A few months ago a simple thought came into my mind and it didn’t left for several months.
‘ With new and cool software like vSAN, Pernixdata FVP, vROPS, vRAC, vSphere 6.0, vSAN, …. You need a new #Homelab to test this stuff’
Yeah… I somehow felt inception-ized ;-).
In the end of last year I had a phone-call with Manfred Hofer form vbrain.info about his great #homelab posts and design-decisions on his blog. Even though I did not chose one of his proposed designs I really like to thank you Fred for your efforts and great sum up.
Since I was asked by multiple people to document my new hardware, I quickly summarized it here:
I had the following requirements for my #homelab:
- min. of 3 nodes (for getting vSAN up and running)
- min. of 96GB RAM
- low-power
- low-noise (currently it’s standing close to my office-desk)
- small
- min. 2 NICS per node
I didn’t really cared about ECC-support, IPMI, etc… Nothing productive will run there… I just need a suitable performance ( ca. 10-16 cores / 2000-3000 4K 70/30 random IOPS / 2-4 TB Disk) capacity / to do some quick’n dirty testing / customer environment simulations. Intel NUCs would have been a perfect choice, but the lack of 32 GB functionality disqualified them ;-/
In the end I decided to go for the following solutions.
Computing
- Shuttle SH87R6
- Intel Core i5-4440S
- 4×8 GB DDR3 memory
- Intel Pro PT 1000 Dual
- 1x Crucial CT256MX
- 1x Crucial CT512MX
Network
- CISCSO SG-300 – 20 ports
- Huawei WS311 Wifi-bridge
Storage
- Synology DS414 Slim
- 1x Crucial CT512MX
- Western Digital RED 1 TB
Currently I have vSphere 6.0 running with a vSAN 6.0 datastore. I have also decided to get a dedicated NFS share on the Synology for maintenance/testing reasons so I can easily demote/recreate the vSAN datastore. Having nearly everything on SSD gives me a performance that is suitable for me and gives me a chance to work efficient with new products (even if the local S-ATA controller are limited in their capabilities, but hey… it’s non-productive ;-).
After optimizing some of my cabling and replacing the fan of the Shuttle Barebone I really like the solution on my desk. It’s powerful, small, scaleable enough for the next things I am planning to do. Even if my requirement for hardware is increasing I can scale-up the solution pretty quickly and easy.
So far I was not able to get the embedded Realtek up and running with vSphere 6.0. But to be honest, I haven’t spent much time with it ;-). Once I have an update here, I will let you know.
April 21, 2015 at 8:59 am
What I really like of your setup is the size of each node, I’m still thinking if getting one of those or bulding up the node myself (mATX), this would be a bit more expensive although the only drawback I see is the case size.
Ideally my lab is going to be a test bed for cloud enviroments, testing different hypervisors and different cloud solutions (i.e Openstack). I want to start with just one node and add more gradually. Do you think the Shuttle is enoguh for this kind of workload?
Why did you choose i5 over i7? I always thought HT provides better performance when doing virtualization.
It has been very interesting reading your setup, also the switch L3 you metion looks quite promising.
Cheers
April 21, 2015 at 11:20 am
For my use-cases I believe that memory will be the limitting factor, but 96Gbyte should be enough (Not for all of my Applications needs to run at the same time). I would recommend a minimum of two nodes even in the beginning, because that’s the only way you can also empty one node up for upgrade, etc.
i5 vs. i7? –> power consumptions is lower (especially with the 4440-S). As I said I don’t expect CPU to be the bottle-neck in my setup. From a pure performance perspective the i7 would be better, but a HT is not a real execution context.
the SG-300 is so far an awesome switch in my lab environment. I don’t want to miss it 😉