cross-posted from: https://discuss.online/post/39090314
Full layout on my forum here. Basic idea is 3 thin clients that fit within:
- 1u space requirement
- Possibly adding 2.5gb nic to the pci-e low profile slots of each hp t740 for Ceph shared storage.
- Kubernetes and/or Proxmox experiments
+----------------------------------------------------------+ | CLUSTER TOTALS | +----------------------+-----------------------------------+ | TOTAL CORES | 12 physical / 24 threads | | TOTAL RAM | 192 GB DDR4 | | TOTAL STORAGE | 1.5 TB NVMe (local only) | | TOTAL NODES | 3 | | TOTAL NICs | 3–7 (depending on 4-port NIC) | | GPU | None | | KVM/IPMI | None | +----------------------------------------------------------+All suggestions greatly appreciated! Never done this before, but hoping to save myself from the future hassle of having to drive to the data center… would much rather log in on Wireguard and address things by remotely rebooting, etc.
I’m not sure what data center will allow you to hodgepodge a 1u cluster of consumer-grade hardware, but heat and power management alone will be a problem.
What’s your plan for when you reboot and it doesn’t come back up? I strongly recommend having some kind of IPMI and virtual console, whether that’s pikvm or whatever. Far too many times I’ve had a server go down and it’s saved me from having to drive down to the datacenter and stand in a loud, frigid room and troubleshoot the issue.
I thought I listed it in the description, but this is something I’m actively wondering about. A KVM over IP, such as PiKVM, Jetkvm, openkvm that allows access to all 3 nodes and saves future drives to the data center by using a wireguard connection instead. PiKVM + HDMI and USB switch is an option. Recommended option from PiKVM docs is https://www.easycoolav.com/products/ezcoo-hdmi-20-switch-4x1-with-usb-30-kvm-4-port-hdmi-switch-for-4k60hz-hdr-and-audio-breakout
The pikvm and pikvm switch are really fun tbh
I really like my Jetkvm, has helped me out.
Cool, have you tried using it with multiple devices on an hdmi switch?
I have not but may look into that as I’m thinking of getting 3 small computers to have a kubernetes cluster at home.
Same, I use Jetkvm for this too, has worked great