I’m looking for storage classes for a multi node cluster. I’m currently using Longhorn and NFS, but I’m not happy with the performance. My cluster doesn’t have beefy nodes, so Ceph/Rook is out of the question (for now).
Nodes:
- 8 GB RAM, 4 cores VM, control plane. 256 GB SSD
- 4 GB RAM, 2 cores, control plane, currently cordoned. 128 GB SSD
- 8 GB RAM, 4 cores, ARM, control plane. 512 GB SSD
- 8 GB RAM, 4 cores. 256 GB SSD
- 16 GB RAM, 6 cores. 256 GB SSD + 1 TB HD
- RPi 4, 4 GB RAM. 128 GB SSD
Thanks! A bit more involved that I’d have thought but still worth considering! Could you update us after your evaluation?
Well actually it is very easy to spin up in docker and most of the configuration happens through env variables.
juicefs itself only exists on the client side, so you basically only have to install and configure the CSI driver with helm.
as it took me a few days to come up with this solution I’d be happy to share my config files.
Performance wise is quite fast on sequential reads (it saturates my 2.5G bandwidth) and slower than I expected on sequential writes (for me it caps at 60MB/s). Postgresql seems happy. I saw no visible performance degradation with Authentic, Immich and Opencloud. Nextcloud installation took ages. I’ve yet to try it with jellyfish and the *arr suite.
A simple NFS share would be faster, but it doesn’t support replication, failover and CSI snapshots.