Hello, I will soon have to setup a small GPU HPC cluster with few servers. It is not something I have ever done, I did setup servers and manage them but not sometof this scale with several users.

I’ve been looking at plausible alternatives and I am looking at openhpc, Ansible and NixOs.

Out of the three I feel like NixOs could be a good choice to have the configuration clearly stored somewhere. I won’t be actively maintaining the cluster at all times and find it nice that if someone needs some software I can point them to the configuration and tell them to modify it and then just update the system.

On the other hand I appreciate how openhpc is designed for that and definitely does support all use cases we need.

Ansible seems like an in between option.

Do you have advice on this choice? I read that NixOs can have problems with cuda drivers and custom compiled software, which would definitely be a deal breaker.

I found some HPC do use NixOs but it is definitely not a widespread option. Thus I’m a bit afraid I won’t be easily able to find solutions to problems I may encounter.

  • moonpiedumplings@programming.devEnglish
    1·
    6 hours ago

    Containers are commonly used to distribute programs that depend on different library versions, including different libc versions.

    And yes, you could compile software for specific hardware and then deploy it via containers.

    What is the advantage in using a container to say run gromacs, rather than pointing the user to the path of the compiled binary

    Different libc versions. There are also some sandboxing and security restrictions that are applied.

    I definitely want to have some persistent services running: databases, a couple web applications and maybe a Jupiter webpage to visualize results.

    Maybe docker swarm mode, with the nvidia container runtime is easier? It lets you spin up services on multiple nodes, which you could then load balance with a conventional reverse proxy. Although if you are trying to do a single instance of Jupyter, then I don’t know about it’s ability to serve multiple users, or the security of that setup. Usually, people go for things like JupyterHub + the Dockerspawner. Jupyterhub handles authentication or

    There is also a Kubernetes spawner, or you can use an alternative web application that can dynamically spin up jupyterlab/notebooks, like Kubeflow, or coder, but I don’t think any of that is what you want in regards to that specific usecase for Jupyter.

    If you just want to present rendered stuff, check out: https://quarto.org/ , which is a static site generator capable of executing jupyter notebooks, and rendering them to websites. It can also render I use it for my blog, which is ironic because I don’t actually use any of the data science features quarto has. I really like quarto because it’s the only static site generator which I found has fulltext search built in/easily enabled. It runs some javascript over a generated index, and you can search the website without any form of backend needed.