I have noticed that my home server is strangely using lots of swap (~5 GB), despite having only a few lightweight processes running and loads of RAM installed (32 GB).
Upon configuring Grafana + Prometheus, I noticed a trend where cache + buffer will progressively increase until swap starts to be used. My system and services combined will use ~8 GB RAM. Upon rebooting, the cache + buffer will start anywhere from 3–10 GB, progressively ramp up to ~25 GB in 1–2h, where swap will start to be needed (~3 GB). See the image attached for reference.
My swap filesystem is on an expensive (to me) SSD, and I would like to reduce its wear by as much as possible. I understand that swap can introduce only minimal wear on SSDs depending on its nature and that it can be harmless, but I am still not sure what is causing this behavior (and why) and whether I should worry about it or not. So I figured I should investigate what is happening here.
My main question is, how can I figure out what is causing this behavior? Is it expected? I am looking for guidance from others who are more experienced than me in the topic.
A little bit about my system:
I am running Debian 12 on an NVMe SSD containing the root partition (btrfs) and docker services. I also have two HDDs, one with persistent data (ext4), and the other with backups (ext4). This is majoritarily a single-user machine. I tried using the following kernel parameters, but it hasn’t helped:
vm.swappiness=10
vm.vfs_cache_pressure=200
My docker services are:
- *arr stack
- jellyfin
- nextcloud
- immich
- open-webui + ollama
- pi-hole
- invidious
- romm
- nginx proxy manager
- grafana + prometheus
- other minor services that I don’t think are doing much (uptime-kuma, stirlingpdf, vaultwarden, etc)
The cache usage being higher on a server than a laptop seems fairly likely as more disk accesses are occurring. When data is loaded from disk, the linux kernel will keep it in memory even after the original requester is done with it, on the principal that it might need it again. This is the cache you’re seeing. If the kernel detects high memory utilisation it will free cache pages before anything else, so you actually want to see near 100% memory utilisation all the time, as it means fewer accesses to disk.
The swap usage is harder to diagnose without more detailed diagnostic work, but you can see from the graph you posted that, even what swap is being used, you have a little free memory. This may well be the kernel preemptively moving little used memory pages to the swap cache so tgat they can be evicted from memory quickly if needed. You could investigate thus by adjusting the ‘swapiness’ value closer to 0, to see if that delays the swap usage and reduces the peak.