I currently use Borg for all my backups. To have a remote backup I placed my old Synology NAS at a family member and connected it to tailscale (via a tailscale docker container). That way I could also can do Borg backups to that remote NAS.
Though I have problems with the stability of the tailscale connection from the NAS. And honestly, I’m a bit tired to fix these problems.
I would like to have a self hosted backup solution, that can reach the synology NAS. I would like to avoid port forwarding at my family members network (for obvious reasons). I have a VPS in the internet that I can use for a common self hosted contact point (like I setup headscale there instead of using official Tailscale servers).
What would be your way for handling these remote backups?
So Tailscale connection instability is not common? I don’t like debugging this, since the NAS is remote and I don’t have much time going to the family member again and again for this. It definitely is something on the remote site, since the tailscale connections at my home are stable.
I will definitely have a look at Kopia. I was in search of a good open source backup solution, since borg also has its flaws. I will check it out, when I setup the remote NAS again (as I also want to put bigger disks in it)
Yes it is uncommon, as people have mentioned it is a Wireguard wrapper. Try just using Wireguard and troubleshoot from there. I would want to know what that ‘simple’ solution is failing. Your workaround could have similar issues.
Troubleshooting wireguard is a pain in the ass, it using UDP and all that.
Why would UDP be any more difficult than TCP (or anything else) for troubleshooting? If nothing else, it should be easier because there are no stateful connections and every packet is “fire and forget”. Now, as a tunneling technology, WG might be more difficult than a simple client/server socket-driven application, but that just means some routing tables need to be inspected and you have to understand IP subnets. In any case, you’d still want to obtain packet traces from both ends of the WG tunnel.
With TCP there are built in listeners on OSs which you can use to test. Also TCP will work before UDP. For whatever reason firewalls and such do weird things with UDP traffic. I always use a cli tool that I can’t remember the name of for testing.
What are these “built in listeners” that you speak of, and which OS’s can I find this on?
This doesn’t really explain why troubleshooting is more difficult, but rather that firewalls often cause trouble for UDP. My question was “why would diagnosing a UDP problem be harder than anything else?”, not “why is UDP more problematic?”. I’m fully aware that some firewalls are configured with absurd parameters, like blocking ICMP, QUIC, or even TLS in some bizarre situations. But my tools for debugging network troubles have always been the same: tcpdump for passive tracing, nc to coerce an active response, and icmp/icmp6 ping for basic connectivity and MTU verification.
Whether it’s UDP or anything other protocol, those tools have proven sufficient for me. Do you use something else?
Never had an issue with Tailscale and have been using it for years.
You don’t have to be on the NAS’ physical network to run some tests, if you can SSH into it, that’s enough to run some speed tests and latency tests to reference servers (e.g. Google, netflix, …), while doing the same from your PC simultaneously. Whenever you encounter “instabilities” you should be able to see which side of the connection is struggling more than usual. You should also be able to monitor the NAS itself for excessive IO or CPU using something like htop (even though that would be rather extraordinary if all that’s going on with it at that time is a Borg backup).
On the topic of Kopia, you can go the easy route of setting up a SFTP remote pointing to your NAS. All you need is SSH access to the NAS (so your current tailscale/borg set-up qualifies as is) and a folder where the SSH user can write. You can then compare backup times vs. Borg (in my use case, going from 18-25min with Borg down to 3-5min with Kopia). If you then want to try the HTTP repository server, there is a container for that, but the set-up is a bit annoying (the password to decrypt the repo needs to be stored on the NAS, you need to generate a SSL certificate pair and have the client whitelist the sha256sum of it, etc). Personally I haven’t found a substantial advantage to having the http repository server but YMMV.
I am also using tailscale via headscale, and my connection has good. Only when the internet where my home server is down is the connection not available, and there isn’t much I can do about that.
I mean it’s a commercial product - if stability issues were a common thing, that would be well known and published.
It’s a mesh VPN so it generally has the same stability challenges as any other: Hamachi, Nord, Meshnet, Wireguard (which is the underlying protocol that Tailscale uses).
If the Tailscale client is running on the NAS itself, I’d consider that the first thing to troubleshoot.
NAS’s generally don’t have much extra processing power, and in my experience apps designed to run on them aren’t the most robust. In this case, the VM/dock client has to consume all TS traffic, perform the encrypt/decrypt, put it back on the net (even if it’s only with the virtual net) over to Borg, which then has to write to the NAS. The NAS hardware has to do all this, with whatever little bit of ram it came with, also while writing all these changes, managing stripes, etc.
While they advertise these things can run stuff like this, it just seems like an opportunity for performance issues when the processing power design was for a NAS/RAID.
I have an ancient NAS on my network that could probably run the TS client, I just don’t bother. TS runs on all desktops and servers (for general access), but connectivity to non-TS devices goes through the server running as a TS router. That can be run on any device running TS - it’s just a feature to enable and configure.