I’m working on self-hosting my own LLMs.

I realized there are things I wanna talk about and research but I don’t want to send it to open AI. Frankly I feel gross about how much I’ve sent to open AI. My desktop is a beefy gaming rig that I don’t use for gaming much. I have a 20thread core, 64gb ram, an Nvidia gtx 3060 and 5 spare TB so why not.

• I keep a few ollama models downloaded and I’m slowly getting to know them and what they can do. Gemma seems to answer the fastest so I’ve been using that. Deepseek is like the reasoning button on chatgpt.
• I use openai-whisper to transcribe meetings I record using OBS. It’s really slow so I have a cronjob transcribe all my meetings for that day overnight.
• Open Web UI is a fantastic LLM frontend. It provides tools, rags, web searching, and model ranking all as a simple to use UI.
• My desktop has a Wireguard server which makes it easy to use my OpenWebUI on my phone.

Now I want to work on giving the LLM access to my Google calendar so it can create reminders for me. I’m sick of forgetting to think about remembering to do things so I hope if I can just ramble at the LLM about what I’m doing or what’s on my mind it can organize my thoughts. What else are these LLM actually for?

I started this about a month ago, absolutely no idea what I was doing, and in that short time this little box has grown a ton. Got the basics for cloud storage, jellyfin with the arr suite, navidrome to replace spotify/tidal, etc. Got my scanner going right into paperless, finally starting a budget planner with actualbudget, even set up homebox to maybe eventually keep track of my collections of random bullshit. Spent 3 days fighting with Wireguard and gluetun to make a single VPN connection that’ll hook me into my LAN but also output all my traffic through Mullvad, using pihole as my DNS - I should get Unbound set up at some point too but that’s a project for another day.

Today I learned about homeassistant, and while I’m not one to care about IoT shit or whatever, just dabbling with NFC tags for the lights and such has been pretty neat.

This week I’m getting a second machine in that I’m going to use exclusively as a NAS and stop relying on USB external hard drives.

I really just wanted a little 24/7 Bob Ross box with a bit of cloud storage, and this project blew up a lot more than I thought it would LOL

A couple things I’ve been working on

First, I spun up a larger VPS to consolidate two smaller ones. This time I dockerized almost everything. Still a docker newb, but karakeep, redmine, mbin, lemmy (still deciding which I want), davical. Asterisk and postfix/dovecot are probably gonna stay on the vps root. I’m using zfs and compression. Interestingly, the postgres database that everything is using seems to get better compression than the mail spool.

A couple weeks ago I picked up a NetApp 7 bay disk shelf for $30. It uses fibre channel (AT-FCX) controllers and I’ve never used that before. I grabbed a $7 FC HBA (QLE2560), a 2m cable and an m2-to-PCIe adapter meant for an eGPU. The idea is to see if I can’t get the RK3588 board I’m playing with to see it. I did something similar with a $50 Dell 12 drive bay and my old C6100.

i just moved almost all of my containers (except for my omada controller) to my VM running fedora and podman off my VM running ubuntu and docker. why? i was in a product sales call (being sold to) and didn’t have any actual work tasks to do during that time. Now there’s an additional VM on the network.

Trying to decide if I’ll move omada as well or just shift everything back. I shouldn’t have fiddled with the stack while I was bored. A video game or something would’ve been a better idea.

I finally bought a tiny PC to replace my aging APU border router/firewall (OpenBSD), so I’m trying to wrap my head around building a router currently inside the network that it will be protecting.

I have Debian installed as hypervisor, Incus, and sticking with OpenBSD for the firewall. pf makes too much sense to me too switch to firewalld. I’ll also move the network-related containers off my main lab host once this is up and running.

I set up my old laptop as a home server, with a vps as reverse proxy via nebula. It runs Mint - strange for a server but that’s so it can still be a laptop. Syncthing keeps it in sync with the more portable laptop.

The ‘server’ now runs immich, which I can use super fast from the laptop itself; a bit slower if I connect with nebula over the LAN (it’s firewalled off from the LAN generally); or still pretty decently via the VPS on Https - and that VPS proxy means the family phones can connect with the apps easily.

Immich runs in podman, with some help from Lemmy about how to set that up.

And filebrowser makes it easy to share files or allow uploads with/from family around the world. With caddy on the VPS, ufw on the server and nebula in between, it’s really easy to add in something like filebrowser on a new subdomain.

Next is to try some other podman containers, or set up mqtt and owntracks.

I have a question on top of my matrix setup. Has any one integrated VoIP? I am trying to bring all communication in house.

I recently setup a full matrix server. What I am currently worried about is my server. I am currently shopping for a used dual Xeon server. I am hosting close to 40 docker containers on 2 1 liter PCs with very low specs. I would love to bring it all in house to a single server with a separate NAD which I do have currently holding 60 terabytes of storage space.

No new devices, but I migrated my homelab from an intel nuc to an old recycled HP z240 with a p1000 gpu I got for free. I had Nextcloud and jellyfin on it, but jellyfin gets the majority of the use.

I then added a gitea docker container to my server for my personal projects. Then I configured a miniflux container with some of my favorite RSS feeds for a lightweight way to view my feeds on my computer.

I would like to get pihole configured again in a docker container(I have only ever run it on a raspberry pi), but I have small children and a baby and they make it hard to find extra time in the day.

I finally finished setting up my Nebula network! An overlay network, as opposed to a true VPN, but excellent for flexibility and remote access. For anyone wanting maximum control over your network with excellent performance, I highly recommend it.

Check out apalrd’s blog for a great tutorial if you’re interested.

I am re-re-factoring my plans for homelab 3.0 and the migration to it. Hardware budget is non-existant so I am trying to figure out how to do everything with what I already own, while re-organizing to better use what I have to make some room. Adding a few sticks of RAM and replacing some older cat5 are all I will do this year.

I installed Jitsi Meet on my YUNOhost server and am very impressed. It works really well and needed basically no setting up after installing.

Currently in a holding pattern because, while I got RAM & SSD for a new-to-me “1-liter” server before tariffs hit, I don’t have the server itself nor any money to buy one, despite looking for 9th or 10th gen Intel which will cost me only $120 to $150 barebones.

Money to buy one is not coming in because the place where I live has nonstop noise & activity and I don’t have a separate room or any door I can close, which severely limits my ability to work as I have auditory hypersensitivity and an absolute need for solitude in order to recharge enough to think. 🤷🏻

Trying to get navidrome routed through Traefik.

I think it’s rejecting it as an untrusted proxy because forwarding the ports locally works.

Also working on getting Traefik up and running on a TuringPi cluster to eventually move my workloads over to it.

My biggest shortcoming at the moment is my NAS is also my gaming PC. It’s pretty inefficient to have that on all the time. But I haven’t had the time to build a dedicated NAS.