Imo the biggest problems with LLMs is that they’re being sold as something that can replace humans. The people peddling the tech are trying to hide the fact that it should never be used without HIL (human in the loop). Not for anything important.
Yeah, my impression is that LLMs are okay at writing code, but not good at software architecture.
It’s an advanced autocomplete, not something that can think, at least not on the level that humans do.
If you need or want to run an LLM on limited hardware, you may want to look into so-called bitnets with ternary connections. These should be efficient enough to run an OK LLM on a CPU with 16 GB of ram if not less. Unfortunately they’re barely out of the experimental stage, so you’ll probably have to compile BitNet.cpp yourself or wait a few months until full support lands in Ollama.
I haven’t run a bitnet myself yet, so I can’t personally vouch for their effectiveness or usefulness.
Sounds like you’re looking for something like archivemount.
I’d suggest getting Linux Mint or another distro with a large user base like Ubuntu, Fedora, Suse or Debian. That way if you do run into problems, there’ll be a lot of people online that can help you.
The main advantage of gaming-focused distros like Bazzite is that Nvidia’s proprietary drivers and Steam come pre-installed. However, if you’re a gamer and a streamer, you’re probably used to a little tinkering, so you should be able to install both of those by hand. (Both can be done through a GUI.) You might also get marginally better performance from these, but IMO this won’t be noticeable enough to be worth the trouble.
If you do choose to go with Mint, I recommend installing the Nvidia drivers through the “Driver Manager” and a newer kernel through the “Update Manager”.
Does your screen not have a built-in settings menu where you can turn off the stretching?