I want to host some LLM’s locally and use more advanced models. Since new hardware is out of the question, I think I should be able to pull something off buying some yesteryear equipment on ebay etc. Did anybody attempt such a project? Does it scale horizontally? (I.e. can I connext two boxes to overcome single box slowness?)
The 64GB Framework Desktop runs just barely over 2k configured minimally, I went that route because I thought it was a better option than the discrete 32GB video card, but there are tradeoffs with compatibility. Something to think about at the 2k, but not quite 3k range though.