SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 14 days agoDo you host your own AI?message-squaremessage-square206linkfedilinkarrow-up1182arrow-down143file-text
arrow-up1139arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 14 days agomessage-square206linkfedilinkfile-text
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up3·edit-213 days agoOr exllama! Vllm, sglang, Lorax. Koboldcpp, Aphrodite, text-generation-webui, LM Studio, powerinfer, ktransformers, mlc-LLM, really whatever floats your boat. Just not ollama.
Or exllama! Vllm, sglang, Lorax. Koboldcpp, Aphrodite, text-generation-webui, LM Studio, powerinfer, ktransformers, mlc-LLM, really whatever floats your boat. Just not ollama.