Stamets@lemmy.world to People Twitter@sh.itjust.works · vor 3 JahrenThe dreamlemmy.worldimagemessage-square249fedilinkarrow-up12K
arrow-up12KimageThe dreamlemmy.worldStamets@lemmy.world to People Twitter@sh.itjust.works · vor 3 Jahrenmessage-square249fedilink
minus-squareCeeBee@lemmy.worldlinkfedilinkarrow-up2·vor 3 Jahren I don’t know of an LLM that works decently on personal hardware Ollama with ollama-webui. Models like solar-10.7b and mistral-7b work nicely on local hardware. Solar 10.7b should work well on a card with 8GB of vram.
minus-squareParetoOptimalDev@lemmy.todaylinkfedilinkarrow-up1·vor 2 JahrenIf you have really low specs use the recently open sourced Microsoft Phi model.
Ollama with ollama-webui. Models like solar-10.7b and mistral-7b work nicely on local hardware. Solar 10.7b should work well on a card with 8GB of vram.
If you have really low specs use the recently open sourced Microsoft Phi model.