Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-219 days agoAm I the only one who is really impressed by Granite4 from IBM?message-squaremessage-square8fedilinkarrow-up111file-text
arrow-up111message-squareAm I the only one who is really impressed by Granite4 from IBM?Possibly linux@lemmy.zip to LocalLLaMA@sh.itjust.worksEnglish · edit-219 days agomessage-square8fedilinkfile-text
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up3·edit-219 days agothere’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
minus-squareBaŝto@discuss.tchncs.delinkfedilinkEnglisharrow-up1·18 days agogranite4:micro-h should be able to run on machines with 4GB RAM
minus-squareXylight@lemdro.idlinkfedilinkEnglisharrow-up2·18 days agoYou can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too
there’s also a “small” and “micro” variant, which are 32b a6b MoE and 3b dense models respectively
granite4:micro-h should be able to run on machines with 4GB RAM
You can run Qwen3 4b thinking at q4 quantization at 2.5GB, which is probably a better model too