colibrì streams Mixture-of-Experts weights from disk instead of loading them, so a 2.8-trillion-parameter model runs on a 32 GB box. Six model families, one engine, CPU or GPU.
Anyone curious enough should be able to open one up: run it, watch every expert fire, and make it better.
Read the engine →A token through GLM-5.2 activates only ~40B parameters, and just ~11 GB of those change from token to token: the routed experts. colibrì keeps the dense weights resident and treats the 19,456 experts as a storage hierarchy. Measured routing heat decides which experts earn VRAM, which earn pinned RAM, and which stream from NVMe, and the router runs a layer ahead so prefetch hides the latency.
Everything is validated token-exact against the reference transformers implementation. Speed never buys drift.
The flagship target: an int4 container, token-exact versus reference, with the full expert atlas published.
Run it →The flagship’s fast sibling: the same 40B active at less than half the total size, and it takes images through the same server.
Run it →The first family to arrive complete: tool calling and a vision tower verified against the upstream reference, on day one.
Run it →The largest family we run: gated recurrence plus MLA attention, 2.8T parameters, now with a Metal backend on Apple Silicon.
Run it →A 975B reasoning MoE with an audio tower. Mixed-precision staging fits it on a consumer box.
Run it →Sparse attention, multi-token prediction and a trained drafter: the most instrumented engine here.
Run it →Gated attention plus Gated DeltaNet. Its CUDA tier keeps hot experts in VRAM: 7x faster than CPU on two 8 GB cards, output bit-identical.
Run it →Every point behind this text is a real measured expert: 13,260 characterised experts, specialists clustering by topic, generalists drifting to the centre. The first expert atlas of a 700B-class model, mapped on ordinary machines.
Measured decode speed on real community machines. The hardware only changes where the experts live, not what the model answers.
…and 15+ more community machines, from an M4 Pro Mac Mini to a 430 GB EPYC server. Full tables, methodology and quality ablations: docs/benchmarks.md
# one C file, no dependencies $ git clone https://github.com/JustVugg/colibri $ make -C colibri/c # point it at a checkpoint and talk to it $ ./coli chat --model /path/to/glm-5.2 # or serve an OpenAI-compatible API + dashboard $ ./coli web --model /path/to/glm-5.2