colibri

A pure-C inference engine that runs giant mixture-of-experts models on everyday hardware by streaming experts from disk with OpenAI-compatible serving.