Agent infrastructure
shimmy
Michael-A-Kuykendall/shimmy
A tiny pure-Rust local inference server for GGUF models with an OpenAI-compatible API and WebGPU acceleration.
Topics
Related projects
koboldcpp12.3kA one-file local LLM application with a built-in web UI for chatting, roleplay, image, video, speech, and agent-assisted creation.
llamafile26.6kA Mozilla project that packs LLMs into single-file executables for local, no-install inference across operating systems.
lmdeploy8.1kA toolkit for compressing, deploying, and serving LLMs with fast inference engines, quantization, and distributed serving.
lemonade6.2kA local AI server that runs optimized LLMs on your own GPU and NPU and serves them to apps via standard OpenAI-compatible APIs.
OGAM3.3kAn offline AI suite for phones and Macs that bundles local LLMs, image generation, vision, speech transcription, tool calling, and a project knowledge base.
AGiXT3.2kA comprehensive AI automation platform that centralizes smart-home, enterprise workflows and crypto tasks through natural conversation with multi-provider support and extensions.