Agent infrastructure
lemonade
lemonade-sdk/lemonade
A local AI server that runs optimized LLMs on your own GPU and NPU and serves them to apps via standard OpenAI-compatible APIs.
Topics
Related projects
lmdeploy8.1kA toolkit for compressing, deploying, and serving LLMs with fast inference engines, quantization, and distributed serving.
LLM-Hub637An open-source on-device AI app for chat, image, video and music generation, speech, translation and device agents, running privately on Android, iOS and desktop.
LocalAI50.3kAn open-source engine to run LLMs, vision, and audio models locally with OpenAI-compatible APIs.
airllm42.3kA Python inference engine that streams huge language models layer by layer to run 70B to 671B models on small GPUs.
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
OpenLLM12.6kAn open-source platform for self-hosting LLMs and serving them as OpenAI-compatible APIs in the cloud.