Agent infrastructure
OpenLLM
bentoml/OpenLLM
An open-source platform for self-hosting LLMs and serving them as OpenAI-compatible APIs in the cloud.
Topics
Related projects
llm-action25.3kA curated Chinese knowledge base sharing LLM training, fine-tuning, inference, and deployment tutorials with hands-on code.
second-brain-ai-assistant-course3.1kAn open-source course teaching how to build a Second Brain AI assistant with RAG, agents, fine-tuning and LLMOps.
LocalAI50.3kAn open-source engine to run LLMs, vision, and audio models locally with OpenAI-compatible APIs.
airllm42.3kA Python inference engine that streams huge language models layer by layer to run 70B to 671B models on small GPUs.
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
promptfoo27.1kAn open-source CLI and library for evaluating prompts and red-teaming LLM apps, with CI integration and model comparisons.