Agent infrastructure
LitServe
Lightning-AI/LitServe
A Python framework for building custom AI inference servers with full control over batching, streaming, and scaling.
Topics
Related projects
ai-guide22.1kA free Chinese AI knowledge hub by Yupi curating tool guides, prompts, industry news, and a comprehensive zero-to-one Vibe Coding tutorial for everyone.
iii18.9kA backend composition runtime and SDKs for composing, observing, and extending services via workers and functions.
carla14.6kAn open-source autonomous driving simulator for developing, training, and validating self-driving systems.
litgpt13.8kA developer framework with from-scratch implementations of 20+ LLMs plus proven recipes to pretrain, finetune, evaluate, and deploy them.
fiftyone11.3kAn open-source toolkit for visualizing, curating, and evaluating high-quality visual AI datasets and models.
LocalAI50.3kAn open-source engine to run LLMs, vision, and audio models locally with OpenAI-compatible APIs.