Agent infrastructure
web-llm
mlc-ai/web-llm
A high-performance in-browser LLM inference engine with WebGPU acceleration and OpenAI-compatible APIs for building private web AI apps.
Topics
Related projects
mlc-llm23.4kA universal ML-compilation engine for deploying large language models with high-performance inference across GPUs, mobile, and browsers.
Prompt-Engineering-Guide79.9kA learning guide and curated resource collection for prompt engineering techniques.
ai-guide22.1kA free Chinese AI knowledge hub by Yupi curating tool guides, prompts, industry news, and a comprehensive zero-to-one Vibe Coding tutorial for everyone.
DocsGPT18.4kA self-hosted AI platform to build grounded agents, workflows, and search over your docs.
nano-vllm16.3kA lightweight vLLM reimplementation in about 1,200 lines of Python delivering fast, readable offline LLM inference.
sie3.5kA self-hosted inference server and production cluster that serves 100+ open models behind one OpenAI-compatible API, covering embeddings, reranking, OCR, classification, and generation for agent workloads.