Agent infrastructure
nexa-sdk
qualcomm/nexa-sdk
An on-device GenAI inference runtime for Snapdragon that runs frontier LLMs and VLMs locally via CLI, SDK, or server.
Topics
Related projects
sglang38.7kA fast open-source inference framework for serving large language and multimodal models at scale.
turbo-fieldfare7.2kAn efficient Gemma 4 inference runtime for Apple-silicon Macs, streaming experts under low-memory budgets and bundling a demo app with evaluation tooling.
tokenspeed2.9kA blazing-fast open-source LLM inference engine tuned for agentic workloads, pairing TensorRT-class speed with vLLM-like ease for production deployments at scale.
GoModel1.3kGoModel is a self-hosted AI gateway written in Go, unifying OpenAI- and Anthropic-compatible APIs with multi-provider routing, caching, budgets, observability, guardrails, and usage analytics.
bigmoeonedge709An inference engine built on stock llama.cpp that runs oversized MoE models on memory-constrained phones by streaming only routed experts from flash without quality loss.
xybrid513A cross-platform on-device AI runtime and SDK collection for running LLMs, speech recognition and text-to-speech locally in mobile apps, desktop software and games.