Agent infrastructure
promptfoo
promptfoo/promptfoo
An open-source CLI and library for evaluating prompts and red-teaming LLM apps, with CI integration and model comparisons.
Topics
Related projects
ragas16.2kAn evaluation toolkit for LLM apps offering objective metrics, test generation, and production feedback loops.
LLM-Engineers-Handbook5.4kThe companion code for the LLM Engineer's Handbook, teaching end-to-end LLM and RAG systems on AWS.
second-brain-ai-assistant-course3.1kAn open-source course teaching how to build a Second Brain AI assistant with RAG, agents, fine-tuning and LLMOps.
opik23.4kAn open-source LLM observability platform for tracing agents, running evaluations, and monitoring production.
OpenLLM12.6kAn open-source platform for self-hosting LLMs and serving them as OpenAI-compatible APIs in the cloud.
opencompass7.6kA one-stop open platform for evaluating foundation models across 100+ datasets, with unified inference, judging, and leaderboards.