Eval framework. Define correct, test against it, get results.

Find what your AI agent gets wrong — before you have a rubric. Qualitative eval for PMs.

A web-based interactive demo for the GuessArena evaluation framework

4-model parallel planning workflow with eval framework — Claude, Gemini, Codex, GLM-5 · OpenClaw ecosystem

Curated AI agent evaluation skills from Microsoft's Eval Guide — plan, generate, run, and interpret eval suites for Copilot Studio agents

Web studio for evaluating LLM agents with rubric-based scoring, LLM-as-judge validation, and side-by-side run diffs. Trace agent execution, compare performance, and iterate with full visibility into tool calls and LLM interactions.

Self-hostable LLM evaluation framework for measuring model performance across configurable skills. Run YAML-defined benchmarks against OpenAI/Anthropic models, score with LLM-as-judge, compare results in a CLI and web dashboard.

Binary safety verdicts (SAFE/HELD/LEAK/MISS/BROKE) + persona fan-out for LLM pipeline evals

MCP server exposing portfolio tools (Semantic Search, Eval Framework, Observability) via Model Context Protocol

Lightweight CLI for versioning prompts and running eval suites. Score outputs with deterministic matching or LLM-as-judge, compare prompt versions with rich terminal diffs. No infra, git-friendly, local-first.

Open-source evaluation framework for AI agents. Define test suites with rubrics, run your agent, get LLM-as-judge scores against criteria, inspect full execution traces, and diff runs to catch behavioral regressions.

Define YAML rubrics, run agents through test scenarios, get LLM-judged per-criterion scores with full trajectory traces, and analyze results in an interactive web dashboard.

Agent evaluation framework: run LLM agents against datasets, capture execution traces, score with rubric-based LLM judges, and view regressions in a web dashboard. Local-first, no external infra required.

🚀 基于Java的开源AI自动化评测框架 / An open source AI automation evaluation framework based on Java

Self-hosted eval harness for LLM agents: YAML-defined scenarios with LLM-as-judge scoring, deterministic tool-call assertions, full execution tracing, and Next.js dashboard for run comparison.