Agent Lens is the conversational quality gate for MLflow. Evaluate, qualify, and govern your agent fleet — in plain English. Works with any framework: LangGraph, Google ADK, LangChain, CrewAI, OpenAI Agents SDK, and more.
Quickstart
Features
Ask "Can this agent be deployed?" and get a structured qualification verdict with evidence. No Python scripts required.
Built entirely on the official MLflow MCP server. Zero custom forks. Your data stays in MLflow.
Runs inside an OpenShell sandbox with defense-in-depth isolation — Linux namespaces, Landlock, seccomp, L7 network proxy. The qualification agent itself is constrained, not just the agents it evaluates.
Skills are portable markdown — not code. Ships with two validated harnesses (Hermes + Google ADK), works with Claude Code, OpenClaw, Goose, or any MCP-capable runtime.
Test for prompt injection, data exfiltration, jailbreaks, and PII leakage with attack-specific judges.
Evaluation-Driven Development: baseline, diagnose failures, fix, re-evaluate, compare. The full MLflow cookbook workflow — conversational.
Compare quality vs cost across models and configurations. Find the optimal tradeoff for your budget.
Create domain-specific LLM judges from natural language. "Check if the agent mentions our privacy policy" becomes a registered scorer.
>=80% scorer pass rate + <5% error rate = QUALIFIED. Evidence-based decisions, not gut feelings.
Evaluates any agent that sends traces to MLflow — LangGraph, Google ADK, LangChain, CrewAI, OpenAI Agents SDK, AutoGen, or custom.
Skills
| Skill | MCP Tools |
|---|---|
| trace-explorer Search and drill into any trace | search_experiments, search_traces, get_trace |
| quality-dashboard Fleet-wide health + cost + latency | search_experiments, search_traces, list_runs |
| analyze-session Trace analysis with status and latency | search_traces, get_trace |
| review-trace Deep-dive + assessments | get_trace, get_trace_assessment |
| create-regression Flag failures for follow-up | update_trace_assessment, set_trace_tag |
| evaluate-agent Run LLM judges on traces | list_scorers, evaluate_traces |
| compare-evaluations Track quality over time | list_runs, describe_run |
| create-judge Build custom scorers from English | list_scorers, register_llm_judge_scorer |
| red-team Adversarial safety evaluation | search_traces, evaluate_traces |
| eval-loop Full EDD improvement cycle | create_run, search_traces |
| cost-quality Cost vs quality tradeoff analysis | list_runs, describe_run, search_traces |
| audit-trail Qualification decision history | search_traces, list_runs, describe_run |
| agent-registry Fleet inventory and status | search_experiments, list_runs, describe_run |
| executive-summary Board-ready health summary | search_experiments, search_traces, list_runs |
| compliance-export JSONL/CSV export for GRC tools | search_traces, list_runs, describe_run |
| aggregate-traces Error rates, latency, token trends | search_traces |
Open source. Self-hosted. Framework-agnostic. Runs on any Kubernetes cluster with MLflow.