01
Outcome evidence
Inputs and outcomes
Keep the task, final response and completion state together so a result can be checked in context.
AGENT & HARNESS
Measure the loop, not just the last answer.
Capture model requests, tool calls, token usage and final outcomes for agent tasks — with a path for external replay scenarios too.
MEASURE · TRACE · IMPROVE
A measured agent loop.
Every step counts
1
Prompt
Task and context
2
Model
qwen-plus
3
Tool call
python_exec
4
Tool result
Evidence returned
5
Nudge
Model continues
6
Submit
Final answer: 18
Key measurements for agent evaluation.
Multi-dimensional evidence
02
Tool evidence
Calls and results
Record tool inputs, outputs and failure states instead of reducing a multi-step run to a single score.
03
Trajectory health
Steps and recovery
Inspect retries, intervention signals and step count to understand how the agent reached its answer.
04
Runtime profile
Tokens, timing and cost
Use runtime signals alongside correctness when choosing an agent design for a real workload.
Native AgentLoop
Use EvalScope’s native loop evaluation with configurable tools, environments and step limits.
External Agent Bridge
Bridge external harnesses and preserve their scenario provenance, trajectory artifacts and validity state.
AGENT BENCHMARKS & DRIFT
Keep an evolving harness tied to its evidence.
Benchmark tasks make agent behavior comparable today. Freezing artifacts, auditing judges and testing held-out tasks are an exploring research protocol — not a shipped drift-prevention claim.
Freeze the run
Keep task, environment, requests and raw artifacts together.
Audit the judge
Separate a scoring-contract change from a behavior change.
Check selection
Use held-out evidence before claiming an improved harness.