GET STARTED

Start with your coding agent, safely.

Install the EvalScope project skill, then describe the task you want to complete. Your agent will clarify the model, endpoint, and other context it needs before running it.

Ask your agent to install the Skill

Install the EvalScope project skill from https://github.com/modelscope/evalscope/tree/main/skills/evalscope. Read it after installation, then tell me what you need from me before using EvalScope.

CHOOSE YOUR TASK

Give your coding agent a bounded job.

Same principle · more possibilities

Evaluate a model

Run a bounded evaluation.

Choose the model and evaluation context with the agent, then review the saved results.

Use the EvalScope project skill to evaluate qwen-plus on five GSM8K samples. Ask me for the model or endpoint configuration you need before running it, then report the results and where the outputs were saved.

COMMAND LINE

Run a small evaluation from the terminal.

Install the service tools, run five API-backed samples, then open the saved outputs locally.

Install service toolsAdds the CLI and local dashboard.

Run five samplesUses the API key from your environment.

Inspect artifactsOpens reports, predictions and configs from ./outputs.

pip install 'evalscope[service]'
export DASHSCOPE_API_KEY='…'
evalscope eval \
  --model qwen-plus \
  --api-key "$DASHSCOPE_API_KEY" \
  --datasets gsm8k \
  --limit 5
evalscope service --outputs ./outputs

Troubleshoot only what changed.

My API endpoint is OpenAI-compatible but not DashScope.

Keep the API-first route; replace the URL, model name and environment variable with the values provided by your service.

I do not have a GPU.

Nothing on this page requires one. Use a remote compatible API and avoid local checkpoint model types.

Where are my results?

EvalScope prints an output directory. Preserve it; reports, predictions, reviews and configs are organized under that run.

Open documentation FAQ