Closed loop
Each virtual user sends its next request after the previous response. Useful for interactive capacity behavior.
PERFORMANCE
Load tests that lead to an operational decision.
Compare serving behavior with RPS, latency, TTFT, throughput, concurrency and success rate — then inspect charts and output artifacts.
LOAD CURVE
Throughput rises with concurrency.
Keep testing until the bottleneck is reached
36.351
68.872
133.224
Six metrics. A complete performance picture.
Beyond a single throughput number
RPSRequests per second
tok/sOutput throughput
LatencyAverage / P99
TTFTTime to first token
ConcurrencySweep range
SuccessCompleted requests
Choose the arrival model before you interpret the curve.
Open loop
Requests arrive at a target rate, independent of completion. Useful for arrival-pressure experiments.
Real results from real runs.
Same data · better insights
Decision-ready evidence
Read the curve, then decide.
A performance run should explain service behavior across load, not promote one isolated number.
- Throughput as concurrency changes
- Latency and time to first token
- Success state and run artifacts
Bound the first run. Keep artifacts.
No model weights or GPU are required when testing a compatible API.
Run an API testevalscope perf \
--model qwen-plus \
--number 24 \
--parallel 1 2 4