Compare

Side-by-side runs on one dataset: different agent setups (models) or pack revisions (versions). Each arm uses your signed-in agent subscription and its quota.

Compare models

Each arm is Label:harness[:model] with harness auto, cursor-agent, codex, or claude. Limit which checks run with --cases (comma-separated ids). Use --dry-run to print the plan without spending. Live runs stop before the next arm when --max-spend-usd is reached (default 5).

shell

skillgate compare models skills/pdf \
  --arm A:cursor-agent \
  --arm B:codex \
  --cases extract-hello \
  --dry-run
skillgate compare models skills/pdf \
  --arm A:codex \
  --arm B:claude \
  --cases extract-hello,rotate-hello \
  --max-spend-usd 5

Compare tables include per-arm token and cost totals when usage is reported.

Compare versions

Compare two prior check runs or two git revisions of the same pack (from a git checkout). Run skillgate compare versions skills/pdf --dry-run with no --run flags to list recent runs and pick ids for the arms.

shell

skillgate compare versions skills/pdf --dry-run
skillgate compare versions skills/pdf --run A --run B --dry-run
skillgate compare versions skills/pdf --run <run_id_a> --run <run_id_b>
skillgate compare versions skills/pdf --git HEAD~1 --git HEAD

List, show, and cancel

shell

skillgate compare list skills/pdf
skillgate compare show <compare_id>
skillgate compare show <compare_id> --json
skillgate compare cancel <compare_id>

cancel stops an in-flight job.