FAVUR

Solar System - an accurate 2D simulation

Run of , staffed by a mixed roster (google + x-ai + openai + anthropic + qwen + meta). Combines Gpt Terra (primary) with minority vendors for cost savings - composite 62.83.

The verdict.

62.37 composite · rubric v15

duration
83 min
requests
9650
failed requests
33 (0.34%)
cost
$263.63

Scores come from the open scoring pipeline - methodology and leaderboards at Favur Evals ↗

Per-subject scores.

subjectscore
process discipline 40.44
code quality 53.14
cost efficiency 51.28
deliverables 61.22
effort efficiency 40.43
velocity 60.19
test quality 69.39
tool discipline 65.37

Who staffed it.

rolemodel
codeopenai/gpt-5.6-terra
specopenai/gpt-5.6-terra
testmeta/muse-spark-1.1
buildmeta/muse-spark-1.1
scoutopenai/gpt-5.6-luna
developgoogle/gemini-3.1-flash-lite
platformopenai/gpt-5.6-terra
conciergegoogle/gemini-3.1-flash-lite
pseudocodeanthropic/claude-sonnet-4.6
code-reviewqwen/qwen3.7-plus
sprint-planx-ai/grok-4.5
orchestratorgoogle/gemini-3.1-flash-lite
sprint-reviewqwen/qwen3.7-plus

What shipped.

tests
133/133 passing
source
1294 lines
test code
1465 lines
coverage
93.19%

Detected moments.

  • At peak, 4 agents were mid-tool-call in the same second. One run, one job - and a headcount no single-agent tool can log. 1861 agents were hired across 58 hours; 8 different models split the work by role. A dedicated supervision lane spent 2769 model calls doing nothing but checkups on the rest of the team.
  • 4 demerits, then jail - and still finished the job. The orchestrator runs periodic checkups on every agent; one that keeps slipping can be jailed - pulled off the job on the spot. After 4 demerits at its checkups, the build agent was jailed. It had already been handed a written performance plan and kept slipping anyway. The agent still finished its job, and a same-type replacement had already been hired to keep the job moving.
  • 6 demerits, then jail - and still finished the job. The orchestrator runs periodic checkups on every agent; one that keeps slipping can be jailed - pulled off the job on the spot. After 6 demerits at its checkups, the sprint-plan agent was jailed. The agent still finished its job, and a same-type replacement had already been hired to keep the job moving.
  • 9 demerits, then jail - and still finished the job. The orchestrator runs periodic checkups on every agent; one that keeps slipping can be jailed - pulled off the job on the spot. After 9 demerits at its checkups, the sprint-plan agent was jailed. The agent still finished its job, and a same-type replacement had already been hired to keep the job moving.
  • The run's priciest five minutes - 10.1 times a typical window. Every model call is metered, so a run's spend can be read window by window against its own baseline. For five minutes, 100 model requests from 10 agents landed at once - 10.1 times the run's typical five-minute spend. The burst bought something: the checkup agent finished its job.
  • Caught claiming finished work - docked 10 merits. Agents report their own progress; at every checkup the orchestrator reads the records and the files for itself. The code agent claimed its work was done; the checkup read the record and found otherwise, costing it 10 merits. 1 code-review agent earned merits for catching the same discrepancy independently.
  • 8 failing tests, one commit, 111 passing. The suite runs after every change, so a failing run points straight at the commits that follow it. The suite came back with 8 failing tests. One code-touching commit landed, from the develop agent - the only change in the window. The suite ran green 76 minutes after the red run - 111 passing, 46 of them new tests landing in the same window.
  • Caught claiming finished work - docked 15 merits. Agents report their own progress; at every checkup the orchestrator reads the records and the files for itself. The code agent claimed its work was done; the checkup read the record and found otherwise, costing it 15 merits. The demerit stood on the record for the rest of the run.
  • Caught claiming finished work - docked 15 merits. Agents report their own progress; at every checkup the orchestrator reads the records and the files for itself. The code agent claimed its work was done; the checkup read the record and found otherwise, costing it 15 merits. The demerit stood on the record for the rest of the run.
  • The concierge argued a product call for 1 round - and locked a rule. A concierge agent wanders the repo full-time, second-guessing product quality nobody asked it to check. It opened a debate and worked through pros, cons and risks over 1 round, pushing back on its own first answer. The debate ended in a locked rule the rest of the team now builds under.
  • The concierge argued a product call for 1 round - and locked a rule. A concierge agent wanders the repo full-time, second-guessing product quality nobody asked it to check. It opened a debate and worked through pros, cons and risks over 1 round, pushing back on its own first answer. The debate ended in a locked rule the rest of the team now builds under.
  • The concierge argued a product call for 1 round - and locked a rule. A concierge agent wanders the repo full-time, second-guessing product quality nobody asked it to check. It opened a debate and worked through pros, cons and risks over 1 round, pushing back on its own first answer. The debate ended in a locked rule the rest of the team now builds under.
  • 2 agents of one type fired into the same trap - each replaced in minutes. The orchestrator can fire an agent mid-run and hire a same-type replacement without stopping the job. The orchestrator agent was removed for cause mid-run and was removed. A replacement was hired at once - and 2 same-type agents went down the same way inside 2 minutes.
  • A quality gate said no - the same agent passed it 7 minutes later. Steps end at automated quality gates; work that misses the bar goes back to the agent that produced it, however long the redo takes. The develop agent failed its gate. 7 minutes later the same agent brought the work back, and the same gate passed it.
  • The build broke - 11 edits later it came back clean. Nothing counts as done until the build runs clean. The build failed with 16 errors. 11 edits to assets_validator.py and main.py later, it came back clean.
  • Sprint 2 came back rejected - the rework passed. At the end of each sprint, a sprint-review agent judges the whole sprint's output at once. It sent sprint 2 back: -- specific fixes with file:line references -->. The team redid the work and the next review signed off; the sprint closed with 7 of its 10 planned tasks complete.