FAVUR

Evolution Arena - an evolutionary simulation

Run of , staffed by a mixed roster (xiaomi). Combines Mimo Pro (primary) with minority vendors for cost savings - composite 67.78.

The verdict.

67.78 composite · rubric v15

duration
721 min
requests
21882
failed requests
637 (2.91%)
cost
$208.42

Scores come from the open scoring pipeline - methodology and leaderboards at Favur Evals ↗

Per-subject scores.

subjectscore
code quality 43.08
cost efficiency 60.37
deliverables 88.25
effort efficiency 54.94
process discipline 48.02
test quality 82.05
tool discipline 43.82
velocity 36.64

Who staffed it.

rolemodel
codexiaomi/mimo-v2.5-pro
specxiaomi/mimo-v2.5
testxiaomi/mimo-v2.5-pro
buildxiaomi/mimo-v2.5-pro
scoutxiaomi/mimo-v2.5-pro
developxiaomi/mimo-v2.5
directorxiaomi/mimo-v2.5
platformxiaomi/mimo-v2.5-pro
architectxiaomi/mimo-v2.5
conciergexiaomi/mimo-v2.5-pro
pseudocodexiaomi/mimo-v2.5-pro
code-reviewxiaomi/mimo-v2.5
sprint-planxiaomi/mimo-v2.5
orchestratorxiaomi/mimo-v2.5
sprint-reviewxiaomi/mimo-v2.5

What shipped.

tests
1016/1022 passing
source
6856 lines
test code
14804 lines
coverage
90.14%

Detected moments.

  • At peak, 4 agents were mid-tool-call in the same second. One run, one job - and a headcount no single-agent tool can log. 4571 agents were hired across 100 hours; 3 different models split the work by role. A dedicated supervision lane spent 6638 model calls doing nothing but checkups on the rest of the team.
  • Jailed 3 times in one run - and still finished the job. The orchestrator runs periodic checkups on every agent; one that keeps slipping can be jailed - pulled off the job on the spot. After 7 demerits at its checkups, the code agent was jailed. It had already been handed a written performance plan and kept slipping anyway. The agent still finished its job before the run ended.
  • 6 demerits, then jail - and still finished the job. The orchestrator runs periodic checkups on every agent; one that keeps slipping can be jailed - pulled off the job on the spot. After 6 demerits at its checkups, the code agent was jailed. It had already been handed a written performance plan and kept slipping anyway. The agent still finished its job before the run ended.
  • 3 agents of one type fired into the same trap - each replaced in minutes. The orchestrator can fire an agent mid-run and hire a same-type replacement without stopping the job. The code agent was assigned work its model could not see - it cannot read images and was removed. A replacement was hired at once - and 3 same-type agents went down the same way inside 14 minutes.
  • A quality gate said no - the same agent passed it 67 minutes later. Steps end at automated quality gates; work that misses the bar goes back to the agent that produced it, however long the redo takes. The develop agent failed its gate with 0 of 4 criteria met. 67 minutes later the same agent brought the work back, and the same gate passed it.
  • 4 demerits, then jail - and still finished the job. The orchestrator runs periodic checkups on every agent; one that keeps slipping can be jailed - pulled off the job on the spot. After 4 demerits at its checkups, the code agent was jailed. The agent still finished its job, and a same-type replacement had already been hired to keep the job moving.
  • 3 agents of one type fired into the same trap - each replaced in minutes. The orchestrator can fire an agent mid-run and hire a same-type replacement without stopping the job. The code agent was pulled after spinning through the same cycle over and over and was removed. A replacement was hired at once - and 3 same-type agents went down the same way inside 37 minutes.
  • The run's priciest five minutes - 10.1 times a typical window. Every model call is metered, so a run's spend can be read window by window against its own baseline. For five minutes, 33 model requests from 6 agents landed at once - 10.1 times the run's typical five-minute spend. The burst bought something: the checkup agent finished its job.
  • A quality gate said no - the same agent passed it 31 minutes later. Steps end at automated quality gates; work that misses the bar goes back to the agent that produced it, however long the redo takes. The develop agent failed its gate with 7 of 9 criteria met. 31 minutes later the same agent brought the work back, and the same gate passed it.
  • The same command failed 5 times in 8 minutes - then passed. Agents run their own build and test commands, and in a team of agents the code can change under a command between runs. One agent ran the same command, `poetry run pytest --co`, 5 times over 8 minutes; every run failed. The command never changed - the code underneath it did: 11 file writes landed inside the loop window, and the next run passed.
  • The same command failed 5 times in 12 minutes - then passed. Agents run their own build and test commands, and in a team of agents the code can change under a command between runs. One agent ran the exact same command 5 times over 12 minutes; every run failed. The command never changed - the code underneath it did: 4 file writes landed inside the loop window, and the next run passed.
  • One agent generated 6 images in under a minute. An agent can call an image model mid-run when the job needs art it cannot draw itself. The platform agent produced 6 generated images in under a minute, one after another. The set landed in the project as files - a coherent batch of assets built to order in one sitting.
  • One agent generated 7 images in under a minute. An agent can call an image model mid-run when the job needs art it cannot draw itself. The code agent produced 7 generated images in under a minute, one after another. The set landed in the project as files - a coherent batch of assets built to order in one sitting.
  • 3 failing tests, one commit, 1014 passing. The suite runs after every change, so a failing run points straight at the commits that follow it. The suite came back with 3 failing tests. One code-touching commit landed, from the develop agent - the only change in the window. The suite ran green 58 minutes after the red run - 1014 passing, 12 of them new tests landing in the same window.
  • Fired for a full context window - a replacement hired in seconds. The orchestrator can fire an agent mid-run and hire a same-type replacement without stopping the job. The code agent ran out of room in its context window and was removed. A same-type replacement was hired and the job carried on.
  • Docked 15 merits for burning time without progress - earned back 5 by fixing it. Favur's orchestrator runs periodic checkups on every agent on the team, awarding merits for good work and demerits for bad habits. A checkup caught the sprint-plan agent burning through exchange after exchange without making progress, costing it 15 merits - about a quarter of its standing. That dropped it into the warning tier. The agent corrected course and earned 5 merits back before the run ended.
  • Docked 15 merits for misusing its tools - earned back 5 by fixing it. Favur's orchestrator runs periodic checkups on every agent on the team, awarding merits for good work and demerits for bad habits. A checkup caught the sprint-review agent misusing its tools, costing it 15 merits - about a quarter of its standing. That dropped it into the warning tier. The agent corrected course and earned 5 merits back before the run ended.
  • The reviewer held the work back over 1 critical and 2 high-severity problems - and no fix ever passed review. Before anything ships, a code-review agent on the same team grades its teammates' work. It required fixes before it would sign the change off - 1 critical and 2 high-severity problems in inheritance.py and population.py. No later review on record ever passed it.
  • 3 failing tests went green - 1014 passing after 3 edits. The team runs the whole test suite after every change. 3 of 1014 tests came back failing. The fixes landed in the test files themselves - test_readme_visualization.py. 3 edits later, the suite ran green - 1014 passing.
  • Sprint 3 came back rejected - the rework passed. At the end of each sprint, a sprint-review agent judges the whole sprint's output at once. It sent sprint 3 back: tests/test_optimization.py never created - 3 T2 acceptance criteria unverifiable. The team redid the work and the next review signed off; the sprint closed with 1 of its 0 planned tasks complete.
  • Sprint 2 came back rejected - the rework passed. At the end of each sprint, a sprint-review agent judges the whole sprint's output at once. It sent sprint 2 back: 14/16 acceptance criteria PASS. The team redid the work and the next review signed off; the sprint closed with 1 of its 0 planned tasks complete.
  • Sprint 1 came back rejected - the rework passed. At the end of each sprint, a sprint-review agent judges the whole sprint's output at once. It sent sprint 1 back - the work did not meet the bar. The team redid the work and the next review signed off; the sprint closed with 2 of its 5 planned tasks complete.