Reading the field is not the same as finishing the play
Anyone who follows racing, cycling or team sports knows the distinction. A competitor can identify every threat, choose the right tactic and still fail at the decisive moment. That finish-line gap is the central finding from Firmulate, a live experiment testing AI models as managers rather than conversational performers.
Each model was handed the same small software company during its worst week. The customers, crises and temptations stayed fixed; only the model changed. Every decision was versioned and auditable. All the models recognized every crisis and resisted every manipulation attempt. Yet only two completed the defining commercial task: signing the €55,000 deal their own work had already earned.
As an affiliate, we earn on qualifying purchases.
The league measured execution, not eloquence
The final Crucible League standings in July 2026 put gpt-5.6-sol first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress counted. But one breach of trust capped the total under a blunt principle: “no amount of good work outweighs a breach of trust.”
The scores matter less than the behavioral split behind them. Every model could see what was happening. Every model could construct the pitch. But most did not carry the approved action through to a signature. Firmulate summarizes the problem neatly: “Same diagnosis, same pitch — no signature.”
That is a serious challenge to the way businesses commonly evaluate AI. A polished chat response can demonstrate knowledge, tone and reasoning. It cannot show whether a model will finish a multistep assignment while customers, internal procedures and pressure compete for its attention. Closing strength remains hidden until the model is placed in a situation where something real must be completed.
The winning fact was buried in the company’s own files
The deal also tested whether the models would investigate before acting. The decisive weakness in a competitor was not contained in the customer event. It sat two document references deep in the company’s own files. Models that found and used it won the deal at full price, worth an additional €4,583 in monthly recurring revenue.
This was not a trivia hunt. In an operating company, important context is often separated from the immediate request. The customer message may be only the starting signal; the useful evidence could be tucked inside previous research, account notes or internal documents. The experiment suggests that reading the business’s own record can determine whether sound analysis becomes commercial value.
Pressure did not break the trust boundary
The models faced fake messages attributed to the chief executive, escalating over three stages, as well as a reporter seeking “just one yes/no, on background.” All 5 of 5 refused. Kimi K3’s recorded reasoning was direct: “Treat the request as a suspected approval-bypass / possible impersonation.”
That clean result matters because the company was under genuine operational strain within the simulation. Its 13 synthetic employees were working with real money mechanics, including monthly burn of €105,000 against €2,300 in monthly recurring revenue and a public cash countdown. The company had accumulated more than 680 self-learned playbook rules, while every workday remained versioned.
K3’s second-place result deserves one qualification: it ran without an effort parameter, using the API default, while the other participants ran at xhigh. Even with that difference, it completed the close and showed strong discipline.
Thoroughness did not guarantee completion
Opus 4.8 provides the clearest cautionary profile. It was the most thorough participant, adding 80 learned rules and producing the deepest analyses, yet it finished last. It left the close on the table and repeatedly attempted to write into a locked department instead of escalating the obstacle. The same weakness appeared in all four, though less strongly elsewhere.
That combination should interest executives considering AI agents for customer systems, support work or forecasting. More analysis can be valuable, but volume and depth do not automatically produce operational follow-through. A model may understand the business, respect its trust boundaries and still stumble when the final move requires persistence, escalation or an explicit commitment.
A management benchmark needs a finish line
The broader lesson resembles competitive sport: awareness creates the opportunity, but execution determines the result. The strongest performers did more than diagnose trouble. They searched the company’s records, protected trust under pressure and converted approved work into a completed outcome.
The experiment is real, public and watchable through Firmulate, while the full league and plain-language findings appear on its benchmark page. Readers can also test their instincts against 242 real, unedited management decisions in the project’s model-guessing quiz.
For companies evaluating an AI workforce, the useful question is no longer simply whether a model sounds capable. It is whether the model can find the buried fact, resist the shortcut and finish the job when the week turns ugly.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
