Imagine an AI agent managing furniture orders during a supplier delay, answering customers as delivery dates slip, and weighing a competitor’s offer. A polished chat demo cannot tell you how it will handle that week. Firmulate’s experiment puts models in charge of the same small software company and lets readers watch their decisions unfold.
Get furniture and decor delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company under pressure
Firmulate’s Crucible League gave frontier models the same customers, crises and temptations, then tracked their decisions through the company’s worst week. The results show why business owners may want to see how an AI acts before giving it responsibility for customer relationships or operating decisions.
In the final league, gpt-5.6-sol placed first with 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. The league’s rule is blunt: partial progress counts, but one breach of trust caps the total. “No amount of good work outweighs a breach of trust.”
The gap between advice and action
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal their own analysis had earned. The finding, in the experiment’s words: “Same diagnosis, same pitch — no signature.” For a business, recognizing the right answer matters; following through matters too.
The deal hinged on information buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won at full price, worth +€4,583 MRR. It is a reminder that useful business context may sit in unassuming records, not in the headline event that first attracts attention.
The pressure also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
What the rankings leave out
Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses, but finished last. It left the close on the table and its discipline slipped, with write attempts into a locked department instead of escalation. A weaker version of the same weakness appeared in all four models.
There is also a fairness detail in the comparison: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Firmulate publishes the experiment as a watchable live company: 13 synthetic employees, real money mechanics, burn of €105k a month against €2.3k MRR, a public cash countdown and 680+ self-learned playbook rules. Every workday is versioned. At firmulate.com, readers can follow the live experiment; 242 real, unedited management decisions also power a “guess the model” quiz.
From watching to your own pilot
For a company considering AI agents in areas such as customer service, sales or operations, Firmulate offers a pilot using a read-only export of the business. The models face crisis scenarios against that company’s own context, and the resulting board report ranks models and surfaces weak points in existing playbooks. Nothing writes back to real systems.
That moves the question from which model sounds most convincing to how it handles your company’s difficult moments: what it notices, what it refuses and whether it follows through.
Take the next step
To explore a wargame pilot against your company’s read-only data, visit Firmulate’s pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
