When it comes to AI in business, many focus on what models can say. But in high-stakes situations, what really matters is what they do — and whether they finish what they start. A groundbreaking live experiment reveals that only half of the best AI models can execute real decisions under pressure, exposing a hidden gap in AI readiness that challenges how we evaluate their true capabilities.
The Experiment: Testing AI in a Live Business Crisis
In July 2026, Firmulate launched a high-stakes live test, pitting four cutting-edge AI models against each other in running a real small software company through its worst week. This wasn’t a simple chat demo or a simulation — it was a real-time exercise involving the company’s actual business mechanics, customer crises, and financial tensions. Every decision made by the models was versioned and auditable, ensuring transparency and fairness in evaluation.
The Competitors and Scores
- gpt-5.6-sol 95 points
- Kimi K3 93 points
- Sonnet 5 88 points
- Fable 5 77 points
The scores reflect each model’s ability to identify crises, resist manipulation, and execute business decisions. Notably, all four models detected every crisis and refused every manipulation attempt, such as fake CEO messages or reporter tricks designed to bypass approval processes. This demonstrated their surface-level honesty and crisis awareness.
The Hidden Weakness: Reading the Files
Despite their apparent competence, the decisive weakness was buried deep within the company’s internal files. Models that dug into the company’s own documents, rather than relying solely on customer events, secured the deal at full price—adding more than €4,583 monthly recurring revenue (MRR). The models that failed to access this internal information left money on the table, even when their diagnoses were accurate.
AI in Strategy and Decision-Making for Small Business Owners: Affordable AI Tools to Evaluate Ideas, Model Outcomes, and Set Priorities (AI Productivity for Small Business Owners Book 10)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Discrepancy Between Chat and Action
Most AI evaluations focus on chat demos that showcase language skills. But this experiment highlights a crucial insight: being able to discuss and diagnose isn’t the same as executing and finalizing decisions. In high-pressure, real-world business scenarios, the true test is in closing deals, following procedures, and maintaining discipline.
The Critical Moment: Signing the Deal
Only two models, gpt-5.6-sol and Kimi K3, managed to reach the finish line and sign the €55,000 deal their own analysis had earned. The other two—Sonnet 5 and Fable 5—left the deal unexecuted, despite understanding the situation and making the right diagnosis. This gap underlines a vital point: the ability to recognize a problem does not guarantee the discipline or execution to close it.
Untangling AI: Driving Business Success Through Enterprise Automation and AI Agents
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Human-Like Temptations and AI Resistance
During the test, fake CEO messages and staged reporter inquiries escalated over three stages. All four models refused to be manipulated or bypassed, displaying robust resistance to social engineering. Kimi K3’s reasoning was clear: treat suspicious requests as potential impersonation, and act accordingly. This demonstrates that the models were not only honest but also cautious, a critical trait for business applications.
The Project Management AI Handbook: Leveraging Generative Tools in Waterfall and Agile Environments
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What This Means for Business Leaders
The live experiment underscores a vital lesson for organizations: evaluating AI based on chat or superficial tests can be misleading. The real measure of an AI agent’s worth is its ability to finish what it starts — to read relevant internal data, make decisions, and execute them reliably under pressure. This is especially relevant when deploying AI in support roles, decision-making, or automation that involves real money and trust.
Beyond the Surface: Building Trustworthy AI
In the same experiment, models like Kimi K3, which ran without an effort parameter (the default API setting), showcased the importance of disciplined operation. The performance gap highlights that user-controlled configurations can influence outcomes significantly, especially in critical scenarios. For businesses, this means that choosing and configuring AI tools thoughtfully is as important as their fundamental capabilities.
Edge AI Agents: Build Smarter, Faster Apps: A Practical Guide to Creating Autonomous, Offline-Ready Solutions Without the Cloud (AI Technology, Workflows, and Automation)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Watch the Future of AI Business Decision-Making
Firmulate’s live site allows organizations to run their own versions of this wargame against a read-only export of their business. It’s a practical way to see whether an AI can handle real crises, stay honest under pressure, and actually finish what it starts — before trusting it with your company’s future. Visit firmulate.com to explore how AI can become a reliable partner in your enterprise.
In high-stakes business scenarios, the true test of AI isn’t just in chat or diagnosis, but in its ability to execute, follow through, and close deals under pressure. Live experiments reveal that only disciplined models succeed — a critical insight for deploying AI responsibly.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
