‍Why Most GenAI Pilots Stall — and How Some Succeed

Why Most GenAI Pilots Stall — and How a Few Succeed

95% of AI projects faill.

Over the summer I had begun to hear some version of this quote being referenced in my circles, eventually the source material for the quotes found its way into my path, it come's from MIT's Project Nanda which had published an extensive report on their findings in June 2025.

The actual quote was "Just 5% of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact."

Much less damning, the findings were focused on GenAI, not all AI, and more specifically, that most GenAI pilots are not scaling to production to a point where they could generate bottom-line impact of over $1 million.

Their findings explained that it wasn't infrastructure (good data), regulation (cybersecurity concerns), or even talent (AI skills).
It’s because most GenAI products share three problems: 

  1. They don’t retain feedback.
  2. They don’t adapt to context.
  3. They don’t improve over time.

General purpose, $20 a month subscriptions like ChatGPT are often better then customized enterprise solutions because they are excellent for writing first drafts and simple work. For complex work that might take days or weeks, work that needs to be done with customers or other teams, 9 out of 10 times, the human is still preferred even over these tools.

A corporate lawyer interviewed by MIT Nanda summed it up nicely:

"Our purchased AI tool provided rigid summaries with limited customization options. With ChatGPT, I can guide the conversation and iterate until I get exactly what I need. The fundamental quality difference is noticeable, ChatGPT consistently produces better outputs, even though our vendor claims to use the same underlying technology "It's excellent for brainstorming and first drafts, but it doesn't retain knowledge of client preferences or learn from previous edits. It repeats the same mistakes and requires extensive context input for each session. For high-stakes work, I need a system that accumulates knowledge and improves over time."."
Source: The GenAI Divide State of AI in Business, MIT Nanda

What Are Those That Succeed Doing Differently?

Many AI (and change) initiatives fail because the people closest to the problem aren’t involved in designing the solution, leaders too far from the day to day provide incomplete or unnecessary solutions. Those companies and startups that have built impactful GenAI take a more distributed approach. Leaders ask teams how AI might help seize opportunities or fix real workflow bottlenecks.

Instead, if leaders want to use GenAI to address opportunities or problems, them simply say so, explain why it must be done, by that's really all it takes, -- ask. Ask your teams "How might we use GenAI in Sales, Operations, Finance, etc.) Then allows teams to describe problems in detail, then enable them to work with engineers to design solutions, those engineers could be yours or those of an external partner.

Bottom-Up Innovation: Leaders need only as how

Business teams will adopt new technologies and ways of working, if there is a purpose and they have a hand in how they will adapt. But with new technologies it's still best to ask people to change less then more. Putting your business teams together with designers and engineers is the best way to do this. When designers and engineers know how your business teams actually work and the technologies in use today, you'll be amazed at what they can produce.

Consider document automation for contracts and forms. One client we worked with had dozens of budget templates and other reporting documents that varied with the type of client they were working for. These documents also updated periodically over the relationship with that client. Automating creation, submission, and approval of key documents across project cycles would require access to project details stored in CRM, ERP, and HriS software. Such a product would know what document needed completion, and what information that user needed to supply, work that may previously have invovled calls or meetings with peers or vendors. In short, such a tool would understand the context of the work at hand, so the user wouldn't need to provide context by prompting, and if it did make a mistake, users feedback would mean it would not make the same mistake again.

Narrow applications like this—embedded directly into existing workflows—are far more likely to succeed and deliver real ROI.

The future of GenAI won’t be defined by bigger and more sophisticated model of its models, but by how well organizations learn to integrate them into the way people actually work. The most successful deployments combine context, memory, and learning, built on process maps and user journeys co-created by business and engineering teams. This is as much a problem of change management as a technology problem, maybe more so.

References: 

The GenAI DivideSTATE OF AI IN BUSINESS 2025 https://www.artificialintelligence-news.com/wp-content/uploads/2025/08/ai_report_2025.pdf