The Pilot Worked. So Why Is No One Using It?
Most of the AI pilots I am asked to look at did what they were built to do. The model summarized the files, drafted the responses or flagged the exceptions. The demo went well and the steering committee was pleased. Six months later the tool is still a pilot, a handful of enthusiasts use it, and nothing in the operating budget has changed.
The data now shows the same pattern at scale. McKinsey's 2026 State of AI survey of 1,719 respondents found that nearly nine in ten report regular use of AI in at least one business function, and 44 percent say AI is scaling across their enterprise, up from 38 percent a year earlier. Yet only 37 percent attribute any EBIT impact to AI use, about the same share as last year (McKinsey, "The state of AI in 2026: On the road to ROI," August 2026). Use is spreading faster than results.
RAND reached a similar conclusion from the technical side. After interviewing 65 experienced data scientists and engineers, its researchers put this root cause of failure first on their list: leaders and technical teams misunderstanding or miscommunicating the problem the AI was meant to solve. Another was chasing the newest technology rather than the real problems of the people meant to use it (RAND, "The Root Causes of Failure for Artificial Intelligence Projects and How They Can Succeed," 2024).
None of this is unique to AI. In 25+ years across Big 4 consulting, a global bank's treasury services and systems integration, I have watched every generation of technology fail the same way. A system that passes its test and misses the work does not get adopted. It gets worked around.
A pilot proves the technology. Adoption proves the change.
Most pilots are designed to answer one question: can the tool do the task? That is a useful question, but it is the easy one. The harder question is whether the people who do the work will do it differently, every day, once the pilot team has moved on.
The McKinsey survey points to the difference. The roughly 6 percent of respondents it classes as AI high performers are far more likely to rebuild the work itself: nearly three-quarters report fundamentally redesigning workflows because of their AI use, against about one-quarter of other respondents. In McKinsey's words, they "fundamentally redesign workflows that are enabled by AI rather than insert AI into existing ones." They are also twice as likely to say their organizations have defined processes to measure the impact of their AI initiatives.
That is the gap I see in most stalled pilots. The tool was inserted into the old process, and no one decided in advance how success in the work would be measured.
Four questions before a pilot is called a success
Before I would call a pilot ready to scale, I ask the sponsor to answer four questions in writing.
Which piece of work changes, and who does it differently next Monday? Name the role, the task and the step that moves into the new tool. "Staff will use AI to be more efficient" is not an answer.
What gets switched off? Every new tool should retire something: a spreadsheet, a manual review step, a legacy screen. If nothing is retired, the organization is paying for two ways of working.
What is the baseline? Measure cycle time, rework or backlog before the pilot starts. Without a baseline, the business case rests on how the demo felt.
Who owns adoption after go-live? The pilot team proves the concept. Someone in the operating line has to own usage, friction and fixes for the first 90 days.
None of these questions is technical. Every one of them decides whether the investment pays back.
What this looks like at scale
When I designed an identity and access architecture governing 50,000+ users for a major U.S. police department, the technical design was only part of the job. Fifty thousand users means fifty thousand daily routines, and access that slows down a routine task invites a workaround. The same lesson held when I led multi-agency platform rollouts for two U.S. state governments: the platform was shared, but each agency's work was its own, and each needed its own plan for how that work would change.
AI raises the stakes on this lesson rather than changing it. A model that drafts a response in seconds saves nothing if a reviewer still rekeys it into the old system.
A 30-day path from pilot to production
If you have a pilot that worked but has not scaled, a month is usually enough to find out why and decide what to do.
Week 1: Sit with the people who do the work. Map the real process the tool is meant to change, including the workarounds.
Week 2: Set the baseline and the adoption measures: which tasks move, what retires, by when.
Week 3: Redesign the workflow around the tool, name the operating owner and agree what gets switched off.
Week 4: Decide with evidence: scale it, fix it or stop it.
Stopping a pilot is a legitimate outcome. Funding a tool no one uses is not.
Process comes before platform. A system that matches how people work gets used; one that does not gets worked around.
If you have an AI pilot or a new platform that has not reached the people it was meant for, book a 30-minute conversation about a Readiness Assessment or Program Health Check. Message me on LinkedIn or write to info@djsc.net.
Comments