The demo went well. That is almost never in question. Someone on the leadership team watched a chatbot answer a tricky customer question, or a drafting tool turn a rough brief into a clean first pass in nine seconds, and the room agreed this was worth doing. Three months later the tool is still technically live, and almost nobody is opening it.
That gap — between "it worked when I tried it" and "the team actually uses it" — is not a rare failure mode. It is the modal outcome. The research on this over the past year is unusually blunt for the subject matter, and unusually consistent across firms that have no reason to agree with each other on much else.
What the numbers actually say
- 95% of generative AI pilots deliver no measurable return to the P&L, according to MIT's 2025 State of AI in Business report, built from 300 public deployments and interviews with 52 executives.
- 97% of executives report deploying AI agents in the past year — but only about 52% of employees report actually using them.
- Gartner expects over 40% of agentic AI projects to be cancelled by the end of 2027, citing escalating cost, unclear business value, and inadequate risk controls.
- Frontline adoption has stalled at roughly 51% even as overall AI usage climbs, per BCG's 2025 workforce survey of over 10,600 employees across 11 countries.
- 68% of employees report zero AI training in the prior twelve months, and training is the single strongest predictor of whether a tool sticks.
01The demo and the rollout are measuring different things
A demo answers one question: can this tool do the task, under conditions someone controlled? A rollout answers a completely different one: will the people who did not choose this tool, who have a way of doing the task already, change their behaviour for it — every day, without being watched?
Those are not the same test, and passing the first tells you almost nothing about the second. MIT's researchers, working from a large sample of enterprise deployments rather than self-reported surveys, put a number on how rarely the second test gets passed: 95% of pilots show no measurable financial impact once they leave the pilot stage. The gap they describe is not that the models are bad. It is what they call a "learning gap" — tools that do not adapt to how a specific team actually works, dropped into workflows that were never redesigned to use them.
The uncomfortable part is that a successful demo can make this worse, not better. A clean demo is evidence the tool works in principle, and leadership treats it as evidence the rollout will go the same way. It rarely does, because the demo never had to survive contact with someone's actual Tuesday.
02Nobody owns whether it gets used
Most software rollouts fail quietly for a boring reason: buying the tool and getting people to use it are two different jobs, and the second one is frequently nobody's job at all. Whoever championed the pilot — often an executive, sometimes an outside vendor — considers their work done once the contract is signed and the pilot posts a good result. Adoption after that point has no owner, no metric, and no deadline.
That absence shows up directly in the research on what makes adoption actually happen. Tools that reached near-universal use in BCG's data shared three things: a named internal sponsor, a real enablement plan, and a shared knowledge base people could return to when something did not work the way the demo suggested. Tools without those three things regressed toward whatever the path of least resistance already was — which is usually the old way of doing it.
The question worth asking before you buy
Not "does this tool work" — the vendor will show you that it does. Ask instead: whose job is it to notice, in week six, that nobody is opening this? If there is no answer, there is no rollout plan, only a purchase.
03The demo used clean data. The team has messy data.
A vendor demo runs on curated inputs — a handful of representative customer emails, a tidy sample spreadsheet, a support ticket chosen because it illustrates the feature well. Real usage runs on whatever actually lands in the inbox: half-finished sentences, attachments in the wrong format, a customer who mentions three unrelated things in one message, records with missing fields going back a decade.
The accuracy gap between those two conditions is well documented outside AI too — it is the same reason a tool that scores well in a lab consistently underperforms once deployed against real-world conditions, sometimes losing 40 to 50 percentage points of accuracy in the transition. Enterprise data specifically compounds this: inconsistent field definitions, missing history, and format mismatches across the systems a small business actually runs are exactly the kind of noise a demo dataset was built to avoid.
The team does not usually conclude "the data was messy." They conclude "the tool was wrong," stop trusting it on the second or third bad answer, and quietly route around it. By the time anyone notices usage has dropped, the team has already decided.
04It was picked by the people who will not use it
Leadership evaluates a tool against a strategic question — does this reduce cost, does this look like where the industry is heading. The team evaluates it against a tactical one — does this make my next hour easier or harder. A tool can win the first argument decisively and lose the second one on day one, and the second argument is the one that determines whether it survives.
This is where the training gap becomes the whole story rather than a footnote to it. BCG's data shows workers who received more than five hours of AI training became regular users at a materially higher rate than those who received less — 79% versus 67% — and the productivity gap compounds from there: trained employees reported saving roughly twice the hours per week of untrained ones using the identical tool. A tool selected without the people who will run it, and handed over without training, is carrying two separate reasons to fail before anyone has typed a single prompt.
| In the demo | In the rollout | Why it breaks |
|---|---|---|
| Curated sample inputs, chosen to show the feature well | Whatever the team actually gets — messy, incomplete, off-format | Accuracy that looked solid in the demo degrades against real-world noise, and trust drops after the first couple of visibly wrong answers. |
| One motivated presenter, running the tool exactly as intended | Everyone else, running it around an existing workflow nobody redesigned | The tool adds a step instead of replacing one, and people default to whatever was already faster. |
| An executive sponsor watching closely and asking good questions | No one checking whether it is still being opened in week six | Usage has no owner, so a quiet decline is invisible until someone asks why the invoice looks the same as usage looks lower. |
| The person who chose the tool is also the one demoing it | The people using it were not in the room when it was chosen | A tool selected on strategic merit can still lose on day-to-day usability, and usability is what decides adoption. |
| A single session, no training required to follow along | Ongoing use, with no training budget or enablement plan | Training is the strongest documented predictor of whether a tool sticks; skipping it is the cheapest-looking way to guarantee it will not. |
Patterns drawn from the MIT, Gartner, and BCG research cited throughout this article. Any individual rollout will differ, but these are the failure modes that recur across the largest published studies.
05Nobody measured whether it was actually being used
This is the pattern behind every other one on this list: the tool's fate gets decided by a metric nobody is tracking. Leadership sees the invoice, which keeps renewing whether or not anyone opens the tool. They do not see login frequency, task completion, or the quiet decline from daily use to "I'll use it when I remember to." The 97%-deployed-but-52%-actually-using-it gap is not a mystery to close — it is a measurement nobody set up in the first place.
Most small businesses that run a pilot could tell you exactly what it cost. Very few could tell you, three months in, how many of the people it was bought for still open it in a normal week. That single number, tracked from week one instead of discovered by accident at renewal, is the cheapest early-warning system available and almost nobody builds it.
06What to actually do
- Pick the metric before you pick the tool. Decide what "actually being used" means — logins per week, tasks completed, whatever fits the workflow — and check it monthly from day one, not at renewal when the answer is already six months stale.
- Name an owner for adoption, separate from whoever championed the purchase. Someone has to be accountable for noticing a quiet decline, and "everyone" is the same as no one.
- Test it on your worst inputs, not your best. Before rollout, run it against the messiest real records you have — the incomplete form, the ambiguous email, the mismatched spreadsheet — not the clean example the vendor supplied.
- Put the people who will use it in the room before you buy it, not after. A tool chosen without them is a tool they had no reason to trust on day one.
- Budget for training as part of the purchase, not as an afterthought. The data is consistent that this is the single highest-leverage line item, and it is usually the first thing cut.
- Redesign the step, don't just add one. A tool that sits on top of the existing workflow as an extra action loses to whatever was already faster; one that actually replaces a step has a real chance.
- Give it a real review date, not an indefinite pilot. Decide up front what "working" looks like at 90 days, and be willing to kill it if the usage number says no — a cancelled pilot is far cheaper than a quietly unused subscription renewing for two years.
07The honest summary
The demo is not lying to you. It is answering a narrower question than the one that decides whether the tool survives. Nothing in the research says AI does not work — it says that most organisations are not doing the unglamorous parts that turn a good pilot into a used tool: naming an owner, training the people who will run it, testing against real conditions instead of curated ones, and actually measuring whether anyone is still opening it once the excitement of the demo has worn off.
None of that is a reason to avoid the pilot. It is a reason to plan the rollout with the same seriousness you planned the demo — because on the numbers above, the rollout is where almost everyone else stops trying.
08Common questions
Why do AI pilots that work in a demo fail once rolled out to the whole team?
A demo tests whether the tool can do the task under conditions someone controlled — curated inputs, a motivated presenter, close attention. A rollout tests whether people who did not choose the tool will change a daily habit for it, using whatever messy real data actually shows up, with no one watching. MIT's 2025 research on enterprise AI deployments found 95% of pilots deliver no measurable financial return once they move past that stage, largely because tools are not adapted to how the specific team already works.
What percentage of AI pilots actually fail?
MIT's State of AI in Business 2025 report, built from interviews with 52 executives and analysis of 300 public deployments, found 95% of generative AI pilots show no measurable impact on the P&L. Separately, Gartner projects that over 40% of agentic AI projects specifically will be cancelled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.
If leadership rolled out an AI tool, why isn't the team using it?
Because leadership and the team are answering different questions. Leadership asks whether the tool advances a strategic goal; the team asks whether it makes the next hour easier or harder, and that second question decides daily use. Surveys show a persistent gap between deployment and actual use — one 2025 report found 97% of executives had deployed AI agents in the past year, while only about 52% of employees reported actually using them.
Does training really affect whether an AI tool gets adopted?
It is one of the strongest predictors researchers have found. BCG's 2025 workforce survey of over 10,600 employees across 11 countries found that employees who received more than five hours of AI training became regular users at a notably higher rate (79%) than those who received less (67%), and reported roughly double the weekly time savings of untrained employees using the same tools. Yet 68% of employees report receiving no AI training at all in the prior year.
Why does an AI tool that scored well in testing perform worse on real work?
Vendor demos and internal pilots typically run on clean, curated sample data chosen to showcase the feature well. Real business data is messier — incomplete records, inconsistent formats, ambiguous inputs — and accuracy commonly degrades significantly when a tool moves from a controlled test to real-world conditions, sometimes by tens of percentage points depending on the domain. Once a team sees a couple of visibly wrong answers on real inputs, trust drops fast and people quietly route around the tool.
How do we know if our team is actually using an AI tool we bought, versus just having access to it?
Access and usage are different facts, and most small businesses only track the first one — the invoice keeps renewing whether or not anyone opens the tool. Pick a concrete usage metric before rollout (logins per week, tasks completed through the tool, whatever fits the workflow) and check it monthly starting in week one, rather than discovering the answer by accident at renewal, months after usage quietly declined.
Whose job should it be to make sure a new AI tool is actually adopted?
It needs a named owner separate from whoever championed the purchase, because buying the tool and driving adoption of it are two different jobs. Research on which pilots reach broad, sustained use consistently points to the same combination: a named internal sponsor, a real enablement and training plan, and a shared resource people can return to when the tool does not behave the way the demo suggested.
Should we just cancel an AI pilot that isn't getting used?
If it has a real review date and a usage number that says no, yes — a cancelled pilot is far cheaper than a subscription that quietly renews for two years while almost nobody opens it. Set the definition of 'working' before rollout starts, not after the invoice has renewed a few times, so the decision to stop is a planned checkpoint rather than something that only gets noticed by accident.
Before you roll it out past the pilot
Tell us what the pilot was and what it looked like when it worked. We will help you name a usage metric, a realistic worst-case data test, and an owner for adoption before it goes to the full team — the three things the research says most rollouts skip. We are not paid more if you buy more tools, so there is nothing in this for us except getting the rollout right.
Ask for a rollout checkSources, read 18 September 2026: MIT NANDA / MIT State of AI in Business 2025 report on the 95% generative AI pilot failure rate ("The GenAI Divide"), as reported by Fortune, Forbes, and the Yahoo Finance summary of the study, August 2025; Gartner's June 2025 press release projecting over 40% of agentic AI projects cancelled by 2027; BCG's "AI at Work 2025" workforce survey (10,600+ respondents, 11 countries) on frontline adoption stalling near 51% and the training-hours-to-adoption relationship; 2025-2026 reporting on the gap between executive-reported AI deployment (97%) and employee-reported usage (52%); and industry reporting on real-world accuracy degradation versus controlled/demo test conditions. Related: The Five-Tool Problem and Shadow AI: Staff Using Tools You Did Not Approve.
Hero image from Unsplash, used under the Unsplash License.