A proof of concept can look successful and still leave the buying decision unanswered. A demo built by enthusiasts, on clean sample data, in front of people who want it to work, almost always works.

Then nothing happens. Six months later someone asks whether "we ever did anything with that AI thing".

The short answer: an AI proof of concept is a short, time-boxed test that answers one business question with real data: can this tool do this specific job well enough, cheaply enough and safely enough to be worth rolling out? To make it lead somewhere, agree the success threshold, the kill criteria, the owner and the production budget before anyone builds anything.

Key points:

  • Gartner predicted at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025 (Gartner).
  • Survey respondents told S&P Global that on average 46% of AI projects are scrapped between proof of concept and broad adoption (S&P Global Market Intelligence).
  • A good POC tests adoption and integration, not just accuracy.
  • Killing a POC on schedule is a success. Letting it drift is the failure.

What is an AI proof of concept?

An AI proof of concept (often shortened to AI POC) is a small, cheap experiment designed to answer a yes-or-no question before you spend real money. Can a language model draft first-pass replies to our support tickets that agents accept with light edits? Can it pull the right fields out of our supplier invoices? Can it triage inbound leads as well as our best salesperson?

A vendor demo is a different animal. It answers the vendor's question ("does our product look good?"), and you need your own answered.

Think of it as a stage gate. I've taught project management for years, and the PRINCE2 habit of committing to the next stage only once the current one has justified itself fits AI work almost perfectly. A proof of concept is a gate, not a launch.

sticky notes on corkboard
Illustrative photograph. Photo: Jo Szczepanska / Unsplash.

What's the difference between an AI POC and a pilot?

People use the words interchangeably, and that confusion causes half the trouble. They answer different questions, so they need different set-ups.

Proof of concept Pilot programme Production
Question it answers Can it do the job at all, on our data? Will our people use it in the real workflow? Does it keep delivering at scale?
Who uses it A small project team A real team doing real work Everyone in scope
Data A representative sample of real cases Live work, with a fallback Live work
Integration Minimal or manual Connected to the tools people already use Fully integrated, monitored, supported
Typical length A few weeks One to three months Ongoing
Output A go, change or stop decision A rollout plan and a business case Results you can measure

Those lengths are working ranges, not rules. The sequence is what matters. Skipping straight from a successful POC to company-wide rollout is how you discover, expensively, that the accuracy was fine and nobody opened the tool.

Why do so many AI proofs of concept go nowhere?

The numbers vary by survey, but they all point the same way. IDC research with Lenovo, reported by CIO, found that for every 33 AI POCs a company launched, only four reached production. S&P Global found the share of companies abandoning most of their AI initiatives before production jumped from 17% to 42% in a year. MIT's NANDA study found that of organisations evaluating enterprise-grade AI tools, 60% looked, 20% reached a pilot and 5% reached production (MIT NANDA, The GenAI Divide). That last one is based on 52 interviews and self-reported outcomes, so treat it as directional.

The reasons are boringly consistent:

  • No business question. The POC exists because the board asked "what are we doing about AI?". The analysts quoted by CIO were blunt: POCs get approved too easily, often under pressure from the top, and often without a business case.
  • Clean data in, messy data later. Gartner's list of reasons for abandonment leads with poor data quality, followed by inadequate risk controls, escalating costs and unclear business value.
  • No owner. The innovation team runs it, then hands it to people who never asked for it.
  • Tested outside the workflow. MIT's interviewees put resistance to adopting new tools at the top of their barriers, ahead of model quality.
  • Nobody budgeted for production. The POC was funded. The integration, security review and support were not.

Most of the failures in why AI projects fail are baked in at this stage.

What should an AI proof of concept actually prove?

Four things. Most POCs test only the first.

  1. Quality. Does the output meet a bar you agreed in advance, such as "at least 8 in 10 drafts accepted with minor edits"? "It looked impressive" is not a bar.
  2. Cost per unit. What does each invoice, ticket or document cost in tool fees and checking time? Saving four minutes with three minutes of review is a rounding error.
  3. Fit. Can it sit inside the system people already work in, or does it need a new tab, a new login and a new habit?
  4. Risk. What happens when it's wrong? Who notices, and how fast?

Get the thresholds signed off before the build starts. Choose the metric after seeing the results and you're writing a press release, not running a test.

Always measure a baseline: how long the task takes today, and what it costs, on the same sample. The guide on how to measure AI ROI covers baselines properly.

How long should an AI POC take, and what does it cost?

For most small and mid-sized businesses testing an off-the-shelf tool or a model through an API, a few weeks is enough to get a decision. If it runs past the agreed end date, reassess scope, data access and the evidence still needed before extending it.

Speed matters. MIT's study found that the strongest mid-market performers took about 90 days from pilot to full implementation, while large enterprises took nine months or longer. Don't throw that advantage away by copying a corporate approval process.

On cost, I'm not going to quote you a market price, because there isn't a credible published benchmark for AI POCs and anyone who gives you a single number hasn't seen your data. What I can tell you is where the money goes:

  • Tool or model costs. Often small. A standard Claude Team seat is listed at US$25 a month on monthly billing, with a minimum of two seats (Claude Team standard seat), and API usage for a few hundred test cases is usually modest.
  • Your people's time. Usually the biggest cost, and the one nobody tracks: pulling sample data, labelling what "good" looks like, reviewing outputs.
  • Outside help. Consultant or developer days, which scale with integration work.

The more important number is the one after the POC. Production needs integration, security review, monitoring, training and support. Sketch that budget now, even roughly. The guide to AI implementation cost will help.

Can you use real customer data in an AI POC?

You should test on real cases, because hand-picked data is how a POC lies to you. But real data usually means personal data, and "it's only a test" is no exemption from data protection law.

In the UK, the ICO lists artificial intelligence and machine learning as examples of innovative technology that can trigger a data protection impact assessment, particularly when combined with other risk factors such as profiling or sensitive data (ICO). EU rules work along similar lines.

Practical rules I'd apply:

  • Use a business or enterprise tier where the vendor doesn't train on your data by default, never a personal account.
  • Minimise. Strip names and identifiers that the task doesn't need.
  • Check where the data is processed and how long it's kept.
  • Write a short note of what you used and why.

If you don't yet have a policy for any of this, a lightweight AI governance framework is worth putting in place before the POC, not after.

What I've seen

The demo is never the problem. One failure pattern is a POC that "succeeded" and then stalled because nobody had decided who would run it on Monday morning. The technology passed. The organisation didn't.

Kill criteria are a kindness. Teams find it oddly hard to stop a pilot: sunk cost, a sponsor's reputation, a vendor promising the next release. A pre-agreed stop condition takes the emotion out. Stopping a POC on schedule is a result, not a failure.

Build the gates in from day one. In my own lab, Ascendancy IQ, AI agents do real work, but under approval gates and controls, with a human signing off anything consequential. It's a working lab rather than a product, and the lesson carries over: decide where the human checkpoint sits during the POC, because it largely sets your real cost and your real risk.

Buy first, then build. Plenty of POCs are built from scratch when a tool already exists. More on that in build vs buy AI.

How to run an AI proof of concept, step by step

  1. Pick one task with a real cost. Something done often, by people, that you can measure today. If you're not sure where to start, the AI readiness checklist helps.
  2. Name an owner from the business. The person whose team will use it, not the person who's keenest on AI.
  3. Measure the baseline. Time, cost and error rate for the current process, on the same sample you'll test.
  4. Write the decision rule. "We scale if X, we change approach if Y, we stop if Z." Sign it off.
  5. Gather a representative sample. A few hundred real cases, including the awkward ones, handled under your data protection rules.
  6. Start with the simplest tool that could work. An existing product or a model through an API before anything custom.
  7. Test inside the workflow where you can. A rough connection to the real system beats a perfect standalone demo.
  8. Time-box it. Set an end date and hold to it.
  9. Decide, in a meeting, on the date. Go to pilot, change and retest once, or stop. Write down why.
  10. If it's a go, fund the production plan, not another POC. Integration, security, training and support, with a named budget.

From there, how to implement AI in your business takes over.

Keep some test cases out of the tuning loop

Separate the examples used to improve prompts from a held-back evaluation set. Include missing information, unusual formats and cases that should be refused or escalated. Otherwise the team can quietly tune the system to pass its own demonstration. Report failures by severity as well as the overall acceptance rate: one harmful answer can matter more than twenty good drafts.

Frequently asked questions

What is an AI proof of concept? An AI proof of concept is a short, time-boxed experiment that tests whether an AI tool can do one specific task well enough on your real data to justify a wider rollout. Its output should be a clear decision to proceed, change approach or stop.

What is the difference between an AI POC and an AI pilot programme? A POC tests whether the technology can do the job at all, usually with a small team and a sample of data. A pilot puts it in front of real users in their real workflow to test adoption, integration and business value before full rollout.

How long should an AI proof of concept take? For most small and mid-sized businesses testing an existing tool or API, a few weeks is enough to reach a decision. If a POC runs past two months, the scope is usually too broad or the data isn't ready.

What percentage of AI proofs of concept make it to production? It depends on the study. IDC research found only four in every 33 POCs reached production, and S&P Global respondents said 46% of projects are scrapped between POC and broad adoption on average.

Do I need a DPIA for an AI proof of concept in the UK? Possibly. The ICO lists AI as an innovative technology that can trigger a data protection impact assessment, especially alongside other risk factors such as profiling or sensitive data, and a test using real personal data is still processing.

Sources

If you're about to commission an AI proof of concept and want the decision rules set before the money goes out, let's talk.

Choose from all 20 guides ↗

Keith De Alwis
WRITTEN BY

Keith De Alwis

Independent thinking. Connected expertise.

Meet the mind behind the notes ↗