5 min read
The AI Pilot Graveyard
Companies are not short on AI ideas. They are short on AI projects that survive contact with a budget review. Here is why pilots die and what the survivors do differently.
I have lost count of the demos I have watched in the last two years. A model reads a contract and pulls out the renewal date. A chatbot answers a support question that used to take a person ten minutes. The room is impressed. Someone says the word transformative. Then six months later I ask how the project is going and the answer is a shrug.
This is the part of the AI story nobody puts in the keynote. Most corporate AI never makes it out of the pilot. It dies quietly, somewhere between the demo that wowed the steering committee and the second quarter when finance asks what it actually saved. I want to talk about why that happens, because the reasons are boringly consistent and almost entirely avoidable.
The demo was never the hard part
A demo runs on three or four inputs that the person building it already knows the model handles. Production runs on the inputs nobody thought about. The contract with the clause written by a lawyer who learned English as a third language. The support ticket that is actually three questions stapled together. The invoice scanned upside down.
The work that separates a demo from a system is not better prompting. It is everything around the model. Where does the input come from. Who checks the output. What happens when the model is confident and wrong. None of that is visible in a slide, so none of it gets funded, and the project arrives in production missing the parts that would have kept it alive.
When we built the meeting capture automation for NSB, the model turning calls into tasks was maybe a fifth of the work. The rest was making sure the tasks landed in the right place, that a human could correct them in seconds, and that the whole thing was visible enough that people trusted it. That is the part that does not demo well and is the only part that matters.
Nobody measured the before
Here is a question that kills pilots faster than any technical failure. What did this cost before we automated it. If the team cannot answer that in numbers within a few days, the project is already in trouble.
A pilot with no baseline cannot prove it worked. It can only claim it worked, and a claim does not survive a budget review run by someone who was skeptical from the start. I have seen genuinely good automations get cut because the team built them, felt the relief, and never wrote down that intake used to take twenty two minutes a document. The relief was real. The evidence did not exist.
Measure the before. Tickets closed per week. Hours logged. Response time. Documents processed. Pick something you can count this week, count it, and write it down before you change anything. The number you capture on day one is the only thing that will defend the project on day ninety.
The pilot solved a problem nobody had at scale
A lot of pilots target the most interesting problem rather than the most common one. Interesting problems make for good demos and bad investments. The model that drafts a beautiful response to an unusual customer complaint is solving a case that happens twice a month. The unglamorous triage of the four hundred routine tickets a week is where the money is, and it never gets picked because it does not sparkle in a meeting.
If you want a pilot to live, aim it at volume. Find the task your team does so often that nobody questions it anymore, the one that is repetitive enough to have a documented process and frequent enough that shaving minutes off it adds up to real hours. That is the work that pays for itself fast enough to earn the next project.
The handoff was never designed
Every model is sometimes wrong. A pilot that pretends otherwise is building on sand. The systems that survive are the ones where being wrong is cheap, because a human catches it before it costs anything, and the correction takes seconds rather than a support escalation.
This means designing the handoff before you design the automation. What does the person see when the model is unsure. How do they fix it. Does the fix make the system better next time or just patch this one case. A pilot that automates the easy ninety percent and hands the messy ten percent to a human with a clean interface will outlive a pilot that tries to automate everything and hides its failures.
What the survivors share
The projects that make it to production are almost always smaller than the ones that died. They picked one repetitive task with a known cost and a cheap failure mode. They shipped something narrow in weeks, proved the number moved, and used that proof to fund the next step. They treated the model as one component in a system that was mostly plumbing and judgment, not as a magic box that would carry the whole thing.
None of this is exotic. It is the same discipline that has always separated software that ships from software that demos. The only thing new is the temptation, because the demos are so much more convincing now that it is easier than ever to fund the wrong thing.
If you are sitting on a pilot that impressed everyone and went nowhere, the problem is probably not the model. Start with how to pick a first project that actually ships, and if you want a second pair of eyes on what to build first, that is exactly what our AI consulting work is for.
Let's Connect
8939 South Sepulveda Boulevard Suite 102
Los Angeles CA 90045
United States