Why AI Pilots Need More than a Great Model
CIOREVIEW >> Agile >> NEWS

Gourmet Specialty Foods

Michael DeMaio, IT Director

Why AI Pilots Need More than a Great Model

Michael DeMaio, IT Director
Michael DeMaio, IT Director, Gourmet Specialty Foods

Michael DeMaio

Decision Data Advisor

Enterprises are running plenty of AI experiments, but few deliver material business results. ISG’s 2025 study of 1,200 prioritized use cases found 31 percent had reached production, and only about one in four met growth expectations. The standard explanation is that the data is not ready. That is often correct but no longer sufficient. Pilots also fail because no one has been assigned the ownership, authority and time to make the data usable. 

The Paradox is Not Only Technical 

AI systems are only as effective as the information they consume. Fragmented sources, inconsistent definitions and duplicate records are amplified by AI, not solved by it. Problems that are tolerable in manual processes become dangerous when decisions are automated at scale.

Every CIO knows this. It persists because the fix has no owner. Repairing how ‘active customer’ is defined spans three departments and sits on nobody’s quarterly objectives, so every new pilot pays the tax again.

The gap is measurable. In a 2024 Precisely– Drexel University study of 565 data and analytics professionals, only 12 percent said their data had the quality and accessibility needed for effective AI, 67 percent did not fully trust the data behind their decisions and 62 percent named weak governance as a primary obstacle.  

What Readiness Actually Means 

Ask three departments what counts as an active customer, and you may get three defensible answers. Records can be clean in every system and still lack shared meaning. Quality attracts the budget because it is easier to tool, and meaning goes unresolved. 

The honest metric is not a quality score. It is the number of spreadsheets analysts keep on the side. Each one signals a gap in trust, access or definitions. This contains decisions that the official system, and therefore the AI, cannot see.

The Cost of Waiting

The cost is not the stalled pilot. It is a number that moves, and nobody notices. Consider a common data-integration failure. Active SKUs carry conflicting unit-of-measure definitions between ERP and warehouse systems.

 Rather than measuring quality only where data is stored, organizations should instrument it where the data is actually consumed. 

A case pack is one unit in one system, twelve in the other. No validation alert fires. The forecast runs on inconsistent inputs, every downstream calculation shifts, and the model takes the blame for a data-contract failure it could not detect. 

Four Pillars, In Order

The order matters more than the list. Establish accountable owners first, then align architecture, quality and adoption around the decisions those owners support. Architecture stalls without governance to define what is being unified. Quality programs measure fields nobody agreed on. Governance first is unpopular because it produces no demo.

Governance is not a policy document. It is a decision system. For each critical data domain, name one business owner who is accountable, define who resolves cross-system conflicts and set a resolution target in days. The owner needs authority to set definitions, assign remediation, and escalate missed deadlines. A committee that meets monthly is a queue.

Once ownership is established, organizations can focus on creating a unified data architecture without making platform consolidation the first milestone. It can take eighteen months to deliver usable value. Instead, integrate the ten or twelve entities that shape decisions—customer, product, supplier, order and location, and publish governed data contracts that other systems can consume.

That foundation makes continuous data quality possible. Rather than measuring quality only where data is stored, organizations should instrument it where the data is actually consumed. Give every production model a freshness and completeness threshold and let it refuse to score when the threshold breaks. A model that fails loudly costs a morning; one that degrades quietly costs a quarter.

Ultimately, these practices have to become part of the organization’s culture. A data-driven culture is not created through a communications campaign but through the incentives that shape which evidence people use and which risks they are willing to take. Trust grows when employees can act on governed data without being punished for a reasonable decision that later proves wrong. 

Buying the Runway

The CIO’s challenge here is political: funding two quarters of essential work that may produce little visible progress.

Rather than making an enterprise-wide platform the starting point, tie the work to a single business decision that already matters, such as a forecast or pricing call. Fix the data entities that the decision depends on, assign clear owners and deliver a measurable improvement. Use that result to justify the next increment.

Then ask five questions: Do we trust our critical business data? Is ownership clearly assigned to a specific person for each domain? Can employees access the data they need without asking another person? Do governance decisions get resolved in days or linger in meetings? Can we explain where any number came from?

Few CIOs, who can build disciplined data foundations rather than simply buying better models, can answer all five.

The next phase of AI will not be decided by access to models; those tools are increasingly available to everyone. It will be decided by whether organizations can supply trusted, governed, decision-ready data faster than their competitors. Without that foundation, organizations will continue buying technology and running pilots without turning them into durable business value.        ​ 

The articles from these contributors are based on their personal expertise and viewpoints, and do not necessarily reflect the opinions of their employers or affiliated organizations.