Demanding Results: Why AI Requires Board-Level KPIs

AI isn’t magic or exceptional. Boards should treat it like any other strategic business initiative and demand clear metrics to guide decision-making.

Artificial Intelligence (AI) is an operational and strategic reality for boards of directors. But while board members’ interest in AI has grown rapidly, many organizations have yet to apply the same level of rigor to AI oversight that they do to other major investments. Too often, AI is treated as an experiment, not a disciplined initiative.

As a result, AI initiatives often stall. Three common culprits include:

1. Fuzzy objectives: “We want to use AI” is not a strategy. Projects need sharp business cases linked to real challenges, whether that’s improving forecasting accuracy or reducing wait times in clinical workflows.
2. Unowned outcomes: When responsibility for success is unclear or shared among multiple parties, accountability erodes. AI efforts need definitive owners, with performance tied to results.
3. Lack of progress measurement: Too many AI projects have no defined baseline, intermediate checkpoints, or timeline for expected value delivery. This would be unacceptable in other investment categories,  such as trading platform upgrades, regulatory compliance initiatives, M&A integrations, or new market expansions. Could you even the reaction of the board to an investment of these types without a clearly stated set of goals and ROI?

This must change.

Boards have a fiduciary responsibility to ensure that their organizations deploy capital wisely. AI investments should be no exception. As with cloud migration, digital transformation, or new market expansion, board members should demand clear, measurable KPIs — financial and non-financial — for every AI initiative. They must then monitor those indicators with the same scrutiny that they apply to other business endeavors.

Why KPIs Matter in AI

The transformative nature of AI can create a sense of exceptionalism. There is a sense that its benefits are intangible or inherently difficult to measure. That narrative is both inaccurate and — worse — risky. AI is not magic; it is a collection of technologies that can be applied to solve real problems. It must earn its place at the table.

Success metrics are what separate experimentation from execution. Without them, organizations risk investing in AI tools that offer little strategic value, lead to cost overruns, or introduce hidden risks.

Consider the strategic misalignment that occurs when an AI model trained to optimize marketing spend focuses on click-through rates without improving conversion or revenue — because success was never clearly defined in business terms. What happens if a hospital deploys a generative AI tool for documentation support but fails to define acceptable productivity gains? Spending two years, say, on customization and integration for such a project will see costs balloon with little measurable return will see costs will balloon with little measurable return. Or how about an AI chatbot that generates inaccurate medical advice due to insufficient guardrails and no KPI in place for clinical accuracy? The fallout could lead to public backlash and loss of patient confidence, to say nothing of the risk to patients.

So, what should be measured and tracked when it comes to AI initiatives?

Financial KPIs might include:

• Reduction in cost per task or transaction (e.g., call center automation reducing agent costs by 30%).
• Increased revenue per customer segment (e.g., personalization engines lifting average order value by 12%).
• Margin improvements from process automation (e.g., invoice processing time cut from 12 hours to 2).
• Shortened time-to-market (e.g., drug discovery pipelines accelerated through AI-driven compound screening).
  

Non-financial KPIs, like those below, are equally critical:

• Improved patient outcomes or customer satisfaction scores (e.g., better clinical decision support increasing diagnosis accuracy).
• Reduction in time spent on repetitive tasks (e.g., physicians spending 25% less time on charting).
• Enhanced data quality or model accuracy (e.g., prediction model improving forecasting error rate by 15%).
•  Explainability, fairness, and bias mitigation (e.g., implementing model audit trails and bias detection protocols).

Tracking these indicators allows boards to evaluate performance, guide resourcing decisions, and make informed go/no-go calls. 

Boards of Inquiry: What to Ask

To govern AI responsibly, boards can also adopt a practice of structured inquiry. Asking pointed questions is nothing new for seasoned board members; the problem with AI may be that they don’t know what areas of inquiry to pursue.

Some valuable questions to ask include:

• What specific business process is this AI project addressing?
• What does success look like, and how is it measured?
• Who owns this initiative, and how often are they reporting progress?
• What are the risks—technical, ethical, regulatory—and what mitigation strategies are in place?
• What is the timeline for impact, and how will it be validated?

Boards that embrace structured inquiry will improve both transparency and results. The aim is not to micromanage these technology initiatives. It is to ensure strategic alignment and enable effective board oversight.

A Call for Discipline

Leading organizations are already acting. Some have embedded AI governance into their existing committees. Others have created dedicated AI subcommittees. In forward-looking enterprises, CIOs, CTOs, and CDOs are partnering with independent directors to review AI portfolios against pre-agreed KPIs. In some cases, executive compensation is now linked to AI deployment success.

That is what it looks like to govern AI effectively —not through hype, but through discipline and accountability. For example, Mayo Clinic has integrated AI governance into its institutional review framework through a cross-functional enterprise AI translation advisory board. The 23-member board, comprised of subject matter experts from areas such as data science; qualitative research; user experience; IT; human factors,  regulatory compliance, ethics, and clinical care; assesses whether an AI technology is suitable for clinical use and lays out requirements to achieve successful translation from pilot to active use.

Turning Promise into Performance

AI holds tremendous potential, but it is just as vulnerable as any other investment to waste, misalignment, or failure. In fact, it may carry greater exposure due to opaque models, evolving regulation, and technical complexity. For example, UnitedHealthcare, via its subsidiary NaviHealth, deployed an AI-powered model called nH Predict to determine post-acute care coverage in Medicare Advantage plans. A notable case involving a 91-year-old patient prompted a class-action lawsuit after his continued nursing-home care recommendations were denied despite strong medical evidence.

Boards must ensure that AI investments meet the same expectations for transparency, alignment, and value delivery as any other strategic initiative. Doing so not only minimizes risks, it maximizes returns. The organizations that treat AI as a core business lever, not a novelty, will be the ones that turn its potential into real and lasting value.