What AI Can Teach Us About Designing Better KPIs

Extractive Summary of the Article | Listen

In 2016, Wells Fargo found itself embroiled in scandal when headlines revealed that its employees, under pressure to meet aggressive sales targets, had opened millions of unauthorized customer accounts. The root cause wasn’t just unethical behavior but a flawed approach to performance measurement. When Wells Fargo’s leadership incentivized employees to sell eight financial products per customer, they inadvertently encouraged gaming behaviors that harmed customers, employees, and ultimately the bank itself. The metric had become more important than its underlying business purpose.

Although Goodhart’s law in action only sometimes rises to the level of scandal, companies everywhere fall prey to it: “When a measure becomes a target, it ceases to be a good measure.” Despite decades of warnings against metric fixation, leaders continue to build incentives around narrow indicators, which results in gaming and ethical lapses and harms business performance.

Traditional solutions to overcoming Goodhart’s law, like balanced scorecards and KPIs, often fail because they remain vulnerable to narrow optimization and gaming behaviors in the absence of careful oversight. The persistence of metric fixation signals a need for a more sophisticated approach to address the problem.

A New Lens: Insights From AI Training.  Leaders increasingly view organizations as systems to be optimized for specific outcomes, much as machine learning researchers optimize algorithms. Since both contexts involve optimizing proxy measures that can diverge from the true goals, solutions from AI research could help solve persistent organizational measurement problems. Of course, organizations consist of people with the agency and complex motivations that algorithms lack, so these techniques provide frameworks that require human adaptation, not mechanistic solutions.

AI researchers have long studied a phenomenon called overfitting, where machine learning models perform well on training data but fail when faced with real-world scenarios because they can’t generalize to predict outcomes with the new data.

The Metric Intelligence Framework in Action.  Machine learning has evolved four strategies to combat overfitting, each with direct implications for organizational design.  These are:  A)  Early stopping: Prevent overoptimization through timely reassessment.  B) Noise injection: Build robustness through controlled randomness.  C)  Capacity alignment: Match metric complexity to organizational capabilities.  And D) Regularization: Create balance through simplicity incentives.

The combination of artificial intelligence research and organizational design offers practical solutions to metric fixation. By applying techniques that keep AI models aligned with real-world goals, managers can improve how organizational performance is measured.

These approaches must work in concert rather than isolation. Organizations adopting metric intelligence treat measurement as an ongoing practice requiring constant refinement, not a “set it and forget it” solution. Successful implementations involve those who will be measured in designing the measurement system, resulting in both better metrics and greater buy-in for the new practices.

show less