Rather than focusing on data science metrics, predictive analytics efforts must be tied to business outcomes.
Predictive analytics serve a range of business needs, from real-time fraud detection to foreseeing customer behavior. However, their business value often is misunderstood, lost in a tangle of irrelevant technical metrics, such as precision, recall, accuracy, and lift.
That’s the word from Eric Siegel, PhD, former Columbia University professor and author of The AI Playbook, who recently explored the issues and opportunities with predictive analytics in a recent webcast and related article in MIT Sloan Management Review.
Metrics such as precision are used by data scientists to serve as a “predictive multiplier. In other words, how many times better than guessing does the model predict?” he explains. While we are seeing predictive analytics boosted exponentially by artificial intelligence and machine learning, most predictive analytics projects “report on the wrong metrics — and this often kills the project entirely,” Siegel says.
Instead, these efforts need to be tied more closely with business outcomes, and specifically, “profitability. ROI, savings, numbers of customers acquired or saved,” he urges.
At the start of any project, data scientists and business users need to understand “what’s predicted, and what’s done about it,” Siegel illustrates. “That defines the use case. Should I drill for oil here? Is this credit applicant likely to be a bad debtor? What do we do about it?”
See also: Predictive Analytics, ML, AI – Do I Have to Choose?
Predictive analytics is more than numbers
As part of this, predictive AI efforts need supportive corporate culture. “It’s not just number-crunching,” he says. “Right now, the world is focused on just the number crunching, the rocket science, instead of its use. We need business steps to establish what you’re predicting and why. The data’s not going to prepare itself, and it’s not the data scientist’s job.”
The framework Siegel proposes to link predictive AI efforts to the business consists of six steps, which he calls bizML:
- Establish the deployment goal
- Establish the prediction goal
- Establish the metrics
- Prepare the data
- Train the model from the data
- Deploy the model
Predictive analytics based on machine learning and analytics delivered through generative AI are different AI approaches, but are based on the same requirements, Siegel emphasizes. “They’re the same thing, but used in a different way,” he explains. “You can also use generative AI as a predictive model, it may perform even better since it uses language-heavy tasks,” he explains. “For both, you need planning, concrete use cases, and ability to demonstrate business value.”
