Machine Learning: Prediction, Decision and Action

Small ups and downs determine your fate and mine, every day. A precise spam filter has a meaningful impact on almost every working hour. We depend heavily on effective internet search for work, health, home improvement and most everything else. We put our faith in personalized music and movie recommendations from Spotify and Netflix. After all these years, my mailbox wonders why companies don’t know me well enough to send less junk mail (and sacrifice fewer trees needlessly). 

These predicaments matter. They can make or break your day, year or life. But what do they all have in common?

These challenges — and many others like them — are best addressed with prediction. Will the patient’s outcome from surgery be positive? Will the credit applicant turn out to be a fraudster? Will the homeowner face a bad mortgage? Will the airfare go down? Will the customer respond if mailed a brochure? With prediction, it is possible to fortify health care, combat risk, conquer spam, toughen crime fighting, boost sales and cut costs.

Gold and Gems

How shall we attack the intricate problem of prediction?

The solution is machine learning — computers automatically developing new knowledge and capabilities by furiously feeding on modern society’s greatest and most potent unnatural resource: data.

As the data piles up, we have ourselves a genuine gold rush. But data isn’t the gold. I repeat, data in its raw form is boring crud. The gold is what’s discovered therein.

The process of machines learning from data unleashes the power of this exploding resource. It uncovers what drives people and the actions they take — what makes us tick and how the world works. With the new knowledge gained, prediction is possible.

This learning process discovers insightful gems such as:

• Early retirement decreases your life expectancy.
• Online daters more consistently rated as attractive receive less interest.
• Rihanna fans are mostly political Democrats.
• Vegetarians miss fewer flights.
• Local crime increases after public sporting events.

When machine learning is applied to improve business operations, it’s often also called predictive analytics (PA).

Even With Limited Accuracy, Predictions Are Valuable

Let me be perfectly clear. It’s fuzzy. Accurate prediction is generally not possible. The weather is predicted with only about 50 percent accuracy, and it doesn’t get easier predicting the behavior of humans, be they patients, customers or criminals.

Good news! PA’s utility withstands quite poor accuracy. We can’t confidently predict the response of any one particular customer. Rather, the value is derived from identifying a group of people who — in aggregate — will tend to behave in a certain way.

Predicting better than pure guesswork, even if not accurately, delivers real value. A hazy view of what’s to come outperforms complete darkness by a landslide.

Machine learning’s task is to find patterns that appear not only in the data at hand, but in general, so that what is learned will hold true in new situations never yet encountered. At the core, this ability to generalize is the magic bullet of PA. There is a true art in the design of these computer methods. The machine actually learns more about your next likely action by studying others than by studying you.

Risk and PA: The Upsides

Organizational risk management, traditionally the act of defending against singular, macrolevel incidents like the crash of an aircraft or an economy, broadens with PA to fight a myriad of microlevel risks.

A little glimpse into the future gives you power because it gives you options. In some cases, for example, the obvious decision is to act in order to avert what may not be inevitable, be it crime, loss or sickness.

At the same time, each time you act on a prediction, you take on risk. For example, a predictive system might boldly declare, “Even though ad A is so strong overall, for this particular user it is worth the risk of going with ad B.”

But navigating this risk carries us to a new frontier of customization.

Malcolm Gladwell said, “What is the great revolution of science in the last 10, 15 years? It is the movement from the search of universals to the understanding of variability. Now in medical science we don’t want to know … just how cancer works; we want to know how your cancer is different from my cancer.”

Quantifying the Clues

Like Sherlock Holmes drawing conclusions by sizing up a suspect, prediction comes of astute observation: What’s known about each individual provides a set of clues about what he or she may do next.

A predictive model is the means by which the attributes of an individual are factored together for prediction. There are many ways to do this.

One is to weigh each characteristic and then add them up.  Each element counts toward or against the final score for that individual. This is called a linear model, generally considered quite simple and limited, although usually much better than nothing.

Other models are composed of rules, like this real example of a student grant and scholarship search website that wanted more clicks on sponsors’ ads.

• IF the individual is still in high school
• AND expects to graduate college within three years
• AND indicates certain military interest
• AND has not been shown this ad yet
• THEN the probability of clicking on the ad for the Art Institute is 13.5 percent.

This rule is a valuable find, since the overall probability of responding to the Art Institutes ad is only 2.7 percent, so we’ve identified a pocket of avid clickers, relatively speaking.

We can go more “supermath” on the prediction problem, employing complex formulas that predict more effectively — but those are often almost impossible to understand by human eyes.

Regardless of approach, all predictive models share the same objective: They consider the various factors of an individual in order to derive a single predictive score for that individual. This score is then used to drive an organizational decision.

Analytics to Action

Unlike a report sitting dormant on the desk, PA leaps out of the lab and takes action. In what it foretells, it mandates movement.

Doctors take a second look at patients predicted to be readmitted, and service agents contact customers predicted to cancel. Predictive scores issue imperatives to mail, call, offer a discount, recommend a product, show an ad, expend sales resources, audit, investigate, inspect for flaws, approve a loan or buy a stock.

As Thomas Davenport and Jeanne Harris put it in Competing on Analytics: The New Science of Winning, “At a time when companies in many industries offer similar products and use comparable technology, high-performance business processes are among the last remaining points of differentiation.”

Enter PA. 

This article was adapted from Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie or Die, with permission from the publisher.

Eric Siegel, Ph.D., served as the inaugural Bodily Bicentennial Professor of Analytics at Darden. A leading consultant and former Columbia University professor, he is also founder of the Machine Learning Week conference series and author of the bestselling Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie or Die and The AI Playbook: Mastering the Rare Art of Machine Learning Deployment.