Are Sports Highlights a Win for IBM’s Watson AI?

Unlike some of the other industries that Watson has tried to master, sports offers a bit more predictability and data sources. IBM’s Cognitive Highlights was on display at the U.S. Open.

Unlike some of the other industries that Watson has tried to master, sports offers a bit more predictability and data sources. IBM’s Cognitive Highlights was on display at the U.S. Open.

Between detecting cancer, improving chatbots, trying to forecast weather, preparing your taxes, and helping surgeons make better decisions, IBM’s Watson platform is a busy little cognitive computer.

Now, IBM has IBM Watson Media, a suite of applications that analyzed images, video, sentiment, and tone, at the U.S. Open tennis tournament, which ended yesterday when Rafael Nadal defeated Kevin Anderson in the men’s final. (Sloane Stephens defeated Madison Keys in the women’s final.)

For this tournament, the United States Tennis Association (USTA) took advantage of a feature called Cognitive Highlights, which automatically creates highlight videos from tennis matches using the most moments that got the biggest reaction from the crowd. The platform takes footage from the entire match, and uses statistical data in combination with crowd reaction sounds, plus facial recognition of the players themselves, to rank each shot in terms of its “overall excitement.” Cognitive Highlights then pulls all the clips together to create a highlight video, which is automatically posted on Facebook and published via US Open mobile apps.

[ Related: How AI and Cognitive Science Can Beat Addiction Treatment Fraud ]

Noah Syken, IBM vice president of Sports & Entertainment Partnerships, said that with 18 concurrent matches, it’s difficult for fans to follow along with everything that’s happening, not to mention analysts and teams responsible for creating these video packages.

“To meet that challenge, Watson is now watching the matches alongside the USTA to help bring fans closer to the best moments across the courts shortly after they happen. We’re seeing this technology come to life through tennis, but the entire IBM Watson Media portfolio has the potential to impact many industries,” Syken said in a statement.

The move into media

Unlike some of the other industries that Watson has tried to master, media and sports offers a bit more predictability, plus plenty of data sources to work from. As part of the Watson Media rollout, IBM is releasing applications that can do intelligent closed captioning that improves by learning industry lingo — relevant not only for the many idiosyncrasies of sports commentary, but also many technical industries. Watson can now also identify violent content, or that with adult language, to provide automatic labeling.

For those courtside at the U.S. Open, Watson is using the Conversation API to power a voice-driven assistant that can help people find places to eat, where to shop, and rest their head for the night. IBM is working closely with the U.S. Open to provide the aforementioned highlights and other video content on plenty of platforms, and then uses a feature called Recommendation Uplift to give better recommendations to viewers based on a more thorough understanding of the selection of content. There’s even Watson features that analyze player and ball position data on particular plays to give better analysis of what’s happening on the court.

Will highlights change Watson’s outlook?

Getting into sports and other media looks like a pivot from Watson’s previous “world changing” applications, but it really seems more like a plan to expand the applications in hopes of a big success story. Because, in the real world, the big picture stuff hasn’t always gone well.

Back in 2013, IBM claimed that Watson was going to “eradicate cancer” by partnering with MD Anderson. Earlier this year, the cancer center at the University of Texas had put the project on hold. Since then, the center has been getting bids for a replacement after losing $62 million on Watson.

[ Related: Could Cognitive IoT Clean Up Our Lakes and Streams? ]

In an interview with Scientific American, Dr. Lynda Chin, once the project lead for MD Anderson, said that Watson is powerful, but difficult to implement in healthcare. For one, it’s difficult for Watson to understand the “idiosyncrasies of medical records: the acronyms, human errors, shorthand phrases, and different styles of writing.” On top of that, the team struggled with inputting enough data into the system—even information from 10,000 patients wasn’t enough to truly leverage Watson’s aggregation abilities.

Gizmodo also published a scathing piece back in August about how marketing might have blown Waton’s real-world capabilities out of proportion, and that many are preferring other, lower-cost AI solutions instead. In response to that and other criticisms, IBM has been quiet, saying only that they believe the marketing presents accurate depictions of how Watson interfaces with users.

In many ways, it feels like Watson could use a win — not all that different from all those seeking a Grand Slam victory. Because IBM seems to want Watson to apply its deep learning algorithms to just about every problem facing humanity, it’s running into the classic “jack of all trades, master of none” problem.

[ Related: Will IBM’s ‘Watson For Everyone’ Approach Work? ]

Meanwhile, the rest of the technology industry isn’t waiting around to see what Watson does. Google has its DeepMind group, Microsoft has Project Oxford, and Baidu is developing Minwa. Smaller companies, such as Flatiron Health, Digital Reasoning, and Narrative Science, are targeting many of the niches under Watson’s umbrella as well. The open source community is vibrant, too, with certain toolkits blossoming in popularity — TensorFlow, Theano, and OpenAI are just a small fraction of the big players.

Given all this movement, Watson not only needs a win, but needs it fast. Straight sets might be the only option.