Opinion mining has been criticized for its accuracy but the field is increasingly ready for prime time. Companies have succeeded in bringing Big Data sentiment analysis to the average user, capturing real-time shifts in public perception. Those shifts have been used to inform marketing, public relations, and investment decisions. Challenges include producing industry-specific analyses and capturing a wider range of emotions.
As the GOP presidential debate aired last week, I logged into LUX2016—a tool for real-time social analysis.
The demonstration was a partnership between ICG Solutions, which provided a streaming analytics platform it initially developed for intelligence uses, and Datawatch Corporation, which offered visual discovery tools.
The big idea with LUX is to bring Big Data analytics to the average user, which is a crucial need for many corporations where analytics must move beyond the esoteric domain of data scientists and produce actionable insight for business managers. With that in mind—and considering that I’m a novice user of real-time analytics—I was impressed with LUX. The system tracked overall sentiment for 16 Republican and three Democratic presidential candidates, as well as how candidates performed on a dozen or so issues from education to immigration. The platform aggregated real-time data and let users visualize that data in increments of 30 minutes, one hour, three hours, and six hours. (Datawatch has products that can visualize up to 25,000 data points per second, but the purpose with LUX was to let users see trends— hence the need for a visual interface that can aggregate data into something meaningful).
On LUX, I could move through the dashboard with ease and see trends, then drill into specific issues on an interactive treemap to see how candidates were faring on those issues. The system also let me find individual tweets and download that data.
Real-Time Sentiment Analysis
As I moved around the LUX platform, I noticed a large bump in overall positive sentiment for Carly Fiorina. That threw me because Fiorina wasn’t on anyone’s map—her polling position didn’t qualify her to speak at the main debate. I later learned that the data reflected her sound performance during the earlier “happy hour” debate.
In other words, the LUX system had successfully captured a real-time shift in positive perception for Fiorina. Not only that, but users could see where and how that shift was occurring and drill into the context for further analysis. It’s a shift that Fiorina can exploit, and it’s the kind of real-time insight that many businesses want to obtain.
“You can break down from the tweet detail perspective who would be the key influencers, and if you’re running a campaign you know this is where you can build up those grassroots efforts,” said Dan Potter, chief marketing officer for Datawatch. “It is pretty easy to sort through by candidate who is being retweeted the most.” (During the debate, Democratic candidate Bernie Sanders—who live-tweeted the GOP debate—got the most number of retweets.)
Political analysts have long predicted that sentiment analysis will soon be a part of every campaign. The 2012 U.S. presidential contest was considered the first Big Data election. The Obama campaign used a mass of data to identify local, persuadable voters, address their concerns, and mobilize them to vote. The first commercial licensing of sentiment analysis software, however, occurred only two years ago in a Singapore election, according to Potter.
Potter stressed the real-time value of Big Data. “Real time is not as important when you’re prepping a candidate. But it would be very important if you have the opportunity to give the candidate feedback during the commercial breaks,” he said.
“The same is true of a business,” he said. “If you have the ability to respond to operational data to see important anomalies, then real-time visualization becomes critical. If you are manufacturing a steam turbine for energy production”—a machine that costs over $100 million—”if that starts to show stress, you want to bring it down and do preventative maintenance.”
Hurdles and Applications for Opinion Mining
Sentiment analysis, also known as opinion mining, has some criticisms. For starters, it tends to capture only a subset of the population that vents on social media. If the data source is Twitter, it also tends to chiefly capture people who are shouting for attention. (How do you reach dissatisfied introverts?)
Another criticism is that a positive or negative tweet often doesn’t carry enough information for a decision. Sentiment analysis has challenges with pinpointing a wider range of emotions, as well as capturing emotions in pictures, video, and emoji. There’s also the hurdle of building industry-specific and foreign-language dictionaries (some companies in this area include Lexalytics and Luminoso). Companies, however, are working to improve in all these areas.
Businesses such as airlines, hotels, and retailers are already using sentiment analysis, said Doug Henschen, a principal analyst at Constellation Research and former executive editor of Information Week. It can be time-consuming and expensive to perform consumer research, but when you have “thousands or tens of thousands of comments—you have readily available data, and it’s at scale,” he said. In some cases, the sentiment analysis can complement a survey, he added—for example to see if hotel users say the same thing about their hotel experience on a survey as they do on social media.
In the business field, I’ve come across several examples where sentiment analysis is already being used or investigated (examples are not exhaustive):
- Barclays Bank introduced Pingit, a mobile payment app, and used sentiment analysis to improve product deficiencies, such as the fact it couldn’t be used by teenagers, and to hone its marketing based on positive reviews.
- TickerTags connects trending social content to stock investments. (The company cites, as one example, the Kim Kardashian Hollywood game, which became a Twitter sensation and doubled the stock of publisher GLU Mobile in a matter of weeks.) Other companies in this field include HedgeChatter and SocialSandbox.
- The city of Toulouse in France uses social data analytics to analyze citizens’ needs posted on public social media.
- Manufacturers could mine opinion to spot defective products–for example car defects–before liability balloons.
- Government agencies could predict or pinpoint civil unrest, political violence, natural disasters, and virus outbreaks.
The main challenge for the field is in finding out why consumers are feeling a certain way, said Henschen. “You want a more subtle understanding.”
Want more? Check out our most-read content:
Goodbye Don Draper, Hello Big Data: An EMA Report on Modern Analytics
Research from Gartner: Real-Time Analytics with the Internet of Things
Becoming an ‘Always On’ Smart Business
Webinar Replay: Better Business Insight with Real-Time Streaming Analysis
Liked this article? Share it with your colleagues using the links below!
