In some use cases, it is impossible for humans to replicate the performance of artificial intelligence. But businesses will need a lot of data for AI systems to be effective.
Maybe you’ve seen an artificial intelligence (AI) system like Watson at work on “Jeopardy!” or have heard of its successes in medical diagnoses or other fields. Maybe you’ve only heard about other similar systems working through incredibly complex and large sets of data to produce results that even non-experts can understand, through visualizations or natural language. Either way, AI systems are impressing many on their march toward becoming essential business processes.
How does artificial intelligence work? AI systems “seem so intensely magical, but they’re not. At the bottom of these systems is hardcore data analytics, and in order for them to work, the data needs to be there,” said Kristian Hammond, a faculty member of the International Institute for Analytics (IIA), during a recent webinar. Hammond, who is also chief scientist of Narrative Science, insists we live in a world absolutely brimming with data—power comes in knowing how to process it.
How NLP-based AI systems work
Watson, developed in IBM’s DeepQA project, is just one example of an AI system capable of churning through terabytes of information in search of an answer to a natural language question. If one asks it, “Who ruled Spain in 1829?” it will turn the language into a number of similar search queries, such as “was king of Spain” or “ruled Spain.”
Hammond says that with these queries, Watson searches through its various networks of sources to find patterns in historical documents or reputable websites, such as Wikipedia. From there, it’s capable of aggregating the evidence into a best guess: for example, 87 percent of the evidence points toward Ferdinand VII, and 17 percent for Maria Christina. At this point, it’s not a stretch to trust in Watson’s analysis, which would be impossible to replicate in human terms.
It’s not unlike a visual recognition engine, which examines millions of images, and all their individual pixels, to understand what patterns indicate a picture of cat versus a picture of anything else.
IBM reported that Watson used more than 100 different techniques for analyzing natural language with Watson, but “what is far more important than any particular technique we use is how we combine them in DeepQA such that overlapping approaches can bring their strengths to bear and contribute to improvements in accuracy, confidence, or speed,” researchers stated.
According to Hammond and the IIA, Watson and other similar AI systems are difficult to develop due to a chicken-and-egg scenario: “Data itself needs to be managed by intelligent machines, but intelligent machines are completely enabled—made possible—by the data around us,” Hammond says.
AI applications in business
How can a business use AI to find flaws in their processes? First, they need to understand what kinds of problems they could actually solve, and which would be meaningful for their business. That becomes the driver for what kind of data collection is necessary, Hammond says. It’s more than likely a business either won’t have the right kind of data, or won’t have enough.
That data comes in three primary forms, according to Hammond: highly structured data, which often comes in a relational database with a schema; high-volume, low-feature data, such as cat pictures; and natural language, which is Watson’s specialty.
There are dozens of potential efficiency gains for each type of data. For example, AI systems can assess business functions, such as detecting fraud, quantifying performance reviews, qualifying customers, and more. Other systems can perform predictions—customer churn, equipment failure, employee retention—and yet others advise on customer communication, employee feedback, and purchase recommendations.
How does natural language processing play a part? Hammond says data can be examined and then re-rendered into natural language to help make it more interpretable. Imagine, instead of a complex spreadsheet of year-over-year sales volumes – something that could exist in a relational database — one could simply read a block of AI-produced natural language: “While still slightly better than sales of the small light widget, sales of the overly large widget have been on the decline over the last year. During the same period, sales of the small light widget have been steadily improving.”
Machine learning and dark data
AI systems may also employ machine learning to process concepts from source data. Watson IoT, for example, uses machine learning to understand current conditions and expected trends from machine data; it also has visual recognition APIs. That opens use cases in predictive maintenance and asset performance/production optimization. Analysis of weather data, meanwhile, might be used for supply chain and logistics, and Twitter data analysis could yield applications in customer experience or brand management. (See: “What IBM can offer with Weather Channel and Twitter data.“)
Another approach, which reportedly goes beyond natural language processing, is to use unsupervised learning to find data relationships that are hidden, even in unstructured data. Loop.ai Labs, for instance, uses unsupervised learning techniques to plumb what it is called “dark data.” (See: Loop AI Labs explains cognitive computing).
In any case, with AI systems becoming easier to implement, more businesses should be able to take advantage of them soon—experts advise it’s better to start intelligently collecting data now, rather than later, to enable the right analyses to make meaningful change.
Related:
Machine learning case studies
How Watson IoT led to a gold medal in cycling
Special report: Frontiers in artificial intelligence for the Internet of Things
