Companies will turn to faster databases, predictive analytics, and machine learning.
We’ve written plenty about the potential of big data throughout 2016. Now, at the year’s end, everyone is angling to make predictions about what’s to come, and businesses are scrambling to act quickly on rapidly-changing technologies. What are companies doing to get themselves ready for the next year of big data innovations?
SQL isn’t going anywhere
Many people think SQL-based databases are out of the picture, and that NoSQL solutions such as MongoDB and Redis are all the new rage. Even with platforms like Hadoop growing in popularity, the trendline for 2016 was that MySQL was on the outs. Not so fast, according to Lloyd Tabb, the founder and chairman of Looker, a company that creates business analytics tools. Tabb says that Google BigQuery will bring many back into SQL’s arms, considering that it is “essentially infinitely scalable,” and that Google has released a new version that’s fully ANSI compliant.
Spark and Presto, which are SQL engines for the Hadoop platform, continue to gain traction as they become leveraged on cloud platforms.
A need for speed
Speaking of cloud platforms—new innovations like Amazon Athena are making it easy to turn a giant S3 “data lake” into actionable analytics without investing in new tools or new infrastructure. Dan Kogan, the director of product marketing at Tableau, says companies are shifting toward “faster databases like Exasol, MemSQL and Hadoop-based stores like Kudu.”
Turning those ‘data lakes’ into something more
Without the right tools or infrastructure to do real analysis, many companies have simply been dumping raw data into one massive warehouse. It contains incredible amount of potential information, but “it doesn’t work if you don’t have a way to make it cohesive when you query it,” says Tabb.
In 2017, businesses will finally be confronted with the reality that they need to either get rid of that data or find some way to make use of it, whether that’s partnering with a technology company or creating something themselves. John Schroeder, executive chairman and founder of MapR, says that delivering individualized recommendations, real-time price checks, personalized content, and more requires “an agile platform that can provide both analytical and operational processing.” Schroeder says they need to stop asking questions and start developing solutions.
They biggest problem is modeling data in a productive way, but Tabb says that companies like Docker and BuzzFeed are already using their data lakes to better understand their customers. It’s possible, just not easy.
‘More’ being predictive analytics
Audelia Boker of Glassbox says that marketers are beginning to track the entire customer journey so that they can rewind it, replay it, and understand exactly what the customer wants. Those data lakes are going to get bigger, but they’re also going to get smarter, and once again, it’s marketing that gets to take the first stab at turning big data into better company decisions.
According to a new report from Autopilot, 48 percent of marketers plan to invest in predictive analytics in 2017.
Dan Kirsch, principal analyst at Hurwitz & Associates says that embedded analytics will help companies make more data-driven decisions without having to ask data analysts or feed the data lake through any number of standalone cloud-based services. At the core of this is the chief data officer—Kirch adds, “The role of the chief data officer is going to become an increasingly strategic [one] in companies.”
With a little bit of machine learning on top
Once those marketing profiles have been created, companies will be able to turn them into different experiences for different customers. Boker says that machine learning will be used to detect anomalies in customer behaviors, and then predict if they’re going to become major issues. In conjunction with that, Schroeder says that more companies will combine microservices and machine learning so that new services can take advantage of large amounts of historical data to better understand the new, streaming data.
The next step, according to Kirsch? “Cognitive solutions that provide breakthrough solutions to difficult problems in vertical markets such as healthcare, manufacturing, and finance.”
Either way, 2017 is going to be an exciting year for big data. By this time next year, it’s going to be smarter, more active, and, of course, much larger than it is now.
