Potential obstacles to implementing streaming analytics include outdated architectures, limited staff availability, and a lack of skills.
As organizations adopt advanced data analytics and artificial intelligence, they are going to increasingly need streaming analytics for real-time insights on events. However, streaming analytics require an all-in shift in mindset that companies aren’t ready to embrace.
That’s the word from Kevin Petrie, who, in a recent Eckerson Group analysis, points to the challenges enterprises face as they move to real-time data capabilities, from missing executive support to skills shortages.
See also: Continuous Intelligence to Benefit from Streaming Analytics Boom
Petrie outlines the challenges being encountered, including strategic confusion, or the pursuit of projects without clear business objectives. “They also might assume faster is always better, and mistakenly replace historical use cases with streaming analytics use cases,” says Petrie.
Skills and time also weigh on streaming analytics projects, he continues. “Data leaders often lack the development staff and skills to take on complex customization. Both business and data leaders can underestimate the need for operational domain knowledge with IoT-related initiatives and fail to collaborate sufficiently with operations experts.”
Inflexible architectures also are an issue, as enterprises “often struggle to copy data from legacy sources like mainframe systems or SAP applications,” says Petrie. “Data teams also face the risk of lock-in. If they customize their stream processor software on cloud Platform-as-a-Service offerings, it might be harder to migrate those workloads to another cloud or back on-premises in the future.”
Petrie makes the following recommendations to put streaming analytics in place:
Evangelize streaming analytics as part of data modernization. Educate executives “about the data modernization benefits of your streaming analytics initiative. This will help you win sponsors, secure budget, and pool resources.”
Rethink the team’s mission. Data teams to “re-design legacy architectures to handle rising data volume, variety, and velocity,” Petrie urges. “These teams need to address requirements for each piece of the puzzle: data sources, data collectors, event brokers, stream processors, data stores, and visualization tools.”
Design your architecture end to end. “Streaming analytics depends on many components: event sources, change data capture and/or APIs, event brokers, stream processors, data stores, and visualization tools. Start with your business requirements, then scope and address the associated technical requirements for each component as they apply to your use case.”
Start with a simple, tactical initiative. “Default to standard, commercial,
suite-oriented offerings where possible, unless your team already has extensive in-house developer staff ready to customize,” says Petrie. “A limited first initiative, if successful, can pave the way for strategic transformation down the road, using a higher-level of customization and potentially open-source tools.”
