Data collection is growing at an exponential rate, but many businesses are still stuck with legacy systems ill-equipped for increased volume and velocity.
Data collection is growing at an exponential rate, with the total amount of data captured and stored doubling every 1.2 years. On top of this, the data collected by businesses is coming from more sources, and has more variety than ever before.
Almost all organizations expect data collection to increase in the next one to five years, according to a survey conducted by data analytics provider Ocient in collaboration with Propeller Insights.
SEE ALSO: Database Security Market a Growing Interest
At the same time, more than half of respondents to the survey are actively looking to switch data warehouse providers. One of the key reasons for this switch is that current systems aren’t comprehensive enough to meet the needs of the business.
Another key reason is modernization, with 40 percent of businesses wanting to shift from legacy systems to a modern solution. Poor flexibility was cited as another reason for the switch, with modern systems offering more interoperability and a variety of cost structures.
“With so many legacy systems on their way to obsolescence, decision makers are putting a lot of thought into what comes next,” said Chris Gladwin, co-founder and CEO at Ocient. “Speed, agility, and integration are top of mind, but replacing legacy systems – some of which have been in place for decades – can be an overwhelming proposition. When the positives of a new data warehouse outweigh the negatives associated with sunsetting familiar technology, it’s time to make a move. Our respondents showed that in addition to speed and performance upgrades, having a flexible and agile system and improving data access and integration across the enterprise are key.”
The boom in data collection and analytics has also forced businesses to be more aware and invested in data security, especially with new laws in place such as GDPR and CCPA. In Ocient’s survey, the main challenge business leaders are facing is maintaining security and compliance, as data volume and velocity grows.
Scaling data management solutions in a cost effective way is another challenge experienced by business leaders. The transition from an all-in-one solution to hundreds of solutions, each customized to fit the organization’s needs, can lead to sprawl if not managed correctly. Streamlining these systems to ensure little to no overlap was another challenge mentioned in the report.
Staffing continues to be an issue across all data and engineering roles. In the Ocient report, 65 percent of respondents said they were concerned or very concerned about their ability to hire experienced specialist talent for data strategy. Three of the most in demand roles are: chief data officer, data analyst, and data scientist.
