How do you achieve real-time analytics while avoiding high costs for racks of storage arrays?
Companies whose success rests on personalized recommendations – like Amazon, Netflix, and Apple Music – require speed and storage. Relevant recommendations must be delivered instantly if they are to have value to customers, and vast amounts of data about customer preferences must be stored. However, storing such data and managing analytic workloads can be costly, and can suffer from structural problems that delay processing of the information. It’s a catch-22: storing performance-hungry data is expensive, but necessary for delivering results.
Another problem is that legacy analytics and data storage solutions are not built to handle high-velocity, high-volume data. The explosion of Big Data disrupted traditional storage solutions; as massive amounts of data flooded into the enterprise, businesses realized it wasn’t feasible to simply add racks and racks of spinning disk. New storage mediums were needed. Further, traditional systems were best suited for structured data with rigid schemas. The data that today’s businesses rely on encompass vast amounts of unstructured data.
Solving the analytics-architecture challenge is essential for success, especially for businesses built on mining data for real-time recommendations. Look at a service like Apple Music: it attracted 15 million users within just six months after its June 2015 launch, yet even with that insane pace of growth, manages to come up with intelligent and personalized recommendations instantly. The service accomplished this feat even though it’s storing a growing warehouse of data relating to users’ listening habits, saved preferences, and most-liked artists.
So, how do you achieve the real-time analytics scenario while avoiding accelerating costs for racks of storage arrays? The answer lies in choosing a recommendation system architecture with storage to match data tasks.
Understand your workloads and their requirements. Not all of the data feeding into a customer recommendation engine needs to be on tap at all times. Some data applications and actions demand high performance, while others need only capacity. The same goes for real-time results: sometimes it’s a must-have, and sometimes it isn’t. Build awareness of what is required to make your business’s applications effective, as well as meaningful to your end user – which will guide how you manage storage.
Deploy a flexible storage architecture. You’re not locked into either high-performance (high cost) storage, or high-capacity (low cost) storage – it’s not an all-or-nothing choice. Instead, you can align workloads with the optimal storage medium for the task. For example, data that feeds into performance-hungry applications should be stored on flash arrays, while data that does not need to be readily or rapidly accessed might be suitable for disks, which can offer some cost savings.
Build your infrastructure with the future in mind. Companies like Apple and Netflix that have done a stellar job of providing recommendations at scale didn’t stumble upon the best storage architecture by mistake. They planned not only for growth, but also for the demands of real-time analytics versus historical data. Too many organizations make the mistake of adopting storage solutions that are good enough for early stages, but can’t easily be expanded if needed.
Personalized recommendations, driven by high-performance, can cement bonds with customers who crave suggestions for content and products they’ll love. But such insights don’t need to take the lion’s share of the IT budget. Instead, enterprises can deploy storage solutions that deliver real ROI in terms of IT costs savings and greater value for the customers.
