How Hybrid Cloud Can Be Used For Analytics and Machine Learning

A hybrid cloud approach lets businesses essentially pick and choose the various aspects of their analytics and ML workloads that they want to keep on-premises and which they want to run in the cloud.

Analytics and machine learning (ML) are some of the fastest-growing workloads today. Businesses want to use insights derived from both for decision support, customer engagement, cost reduction, risk management, and more. 

Unfortunately, businesses looking to unlock the power of analytics and ML face significant challenges. Many lack staff with expertise or training in analytics and ML. Most do not have the infrastructure for analytics and ML. Specifically, they do not have the capacity to store data needed to build and train models, nor do they have the compute power to run the models and workloads.

Making matters more complicated is that there are distinct phases in rolling out analytics and ML projects. They include building/training, tuning/testing, and deploying/running in production. Each stage has distinct compute and storage requirements. For example, the building/training stage typically uses large datasets, the tuning/testing stage needs many high-performance computing cores or instances, and the deploying/running stage needs reliable compute power that delivers results or insights in a timely manner upon which businesses can take action.

And those needs may vary greatly over time. For instance, a financial services firm might want to refine an analytics or ML-based model or application by incorporating an additional economic dataset and retrain it using that new data. Or a business might make a production version of an analytics and ML model, application, or tool available to a small set of users and, once proven of value, opt to expand its use to the entire company.

Hybrid cloud steps in

The three big public cloud providers, Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP), want developers and data scientists to develop, test, and deploy machine learning models on their clouds. Each offers multiple data storage options, including serverless databases, data warehouses, data lakes, and NoSQL datastores. They offer popular machine learning frameworks, including TensorFlow and PyTorch. All three offer a growing number of capabilities to support the full analytics and ML life cycle.

Even with these options, most businesses want to keep some aspects of their analytics and ML projects on-premises. For example, a business might rely on a massive customer dataset that resides on a legacy system. It would be impractical (and most likely highly expensive) to migrate such a dataset to a public cloud. The smart move would be to keep the data where it is.

That’s where hybrid cloud plays a role. Hybrid cloud can be used to run different cloud infrastructures in an interoperable fashion to optimize a single workload. How does that help with analytics and ML? A hybrid cloud approach lets businesses essentially pick and choose the various aspects of their analytics and ML stages that they want to keep on-premises and which they want to use in the cloud. For instance, a business might rely on cloud storage to accommodate the large volumes of data needed to train models. Or they might opt to use analytics and ML-specific instances to accelerate certain workloads.

The cloud services in each area (data storage, compute power, analytics and ML acceleration via special processors, and more) provide the capabilities to scale up or down to meet the elastic demands of analytics and ML workloads.

Well-suited for analytics and ML

Hybrid cloud offers many benefits for analytics and ML, and it is well-suited to meet the varying demands over time. However, hybrid cloud environments can be fairly complex. And that can lead to problems.

There are several issues at hand. Many tools that help manage and troubleshoot infrastructure or workload performance are designed for one environment. A business might have a grasp of what’s going on on-premises. But that does not extend to the public cloud. Additionally, while cloud providers may offer their own tools and management systems, most will not provide any help for on-premises workloads or other cloud services.

The reason this is a problem is that when building, testing, and deploying analytics and ML on a hybrid platform, these disparate tools make it very hard to troubleshoot problems, identify root cause problems, spot issues in the making, or notice anomalies that are pre-cursors to security and performance problems.

What’s needed is a solution that provides visibility across all clouds. An ideal solution would help businesses build, manage, govern, and optimize a complex hybrid cloud environment. It should also combine the management of the hardware resources of multiple public clouds with a private cloud in an on-premises data center. Those are the basics. Increasingly, what is also needed in such a solution is the ability to orchestrate IT processes and provide cloud and cost governance, all in an automated manner.

Enter Vertica, a Micro Focus line of business. Vertica’s SQL database and in-database machine learning support the entire predictive analytics process with massively parallel processing and a familiar SQL interface. And it allows companies to deploy machine learning in hybrid deployments. You can operationalize ML and scale ML with Vertica.

Learn more: https://www.vertica.com/product/database-machine-learning/