For businesses that routinely use real-time applications, cloud-native architectures significantly simplify application design, coding, and customization.
Continuous intelligence (CI) requires real-time analysis of streaming data to make decisions in milliseconds to seconds. Data issues with CI alone can be challenging given the volume of data generated these days from smart devices, social media platforms, clickstreams, and more. Making matters more difficult is that most CI applications do not exist in a bubble. They often are part of a larger business process that involves multiple datasets analyzed using a variety of tools.
See also: What Is Cloud-Native and Why Does it Matter for CI?
A simple example illustrates the issues businesses face. A retailer might use CI on social media streams to identify customer sentiment in the moment and on an online customer’s current session clickstream information. Information derived in such analysis might be combined with the customer’s preferences and past purchases to make product recommendations in real time. Each process uses different data sets, typically different databases, and different analytics methods. Yet, all the steps must be executed at the same time, so the derived information can be used to make on-the-spot offers personalized for the customer online at that moment.
In the past, a single, all-encompassing application would be developed, incorporating these distinct computational chores. The problem with such an approach is once developed, any changes (such as wanting to use a new data source or analysis application) require reworking the entire application.
As discussed in a previous blog in this series, a better approach is to develop CI applications using a cloud-native architecture based on loosely connected microservices. A cloud-native approach is particularly well-suited for CI for several reasons.
First, a cloud-native approach can more easily accommodate data used in CI applications. One issue here is that data streams often are highly variable. For example, a financial services application to detect fraudulent transactions in real time would typically see the most massive streams of data when markets open and then when they close. The volume of data the rest of the day would be significantly lower. Cloud-native architectures can accommodate such fluctuations because they scale easily.
Second, many application areas where CI is being used are highly dynamic. Businesses frequently need to use new and different datasets and innovative analytics solutions as they emerge. Again, a cloud-native approach helps in that each component of a CI application can be deployed and maintained as a self-contained microservice. If any element of a broad application needs to be updated or changed, only that element needs to be touched. The rest of the conglomerate application remains the same.
Third, with mergers, acquisitions and decentralized organizations, organizations have a wide range of infrastructure. Private data centers, public and private clouds from multiple vendors is the rule, and not the exception. By using microservices running in this multi-cloud universe, new applications can be implemented more quickly using cloud-native technologies without regard to location.
Data Services in a Decentralized Model
The boom in the use of CI, artificial intelligence (AI), and sophisticated analytics for mainstream business processes is placing new demands on the ways in which data services are handled. Companies need to get data from its source of generation, get it prepped and put into a suitable format, and then feed it into compute resources to run analysis programs.
Traditional database methods do not work for CI. In the past, all data might be saved to a single data warehouse allowing analysis of historical data over time. With CI, streaming data must be ingested, prepped, and analyzed as the data is generated. That requires a different data services model.
In many cases, streaming data also must be saved. For instance, a CI predictive maintenance application might use streaming data from the Internet of Things (IoT) devices embedded in equipment on a manufacturing floor to spot and correct real-time incidents that cause downtime. After the fact, that data, if saved, could be analyzed to find root causes of typical problems. In each application (real-time detection of problems and analysis of historical data to find root causes), data is handled and managed in different ways and analyzed using different tools.
Another factor that is placing new demands on data services for CI applications is that data and analysis must be done in different environments. In the past, all data might be backhauled to a data center or cloud service center for analysis. CI applications often require that the database run on a mobile or edge device and scale to enormous capacity.
The way to accommodate dynamic environments and data services is to use a cloud-native architecture with application elements running as microservices, with access to local data or virtualized access to remote data as appropriate.
Using a microservices approach, changes can be made in existing infrastructure without disruption, allowing improvements and the use of new data and tools over time. The right architecture makes all the needed data available for line-of-business analytics, executive insights, reporting, compliance, governance, and data science that uses machine learning and artificial intelligence.
A cloud-native architecture using microservices also helps bring together data sources and analytics elements that, in the past, would have been siloed and not easily accessible. Specifically for data services, critical benefits of cloud-native include:
- A core data platform capability that supports high availability, security, performance, scalability, and management
- Data integrity and consistency to deliver trusted data
- Ability to use and integrate a wide variety of tools for data management functions
- Analytic models created in many languages and tools such as Python, R, Java or C/C++
- Support for concurrent queries, transactions, reports, and data access patterns
- Deployment on-premises or in the cloud
- Easily scalable microservices that can be changed without impacting the rest of the application, because each runs independently in a decentralized manner
- Access to data using standard connectivity such as SQL, ODBC/JDBC, XML, or REST
For businesses that routinely use real-time applications, cloud-native architectures significantly simplify application design, coding, and customization. Such an architecture platform can support real-time insights, machine learning, streaming analytics, and high-performance transactional processing.
Teaming with a Cloud-Native Technology Partner
IBM has been a leader in aiding the industry move to cloud-native applications. In the past few years, it has transformed its software portfolio to be cloud-native and optimized it to run on Red Hat OpenShift. With this transformation, businesses can build mission-critical CI applications once and run them on all leading public clouds—including Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform, Alibaba, IBM Cloud—as well as on private clouds.
With specific regard to CI, IBM addresses streaming data ingestion and analysis issues with IBM Cloud Pak for Data. IBM Cloud Pak for Data is a fully integrated data and AI platform that helps businesses collect, organize, and analyze data and infuse AI throughout their organizations. Built on Red Hat OpenShift, IBM Cloud Pak for Data integrates IBM Watson AI technology with IBM Hybrid Data Management Platform, DataOps, data governance, streaming analytics, and business analytics technologies. Together, these capabilities provide the architecture for CI that can meet ever-changing business needs.
IBM Cloud Pak for Data is easily extendable using a growing array of IBM and third-party services. It runs across any cloud, allowing businesses to integrate their analytics and applications to speed innovation.
Complementing IBM Cloud Pak for Data, IBM Cloud Pak for Data System is a cloud-native data and AI platform in a box that provides a pre-configured, governed, and secure environment to collect, organize, and analyze data. Built on the same Red Hat OpenShift Container Platform, IBM Cloud Pak for Data System gives businesses access to a broad set of data and AI services and allows quick integration of these capabilities into applications to accelerate innovation. The hyperconverged, plug-and-play system is easily deployable in four hours.
To learn more about how a cloud-native architecture can help with your CI efforts, visit Cloud Pak for Data.
Read the other blogs in this series:
- What Is Cloud-Native and Why Does it Matter for CI?
- How Kubernetes and Containers Enable Highly Scalable CI Applications
- Considerations for Successful Continuous Data Ingestion and Analysis
- The Benefits of Using Self-Service, Elastic, Cloud Infrastructures for Continuous Intelligence
