Some of the most successful companies these days have mastered “intelligent integration,” in which applications can analyze and route data from multiple systems in pursuit of a business result.
Curious what data systems integration can do? Look no further than Amazon, which is able to connect separate data systems and applications for a real-time recommendation system. To the user shopping on Amazon, the process appears seamless. Product recommendations arrive embedded on their personalized homescreen, in an easy-to-use web interface or mobile app.
Behind the scenes, data integration and business applications are at work. Algorithms combine data from a customer’s past-purchase history and current browsing patterns to make product recommendations, and of course Amazon also analyzes product and warehouse data to enable efficient shipping. (Related: “Omnichannel marketing with data: Beating Amazon at its own game.”
Another example is Netflix. To the user, movie recommendations are embedded on the Netflix interface in several categories, whether “recommended for you” or “popular now.” Peel back the curtains, and near real-time user events are managed through a distributed computing system called Manhattan. As Netflix explains, data flow is accomplished logging through Apache Chukwa to Hadoop. A machine-learning algorithm parses data from primary data stores including Apache Cassandra, EV Cache, and MySQL. (See: “How machine learning fuels your Netflix addiction.”)
In another use case, that of retail, a business application may need to analyze data from point-of-sale systems and an inventory database. Such analysis can guide real-time decisions on product pricing and inventory control to enhance sales.
Taking it a step further, in the case of brick-and-mortar retailers that use beacons, an application may need to read geo-data from a customer’s smart phone, data from a customer-relationship management system to see the customer’s past purchases, and data from in-store inventory. An analysis could then push promotions for certain items.
Data system integration in the industrial IoT
As data moves from its origin of creation to back-end data storage systems, the use of data integration and application integration — what is know being called “intelligent integration” — becomes a pressing issue. In industrial environments especially, the use of gateways to process data from edge devices is crucial.
From an architectural standpoint, business processing and reporting might occur at the data center level. Gateways, however, might perform functions such as processing, messaging, and filtering of data to enable real-time analytics.
In a rail system, for instance, data might come from the train’s onboard control system which measures speed; from waypoint systems that monitor rail traffic; passenger counting systems; a gateway system that analyzes and routes edge data; and from a data store that contains train schedules. (See: “IoT automation in smart transportation systems.”)
Or, in a factory, machine sensors on a piece of equipment might feed data to a gateway, which performs some analysis and filtering functions before sending that data onto the cloud. The overall architecture, meanwhile, might also involve the use of an historical data store; a customer relationship management system; and an enterprise resource planning system.
To enable a business use case such as predictive maintenance of equipment, applications might need to be deployed on any layer of the architecture: For instance, an analytic model to optimize asset performance of equipment might be developed in the cloud and pushed out to the gateway.
