Data Management and AI: Creating Value at the Edge

By applying edge-specific capabilities and AI integration at the data source, businesses can eliminate inefficiencies, gain greater operational awareness, and turn data at the edge into revenue-generating opportunities.

Edge adoption is skyrocketing, forecast to reach $168 billion with a CAGR of 24.51% by 2030. Why the surge in growth? Edge data and the real-time insights derived from it contain immense business value, with the ability to drive revenue, transform customer experience and boost the bottom line.

Yet, traditional methods of capturing, storing, and analyzing data don’t always deliver when it comes to the edge. New approaches are needed to manage and analyze the vast volumes of streaming data coming from edge devices and to turn that data into business benefits. This includes addressing the special data management considerations that arise when working at the edge and how to apply AI protocols at the source to gain real-time insight.

To explore the challenges and technologies surrounding edge data value, RTInsights recently sat down with  Bill Schmarzo, the Dean of Big Data, to discuss possible solutions, such as its Streaming Data Platform.

Here is a summary of our conversation.

RTInsights: Why is there so much interest in the “edge,” and why now?

Schmarzo: The edge is more than just a collection of new devices and data sources deployed at edge locations. At its best, the edge has the ability to ingest, analyze, infer, and act upon business-critical data in real or near real-time, essentially transforming the business. 

As such, the edge enables new business and operational use cases – everything from smart places to predictive maintenance to robotic process automation (RPA) and location-based customer engagement opportunities.

For many organizations, the edge is the front line of customer and operational engagement. It’s where business gets done. The edge has become the new monetization and value creation frontier.

RTInsights: How does the edge help businesses with respect to monetization and value creation?

Schmarzo: Monetization and value creation happen when businesses use edge to collect and analyze data and are then able to act upon the insights derived from that data.

This capability lets us connect with customers and devices in action, flagging anomalies and engagement opportunities as they appear. Such instances and insights can be used, for example, to push an offer to in-store customers just as they approach a specific display.

Encounters and engagements with customers via edge devices (e.g., smartphone apps) can reduce customer support calls and fraudulent transactions. The edge can also play a role in more complex engagements that lead to revenue generation. For example, a financial services business might seek to increase customer retention using the edge to provide hyper-personalized offerings. The same information and insights could be used to improve location-based marketing effectiveness.

A manufacturing company could use edge solutions to reduce unplanned operational downtime, accelerate fault detection, improve maintenance, reduce repair times, improve fix-first-time effectiveness, reduce labor and materials costs, and improve operational effectiveness.

A healthcare organization could use edge to improve the quality of care. Edge use cases in healthcare include accelerating issue detection, improving the quality of care, reducing hospital stays, unplanned readmissions, hospital-acquired infections, and improving caregiver satisfaction.

In all these instances, there is an impact on the bottom line. It could be costs avoided, costs reduced, improved efficiency, and increased transactional opportunities. All of these are made possible by interacting with data at the edge.

RTInsights: What challenges do businesses have to address to monetize or create value using the edge?

Schmarzo: While edge generates many new value-creation opportunities, there are planning, implementation, management, and governance challenges that are different from those found in traditional IT systems.

One critical difference is that edge implementations frequently rely on streaming and real-time data. In many applications, this data comes from a wide and diverse variety of edge devices and data sources. An effective edge solution must be capable of analyzing that data and combining it with historic data.

The challenge is that traditional data management applications have been optimized to support the batch ingestion of data from ERP, CRM, SFA, and other enterprise applications. The processes and governance policies that worked for those workstreams are not always sufficient for the variety and streaming speed of data generated at the edge.

Another difference is that edge applications typically include different types of data sources. Traditional data management has focused on managing structured data. But edge data comes in semi-structured such as log files and JSON files and unstructured formats, like text, video, audio, and sensor data.

Furthermore, edge devices, particularly in asset-heavy industries like manufacturing, are part of Operational Technology OT systems, not IT systems. The devices directly monitor or control industrial equipment, assets, processes, and events. They communicate and share data using different protocols than IT devices do. So, many edge applications must be able to integrate IT and OT technologies.

Possibly the biggest edge challenge comes with ingesting, analyzing, and acting on data as it streams off different edge devices and sources. Traditional methods of capturing edge data, storing it in a cloud data warehouse or lake house, and then analyzing it negate the benefits of edge. Edge innovations like autonomous vehicles and intrusion detection depend upon at-the-source data analysis to make rapid, real-time decisions.

RTInsights: How does Dell Technologies help?

Schmarzo: The Streaming Data Platform, or SDP, automates and scales repetitive data management tasks, including data ingestion, analysis, and IT/OT integration, which are so critical for edge use cases. 

SDP helps businesses integrate real-time edge data with deep historical data in a single, auto-scaling infrastructure. It offers many other edge-centric data management capabilities. Here are a few:

Rapid ingest, inference, profiling, and analysis of data during streaming: Accelerates the ingestion of structured and unstructured data from source systems, automates analysis and metadata tagging of data during ingest, data profiling (data statistics), and data transformation (enrichment, normalization, n-tiling).

Intelligent data movement, monitoring, and management via analytics-infused data pipelines: Replaces outdated and sprawling ETL/ELT jobs with general smart data pipelines that accelerate the integration and analysis of data. Allows for continuous monitoring of data pipeline performance and health. Pipeline intelligence flags data discrepancies at the source for rapid problem resolution.

Integrates analytics results with operational systems and management dashboards: Leverages open-source data visualization tools to accelerate the inspection of data trends and relationships. Integrates data visualizations into management and operational dashboards.

Acceleration of edge ML Model development and monitoring: Accelerate the building, testing, and refinement of data models with Spark, Flink, and TensorFlow. Share data and analytic project learnings using Jupyter Notebooks. Speed the transition from exploratory projects to scalable data products.

RTInsights: Can you give us a use case example?

Schmarzo: An example of a smart manufacturing edge solution is the Dell Technologies Validated Design (DTVD) with Litmus.

Litmus Edge is a complete operating system based on Linux that eases deployment and management at the edge. It can be run on a bare-metal hardware device or within a VM. It enables connectivity of manufacturing assets, with more than 250 different OT-based protocols available out of the box and the capability to build custom connectivity. It can collect, normalize, and contextualize data for further use on edge devices or northbound to your enterprise IT systems and platform.

This DTVD solution is built in partnership with Litmus and centrally manages and orchestrates industrial edge devices, data, and applications, from the factory floor to the enterprise cloud. It accelerates smart manufacturing outcomes around Overall Equipment Effectiveness or OEE, predictive maintenance, product quality, yield optimization, and more by using available data.

To summarize, Dell understands that gaining value from edge investments is not as simple as building out systems and devices at the edge while relying on traditional data management and analytics processes. By applying edge-specific capabilities and AI integration at the data source, businesses can eliminate inefficiencies, gain greater operational awareness, and start turning data at the edge into revenue-generating opportunities.