A successful enterprise IoT architecture needs fast ingestion, an operational database, event triggers, and data export for longer-term analytics.
Enterprise architecture is an understated yet essential piece of the real-time, Internet of Things story. The capabilities and services delivered on the front end – described in a large cache of our case studies – represent exciting new horizons for today’s businesses.
Enabling IoT analytics requires great architecture — the planning and scoping of technology resources into a master plan to help guide decisions and purchases. While built on well-laid plans, enterprise architecture also needs to be highly adaptable, and open to ever-shifting technology innovations.
IoT is still relatively new to organizations, so many architectural plans are still works in progress. The state of enterprise IoT architecture – and the elements in place that are making it work – were recently documented by Ryan Betts, author of Architecting for the Internet of Things. Betts points out that IoT data management platforms need to handle data in motion — fast data — as well as big data, which is usually at rest.
Dealing with legacy data in an IoT world
The challenge – and this is where enterprise architects earn their pay – is in bringing legacy back-end systems into the IoT world. To put it simply, most legacy systems – those built and installed before 2010 – weren’t made for the deluge of real-time IoT data and processing. There are even many organizations running mainframes and applications first written in the 1970s and ‘80s.
In his book, Betts, who is currently CTO of VoltDB, cites examples of how some back-end systems are being moved from batch-processing mode to support real-time IoT. “An example of this is the validation, estimation, and error platforms sitting behind real-time smart grid concentrators. There are many use cases (real-time consumption, pricing, grid management applications) that need to process incoming readings in real time.”
A good enterprise architecture methodology will recognize what legacy applications are capable of being rewritten to accommodate real-time networks, and which would benefit from a “platform that offers real-time capabilities to real-time applications while supporting stateful buffering of the feed for downstream batch processing,” he says.
Betts documents the rise of a new reference architecture that supports sensor data, machine-to-machine communications, big data, cloud computing, distributed systems, networking, mobile and telco, apps, and smart devices.
There are four elements that enterprise IoT architecture needs to address to ensure success:
Fast ingest
The ability to ingest data from potentially thousands of events is key to real-time IoT. “The system needs the performance to scalably ingest these events and to be able to process these events as they arrive, discrete from one another,” Betts says. “Applications need high-speed ingestion, in-memory performance, and horizontal scalability to provide a single ingestion point for very high-velocity inbound data feeds.”
Explore and analyze
An IoT-enabled architecture requires “real-time access to applications and querying engines, enabling queries on the stream of inbound data that allow rules engines to process business logic,” says Betts. “As these events are received, stored, and processed in the operational database, the system needs to allow access to events for applications or querying engines.”
Act
“Applications also require the ability to trigger events and make decisions based on the inbound stream: thresholds, rules, policy-processing events, and more,” Betts advocates, adding that “triggered events” range from “updates to a simple notification service or to a simple queuing service that are pushed, based upon some business logic that’s evaluated within the operational database.”
Export
An IoT-enabled application “needs the ability to export accumulated, filtered, enriched, or augmented data to downstream systems and long-term analytics stores,” says Betts. “As this data is collected into a real-time intraday repository or operational system, you can start to write real-time applications that track real-time pricing or real-time consumption, for example, and then begin to manage data or smart sensors or devices in a more efficient way than when data is only available at the end of day.”
