A generation of video applications treats cameras more like the internet of things (IoT) devices by monitoring asset conditions, identifying equipment using bar codes, license plates, or the movement of vehicles and personnel.
Video cameras are ubiquitous in the energy sector and have been for many years. Being asset-intensive industries, they have a lot of very expensive equipment and operations that may pose hazards to protect. As a result, perimeter security and general surveillance are critical applications for video in energy. However, applications are emerging that use cameras for completely different tasks – creating a new and expanding role for video and, especially, analytics.
See also: Too Much Real-Time Video? Visual Analytics Can Help
This new generation of video applications treats cameras more like the internet of things (IoT) devices by monitoring asset conditions, identifying equipment using bar codes, license plates, or the movement of vehicles and personnel. They are also much more efficient at real-time video security. The key technology innovation behind this growth is the use of sophisticated video analytics based on machine learning and artificial intelligence (AI).
The concept of video analytics is simple enough. To use the example of perimeter surveillance, 99% of the time, the scene that a camera records is virtually the same thing day after day. So, when an intruder actually cuts through the fence, an analytics application may not know exactly what it is seeing, but it knows it is different than what it normally surveils. It is, in the language of machine learning, “an anomaly.”
When the surveillance analytics confront an anomaly, the application can alert someone in security to view that footage. Security personnel can then immediately verify if it is an intrusion event or not; the application doesn’t have to be that smart. This saves personnel from having to watch hundreds, or thousands, of hours of video just to catch the few minutes when a potential intrusion occurs. A video analytics program essentially filters the footage and presents only the bits that might actually matter.
If it turns out that the anomaly is simply local wildlife or a wandering dog, personnel can “teach” the program to ignore it. The application then stores the pattern of the dog and no longer treats it as a threat.
Early attempts to create similar applications took a direct approach by trying to pre-configure the analytics application with known threat or incident patterns. However, these were less successful unless the threat was very well known and predictable in form, such as a deformation in a round object, such as a gear or a wheel. For many kinds of surveillance and monitoring, though, the incident pattern is not consistent enough to pre-program.
This is where machine learning and real-time analytics are coming to the fore. For instance, a camera that is watching an intersection in a coal mining operation is constantly recording the movement of trucks as they go by hauling ore. This isn’t just a static perimeter fence where nothing ever happens. Nonetheless, the application can learn over time to understand which patterns of activity are normal and which are not. In this way, a real-time video application can recognize a problem when an autonomous truck breaks down in the middle of the intersection, or an object falls off and blocks the roadway. The program doesn’t have to have anything programmed into it. It simply deduces what is normal and what isn’t, with occasional feedback from personnel to tell it when an anomaly can be considered “normal.”
From an operational IT perspective, one of the great advantages of video analytics is that it also cuts down on the amount of video that must traverse a network. For example, traditional CCTV solutions required dedicated networks for the cameras, whereas video analytics can actually be hosted at the edge of the network as video is processed locally. The 99% of video being recorded that is of no interest can then also be discarded or stored locally, while the anomalous video footage, which is much reduced, can be sent over the primary cabled or wireless network to a central cloud server.
Video analytics programs can also add data from other sources such as audio, telemetry, and data from IoT sensors. For instance, with the stalled autonomous ore hauler, the video analytics program might also be monitoring heartbeat messages between the truck and the communications network and understand that the truck has lost its network connection and is waiting to acquire a signal — a common occurrence with Wi-Fi mesh networks. It could also be taught to ignore the stalled truck in this case, but not in others.
There are different use cases for every part of the energy and utility sector. Wind farms, for example, might use real-time video analytics to identify flocks of birds. When the camera records the approach of a flock, the turbine blades are shut down to avoid collisions. Another application for gas pipelines uses infrared cameras in combination with IoT gas monitors. When the presence of a leak triggers an alarm, the video analytics program can be prompted to scan for heat signatures along the pipeline to locate where it is occurring.
Video can also play its part in general environmental monitoring alongside IoT sensors that measure chemical contamination in air and water, moisture and vibrations in soil, weather tracking, as well as ambient heat and humidity. Video has been used to record water flow in streams to detect the risk of flooding, which can be used with other sensor data to populate hydrological models that can predict possible risks to assets or safety from flooding. On the simpler side, real-time video analytics can also be used to log events and store them for forensic analysis later. Only video footage that departs from the normal pattern is stored and can later be called up when investigating the causes that might have led to a major incident.
Video analytics are beginning to play a critical role in preventive maintenance of energy assets as well, including pipelines, powerlines, and equipment. Predictive maintenance leverages IoT data, audio, and video streams – using advanced data analytics – to predict the optimum time for maintenance and replacement of assets. This is in contrast to traditional preventive maintenance, which can lead to waste by refurbishing or replacing assets that are actually still in serviceable condition. It’s akin to throwing out a can of food just because it has reached the stamped expiration date, without first opening it to see if the contents are still good.
Condition-based assessment, on the other hand, is able to use real-time analytics to predict failure times and optimize maintenance options. It also reduces costs, increases utilization, enhances safety, and minimizes delays and revenue loss. Advanced analytics can also break down data silos by correlating video and data from IoT sensors, environmental information, and historical trends to provide operations intelligence, solve specific operational and maintenance pain points, and optimize asset lifecycles.
Clearly, real-time data analytics using machine learning and AI are revolutionizing almost every corner of the energy business. IoT devices and sensors are getting the lion’s share of the attention, but video is also getting a second life as a critical source of visual data. These analytics systems overcome one of the principal issues that plagued the first generation of CCTV; namely, the firehose of footage they would produce was simply too expensive to monitor. Video analytics, mixed with other kinds of IoT sensing, are now able to present consumable amounts of data with a much higher rate of relevancy for the industry and are helping to focus that fire hose where it’s needed most.
