IoT Security Solutions for Critical Infrastructure

Multi-site critical infrastructure operations managers must secure their sites to ensure uninterrupted services for their customers that depend on them.

For Critical Infrastructure, security is non-negotiable. Protecting networks, systems, and assets for uninterrupted operations is critical to the security of an organization, a city, and a country. It is a sad fact that cybercriminals can penetrate 93% of company networks (betanews.com). Given this, the best protection is making it as difficult as possible, forcing cybercriminals to move on to other targets given the time and effort required to deal with a well-protected network and the prevalence of other targets. In 2021, businesses suffered 50% more cyberattack attempts per week (darkreading.com) when compared to 2020. What’s more alarming is Accenture’s Cost of Cybercrime Study found small businesses get 43% of cyberattacks–but only 14% of them can defend themselves. Security is of utmost importance in this day and age of the proliferation of ‘connected’ devices, yet at the same time, there is a deafening silence to both the risk and the solutions, given the fear of public knowledge of a breach to an organization.

A security solution is no longer just the purchase of a virus scanning package but requires day-in and day-out analysis of the ever-increasing portfolio of devices that both enter and leave a network. Here’s a list of what these devices are and the recommended security for integrating them via an IoT platform within a greenfield or brownfield environment.

The Typical Building ‘Edge’ Solution

These edge solutions often consist of many parts and pieces, each from different manufacturers and often different models and generations of equipment. IoT bridge devices are commonly used to collect and rationalize this data into a usable form and, most importantly, storage buckets in the cloud for real-time and historical outcome-based analytics. This starts our story with the generally private network of the ‘Edge.’ 

The Edge Private Network and RTU – an environment of an edge bridge or remote terminal units (RTUs) that collects information from far edge devices’ native protocols. This can often include sensors or systems communicating on things such as Modbus, BACnet, SNMP, and others. Often there are also TCP/IP-based devices, but this is considered a non-public network at the edge, and the edge bridge operates as its collector of data and also a service provider for things like DHCP, DNS, and other networking services. Any way you look at this, however, it’s a private network with the Bridge as the single source of truth to feed the cloud. Access to this is commonly protected through private networking that is not directly on the internet, and even in a situation where cellular is required, these operate on private APNs over the cellular carriers to limit traffic. Access to these Edge devices is key in any security policy.

Private networks should encompass all the far edge TCP/IP enabled devices, RTUs, and cloud infrastructure. Most often, the outside vector of attack for vulnerabilities comes through the user interface (UI) since it would most likely have an open port.

Standard IoT solutions via edge bridging often offer configurable trusted clients to allow access to internal device UIs. These can be put behind virtual private networks (VPN)s with established two-factor authentication (2FA) schemes. If external clients are absolutely necessary without the deployment of a VPN, make sure your web server is deployed at a minimum using:

• An SSL Certificate to deploy SSL/TLS encryption allowing HTTPS to the platform’s web server.
• Configured and restricted firewalls to the ports required, and ideally, IP whitelisting.
• Router and firewall provisioning to address brute force attacks using automated IP address banning.
• Vendors may offer several edge RTUs for simple installation with specific requirements. With a multi-site critical facility solution, the architecture typically has an edge RTU at each site that connects upstream to a cloud instance of the solution on a VM. If you’re using this type of solution, both the edge RTU and cloud VM should reside in the same private network. Any IoT solution should allow its software to run both on proprietary hardware as well as customer-supplied hardware. This allows scalability and flexibility for customers without the long-term vendor lock problem.  
• It’s recommended to pay close attention to far edge devices, particularly those that communicate via TCP/IP–especially when there’s no RTU as an arbitrator.

Note: Often, edge devices require upgrading. Upgrading is highly recommended as they often include security enhancements. This is best performed on regular maintenance intervals that require manual intervention either in person to perform the upgrade or, if this is not possible – the manual opening and then closing of ports to allow for upgrades. This does require more effort on behalf of the operator but is seen as the most secure way of edge device upgrades for device security when internal upgrade services are not possible.

See also: Bringing AI to the Edge: Focus on the Plumbing First

The Critical Facility Monitoring Software (CFMS) is a cloud and Edge based solution that enables monitoring, data aggregation, packaging of data, diagnostics, and remote control of far edge device information. It also enables remote multi-site management from a single or centralized location–connecting to a database (either internal or external) to allow long-term storage of Far Edge Data. As previously mentioned, these often sit as an arbitrator and firewall between edge field IoT devices and the inner network where the storage of device data and remote operation occur.  

• Though these devices are on the interior network, they are a point of significant interest to any party looking to access the network, given they operate as the intermediary between other protocols and the internal TCP/IP network:
  • Use a strong user password within the IoT platform’s user management–preferably single sign-on (SSO) integration.
  • While certain solutions have built-in protection, it’s critical to manage all your file permissions.
  • When planning your user rights management, pay particular attention to the granular permissions of data sources to ensure allocation of all role-based access. Make sure to pay particular attention to inheritance and enhanced access control.
  • Use whitelisting on any edge device firewall to only whitelist the approved cloud connections for data transfer.

IoT Cloud Services typically include virtual computer resources, database technology for long-term edge data storage, firewalling, and networking to enable edge devices. Amazon (Amazon Web Services), Google (Google Cloud Platform), Microsoft (Azure), and Digital Ocean are among the common cloud providers you can use though it is not uncommon to use ‘bare metal’ instances in environments that are not willing to use cloud providers.

• Depending on your vendor, Cloud platform operations and naming conventions may vary. It is typical for cloud environments to run hybrid connecting to other cloud services by other vendors, increasing the complexity of security. The choice of a vendor can have massive security implications, but more importantly, who is managing that cloud service plays a pivotal role. The following are standard recommendations for cloud services:
  • Limit public network ports to the required CFMS ports only.
  • Restrict private network side ports to an ‘as need’ basis. This can heavily depend on your IoT solution. Fewer ports are always better.
  • If you decide to deploy an independent CFMS database instead of the integral database from your vendor, make sure the database doesn’t reside on the same VM as the vendor solutions’ CFMS runtime–and uses a virtual private cloud (VPC) to communicate with the database to avoid any requirement to open public ports. At no time, make public ports available to a database.
  • If the CMFS requires external web access, always use SSL certificates and secure ports.

CFMS Database is the location where data is collected from far edge devices. Typically, this is from your cloud provider and where all your data from the Edge locations flows after being packaged. It is the sole source of where your analytics feed from. Don’t confuse this with instances where external data lakes also are connected to the cloud provider to allow data storage for custom applications. The database is the single source of truth, and often the CFMS contains a convenient API to access this database through a REST API or gRPC.

• For scalable solutions and larger projects, use a managed service database environment from a major cloud provider, which will allow you to build a simple and effective MySQL managed database solution. At all times, this should be external to the CFMS allowing common data portability and backups through standard I.T. means.
• As mentioned above, at no time should this database be publicly accessible. External parties should access this via an API available from the CFMS–the CFMS should access the database only through a VPC connection avoiding any public access. 

Network Infrastructure is a collection of virtual or physical networking components, including routers, switches, firewalls, DHCP servers, cellular modems, etc., that facilitate Ethernet traffic between devices in a private or public network. In any cloud environment, this is often inherent in the system.

• Plan on a network infrastructure solution before deployment and use standards that fit the needs of the business while offering a secure environment. Often a vendor offers a fully managed solution that builds this infrastructure, but if it’s required that you deploy into the existing architecture, it would fall on your organization to provide servers and security policies to meet the solutions’ needs.
• Given the vastly different cloud environments available, it is imperative that you either work with someone that understands the specific cloud instance and their respective security solutions; or optionally use a fully managed service IoT solution that takes care of this for you. This, by far, is the riskiest surface of attack in any solution.

Brownfield Systems are typically edge systems that are running without an RTU. They are often pre-existing and are deemed too expensive with no tangible ROI to replace. They could be running on TCP/IP or other proprietary protocols. Examples of these are lighting systems, building management systems (BMS), and fire alarm and suppression systems.

• While similar to far edge devices, typically, a Brownfield system employs a gateway that facilitates a translation between a proprietary protocol (or wire plant) and Ethernet protocols. Most commonly, these are single NIC devices with proprietary firmware that may require custom programming to facilitate the mapping of objects and functions to make logical sense to any other system. A wide range of these systems exist. Should they exist, it is important to understand if they have a REST API that an IoT CMFS can just exploit or if there is something more proprietary that must be done to facilitate connections.
  • This style of devices, even if they are TCP/IP, should not openly traverse the IoT cloud. They should be relegated to just the edge and allow the RTU to perform the communications function with these bridges to limit risk.
  • Practice the principle of least privilege to verify you are granted only the minimum necessary to complete the required functionality with any communication with these devices.
  • Ideally, encrypt payload traffic using TLS, if available.

Upstream Connection (Latent) is the physical connection of Edge to Cloud. This is a TCP/IP-based connection to the location where the critical facility management software resides and operates. It is considered latent unreliability, given any IoT solution should be able to tolerate upstream instability in situations such as cellular, satellite, and Wi-Fi.

• Use a private network when at all possible. Hard-line ethernet is always the best, followed by cellular. Wi-Fi should be avoided.
• Cellular–though not recommended for primary communications to the Edge–is a viable option for some applications as a primary and, more commonly, a backup connection to the cloud. Using cellular technology as a backhaul to the cloud, the most secure path is via a private APN from the service provider–which, although not always possible, public APNs can be used so long as your RTU has a firewall.

Service Bus is an authenticated REST API that allows third-party software and analytics to use the data (both current and historical) stored by the Critical Facility Monitoring Software. It also allows external access to the CFMS Database, typically for external analytics.

• When feasible, use solutions based on solid, proven authentication and authorization mechanisms–such as OAuth2.0 or key exchange for external authorization to the vendor’s API.
• Practice the principle of least privilege verifying to only expose pertinent information to the foreign entity via the REST API.
• Use TLS for transport.
• Use rate limiting in the network infrastructure to reject subsequent requests at a threshold (10,000 requests a day or lower) to prevent DOS attacks.
• Use IP whitelisting.

Security threats are not going away. In fact, last year, the U.S., Australia, and the UK’s cybersecurity authorities reported an increase in sophisticated, high-impact ransomware incidents against critical infrastructure organizations globally. It’s ever more critical that multi-site critical infrastructure operations managers take steps to secure their sites and to ensure uninterrupted services for their customers and organizations that depend on them. These recommendations are only the start of what should be considered fundamental in any IoT solution globally.

The best bet for any organization is to engage with an IoT Platform firm to discuss their existing infrastructure and potential needs into the future.