How to Tackle OT Challenges: Asset Inventory and Vulnerability Assessment

The plurality of devices and protocols found in operational technology (OT) environments makes asset discovery and remediation a challenge. Here’s how Tenable OT Security can help.

The Operational Technology (OT) cybersecurity sector is facing new opportunities and challenges as the complexity and vulnerability of formerly isolated OT/ICS networks have expanded due to the convergence of IT, IoT, and OT networks. The number of vulnerabilities disclosed in OT systems is constantly growing bringing a new way to think about OT systems; with the increasing adoption of Industry 4.0¹ and smart infrastructure, OT has become of critical importance to organizations. Obtaining an accurate asset inventory and performing vulnerability assessment on those assets is challenging for several reasons, including:

• OT systems are often legacy, proprietary, or customized, making them difficult to update or patch
• They have different performance and availability requirements than IT systems, and may not tolerate frequent scanning or monitoring
• Most OT environments have a plurality of devices, including programmable logic controllers (PLCs), remote terminal units (RTUs), human machine interfaces (HMIs), sensors and meters, safety controllers and industrial internet of things (IIoT) devices, just to name a few
• Each OT vendor has specific characteristics and protocols for its equipment, including proprietary or open communication protocols and standards

The above factors are some of the reasons organizations are hesitant to actively monitor or patch OT devices. The traditional methods of asset discovery and remediation that work in the IT space can cause disruptions and downtime when applied in OT environments. At the same time, the convergence of IT and OT environments is expanding the attack surface. The ability to maintain a comprehensive view of OT assets and to take a deep dive into device information requires being able to talk to each device in a language it understands and, even before that, to know which device to talk to.

An OT asset's visibility depends on its network activity. As an example, industrial control system (ICS) protocols, such as the common industrial protocol (CIP), do not commonly broadcast on the network. It’s easy to assume this adds a layer of stealth that would keep the underlying assets invisible to attackers, but it’s not true. An attacker would still be able to actively discover such an asset and, as a defender, you have to remain one step ahead and maintain a holistic and real-time view of your infrastructure.

In this blog, we use examples from our lab environment to demonstrate how Tenable OT Security can help you tackle these hurdles while both passively monitoring and actively querying your assets for in-depth asset visibility.

Passive network detection

Passive scanning is a technique that involves silently gleaning network data to detect asset information and weaknesses, without actively interacting with endpoints. It is implemented by sniffing all network traffic and parsing communication protocols that will identify endpoints.

Using this approach exclusively has some major limitations; “dormant” devices will not be discovered and devices’ metadata — like firmware, model, version and other relevant information — will not be detected. These are major obstacles to an effective asset mapping and vulnerability assessment.

Also, passive monitoring requires network mirroring. It could be achieved in a variety of ways, which are explored in the SANS white paper Practical Industrial Control System Cybersecurity: IT and OT Have Converged – Discover and Defend Your Assets sponsored by Tenable. Also, often, passive monitoring is not even possible in OT environments due to factors such as the age of the infrastructure, architectural constraints, technological limitations, or specific process and customer restrictions. Implementing passive monitoring in older environments can incur significant time and financial costs.

With passive monitoring, it is most likely that you will not receive any relevant packets without a PLC being requested by a workstation. Moreover, if you do receive some data, you may not read it because of encryption. In the Tenable lab environment, we got zero CIP packets on our first interface even with multiple backplanes administered from the same network. Capturing from other interfaces, we received some broadcasted UDP ListIdentity requests from other workstations:

It also shows the importance of choosing/combining the right monitoring solutions (port mirroring, hardware test access point [TAP]), optimally located on the network, in order to be able to sniff out the maximum amount of data.

Now, to make our comparison between passive monitoring and active querying, we’ll suppose that, while passively listening on the wire, we are able to capture the initial exchanges between an engineering workstation and some endpoints (one of the best-case scenario):

Looking at a random Ethernet/IP CIP “Get Attribute All” service response provides identification and general information about the Ethernet/IP-enabled device. In this case, we can assume that the asset located at 10.100.101.157 is at least composed of 1768-ENBT communication adapter.

Because the network capture was running when the operator went online with the PLC, it also contains more interesting packets. Let’s take a deeper look at what kind of information we have in them:

1. The CIP packet class indicates this is an Identity Object, which is the most classic object type that all CIP devices must have
2. We know that the device is located in the first slot of the backplane (0)
3. The device is a Rockwell PLC
4. The PLC runs on firmware 20.13
5. The PLC is a 1768-L43S/B i.e a Compact GuardLogix CPU L43S

Looking later in the capture also allows identifying the communication adapter located in the second slot (see 1) of the backplane (the same one identified earlier in “Get Attribute All” service response):

We are aware of a few endpoints on the networks and identified some models. If we stick to the 10.100.101.157 IP address, we know that it matches a Rockwell Automation/Allen-Bradley Compact GuardLogix (1768-L43S/B) with its communication adapter (1768-ENBT/A). If we miss those initial packets, we would not have much data on these devices. Indeed, as most packets sent afterward are service packets related to the actual PLC state and attributes, we would not get information such as the firmware version which is essential for a vulnerability assessment. The visibility of these industrial systems via passive network monitoring is limited because these controllers don't actively broadcast their presence. They only become more visible when there is network activity initiated by an engineering station or other devices interacting with them. This is a common characteristic in industrial control systems to enhance security and minimize network noise.

The limitation of passive solutions becomes even more obvious when facing encrypted communication protocols such as S7CommPlus used to communicate with Siemens S7-1500 /1200 devices.

Active device queries

Now, let’s go back to focus on our Rockwell chassis example (10.100.101.157). Rockwell proposes a variety of communication modules that support CIP networks (EtherNet/IP, DeviceNet/IP, ControlNet/IP and CompoNet/IP). CIP is a media independent, object-oriented protocol using a multicast producer-consumer communication model.

For our tests, we are using the pycomm3 Python library which helps communicate with Allen-Bradley PLCs using Ethernet/IP. We are now switching from the passive “wait and see” approach to an active discovery:

>>> from pycomm3 import CIPDriver
>>> assets=CIPDriver.discover()

Here is what happens on the wire:

The discover() method broadcasts a UDP “List Identity” message. As mentioned in the ODVA CIP documentation; “the ListIdentity command typically is sent as a broadcast UDP message to tell all EtherNet/IP devices to return a data set with identity information. This command is used by software tools to browse a network”.

First of all, we can see that many more assets show up thanks to this active discovery. Let’s now take a look at our reference backplane ('10.100.101.157'). In the best scenario possible for passive listening, we found two assets, respectively, in slots 0 and 1.

We previously sent an “unregistered” request as we had no prior knowledge of the device’s identities. Let’s now establish a session and actively query 10.100.101.157:

>>> driver=CIPDriver('10.100.101.157')
>>> driver.open() # Establishes a session with the device
True

Even if there is no absolute limit to the number of slots that can be provided in a backplane, we are using a practical limit of 20 for the purpose of this exercise:

>>> for i in range(0,20):
...     try:
...         print(f'slot {i} -> {driver.get_module_info(i)}\n')
...     except:
...         pass
... 
slot 0 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Programmable Logic Controller', 'product_code': 101, 'revision': {'major': 20, 'minor': 13}, 'status': b'p0', 'serial': '604e6055', 'product_name': '1768-L43S/B LOGIX5343SAFETY'}

slot 1 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Communications Adapter', 'product_code': 151, 'revision': {'major': 4, 'minor': 5}, 'status': b'`\x00', 'serial': '6074a847', 'product_name': '1768-ENBT/A'}


slot 5 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Programmable Logic Controller', 'product_code': 103, 'revision': {'major': 20, 'minor': 13}, 'status': b'\x00 ', 'serial': '2d009083', 'product_name': '1768-LSP/B LOGIXSAFETY'}

We found a new asset in slot 5 (1768-LSP/B LOGIXSAFETY). The ‘S’ in the controller 1768-L43S means that it comes with its safety coprocessor that was just spotted. Even if these two come together as a whole (1768-LSP/B has not been added retrospectively in the chassis), only active querying can provide this level of granularity in asset discovery.

This way of enumerating slots is simplistic and a bit naive, but self-explanatory enough to show that we are getting better asset coverage, even against the best passive monitoring scenario, as previously shown. Now we found a new processor in slot 5 which we did not discover using passive monitoring. In addition, techniques like reading backend configuration files or requesting simple network management protocol (SNMP) communities, will provide a more comprehensive view of your assets. Also, some devices embed file transfer protocol (FTP) or web servers which provide interesting information but may require authentication.

Here is an SNMP example using public community for the 1768-ENBT communication adapter:

$ snmpwalk -v1 -c public 10.100.101.157 1.3.6.1.2.1.1.1.0 # SysDescr
iso.3.6.1.2.1.1.1.0 = STRING: "Rockwell Automation 1768-ENBT"

Also, the vendor private management information base (MIB) can give you more detailed information about the requested device.

Let’s provide another example with a 1756-L81E/B ControlLogix (10.100.101.150) which responded to our previous ListIdentity request:

Let’s see if we can simply enumerate the chassis using our previous Python script:

slot 0 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Communications Adapter', 'product_code': 14, 'revision': {'major': 12, 'minor': 5}, 'status': b't\x00', 'serial': '00ceb12f', 'product_name': '1756-DNB/E'}

slot 1 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Communications Adapter', 'product_code': 166, 'revision': {'major': 10, 'minor': 7}, 'status': b'0\x01', 'serial': '00d86889', 'product_name': '1756-EN2T/D'}

slot 2 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Communications Adapter', 'product_code': 7, 'revision': {'major': 11, 'minor': 4}, 'status': b'0\x00', 'serial': '008634cf', 'product_name': '1756-CNB/E 11.004'}

slot 3 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Programmable Logic Controller', 'product_code': 164, 'revision': {'major': 33, 'minor': 12}, 'status': b'`0', 'serial': '00d967f6', 'product_name': '1756-L81E/B'}

slot 4 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Programmable Logic Controller', 'product_code': 92, 'revision': {'major': 30, 'minor': 11}, 'status': b'`0', 'serial': '00c47bca', 'product_name': '1756-L71/B LOGIX5571'}

slot 5 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Programmable Logic Controller', 'product_code': 54, 'revision': {'major': 20, 'minor': 55}, 'status': b'`0', 'serial': '00d7d63d', 'product_name': '1756-L61/B LOGIX5561'}

slot 6 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Communications Adapter', 'product_code': 200, 'revision': {'major': 10, 'minor': 10}, 'status': b'0\x00', 'serial': '00d792a1', 'product_name': '1756-EN2TR/C'}

slot 7 -> {'vendor': 'Rockwell Automation/Allen-Bradley', 'product_type': 'Communications Adapter', 'product_code': 18, 'revision': {'major': 6, 'minor': 3}, 'status': b'4\x01', 'serial': '0056ff36', 'product_name': '1756-DHRIO/D '}

Once again, only active querying is able to map the above-mentioned chassis with this fine granularity. Only relying on passive monitoring will leave an infrastructure at risk with a huge gap in asset discovery, providing an incomplete mapping in the best scenario or more likely no information at all.

Tenable’s active querying is patented (#US-10261489-B2) and communicates with your OT devices in their native language to eliminate blind spots caused by passive network monitoring alone. While actively querying devices, you can get non-disruptive, detailed device information even if it is dormant or if it infrequently connects to your network and, as a result, gather up-to-date, detailed information about every device on your network.

This capacity to obtain granular information is crucial in order to perform a fine-grained vulnerability assessment. Also, it is common to see vendors referring to vulnerable assets thanks to their unique identifiers called serial or order numbers. This information is only accessible when actively querying the devices.

In Tenable OT Security, you can configure and periodically/manually run several active queries types targeting multiple services and protocols for OT as well as IT systems.

Responsive and accurate vulnerability detection

Tenable OT security actively monitors your OT devices and queries them in their native communication protocols. It provides detailed information like specific models, OS and firmware versions, serial numbers and more. This discriminating information is essential to have a fine-grained vulnerability assessment. Indeed, one of the biggest drawbacks in common OT security tools is that customers are facing an extremely high level of false positives and negative alerts:

• false negatives because of a lack of assets discovered; and
• false positives because of inefficient asset fingerprinting, leading to an overly wide and distorted vulnerability detection

Also, responsiveness to the most recent and critical vulnerabilities is key.

Tenable has more than 20 years of experience in developing an extensive library of plugins and is a recognized leader in vulnerability assessment. Plugins are easy to use, highly customizable and updated on a daily basis.

Tenable OT Security is continuously expanding its coverage of industrial assets, both from a protocol support point of view and a vulnerability tracking one. The granularity of the information gathered during asset discovery is key to tightening up vulnerability detection.

As shown in the previous section, multiple sources may be required to obtain the exact asset model and version, but this is only the first step of vulnerability identification. Indeed, in addition to proof checking each vulnerability source, a deep understanding of each classified asset is required to fit vendor specificities, like version nomenclature.

As an example, some Wago or Siemens devices use standard version format with prefixes like FW01 or V1.0. Nonetheless, very specific patterns like R150-V2 (version 2 of the firmware revision 150), or even 5.2(1)SV3(4.1A), including identifiers specific to Cisco’s software lifecycle management, are also expected. Of course, security advisories are not limited to firmware version matching and often include specific models and serial/order numbers, requiring just as much attention to achieve accurate vulnerability detection.

Deep diving into asset discovery and having an accurate understanding of industrial vendors is a game changer for achieving a fine-tuned detection.

Conclusion

Obtaining in-depth visibility of industrial assets is challenging. Several nested devices serving multiple purposes create a complex, sprawling network with a lot of hurdles to tackle. The convergence between IT and OT infrastructure creates more attack paths and increases opportunities for adversaries. Most of the compromises seen in OT follow a lateralization from an IT network. Due to this, these converged environments need greater visibility for full security, which includes active device querying, passive network detection, a thorough inventory of all of your assets, and real-time data on all of your assets and threats. Without it, you'll constantly put your organization in danger due to blind spots.

To learn more about Tenable OT Security, please visit our product page where you can schedule a demo.

¹The industrial internet of things (IIoT) is also referred to as Industry 4.0, which has introduced new internet of things (IoT) devices into OT environments, creating new and increasing risks for OT security. See What is operational technology (OT)? | Tenable

Learn more

• What's New in Tenable OT Security 3.16: Elevating Building Management System Security and User Experience
• Finding Rockwell Automation Allen-Bradley Communication Modules Affected by CVE-2023-3595 and CVE-2023-3596 in OT Environments
• ODVA Technologies' Common Industrial Protocol (CIP)