Tenable CIO Patricia Grant and CSO Robert Huber share insights and best practices to help IT and cybersecurity leaders and their teams weather the next cyber crisis of Log4j proportions.
A year ago, the discovery of the Log4Shell vulnerability sent IT and cybersecurity teams scrambling to assess and address their organizations’ exposure to this flaw in the ubiquitous Log4j open source component.
Shifting quickly into “all hands on deck” mode, IT, engineering and cybersecurity teams put in long days and nights, racing to prevent breaches stemming from this easy-to-exploit and extremely severe zero-day vulnerability.
Now, the one-year anniversary of the earth-shaking Log4j event offers an opportunity to take stock and ponder key questions, such as: “What did we learn?” and “Are we ready for the next Log4j-like vulnerability?”
They’re important questions to ponder, especially with the findings about the current state of Log4j remediation that Tenable recently disclosed. After analyzing a representative sampling of telemetry data from its 40,000 customers globally, Tenable found that, as of October 1, 2022:
- 72% of organizations remain vulnerable to Log4Shell.
- 29% of vulnerable assets saw the reintroduction of Log4Shell after full remediation was achieved.
- Nearly one third of North American organizations have fully remediated Log4j (28%), followed by Europe, Middle East and Africa (27%), Asia-Pacific (25%) and Latin America (21%).
In this blog, Tenable CIO Patricia Grant and CSO Robert Huber share their insights, recommendations and expertise for dealing with high-impact vulnerabilities like Log4Shell, including the importance of a detailed playbook, cross-team collaboration and asset visibility.
Communicate and collaborate — constantly and enthusiastically
At the foundation of an effective response to a Log4j-type event is a basic, yet sometimes elusive, principle: A pre-existing good and constant communication among the CIO, the CSO and all other stakeholders involved in protecting the company from cyberthreats.
For example, a key partnership between the CIO and the CSO involves the establishment of protective policies and procedures regarding the secure and compliant use of technology, which helps set a solid security hygiene foundation for managing and reducing cyber risk. The enforcement and detection of policy violations, vulnerabilities, misconfigurations and other flaws should be continuous and automated.
“Having visibility into your assets, with dashboards for reporting progress, is the best way to respond quickly and manage communications between the CIO and CSO on status at all times,” Grant says.
Not only is communication between the CIO and CSO important, but it’s essential for the employees as well. All employees need to understand the importance of security in their company and ensure that they understand and follow all policies, which are there for a reason. Failure to do so could have negative consequences for the whole company.
“It’s important to not only communicate ‘the what’, but also ‘the why’ and the impact of not complying.” Grant says.
Boosting the staff’s security awareness increases end user compliance and cooperation, and reduces the incidence of risky behavior, such as the use of unapproved IT products. When dealing with a high-risk, zero-day vulnerability like Log4j, these “shadow IT” tools represent an extremely dangerous attack vector for an organization.
Get full asset visibility
Another foundational piece that must be in place for successfully dealing with high-impact vulnerabilities is comprehensive asset visibility. Once you’ve understood the nature of the vulnerability and its severity, as well as the implications of getting breached, you must quickly determine where and to what extent you’re impacted.
To do that, it’s critical for IT and cybersecurity to have an actively managed and constantly updated inventory of IT assets in a configuration management database (CMDB) or similar repository, such as an asset data lake fueled by your IT service management (ITSM), vulnerability management (VM) and endpoint detection and response (EDR) solutions.
“Otherwise, it’ll be difficult to swiftly and precisely answer the question of where this vulnerability is in your IT environment,” Huber says. This continuously updated IT asset inventory should contain all hardware and software assets on-premises, in the cloud and in remote locations, with granular details of all their components.
Draft a response playbook
Organizations felt an intense urgency to address Log4Shell as quickly as possible — including detecting where it was in their environment and promptly patching the impacted assets. Based on how that went, you should have gotten a good idea of whether your vulnerability management program can handle a future Log4j-like crisis, Huber says.
Here are five key questions to ask yourself:
- Do you have enough scanners and infrastructure in place to respond?
- Can you scan quickly enough?
- Can you patch immediately and automatically?
- Can your business weather the operational disruptions of a massive and urgent vulnerability assessment and remediation campaign?
- Can you coordinate all of these things quickly?
At Tenable, Huber and Grant deploy a rapid-response (RR) playbook for dealing internally with high-impact vulnerabilities. It outlines concrete steps to take and details the tasks and responsibilities of different teams, including research, security, IT, operations, legal, marketing, product management and the C-level suite.
“This is very similar to an incident response plan, but very specific for high profile vulnerabilities. The IR process could be co-opted for such events as it is usually well understood and agreed upon by all stakeholders,” Huber says.
With this RR playbook in place, Tenable’s goals are to:
- Quickly address and respond to the vulnerability
- Ensure teams involved collaborate and communicate effectively
- Ensure everyone is clear about their responsibilities and expectations
- Ensure that internal and external communications are clear and consistent
Once the RR plan is activated, Tenable goes through the following stages:
- Convene the RR executive team that’ll manage strategic decisions.
- Assemble the RR team with representatives from each team involved.
- Establish the key goals and objectives that must be attained.
- Execute playbook action items.
- Track and document progress and modify strategy as needed.
- With the incident contained, wrap up and shift back to normal operations.
Be ready to respond to queries from customers and partners — quickly!
As you’re scrambling to address the vulnerability, you’ll get bombarded with queries from customers and partners wanting to know whether your organization has been impacted and if your software will need to be patched. It’s a consequence of the greater awareness we all now have about software supply chain risks.
Having answers ready is a must, and a challenge, so Grant and Huber recommend the following:
- If you haven’t already done so, create detailed software bills of materials (SBOMs) for all your software, so that you know all the “ingredients” present in these products.
- Embed security checks and tests early in your software development lifecycle, using tools for dynamic and static application security testing; third-party dependency or software composition analysis; vulnerability management; and container security. This will improve the trust and assurance of the software you release to customers.
Keep close tabs on the security posture of your suppliers
On the flip side, you should have a stringent process for evaluating the cybersecurity preparedness and practices of all your IT providers, and manage that third-party vendor risk so that, for example, your software vendors and managed service providers don’t put you in undue danger of breaches through carelessness on their part.
“You’re only as safe as the weakest link in your vendor supply chain,” Grant says.
Not “if” but “when”
Bottom line: it’s not a question of “if” but rather of “when” IT and cybersecurity leaders will again face a crisis of Log4j proportions. Taking steps such as the ones outlined in this blog will go a long way towards putting your organization in a much better position to successfully respond to the next “all hands on deck” vulnerability.
“Run tabletop exercises for this specific scenario and create some muscle memory and identify opportunities for improvement,” Huber says.
For more information about Log4j, check out the Tenable Log4j resources page, which includes links to FAQs, white papers, blogs, plugins, how-to videos, on-demand webinars and more.
