On February 17, 2017 a Google researcher stumbled onto a situation that some are calling Cloudbleed, where services running on Cloudflare servers were inadvertently causing chunks of uninitialized memory to be mixed with valid data. The Google researcher posted this description on the discovery. The uninitialized memory can contain encryption keys, passwords and other sensitive data. This data leakage is very critical due to the amount of caching found on the internet today. With the widespread caching services, the extent of the leakage may be very hard to determine. Cloudflare reports that the bug has been patched and resolved; you can read more about this bug on the Cloudflare blog.
What does this mean to your company?
As this breach is passive in nature, the cached data has not yet been reported to be exploited. With the risk of passwords, encryption keys and other Personally Identifiable Information (PII) as part of the possible data leak, your company must be able to determine if data has been compromised or not. There are several lists of domain names published on github.com. However, for customers using SecurityCenter Continuous View® (SecurityCenter CV™) with Passive Vulnerability Scanner® (PVS™) and Log Correlation Engine® (LCE®), you can easily track and identify which internal systems are using services running on Cloudflare systems. After identifying the hosts and services used, the security analysts can begin to understand the risk to your organization.
Locating the data
When using PVS and LCE, the best practice is to have the PVS real-time logs sent to LCE for further analysis. As part of the configuration of PVS, there is a section called Realtime Events. In the Realtime Events, there are two settings to enable Log Realtime Events To Realtime Log File and Enable Realtime Event Analysis. These settings enable PVS to log session level events similar to NetFlow. Next, you must set up the syslog settings to send the data to LCE. Once real-time event data is sent to LCE, you will be able to see who is communicating with services using Cloudflare. Additionally, you can install the LCE client on DNS servers, which enables LCE to track DNS queries.
SecurityCenter CV has several types of asset lists that you can use to identify traffic patterns or groups of hosts with similar vulnerabilities or risks. The asset list best suited for detecting Cloudflare is a Watchlist asset. The Watchlist asset is a group of IP addresses that are of interest and need to be monitored, but which may not be local to your environment; for example, Cloudflare IPs. We looked up Cloudflare IP address blocks using American Registry for Internet Numbers (ARIN). To create the asset, you can go to Assets and click Add. Next click on Type Watchlist, and give the asset the name Cloudflare add the following subnets to newly created asset:
- 104.16.0.0/12
- 108.162.192.0/18
- 162.158.0.0/15
- 172.64.0.0/13
- 173.245.48.0/20
- 198.41.128.0/17
- 199.27.128.0/21
Now click on Submit to save the asset. After creating the asset, and before proceeding to Analysis, allow the asset to update.
Locating systems with a possible data leakage
To locate the events that are evidence of hosts using services running Cloudflare, you must first go to Analysis > Events. According to the Cloudflare blog post, the dates of the greatest risk are February 13, 2017 to February 18, 2017. By expanding the filters, you can add in the explicit dates and the Cloudflare Asset. When adding the first date, be sure to set the time to 00:00; this will ensure that the filter starts at the beginning of February 13. Next, for the second date, set the time to 23:59, to ensure that the full day is captured.
The next step is to add the asset as part of the filter; this a two step process. First, click on select filters, and then add the Asset filter. The Asset filter is now available on the left hand side of the screen, and you can click All in the Asset field and enter the name of the Cloudflare asset:
Next click on Apply All to see the events related to Cloudflare. The first view you will see is the List of Event Types; these are the high level summary categories of events. For example, here are several event types that can help determine the risk your network is exposed to:
The web-access shows PVS tracking the type of HTTP calls made, such as web content, JPG files, PDF files, HTTP requests, and several others. Click on web-access, then select Jump to Raw Syslog Events in the upper right hand corner of the screen. Click on the plus sign + next to each log, and you can review the URL related HTTP request parameters. You can then review the details such as the source of the HTTP request and the URL visited. At this point, you must create a list of URLs that are related to your business risk and begin to investigate if your organization is at further risk.
Another great feature of tracking PVS event data with LCE is the ability to historically track vulnerabilities. In the following sample, you can see my lab has a Mac OS X system running a vulnerable browser. In this case, the vulnerability might not increase risk of the Cloudflare breach, but getting a good historic view of vulnerabilities detected by PVS is a great feature when combining PVS and LCE together.
Wrapping up
SecurityCenter CV is a powerful tool when fully implemented, and can aid your investigations when there are large data breaches such as Cloudbleed. By using LCE to track real-time events in PVS, you have a good historic view of vulnerability data and protocol level events. Combining PVS and LCE enables your organization to see the traffic and understand the content of the session. As the context of the Cloudflare traffic is revealed, you can better understand and assess the risk to your organization. Tenable provides our customers with a full-featured threat and vulnerability analysis that far exceeds those of our competition.