After further analysis of the memory leak bug disclosed last week, Cloudflare officials say they haven’t found any instances of customer passwords, credit card data, or health records leaking while the vulnerability was exposed.
The vulnerability, now known as Cloudbleed, has joined the pantheon of Internet-scale bugs to emerge in the last few years, even though the exposure was not quite as wide as some of the other weaknesses, such as Heartbleed. Caused by an issue that arose during the company’s migration from an older HTML parser to a newer one last year, the vulnerability resulted in uninitialized memory leaking in some cases. A researcher from Google’s Project Zero team discovered the problem while he was researching a separate issue last month.
Cloudflare engineers were able to mitigate the vulnerability within a few minutes of Google alerting them to its existence, but then the real work of discovering what had happened and what data may have leaked began. CEO Matthew Prince said Wednesday that the company’s ongoing analysis of the situation has led to the preliminary conclusion that much of the data that leaked was internal Cloudflare data and not sensitive customer information, and also that there is no direct evidence that any attackers exploited the bug.
“The summary is that, while the bug was very bad and had the potential to be much worse, based on our analysis so far: 1) we have found no evidence based on our logs that the bug was maliciously exploited before it was patched; 2) the vast majority of Cloudflare customers had no data leaked; 3) after a review of tens of thousands of pages of leaked data from search engine caches, we have found a large number of instances of leaked internal Cloudflare headers and customer cookies, but we have not found any instances of passwords, credit card numbers, or health records; and 4) our review is ongoing,” Prince said in a post explaining the company’s analysis and conclusions.
Podcast with Nick Sullivan of Cloudflare on the Cloudbleed bug
The bug was only present under limited circumstances on a subset of pages delivered by Cloudflare’s infrastructure, and Prince said that if an attacker had been aware of the vulnerability before it was fixed, he could have set up a system to send a large volume of requests to vulnerable pages. The attacker could then log the responses, which would include the leaked data. Prince said the company’s engineers have spent the last week-plus looking for indications of that kind of activity and have found none.
“The nightmare scenario we have been worried about is if a hacker had been aware of the bug and had been quietly mining data before we were notified by Google’s Project Zero team and were able to patch it. For the last twelve days we’ve been reviewing our logs to see if there’s any evidence to indicate that a hacker was exploiting the bug before it was patched. We’ve found nothing so far to indicate that was the case,” he said.
But that wasn’t the end of the potential for harm. The nature of the vulnerability meant that sensitive data could have been cached by search engines across the web before the issue was fixed. So Cloudflare’s team worked with the search engines to purge their caches of the pages that could have had sensitive customer data on them and then analyzed the information on those cached pages to see what was actually there. They found that more than 67 percent of the data in a given leak was Cloudflare header information, less than one percent contained cookie data.
“Since this is just a sample, it is not correct to conclude that no passwords, credit cards, health records, social security numbers, or customer encryption keys were ever exposed. However, if there was any exposure, based on the data we’ve reviewed, it does not appear to have been widespread. We have also not had any confirmed reports of third parties discovering any of these sensitive data types on any cached pages,” Prince said.
As the post-mortem analysis continues, Prince said Cloudflare has hired Veracode to audit the company’s internal code and look for other unknown vulnerabilities.
Image: Marlis Borger, CC By license.
