Google’s Halloween lock-out caused by false positive

Who is in charge of files created and stored on Google Docs and Drive?

Most people assume it’s the user or team sharing them but an incident affecting these services on Halloween has reminded everyone that there is always a superuser with absolute power sitting above this – Google itself.

On that particular day, a portion of Docs users started finding themselves blocked from opening or editing specific documents. Many reported seeing the following message:

This item has been flagged as inappropriate and can no longer be shared.

Except the files were wholly innocent of the charge, something that was quickly pointed out to Google using the preferred medium of modern complaint, Twitter. A few hours later, access to the files was restored.

All back to normal? Not exactly.

On Friday, Google offered an official explanation for what went wrong:

A short-lived bug that incorrectly flagged some files as violating our terms of service (TOS). [This] caused the Google Docs and Drive services to misinterpret the response from these protection systems and erroneously mark some files as TOS violations, thus causing access denials for users of those files.

What Google is saying is that its “unparalleled automatic, preventive security precautions … using both static and dynamic antivirus techniques” suffered what is known in the trade as a false positive.

This happens when a security system incorrectly flags something as suspect that isn’t, a phenomenon affecting all systems from time to time.

While not fun they’re still less worrisome than a false negative, which happens when a genuinely malicious file slips through unnoticed.

Nonetheless, the incident makes it clear that every time a user creates a file on Drive (which is where Docs files are stored), there is a possibility that it might at some point be scanned by Google’s security software to decide whether it’s “inappropriate” or not.

Drive has been widely abused to host malicious (boobytrapped) files, command and control and even crude phishing attacks, so you can understand why Google might want to do such a thing.

The deeper issue is how this is done and whether it in any way compromises privacy over and above the implicit fact (as stated in the terms and conditions) that Google can be legally compelled to hand files over to law enforcement if presented with a court order.

On the basis of Google’s policies it seems unlikely to me that the system reads the contents of files or scans each individually as it is created and used. Rather, periodic scans are run on groups of files as a way of spotting patterns that indicate something suspicious is afoot.

We have no way of knowing how well this system spots malice, but we can say from the rarity of events like this, where large numbers of users are locked out, that disruptive false positives are rare.

It’s possible individual users can protect themselves against this kind of glitch by mirroring Drive files to a local machine and working on those offline. This definitely won’t work for G Suite (formerly Apps for Work) files shared across multiple users and hosted online, however.

The lesson from the Halloween lock-out remains that while content sitting on Drive or created through Docs might belong to the user, the service itself is always Google’s domain. If only more people read the T&Cs.