Big bad decryption bug in OpenSSL – but no cause for alarm

The well-known and widely-used encryption library OpenSSL released a security patch earlier this week.

Annoyingly for those who like lean, modern, sans serif typefaces, the new version is OpenSSL 1.1.1l, which is tricky to interpret if you use a font in which upper case EYE, lower case ELL and the digit ONE look at all similar.

To spell it out phonetically, you’re after OpenSSL version ONE dot ONE dot ONE LIMA.

(At the time of writing, Naked Security’s official typeface is Flama, a Bauhaus-inspired font family derived from DIN 1451, which itself arose out of early 20th century German railway and road lettering styles. Our lower case ELLs have a neat looking rightwards curl at the bottom to improve their legibility, and ONEs get a classically European look with a crossbar at the bottom and a little leftward flick at the top. But not all typefaces are made that way.)

The bugs

OpenSSL, as its name suggests, is mainly used by network software that uses the TLS protocol (transport layer security), formerly known as SSL (secure sockets layer), to protect data in transit.

Although TLS has now replaced SSL, removing a huge number of cryptographic flaws along the way, many of the popular open source programming libraries that support it, such as OpenSSL, LibreSSL and BoringSSL, have kept old-school product names for the sake of familiarity.

Despite having TLS support as its primary aim, OpenSSL also lets you access the lower-level functions on which TLS itself depends, so you can use the libcrypto part of OpenSSL to do standalone encryption, compute file hashes, verify digital signatures and even do arithmetic with numbers that are thousands of digits long.

There are two bugs patched in the new version:

• CVE-2021-3711: SM2 decryption buffer overflow.
• CVE-2021-3712: Read buffer overruns processing ASN.1 strings.

Strings, long and short

The second of these bugs, CVE-2021-3712, is the less dangerous of the two, and ironically relates to how OpenSSL handles encoded cryptographic keys and certificates.

The raw data inside TLS key and certifcate files is packaged up in a format called DER, short for Distinguished Encoding Rules, which is a form of ASN.1, short for Abstract Syntax Notation version 1, a structured way of representing binary data.

(Note that if you’ve ever looked at TLS keys or certificates, you’ve probably seen something like this:

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE+LXZfjSOTE0cigDmC3Vlbm0VABgl
Zkmp1zbZsiN9ILxqSQy5Krrza94c/eVZORK03gteh9txboKKQOh6LyAftg==
-----END PUBLIC KEY-----

That’s just a topped-and-tailed, base64 encoded version of the raw DER data, used to make the file easier to recognise and less likely to get mangled in transit than a pure binary file.)

Those names are quite a mouthful, but the important part is not the jargon, but the fact that text strings in ASN.1 are stored in a similar way to how they are in programming languages like Pascal, namely with a length field followed by exactly that much data.

In C, however, strings are stored without any length field: you just get the raw text data, ended with a zero (NUL) byte.

That makes C strings much simpler to use, but it can be annoying for three reasons.

Firstly, it means you can never be sure how long a string is until you traverse the whole thing to find out where the NUL byte is; secondly, you can’t have a NUL byte in the middle of a string, even if you want to; and thirdly, if the final NUL byte gets left out, then copying or printing out a string could go on and on for ages, and include way more information than you intended, assuming that the unterminated string is followed by a large block of non-zero data.

So, ASN.1 gives you structure and control, while C gives you simplicity and speed.

To give you the best of both worlds, OpenSSL always adds a NUL byte to its ASN.1 strings, even though this is not necessary.

This means you can access those strings via OpenSSL’s special ASN.1 functions, as usual, but also safely read them out directly from C, for example if you want to print out a message that contains the relevant string.

Actually (and this is the problem), that’s not quite true.

You can can safely treat OpenSSL’s ASN.1 strings as C strings, but only if they were generated by OpenSSL’s special “always add the NUL byte” functions, otherwise you could end up with an unterminated string, and all the problems that can cause.

If you construct the ASN.1 data yourself, maliciously or otherwise, you don’t need to include those pesky NUL bytes, and the resulting strings will not be safe to use directly from C – you will need to use OpenSSL’s special access functions all the time.

Unfortunately, a few of OpenSSL’s own functions were found to be taking shortcuts and relying on directly accessing ASN.1 strings from C, even when they couldn’t be sure that the original data had been created with those all-important NUL bytes tacked on the end.

As a result, with clever shenanigans, it might be possible for an attacker to trick OpenSSL into printing out data that goes beyond the end of the memory buffer.

This sort of read buffer overflow could lead to to data leaks, where private data accidentally gets revealed in output such as a log file or a system message.

Alternatively, read overflows could lead to memory access errors, where OpenSSL reads so much extra data beyond the missing NUL that an access violation occurs, leading to a software crash.

Data breached due to a read overflow is exactly what happened in the infamous Heartbleed bug, where code that was supposed to send a reply containing just a few bytes from memory inadvertently copied up to 64Kbytes every time instead.

This leaked out whatever else was adjacent in memory at the time, possibly including passwords or decryption keys that just happened to be nearby.

Decrypting too much

The more serious bug of the two, CVE-2021-3711, also involves a buffer overflow, but this time it’s a write overflow, making it much more dangerous.

If you can write past the end of an officially allocated block of memory and modify data that controls some other part of a program, then you may be able to manipulate the behaviour of the program in the future.

For example, you may be able to trick it into thinking that something succeeded (or failed) when it didn’t, or even to take over the flow of program execution entirely.

Using booby trapped data to take over a running program is known as RCE, short for remote code execution, which means exactly what it says: someone else, perhaps even on the other side of the world, gets to control your computer without so much as a popup dialog or a warning.

The CVE-2021-3711 bug relies on a common programming idiom used in software code that generates output data.

That idiom involves using the same data output function twice in succession: first, you run the function but say “don’t actually generate the data, just tell me how much there will be when I do it for real”; then, after setting aside a buffer of the right size, you run the function again to produce the actual data.

That way, in theory, you can reliably avoid buffer overflows by making sure that you have enough memory space before you start.

Except that in one specific case in OpenSSL, namely when using the Chinese government’s cryptographic algorithm known as ShangMi (SM), the software may end up telling you that you’ll need a buffer size up to 62 bytes too small.

That means that booby-trapped encrypted data sent into for decryption could trigger a significant, writable buffer overflow inside OpenSSL.

In particular, it looks as though the buggy code could end up getting called if OpenSSL is asked to set up a TLS connection using the ShangMi ciphersuite and then to validate the web certificate presented by the other end during the TLS handshake.

So far, we’re not aware of any working exploits using this vulnerability.

With a bit of luck

The good news here is that official TLS support for ShangMi was only introduced in RFC 8998, dated March 2021, so it’s a newcomer to the world’s cryptographic stable.

So, although OpenSSL includes implementations of the SM algorithms (SM2 for key agreement and digital signatures, SM3 for hashing, and SM4 for block encryption)…

…it doesn’t yet include the code needed to allow you to choose these algorithms as a ciphersuite for use in TLS connections.

You can’t ask your TLS client code to request a ShangMi connection to someone else’s server, as far as we can see; and you can’t get your TLS server code to accept a ShangMi connection from someone else’s client.

So the bug is in there, down in the low-level OpenSSL libcrypto code, but if you use OpenSSL at the TLS level to make or accept secure connections, we don’t think you can open up a session in which the buggy code could be triggered.

In our opinion, that greatly reduces the likelihood of criminals abusing this flaw to implant malware on your laptop, for example by luring you to a booby-trapped website and presenting you with a rogue certificate during connection setup.

What to do?

• Upgrade to OpenSSL 1.1.1l if you can. Although most software on Windows, Mac, iOS and Android will not be using OpenSSL, because those platforms have their own alternative TLS implementations, some software may include an OpenSSL build of its own and will need updating independently. If in doubt, consult your vendor. Most Linux distros will have a system-wide version of OpenSSL, so check with your distro for an update. (Note: Firefox doesn’t use OpenSSL on any platforms.)
• Consider rebuilding OpenSSL without ShangMi support if you can’t upgrade. Passing the options nosm2 nosm3 nosm4 to the OpenSSL config script before rebuilding will do the trick. Given that ShangiMi can’t yet be selected for use in TLS connections, you should find this to be a non-disruptive change.
• If you’re a programmer, always assume the worst about data. Never assume that data you are called upon to deconstruct was created by the matching construction functions that you carefully coded into your own library. Crafting booby-trapped data packets that don’t look the way you expect (and weren’t covered in your own testing) is what a lot of cybersecurity researchers do for a living. Unfortunately, not all of them work for the Good Guys.