Mozilla patches critical “BigSig” cryptographic bug: Here’s how to track it down and fix it

Renowned bug-hunter Tavis Ormandy of Google’s Project Zero team recently found a critical security flaw in Mozilla’s cryptographic code.

Many software vendors rely on third-party open source cryptographic tools, such as OpenSSL, or simply hook up with the cryptographic libraries built into the operating system itself, such as Microsoft’s Secure Channel (Schannel) on Windows or Apple’s Secure Transport on macOS and iOS.

But Mozilla has always used its own cryptographic library, known as NSS, short for Network Security Services, instead of relying on third-party or system-level code.

Ironically, this bug is exposed when affected applications set out to test the cryptographic veracity of digital signatures provided by the senders of content such as emails, PDF documents or web pages.

In other words, the very act of protecting you, by checking up front whether a user or website you’re dealing with is an imposter…

…could, in theory, lead to you getting hacked by said user or website.

As Ormandy shows in his bug report, it’s trivial to crash an application outright by exploiting this bug, and not significantly more difficult to perform what you might call a “controlled crash”, which can typically be wrangled into an RCE, short for remote code execution.

The vulnerability is officially known as CVE-2021-43527, but Ormandy has jokingly dubbed it BigSig, because it involves a buffer overflow provoked by submitting a digital signature signed with a cryptographic key that is bigger than the largest key NSS is programmed to expect.

Buffer overflow

A buffer overflow is triggered when a memory area that only has space for X bytes is inadvertently stuffed with Y bytes of data, where Y > X.

Those superfluous extra (Y-X) bytes of “overflow” typically end up overwriting an adjacent block of memory that is already in use for something else, like a surfeit of ill-behaved guests at a hotel room party who end up spilling out into the corridor, barging into neighouring rooms, and generally making a nuisance of themselves.

Typically, this sort of memory corruption causes the vulnerable application to veer off course into some unchartered and unknown memory region where the operating system has no choice but to shut it down right away, causing a simple crash.

But in an RCE, the attackers orchestrate the crash in such a way as to misdirect the application into code they’ve supplied themselves.

An RCE is like a rogue hotel partygoer who not only barges into your room and creates a disturbance that wakes you up, but also deliberately takes advantage of your temporary confusion by stealing your laptop and your wallet under cover of pretending to apologise while you chase them out.

The bad news is that any application that uses the NSS library could be affected by this bug, including most Mozilla apps and several other popular open source programs.

Mozilla explicitly lists the following as impacted:

• Thunderbird, Mozilla’s own email client.
• LibreOffice, a popular free alternative to Microsoft Office.
• Evolution, an open source calendaring app.
• Evince, a popular multi-format document viewer for PDFs and images.

The good news, if you like to think of it that way, is that this bug can’t be triggered in Firefox, so Mozilla’s popular browser is not affected.

Of course, there many be other apps that are vulnerable too – for example, we’re not sure whether the still-active Seamonkey project, which is essentially a Firefox-like browser and a Thunderbird-like email client packaged into a single app, is at risk.

What happened?

The bug is down to code that made the infamous, and so often dangerous, assumption that “this is so unlikely that it it almost certain never to happen, therefore it will never happen, therefore there is no need to check if it has”.

When verifying a digital signature, NSS allocates a chunk of memory to store all the data relevant to the calculations, including the cryptographic public key required for the validation.

The space reserved for the public key is chosen by working out the size of the largest possible DSA key supported by NSS, the largest possible Elliptic Curve (EC) key supported by NSS, and the largest RSA key, and then using the largest of those values to ensure a buffer that is “always big enough”.

RSA keys are notoriously much larger that those of other cryptographic algorithms (this is one reason why EC cryptography is taking over from RSA), typically reaching 2048 or even 4096 bits, instead of the 256 or 512 bits typically required for EC keys.

But RSA keys bigger than 4096 bits are astonishingly rare, not only because they would be much larger than is strictly needed to resist today’s cracking tools, but also because they’re much slower to create and use than smaller keys, even on fast computers.

We’ve never seen, or even heard of, RSA keys of 16384 bits in real-life use, given that they’re typically between 500 and 1000 times slower to generate than 2048 bit keys, which are still currently considered acceptably large to resist attack.

Indeed, the public key buffer allocated for NSS signature verification is 16384 bits long, a size that ought to be more than enough for many years to come…

…and the code that copies an incoming public key into that buffer therefore assumes that no one would go to the trouble of generating a larger RSA key, so it doesn’t bother checking that the key it just received actually fits.

The bug fix was to add in the size-checking code that ought to have been there all along.

What to do?

• Update NSS. Many Linux distros will have a central copy of the NSS library, but some installed apps may include and use their own versions of the library. You can search for the file libnss3.so to find how many NSS instances are on your computer. Windows apps that use NSS will typically include their own versions; search for NSS3.DLL. You need version 3.73 or later, or 3.68.1 ESR if you are using the extended support release. For advice on how to locate any NSS library files on your computer, and how to check what version you have, see below.
• Never skimp on error checking. Just because most people won’t generate huge cryptographic keys doesn’t mean that no one will, whether they do so by accident (which in this case would cause a Denial of Service attack by crashing your app) or by design (in order to hack into your computer on purpose).

TIPS FOR FINDING AND VERSIONING NSS FILES

On Linux, you can search for copies of the NSS library code with the find command. The output from our system is shown as an example.

We have Firefox, Tor and LibreOffice installed, so we conclude from this output that Firefox and Tor have their own NSS library copies, while LibreOffice is relying on the one provided by our distro in /usr/lib64:

   $ find / -type f -name 'libnss3.so' 2>/dev/null
   /usr/lib64/libnss3.so
   /opt/firefox/libnss3.so
   /opt/tor-browser_en-US/Browser/libnss3.so

On Windows, try the DIR command shown below, from a regular command prompt window (i.e. run CMD.EXE, not PowerShell).

We have installed Firefox and LibreOffice, both of which contain their own copy of the NSS3 library file, and will therefore need updating via their own download sources. Remember that Firefox is not affected by this bug, but LibreOffice is.

   C:\Users\duck> DIR C:\NSS3.DLL /S
   [. . .]
    Directory of c:\Program Files\LibreOffice\program
   19/11/2021  11:18         1,089,680 nss3.dll
                  1 File(s)      1,089,680 bytes

    Directory of c:\Program Files\Mozilla Firefox
   19/11/2021  15:31         2,186,168 nss3.dll
                  1 File(s)      2,186,168 bytes

        Total Files Listed:
                  2 File(s)      3,275,848 bytes
   [. . .]

Identifying the internal version numbers of the NSS files that turn up in tour search can be tricky, given that the only reliable way to do so is to load the library and ask it to report on itself.

On Linux

The code below worked for us on Linux. Save as nsschk.c, compile with gcc -o nsschk nsschk.c -ldl, and run ./nsschk with the NSS library file you wish to check on the command line:

#include <stdio.h>
#include <stdlib.h>
#include <dlfcn.h>

void bail(char *msg) {
   fprintf(stderr,"%s\n",msg);
   exit(1);
}
   
int main(int argc, char **argv) {
   /* Use the command argument as the NSS library name,  */
   /* otherwise pick a sensible default for your distro. */
   char *libname = argc>1 ? argv[1] : "/usr/lib64/libnss3.so";
   printf("Using library file: %s\n",libname);

   void *nsslib  = dlopen(libname,RTLD_LAZY);
   if (nsslib == NULL) { bail("Can't dlopen() that file"); }

   int   (*initfn)(char *dir) = dlsym(nsslib,"NSS_NoDB_Init");
   char *(*getvfn)(void)      = dlsym(nsslib,"NSS_GetVersion");

   if (initfn == NULL) { bail("Can't find NSS_NoDB_Init function"); }
   if (getvfn == NULL) { bail("Can't find NSS_GetVersion function"); }
 
   if ((*initfn)(".") != 0) { bail("Failed to initialise NSS"); }
 
   printf("NSS Version: %s\n",(*getvfn)());
   return 0;
}

Our NSS files (see above) showed up as follows:

$ ./nsschk 
Using library file: /usr/lib64/libnss3.so
NSS Version: 3.73

$ ./nsschk /opt/firefox/libnss3.so
Using library file: /opt/firefox/libnss3.so
NSS Version: 3.71

$ ./nsschk /opt/tor-browser_en-US/Browser/libnss3.so
Using library file: /opt/tor-browser_en-US/Browser/libnss3.so
NSS Version: 3.68

Our distro-managed version, as used by the vulnerable LibreOffice, is up to date. Firefox and Tor will presumably be updated soon by Mozilla and the Tor Project respectively, but as they are both apparently immune to this bug, we consider them safe.

On macOS

On a Mac, you can use the same code, but you will explicitly need to tell macOS what directory to use for the NSS library files, or change the current directory to the location of the libnss3 file first. Also, search for both libnss3.so and libnss3.dylib, because both extensions are used in macOS builds.

On our test Mac, for example, we searched the /Applications folder for NSS libraries:

   $ find /Applications -type f -name 'libnss3.*'
   /Applications/Firefox.app/Contents/MacOS/libnss3.dylib
   /Applications/LibreOffice.app/Contents/Frameworks/libnss3.dylib
   /Applications/Thunderbird.app/Contents/MacOS/libnss3.dylib
   /Applications/TorBrowser.app/Contents/MacOS/libnss3.dylib

   $ DYLD_LIBRARY_PATH=/Applications/Firefox.app/Contents/MacOS ./nsschk libnss3.dylib
   Using library file: libnss3.dylib
   NSS Version: 3.71

   $ DYLD_LIBRARY_PATH=/Applications/Thunderbird.app/Contents/MacOS ./nsschk libnss3.dylib
   Using library file: libnss3.dylib
   NSS Version: 3.68

   $ DYLD_LIBRARY_PATH=/Applications/TorBrowser.app/Contents/MacOS ./nsschk libnss3.dylib
   Using library file: libnss3.dylib
   NSS Version: 3.53.1

   $ DYLD_LIBRARY_PATH=/Applications/LibreOffice.app/Contents/Frameworks ./nsschk libnss3.dylib
   Using library file: libnss3.dylib
   NSS Version: 3.55

On Windows

A few modifications produced code that worked for us on Windows. To ensure that Windows finds all the additional DLLs that the NSS3.DLL library needs, change directory to where the NSS3.DLL version resides, and run the NSSCHK.EXE command in that directory.

#include <windows.h>
#include <stdio.h>
#include <stdlib.h>

void bail(char *msg) {
   fprintf(stderr,"%s\n",msg);
   exit(1);
}
   
int main(int argc, char **argv) {
   /* On Windows, we look for NSS3.DLL in the current    */
   /* directory only, to help ensure we find its friends */
   char *libname = "./NSS3.DLL";
   printf("Using library file: %s\n",libname);

   HMODULE nsslib = LoadLibrary(libname);
   if (nsslib == NULL) { 
      fprintf(stderr,"Error: %d\n",GetLastError());
      bail("LoadLibrary() failed on that file");
   }

   int   (*initfn)(char *dir) = GetProcAddress(nsslib,"NSS_NoDB_Init");
   char *(*getvfn)(void)      = GetProcAddress(nsslib,"NSS_GetVersion");

   if (initfn == NULL) { bail("Can't find NSS_NoDB_Init() function"); }
   if (getvfn == NULL) { bail("Can't find NSS_GetVersion() function"); }
 
   if ((*initfn)(".") != 0) { bail("Failed to initialise NSS"); }
 
   printf("NSS Version: %s\n",(*getvfn)());
   return 0;
}

Our results were as follows:

C:\Users\duck>cd "\Program Files\Mozilla Firefox"

C:\Program Files\Mozilla Firefox>\Users\duck\NSSCHK.EXE
Using library file: ./NSS3.DLL
NSS Version: 3.71

C:\Program Files\Mozilla Firefox>cd "\Program Files\LibreOffice\program"

C:\Program Files\LibreOffice\program>\Users\duck\NSSCHK.EXE
Using library file: ./NSS3.DLL
NSS Version: 3.55

We infer from the output above that LibreOffice on Windows is currently vulnerable (we downloaded the latest version to do this test), so watch out for an update notification and grab the new version as soon as a patched build is avilable.

Go to the Options > LibreOffice > Online Update dialog and click [Check Now] to see if a new version is available.

Update. Shortly after writing this article, LibreOffice 7.2.4 was released. This includes an NSS3.DLL file dated 2012-12-03 that our code above (and the Explorer Properties dialog) reports as NSS 3.73.

You can also right-click on the NSS3.DLL file in Windows Explorer and choose Properties > Details, but the version string seems to depend on how the application package was built, so it may not reveal the actual NSS version number.

For example, on our Windows computer, the NSS3.DLL delivered as part of the Firefox app was labelled with the top-level Firefox version details; the LibreOffice DLL revealed the NSS-specific version string: