How to Write Fuzz Targets for Java Applications

In this article, Fabian Meumertzheim, Senior Software Engineer at Code Intelligence, gives a brief introduction on how to write fuzz targets with the open-source Java fuzzer Jazzer.

What Is Jazzer?

Jazzer is an open-source fuzzing engine for the Java Virtual Machine (JVM). With Jazzer, developers can increase their test coverage to find edge cases and avoid software bugs more effectively. 

We built Jazzer based on popular and proven tools, including the libFuzzer fuzzing engine and JaCoCo for coverage instrumentation. Like most modern fuzzers, Jazzer is a coverage-guided and in-process fuzzer, which means that it runs Java targets in its own JVM instance while monitoring the reached code paths. This allows developers to test their software at high speed. Since Jazzer operates directly on the JVM bytecode, this is possible even without access to the source code.

libFuzzer vs. Jazzer

Jazzer has all the advanced libFuzzer features and ports them to Java while also adding some additional Java smarts, like a “keep going” mode. This mode allows developers to quickly skip uninteresting exceptions in favor of more interesting bugs and vulnerabilities.

Tried and Tested libFuzzer Features:

• Minimization with deduplication
• Parallel fuzzing in fork mode
• Value profiling

Additional Java Features:

• Pure Java reproducers
• Keep going mode
• Method hooking framework

With Jazzer, we trigger a lot of bugs. But as a Java developer, you might want to debug the code without all the fuzzer "overhead". For this, we create simple java classes with a main function triggering the malicious inputs, so-called reproducers. 

Method hooking is an advanced feature that can be used to write bug detectors and sanitizers (for more information on Method hooking, have a look at our GitHub documentation.)

How to Build a Fuzz Target With Jazzer?

Below, you can see a small but challenging example application. The application consumes a String and two longs. First, it encodes the String using Base64 and checks whether it equals a predefined String. (SmF6emVy is the Base64 encoding of “Jazzer”). Only if the Base64 encoding matches does the program continue. Next, the application uses a simple XOR encryption with a static key on the two longs and again compares them to predefined values. If both match, the program continues and reaches the planted bug. While this is a simple example for humans to solve, fuzzers can easily struggle with this kind of code since very specific “magic” values need to be found to pass the checks. In this case, the fuzzer also needs to “break” the XOR encryption.

class ApplicationUnderTest {
    private static String base64Encode(String input) {
        return Base64.getEncoder().encodeToString(input.getBytes());
    }

    // insecure encrypt with xor and static key
    private static long insecureEncrypt(long input) {
        long key = 0xefe4eb93215cb6b0L;
        return input ^ key;
    }

    public static void consume(String str, long input1, long input2) {
        if (base64Encode(str).equals("SmF6emVy")) { // base64Encode("Jazzer")
            // To get here, the fuzzer has to guess the expected input byte by byte with value profile
            if (insecureEncrypt(input1) == 0x9fc48ee64d3dc090L) { // insecureEncrypt(Long.toString("value"))
                // To get here, the fuzzer has to guess the expected input bitwise with value profile
                if (insecureEncrypt(input2) == 0x888a82ff483ad9c2L) { // insecureEncrypt(Long.toString("profile"))
                    // To get here, the fuzzer needs fake program counters and value profile
                    throw new FuzzerSecurityIssue("You found a Bug");
                }
            }
        }
    }
}

Jazzer Makes It Look Easy

Jazzer, however, can solve this problem with very little effort. Below you see the Jazzer fuzz target, which is basically just a static fuzzerTestOneInput function. The function takes a FuzzedDataProvider, a helper object for splitting the raw fuzzer input into useful primitive types, and gets the String and two longs required by the sample application. Then all it does is call the application with that data. It is as simple as that. 

public class ExampleValueProfileFuzzer {
    // consume data from fuzzer and give it to function under test
    public static void fuzzerTestOneInput(FuzzedDataProvider data) {
        Sting str = data.comsumeString(100); // get a string with max_len 100
        long input1 = data.consumeLong( );
        long input2 = data.consumeLong( );
        // call function under test with fuzz data
        ApplicationUnderTest.consume(str, input1, input2);
    }
}

If you run Jazzer, it will only take about five seconds to crash this target. After about 2.5 million runs with over 600 000 runs per second, Jazzer will have found the “magic” values that get it past the Base64 encoding and XOR encryption and trigger the bug. The reason Jazzer can get past these hurdles so quickly is that like libFuzzer, Jazzer uses value profiling to guide the fuzzer past these kinds of checks much more efficiently than simply hoping to stumble on the exact value by chance. For more on this see https://llvm.org/docs/LibFuzzer.html#value-profile and https://github.com/CodeIntelligenceTesting/jazzer/#value-profile.

$ ./jazzer --cp=. --target_class=ExampleFuzzer -use_value_profile=1
...
#2426817 REDUCE cov: 12 ft: 324 corp: 291/5492b lim: 4096 exec/s: 606704 rss: 312Mb L: 22/27 MS: 1 EraseBytes-
#2477313 REDUCE cov: 12 ft: 324 corp: 291/5490b lim: 4096 exec/s: 619328 rss: 316Mb L: 22/27 MS: 1 EraseBytes-
#2506214 REDUCE cov: 12 ft: 324 corp: 291/5489b lim: 4096 exec/s: 501242 rss: 319Mb L: 23/27 MS: 1 EraseBytes-
#2561055 REDUCE cov: 12 ft: 324 corp: 291/5488b lim: 4096 exec/s: 512211 rss: 324Mb L: 22/27 MS: 1 EraseBytes-

== Java Exeption: java.lang.IllegalStateExeption: not reached
at ExampleFuzzer.fuzzerTestOneInput(ExampleFuzzer.java:17)
DEDUP_TOKEN: 985e7866de639615
== libfuzzer crashing input ==
MS: 1 ChangeBinInt-; base unit: 17d76cac0d2d25c007fdffd6758a992a4fe919f
0x4a, 0x61, 0x7a, 0x65, 0x72, 0x20, 0x76, 0x61, 0x6c, 0x75, 0x65, 0x20, 0x70, 0x72, 0x6f, 0x66, 0x69, 0x6c, 0x69, 0x6e, 0x67,
Jazzer value profiling
artifact_prefix='/tmp/'; Test unit written to /tmp/crash-293dc3aec0ffd72fbdf63dce2a853976a7879202
Base64: SmF6emVyIHZhbHVlIHByb2ZpbGluZw==
reproducer_path='/tmp'; Java reproducer written to /tmp/crash-293dc3aec0ffd72fbdf63dce2a853976a7879202.java

Why Fuzz Memory-Safe Languages Like Java?

Among developers, it is a common misconception, that fuzz testing is only suitable for detecting memory corruption bugs. And since Java is a memory safe language, there is no need to fuzz Java. 

However, fuzzers can find many more bugs than only memory corruptions. It’s just that languages like C and C++ have so many memory bugs with high impact that they get all the attention. Indeed, fuzzers can find any type of bug you care to specify. Simple yet common examples are uncaught exceptions and memory leaks that crash services or applications. Infinite loops that can be used in DoS attacks can also be found. But more interestingly, anything that can be expressed as an assertion can be found quite simply, e.g. multiple implementations or libraries that should produce the same output for a given input but diverge in edge cases. 

It is also possible to find SQL injections or even remote code execution bugs. So in our view, there is no question that memory-safe languages need to be fuzzed since all languages are susceptible to functional bugs as well as severe security vulnerabilities.

Common Java Vulnerabilities That Can Be Found With Fuzzing

Functional Bugs

• Uncaught exceptions
• Assertions
• Inconsistent Implementations
  (differential fuzzing)

Security issues

• Infinite Loops
• OutOfMemoryError
• SQL injections
• XSS (Framework, Sanitizers, …)
• RCE (Jakarta, Serialization, …)
• ...

Use Case: Fuzzing a Popular Java Library

Let's have a look at a real bug we found with Jazzer. In this particular use case, we will talk about the JSON sanitizer, which is a popular Java library developed at Google and maintained by the OWASP Foundation. You can feed the JSON-sanitizer arbitrary bytes and strings while it ensures that the output is always valid JSON. The JSON Sanitizer guarantees that the output will never contain certain substrings that, due to the nature of how HTML is parsed, might mess up your scripts or even cause XSS. Thus, after running untrusted input through the sanitizer, you should be able to safely embed it into a script block in your HTML, like in the orange box below. 

JSON-sanitizer: example of an embedded script.

We set this up with a property-based fuzzing approach in Jazzer. We consume a string and pass it through the sanitizer. Then, we assert that the “safe” JSON that comes out of the sanitizer does not contain the specified substring. Because if it does, we would be vulnerable to cross-site scripting.

Property-based fuzz target. See full gist.

XSS in JSON Sanitizer (CVE-2021-23899)

With this fuzz target, we found a vulnerability in the JSON sanitizer, which is not hard to trigger. By “escaping” the first letter of an HTML tag in your JSON string, you can pass it through the sanitizer, and it will output the original tag. By the nature of HTML, this closing script tag, even if contained in a JavaScript string literal, will close the script block. You then start a new one, you insert an alert, and you get a cross-site scripting exploit.

XXS-Bug in JSON Sanitizer (CVE-2021-23899).

Long story short: Fuzzing allowed us to find a high-severity security vulnerability in this popular and well-tested Java project with very little effort. If you want to have a closer look at Jazzer, the open-source Java fuzzer we used to find this bug, check out its GitHub repo.

Jazzer Findings

Since Jazzer's release in mid-February 2021, we have already found over 100 bugs in more than 20 open-source projects that we fuzzed. 8 out of the 50 bugs were security issues, 5 of which were found in stable releases and thus assigned CVEs. (See full trophy list).

• >100 bugs found
• >20 OSS projects fuzzed
• 8 Security Issues
• 5 CVEs

I hope this post gave you a good first impression of how easy it is to find bugs in Java applications with Jazzer. If you want to learn more about Jazzer, I also recommend the article on “Engineering Jazzer.” Or just check out Jazzer on GitHub.

If you find a bug with it, I would be happy to hear from you (once it’s public) on Twitter (@fhenneke).

About Fabian Meumertzheim

Fabian Meumertzheim is a Senior Software Engineer at Code Intelligence. He maintains and contributed to multiple open-source projects, such as Chromium, Jazzer, systemd, and Android Password Store, all with the aim of making security unobtrusive and ubiquitous.