Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fuzzing halts on expected errors #570

Open
MischaU8 opened this issue Aug 26, 2023 · 7 comments
Open

Fuzzing halts on expected errors #570

MischaU8 opened this issue Aug 26, 2023 · 7 comments

Comments

@MischaU8
Copy link

When an expected error is received, the fuzzing halts. My expected behaviour is that the fuzzing will continue and ignore the error, unless it is actually not received at all.

Minimal example to reproduce this:

$ npx jazzer fuzz-expected-error-bug.js --sync -x IgnoreThisError
Dictionary: 4 entries
INFO: Running with entropic power schedule (0xFF, 100).
INFO: Seed: 4054261925
INFO: Loaded 1 modules   (512 inline 8-bit counters): 512 [0x1280c0000, 0x1280c0200),
INFO: Loaded 1 PC tables (512 PCs): 512 [0x115000000,0x115002000),
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
INFO: A corpus is not provided, starting from an empty corpus
#2	INITED cov: 4 ft: 4 corp: 1/1b exec/s: 0 rss: 132Mb
INFO: Received expected error "IgnoreThisError".

$

contents of fuzz-expected-error-bug.js:

"use strict"
const { FuzzedDataProvider } = require("@jazzer.js/core")

module.exports.fuzz = function(fuzzerInputData) {
    let data = new FuzzedDataProvider(fuzzerInputData)
    let number = data.consumeIntegralInRange(1, 2**35-31)

    if (number > 100) {
        throw new IgnoreThisError("Nothing to see here")
    } else if (number == 25) {
        throw Error("You found it!")
    }
}

class IgnoreThisError extends Error {
    constructor(message) {
      super(message)
      this.name = "IgnoreThisError"
    }
}

Jazzer 1.6.1 on OSX, node v18.17.1.
@bertschneider
Copy link
Contributor

bertschneider commented Aug 29, 2023 •
edited

Hi, thx for reporting this!

expected_errors is used to influence the application exit code, so that external applications can detect expected and unexpected errors, e.g. this comes in handy in CI pipelines and scripts.

Skipping findings, as you suggested, is still on the list of features to implement. It shouldn't be too hard to build this, though. The general idea is to hash a finding's stack trace and and print that as de-duplication token on exit. In subsequent runs it can be used to ignore findings resulting in the same token/hash. Do you want to give it a try?

BTW: Nice blog post you wrote there on Jazzer.js 👍

@harisab2547
Copy link

Yes, the Jazzer js halts on expected error but you can control by your own code to ignore those,
I have made a list of errors to ignore the errors so that the Fuzzer continue without halts.
If you want i can share in the thread

@karfau
Copy link

karfau commented Oct 1, 2023

Good to know:
https://www.llvm.org/docs/LibFuzzer.html#id41

It may be desirable to reject some inputs, i.e. to not add them to the corpus.

For example, when fuzzing an API consisting of parsing and other logic, one may want to allow only those inputs into the corpus that parse successfully.

If the fuzz target returns -1 on a given input, libFuzzer will not add that input top the corpus, regardless of what coverage it triggers.

@bertschneider
Copy link
Contributor

True, but Jazzer.js does not support this, at least for now.

Actually, Jazzer.js uses a patched libFuzzer version where returning -2 from the fuzz target will exit the fuzzing loop so that the application can be shut down orderly. The result of the JS fuzz target is not evaluated on the C++ side.
Looking up usages of RETURN_CONTINUE and RETURN_EXIT (https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/packages/fuzzer/shared/libfuzzer.h#L30) should probably point to all required places, though.

Happy to help you implement this feature 😄

@karfau
Copy link

karfau commented Oct 4, 2023

Well, that's good to know.
Maybe it's worth documenting the differences compared to the official docs on libFuzzer side?

It also took me a while to find the documentation for the output, and I would have loved to pass -ignore_c_rashes flag when running with -fork, but I'm told that the flag is not supported...

I guess there are more differences, and a quick overview about those would be helpful to understand what is possible and what is not.

PS: Even if I would have the time to look into it, I have no clue regarding C/C++ so I don't feel capable of diving into the code base to implement this.

@bertschneider
Copy link
Contributor

No worries, thx for bringing this up! I added it to our list of missing features.

Ideally users would not need to know which or even if an external fuzzing engine is used internally. As libFuzzer has quite a few useful features not directly exposed by Jazzer.js it's possible to pass in fuzzer engine arguments directly. And you're definitely right that documentation is lacking in this regard.

To enable fork mode and ignore crashes you could use something like this:
npx jazzer <fuzz_target> -- -fork=1 -ignore_crashes=1

In general libFuzzer parameters can be passed in after --. Please note that libFuzzer parameters use a single -, whereas Jazzer.js parameters use a --.

@karfau
Copy link

karfau commented Oct 6, 2023

Ah, I was missing the =1 part when passing the argument!
Already did gave uptrying it, that's great news.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@bertschneider @karfau @harisab2547 @MischaU8