-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to configure the corpus for jest in regression mode? #637
Comments
Jazzer.js jest-runner can run in two modes: regression, and fuzzing. Jazzer.js auto-generates the directories for each mode and test. In regression mode for your tests in
Files in these three directories will be used for their corresponding tests. In fuzzing mode, the directory is in the main project directory .cifuzz-corpus, where you put some files already (the first two will be ignored, because they are not associated with any test):
The fuzzing mode also uses inputs from the regression directories of for each test, however regression mode only uses inputs from the regression directory. Also, the first input that the fuzzer tries is an empty input (does not contain any bytes). If you have no files in the regression directory of the corresponding test, this will be the only input that the fuzzer will try. That should explain your screenshot above. |
@oetr thank you for the explanation. I would have expected that the two samples are going to be logged by the test using console.log.
Do you mean I should add a (Is there any documentation/executable example that shows this?) |
Jazzer.js automatically generates all necessary directories for both modes (regression, fuzzing) after you run your jest fuzz tests once. The regression tests are always in some folder relative to where the test resides.
You can check out the example in https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/examples/jest_integration/integration.fuzz.js, where the regression tests are in the subfolder It is somewhat cryptically mentioned in https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/docs/jest-integration.md#fuzzing-mode, but I will add a ticket to improve this part of our documentation. Thanks for reporting this! |
Worked like a charm. Here is my "generic" solution for a folder that contains a single regression testsuite and multiple fuzz targets, which reflects the directory structure that jazzer.js generates and will yell at you for targets without any input files and verifies that each target it called the expected amount of times: 'use strict';
const { describe, expect, test, beforeAll } = require('@jest/globals');
const fs = require('fs');
const path = require('path');
const TARGETS = fs.readdirSync(__dirname).filter((file) => file.endsWith('.target.js'));
TARGETS.forEach((target) => {
describe('', () => {
beforeAll(() => {
const testfiles = fs.readdirSync(path.join(__filename.replace(/\.js$/, ''), target));
expect(testfiles.length).toBeGreaterThan(0);
});
const module = require(path.join(__dirname, target));
test.fuzz(target, (data) => module.fuzz(data));
});
}); (Update: I needed to drop the assertion on the calls made to the spy, it didn't work reliably.) |
I read through https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/docs/jest-integration.md and am also aware about https://github.com/CodeIntelligenceTesting/jazzer.js/blob/main/packages/core/options.ts but I was not able to derive, how to configure those tests where to pick the data from.
Do they have to stay in the top level as
crash-<hash>
files?I would love to point to a separate directory like when passing "corpus" as an argument, so that the test runner picks up the files contained in it and uses those for the regression test.
Is that possible?
I enabled verbose logging and even though the test is being reported as being run, what data is passed to it if no input is being found in regression mode?
Because I found this code:
jazzer.js/packages/jest-runner/corpus.ts
Line 21 in d839e51
I tried to add my
crash-
files to the.cifuzz-corpus
directory, but didn't see anything happening.When running the tests in regression mode I observed that it creates a directory structure inside that folder, which reflects the name of the test file, the describe message and the
it.fuzz
/test.fuzz
messages, so I also added my crash files there, but it didn't change anything.To understand what is being passed to the targets, I added the following to my test suite:
which only leads to the following output:
Here is where you can see all the changes I did so far: xmldom/xmldom#556
(2 commits pushed to that branch at the point of posting this)
The text was updated successfully, but these errors were encountered: