Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/run-all-tests-pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,9 @@ jobs:
- name: Copy Bazel log
if: always()
shell: bash
run: cp "$(readlink bazel-out)"/../../../java.log* .
run: |
cp "$(readlink bazel-out)"/../../../java.log* .
cp -r selffuzz/src/test/resources/.corpus ./corpus_backup

- name: Upload test logs
if: always()
Expand All @@ -81,3 +83,4 @@ jobs:
path: |
bazel-testlogs*/**/test.log
java.log*
corpus_backup/**
157 changes: 157 additions & 0 deletions docs/mutation-framework.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,163 @@ class SimpleClassFuzzTests {
}
```

## @ValuePool: Guide fuzzing with custom values

The `@ValuePool` annotation lets you provide concrete example values that Jazzer's mutators will use when generating test inputs.
This helps guide fuzzing toward realistic or edge-case values relevant to your application.

### Basic Usage

You can apply `@ValuePool` in two places:
- **On method parameters** - values apply only to that parameter's type
- **On the test method itself** - values propagate to all matching types across all parameters

**Example:**
```java
@FuzzTest
void fuzzTest(@ValuePool(value = {"mySupplier"}) Map foo) {
// Strings from mySupplier feed the Map's String mutator
// Integers from mySupplier feed the Map's Integer mutator
}

@FuzzTest
@ValuePool(value = {"mySupplier"})
void anotherFuzzTest(Map foo, String bar) {
// Values propagate to ALL matching types:
// - String mutator for Map keys in 'foo'
// - String mutator for 'bar'
// - Integer mutator for Map values in 'foo'
}

static Stream mySupplier() {
return Stream.of("example1", "example2", "example3", 1232187321, -182371);
}
```

### How Type Matching Works

Jazzer automatically routes values to mutators based on type:
- Strings in your value pool → String mutators
- Integers in your value pool → Integer mutators
- Byte arrays in your value pool → byte[] mutators

**Type propagation happens recursively by default**, so a `@ValuePool` on a `Map<String, Integer>` will feed both the String mutator (for keys) and Integer mutator (for values).

---

### Supplying Values: Two Mechanisms

#### 1. Supplier Methods (`value` field)

Provide the names of static methods that return `Stream<?>`:
```java
@ValuePool(value = {"mySupplier", "anotherSupplier"})
```

**Requirements:**
- Methods must be `static`
- Must return `Stream<?>`
- Can contain mixed types (Jazzer routes by type automatically)

#### 2. File Patterns (`files` field)

Load files as `byte[]` arrays using glob patterns:
```java
@ValuePool(files = {"*.jpeg"}) // All JPEGs in working dir
@ValuePool(files = {"**.xml"}) // All XMLs recursively
@ValuePool(files = {"/absolute/path/**"}) // All files from absolute path
@ValuePool(files = {"*.jpg", "**.png"}) // Multiple patterns
```

**Glob syntax:** Follows `java.nio.file.PathMatcher` with `glob:` pattern rules.

**You can combine both mechanisms:**
```java
@ValuePool(value = {"mySupplier"}, files = {"test-data/*.json"})
```

---

### Configuration Options

#### Mutation Probability (`p` field)
Controls how often values from the pool are used versus randomly generated values.
```java
@ValuePool(value = {"mySupplier"}, p = 0.3) // Use pool values 30% of the time
```

**Default:** `p = 0.1` (10% of mutations use pool values)
**Range:** 0.0 to 1.0

#### Type Propagation (`constraint` field)

Controls whether the annotation affects nested types:
```java
// Default: RECURSIVE - applies to all nested types
@ValuePool(value = {"mySupplier"}, constraint = Constraint.RECURSIVE)

// DECLARATION - applies only to the annotated type, not subtypes
@ValuePool(value = {"mySupplier"}, constraint = Constraint.DECLARATION)
```

**Example of the difference:**
```java
// With RECURSIVE (default):
@ValuePool(value = {"stringSupplier"}) Map> data
// Strings feed both Map keys AND List elements

// With DECLARATION:
@ValuePool(value = {"stringSupplier"}, constraint = DECLARATION) Map> data
// Strings only feed Map keys, NOT List elements
```

---

### Complete Example
```java
class MyFuzzTest {
static Stream edgeCases() {
return Stream.of(
"", "null", "alert('xss')", // Strings
0, -1, Integer.MAX_VALUE, // Integers
new byte[]{0x00, 0xFF} // A byte array
);
}

@FuzzTest
@ValuePool(
value = {"edgeCases"},
files = {"test-inputs/*.bin"},
p = 0.25 // Use pool values 25% of the time
)
void testParser(String input, Map config, byte[] data) {
// All three parameters get values from the pool:
// - 'input' gets Strings
// - 'config' keys get Strings, values get Integers
// - 'data' gets bytes from both edgeCases() and *.bin files
}
}
```

---

#### Max Mutations (`maxMutations` field)

After selecting a value from the pool, the mutator can apply additional random mutations to it.
```java
@ValuePool(value = {"mySupplier"}, maxMutations = 5)
```

**Default:** `maxMutations = 1` (one additional mutation applied)
**Range:** 0 or higher

**How it works:** If `maxMutations = 5`, Jazzer will:
1. Select a value from your pool (e.g., `"example"`)
2. Apply up to 5 random mutations (e.g., `"example"` → `"exAmple"` → `"exAmple123"` → ...)

This helps explore variations of your seed values while staying close to realistic inputs.


## FuzzedDataProvider

The `FuzzedDataProvider` is an alternative approach commonly used in programming
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ load("//bazel:fuzz_target.bzl", "java_fuzz_target_test")

java_fuzz_target_test(
name = "ArgumentsMutatorFuzzTest",
timeout = "long",
srcs = [
"ArgumentsMutatorFuzzTest.java",
"BeanWithParent.java",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ private static Path addInputAndSeedDirs(
Paths.get(context.getConfigurationParameter("jazzer.internal.basedir").orElse(""))
.toAbsolutePath();

System.setProperty("jazzer.internal.basedir", baseDir.toString());

// Use the specified corpus dir, if given, otherwise store the generated corpus in a per-class
// directory under the project root.
// The path is specified relative to the current working directory, which with JUnit is the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,17 @@
import java.lang.reflect.AnnotatedType;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;

public final class ArgumentsMutator {
private final ExtendedMutatorFactory mutatorFactory;
private final Method method;
private final InPlaceProductMutator productMutator;

private static final Map<Method, ArgumentsMutator> mutatorsCache = new ConcurrentHashMap<>();

private Object[] arguments;

/**
Expand Down Expand Up @@ -78,15 +82,14 @@ private static String prettyPrintMethod(Method method) {
}

public static ArgumentsMutator forMethodOrThrow(Method method) {
return forMethod(Mutators.newFactory(new ValuePoolRegistry(method)), method)
.orElseThrow(
() ->
new IllegalArgumentException(
"Failed to construct mutator for " + prettyPrintMethod(method)));
}

public static Optional<ArgumentsMutator> forMethod(Method method) {
return forMethod(Mutators.newFactory(new ValuePoolRegistry(method)), method);
return mutatorsCache.computeIfAbsent(
method,
m ->
forMethod(Mutators.newFactory(new ValuePoolRegistry(m)), m)
.orElseThrow(
() ->
new IllegalArgumentException(
"Failed to construct mutator for " + prettyPrintMethod(m))));
}

public static Optional<ArgumentsMutator> forMethod(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
import java.lang.annotation.Target;

/**
* Provides values to user-selected mutator types to start fuzzing from.
* Provides values to user-selected types that will be used during mutation.
*
* <p>This annotation can be applied to fuzz test methods and any parameter type or subtype. By
* default, this annotation is propagated to all nested subtypes unless specified otherwise via the
Expand Down Expand Up @@ -72,17 +72,44 @@
* don't need to match the type of the annotated method or parameter. The mutation framework will
* extract only the values that are compatible with the target type.
*/
String[] value();
String[] value() default {};

/**
* Specifies glob patterns matching files that should be provided as {@code byte []} to the
* annotated type. The syntax follows closely to Java's {@link
* java.nio.file.FileSystem#getPathMatcher(String) PathMatcher} "glob:" syntax.
*
* <p>Relative glob patterns are resolved against the working directory.
*
* <p>Examples:
*
* <ul>
* <li>{@code *.jpeg} - matches all jpegs in the working directory
* <li>{@code **.xml} - matches all xml files recursively
* <li>{@code src/test/resources/dict/*.txt} - matches txt files in a specific directory
* <li>{@code /absolute/path/to/some/directory/**} - matches all files in an absolute path
* recursively
* <li>{@code {"*.jpg", "**.png"}} - matches all jpg in the working directory, and png files
* recursively
* </ul>
*/
String[] files() default {};

/**
* This {@code ValuePool} will be used with probability {@code p} by the mutator responsible for
* fitting types.
*/
double p() default 0.1;

/**
* If the mutator selects a value from this {@code ValuePool}, it will perform up to {@code
* maxMutations} additional mutations on the selected value.
*/
int maxMutations() default 1;

/**
* Defines the scope of the annotation. Possible values are defined in {@link
* com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. By default it's {@code
* com.code_intelligence.jazzer.mutation.utils.PropertyConstraint}. By default, it's {@code
* RECURSIVE}.
*/
String constraint() default RECURSIVE;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@
import java.util.Optional;
import java.util.function.Predicate;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class ValuePoolMutatorFactory implements MutatorFactory {
/** Types annotated with this marker wil not be re-wrapped by this factory. */
Expand Down Expand Up @@ -75,12 +74,17 @@ private static final class ValuePoolMutator<T> extends SerializingMutator<T> {
private final SerializingMutator<T> mutator;
private final List<T> userValues;
private final double poolUsageProbability;
private final int maxMutations;

ValuePoolMutator(
SerializingMutator<T> mutator, List<T> userValues, double poolUsageProbability) {
SerializingMutator<T> mutator,
List<T> userValues,
double poolUsageProbability,
int maxMutations) {
this.mutator = mutator;
this.userValues = userValues;
this.poolUsageProbability = poolUsageProbability;
this.maxMutations = maxMutations;
}

@SuppressWarnings("unchecked")
Expand All @@ -91,14 +95,9 @@ static <T> SerializingMutator<T> wrapIfValuesExist(
return mutator;
}

Optional<Stream<?>> rawUserValues = valuePoolRegistry.extractRawValues(type);
if (!rawUserValues.isPresent()) {
return mutator;
}

List<T> userValues =
rawUserValues
.get()
valuePoolRegistry
.extractUserValues(type)
// Values whose round trip serialization is not stable violate either some user
// annotations on the type (e.g. @InRange), or the default mutator limits (e.g.
// default List size limits) and are therefore not suitable for inclusion in the value
Expand All @@ -112,7 +111,8 @@ static <T> SerializingMutator<T> wrapIfValuesExist(
}

double p = valuePoolRegistry.extractFirstProbability(type);
return new ValuePoolMutator<>(mutator, userValues, p);
int maxMutations = valuePoolRegistry.extractFirstMaxMutations(type);
return new ValuePoolMutator<>(mutator, userValues, p, maxMutations);
}

/**
Expand Down Expand Up @@ -144,8 +144,8 @@ private static <T> boolean isSerializationStable(SerializingMutator<T> mutator,
@Override
public String toDebugString(Predicate<Debuggable> isInCycle) {
return String.format(
"%s (values: %d p: %.2f)",
mutator.toDebugString(isInCycle), userValues.size(), poolUsageProbability);
"%s (values: %d p: %.2f, maxMutations: %d)",
mutator.toDebugString(isInCycle), userValues.size(), poolUsageProbability, maxMutations);
}

@Override
Expand Down Expand Up @@ -180,19 +180,28 @@ public T init(PseudoRandom prng) {
@Override
public T mutate(T value, PseudoRandom prng) {
if (prng.closedRange(0.0, 1.0) < poolUsageProbability) {
if (prng.choice()) {
return prng.pickIn(userValues);
} else {
// treat the value from valuePool as a starting point for mutation
return mutator.mutate(prng.pickIn(userValues), prng);
value = prng.pickIn(userValues);
// Treat the user value as a starting point for mutation
for (int i = 0; i < prng.closedRange(0, maxMutations); i++) {
value = mutator.mutate(value, prng);
}
return value;
}
return mutator.mutate(value, prng);
}

@Override
public T crossOver(T value, T otherValue, PseudoRandom prng) {
return mutator.crossOver(value, otherValue, prng);
if (prng.closedRange(0.0, 1.0) < poolUsageProbability) {
value = prng.pickIn(userValues);
// Treat the user value as a starting point for crossOver
for (int i = 0; i < prng.closedRange(0, maxMutations); i++) {
value = mutator.crossOver(value, prng.pickIn(userValues), prng);
}
return value;
} else {
return mutator.crossOver(value, otherValue, prng);
}
}
}
}
Loading
Loading