nesfuzz/README.md

# nesfuzz 

nesfuzz is a fuzzer for Nes Games by [@SarahJamieLewis](https://twitter.com/sarahjamielewis)

nessfuzz built on top of the [nestur](https://github.com/spieglt/nestur) emulator by [@spieglt](https://github.com/spieglt).

## Usage & Methodology 

To begin fuzzing you will need a rom file, and a sample input file. For sample inputs see [TasVids](http://tasvideos.org/).

`nessfuzz <rom> <tas file>` 
`nessfuzz smb.rom happylee-supermariobros,warped.fm2`

nesfuzz uses the same input to see novel RAM configurations and search the possible input space. It will also
tile 28 (by default), windows to allow you to see the fuzzing happen.

![](./fuzzimages/screenshot.png)


## Parameters

Found at the top of `main.rs` a few parameters control the types and effectiveness of fuzzing.

    // The number of cpu instances to spawn..
    const NUM_THREADS: usize = 28;
    
    // The number of frames to fuzz and process
    // A small number exploits the current point more at the expense of
    // large exploration - and vice versa.
    const FRAMES_TO_CONSIDER: usize = 400;
    
    // Same input should generate the same output...
    // (I make no guarantee of that at the moment)
    const RNG_SEED: u32 = 0x5463753;
    
    // If set to a low number, this disables start presses after the given frame
    // Useful for some games where pausing does nothing to advance the game...
    const DISABLE_START_PRESSES_AFTER: usize = 50;
    
    // The rate at which seed inputs become corrupted..
    const MUTATION_RATE: f64 = 0.1;
    
    // The rate at which seed inputs may become soft resets..
    const MUTATION_RATE_SOFT_RESET: f64 = 0.000;


## Known Issues

The only game that really works as expected is Super Mario Bros. with the `happylee-supermariobros,warped.fm2` input.
This is probably because of issues in the underlying emulator / differences in the expected behaviour of the system the
tas inputs are produced for v.s. the emulator.

Other games like Legend of Zelda, Megaman, Super Mario Bros. 3, Final Fantasy II etc. will run, but
tas inputs from them quickly become out of sync with the actual gameplay. Further research is needed to as to why
that is. Help appreciated.

As noted above, many speed runs, and some of the more interesting bugs require exploiting input from a second controller.
I didn't have time to dive into exactly how player 2 controllers work on the NES this weekend and so my first attempt
at this implementation is buggy. It seems to work fine for Legend of Zelda, but causes issues if the feature is
enabled in other games.

Finally, there is an issue with the cpu clock / soft resest which causes a one frame difference in behaviour between
emulated runs and fuzzed runs. This means a tiny modification needs to be made to runs exported from nesfuzz before
they can be run in an emulator. This might also be related to the issue described above.

## Future Extensions

Right now novelty is driven by the hamming distance of the ram of the cpu compared to observed values. You can get
better performance by changing the novelty to focus on specific values / be more game specific.

There are also a number of possible extensions in the replacement algorithm. Right now the fuzzer makes no
attempt to "lock-in" good paths and so the engine is likely to reconsider all past values. This leads to the
queue of inputs growing without bound (eventually causing the application itself to be refused memory from the kernel).


## Emulator Tests

### CPU Timing Test

Passess

### Sprite Hit

| Test Name | Current Status |
|-----------|----------------|
| 01.basics.nes| Pass | 
| 02.alignment.nes| Pass|
| 03.corners.nes | Pass|
| 04.flip.nes| Pass|
| 05.left_clip.nes| Pass|
| 06.right_edge.nes| Pass|
| 07.screen_bottom.nes| Fail #4 - `y=255 should not hit`|
| 08.double height.nes| Pass |
| 09.timing basics.nes| Pass|
| 10.timing order.nes| Pass|
| 11.edge_timing.nes| Pass|