sarah
/
nesfuzz
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Parameter Clean up, Single Shot Mode, Export stub

This commit is contained in:
Sarah Jamie Lewis 2021-11-22 14:45:52 -08:00
parent f4b677d52a
commit 7280bdb68e
3 changed files with 216 additions and 76 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
README.md
View File

@ -16,6 +16,7 @@ tile 28 (by default), windows to allow you to see the fuzzing happen.
![](./fuzzimages/screenshot.png)
## Parameters
Found at the top of `main.rs` a few parameters control the types and effectiveness of fuzzing.
@ -49,6 +50,25 @@ The only game that really works as expected is Super Mario Bros. with the `happy
This is probably because of issues in the underlying emulator / differences in the expected behaviour of the system the
tas inputs are produced for v.s. the emulator.
Other games like Legend of Zelda, Megaman, Super Mario Bros. 3, Final Fantasy II etc. will run, but I have had any
Other games like Legend of Zelda, Megaman, Super Mario Bros. 3, Final Fantasy II etc. will run, but
tas inputs from them quickly become out of sync with the actual gameplay. Further research is needed to as to why
that is. Help appreciated.
that is. Help appreciated.
As noted above, many speed runs, and some of the more interesting bugs require exploiting input from a second controller.
I didn't have time to dive into exactly how player 2 controllers work on the NES this weekend and so my first attempt
at this implementation is buggy. It seems to work fine for Legend of Zelda, but causes issues if the feature is
enabled in other games.
Finally, there is an issue with the cpu clock / soft resest which causes a one frame difference in behaviour between
emulated runs and fuzzed runs. This means a tiny modification needs to be made to runs exported from nesfuzz before
they can be run in an emulator. This might also be related to the issue described above.
## Future Extensions
Right now novelty is driven by the hamming distance of the ram of the cpu compared to observed values. You can get
better performance by changing the novelty to focus on specific values / be more game specific.
There are also a number of possible extensions in the replacement algorithm. Right now the fuzzer makes no
attempt to "lock-in" good paths and so the engine is likely to reconsider all past values. This leads to the
queue of inputs growing without bound (eventually causing the application itself to be refused memory from the kernel).

63
src/input.rs
View File

@ -1,7 +1,8 @@
use crate::input::ConsoleAction::SoftReset;
use crate::{Rng, DISABLE_START_PRESSES_AFTER, MUTATION_RATE, MUTATION_RATE_SOFT_RESET};
use std::fs::File;
use std::io;
use std::io::BufRead;
use std::{fs, io};
#[derive(Clone, Debug, Eq, PartialEq, Hash)]
pub enum ConsoleAction {
@ -28,6 +29,66 @@ impl FuzzingInput {
self.disable_start_after = frames;
}
pub fn export(&self, filename: String, frames: usize) {
let mut tas = format!("");
println!("Found path to world werid...dumping tas");
for i in 0..frames {
tas = format!("{}|", tas);
let input = self.get_frame_input(i).unwrap();
if input.console_action == SoftReset {
tas = format!("{}1", tas);
} else {
tas = format!("{}0", tas);
}
tas = format!("{}|", tas);
if input.player_1_input >> 7 == 1 {
tas = format!("{}R", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 6) & 0x01 == 1 {
tas = format!("{}L", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 5) & 0x01 == 1 {
tas = format!("{}D", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 4) & 0x01 == 1 {
tas = format!("{}U", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 3) & 0x01 == 1 {
tas = format!("{}T", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 2) & 0x01 == 1 {
tas = format!("{}S", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 1) & 0x01 == 1 {
tas = format!("{}B", tas);
} else {
tas = format!("{}.", tas);
}
if (input.player_1_input >> 0) & 0x01 == 1 {
tas = format!("{}A", tas);
} else {
tas = format!("{}.", tas);
}
tas = format!("{}|||\n", tas);
}
fs::write(filename, tas).expect("Unable to write file");
}
pub fn mutate_from(&mut self, rng: &mut Rng, frame: usize, frames_to_consider: usize) {
self.mutated = true;
for frame_num in frame..(frame + frames_to_consider) {

205
src/main.rs
View File

@ -1,4 +1,4 @@
#![feature(total_cmp)]
#![feature(int_log)]
mod apu;
mod audio;
@ -13,15 +13,18 @@ mod state;
use apu::Apu;
use ppu::Ppu;
use crate::cpu::Cpu;
use crate::fuzzing_state::{FuzzingInputState, FuzzingState};
use crate::input::ConsoleAction::SoftReset;
use crate::input::{ConsoleAction, FuzzingInput};
use crate::screen::Screen;
use minifb::{ScaleMode, Window, WindowOptions};
use priority_queue::double_priority_queue::DoublePriorityQueue;
use priority_queue::priority_queue::PriorityQueue;
use std::collections::HashSet;
use std::sync::mpsc::{Receiver, Sender};
use std::sync::{mpsc, Arc, Mutex};
use std::thread;
use std::{fs, thread};
// The number of cpu instances to spawn..
const NUM_THREADS: usize = 28;
@ -29,21 +32,30 @@ const NUM_THREADS: usize = 28;
// The number of frames to fuzz and process
// A small number exploits the current point more at the expense of
// large exploration - and vice versa.
const FRAMES_TO_CONSIDER: usize = 400;
const FRAMES_TO_CONSIDER: usize = 500;
// Same input should generate the same output...
// (I make no guarantee of that at the moment)
const RNG_SEED: u32 = 0x5463753;
const RNG_SEED: u32 = 0x35234623;
// If set to a low number, this disables start presses after the given frame
// Useful for some games where pausing does nothing to advance the game...
const DISABLE_START_PRESSES_AFTER: usize = 50;
const DISABLE_START_PRESSES_AFTER: usize = 500;
// The rate at which seed inputs become corrupted..
const MUTATION_RATE: f64 = 0.1;
const MUTATION_RATE: f64 = 0.25;
// The rate at which seed inputs may become soft resets..
const MUTATION_RATE_SOFT_RESET: f64 = 0.000;
const MUTATION_RATE_SOFT_RESET: f64 = 0.0000;
// The number of cases to fuzz from a given seed input at each consideration point..
const SEED_CASES: usize = 500;
// Only add the original seed case top the queue (hammer a single state)
const SINGLE_SHOT: bool = false;
// Fast forward the seed state to a given frame
const PLAY_FROM: usize = 0;
fn main() -> Result<(), String> {
let argv = std::env::args().collect::<Vec<String>>();
@ -67,23 +79,26 @@ fn main() -> Result<(), String> {
let mut seed_input = FuzzingInput::load(fm2_file.as_str());
seed_input.disable_start_after(DISABLE_START_PRESSES_AFTER);
let mut starting_state = FuzzingState::default(rom_filename.clone());
if PLAY_FROM > 0 {
println!("Playing {} frames prelude...", PLAY_FROM);
let (mut cpu, _) = starting_state.load_state(rom_filename.clone());
play_frames(&mut cpu, PLAY_FROM, &seed_input);
starting_state = FuzzingState::save_state(cpu, PLAY_FROM);
}
let mut fuzzing_queue: PriorityQueue<FuzzingInputState, u64> = PriorityQueue::new();
fuzzing_queue.push(
FuzzingInputState(
seed_input.clone(),
FuzzingState::default(rom_filename.clone()),
),
0,
FuzzingInputState(seed_input.clone(), starting_state.clone()),
10000,
);
for _i in 1..NUM_THREADS {
let mut mutated_input = seed_input.clone();
mutated_input.mutate_from(&mut rng, 0, FRAMES_TO_CONSIDER);
mutated_input.mutate_from(&mut rng, PLAY_FROM, FRAMES_TO_CONSIDER);
fuzzing_queue.push(
FuzzingInputState(
mutated_input.clone(),
FuzzingState::default(rom_filename.clone()),
),
0,
FuzzingInputState(mutated_input.clone(), starting_state.clone()),
rng.next() as u64,
);
}
@ -112,11 +127,9 @@ fn main() -> Result<(), String> {
loop {
println!("Prospective Cases: {}", fuzzing_queue.len());
let mut temp_scores = vec![];
for i in 0..NUM_THREADS {
if fuzzing_queue.is_empty() == false {
let (state, score) = fuzzing_queue.pop().unwrap();
temp_scores.push(score);
// Send to thread for process...
result_channels[i]
.0
@ -125,10 +138,7 @@ fn main() -> Result<(), String> {
} else {
result_channels[i]
.0
.send((
seed_input.clone(),
FuzzingState::default(rom_filename.clone()),
))
.send((seed_input.clone(), starting_state.clone()))
.expect("error sending new fuzzing input to thread");
}
}
@ -139,10 +149,8 @@ fn main() -> Result<(), String> {
Ok((fuzzing_input, fuzzing_state)) => {
let mut lowest_similarity = u64::MAX;
for (_j, state) in previous_states.iter().enumerate() {
let similiarty = hamming::distance(
state.as_slice(),
fuzzing_state.cpu_state.mem.as_slice(),
);
let similiarty =
hamming::distance(&[state[0x86]], &[fuzzing_state.cpu_state.mem[0x86]]);
// println!("{} {} {} ",i,j,similiarty);
if similiarty < lowest_similarity {
lowest_similarity = similiarty
@ -151,16 +159,56 @@ fn main() -> Result<(), String> {
previous_states.insert(fuzzing_state.cpu_state.mem.clone());
// Seed Thread
if fuzzing_input.mutated == false {
// Seed input handling...
fuzzing_queue.push(
FuzzingInputState(fuzzing_input.clone(), fuzzing_state.clone()),
10000,
);
if SINGLE_SHOT {
if lowest_similarity > 0 {
for i in 0..lowest_similarity.log10() {
let mut mutated_input: FuzzingInput = fuzzing_input.clone();
mutated_input.mutate_from(
&mut rng,
starting_state.frames,
FRAMES_TO_CONSIDER,
);
// Add the mutated input and the regular input to the queue...
fuzzing_queue.push(
FuzzingInputState(
mutated_input.clone(),
starting_state.clone(),
),
lowest_similarity,
);
}
}
continue;
} else {
// Seed Thread
if fuzzing_input.mutated == false {
// Seed input handling...
fuzzing_queue.push(
FuzzingInputState(fuzzing_input.clone(), fuzzing_state.clone()),
10000,
);
// Add seed cases..
for _i in 0..SEED_CASES {
let mut mutated_input: FuzzingInput = fuzzing_input.clone();
mutated_input.mutate_from(
&mut rng,
fuzzing_state.frames,
FRAMES_TO_CONSIDER,
);
// Add the mutated input and the regular input to the queue...
fuzzing_queue.push(
FuzzingInputState(mutated_input.clone(), fuzzing_state.clone()),
lowest_similarity,
);
}
} else if lowest_similarity > 0 {
// Only add to the queue if strictly better...
fuzzing_queue.push(
FuzzingInputState(fuzzing_input.clone(), fuzzing_state.clone()),
lowest_similarity,
);
// Add 10 random cases..
for _i in 0..10 {
let mut mutated_input: FuzzingInput = fuzzing_input.clone();
mutated_input.mutate_from(
&mut rng,
@ -173,29 +221,24 @@ fn main() -> Result<(), String> {
lowest_similarity,
);
}
} else if lowest_similarity > 0 {
// Only add to the queue if strictly better...
fuzzing_queue.push(
FuzzingInputState(fuzzing_input.clone(), fuzzing_state.clone()),
lowest_similarity,
);
let mut mutated_input: FuzzingInput = fuzzing_input.clone();
mutated_input.mutate_from(
&mut rng,
fuzzing_state.frames,
FRAMES_TO_CONSIDER,
);
// Add the mutated input and the regular input to the queue...
fuzzing_queue.push(
FuzzingInputState(mutated_input.clone(), fuzzing_state.clone()),
lowest_similarity,
);
}
}
_ => {}
}
}
if SINGLE_SHOT {
// Add seed cases..
for _i in 0..SEED_CASES {
let mut mutated_input: FuzzingInput = seed_input.clone();
mutated_input.mutate_from(&mut rng, starting_state.frames, FRAMES_TO_CONSIDER);
// Add the mutated input and the regular input to the queue...
fuzzing_queue.push(
FuzzingInputState(mutated_input.clone(), starting_state.clone()),
rng.next() as u64,
);
}
}
}
}
@ -258,6 +301,7 @@ fn run_game(
while new_frames < FRAMES_TO_CONSIDER {
// step CPU: perform 1 cpu instruction, getting back number of clock cycles it took
let cpu_cycles = cpu.step();
// clock PPU three times for every CPU cycle
@ -313,25 +357,40 @@ fn run_game(
}
}
/*
TODO:
- untangle CPU and APU/PPU?
- better save file organization?
fn play_frames(cpu: &mut Cpu, num_frames: usize, fuzzing_input: &FuzzingInput) {
let mut played_frames = 0;
while played_frames < num_frames {
// step CPU: perform 1 cpu instruction, getting back number of clock cycles it took
let cpu_cycles = cpu.step();
Timing notes:
The PPU is throttled to 60Hz by sleeping in the main loop. This locks the CPU to roughly its intended speed, 1.789773MHz NTSC. The APU runs at half that.
The APU gives all of its samples to the SDL audio device, which takes them 60 times per second in batches of 735 (44,100/60). It selects the ones
it needs at the proper interval and truncates its buffer.
// clock PPU three times for every CPU cycle
for _ in 0..cpu_cycles * 3 {
let (_, end_of_frame) = cpu.ppu.clock();
if end_of_frame {
played_frames += 1;
}
Failed tests from instr_test-v5/rom_singles/:
3, immediate, Failed. Just unofficial instructions?
0B AAC #n
2B AAC #n
4B ASR #n
6B ARR #n
AB ATX #n
CB AXS #n
7, abs_xy, 'illegal opcode using abs x: 9c'
// Checking for Inputs...
*/
match fuzzing_input.get_frame_input(played_frames) {
Some(input) => {
match input.console_action {
ConsoleAction::None => {}
ConsoleAction::SoftReset => {
cpu.soft_reset();
}
}
if cpu.strobe & 0x01 == 0x01 {
cpu.button_states = input.player_1_input;
// FIXME PLayer 2 doesn't play nicely with some games (e.g. mario)
// So to enable player 2 controls you also have to uncomment the
// bus in cpu/mod.rs
cpu.button_states2 = input.player_2_input;
}
}
_ => {}
};
}
}
}