Startup

The search parameters are contained in a Configuration object. The configuration is initialised with various parameters and the type of chromosome to produce. The Genotype represents the population, and exposes the GA component to the user (via the evolve and getFittestIndividual methods).

Genotype

Genotypes can be bootstrapped from a Configuration via the randomInitialGenotype method. This constructs the population (which uses randomInitialChromosome to produce random individuals) and returns the initialised Genotype object.

Genotype.evolve()

evolve has three overloaded methods, the two that take arguments internally call the no-argument version.

The evolve method simply obtains an IBreeder object from the Configuration and uses it to construct a new Population, before setting the current population to the newly constructed version.

IBreeder

IBreeders coordinate the evolution step, however they do not actually perform or store any of the genetic operators. These are kept in the Configuration object’s m_geneticOperators field.

GABreeder

The default Breeder used is the GABreeder. The following stages are applied to the population:

Fitness Evaluation

The fitness of each individual in the population is first calculated, using the bulk fitness function, or via updateChromosomes if one isn’t provided.

Natural Selectors (Before Genetic Operators)

Each of the NaturalSelectors registered for application before the genetic operator chain runs is run on the population. The default configuration does not set any of these selectors.

Genetic Operators

Each of the genetic operators registered in the configuration is run, one at a time, in order of addition to the Configuration. Implementation for this step appears in BreederBase.applyGeneticOperators. The genetic operators implement an operate method, which runs over the population. These may only add to the population, however; an operator should never modify an individual in the population. A List of IChromosomes is passed as well as a Population, although this is just a reference to the Population‘s internal list.

DefaultConfiguration uses a Crossover operator, followed by a Mutation operator. Parameters to these are given at the end of this post.

Evaluation

The population is then re-evaluated, as it contains new individuals not present before the first round of selection.

Natural Selectors (After Genetic Operators)

Finally, the new population is produced by applying a NaturalSelector to it. If the selector selects too few individuals, JGAP can optionally fill the remainder with random individuals – change MinimumPopSizePercent to turn this on, but it’s off by default.

The Default JGAP Configuration

DefaultConfiguration sets up the following operator chain:

Genetic Operators

CrossoverOperator:

Rate: 35%, which translates to populationSize * 0.35 crossover operations per generation. Each crossover produces 2 individuals. Crossover point is random.

MutationOperator:

Desired Rate: 12. The rate for this one is a little different; mutation occurs if a random integer between 0 and the rate (12) is 0, thus it equates to a probability of 1/12. This operation is applied to each gene in each element of the population (excluding those produced by crossover), so the rough likelihood of a mutation being applied is (1/rate) * popSize * chromosomeSize. For each gene that is to be mutated, a copy of the whole IChromosome that contains it is made, with the selected Gene mutated via its applyMutation method.

Natural Selector

One selector is defined by default, which is the BestChromosomesSelector. This selector is given the following parameters:

Original Rate: 0.9. 90% of the population size allotted to this selector will actually be used. The selector is elitist, so this will return the top populationSize * 0.9 elements. The remaining 10% is filled by cloning the selected elements, one by one, in order of fitness until the population size reaches the limit.

Doublette Chromosomes Allowed? Yes. Doublettes are equivalent chromosomes (by equals()).

Configuration Parameters

The configuration specifies that the pool of selectors will select 100% of the population size. However, the only selector is configured to produce only 90% of the population size each time.

keepPopulationSizeConstant is also set to true, which dictates that the population is trimmed to size at each iteration. Somewhat counter-intuitively, this does not cause the population size to be increased if it falls below the user supplied value; it doesn’t enforce a minimum, only a maximum

minimumPopSizePercent is set to zero, so random individuals will never be used to fill the population if it declines below the limit (won’t ever happen under these conditions).

So What is the Default?

The default configuration is an elitist ranking selector that clones the top 90% of the user-specified population size, repeating it to fill the rest. Crossover is random point with a rate of populationSize * 0.35. Mutation is random and applied at  to 1 in 12 genes in the whole population (i.e. the rate is dictated by chromosome size * population size / 12).

On rebooting it turned out that my choice was unwise – I didn’t have sudo access nor did I have the root password for recovery. Unfortunately, the trick of using /bin/bash as an init replacement to get a root shell didn’t work either (it’s a common problem).

The fix was to write a small C program which just executed a script with /bin/sh to replace the sudoers file:

#include<stdio.h>
#include<stdlib.h>
#include<unistd.h>
int main(void){
	execl("/bin/sh", "-e", "/home/mat/replace_sudoers.sh",0);
	return 0;
}

Compile this with gcc: gcc -o replace replace_sudoers.c then create a replace_sudoers.sh with the following contents (with the home directory changed):

echo "Backing up old sudoers"

cp /etc/sudoers /home/mat/old_sudoers
chmod 777 /home/mat/old_sudoers

echo "moving new one"
cp -fr /home/mat/sudoers /etc/sudoers
chown root /etc/sudoers
chmod 0440 /etc/sudoers

echo "moved new sudoers successfully, will halt in 5s"
sleep 5
halt

I replaced my old sudoers with the following bare-bones one:

#/etc/sudoers

Defaults env_reset

root	ALL=(ALL) ALL
mat	ALL=(ALL) ALL
%admin	ALL=(ALL) ALL

Now, instead of using init=/bin/sh, init=/home/mat/replace can be used which will launch the script and overwrite the old sudoers file. On rebooting, you’ll have your sudo privileges back.

Luckily, thanks to the Pandocs [1] and a lot of patience from other developers, people have started building their own dumpers. This post details another implementation, built for PIC16 devices instead of the existing Arduino and AVR implementations. It is heavily based on the InsideGadgets Arduino GBCartRead project [2].

There's a ROM dumper under this rat's nest, honest.

The Hardware

I built my dumper around a PIC16F690. It’s got just enough pins to drive a couple of shift registers, a UART and to leave enough pins left over for I/O. The most difficult part to get is the cartridge connector – a donor Game Boy is required unfortunately.

The schematic is fairly straightforward. The PIC communicates with the PC via a MAX232. The address lines are driven via two 74HC594 shift registers. Data and control pins take up the remaining pins. Power is supplied via a LM7805 linear regulator (not shown).

The cartridge requires a connection to GND and +5V, as well as a pull-up resistor from +5V to the /RST pin (30)

The Software

Game Boy cartridges are fairly simple devices. They use a 32-pin connector which consists of power, three control lines, a 16-bit address bus and an 8-bit data bus. The cartridge shares the address bus with other devices and is mapped to the region 0×0000 to 0x7FFFF [3].

Talking to a cartridge is fairly simple:

1. Write the address you want to read or write to the address bus
2. (optional) Write the data you want to write to the data bus
3. (optional) Raise MREQ if you’re reading SRAM (Pokemon live there)
4. Raise RD or WR to read or write respectively
5. Drop RD, WR and MREQ
6. (optional) Read the data from the address bus

The WR, RD and MREQ pins are all active high.

Cartridges larger 16KB require bank switching which is handled by  a device called the memory bank controller (MBC). The MBC also controls access to the SRAM, as well as any other peripherals on the cartridge.

The MBC is controlled by writing commands to specific addresses. These addresses vary between MBCs, although most seem to enable RAM when 0xA0 is written to 0×0000. The MBC type is readable from the cartridge descriptor which is at 0×0134-0×0148 [1].

The firmware for the PIC is available at GitHub [4].

Rescuing the Pokemon

To use the dumper just connect to it via the serial port. The interface is fairly straight forward. It waits for confirmation before reading the descriptor, it’s pretty straight forward from there. The most important mode to use first is the diagnostics mode, which needs to be run without a cartridge connected.

On the Importance of Testing

The diagnostics mode is ideal for catching faults before they have real implications.

Faults like this one. In this case the WR pin hadn’t been properly soldered. Since the pins are active low and they’re connected via pull-down resistors. That means that the WR signal was 1, so the attempt to read RAM ended up writing garbage to it. I euthanised my Pokemon.

Dumping Data

Retrieving data (hopefully having not destroyed it) can be done using RealTerm [5]. It features true raw logging which is necessary, as, for example, PuTTY interprets some control characters, even in raw logging mode.

Future Work

At the moment the interface is a bit clunky, as is having to use RealTerm to dump things. I’ll be changing it to speak the same protocol as the original Arduino GBCartRead [2] which will make dumping things easier.

Links

[1] : The Pandocs – Everything you wanted to know about Game Boy but were afraid to ask

[2] : GBCartRead

[3] : Game Boy memory map

[4] : GBCartRead-PIC16

[5] : RealTerm

Some terminology needs explaining here before we go further.  AspectJ defines a point in the source to stop execution and pass it to your code as a “Pointcut”, the actual code executed on this pause is termed “advice”.  Although I use the word “pause” it’s not strictly correct; AspectJ “weaves” the advice code for matching pointcuts at compile time. Additionally (and perhaps most significantly) AspectJ can weave on class load time (“Load Time Weaving”), this is what we’ll be using.

So we have a way of inserting code in places in an existing code base, what can we do with it?  A lot of horrible things (playing around with returned values and parameters to name a few); but for now we’ll stick to what we want to do, tracing method calls.

The most obvious way to instrument a program is to insert calls to some logging library at the top of every method.  Although this works it takes a lot of time, either manually logging everything or figuring out how to use a regex/parser to do it for you.  With AspectJ we can simply weave code that logs the current method at every method execution point.

So how do we do that?  First we need to get AspectJ working; Eclipse will do that for you, alternatively you can do everything from the command line (assuming you have AspectJ installed, including the ajc command on your PATH).

We’ll define an Aspect that defines a pointcut on every method execution, as well as some advice to run when they turn up when the code is executed.

package aspects;

import java.util.logging.Level;
import java.util.logging.Logger;

import org.aspectj.lang.Signature;

aspect Trace{


	pointcut traceMethods() : (execution(* *(..))&& !cflow(within(Trace)));

	before(): traceMethods(){
		Signature sig = thisJoinPointStaticPart.getSignature();
		String line =""+ thisJoinPointStaticPart.getSourceLocation().getLine();
		String sourceName = thisJoinPointStaticPart.getSourceLocation().getWithinType().getCanonicalName();
		Logger.getLogger("Tracing").log(
                Level.INFO, 
                "Call from "
                	+  sourceName
                    +" line " +
                    line
                    +" to " +sig.getDeclaringTypeName() + "." + sig.getName()
		);
	}

}

So what exactly does this do? Firstly, the pointcut traceMethods() defines a new pointcut called traceMethods. This pointcut matches execution of every method in every class, as long as the control flow isn’t in the current class (Trace). The latter constraint is to stop an infinite loop occurring.

The before(): part of the class defines advice. This is the code that gets inserted just before the execution of the method. Advice can also be given after a pointcut is hit (using the after keyword instead). Our “advice” doesn’t modify execution flow, it just logs some information about the state of the program when the pointcut was hit, but it could quite easily start modifying control flow.

Save this file as Trace.java and compile it using: ajc -outxml -outjar aspects.jar Trace.java.

This compiles the aspect and puts it into a jar ready for use. The -outxml parameter will cause ajc to automatically generate an aop.xml file and save it in the jar’s META-INF directory. The load time weaving agent will read this to determine which aspects to weave with the classes it loads. The aop.xml file can do more than that, it can be used to set namespaces to ignore when weaving (such as java.*) to avoid mucking with the control flow of libraries.

So now we have our aspects.jar how do we use it? The AspectJ weaving agent must be available somewhere on the filesystem (aspectjweaver.jar) and your aspect.jar (and the target application) must be on the classpath.

Once that’s sorted to trace the application, use

java -javaagent:<path to aspectjweaver.jar> -cp <path to aspects.jar>:<path to target jar/folder> <name of main class to run>

Hopefully that will run and you should see a large amount of console output:

INFO: Call from main.RunFile line 206 to main.RunFile.main
Mar 31, 2011 2:52:53 PM aspects.Trace ajc$before$aspects_Trace$1$b314f86e
INFO: Call from main.RunFile line 186 to main.RunFile.runList
Mar 31, 2011 2:52:53 PM main.RunFile main
INFO: Starting clustering of 0 files
Mar 31, 2011 2:52:53 PM aspects.Trace ajc$before$aspects_Trace$1$b314f86e
INFO: Call from main.ExperimentRunner line 59 to main.ExperimentRunner.runExperiments


AspectJ lets you do far more than this, it’s an understatement to say this is all it can be used for. Using LTW does make for some interesting possibilities, one obvious use would be monkey patching existing projects to fix bugs where source isn’t available (and the license permits it, of course).

AspectJ is an interesting piece of work; the official project page has far more resources on the types of pointcuts you can define and the more nitty-gritty details of making it mess with the traced program.