Use MAME's debugger to reverse engineer and extend old games

For the Church of Robotron's installation at Toorcamp 2012, we needed to be able to trigger physical events when game events happened in Robotron 2084. A quick summary for context:

We had an Altar that contained a Linux box that ran MAME and Robotron 2084
We had joysticks that acted as HID devices
Player would kneal down and play the game. As events happened in the game (player death, human death, lasers firing), we would trigger physical events.

We choose to use MAME's debugger to detect game events and notify other pieces of software when they happened. This is a quick tutorial for others (and a reminder to ourselves) if you're interested in doing similar things. We're going to find out how to detect player death!

Prereqs:

MAME
Robotron ROMs
Patience

Here we go!

Start mame in windowed mode with the debugger active:

mame -window -debug robotron

You'll see two windows appear, one is the emulation, the other is the debugger window. You can type "help" at the console and get good documentation on the debugger capabilities. You can also click on the down arrow to access a menu that allows you to run the emulation and later pause it at different points.

Memory dump

The first thing to do to find the memory location of the number of lives the player currently has. One way to do this is to take advantage of other peoples work. ;) Sean Riddle has a great site with a lot of this information already available! The other way is the manual way. Let's do that!

Start the game by hitting F5
Start playing a game, by hitting 5 to drop a coin, and 1 for one player game.
Pause the game by hitting F8
In the debugger console, type "Help Memory"
Ah, nice, there's a dump program memory command. Type: "dump lives3.txt,0,0xFFFF"
Press F5, then click on the emulation screen
Hit F2 to access the Operator Screens of Robotron.
Hit F2 until you see "Game Adjustment"
We're going to change the default number of lives to 5 and see if we can find the difference in the dump.
Press "D" to go down to "Turns Per Player"
Press "I" to change the number of lives to "5"
Press F2
Reselect the debugger screen
Press F8 to pause
Press the Down Arrow, Select Reset, then Hard
Start playing a game, by hitting 5 to drop a coin, and 1 for one player game.
Let the game get to the main screen, then pause the game by hitting F8
Type "dump lives5.txt,0,0xFFFF"

Now we have dumps of the game with 5 lives and 3 lives as the default. Let's use a diff tool to see what changed in memory! On OSX, you can use Filemerge or standard old "diff". Scanning through the differences, we don't see anything that correlates to 5 lives and 3 lives. But we do see two spots that seem to correlate to 4 and 2 lives:

2722,2727c2722,2727

AA60: 18 07 FF 06 5F 04 1D AA 58 39 CD 02 00 00 00 00 ...._...X9...... AA60: 18 07 FF 06 43 03 E9 AA 58 39 CD 04 00 00 00 00 ....C...X9......

This address is AA6B.

3039c3039

BDE0:  00 00 00 00 00 00 01 00 00 02 50 00 02 01 14 0A  ..........P.....
BDE0:  00 00 00 00 00 00 01 00 00 02 50 00 04 01 14 0A  ..........P.....

This address is BDEC.

Setting watchpoints

Let's see if we can test our guesses!

Go back to the debugger:

Type "help watchpoints", a Watchpoint will cause program execution to stop whenever a memory address is written to or read from.
Find out more info on setting a watchpoint by typing "help wpset"
Let's set our watchpoints
Type "wpset 0xAA6B,1,w" This will set a watchpoint that is triggered whenever our first memory address guess is written to
Type "wpset 0xBDEC,1,w"
Run the game
Assuming you didn't die, you'll notice the first watchpoint has been hit and it doesn't have anything to do with player death. Lets clear that one.
Type "wpclear 1"
Start the game again, and play hard until the Mutant Savior dies.
You'll notice the second watchpoint is triggered!

We've found the memory location for player lives! We can also take note of the instruction pointer address. It's at 27AC, let's set a breakpoint here.

Setting breakpoints

Type "bpset 0x27AC"
Type "wpclear"

Continue playing. You'll notice the breakpoint is hit for every player death except the last one. That's a bummer! Let's walk up the stack and figure out a better spot. On some CPU types, you can do just that, but with the 6809 that's powering Robotron you can't. So we'll turn the trace file, run the game until we hit our breakpoint, and then close everything. We can step back up from the bottom of the file and see if there are interesting addresses to set breakpoints on.

Trace file

Type "trace trace.txt"
Start a game and die
In the debugger type "traceflush"
Then type "trace off"

Open the trace.txt file in an editor. Go all the way to the bottom. You'll see a nice trace with repeated blocks called out. I use these blocks as starting points for new breakpoints. If you scan up the file, you'll see repeated addresses of DC56. Skip above those, they're not interesting. We want to find the first branch of code that doesn't repeat. Just scanning here are the addresses that seem interesting to me:

0xD676, 0xD8BC, 0xD1ED, 0x30FE

Let's set some breakpoints:

Type "bpset 0xD676"
Type "bpset 0xD8BC"
Type "bpset 0xD1ED"
Type "bpset 0x30FE"

Start running!

You'll notice D1ED gets triggered alot, so disable that one with the "bpclear (breakpoint number)" The same thing happens to D8BC and D676. But, 0x30FE seems to work!

Summary

I hope this helps people get used to the MAME debugger. After you have this information, you may want to do something with it. We choose to broadcast game events over UDP. You can see how by looking at our Github repo here.

PRAISE THE MUTANT SAVIOR!

Crossposted from here.