(Link to video)
Pokemon Yellow Total Control Hack. Reprogramming the game from the inside!

Game objectives

  • Emulator used: vba-rerecording 23.5
  • Reprogram the Game from the inside

Comments

I've included a detailed writeup here: http://aurellem.org/vba-clojure/html/total-control.html
The following are the highlights:

Introduction

Think of pokemon yellow as creating a little universe with certain rules. Inside that universe, you can buy items, defeat rival trainers, and raise your pokemon. But within that universe, you are bound by the rules of pokemon. You can't build new buildings, or change the music, or change your clothes.. There are some games (like chess), where it is not possible to alter the rules of the game from within the game. No matter what moves you make in chess, you can never change the rules of the game so that it becomes checkers or basketball. The point of this run is to show that you CAN change the rules in pokemon yellow. There is a certain sequence of valid actions (like walking from one place to another or buying items) that will allow you to transform pokemon yellow into Pacman, or Tetris, or Pong, or a MIDI player, or anything else you can imagine.

Background

The speedrun (#2913: p4wn3r's GBC Pokémon: Yellow Version in 01:36.95) by Felipe Lopes de Freitas (p4wn3r), beats pokemon yellow in only 1 minute and 36 seconds. It does it by corrupting the in-game item list so that he can advance the list past its normal limit of 20 items. The memory immediately after the item list includes the warp points for the current map, and by treating that data as items and switching and dropping them, he can make the door from his house take him directly to the end of the game.
When I first saw that speedrun, I was amazed at how fast pokemon yellow could be beaten, and that it was possible to manipulate the game from the inside, using only the item list. I wondered how far I could extend the techniques found in p4wn3r's run.
The gameboy is an 8 bit computer. That means that ultimately, anything that happens in pokemon is a result of the gameboy's CPU reading a stream of 8 bit numbers and doing whatever those numbers mean. For example, in the gameboy, the numbers:
62 16 37 224 47 240 37 230 15 55
mean to check which buttons are currently pressed and copy that result into the "A" register. With enough numbers, you can spell out an interactive program that reads input from the buttons and allows you to write any program you want to the gameboy. Once you have assembled such a program and forced the game to run it, you have won, since you can use that program to write any other program (like Tetris or Pacman) over pokemon yellow's code. I call a program that allows you to write any other program a "bootstrapping program". So, the goal is to somehow get a bootstrapping program into pokemon yellow and then force yellow to run that program instead of its own.
How can we spell out such a program? Everything in the game is ultimately numbers, including all items, pokemon, levels, etc. In particular, the item list looks like:
item-one-id         (0-255)
item-one-quantity   (0-255)
item-two-id         (0-255)
item-two-quantity   (0-255)
.
.
.

Let's consider the button measuring program [37 62 16 37 224 37 240 37 230 15 55] from before. Interpreted as items and item quantities, it is
lemonade     x16
guard spec.  x224
leaf stone   x240
guard spec.  x230
parlyz heal  x55
So, if we can get the right items in the right quantities, we can spell out a bootstrapping program. Likewise, when writing the bootstrapping program, we must be careful to only use numbers that are also valid items and quantities. This is hard because there aren't many different items to work with, and many machine instructions actually take 2 or even 3 numbers in a row, which severely restricts the types of items you can use. I ended up needing about 92 numbers to implement a bootstrap program. Half of those numbers were elaborate ways of doing nothing and were just there so that the entire program was also a valid item list.
The final part of the hack is getting pokemon yellow to execute the new program after it has been assembled with items. Fortunately, pokemon keeps a number called a function pointer within easy reach of the corrupted item list. This function pointer is the starting point (address) of a program which the game runs every so often to check for poison and do general maintenance. By shifting an item over this function pointer, I can rewrite that address to point to the bootstrapping program, and make the game execute it. Without this function pointer, it would not be possible to take over the game.

The Run

Pallet

I start off and name my rival Lp/k. These characters will eventually be treated as items and shifted over the function pointer, causing it to execute the bootstrapping program that will soon be constructed. I start the run the same as p4wn3r's and restart the game while saving, so that the pokemon list is corrupted. By switching the 8th and 10th pokemon, I corrupt the item list and can now scroll down past the 20th item. I shift items around to increase the text speed to maximum and rewrite the warp point of my house to Celadon Dept. Store. (p4wn3r used this to go directly to the hall of fame and win the game in his run.) I deposit many 0x00 glitch items into the PC from my corrupted inventory for later use. Then, I withdraw the potion from the PC. This repairs my item list by overflowing the item counter from 0xFF back to 0x00, though the potion is obliterated in the process. I then take 255 glitch items with ID 0x00 from the computer into my personal items.

Celadon Dept. Store

Leaving my house takes me directly to Celadon Dept. store, where I sell two 0x00 items for 414925 each, giving myself essentially max money. I hit every floor of the department store, gathering the following items:
 +-------------------+----------+
 |##| Item           | Quantity |
 +--+----------------+----------+
 |1 | TM02           |  98      |
 |2 | TM37           |  71      |
 |3 | TM05           |   1      |
 |4 | TM09           |   1      |
 |5 | burn-heal      |  12      |
 |6 | ice-heal       |  55      |
 |7 | parlyz-heal    |  99      |
 |8 | parlyz-heal    |  55      |
 |9 | TM18           |   1      |
 |10| fire-stone     |  23      |
 |11| water-stone    |  29      |
 |12| x-accuracy     |  58      |
 |13| guard-spec     |  99      |
 |14| guard-spec     |  24      |
 |15| lemonade       |  16      |
 |16| TM13           |   1      |
 +--+----------------+----------+
After gathering these items, I deposit them in the appropriate order into the item PC to spell out my bootstrapping program. Writing a full bootstrap program in one go using only items turned out to be too hard, so I split the process up into three parts. The program that I actually construct using items is very limited. It reads only from the A, B, start, and select buttons, and writes 4 bits each frame starting at a fixed point in memory. After it writes 200 or so bytes, it jumps directly to what it just wrote. In my run, I use this program to write another bootstrapping program that can write any number of bytes to any location in memory, and then jump to any location in memory. This new program can also write 8 bits per frame by using all the buttons. Using this new bootstrap program, I write a final bootstrapping program that does everything the previous bootstrapping program does except it also displays the bytes it is writing to memory on the screen.

Finale

After completing this bootstrapping program, I go to the Celadon mansion, because I find the metaness of that building to be sufficiently high to serve as an exit point for the pokemon universe. I corrupt my item list again by switching corrupted pokemon, scroll down to my rival's name and discard until it is equal to the address of my bootstrapping program, and then swap it with the function pointer. Once the menu is closed, the bootstrapping program takes over, and I write the payload....

Other comments

The entire video was played by the computer using bots. I used functional programming to write search programs over different possible game states to find the most efficient way of performing general actions. Some interesting things I developed but didn't use were pretty printing functions to display the game's internal data structures, and an "improbability drive" that forces improbable events to happen automatically using search.
Here are a few example scripts:
 (defn-memo viridian-store->oaks-lab
   ([] (viridian-store->oaks-lab
        (get-oaks-parcel) ) )
   ([ script \]
      (->> script
           (walk [↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ← ← ← ← ← ← ← ← 
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ←
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  → → → → → → → →
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ← ← ← ←
                  ↓ ↓ ↓ ↓
                  ])
           (walk-thru-grass
            [↓ ↓ ↓ ↓ ↓ ↓ ↓])
           (walk [↓ ↓ ← ↓ ↓ ↓ ←
                  ↓ ↓ ↓ ↓ ↓ ↓
                  → → → ↑])
                 
           (do-nothing 1) ) ) )
This script walks from the Viridian City pokemon store to Oak's Lab in the most efficient way possible. The walk-thru-grass function guarantees that no wild battles will happen by manipulating the game's random number generator.
 (defn-memo hacking-10
   ([] (hacking-10 (hacking-9) ) )
   ([ script \]
      (->> script
           begin-deposit
           (deposit-held-item 17 230)
           (deposit-held-item-named :parlyz-heal 55)
           (deposit-held-item 14 178)
           (deposit-held-item-named :water-stone 29)
           (deposit-held-item 14 32)
           (deposit-held-item-named :TM18 1)
           (deposit-held-item 13 1)
           (deposit-held-item 13 191)
           (deposit-held-item-named :TM02 98)
           (deposit-held-item-named :TM09 1)
           close-menu) ) ) 
 
This script calculates the fastest sequence of key presses to deposit the requested items into a PC, assuming that the character starts out in front of a computer.

Other Comments

The final payload program is multiple programs. I created a reduced form of MIDI and implemented it in gameboy machine language. Then I translated a midi file from [dead link removed] into this reduced MIDI language. The payload program contains both the music data and the MIDI interpreter to play that data. The picture works in a similar way. There is code to translate a png file into a form that can be displayed on a gameboy, and other code to actually display that image. Both the image and the display code are also written by the final bootstrapping program. Even though my final payload is rather simple, you can write any program at all as the payload. The source for the sound and image displaying code is at http://hg.bortreb.com/vba-clojure
This entire project is open source and I encourage anyone who wants to take the code and play around!

Suggested Screenshots

Or whatever you all think would be best.
I encoded the video with/without button visualization here:

FractalFusion: Replaced movie file to fix time (note that the parser works in such a way so that the time listed for a VBM can easily be faked).
FractalFusion: Response is well-received for the new concept. Accepting for publication.
natt: Processing...
FractalFusion: Changed branch to "Executes Arbitrary Code".

1 2 3 4
7 8
Active player (421)
Joined: 3/21/2011
Posts: 127
Location: Virginia (United States)
jlun2 wrote:
Also, any other game that can also do something like this?
SMW is a possibility, as one of the glitches in it jumps to OAM, which can be manipulated to run codes (see the glitched any%). Make a code that writes input to RAM and jumps to it, and there you go. Anyway, very interesting run. For-sure yes vote. Though, I have to ask, why are there so many 00's in the code you wrote?
YouTube Channel - Twitter Current projects: Sutte Hakkun, Hyper VI, RTDL, own hacking projects
Emulator Coder, Skilled player (1141)
Joined: 5/1/2010
Posts: 1217
bortreb wrote:
HOWEVER, you can only actually write 60 bytes per second because that is the maximum input rate of the buttons (it actually isn't, but you'd need a different emulator than vba-rerecording to go any higher.)
Any estimates how fast one could read data if the emulator used supported sub-frame input (so one could read in fast loop)?
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
kaizoman666 wrote:
Anyway, very interesting run. For-sure yes vote. Though, I have to ask, why are there so many 00's in the code you wrote?
The 00s are part of the data for the image. I need to define a few empty or near-empty tiles to make the strings and background. Also, the balloon tiles have a lot of 00s in them as well.
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
Ilari wrote:
bortreb wrote:
HOWEVER, you can only actually write 60 bytes per second because that is the maximum input rate of the buttons (it actually isn't, but you'd need a different emulator than vba-rerecording to go any higher.)
Any estimates how fast one could read data if the emulator supported sub-frame input?
It depends on your input program, and how physically realistic you want it to be for a real gameboy. The simpliest input reading program is just a tight loop of about say 20 opcodes, and the processor operates at around 4Mhz (8Mhz if you force the CPU into Double Speed Mode). So that's something like 400kb per second with unrealistic physical assumptions. But if we're talking about actual physical buttons, they have a certain amount of time to physically move forwards and backwards, and then the electrical resonance in the detection circuit has to settle down, and all that. I would be surprised if you could go any higher than say 5 times higher than the current artificial limit, so maybe 300bytes a second max? Now, you could also use the IR link to push data into RAM. That operates at 65kb a second, though I think it is sort of cheating to use the IR port.
Editor, Skilled player (1941)
Joined: 6/15/2005
Posts: 3247
bortreb wrote:
When writing your payload program, you are not allowed to use any interrupts or the fixed jump instructions, since the interrupt handling code and fixed jump vectors are inside pokemon yellow's ROM. Without interrupts it's harder to play music and get input.
Does this apply to all programs written in RAM, or only the payload program? The initial bootstrap program uses absolute jumps, as detailed in this post. Also, since interrupts are like subroutines, does it mean that call routine instructions cannot be used? Edit: bortreb, you may use the edit button in the top right corner of each post.
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
FractalFusion wrote:
bortreb wrote:
When writing your payload program, you are not allowed to use any interrupts or the fixed jump instructions, since the interrupt handling code and fixed jump vectors are inside pokemon yellow's ROM. Without interrupts it's harder to play music and get input.
Does this apply to all programs written in RAM, or only the payload program? Also, since interrupts are like subroutines, does it mean that call routine instructions cannot be used?
The absolute jump and call routine instructions are fine, you just can't use interrupts (because they are registered with code in ROM as part of the gameboy's architecture), and you can't use the opcodes 0xCF, 0xDF, 0xEF, or 0xFF since those all call routines in ROM as well. See http://imrannazar.com/Gameboy-Z80-Opcode-Map for more details. The restriction doesn't limit what you can program, it just makes you have to constantly check on things instead of relying on interrupts. Thanks for letting me know about the Edit button. Is there something specific you wanted me to edit?
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
FractalFusion wrote:
I played this run back in VBA and looked at the memory. I'm very pleased with how it works. ... It actually starts writing a bit behind C000 but it doesn't matter since only the region C000-DFFF can be written to.
This is a very accurate view of the bootstrapping process! The third bootstrapping program does start at 0xC000 though. The first few numbers you see are part of the interface to the bootstrapping program that tells it where to start writing and how many bytes to write.
Emulator Coder, Skilled player (1141)
Joined: 5/1/2010
Posts: 1217
bortreb wrote:
Thanks for letting me know about the Edit button. Is there something specific you wanted me to edit?
I think he was referring to editing your posts instead posting multiple posts back-to-back.
Brandon
He/Him
Editor, Player (190)
Joined: 11/21/2010
Posts: 913
Location: Tennessee
I've been hoping for a run like this for a long time...granted, I was hoping to see Pokemon hacked into a different game and see that get TASed, but that would probably take too long, so this is definitely worth my yes vote.
All the best, Brandon Evans
Glitcher
He/Him
Joined: 3/24/2007
Posts: 216
Location: London, U.K.
bortreb wrote:
The 00s are part of the data for the image. I need to define a few empty or near-empty tiles to make the strings and background. Also, the balloon tiles have a lot of 00s in them as well.
But why draw balloons at all? If the background music is about ponies, wouldn't it be better to draw... I don't know... ponies?
Joined: 1/7/2011
Posts: 8
Glitcher wrote:
bortreb wrote:
The 00s are part of the data for the image. I need to define a few empty or near-empty tiles to make the strings and background. Also, the balloon tiles have a lot of 00s in them as well.
But why draw balloons at all? If the background music is about ponies, wouldn't it be better to draw... I don't know... ponies?
The balloons are very relevant to MLP. See here.
Joined: 12/2/2005
Posts: 139
Location: New York, United States
So it would seem everyone on this board is either a computer science major or a Brony. It's all greek to me, baby.
What's a man like me supposed to do with all this extra savoir-faire?
Spikestuff
They/Them
Editor, Publisher, Expert player (2305)
Joined: 10/12/2011
Posts: 6341
Location: The land down under.
Glitcher wrote:
bortreb wrote:
The 00s are part of the data for the image. I need to define a few empty or near-empty tiles to make the strings and background. Also, the balloon tiles have a lot of 00s in them as well.
But why draw balloons at all? If the background music is about ponies, wouldn't it be better to draw... I don't know... ponies?
The balloons is actually the mark of the character Pinkie Pie. So maybe in code making the whole thing pink would be a pain.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Post subject: Then we shall Speak Ελληνικά!!
Spikestuff
They/Them
Editor, Publisher, Expert player (2305)
Joined: 10/12/2011
Posts: 6341
Location: The land down under.
notBowen wrote:
It's all greek to me, baby.
Τι κάνετε; Είστε καλά;
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Editor, Experienced player (608)
Joined: 11/8/2010
Posts: 4012
Spikestuff wrote:
The balloons is actually the mark of the character Pinkie Pie.
Why are her eyes so scary-looking? I would like to see all that could be done with the code as well, but I understand it's no small task to even write out this simple program, and I suppose it's still really impressive.
Joined: 12/31/2007
Posts: 10
Well this was by far the most entertaining run I've seen in a long time. I'll toss making a game with this on my cool things to do if I ever have free time list.
Spikestuff
They/Them
Editor, Publisher, Expert player (2305)
Joined: 10/12/2011
Posts: 6341
Location: The land down under.
CoolKirby wrote:
Spikestuff wrote:
The balloons is actually the mark of the character Pinkie Pie.
Why are her eyes so scary-looking?
This shit is scarier
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Active player (405)
Joined: 3/22/2006
Posts: 708
I honestly don't know how to vote here. I think this is very impressive conceptually, but it's not really fun to watch.
Joined: 2/20/2010
Posts: 209
Location: I'm in space
CoolKirby wrote:
Why are her eyes so scary-looking?
BLASPHEMY
Oh, play it cool. Play it cool. Here come the space cops.
Player (12)
Joined: 11/23/2012
Posts: 94
Every time you derail the topic from the technical merit of the run, I wish I weren't a newbie so I could vote "meh" so hard on this. Talk about all the possibilities that we've discovered for making a game become a completely different game. :) If we could make Super Mario World as broken as Pokémon Yellow, I think it would be fantastic to make a Pokémon reference in SMW, and a Mario reference in Yellow.
Experienced player (599)
Joined: 2/8/2009
Posts: 656
This is by FAR the most impressive run I have ever seen. All I love about glitch hunting and TASing in general is to see all the things you can do with the game just by using proper inputs. It's already awesome to see people completely break games apart and beat them within a ridiculous fast time. This on the other hand brings this whole concept to a new level. Kudos, bortreb! You are a mad genius ;)
Editor, Skilled player (1506)
Joined: 7/9/2010
Posts: 1317
Yes vote for awesomeness!
Favorite animal: STOCK Gt(ROSA)26Sortm1.1(rtTA,EGFP)Nagy Grm7Tg(SMN2)89Ahmb Smn1tm1Msd Tg(SMN2*delta7)4299Ahmb Tg(tetO-SMN2,-luc)#aAhmb/J YouTube Twitch
Joined: 8/10/2004
Posts: 173
Location: Bethel, VT
Yes vote from me... a MLP fan AND a computer engineering major. The ending made me laugh pretty good. As for what can be done... well, depending on how much code you can input at once, I imagine you can make a program that converts PC assembly to gameboy assembly (if they are even different, and if they share instructions), and then make a bot that inputs that in to the game. It would be some work at first, but then you could start doing all kinds of things, I imagine.
Joined: 4/17/2010
Posts: 10
I kinda agree with the feeling that the ending is a little disappointing, but I don't think this run deserves to be rejected because of that. There's a lot of possibilities for entertainment here and publishing this would increase the chances of that happening. I vote yes for the moon tier and I think this really should get an star with a more elaborated ending.
Glitcher
He/Him
Joined: 3/24/2007
Posts: 216
Location: London, U.K.
MCXD wrote:
The balloons are very relevant to MLP. See here.
Spikestuff wrote:
The balloons is actually the mark of the character Pinkie Pie. So maybe in code making the whole thing pink would be a pain.
You're missing the point. If the music was about the Looney Tunes, I would not draw a picture of Yosemite Sam's guns. If it was about TMNT, I would not draw a picture of Spinter's cane. So why draw a picture of a pony's cutie mark when you can draw the ponies themselves?
1 2 3 4
7 8