(Link to video)
Pokemon Yellow Total Control Hack. Reprogramming the game from the inside!

Game objectives

  • Emulator used: vba-rerecording 23.5
  • Reprogram the Game from the inside

Comments

I've included a detailed writeup here: http://aurellem.org/vba-clojure/html/total-control.html
The following are the highlights:

Introduction

Think of pokemon yellow as creating a little universe with certain rules. Inside that universe, you can buy items, defeat rival trainers, and raise your pokemon. But within that universe, you are bound by the rules of pokemon. You can't build new buildings, or change the music, or change your clothes.. There are some games (like chess), where it is not possible to alter the rules of the game from within the game. No matter what moves you make in chess, you can never change the rules of the game so that it becomes checkers or basketball. The point of this run is to show that you CAN change the rules in pokemon yellow. There is a certain sequence of valid actions (like walking from one place to another or buying items) that will allow you to transform pokemon yellow into Pacman, or Tetris, or Pong, or a MIDI player, or anything else you can imagine.

Background

The speedrun (#2913: p4wn3r's GBC Pokémon: Yellow Version in 01:36.95) by Felipe Lopes de Freitas (p4wn3r), beats pokemon yellow in only 1 minute and 36 seconds. It does it by corrupting the in-game item list so that he can advance the list past its normal limit of 20 items. The memory immediately after the item list includes the warp points for the current map, and by treating that data as items and switching and dropping them, he can make the door from his house take him directly to the end of the game.
When I first saw that speedrun, I was amazed at how fast pokemon yellow could be beaten, and that it was possible to manipulate the game from the inside, using only the item list. I wondered how far I could extend the techniques found in p4wn3r's run.
The gameboy is an 8 bit computer. That means that ultimately, anything that happens in pokemon is a result of the gameboy's CPU reading a stream of 8 bit numbers and doing whatever those numbers mean. For example, in the gameboy, the numbers:
62 16 37 224 47 240 37 230 15 55
mean to check which buttons are currently pressed and copy that result into the "A" register. With enough numbers, you can spell out an interactive program that reads input from the buttons and allows you to write any program you want to the gameboy. Once you have assembled such a program and forced the game to run it, you have won, since you can use that program to write any other program (like Tetris or Pacman) over pokemon yellow's code. I call a program that allows you to write any other program a "bootstrapping program". So, the goal is to somehow get a bootstrapping program into pokemon yellow and then force yellow to run that program instead of its own.
How can we spell out such a program? Everything in the game is ultimately numbers, including all items, pokemon, levels, etc. In particular, the item list looks like:
item-one-id         (0-255)
item-one-quantity   (0-255)
item-two-id         (0-255)
item-two-quantity   (0-255)
.
.
.

Let's consider the button measuring program [37 62 16 37 224 37 240 37 230 15 55] from before. Interpreted as items and item quantities, it is
lemonade     x16
guard spec.  x224
leaf stone   x240
guard spec.  x230
parlyz heal  x55
So, if we can get the right items in the right quantities, we can spell out a bootstrapping program. Likewise, when writing the bootstrapping program, we must be careful to only use numbers that are also valid items and quantities. This is hard because there aren't many different items to work with, and many machine instructions actually take 2 or even 3 numbers in a row, which severely restricts the types of items you can use. I ended up needing about 92 numbers to implement a bootstrap program. Half of those numbers were elaborate ways of doing nothing and were just there so that the entire program was also a valid item list.
The final part of the hack is getting pokemon yellow to execute the new program after it has been assembled with items. Fortunately, pokemon keeps a number called a function pointer within easy reach of the corrupted item list. This function pointer is the starting point (address) of a program which the game runs every so often to check for poison and do general maintenance. By shifting an item over this function pointer, I can rewrite that address to point to the bootstrapping program, and make the game execute it. Without this function pointer, it would not be possible to take over the game.

The Run

Pallet

I start off and name my rival Lp/k. These characters will eventually be treated as items and shifted over the function pointer, causing it to execute the bootstrapping program that will soon be constructed. I start the run the same as p4wn3r's and restart the game while saving, so that the pokemon list is corrupted. By switching the 8th and 10th pokemon, I corrupt the item list and can now scroll down past the 20th item. I shift items around to increase the text speed to maximum and rewrite the warp point of my house to Celadon Dept. Store. (p4wn3r used this to go directly to the hall of fame and win the game in his run.) I deposit many 0x00 glitch items into the PC from my corrupted inventory for later use. Then, I withdraw the potion from the PC. This repairs my item list by overflowing the item counter from 0xFF back to 0x00, though the potion is obliterated in the process. I then take 255 glitch items with ID 0x00 from the computer into my personal items.

Celadon Dept. Store

Leaving my house takes me directly to Celadon Dept. store, where I sell two 0x00 items for 414925 each, giving myself essentially max money. I hit every floor of the department store, gathering the following items:
 +-------------------+----------+
 |##| Item           | Quantity |
 +--+----------------+----------+
 |1 | TM02           |  98      |
 |2 | TM37           |  71      |
 |3 | TM05           |   1      |
 |4 | TM09           |   1      |
 |5 | burn-heal      |  12      |
 |6 | ice-heal       |  55      |
 |7 | parlyz-heal    |  99      |
 |8 | parlyz-heal    |  55      |
 |9 | TM18           |   1      |
 |10| fire-stone     |  23      |
 |11| water-stone    |  29      |
 |12| x-accuracy     |  58      |
 |13| guard-spec     |  99      |
 |14| guard-spec     |  24      |
 |15| lemonade       |  16      |
 |16| TM13           |   1      |
 +--+----------------+----------+
After gathering these items, I deposit them in the appropriate order into the item PC to spell out my bootstrapping program. Writing a full bootstrap program in one go using only items turned out to be too hard, so I split the process up into three parts. The program that I actually construct using items is very limited. It reads only from the A, B, start, and select buttons, and writes 4 bits each frame starting at a fixed point in memory. After it writes 200 or so bytes, it jumps directly to what it just wrote. In my run, I use this program to write another bootstrapping program that can write any number of bytes to any location in memory, and then jump to any location in memory. This new program can also write 8 bits per frame by using all the buttons. Using this new bootstrap program, I write a final bootstrapping program that does everything the previous bootstrapping program does except it also displays the bytes it is writing to memory on the screen.

Finale

After completing this bootstrapping program, I go to the Celadon mansion, because I find the metaness of that building to be sufficiently high to serve as an exit point for the pokemon universe. I corrupt my item list again by switching corrupted pokemon, scroll down to my rival's name and discard until it is equal to the address of my bootstrapping program, and then swap it with the function pointer. Once the menu is closed, the bootstrapping program takes over, and I write the payload....

Other comments

The entire video was played by the computer using bots. I used functional programming to write search programs over different possible game states to find the most efficient way of performing general actions. Some interesting things I developed but didn't use were pretty printing functions to display the game's internal data structures, and an "improbability drive" that forces improbable events to happen automatically using search.
Here are a few example scripts:
 (defn-memo viridian-store->oaks-lab
   ([] (viridian-store->oaks-lab
        (get-oaks-parcel) ) )
   ([ script \]
      (->> script
           (walk [↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ← ← ← ← ← ← ← ← 
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ←
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  → → → → → → → →
                  ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
                  ← ← ← ← ←
                  ↓ ↓ ↓ ↓
                  ])
           (walk-thru-grass
            [↓ ↓ ↓ ↓ ↓ ↓ ↓])
           (walk [↓ ↓ ← ↓ ↓ ↓ ←
                  ↓ ↓ ↓ ↓ ↓ ↓
                  → → → ↑])
                 
           (do-nothing 1) ) ) )
This script walks from the Viridian City pokemon store to Oak's Lab in the most efficient way possible. The walk-thru-grass function guarantees that no wild battles will happen by manipulating the game's random number generator.
 (defn-memo hacking-10
   ([] (hacking-10 (hacking-9) ) )
   ([ script \]
      (->> script
           begin-deposit
           (deposit-held-item 17 230)
           (deposit-held-item-named :parlyz-heal 55)
           (deposit-held-item 14 178)
           (deposit-held-item-named :water-stone 29)
           (deposit-held-item 14 32)
           (deposit-held-item-named :TM18 1)
           (deposit-held-item 13 1)
           (deposit-held-item 13 191)
           (deposit-held-item-named :TM02 98)
           (deposit-held-item-named :TM09 1)
           close-menu) ) ) 
 
This script calculates the fastest sequence of key presses to deposit the requested items into a PC, assuming that the character starts out in front of a computer.

Other Comments

The final payload program is multiple programs. I created a reduced form of MIDI and implemented it in gameboy machine language. Then I translated a midi file from [dead link removed] into this reduced MIDI language. The payload program contains both the music data and the MIDI interpreter to play that data. The picture works in a similar way. There is code to translate a png file into a form that can be displayed on a gameboy, and other code to actually display that image. Both the image and the display code are also written by the final bootstrapping program. Even though my final payload is rather simple, you can write any program at all as the payload. The source for the sound and image displaying code is at http://hg.bortreb.com/vba-clojure
This entire project is open source and I encourage anyone who wants to take the code and play around!

Suggested Screenshots

Or whatever you all think would be best.
I encoded the video with/without button visualization here:

FractalFusion: Replaced movie file to fix time (note that the parser works in such a way so that the time listed for a VBM can easily be faked).
FractalFusion: Response is well-received for the new concept. Accepting for publication.
natt: Processing...
FractalFusion: Changed branch to "Executes Arbitrary Code".

1 2 3 4 5 6 7 8
Spikestuff
They/Them
Editor, Publisher, Expert player (2300)
Joined: 10/12/2011
Posts: 6341
Location: The land down under.
Glitcher wrote:
You're missing the point. If the music was about the Looney Tunes, I would not draw a picture of Yosemite Sam's guns. If it was about TMNT, I would not draw a picture of Spinter's cane. So why draw a picture of a pony's cutie mark when you can draw the ponies themselves?
It would be buggs bunny then as he is the main face It takes longer to do the code.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
LSK
Joined: 4/17/2006
Posts: 159
Dessyreqt wrote:
What's next? Write Tetris to memory and then TAS that? Of course, the Game Boy version of Tetris would take a little over 9 minutes to write, but it would be interesting nonetheless!
I would love to see Super Mario Land written into memory and then TAS'd. That's the sort of thing that should be done just for the sake of doing it. Also, it would be hilarious to watch in an emulator.
Player (33)
Joined: 3/8/2012
Posts: 398
Location: Windfall Island
I'd rather not wait the long amount of time it would take to code an entire game over Pokemon Yellow lol. But it would be really interesting.
IronSlayer wrote:
Your counterargument would be like me saying that the Earth is round and then you telling me that I need to show it's flat so I can "prove us all wrong".
Seems legit.
Emulator Coder, Skilled player (1141)
Joined: 5/1/2010
Posts: 1217
Abahbob wrote:
I'd rather not wait the long amount of time it would take to code an entire game over Pokemon Yellow lol. But it would be really interesting.
Using subframe input (and ignoring the physics of the buttons / detection circuits) would cut down the waiting time a lot.
Former player
Joined: 12/19/2010
Posts: 66
Location: Porto Alegre, Brazil
Ilari wrote:
Abahbob wrote:
I'd rather not wait the long amount of time it would take to code an entire game over Pokemon Yellow lol. But it would be really interesting.
Using subframe input (and ignoring the physics of the buttons / detection circuits) would cut down the waiting time a lot.
Does VBA support this? Or it's not an emulator thingy?
Everytime I get home My neighbour's cockroach is in my bed
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
Lollorcaust wrote:
Ilari wrote:
Abahbob wrote:
I'd rather not wait the long amount of time it would take to code an entire game over Pokemon Yellow lol. But it would be really interesting.
Using subframe input (and ignoring the physics of the buttons / detection circuits) would cut down the waiting time a lot.
Does VBA support this? Or it's not an emulator thingy?
VBA does not support subframe input to my knowledge.
Joined: 8/10/2004
Posts: 173
Location: Bethel, VT
This is how I imagine the main character when watching this:
Joined: 11/7/2012
Posts: 23
Should have used the ending from a totally different game as the final payload.
Joined: 2/20/2010
Posts: 209
Location: I'm in space
Itstoearly wrote:
This is how I imagine the main character when watching this:
I think it's more something like this:
Oh, play it cool. Play it cool. Here come the space cops.
Spikestuff
They/Them
Editor, Publisher, Expert player (2300)
Joined: 10/12/2011
Posts: 6341
Location: The land down under.
goldfish wrote:
I think it's more something like this: image
Seems legit. 10/10
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Glitcher
He/Him
Joined: 3/24/2007
Posts: 216
Location: London, U.K.
goldfish wrote:
I think it's more something like this:
This should be the screenshot. For realz.
Joined: 2/25/2006
Posts: 407
Any encode available on Youtube without the buttons along the bottom? All the button presses aren't visible in it making it pointless and the bright yellow is an eye sore when contrasted against a near monochrome game.
Ryzen 3700X, ASUS Crosshair VIII Hero (WiFi) Motherboard, 32GB 3600MHz RAM, MSI Geforce 1070Ti 8GB, Windows 10 Pro x64 http://tasvideos.org/Nach/FranpaAlert.html
Former player
Joined: 9/3/2012
Posts: 40
Location: boston
franpa wrote:
Any encode available on Youtube without the buttons along the bottom? All the button presses aren't visible in it making it pointless and the bright yellow is an eye sore when contrasted against a near monochrome game.
I don't have anything on youtube but there's an encode on my website with no buttons. It's here: http://aurellem.org/pokemon-hack/rlm-yellow-hack-no-buttons.avi
Joined: 4/3/2007
Posts: 29
Joined: 11/24/2012
Posts: 1
I'm nearly certain that Gary made this TAS. It's obvious, he got 10 Kanto's badges after all... http://www.youtube.com/watch?v=0W8f3A_VvgQ
Emulator Coder, Skilled player (1141)
Joined: 5/1/2010
Posts: 1217
bortreb wrote:
VBA does not support subframe input to my knowledge.
Using subframe input with GBC would mean using lsnes (nothing else capable of emulating GBC can do subframe input currently.)... Of course, there's making that emulator work (and possibly hacking it).
Editor, Skilled player (1941)
Joined: 6/15/2005
Posts: 3247
This run is a playaround but I think the branch should be changed. To me, "Total Control Hack" is misleading, but I can't think of another name for it at the moment. "Hacking" is fine, but not "Hack". Also, which tier do you think this should go in?
Former player
Joined: 12/19/2010
Posts: 66
Location: Porto Alegre, Brazil
FractalFusion wrote:
This run is a playaround but I think the branch should be changed. To me, "Total Control Hack" is misleading, but I can't think of another name for it at the moment. "Hacking" is fine, but not "Hack". Also, which tier do you think this should go in?
Star, of course.
Everytime I get home My neighbour's cockroach is in my bed
Editor, Experienced player (608)
Joined: 11/8/2010
Posts: 4012
Lollorcaust wrote:
FractalFusion wrote:
Also, which tier do you think this should go in?
Star, of course.
I don't think it fits in with the Star tier's current qualifications, but it really should be featured in a similar way to Star movies.
Editor, Player (44)
Joined: 7/11/2010
Posts: 1022
"Total control" makes sense as a category by itself, and it's the usual name I've heard for it.
Joined: 2/20/2010
Posts: 209
Location: I'm in space
Lollorcaust wrote:
FractalFusion wrote:
Also, which tier do you think this should go in?
Star, of course.
I second that -- the TAS is just sooo unique, I really think it should get the 'star' designation to put it out in the front with the other especially noteworthy TASes, even if only to demonstrate that this sort of extreme hacking is possible through TASing.
Oh, play it cool. Play it cool. Here come the space cops.
Skilled player (1707)
Joined: 9/17/2009
Posts: 4952
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
ais523 wrote:
"Total control" makes sense as a category by itself, and it's the usual name I've heard for it.
"Perfect Control"?
Banned User, Former player
Joined: 3/10/2004
Posts: 7698
Location: Finland
I find it curious that the Gameboy would execute anything from RAM at all. I would assume that the most rational design for a handheld console (or any console where the program code is not read into memory but executred directly from ROM cartridges) would be for the game ROM and the RAM being in completely separate address spaces. Does the Gameboy map the game ROM and the RAM into the same address space? Anyway, as cool as this demonstration is, I have to question its validity as a TAS. It does not finish the game, so it's not a TAS in the intended sense. It also doesn't have any well-defined goal. It doesn't actually "play" the game at all (more than what's absolutely necessary to modify the RAM and make the program counter jump there.) However, this is way too cool, way too popular, and showcases too well what tool-assistance is all about (breaking games), that there's no option but to publish it. I suppose that what I'm trying to say is that this needs its own category, separate from all the others. The "demo" category (which has been suggested many times in the past) would be a perfect fit for this. (Edit: Oh, and btw. As much as I love the ending, perhaps a better joke would have been if it had literally been a rickroll. With Rick Astley's image and all.)
Warepire
He/Him
Editor
Joined: 3/2/2010
Posts: 2174
Location: A little to the left of nowhere (Sweden)
Warp wrote:
Does the Gameboy map the game ROM and the RAM into the same address space?
The GameBoy puts everything on the same bus. So the program counter can be made to point anywhere you want, be it ROM, RAM, External Regs, Video RAM, Sound controllers...
Editor, Active player (296)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
Warp wrote:
I find it curious that the Gameboy would execute anything from RAM at all. I would assume that the most rational design for a handheld console (or any console where the program code is not read into memory but executred directly from ROM cartridges) would be for the game ROM and the RAM being in completely separate address spaces.
It is not really that surprising. Examples of other well-known platforms where there is no difference between RAM and ROM from the viewpoint of the CPU's instructions and addressing modes*: - NES - SNES - 8086 PC (and all of its descendants, including modern x86_64 PCs) - Commodore 64 You can even execute memory mapped I/O if you are so inclined. I have a test for NES which does exactly that: Ensures that a combination of I/O ports will return predictable values, and then transfers control to those addresses. The CPU will execute whatever opcodes corresponding to the values read from those I/O ports. On the PC you might execute code from the CGA/VGA video memory. (I believe this has been done in some demos.) *) Sure, on the NES there are addressing schemes which can only access a particular 256-byte region of the RAM, and the stack can only be in a very specific location in the RAM, but it doesn't mean you are limited to using those schemes for addressing the RAM, nor that all access is done through those schemes.
1 2 3 4 5 6 7 8