(Link to video)
Submission Text Full Submission Page Publication Page
I am Igor Guevara and if this TAS is not accepted I will eat TASVideos.
This TAS has been console verified by TiKevin83 and commentated by both luckytyphlosion and CasualPokePlayer during the 2020 PSR Marathon:

Introduction

This TAS is an improvement to the previous Catch 'Em All TAS. The primary improvements come from the use of a new farming method, called Faster Than LWA (or FTL for short) as well as an alternative way of acquiring the tools to set up this farming method involving Save Corruption.

Categories

  • 100% completion
  • One luck manipulation
  • Uses save data corruption
  • Corrupts memory
  • Major skip glitch
  • Heavy glitch abuse

Game Objectives

  • Emulator used: Bizhawk 2.4.2, Gambatte core, CGB-GBA mode for console verification
  • Aims for fastest completion time.
  • Catches Obtains Registers all 151 Pokémon as quickly as possible.
  • ...even more quickly than the previous TAS!
  • Does not modify Pokédex flags for completion
  • Does not use arbitrary code execution, arbitrary ROM execution, nor arbitrary memory corruption.

Description Notes:

  • Party Swap indices mentioned are all 0-indexed, while Item Underflow slots mentioned are all 1-indexed.
  • Party Swaps will use the shorthand notation A <> B, which means the Pokémon in slot A is swapped with the Pokémon in slot B.
  • This explanation makes heavy use of WRAM labels from the pokered disassembly (old commit with addresses) which use Hungarian notation, prefixed with w. Some HRAM labels are used as well, found in (hram.asm), and also some ROM labels, which can be searched using the GitHub search function.

Major Terminology/Concepts:

These explanations are partially copied from the previous Catch 'Em All TAS and MrWint's Save Corruption TAS.

Echo RAM:

The Game Boy has a region of memory called Work RAM, or WRAM, at $c000-dfff. The memory at $c000-ddff is mirrored to $e000-fdff, such that reading from $e000-fdff is equivalent to reading from $c000-ddff, and writes to $c000-ddff will appear in $e000-fdff and vice versa. Thus, whenever you see an address in the $e000-fdff range, it is implied that this really corresponds to the $c000-ddff memory range.

Quantity 255

When the game has to completely remove an item from the inventory, it shifts the items below in order to fill up the empty item slot, until it reaches the end of list terminator (marked as CANCEL). If an item with a quantity of 255 is below the tossed item, then the game mistakes the quantity of 255 for the end of list terminator and stops shifting the remaining items upwards. This leads to interesting side effects such as duplicating the 255 quantity item, or having another item represent the cancel button (despite not actually being the 0xff item). While the former effect mildy helps in some ways, the latter effect is relevant in obtaining Item Underflow.

Item Underflow/Dry Underflow

When the game merges two same items via swapping, it checks if the on-screen menu needs to be resized (note that this only refers to the current four items shown on screen). If the resulting list length is one, then the menu should be resized to 01 (only being able to use the first item). However, if a merge was somehow accomplished so that the resulting list length is zero (or any other value), then the menu is not resized. In normal gameplay, this does not matter as it is impossible to merge two items with only one item in the inventory. However, by having a non-CANCEL item act as the cancel button, we can merge two items together while having only one item in the inventory. By doing this twice, we can underflow the number of items in the inventory to 255.
Item Underflow allows us to modify the memory in range [$d31e, $d41d]. The powerful part of Item Underflow is that we can manipulate memory on the individual byte level, and the memory which we can manipulate can produce useful effects.

Party Overflow Swaps

Swapping Pokémon outside the first 6 slots is one of the memory manipulation techniques used. While not strictly necessary (it's actually possible to set up FTL with only Item and Box Underflow), Party Overflow greatly speeds up the setup. To see how, let us look at the actual mechanics of swapping Pokémon. When two Pokémon A and B are swapped, four things happen in this order:
  • The species is swapped (1 byte): $d164 + A <-> $d164 + B
  • The Pokémon data is swapped (44 bytes): $d16b + 44*A <-> $d16b + 44*B
  • The Pokémon OT name is swapped (11 bytes): $d273 + 11* A <-> $d273 + 11*B
  • The Pokémon nickname is swapped (11 bytes): $d2b5 + 11*A <-> $d2b5 + 11*B
Given that we can access slots from 0-255, we see that the range that we can modify with Party Overflow is $d164-fd6a. This actually encompasses all of WRAM (due to how Echo RAM works), which means that manipulating any memory in WRAM is as simple as performing a party swap. This does NOT violate Arbitrary Memory Corruption, as we are unable to modify HRAM, SRAM, or VRAM (as well as ROM), and needing to modify SRAM is actually vital to the setup.
Combined with Item Underflow, we can perform very precise memory manipulation by tossing item slots, then swapping Pokémon whose memory regions correspond to said item slots.
There are also some constraints we have to respect. Namely, in most cases we cannot perform any swaps which corrupt the following:
  • Pokedex flags: Corrupting these flags do not count as completion as part of the 100% rules.
  • wMapScriptPtr: Every overworld frame, the game switches to a predefined ROM bank depending on wCurMap, and then calls the pointer stored in wMapScriptPtr. Manipulating this value is generally seen as arbitrary ROM execution.
  • wCurMap: While modifying wMapScriptPtr is generally seen as arbitrary ROM execution, it is a question whether modifying wCurMap where the map bank is different but keeping the value of wMapScriptPtr unchanged actually constitutes arbitrary ROM execution. This TAS assumes that modifications of wCurMap such that the map ROM Bank changes counts as arbitrary ROM execution.
In addition, we need to warp to the Hall of Fame. The fastest way to do this is with a Connection Warp (a warp that happens when walking off the edge of the map), where we modify the connected map ID, then walk off the respective edge of the connection. There is also a base destination pointer associated with a connection and stored in memory, used to tell the game where to write the metatiles of the maps adjacent to the current map in the overworld map buffer in memory. The important part is that modifying this base destination pointer and then crossing a connection would constitute Arbitrary Memory Corruption. Thus, we must ensure that the destination pointer of the connection we choose stays untouched, which means we cannot perform any party swaps which modify this destination pointer.
Within the actual routing, a Python script was made to determine the party slot which controlled a section of memory, and whether swapping the slot would cause Pokédex corruption. The script (as well as some other scripts used) can be found here. Like MrWint's various scripts, these were never meant to be easily usable.

Lag reduction:

Part of the route is optimized around reducing item name lag, most notably the glitch item []j. with ID $00. This item has the laggiest name in the game, requiring 410220 cycles or 6-7 frames (depending on when the lookup occurred in the current frame) to look up. If the entire menu is redrawn with all []j. items (which happens when scrolling past the bounds of the visible window), then the game takes 25 frames just to print item names to the screen. Thus, we generally want to avoid seeing []j. items, which is done in the route by manipulating what is seen in the Item View, and also by using Party Swaps to swap lagless items into regions of $00 which we scroll through.

Explanation of FTL:

Every map has a map text pointer (wMapTextPtr), which is a table that contains pointers to the possible texts that a map can display (excluding things like text predefs[1]). To display a message in the overworld, the game calls DisplayTextID with the id of the text to display in hSpriteIndexOrTextID. This function may be called when talking to NPCs and signs, within map scripts (not covered here), and possibly other places. When talking to an NPC, hSpriteIndexOrTextID is loaded with the internal ID of the NPC's sprite, while with a sign, hSpriteIndexOrTextID is loaded with the sign specific text ID. Retrieving the individual text pointer associated with the Text ID is done by reading the (id-1)th pointer from the map text pointer table (i.e. [[wMapTextPtr] + (id - 1) * 2]). There are some reserved Text ID values (most notably, an ID of 0 corresponds to opening the start menu, which is why the -1 subtraction is necessary), but these aren't relevant.
Once the individual text pointer is retrieved, the game checks the first byte of the text. If it corresponds to a specific text script ID, then the game performs some alternate behaviour such as bringing up the Pokémon Center heal dialogue, and otherwise will pass the text pointer to the text engine (PrintText_NoCreatingTextBox, PrintText is the general function). One of these specific text script IDs, with a value of $f7, brings up the game corner prize menu.
In order to determine the prizes to display, the game uses hSpriteIndexOrTextID as an index to the prizes table. Ignoring TMs, the format of the Pokémon prizes table is as follows:
  • Each entry (prize selection) of the prizes table consists of two pointers, a pointer to the Pokémon entries table, and a pointer to a table describing the price of each Pokémon entry.
  • The format of the prize selection table is a sentinel terminated list (terminated by a $50 byte, also the string end value), which is odd because the menu was designed to hold exactly three entries, and does not allow selecting past the 4 menu slots. The format of the prices table is just a fixed list of three elements.
  • The list element of the Pokémon entries table is the internal Pokémon ID, while the list element of the prices table is a 2 byte BCD number. Naturally, the nth element in the Pokémon entries table has the price described by the nth element in the prices table.
In normal gameplay, reusing hSpriteIndexOrTextID poses no issue as the conversion from hSpriteIndexOrTextID to the prize table index is hardcoded around the prize vendors having sprite IDs 3, 4, and 5 (the conversion being text ID minus 3). However, if we were to use a custom map text pointer where the individual text pointer has an ID outside this range, then the game would read an out of bounds invalid prize selection pointer. Since the prize table is in ROM, there are only a finite number of prize selection entries. By sheer luck, one of the available price selections has a Pokémon entry table prize list which points to an item slot reachable within Item Underflow (a full list of invalid price selections can be found here). More specifically, this entry has a text ID of 59, a Pokémon entries pointer of $d3fa (where the range of addresses reachable in Item Underflow is [$d31e, $d41d]), and a price pointer of $a7cc. It quickly becomes apparent that we can engineer a Pokédex registering machine similar to LWA, where we convert an item quantity to the Pokémon we want to register, by starting at the highest Pokémon ID, obtaining a Pokémon, tossing downwards to the next valid ID, and repeating until we get all Pokémon registered. This setup, known as FTL, is much Faster Than LWA (explaining the name etymology), as we skip the ball toss animation and Pokédex registration screen.
A small note is that the way that Item Underflow maps addresses to item slots means that we'll be buying from the 2nd slot of the prize selection menu (instead of the first). This means that $d3fb will contain the quantity, and $a7ce-$a7cf will contain the prize price.
So our initial route looks something like this:
  1. Acquire Item Underflow
  2. Set up something which allows us to repeatedly pull up a textbox with a text ID of 59, such that the individual text pointer points to an $f7 byte
  3. Set the price of the 2nd slot and the number of owned coins to values where we can buy all the Pokémon we don't have.
  4. Acquire a Coin Case
  5. Perform FTL
  6. Warp to the Hall of Fame
The old route which used LWA and the current RTA route which uses FTL obtain the 255 stack required by Item Underflow by catching Missingno. twice via the Cooltrainer glitch (explanation provided from the previous TAS description). However, it turns out that it is possible to obtain Item Underflow almost immediately at the start of the game, by using Party Overflow obtained via Save Corruption. It was previously thought that getting Item Underflow via Party Overflow was impossible without corrupting Pokédex flags, because wPokedexOwned precedes wNumBagItems and the possible operations of Party Overflow Swaps mean that wPokedexOwned is corrupted if wNumBagItems is corrupted. However, we can instead corrupt wNumBoxItems and wBoxItems to obtain an item with a quantity of 0 or 255, which we can withdraw to perform standard Dry Underflow. The swaps required to corrupt box items do not corrupt wPokedexOwned. Note that the reason this isn't used in the current RTA route is because of legacy rules (mostly arbitrarily decided by me) which unconditionally disallow the type of save corruption at the start of the game.
Updating our current route sketch, we get:
  1. Save corrupt
  2. Corrupt box items to be able to withdraw a 0 or 255 quantity item
  3. Perform Dry Underflow
  4. Set up something which allows us to repeatedly pull up a textbox with a text ID of 59, such that the individual text pointer points to an $f7 byte
  5. Set the price of the 2nd slot and the number of owned coins to values where we can buy all the Pokémon we don't have.
  6. Acquire a Coin Case
  7. Perform FTL
  8. Warp to the Hall of Fame
The next step is to address how Step 4 will be achieved. Recall that we mentioned that overworld messages are predominantly displayed through NPCs, signs, or map scripts. Map scripts aren't really an option for reasons not mentioned here, and to be able to interact with an NPC with a sprite ID of 59, the memory location of the sprite data used to determine if we can interact with the sprite is within wTileMap (20x18 buffer of screen tiles), a highly volatile memory region[2]. However, the sign data format enables it to be easily manipulated for our purposes. The format of sign data is as follows:
  • wNumSigns: Number of signs in the current map, one byte. Must be set to a value greater than the sign entry we want the game to choose
  • wSignCoords: Y/X coordinates of each sign, two bytes per entry (16 entries total)
  • wSignTextIDs: Text ID of each sign, one byte per entry (16 entries total)
Thus, we can simply set wNumSigns to a non-zero value, and set the coordinates of a sign entry to coordinates reachable by the player and the sign text ID to 59[3]. This is assumed to be possible to do by manipulating some free space in Item Underflow, then using party swaps to swap the aformentioned space into sign data.
As for wMapTextPtr itself, modifying it via party swaps is infeasible (and probably slower), because it would corrupt wCurMap and/or wMapScriptPtr, which could cause a crash, but more importantly would potentially violate arbitrary ROM execution. However, wMapTextPtr is accessible via Item Underflow, thus we can just swap an item into the memory corresponding to the pointer. To determine the new value of wMapTextPtr, we simply follow the map text pointer format discussed earlier. Firstly, we need an individual text pointer in memory (ROM or RAM) which points to an $f7 byte. Since we're using a text ID of 59, we must set the map text pointer to a value such that the 59th (1-indexed) entry is the aforementioned individual text pointer. A Python script (linked within the collection of scripts) was made to find all map text pointers which fit this criteria (such that only ROM locations are referenced). Out of the nine pointers generated, one of them, $10f3, can be generated completely with items derived from wRivalName (TM43 x??, later tossed down to TM43 x16), accessible via Item Underflow.
Updating our current route sketch, we get:
  1. Save corrupt
  2. Corrupt box items to be able to withdraw a 0 or 255 quantity item
  3. Perform Dry Underflow
  4. Stage some values in Item Underflow to be swapped into Sign Data
  5. Set wMapTextPtr to TM43 x16, corresponding to $10f3 in little endian.
  6. Set the price of the 2nd slot and the number of owned coins to values where we can buy all the Pokémon we don't have.
  7. Acquire a Coin Case
  8. Perform FTL
  9. Warp to the Hall of Fame
Next is resolving Step 6. What is interesting is that the coins table is within SRAM, which is normally disabled (no reads/writes) in well programmed games. However, when the game loads a pic (Battle sprites), it uses SRAM as a decompression buffer, and then leaves SRAM enabled. From here, SRAM will stay enabled until the game performs a process which disables SRAM, which only happens after the game saves, and when loading the player battle back pic. This is actually not the case in Pokémon Yellow; the game will properly disable SRAM after decompressing a pic. Anyway, the fastest way to load a pic (and thus enable SRAM) is by viewing the Trainer Card, as that loads the player front pic.
Now, it seems like there isn't a way to corrupt $a7ce-$a7cf. Party Overflow, despite its far reaching powers, cannot actually corrupt any SRAM address. Given the circumstances, the only two reasonable options[4] to corrupt this value are as follows:
  1. View glitch Pokémon pics with dimensions that exceed the allocated decompression buffer.
  2. Crash the game while SRAM is open.
During the initial routing, I immediately dismissed looking at glitch pics, as it seemed too complex. Upon writing the description, I decided to take a stab at it. My understanding of glitch pics is that certain glitch Pokémon have pic dimensions larger than what the game has reserved space for (i.e larger than a width/height of 7 tiles). When these glitch pics are loaded, a buffer overflow occurs in order to decompress the large sprite. The normal buffer ends at $a498, and we need to corrupt $a7ce-a7cf, so it is reasonable to assume that a slightly larger sprite would be able to reach that address. However, the results of loading every single glitch Pokémon's pic proved unfruitful: Out of all glitch Pokémon, only two are able to corrupt up to $a7cf (and both use the same input pointer, so it's really only one). Only the area allocated for pics with normal dimensions is cleared when a pic is loaded, and observation of the full buffer in BGB's memory viewer showed that viewing the pic multiple times caused different output in the out of bounds buffer, leading myself to conclude that a feedback process was occuring between the previous buffer and the resulting buffer. However, repeatedly loading the pic for multiple attempts did not produce a low enough price (needs to be somewhere under $0200), and I concluded that it was probably not worth it. A possibility would be to load multiple different glitch pics, which by observation cause diferent corruptions, but this would take far too long to test manually (recorrupting wPlayerCoins isn't an option as explained later).
The second option is to crash the game while SRAM is open. More specifically, perform something for the game to call an rst vector. rst vectors are basically one-byte predefined calls, used to save space over traditional 3 byte calls. However, Game Freak opted not to use rst vectors, and left them as rst $38. This means that if any rst vector is called, the game will call rst $38, which calls rst $38, and so on infinitely. This will push the return address of the call onto the stack infinitely, eventually wiping all of memory. The important part about rst $38 is that:
  1. As it writes to every possible address, it will corrupt $a7ce-f (This is probably not AMC, as you can't choose a single address to write to)
  2. The return addressed pushed onto the stack is $0039, which is a low enough value to be able to buy all Pokémon with a high coin count.
However, it isn't as simple as using any glitch item which calls rst $38; this is because of the VBlank interrupt.
Basically, VBlank is an event that occurs at the end of every frame, where the CPU is free to access video and sprite memory that would normally be disabled during the rest of the frame (except in HBlank/Mode 2, not relevant). GB games can enable a VBlank interrupt, where at the start of this VBlank period, the Game Boy will forcefully switch execution to a user defined segment of code. In the case of Pokémon Red/Blue, and presumably other games, this code will copy video related buffers to video memory, and also update per-frame events like audio and joypad.
So, even though the game will hammer rst $38 ad infinitum, eventually the VBlank interrupt will kick in. This becomes a problem when the return address of rst $38 corrupts wAudioROMBank to $39. First, let us explain what a ROM Bank is. A ROM Bank is basically a $4000 byte swappable region of ROM (located at $4000-7fff), which allows the Game Boy to access more ROM than the address range of the Game Boy ($0000-ffff)[5]. To access data in a ROM Bank, the game must switch to the ROM Bank, by writing the ROM Bank number (also referred to as ROM Bank) to $2000-3fff. When the game needs to update audio, the game will first switch to the ROM Bank in wAudioROMBank, and then will call one of three routines depending on the value of wAudioROMBank. By default (if wAudioROMBank doesn't match the other two ROM Banks), it will call the function address for the third audio routine. This means that if wAudioROMBank is an invalid value, the game will use the function address ($5177) for the third routine, but with a ROM Bank not matching the ROM Bank of the audio routine, thus running completely garbage data as code. ROM Bank $39 is completely empty, so when the game calls 39:5177, it will fall through all the way to Video Memory, performing ACE. While some crashes will still manage to run into an rst $38 due to disabled Video Memory, others will just lock up by running into a Stop opcode (There is also a small period after each scanline where Video Memory is accessible), which is basically a hard lock in the context of Pokémon[6]. Regardless, as ACE is performed, we need to find alternate ways of crashing that don't involve wAudioROMBank corruption.
Actually, the method used is a method of wAudioROMBank corruption. When the player gets off the bicycle, the game needs to play the default map music. When the game runs the routine to do so, it checks if [wAudioROMBank] equals [wMapMusicROMBank], and writes [wMapMusicROMBank] to [wAudioROMBank] if not. We can invoke a crash by modifying [wAudioROMBank] to a value such that within VBlank, the audio routine called will eventually execute an rst $38. While we said earlier that wAudioROMBank corruption was bad, setting up a crash like this is good, because interrupts are disabled while in an interrupt, so we do not have to worry about another VBlank interrupt happening.
Note that the number of invalid ROM Banks which work is reduced because when switching back to the default map music, the game will call another routine to play the map music out of a set of three routines similar to the routines in VBlank, thus the requirements for the invalid ROM Bank are that it must not crash when running the play sound routine, and must crash when running the update music routine in VBlank.
Out of all the invalid ROM Banks, only one, with a value of $1d, fits this criteria. wMapMusicROMBank is accessible within Item Underflow, so we can modify this value by swapping an item into the Item Underflow slot which corresponds to wMapMusicROMBank. We can use a value of $9d, generated from items accessible in wRivalName, as Red/Blue only use $40 ROM Banks, and ROM Banks >= $40 are truncated modulo $40. We achieve Item Underflow via Party Swaps. This corrupts dex flags temporarily, but the game will revert back to the previous save after the intentional crash (and dex flags are also corrupted via the rst $38 anyway).
In order to be able to ride the bicycle, wCurMapTileset must be a value which allows bicycling. We simply modify this value while in Item Underflow.
Note that when writing this, I realized that it would be simpler to just use Party Swaps to swap a value into wAudioROMBank, but it would require a near complete rewrite of the TAS (as we would skip getting $9d and would replace it with a less laggy item, thus timings when scrolling past the item would have to be adjusted)
Updating our current route sketch, we get:
  1. Save corrupt
  2. Acquire Item Underflow via Party Swaps
  3. Swap $9d into wMapMusicROMBank
  4. Acquire a Bicycle
  5. Modify wCurMapTileset so we can ride the Bicycle
  6. View the Trainer Card
  7. Get on and off the Bicycle, crashing the game and setting [$a7ce-a7cf] to $0039. (We then reload the save)
  8. Corrupt box items to be able to withdraw a 0 or 255 quantity item
  9. Perform Dry Underflow
  10. Stage some values in Item Underflow to be swapped into Sign Data
  11. Set wMapTextPtr to TM43 x16, corresponding to $10f3 in little endian.
  12. Set the number of owned coins to a value where we can buy all the Pokémon we don't have.
  13. Acquire a Coin Case
  14. Perform FTL
  15. Warp to the Hall of Fame
Step 12 can be achieved via a simple Party Swap. Step 14 will be explained in the upcoming section, while Steps 13 and 15 will be explained in the actual route explanation.
Thus, we have a final route now. However, we can perform some additional optimizations to save some time.
Optimization 1: Optimize FTL itself. First, a quick explanation of how the basic FTL loop works:
  1. Toss the item quantity to the ID of the Pokémon wanted
  2. Exit the Item Menu and buy the Pokémon from the Game Corner menu
  3. Open the Item Menu
  4. Repeat until all Pokémon are acquired
We can optimize this in two ways. First, let Repels be the item to be tossed, as using a Repel is faster than tossing an item. (in fact, using two Repels in succession is still faster than tossing two, by 4 frames). Secondly, make use of the glitch item "8 8" to quickly exit the menu. 8 8 is a special item. The short explanation[7] is that 8 8 will return to the main overworld loop if used in the overworld, which is faster as we skip needing to exit the item and start menu.
Optimization 2: Skip needing to deal with Pokémon boxes. Recall that the prize selection table of a prize selection entry is sentinel terminated by a $50 byte. When loading the prize menu, the game will copy the prize selection table to memory, starting at wPrize1. This means that if the terminator to the prizes table appears much later, then the game will happily copy data up until the terminator and overwrite whatever memory that is past the prize table in memory. It turns out that wPartyCount is not that far away from wPrize1 in memory, so we could intentionally set the terminator to be further than expected so that whenever we open the prize menu, wPartyCount is overwritten with zero. This means that Pokémon will never be sent to the box, thus the "sent to box" message will never be displayed, plus we do not need to worry about changing boxes either. As a side effect, once we start talking to the prize menu sign, we cannot perform any further out of bounds party swaps (since wPartyCount will be set to zero). This is the reason why we cannot use a larger prize price and use party swaps to refresh wPlayerCoins. Note that we can't set the terminator too late, as right after Party Data is Pokedex Flags, which we are not allowed to corrupt.
Optimization 3: Faster Hall of Fame animation. Before warping to the Hall of Fame, we buy an additional glitch Pokémon with ID of $ff. $ff indicates the end of the party species list (wPartySpecies), so even though this $ff represents an actual Pokémon, the Hall of Fame animation code will interpret this $ff as the end of the party, thus skipping the Hall of Fame Pokémon animation.
The final route sketch is now:
  1. Save corrupt
  2. Acquire Item Underflow via Party Swaps
  3. Swap $9d into wMapMusicROMBank
  4. Acquire a Bicycle
  5. Modify wCurMapTileset so we can ride the Bicycle
  6. View the Trainer Card
  7. Get on and off the Bicycle, crashing the game and setting [$a7ce-a7cf] to $0039.
  8. Corrupt box items to be able to withdraw a 0 or 255 quantity item
  9. Perform Dry Underflow
  10. Stage some values in Item Underflow to be swapped into Sign Data
  11. Set wMapTextPtr to TM43 x16, corresponding to $10f3 in little endian.
  12. Set the number of owned coins to a value where we can buy all the Pokémon we don't have.
  13. Set a $50 byte somewhere in memory such that wPartyCount is corrupted to zero upon opening the prize menu, but Pokedex flags remain uncorrupted.
  14. Acquire a Repel based item and the glitch item 8 8, and swap these into the FTL item slots.
  15. Acquire a Coin Case
  16. Perform FTL
  17. Buy the glitch Pokémon $ff.
  18. Warp to the Hall of Fame
With the ideas behind the route explained, let us go to the actual route. The actual route does some of these steps out of order, but still follows the genera idea of the route.

Route Explanation

Version choice

Red version is chosen over Blue because of a faster Trainer ID manipulation, the Charmander cry in the intro is faster than Squirtle's, and Red version has the shortest default player name (displayed only twice).

Route

As a reminder, $d31e-$d42d is the region of memory accessible by Item Underflow, so whenever a bullet point mentions addresses within that range without mentioning a pokered label, it is implied that the address will be scrolled through or accessed in some way.

Intro to Crash

  • To my dismay, we are obligated not to clear save data, as per TASVideos decree, which to me is a sign of MORAL IMPURITY.
  • Options are not set as there is no need to. We never enter any battles and we can advance text to the fastest (intended) speed by holding A or B.
  • A Trainer ID of $7145 is manipulated. The $71 is not important, while the $45 corresponds to the internal ID of the Coin Case item, required to access the prize menu.
  • The player name is irrelevant and is named the default RED for speed.
  • The rival is named ?!♂;///. The ?!♂ portion is not explicitly required, but the internal IDs correspond to TMs which produce no lag in the item menu, and it is the fastest pattern of characters corresponding to TMs which can be typed in while moving the cursor to the ; character. The ; corresponds to $9d, which is the value decided earlier that will be swapped into wMapMusicROMBank. The 7th slash is used as the TM43 x16 to be swapped into wMapTextPtr. The remaining slashes are just lagless filler items.
  • Immediately, we save corrupt. This fills our entire party with $ff bytes, including wPartyCount. The first six Pokémon being filled with $ffs will become useful for item lag reduction.
  • Upon reloading, we walk to a specific part of the map. This sets wXCoord to $6, corresponding to the Bicycle item in Item Underflow, and wYBlockCoord to 0, corresponding to the Bicycle item's quantity.
  • We open the party menu and swap 0x1 with 0x9, and 0x9 with 0xa. This sets wNumBagItems to $ff, giving us Item Underflow, and also fills a large portion of item slots with $ff (as opposed to $00), which when rendering the item menu the game prints CANCEL and stops rendering any further items. These item slots would normally be filled with []j., thus saving time by not needing to lookup the []j. item name.
  • Q r 4 h i ($9d) is selected and swapped with wMapMusicROMBank (item) Ultra Ball x0. Q r 4 h i was swapped from wRivalName into an earlier portion of memory as a side effect of the above two swaps.
  • The Bicycle x0 in wXCoord (item) and wYBlockCoord (qty) (manipulated earlier) is swapped with []j. x4 in wCurMapTileset (qty). The quantity of the Bicycle is one which allows biking.
  • We get on the Bicycle, then view the Trainer Card to enable SRAM.
  • We get off the Bicycle, crashing the game and setting [$a7ce-a7cf] to $0039.

Party Swaps 1

  • Upon reloading the game, we open the party menu to swap some Pokémon.
  • Swap 0x0 <> 0xb4. This has the effect of writing $ff to wLowHealthAlarm which will cause the game to skip waiting for sound effects that the game would normally wait for, thus saving time.
  • Swap 0xc8 <> 0x1. This swaps $ffs from slot 0x1 into the region $f3cb-f3f6, which will be scrolled through in order to get to the slot corresponding to $d3fa for FTL.
  • Swap 0x2 <> 0xa. This just swaps $ffs into $d323-d34e (near the start of wBagItems) for lag reduction purposes.
  • Swap 0x3 <> 0x1b. This swaps $ffs into $d39c-d3a6 and $d3de-d3e8 (some connection/misc data that is mostly $00) for lag reduction purposes.
  • Swap 0x39 <> 0x3a and 0x3a <> 0x44. The first swap modifies collision allowing us to walk off the north side, and the second swap corrupts the first three Box Items to items we can use to perform Dry Underflow (as well as allowing us to access these three items)[8].
  • Swap 0x19 <> 0x4, swapping $ffs into $d386-d390 and $d3c8-d3d2 for lag reduction.
  • Swap 0x5 <> 0x18. This swaps $ffs into $d37b-d385 and $d3bd-d3c7 for lag reduction, AND modifies wPlayerCoins to $ffff, which is plenty enough to buy the 152 Pokémon we need.

Item Manipulation

Due to the large amount of $ffs in the Item Underflow region, some items will not be visible as they're manipulated with, e.g. the TAS may toss TM43 x64 from TM43 x80 even though it isn't seen in the Item Menu View.
  • The party menu is exited, and we make our way to the PC. Accessing the PC from the right is faster as wXCoord is 1 instead of 0 in this position, corresponding to a Master Ball (lagless) instead of a []j. in Item Underflow.
  • In the PC menu, the Withdraw Menu is accessed. Master Ball x1, []j. x176, and X Attack x255 are withdrawn in this specific order. The Master Ball and []j. are just items used to get to 3 items in the player inventory, as required for Dry Underflow. The quantity of Master Ball x1 is irrelevant, while withdrawing []j. x176 leaves []j. x80 in the PC, which the quantity is used as the $50 byte terminator required by the Prize Table. The X Attack x255 is used as the 255 stack required for Dry Underflow.
  • The PC is exited from, and the Item Menu is opened.
  • Dry Underflow is performed. []j. is tossed first to avoid displaying the name later, then Master Ball x1.
  • TM55 ($ff) x254 is tossed from TM55 x255 in slot 4 to get TM55 x1. The resulting quantity is used as the Y Coord of the sign used for the Prize Menu. The X Coord of the sign is the $ff byte right after slot 4 quantity, which is the west edge of the map.
  • X Attack x0 is swapped with the Coin Case in wPlayerID+1 (item). For the game to recognize that the player has a Coin Case, it must be placed before the first $ff item (TM55/CANCEL) in the player's bag.
  • X Attack x0 is then swapped with []j. x3 in wOptions (quantity). This sets the bits responsible for text speed to 0, meaning that all subsequent text will print instantly.
  • TM55 x196 is tossed from TM55 x255 in slot 20 ($d344-5) to get TM55 x59, which will be swapped into the text ID of the Prize Menu sign.
  • TM43 x64 is tossed from TM43 x80 in wRivalName+6 (item+quantity) to get TM43 x16.
  • TM43 x16 is swapped into TM07 x64 in wMapTextPtr (item+quantity). This fulfills the requirement to set the map text pointer to TM43 x16
  • []j. x137 is tossed from []j. x255 in the slot (quantity) corresponding to the map ID of the north connection (quantity) to get []j. x118. 118 is the map ID of the Hall of Fame, so performing a north connection warp will cause the player to warp to the Hall of Fame.
  • Full Restore x64 is swapped with []j. x0 in the slot corresponding to the north connection coordinate alignments (Y: item, X: quantity). The coordinate alignments are values which are added to the player's X and Y coordinate upon performing a connection warp, in order to adjust the player's coordinates to the correct position in the new map, relative to the coordinates of the old map. Before the swap, these values were both zero. This is important for the Y coord alignment value, as upon entering the Hall of Fame, the player is scripted to automatically walk upwards. Without modifying the Y coord alignment, upon warping to the Hall of Fame, the player's y coord would stay at 0, and then after one step upwards, the game would try to perform another north connection warp. Since the Hall of Fame has no north connection, the north connection map ID has a default value of 255, so the game will try to load the invalid map 255, crashing the game. Swapping Full Restore x64 (the closest safe item to swap) sets the player's Y coord alignment to 16 (ID of Full Restore), which is more than enough room for the scripted movement to play out without loading another map.
  • TM55 x225 is tossed from TM55 x255 in slot 50 to get TM55 x30.
  • TM55 x124 is tossed from TM55 x255 in slot 51 to get TM55 x124. The resulting quantities of the above and this TM55 correspond to the IDs of Repel and 8 8. A party swap will be performed later which converts these quantities into items and places the Repel quantity into $d3fb, and the 8 8 item right below the Repel. Both items will have a quantity of 255, which is important for reasons explained later.
  • The cursor is scrolled to the item slot corresponding to $d3fa-d3fb.

Final Party Swaps and FTL

  • The party menu is opened one last time.
  • Swap 0x13 <> 0xa. This swaps the item memory manipulated earlier into the actual sign coordinates and sign text ID, and also swaps an $ff into wNumSigns.
  • Swap 0x12 <> 0x1d. This swaps the manipulated TM55 x30 into the first FTL item slot ($d3fa-d3fb) and TM55 x124 into the item under the first FTL slot ($d3fc). Due to how nicknames are aligned, this swap converts quantities into items, and vice versa, thus placing a Repel x255 and 8 8 x255 into the aforementioned item slots.
  • The party menu is exited, and the trainer card is viewed, enabling SRAM for the prize price.
  • The item menu is closed, and the player walks one to the left, in position to start FTL. Doing this after the following step would be faster, but it's something I missed in the original route.
  • The item menu is opened again, and Repel x65 is tossed from Repel x255 to get Repel x190. 190 is the ID of Victreebel, and is the Pokémon with the highest ID.
  • 8 8 is used to return us to the Overworld. From here on, farming commences. By coincidence, the IDs of Repel and 8 8 map to Pokémon we need (Tangela and Metapod), so we buy those from the first and third slots respectively, instead of the Repel quantity slot.
  • Once all Pokémon are farmed, the Repel is used one last time. The last Pokémon obtained was Rhydon, which has an ID of 1, so using the Repel will remove it from the inventory, and all items below the Repel will be shifted up. Due to the mechanics of x255 quantities, an 8 8 x255 will take the place of the Repel x1. One final Pokémon is bought (known as 'M), which has an ID of $ff.
  • The player takes the shortest path to the north edge of the map allowed by the collision to warp to the Hall of Fame.
  • Because of the $ff Pokémon, no Pokémon are animated in the Hall of Fame.

Possible optimizations

  • Faster rst $38 Crash: As mentioned in the route, it is possible to use party swaps to overwrite wAudioROMBank. This would save at most 10 seconds by my estimate.
  • Achieving Instant Text earlier: While this could speed up the interactions before Instant Text is achieved in the current route, it turns out that getting Instant Text via Party Swaps is not that trivial. In addition to setting the low 4 bits of wOptions to zero, bit 0 of wLetterPrintingDelayFlags must be 1, and bit 1 must be 0. Finding this combination in memory to swap into wOptions and wLetterPrintingDelayFlags turned out to be very difficult.
  • Faster Hall of Fame: There are other strategies to warp to the Hall of Fame such as a tile warp, or by using the glitch item 10F combined with modifying wCurMap. In addition, we could also set wHallOfFameCurScript to 2, which would basically skip to the Hall of Fame Oak Cutscene.
  • Better lag optimization: Much of the lag optimization in this route was not actually timed to be faster, mainly out of laziness but also because the amount of possible permutations of party swaps would be too exhausting to test by hand.
  • Better setup in general: Glitch setups with Party Overflow in general have not been explored much. Perhaps there is a faster combination of party swaps that is yet to be discovered.

Final notes

As a fun aside, with the old TAS, at least you had all 151 Pokémon somewhere in the Box or Party. With this TAS, we achieve a whole new level of pointlessness:
  • We don't actually get to keep the Pokémon we buy, since wPartyCount is cleared every time we buy a Pokémon, so the only purpose of this effort is just registering all 151 Pokémon.
  • To check if a save exists, the game checks if wPlayerName is terminated with a $50. When the prize entry table is copied to memory, it overwrites wPlayerName along the way to wPartyCount, meaning that wPlayerName is now unterminated. This means that the save won't even load after we beat the game!
And despite the massive improvement due to the use of Save Corruption, I don't really enjoy the setup from an aesthetics point. The setup to set the prize prices to $0039 properly captures the magic of Gen 1 Glitch Runs, but this portion is short lived. The "Party Swaps 1" section is just too unidentifiable to the user, namely that the action of swapping out of bounds Party Pokémon produces very little visual feedback, which doesn't live up to the craziness factor of the Item Manipulation seen in other runs. The Item Underflow section is a bit better, but compared to other glitch runs there's a lot less substance in the item manipulation (a good portion of it is mostly scrolling), and a good chunk of the item manipulation is just tossing TM55 quantities to be swapped with Party Swaps, which is pretty uneventful and doesn't contain the craziness factor of more complex item manipulations required in other runs. Namely, if you look at the main setup portion of the RTA (Classic) Catch 'Em All run, the item manipulation and techniques required are much more visually striking to me compared to this route.
The farming part of the run is actually pretty cool to look at, mainly because the farming loop requires precise menuing which would be very difficult for a human to accomplish, which looks even more impressive given the menuing speed. I might even say that it outranks the setup portion of the run, apart from the setup to set the prize prices to $0039 (which would be routed out in a future route anyway).

Credits

  • CasualPokePlayer: Actually created the input file, based on the route, and also suggested minor improvements to the route.
  • luckytyphlosion: Coming up with the route for the TAS, creating the FTL strategy, and writing most of the description. Like the previous TAS, I did not touch a single bit of input.
  • MrWint: Took some of the description from his Save Corruption TAS and used it here.
  • BGB: Game Boy emulator with an amazing debugger that helped with routing out the TAS.
  • the pokered disassembly: General massive help in documenting the game's code allowing numerous tricks, including the FTL strategy, to be discovered.
  • Igor Guevara: The true creator of this TAS.

Suggested Screenshots

(Preview only, do not use these screenshots, since they were taken from the YouTube encode)

Footnotes

[1] - Text predefs are a global list of texts handled by alternative mechanisms such as hidden objects
[2] - Actually, my initial assumption is that only 15 NPCs are checked (thus only IDs from 1-15 could be used), but upon further investigation it turns out that the number of NPCs checked is equivalent to the value in wNumSprites.
[3] - Also, the coordinates of the sign entries before the entry slot we choose must be different than the coordinates of the chosen sign, otherwise the coordinates would match up with the previous sign. This turns out to be irrelevant since we just use the first sign entry.
[4] - Another option would be to load a battle with a glitch sprite, but this isn't feasible because the expanded party causes the battle to eventually crash.
[5] - There's also the home bank, located at $0000-3fff and cannot be swapped, but this isn't relevant.
[6] - The Stop opcode switches the Game Boy to "very low power standby mode", which can only be interrupted with a button press, however, button presses are only enabled for a very short time during VBlank, so under normal circumstances the game will never resume from a Stop opcode.
[7] - The long explanation (which you may skip if you so desire), is that what 8 8 does depend on when it is used, as the exact location where it jumps is within the VBlank routine. What it does in the VBlank routine isn't so important, rather, it's what it does at the end which is important. Basically, the Game Boy has a stack, which operates similar to a real life stack of papers. You can add data to the top of the stack via the push opcode, and you can retrieve data from the top of the stack via the pop opcode. The call and rst opcodes will push where execution returns after a function call, and all return opcodes will return to the address stored onto the stack. Since the VBlank routine can happen at almost any time, we need to save the registers (internal "global" variables) onto the stack. At the end of VBlank, these registers are restored, and execution returns to the addressed pushed onto the stack (as part of how interrupts work). However, where 8 8 jumps to is in the middle of the VBlank routine, after the registers are pushed. So at the end of VBlank, the game will pop off values off the stack which were saved by other routines, and returns to whatever is at the top of the stack. When used in the overworld, by sheer chance the game will return to where the game resumes execution after closing the start menu (location within the pokered disassembly here), in essence returning back to the overworld.
[8] - The 0x39 <> 0x3a swap was intended as an intermediate swap, removing $ffs so that tossing an item in Box Item Underflow would cause an item with a large value to shift into the item slot for wPlayerCoins. It turns out this is irrelevant because another Party Swap swaps $ffff into wPlayerCoins anyway. However, the 0x39 <> 0x3a swap also modifies collision such that it is possible to step off the map from the north side, required as part of the Hall of Fame warp. Initial tests suggest that this simple swap is faster than trying to manipulate collision in Item Underflow to allow warping to the Hall of Fame, so this swap still turns out to be faster.
Samsara: okay but what about that guy who's gonna eat tasvideos
Samsara: Aw, here it goes.
Samsara: Y'know I should probably take this one back, honestly.
Samsara: This is actually surprisingly straightforward of a judgement. Although this run adds save corruption to the already massive pile of glitches that the published run happens to be, looking at them side-by-side more or less tells me that they're working under the same general principle, just using different methods. In layman's terms, the published run catches 'em all, while this run buys 'em all, which is a perfectly valid method of obtaining Pokemon. It's just used in an INCREDIBLY INVALID-LOOKING WAY. As far as the game is concerned, all 151 Pokemon are obtained. As far as we're concerned, they are all obtained through in-game methods and not ACE. ACE is not achieved at any point in this run, despite the memory/save corruption, which is similar in execution to the currently published run. The save ends up too corrupted to even load once the game returns to the title screen, but... Well, we've had worse on the corrupted save front, so that's not an issue either.
All in all, accepting as an improvement to the published run (remaining in Vault due to the middling feedback), and what the hell has happened to this game from my childhood.
Spikestuff: Publishing.

Show more

Hide
1 2 3
Posted: 5/17/2021 5:14 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
Acumenium wrote:
CasualPokePlayer wrote:
BigBoct wrote:
If I was to watch this movie without reading the submission text, I would find the setup to be doing very similar things to [3358] GBC Pokémon: Yellow Version "arbitrary code execution" by MrWint in 05:48.28, which is an ACE playaround, and I would question what exactly you are refusing to do that the other movie permits.
Visually perhaps, but how it's technically done is nowhere alike. This movie does not setup any ACE payload (i.e., it does not setup anything that jumps to RAM), rather it just does a ton of memory manipulation (which given the currently published movie, memory manipulation is allowed for 100% categories as long as it is not arbitrary memory manipulation(/corruption)) to just setup an invalid game corner to farm Pokemon.
Is there any particularly difference between arbitrary code execution and rearranging and redirecting memory to do things in a specific way? I feel like "memory manipulation" is just "ACE, but self-limited". The same glitches are used to start both, so they appear the same. ACE itself is achieved via rearranging and redirecting memory addresses in a way that it runs a program based on it... which this is doing. You made a payload that when using a certain glitch item, loads a glitchy Game Corner, which reads other parts of RAM (your inventory, specifically, it seems a Repel item count?) to determine what the prize is. Defining ACE to just be "when you make Tetris or Pong" seems arbitrary in itself, and thankfully it appears I am not wrong with this specific part from what I've seen some Judges like Mothrayas say in other threads. I don't see at all how this run isn't ACE. I do have one other question, is this save file okay? I know you can eventually fix the "255 Pokemon" and even buggy boxes, but the money seems blanked out. Is it a very high but playable number? Or is it 0? Or is it not actually usable anymore?
"Limited" is the exact point on why limited memory manipulation is allowed. The reason ACE (along with ARE/AMC) is banned is due to the fact you can in principle do anything with it, rendering any sort of restriction like "100%" moot. If it's limited, by principle you cannot just do anything with it, thus this mootness does not apply. Also, can you please actually read the submission text? All this "glitch item" does is jump to the overworld loop. That's it. It's just an optimization so the start menu doesn't have to be closed to interact with the Game Corner (which is a glitched invisible sign that is set up to act like a Game Corner). "Defining ACE to just be "when you make Tetris or Pong" seems arbitrary in itself, and thankfully it appears I am not wrong with this specific part from what I've seen some Judges like Mothrayas say in other threads. I don't see at all how this run isn't ACE." Well that is not how I would define ACE at all, here is my own definition: Arbitrary Code Execution: If the PC (Program Counter) touches any RAM currently visible on the system bus, it is ACE. "RAM" is definable as any memory or I/O (Input/Output, which isn't RAM per se, but in spirit of ACE it can act like such) which is R/W (Read/Write). However, code put into RAM by the game and executed is exempt from this, unless this code is at all modified by the player and executed, which then it becomes ACE. This is not done, and therefore there is no ACE involved with this run. Again, the entire setup is explained in the submission text, which you did not bother to read apparently. Also the save file state is explained in the submission text. The money is probably some high number (I don't know, I don't care, it doesn't affect loading the save file anyways). Also, the submission text explains the Game Corner ends up overwriting the party count (we don't have "255" Pokemon anymore) and name (as explained in the submission text, this means the save file can't be loaded since the game uses a valid name as a "check if save file exists". If it was a valid name anyways, it would load the save file just fine with the completed dex, so probably doesn't mean anything for validity of the run.)
Posted: 5/17/2021 6:05 AM
Quote
FractalFusion
Editor, Expert player (2073)
Joined: 6/15/2005
Posts: 3282
Glitchy TAS of 2013 Gameboy TAS of 2013 PSX TASer of 2010
Acumenium wrote:
ACE itself is achieved via rearranging and redirecting memory addresses in a way that it runs a program based on it... which this is doing.
Literally all code runs a program based on something. That doesn't say anything. :) From my understanding, ACE is a very narrowly defined type of game-breaking state (distinct from other types of game-breaking states), based on what is being done (and not what *can* be done). ACE is a state where any desired sequence of values the author so chooses is promptly written to any desired addresses in RAM, and is immediately executed thereafter (or such equivalent power). Usually "promptly" is understood to be in the order of at least one byte per frame. As far as I know, none of the non-playaround game-end glitch TASes even use the term "arbitrary code execution" as a branch. They just use "game-end glitch", even if the glitch used has the potential to cause ACE. This is because defining ACE in terms of potential causes is far too vague (Ex: Many games have ACE exploits in them. Does that make playing the game itself "ACE", because the game is capable of ACE? If not, where do we draw the line?) That being said, Pokemon is so broken that I can't tell "how close to ACE" a glitch is. And I'm not familiar with the glitches in this TAS, so I certainly can't tell you exactly what limitations are being used here.
Posted: 5/17/2021 6:20 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
FractalFusion wrote:
Acumenium wrote:
ACE itself is achieved via rearranging and redirecting memory addresses in a way that it runs a program based on it... which this is doing.
From my understanding, ACE is a very narrowly defined type of game-breaking state (distinct from other types of game-breaking states), based on what is being done (and not what *can* be done). ACE is a state where any desired sequence of values the author so chooses is promptly written to any desired addresses in RAM, and is immediately executed thereafter (or such equivalent power). Usually "promptly" is understood to be in the order of at least one byte per frame.
Or just: Arbitrary Code Execution: If the PC (Program Counter) touches any RAM currently visible on the system bus, it is ACE. "RAM" is definable as any memory or I/O (Input/Output, which isn't RAM per se, but in spirit of ACE it can act like such) which is R/W (Read/Write). However, code put into RAM by the game and executed is exempt from this, unless this code is at all modified by the player and executed, which then it becomes ACE. And if you want to go ask about the other two ACE-level glitches (ARE/AMC), here's definitions for them here too: Arbitrary ROM Execution: If a glitch can cause the PC touch any byte in ROM currently visible on the system bus, it is ARE. "ROM" is definable as any memory or I/O given by the game medium which is Read Only. Arbitrary Memory Corruption: If a glitch can corrupt any byte in writable memory or I/O currently visible on the system bus, it is AMC.
Posted: 5/17/2021 6:45 AM
Quote
FractalFusion
Editor, Expert player (2073)
Joined: 6/15/2005
Posts: 3282
Glitchy TAS of 2013 Gameboy TAS of 2013 PSX TASer of 2010
CasualPokePlayer wrote:
Arbitrary Code Execution: If the PC (Program Counter) touches any RAM currently visible on the system bus, it is ACE.
What about glitches that cause the Program Counter to enter RAM, but the author does not quite have the control to modify that part of memory? (This is mostly hypothetical, since a game that is broken enough to have such a glitch will almost certainly have a glitch that leads to the type of ACE which I was talking about.)
CasualPokePlayer wrote:
Arbitrary ROM Execution: If a glitch can cause the PC touch any byte in ROM currently visible on the system bus, it is ARE.
CasualPokePlayer wrote:
Arbitrary Memory Corruption: If a glitch can corrupt any byte in writable memory or I/O currently visible on the system bus, it is AMC.
I'm pretty sure the existence of these two correlate with the existence of Arbitrary Code Execution. I'm actually not sure if anyone used these terms seriously before.
Posted: 5/17/2021 6:59 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
FractalFusion wrote:
CasualPokePlayer wrote:
Arbitrary Code Execution: If the PC (Program Counter) touches any RAM currently visible on the system bus, it is ACE.
What about glitches that cause the Program Counter to enter RAM, but the author does not quite have the control to modify that part of memory? (This is mostly hypothetical, since a game that is broken enough to have such a glitch will almost certainly have a glitch that leads to the type of ACE which I was talking about.)
In principle, this is never the case. Supposedly with all known ways to modify that memory you cannot execute some total control thing, but this does not mean some new way can't come along to allow it. You end up saying, "well this is not ACE until new trick comes along then it is ACE". RAM is modifiable, and in principle you can probably modify it to whatever want.
FractalFusion wrote:
I'm pretty sure the existence of these two correlate with the existence of Arbitrary Code Execution. I'm actually not sure if anyone used these terms seriously before.
They exist because Pokemon Speedrunning community really (i.e. blame lucky), and ARE (which fun fact, is exactly what the "save glitch" TAS does; the "save glitch" TAS technically does not use ACE) and AMC are actively avoided in this submission as the submission text will note.
Posted: 5/18/2021 5:31 AM
Quote
Acumenium
He/Him
Banned User
Joined: 6/11/2020
Posts: 73
CasualPokePlayer wrote:
Acumenium wrote:
CasualPokePlayer wrote:
BigBoct wrote:
If I was to watch this movie without reading the submission text, I would find the setup to be doing very similar things to [3358] GBC Pokémon: Yellow Version "arbitrary code execution" by MrWint in 05:48.28, which is an ACE playaround, and I would question what exactly you are refusing to do that the other movie permits.
Visually perhaps, but how it's technically done is nowhere alike. This movie does not setup any ACE payload (i.e., it does not setup anything that jumps to RAM), rather it just does a ton of memory manipulation (which given the currently published movie, memory manipulation is allowed for 100% categories as long as it is not arbitrary memory manipulation(/corruption)) to just setup an invalid game corner to farm Pokemon.
Is there any particularly difference between arbitrary code execution and rearranging and redirecting memory to do things in a specific way? I feel like "memory manipulation" is just "ACE, but self-limited". The same glitches are used to start both, so they appear the same. ACE itself is achieved via rearranging and redirecting memory addresses in a way that it runs a program based on it... which this is doing. You made a payload that when using a certain glitch item, loads a glitchy Game Corner, which reads other parts of RAM (your inventory, specifically, it seems a Repel item count?) to determine what the prize is. Defining ACE to just be "when you make Tetris or Pong" seems arbitrary in itself, and thankfully it appears I am not wrong with this specific part from what I've seen some Judges like Mothrayas say in other threads. I don't see at all how this run isn't ACE. I do have one other question, is this save file okay? I know you can eventually fix the "255 Pokemon" and even buggy boxes, but the money seems blanked out. Is it a very high but playable number? Or is it 0? Or is it not actually usable anymore?
"Limited" is the exact point on why limited memory manipulation is allowed. The reason ACE (along with ARE/AMC) is banned is due to the fact you can in principle do anything with it, rendering any sort of restriction like "100%" moot. If it's limited, by principle you cannot just do anything with it, thus this mootness does not apply. Also, can you please actually read the submission text? All this "glitch item" does is jump to the overworld loop. That's it. It's just an optimization so the start menu doesn't have to be closed to interact with the Game Corner (which is a glitched invisible sign that is set up to act like a Game Corner). "Defining ACE to just be "when you make Tetris or Pong" seems arbitrary in itself, and thankfully it appears I am not wrong with this specific part from what I've seen some Judges like Mothrayas say in other threads. I don't see at all how this run isn't ACE." Well that is not how I would define ACE at all, here is my own definition: Arbitrary Code Execution: If the PC (Program Counter) touches any RAM currently visible on the system bus, it is ACE. "RAM" is definable as any memory or I/O (Input/Output, which isn't RAM per se, but in spirit of ACE it can act like such) which is R/W (Read/Write). However, code put into RAM by the game and executed is exempt from this, unless this code is at all modified by the player and executed, which then it becomes ACE. This is not done, and therefore there is no ACE involved with this run. Again, the entire setup is explained in the submission text, which you did not bother to read apparently. Also the save file state is explained in the submission text. The money is probably some high number (I don't know, I don't care, it doesn't affect loading the save file anyways). Also, the submission text explains the Game Corner ends up overwriting the party count (we don't have "255" Pokemon anymore) and name (as explained in the submission text, this means the save file can't be loaded since the game uses a valid name as a "check if save file exists". If it was a valid name anyways, it would load the save file just fine with the completed dex, so probably doesn't mean anything for validity of the run.)
How is this differing from ACE/ARE/AMC? I did read the submission description. All it does is proudly boast that it does not use ACE/ARE/AMC and gives zero explanation on how this run is somehow different. If someone who actually knows what these terms or glitches are still has questions reading the description, it's profoundly bad for a layperson who doesn't. Your submission text should give a clear explanation on what you are doing and how you are doing it---and you do this with the glitches used, you know, swapping memory addresses around, changing memory values, running custom routines based on what was swapped around... Jumping to the overworld loop is a fancy way of saying: - Memory is rearranged to change current room's "overworld loop" to act as the Game Corner - Memory is rearranged to change what Pokemon are available from the Game Corner, via modifying one memory address that handles the reward So again, how is this not ACE/ARE/AMC? If the savefile can't be loaded due to an empty name, shouldn't this be rejected? This is no different than it crashing mid-credits or looping on the HoF which has come up for runs before. Although via totally-not-arbitrary-memory-editing you could just give yourself a name before the Elite Four warp. Frankly your condescension is really grating. I can see the original submission text right here: http://tasvideos.org/forum/viewtopic.php?p=504335#504335 At no point do you mention the money thing. At no point. You didn't edit it in either: http://tasvideos.org/7054S.html Additionally, I see no mentions of what ACE is, and how this run isn't ACE, just that it isn't.
Posted: 5/18/2021 6:03 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
Acumenium wrote:
How is this differing from ACE/ARE/AMC? I did read the submission description. All it does is proudly boast that it does not use ACE/ARE/AMC and gives zero explanation on how this run is somehow different. If someone who actually knows what these terms or glitches are still has questions reading the description, it's profoundly bad for a layperson who doesn't. Your submission text should give a clear explanation on what you are doing and how you are doing it---and you do this with the glitches used, you know, swapping memory addresses around, changing memory values, running custom routines based on what was swapped around... Jumping to the overworld loop is a fancy way of saying: - Memory is rearranged to change current room's "overworld loop" to act as the Game Corner - Memory is rearranged to change what Pokemon are available from the Game Corner, via modifying one memory address that handles the reward So again, how is this not ACE/ARE/AMC? If the savefile can't be loaded due to an empty name, shouldn't this be rejected? This is no different than it crashing mid-credits or looping on the HoF which has come up for runs before. Although via totally-not-arbitrary-memory-editing you could just give yourself a name before the Elite Four warp. Frankly your condescension is really grating. I can see the original submission text right here: http://tasvideos.org/forum/viewtopic.php?p=504335#504335 At no point do you mention the money thing. At no point. You didn't edit it in either: http://tasvideos.org/7054S.html Additionally, I see no mentions of what ACE is, and how this run isn't ACE, just that it isn't.
Whether or not these runs are "ACE" or not and whether that invalidates the run gets determined by judges, not laypeople. Most "laypeople" would frankly not understand what constitute as ACE, as most just see ACE as either "super glitchy thing" or "create pong or tetris" and not understand what it means at its core (executing RAM w/ caveats). "- Memory is rearranged to change current room's "overworld loop" to act as the Game Corner" I quite literally told you the Game Corner is a glitched invisible sign, and this is explained in the submission text. "jump to the overworld loop" quite literally means jump to the code which is after closing the start menu (which is in ROM). That's it. It's just a slight optimization so the start menu doesn't need to be closed, the Game Corner works regardless of this item (which as a note, this item is not used when getting 'M, as your cursor jumps back to the top of the bag so it's faster to just close the start menu like normal). And no that does not count as ACE/ARE/AMC, as this glitch item's pointer is a constant (i.e. the pointer is in ROM) that points towards ROM. "- Memory is rearranged to change what Pokemon are available from the Game Corner, via modifying one memory address that handles the reward" That's just like what the previously published run did with LWA (well, changing it so different Pokemon are caught depending on Master Ball quantity, same principle anyways), and that was not considered AMC (because the memory manipulation was limited, thus not arbitrary). "At no point do you mention the money thing. At no point." Never said I did, I said the save file's state was explained. The money doesn't matter towards the save file's state so it's irrelevant to explain in the submission. "This is no different than it crashing mid-credits or looping on the HoF which has come up for runs before." It is different, the game does do its end game save routine (which the crashing mid-credits and looping on the HoF were critically missing). While the save is fucked anyways, the game does run all the save routines as it normally should at the end, thus this does count as a proper completion. "running custom routines based on what was swapped around" This is not done. Custom routines imply player modified RAM is executed, which it isn't. If you want to keep arguing it is, well the tracelogs won't lie in the end.
Posted: 5/19/2021 10:05 PM
Quote
Acumenium
He/Him
Banned User
Joined: 6/11/2020
Posts: 73
Whether or not these runs are "ACE" or not and whether that invalidates the run gets determined by judges, not laypeople. Most "laypeople" would frankly not understand what constitute as ACE, as most just see ACE as either "super glitchy thing" or "create pong or tetris" and not understand what it means at its core (executing RAM w/ caveats).
It is up to the Judges to make sure the layperson understand this as they're effectively the ones "marketing" the runs to the laypeople, the public. Ideally, the submitter would help too, but that shouldn't be required. You did a great job explaining what you did here, but not how it isn't [x] term for a glitch. Looking up the term "arbitrary code execution": > In computer security, arbitrary code execution (ACE) is an attacker's ability to execute arbitrary commands or code on a target machine or in a target process. It doesn't seem to specify that you have to have absolute control. Limited-scope control seems to qualify. Put it this way: is using 8F still ACE in Pokemon Red 3DS or no, because it can't access the 3DS firmware? When is it "total" control?
That's just like what the previously published run did with LWA (well, changing it so different Pokemon are caught depending on Master Ball quantity, same principle anyways), and that was not considered AMC (because the memory manipulation was limited, thus not arbitrary).
I feel like we need to determine if party and/or item underflow should or shouldn't be allowed. You have 484 RAM addresses to modify between the two, if memory serves. I say "if memory serves" because I somewhat recall that the first 20 item slots still correspond to the same RAM they would normally, and the same goes for the first six party slots? That's not limitless, but it's more than enough to run complicated routines like the customized "Yami Shops" or "Game Corner" (this run) which call for modifiable RAM addresses (the "Master Balls" or "Repels"). When is it arbitrary? When all RAM addresses are possible to access and modify? That in and of itself is an arbitrary distinction as you're saying a program is only a program when it reaches a certain size. If memory serves, Pong can fit in 192 bytes. So would that be ACE, or no, because it's small enough to fit in easily modified "limited" addresses?
It is different, the game does do its end game save routine (which the crashing mid-credits and looping on the HoF were critically missing). While the save is fucked anyways, the game does run all the save routines as it normally should at the end, thus this does count as a proper completion.
Feels like we're splitting hairs here. There's not enough of a difference I think. Recall that Pokemon Red does have a metric to see if you've beaten the game: Cerulean Cave. We can't verify that here as it never saves to anything.
This is not done. Custom routines imply player modified RAM is executed, which it isn't. If you want to keep arguing it is, well the tracelogs won't lie in the end.
This is very interesting. So at no point are you swapping around items that correspond to RAM values, then running an otherwise not possible routine, based on the RAM modified?
Posted: 5/19/2021 10:38 PM
(Edited: 5/20/2021 2:01 AM)
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
"It is up to the Judges to make sure the layperson understand this as they're effectively the ones "marketing" the runs to the laypeople, the public." No? It's not their responsibility to do PR for every run (if anything that's the publisher's job, although not really in this case). And whether or not this run is ACE or whatever isn't an issue of PR, it's an issue of "does this follow movie rules." "Looking up the term "arbitrary code execution": > In computer security, arbitrary code execution (ACE) is an attacker's ability to execute arbitrary commands or code on a target machine or in a target process." Well, that is correct. But, where are these arbitrary commands/code going to come from? The only place such could be written would be RAM, oh, that ends up being "executing RAM." Which this run does not do. Heck you could make a lua script to go check really, that's not that hard to figure out. "Put it this way: is using 8F still ACE in Pokemon Red 3DS or no, because it can't access the 3DS firmware?" Is using 8F still ACE in Pokemon Red Gambatte or no, because it can't access the PC firmware? "target process" is the key here. The target process being the emulated instance in both cases. "We can't verify that here as it never saves to anything." It does save, the game just thinks the save doesn't exist since the name is not terminated correctly. But save is there. "This is very interesting. So at no point are you swapping around items that correspond to RAM values, then running an otherwise not possible routine, based on the RAM modified?" The modified RAM is not executed.
Posted: 5/20/2021 1:58 AM
Quote
moozooh
Senior Moderator
Joined: 8/4/2005
Posts: 5777
Location: Away
Acumenium, it seems to me that you either haven't read my previous post to you, or decided for some reason that you shouldn't take it to heart because you didn't notice that I'm a moderator and you shouldn't ignore what a moderator says when they address you specifically. (I know it's not immediately evident, which is why I'm not holding that particular thing against you.) In any case, I see no significant changes in your conduct since that time, which is not a great thing. You still routinely make arguments without a proper understanding of the subject, you're being combative, and besides, it's getting really hard to understand what it even is that you're arguing. So here's my task for you: limit your next post here to two short paragraphs. In the first paragraph, recap the things you're upset with in regards to this movie, its authors, or the judging process. In the second, propose what should be done instead or how it should be done. Do both without being combative or rude. If you have no such points to make, state so clearly. Even if you decide to drop the discussion here for any reason, the same task will apply to whatever next argument you make in any other thread. This will determine whether you're even capable of stating your disagreements in a polite and concise manner, and hence how we should treat your presence on the site. It's in your best interests to demonstrate that you are, in fact, capable of doing this right.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Posted: 5/20/2021 3:47 AM
Quote
DJ_Incendration
He/Him
Player (52)
Joined: 4/1/2016
Posts: 294
Location: Cornelia Castle
I'm sure that we are all capable of being polite and concise, while saying what we disagree about. I don't feel like anyone is being combative here, and I believe it's fine to ask questions because someone doesn't understand something.
DJ Incendration Believe in Michael Girard and every speedrunner and TASer!
Posted: 5/20/2021 10:40 PM
Quote
Acumenium
He/Him
Banned User
Joined: 6/11/2020
Posts: 73
moozooh wrote:
Acumenium, it seems to me that you either haven't read my previous post to you, or decided for some reason that you shouldn't take it to heart because you didn't notice that I'm a moderator and you shouldn't ignore what a moderator says when they address you specifically. (I know it's not immediately evident, which is why I'm not holding that particular thing against you.) In any case, I see no significant changes in your conduct since that time, which is not a great thing. You still routinely make arguments without a proper understanding of the subject, you're being combative, and besides, it's getting really hard to understand what it even is that you're arguing. So here's my task for you: limit your next post here to two short paragraphs. In the first paragraph, recap the things you're upset with in regards to this movie, its authors, or the judging process. In the second, propose what should be done instead or how it should be done. Do both without being combative or rude. If you have no such points to make, state so clearly. Even if you decide to drop the discussion here for any reason, the same task will apply to whatever next argument you make in any other thread. This will determine whether you're even capable of stating your disagreements in a polite and concise manner, and hence how we should treat your presence on the site. It's in your best interests to demonstrate that you are, in fact, capable of doing this right.
If you want someone to be aware of your discussion to them, PMs work. I had no idea you gave me any sort of warning, moozooh. I stopped checking that thread because I realized I was wasting my time. I was speaking from a minority platform in the first place, and was well aware of that. There has also been zero hostility other than two minor comments where I felt CasualPokePlayer was being condescending and it could've merely been a misunderstanding. I am very curious what exactly it is you think was "combative" or "hostile" here. Yes, I am in disagreement with labeling this as "not using ACE". I have outlined why that is pretty well, I think, as the delineation mentioned in this thread by the run creator (CPP) was arbitrary and makes it seem as if ACE is only when a program is a certain size or if it hijacks the entire RAM (which I already proved is not how ACE is defined). This is most definitely running customized routines (programs), they just aren't very big. You know it is possible for people to disagree civilly, right? Memory corruption would fall more under the "reset curse" in A Link to the Past (GBA)---famously used to beat the Four Swords Palace in record time here. This isn't memory corruption, it's calculated manipulation to run specific routines---you know, code execution? If we aren't allowed to discuss (read: say something that isn't praise) runs to any capacity, which is what you're trying to imply here, leave these threads closed to posts but allow voting only. So far I'm not seeing where anyone was uncivil. "Combative" implies hostility and again, except for two comments I probably overthought, I have not seen a single ounce of hostility from CPP and I have not said one negative thing about him or anyone else.
Posted: 5/20/2021 11:03 PM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
Acumenium wrote:
I think, as the delineation mentioned in this thread by the run creator (CPP) was arbitrary and makes it seem as if ACE is only when a program is a certain size or if it hijacks the entire RAM (which I already proved is not how ACE is defined).
I never said that. I literally told you if any RAM is executed, it is ACE (with the minor exception of code intentionally placed by the game into RAM and executed is not ACE, unless said code is at all corrupted by the player and executed, in which case it does become ACE. The only code that can fall under this exception would be the OAM DMA routine the game places into FF80-FF89 of HRAM, which is not modified at all). This run does not violate this, despite your baseless claims that it does. "it's calculated manipulation to run specific routines---you know, code execution?" If you're saying that memory is manipulated then that manipulated memory is executed, then you are objectively wrong. If you are talking about "memory is manipulated then code in ROM doesn't something unintentional" that can really end up just covering a ton of glitches. You can argue the previous run is covered under this too. As a reminder, the save corruption is not required to perform the FTL (Invalid Game Corner) strategy, it is performable without save corruption, save corruption just makes this faster. "This is most definitely running customized routines (programs), they just aren't very big." If you are going to keep making this claim, can you at least provide any sort of proof? Looking at the encode is not going to tell you what happens under the hood. Either go download the movie file, tell me which frame I could pull up the tracelogger and find ACE, or stop asserting this.
Posted: 5/21/2021 12:39 AM
Quote
£e_Nécroyeur
Skilled player (1672)
Joined: 7/1/2013
Posts: 447
Arcade TASer of 2017 Arcade TAS of 2016 Exotic platforms TASer of 2014 Exotic platforms TAS of 2014
DJ Incendration wrote:
I don't feel like anyone is being combative here, and I believe it's fine to ask questions because someone doesn't understand something.
Agreed. This sort of discussion can be inherently challenging for both sides; a bit of frustration is understandable.
Posted: 5/21/2021 2:30 AM
Quote
Acumenium
He/Him
Banned User
Joined: 6/11/2020
Posts: 73
CasualPokePlayer wrote:
Acumenium wrote:
I think, as the delineation mentioned in this thread by the run creator (CPP) was arbitrary and makes it seem as if ACE is only when a program is a certain size or if it hijacks the entire RAM (which I already proved is not how ACE is defined).
I never said that. I literally told you if any RAM is executed, it is ACE (with the minor exception of code intentionally placed by the game into RAM and executed is not ACE, unless said code is at all corrupted by the player and executed, in which case it does become ACE. The only code that can fall under this exception would be the OAM DMA routine the game places into FF80-FF89 of HRAM, which is not modified at all). This run does not violate this, despite your baseless claims that it does. "it's calculated manipulation to run specific routines---you know, code execution?" If you're saying that memory is manipulated then that manipulated memory is executed, then you are objectively wrong. If you are talking about "memory is manipulated then code in ROM doesn't something unintentional" that can really end up just covering a ton of glitches. You can argue the previous run is covered under this too. As a reminder, the save corruption is not required to perform the FTL (Invalid Game Corner) strategy, it is performable without save corruption, save corruption just makes this faster. "This is most definitely running customized routines (programs), they just aren't very big." If you are going to keep making this claim, can you at least provide any sort of proof? Looking at the encode is not going to tell you what happens under the hood. Either go download the movie file, tell me which frame I could pull up the tracelogger and find ACE, or stop asserting this.
Okay, we should probably cover something again then. What defines "executing RAM"? Using the limited RAM editing capabilities of an underflowed inventory/party to create or trigger new functions (Invalid Game Corner with a fully editable reward table) isn't it? Why? When does "executing RAM" or "RAM manipulating" become such? The underflowed inventories are literally a form of RAM editor. Very limited in scope---as I said earlier I think it's somewhere in the 480 range for what both combined can access, and there may be overlap, but the point still stands. You triggered, via swapping and altering memory values in the underflowed menus, a Game Corner that never existed, especially in a location where it never existed, with prizes that no existing Game Corner has. From there, you also, via swapping and altering memory values, altered what the new Game Corner has for prizes. Basically, and I feel I've covered this enough (as have you), my issue is with the use of the underflowed menus. Those are essentially limited-function RAM editors, which means that they're effectively executing arbitrary code. I've already illustrated that ACE does not require it to be a big program or have total control. Pokemon is in a weird situation as you said where it is so buggy that conceivably many categories of what is or isn't ACE can be created, but at that point you're just delineating how much arbitrary code should be executed before it's a new category. New routines that didn't exist in the RAM before swapping and altering their values now exist thanks to it such as the (invalid) Game Corner and its prizes that can be altered based on a RAM address corresponding to the Repels. However I also have to concede that due to the existence of the glitchless Co-Op Diploma run, I don't know what could exist alongside it other than a run that isn't full-blown altering the Pokedex count (like the original save corruption TAS that did it to 152 incidentally). Wouldn't relegating memory corruption (Mew Glitch/Cooltrainer/other non-underflow methods) for only the missing slots be terribly arbitrary? A no trades full completion run (124 in RB, 129 in Y, IIRC) could be interesting I guess, but would it be redundant alongside a Co-Op Diploma?
Posted: 5/21/2021 2:59 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
Acumenium wrote:
Okay, we should probably cover something again then. What defines "executing RAM"?
The Program Counter ("PC") points to read/writable memory or I/O, which will cause the CPU will execute code from RAM. That is the objective definition for "executing RAM." "Using the limited RAM editing capabilities of an underflowed inventory/party to create or trigger new functions (Invalid Game Corner with a fully editable reward table) isn't it?" 1. No. 2. This doesn't create a "new function." This triggers an already existing function with the wrong arguments essentially, which has the result you see in the TAS. As a note, you cannot arbitrarily control this invalid Game Corner, you have to use whatever ones that are already per-determined in ROM (which one invalid one happens to source prizes in underflowed bag space and take prize money from SRAM). "The underflowed inventories are literally a form of RAM editor. Very limited in scope---as I said earlier I think it's somewhere in the 480 range for what both combined can access, and there may be overlap, but the point still stands." Well what do you think "memory manipulation" is? I don't get how this is a point. "they're effectively executing arbitrary code" No, they're not. "New routines that didn't exist in the RAM" I'm not sure why you keep saying "routines." Routines are code, and thus are meant to be executed. You are saying this TAS is inserting code into RAM and is executing it. It's not, and that can be objectively proven so, and you have not proven otherwise. "thanks to it such as the (invalid) Game Corner and its prizes that can be altered based on a RAM address corresponding to the Repels." Again, this isn't some function created in RAM to go create the Game Corner. It's a sign created, which has its text script point to someplace that has the text script ID which corresponds to the Game Corner, and the Game Corner ID is set to 59, which by luck happens to have prize data in underflowed inventory and price data in SRAM.
Posted: 5/21/2021 3:19 AM
Quote
TiKevin83
He/Him
Ambassador, Moderator, Site Developer, Player (155)
Joined: 3/17/2018
Posts: 358
Location: Holland, MI
Rookie of 2018 Gameboy TASer of 2018 Gameboy TAS of 2018
The word "arbitrary" is what indicates an unlimited scope of control of the hardware. If you dig deeper into the boilerplate wikipedia explanation, it clarifies that ACE usually refers to having gained unlimited control over where the program counter is pointing to, and unlimited control over the data at that location in memory (eg you've made the location of the next instruction to execute the same location as an input buffer). The process of setting up the glitch game corner and swapping buyable pokes explicitly avoids doing those things (as CPP has also clarified above). Whether an exploit looks like ACE to a casual viewer isn't the point of the ACE flag, it's a technical point of merit for infosec-minded runs. I personally would love to see more games approached with infosec and broken with both ACE and non-ACE attack vectors.
Posted: 5/21/2021 2:55 PM
Quote
Sanqui
Any
Player (26)
Joined: 4/25/2011
Posts: 33
Just chiming in to say that to my best knowledge, this run does indeed not perform arbitrary code execution (ACE). Since people are being so skeptical of it, though, I wonder if somebody could made a Bizhawk fork that would report whether ACE has been performed on a given movie? The OAM DMA routine might require a special case but beyond that it shouldn't be that difficult. I would take a shot at it but running Bizhawk is troublesome on Linux so there's my excuse. I just imagine a "ACE verification" pass might be more convincing, since the rules would be, well, codified.
ovo
Posted: 5/21/2021 4:52 PM
Quote
DJ_Incendration
He/Him
Player (52)
Joined: 4/1/2016
Posts: 294
Location: Cornelia Castle
I don't think that's necessary. It even says as an objective in the submission text, "Does not use arbitrary code execution, arbitrary ROM execution, nor arbitrary memory corruption." In the PSR video, it says is does not. Because of this, I know it does not.
DJ Incendration Believe in Michael Girard and every speedrunner and TASer!
Posted: 5/27/2021 8:44 PM
Quote
ais523
Editor, Player (44)
Joined: 7/11/2010
Posts: 1029
I agree that this run does not use arbitrary code execution or arbitrary ROM execution. However, I dispute that it isn't using arbitrary memory corruption. The Game Boy + cartridge combination has multiple different types of memory. This run uses a glitch (item underflow into party overflow) to gain control of one of those types of memory (the WRAM), and is able to corrupt it at will; it's definitely using an arbitrary WRAM corruption glitch. Now, this glitch isn't capable (without extending it into ACE/ARE) of corrupting other areas of memory, such as the SRAM, which is why I think the author isn't considering it to be an arbitrary memory corruption glitch. But the WRAM is a large and important area of memory that stores almost (but not quite) everything that affects gameplay, so an arbitrary WRAM corruption glitch is very similar in spirit to a more general arbitrary memory corruption glitch. We don't call a glitch "not arbitrary memory corruption" just because it isn't capable of altering the ROM (which is after all a type of memory). I think this is a similar situation: being unable to change the SRAM, VRAM, etc. doesn't change the fact that the one glitch is capable of affecting almost anything in the game's memory that actually matters. (The glitch could be used to set Pokédex flags directly, for example; the author explicitly refuses to use it to do so, in order to meet a 100% definition, but it would have been powerful enough to do so if necessary, and it's used to do a lot of other things in the game. SDA used to have a rule along the lines that "if a glitch you use is capable of doing X, you can't use it in a no-X category, even if you actually use it to do something other than X", and I think this is a similar case.)
Posted: 5/27/2021 9:00 PM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
ais523 wrote:
But the WRAM is a large and important area of memory that stores almost (but not quite) everything that affects gameplay
"important" is subjective. For this game, this could be argued due to nearly everything using it, but not all games are like this. There are several games in fact that just use SRAM as effective WRAM, making it just as important. In contrast, "arbitrary" can be objectively defined. If any memory visible to the CPU is corruptible with a glitch, then you can "arbitrarily" corrupt memory with that glitch. These terms are used for the Vault, which is meant to be objective, not subjective.
ais523 wrote:
We don't call a glitch "not arbitrary memory corruption" just because it isn't capable of altering the ROM (which is after all a type of memory).
You can still try to write to ROM, and that could end up being linked up to a memory mapper (so that I/O is corruptible). Although even then that's still just semantics, AMC referring to RAM specifically is implied.
ais523 wrote:
(The glitch could be used to set Pokédex flags directly, for example; the author explicitly refuses to use it to do so, in order to meet a 100% definition, but it would have been powerful enough to do so if necessary, and it's used to do a lot of other things in the game. SDA used to have a rule along the lines that "if a glitch you use is capable of doing X, you can't use it in a no-X category, even if you actually use it to do something other than X", and I think this is a similar case.)
You do realize by taking this to an extreme you effectively ban a ton of glitches (to the point where you probably can't do a true CEA run), right? Let's say the old classic trainer escape glitch. That could be used for ACE. Should that be banned? Or let's take underflowed inventory, that could be used for ACE, should that be banned? Or let's take (nearly every glitch), that could be used for ACE, should that be banned?
Posted: 5/28/2021 12:02 AM
Quote
ais523
Editor, Player (44)
Joined: 7/11/2010
Posts: 1029
Let's see, there's a continuum as far as memory corruption is concerned. Being able to corrupt all but 1 address is effectively arbitrary memory corruption (most glitches can't corrupt quite every address, e.g. because you need one to hold the cursor). Being able to corrupt only 1 address isn't arbitrary at all, it's just a specific-purpose glitch. So a cut-off has to be drawn somewhere between arbitrary memory corruption and "regular" memory corruption. If we're talking about Vault requirements for 100%, one of the only objective places to draw the line, as far as I can tell, is "can this corrupt the memory that stores the information about whether the 100% requirements are met?". In this case, party overflow could, so we shouldn't allow it. (The only other place I could see to draw the line is "is there any way at all to affect/control which address this glitch corrupts, even within a narrow range?", but that's much more restrictive.) I think the bigger conceptual problem is "how do you define what a single glitch is?". The "classic" version of trainer-escape, for example, generally has well defined behaviour for what it does (forcing an encounter based on a value left in memory from a previous battle), and probably isn't really classifiable as a memory corruption glitch at all (I'd put it in the "storage" group of glitches); the main complication is that some memory in Pokémon is used for two different purposes, and we're corrupting it from one possible point of view but not the other. With trainer-escape into ACE, the execution is a bit different (you need to manipulate stored values in other ways between the escape and the re-entry, e.g. by reading signs, rather than simply just starting a trainer battle at range). So are these the same glitch or not? With storage glitches, the order in which you do things can be critically important, so I could understand a classification of "trainer-escape → fight trainer → re-enter" as being treated separately from "trainer-escape → fight trainer → read sign → re-enter". I think there's a big conceptual difference, though, between "let's do this glitch, but add extra steps so that it gets a different effect", and "let's do this glitch, but move the cursor to a different location so that we're affecting a different part of memory".
Posted: 5/28/2021 12:19 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
ais523 wrote:
(most glitches can't corrupt quite every address, e.g. because you need one to hold the cursor).
I suppose this is true. In practicality, the only glitches that can chose any memory address to corrupt would be corrupt some pointer the game refers to to write someplace. Which there is one noted in the TAS that had to be avoided, so it's not entirely impossible. Also, on trainer escape, I was more referring to "trainer escape into go fight glitch mon that just suddenly gives ACE" which is possible. Also, from what I understand, the rule against corrupting memory for progress is just "it doesn't count towards progress" (basing my understanding from the judgement note of the movie this obsoletes). If you set every flag/etc with in game methods, it counts as progress, but otherwise it doesn't count as progress. Note with your reasoning, you can essentially say "you are not allowed to crash the game" (like this movie does) as that can (and does in this movie) corrupt Pokedex flags, but even so it's effectively moot since the player is forced to revert back to the previous save before the corruption.
Posted: 5/29/2021 1:01 AM
Quote
Acumenium
He/Him
Banned User
Joined: 6/11/2020
Posts: 73
CasualPokePlayer wrote:
Acumenium wrote:
Okay, we should probably cover something again then. What defines "executing RAM"?
The Program Counter ("PC") points to read/writable memory or I/O, which will cause the CPU will execute code from RAM. That is the objective definition for "executing RAM." "Using the limited RAM editing capabilities of an underflowed inventory/party to create or trigger new functions (Invalid Game Corner with a fully editable reward table) isn't it?" 1. No. 2. This doesn't create a "new function." This triggers an already existing function with the wrong arguments essentially, which has the result you see in the TAS. As a note, you cannot arbitrarily control this invalid Game Corner, you have to use whatever ones that are already per-determined in ROM (which one invalid one happens to source prizes in underflowed bag space and take prize money from SRAM).-game func "The underflowed inventories are literally a form of RAM editor. Very limited in scope---as I said earlier I think it's somewhere in the 480 range for what both combined can access, and there may be overlap, but the point still stands." Well what do you think "memory manipulation" is? I don't get how this is a point. "they're effectively executing arbitrary code" No, they're not. "New routines that didn't exist in the RAM" I'm not sure why you keep saying "routines." Routines are code, and thus are meant to be executed. You are saying this TAS is inserting code into RAM and is executing it. It's not, and that can be objectively proven so, and you have not proven otherwise. "thanks to it such as the (invalid) Game Corner and its prizes that can be altered based on a RAM address corresponding to the Repels." Again, this isn't some function created in RAM to go create the Game Corner. It's a sign created, which has its text script point to someplace that has the text script ID which corresponds to the Game Corner, and the Game Corner ID is set to 59, which by luck happens to have prize data in underflowed inventory and price data in SRAM.
If you were to rearrange the RAM just slightly more than done here, you'd have full control. As it stands, you don't have full control as of this video, just 483 (255*2 -6 (Party Pokemon) and -21 (20 Items + Cancel)) addresses, if memory serves right. At that point, what makes using ACE to trigger the credits (Super Mario World, Donkey Kong Country, etc.) actually "ACE"? I mean, you're just triggering an in-game function, aren't you? It's not like it's "total" control. I would also like to point out that Pokemon can and often does run code (when using ACE) from SRAM. Is this still "ACE"? Or is SRAM not actually RAM? Literally anything is "code". I keep saying "routine" because I fail to see what else I am supposed to call a created function from code, of which RAM of any kind can be if you execute it properly. Let's reevaluate, shall we? - The game features two Game Corner prize tables that award Pokemon. Prizes vary by version, but in this one, they award, in order: - Prize Table #1: Abra (180C), Clefairy (500C), Nidorina (1200C) | Prize Table #2: Dratini (2800C), Scyther (5500C), Porygon (9999C) Yours is not only located in the Celadon Game Corner, where the others are, it has a displayed prize table of: - Tangela (39C), Vileplume (39C), Metapod (39C) Additionally, yours is triggered via "a sign" (not an NPC), that seems to be triggered by some use of an item? In reality, the prize table for yours is for show, the first option seems to award something based on a memory address that the Repel count is tied to.
Again, this isn't some function created in RAM to go create the Game Corner. It's a sign created, which has its text script point to someplace that has the text script ID which corresponds to the Game Corner, and the Game Corner ID is set to 59, which by luck happens to have prize data in underflowed inventory and price data in SRAM.
...so, to reiterate: - you did NOT create the Game Corner. You just created a sign where there was none that creates a (new) Game Corner based on manipulated memory. I'm sorry, but what part of this isn't ACE? Arbitrary is the polite and honest way to describe how you're going about defining everything to, ironically, avoid it being called arbitrary (code execution)! To make matters worse, even if this is accepted with your frankly flimsy reasoning defending it from being considered ACE (if it's not ACE, it's not ACE, the lengthier the "defense" against accusations goes the more flimsy the reasoning behind it not being it, looks), it shouldn't be accepted on the basis that it isn't even a proper completion. It doesn't---and can't save. That has absolutely no difference from a run crashing during the credits. It's okay if it didn't have a battery, it's not required that the game hold the save, but this isn't even creating one.
Posted: 5/29/2021 1:52 AM
Quote
CasualPokePlayer
Emulator Coder, Judge, Experienced player (729)
Joined: 2/26/2020
Posts: 778
Location: California
Arcade TAS of 2023 TASer of 2021 Gameboy TASer of 2021 Rookie of 2020 Gameboy TASer of 2020 Lucky TAS of 2020
"At that point, what makes using ACE to trigger the credits (Super Mario World, Donkey Kong Country, etc.) actually "ACE"? I mean, you're just triggering an in-game function, aren't you? It's not like it's "total" control." You're executing code you've made in RAM. Therefore, it is ACE. "I would also like to point out that Pokemon can and often does run code (when using ACE) from SRAM. Is this still "ACE"? Or is SRAM not actually RAM?" I literally defined RAM some time ago for this argument and SRAM fits the book perfectly fine. "that seems to be triggered by some use of an item" Can you not read? The item used is an item that happens to jump to the overworld loop. It's effectively just a faster way to close the menu. That's it. Actually using the Game Corner is as simple as pressing A in the overworld (just like a sign/NPC!). The item can be removed entirely from the TAS and it would still be able to use the Game Corner just fine. "I'm sorry, but what part of this isn't ACE?" Well, for one, it doesn't involve code execution in RAM. But how about I explain this differently. First, to clarify, when I said "Game Corner" earlier, I meant the prize menu that appears. Now, how does the game actually handle the prize menu within the actual Game Corner. Internally, it just creates signs in each booth which have their IDs point to each prize table. They're just as much "signs" as the "signs" created in this TAS. Now, when the sign ID is 59, the game ends up reading unrelated ROM data to figure out where the prize data is located (in fact, arguably), and by chance it points to underflowed inventory. Thus all the prizes end up corresponding there: The first prize corresponding to Repel's item ID, the second corresponding to Repel's quantity, and the third corresponding to "8 8"'s item ID. At no point in this process does it involve executing some custom code created, thus, there is no ACE involved. "(if it's not ACE, it's not ACE, the lengthier the "defense" against accusations goes the more flimsy the reasoning behind it not being it, looks)" I mean, again, a lua script can trivially be made to check for ACE specifically, said lua script will not detect ACE. Still waiting on which frame I could just look at the tracelogger and apparently find ACE you claim exists. "can't save" It does save. In fact, all the save routines are executed the same as a typical completion. Now, the game can't actually load the save since it considers the save file as non-existent, but that's a different thing. "this isn't even creating one" It did create a save. Also, to clarify, all "saving the game" is is just the game copying memory from WRAM into SRAM then checksuming the data (then storing the checksum). That's it. Is that done in this movie? Yes? Then the game saved.
1 2 3