It's also worth noting that I looked into Bruno's movie (
this one) and found that the effect shown there at least is not at all ACE, despite how it looks. There are, of course, many paths that enter ACE territory, but not all of them.
To explain that particular code path, it causes the Chuck's phase pointer to jump into the sprite load loop ($C2 = 0x51 jumps to $02A9AA). What happens from there is that the game tries to load some sprite data (from [$CE + Y], which happens to have Y = 0x51 from before) into sprite slots 03-15 (note that the normal sprite slot cap is only 0B). As it happens, the "positions" of all those invalid sprites end up being to the left of the screen, so the game just skips over them and does nothing with the data, and finally returns back to the Chuck without issue. It ends at X = 0x15 because that one is considered to be to the "right" of the screen so it determines the loop is done.
However, the value of X is still retained from that loop; that is, the Chuck's is returned to with X = 0x15. In SMW, sprites use the X index to determine the sprite slot currently being processed, and it then decrements until it hits 0, which means... we're now dealing with sprite slot 15, which doesn't exist. And before anything returns back to normal, there are invalid sprite slots 0C-14 in the way, which also have to be processed.
Thankfully, SMW happens to put its sprite status table (which determines what routine to run for a sprite) at $14C8, and immediately following that at $14D4 is the the sprite Y position high byte table. In a horizontal level, every value in that table will be either 0 or 1; 0 corresponds to "no sprite", and 1 corresponds to "initialize sprite". So no main sprite routines will end up being run with this invalid index, and only sprite's whose "real" counterpart indices (that is, the sprite slot minus 0x0C) had a high Y position of 1 will actually end up doing anything. And thankfully, the initialization routines for most sprites are extremely short; usually only a few lines at best, which means there won't be many adverse effects.
However, some of these routines do end up being run, and with that invalid X index no less, which causes some bad indexing to sprite tables during that initialization. Fortunately, most sprite tables are immediately followed by other sprite tables, which means we can't really do anything openly game-breaking to RAM. At best, we can only write some values to existing sprite tables, messing with the sprites that are currently spawned. For that most part, that's harmless, but this particular effect is the result of one of those misc table writes. Specifically, during the processing, the slot whose "real" counterpart is Yoshi ends up running the initialization routine for sprite 0B, the buzzy beetle (because the sprite ID table at $9E is immediately followed by the sprite Y speed table at $AA, and Yoshi's "Y speed" is 0B at the time). The buzzy beetle's initialization routine doesn't do much, but one thing it does do is that it sets a random value to the sprite table at $1570 (which then indexes into another sprite table at $157C). Yoshi uses $157C as his direction; 0 = left, 1 = right. And here's where the real bug actually happens.
While Mario is riding Yoshi, Yoshi's direction is transfered to Mario's direction every frame. As a result, Mario ends up getting this weird direction value written to him as well. And, of course, that causes everything you see in Bruno's video. His direction is used during indexing of Mario's tilemap tables, so an invalid value ends up there and Mario looks like he's now having a really weird day. And Mario's direction is also used during flight, to determine the index to the Mario X speed acceleration tables. So that pulls an invalid value, and later when indexing the speed tables, it pulls "80FF" as his X speed (an
extremely high leftward speed). To then get the rightward speed, you then have to press >+B to "decelerate" Mario. That, of course, also throws out an invalid value due to Mario's wack direction, resulting in a new speed of "6100" (an still-extremely-high rightwards speed). So you basically have freedom to go really fast left or really fast right like this.
That said, you can only really abuse this while flying. When not flying, Mario's directly gets reset from any left/right inputs; it can also get reset by capespinning, so you can't do that. However, relanding on Yoshi (who got shoved way over to the right as another side effect of this glitch) will break Mario's direction again.
So, that's that particular effect. And there's a huge range of similar effects you can probably get from just this one particular code path, by e.g. messing with Yoshi's Y speed do run different sprite initialization routines. So the question is, how legal is this? There's theoretically a wide range of effects, assuming we can find code paths that don't enter ACE territory. We can at least note that:
- With the lava method, there's a limited number of levels that both have lava and allow Yoshi to enter. The only ones that come to mind are VD1 (useless), VD3 (no viable sprites, probably), CI4 (useless), CIS (useless), and VoB4. I might be forgetting a few there, but VoB4 seems like it'd be the only one this method work be useful in.
- The number of sprites that naturally both use $1558 and use $C2 as a phase pointer is limited. There is only the Diggin'/Pitchin' chuck, Monty Moles, Fishbones, and the Bowser statue; the last two can't even be reached with a Yoshi anyway.
- The Koopa method depends on the value of $1540 during or close to initialization, and the number of sprites that both use $C2 as a phase pointer and set $1540 relatively early is pretty small.
A preliminary search led to this list of sprites that potentially can be used to trigger the glitch in the original game: Volcano Lotuses, Chargin' Chucks, Splittin' Chucks, Bouncin' Chucks, Clappin' Chucks, Pitchin' Chucks, Diggin' Chucks, Sumo Bros., Pipe Lakitus, and Monty Moles. A quick search for all instances of these led to the following list of viable locations for application in the original game:
volcano lotus: DP1, Groovy
chargin': YI2
pitchin': DP1, Funky
diggin': VoB4
sumo: Funky
pipe lakitu: VS2, FoI4
monty mole: YI2
All other instances of the sprite either would likely not be useful (e.g. right at the end of the level) or were not reachable with both a Yoshi and Koopa (e.g. in castle or not a level with a shell). So you can essentially consider that the potential scope of application within the original game, at least as far as I'm aware of. Exactly what kind of applications each of those sprites can have is still a mystery, though; we know about the VoB4 application with the Diggin' Chuck, but that's only one sprite with one pointer. How much mileage we can get out of other code paths is still up for investigation, and it's something we should probably decide on before considering its use in any runs.
