Submission #7726: Sauraen, dwangoAC & Savestate's N64 The Legend of Zelda: Ocarina of Time "Triforce% ACE Showcase" in 04:37.48

(Link to video)
Nintendo 64
Triforce% ACE Showcase
Bizhawk 2.0.0
16649
60
228585
PowerOn
Sauraen, dwangoAC
(Additionally: Savestate)
Legend of Zelda, The - Ocarina of Time (USA)
Submitted by dwangoAC on 10/8/2022 7:11:21 AM
Discuss and Vote Queue All by submitter Download
Submission Comments

Triforce% OoT ACE Showcase - abstract

Triforce% OoT ACE Showcase was presented by Sauraen, Savestate, and dwangoAC with the help of TASBot under the title Ocarina of Time Beta Showcase during the SGDQ 2022 TAS block after helping raise more than $228k for Doctors Without Borders. Triforce% was performed using an original, unmodified US 1.0 release cartridge of The Legend of Zelda: Ocarina of Time on a real, original N64 (the Nintendo 64 had a passive RGB mod for a clean video signal that did not affect gameplay in any way). Everything in this run was done live in front of an audience using only button presses on controllers, ultimately achieving ACE (Arbitrary Code Execution) using Stale Reference Manipulation (SRM, a form of Use-After-Free exploit).

Showcase submission notes

This run is (finally) being submitted by dwangoAC on behalf of the director of the project (Sauraen), the human speedrunner (Savestate), and the rest of the 25+ strong team who worked on it. This is being submitted with a stub file as a showcase submission following discussions in the forum thread "Let's figure out how to publish TASBot's GDQ showcases". The submission file itself is a placeholder movie file prepared by BruceShankle (used with permission - thanks!) which does not accurately represent the 53.05.3 runtime and 1 hour of content that the Triforce% team presented. Instead of attempting to watch the movie file in BizHawk, please watch the official Triforce% video or the 10-minute Triforce% Highlights video.
Over two dozen people put immense effort into creating and documenting this run, almost entirely in secret due to the nature of the content. The team developed a website at https://GetTriforce.link complementing the full source code of everything the team created to document the project and I've opted to take the excellent documentation from there and supply it here mostly unchanged as it covers most aspects of the run. The sections below contain substantial spoilers, so please pause reading now and watch the run if you haven't already before coming back to the description and FAQ below. One other note: I (dwangoAC) would like to specifically call out Sauraen for the heart he put into this as Triforce% would not exist without his passion over two and a half years of effort as well as to Savestate for how amazing they were in their performance in front of a live audience.
Triforce% is a unique type of artistic work: a “speedrun” performance of a video game which uses glitches to modify the console’s memory on-the-fly to create a new story and a new experience within the game. The game cartridge and ROM are completely unmodified; the changes to the game content are accomplished entirely through glitches and controller input. TASBot with a replay device is used to stream controller inputs extremely quickly into the console, but in principle, a version of this could be accomplished by a perfect human player, just much slower.
The storyline created through ACE ties together beta content from Ocarina of Time–some content actually left in the cartridge, some shown in pre-release screenshots and videos and recreated by the Triforce% team–with urban legends about OoT from the early days of the internet, creating a plot leading to the Triforce. Then, Link uses the power of the Triforce to warp to the future, becoming his Breath of the Wild self and bringing in messages of unity from the live Twitch chat, all in real time on a Nintendo 64 console via controller input.

FAQ

What is Triforce% OoT ACE Showcase?

Triforce% is a hybrid RTA/TAS superplay (“speedrun”) of The Legend of Zelda: Ocarina of Time (1.0) (U) (Nintendo 64) which uses Arbitrary Code Execution (ACE) to install a set of data loaders in the console’s memory, enabling arbitrary assets (scenes, objects, music, etc.) in the game ROM to be seamlessly live-patched via the TAS replay device. By modifying the game programming and assets, beta content cut from the final version of OoT is restored and brought to life on screen. A new plot is constructed based on this beta content and on urban legends from the late ’90s, culminating in Link obtaining the Triforce, all within the vanilla game.
Then, Link uses the power of the Goddesses–and we use the power to make the game whatever we can imagine–to go far beyond what would have been possible in 1998, and bring together the community in a meaningful way.

Okay, explain that to me like I’m five?

A human speedrunner and a robot sit down in front of a completely unmodified copy of Ocarina of Time on N64. They press buttons on the controllers very quickly and very precisely, and activate complicated glitches in the game. These glitches let them hack into the game and start changing the plot, but this is still all done just by pressing buttons on the controllers.

How does Triforce% relate to OoT Beta Showcase?

OoT Beta Showcase is the name we were using for Triforce% in order to avoid spoiling that we were going to get the Triforce. They are the same project. The idea was to tie some elements of real beta content together with a plot made up by the Triforce% team, and create a plausible “alternate ending” to OoT involving the Triforce, all through ACE and controller input.

How much/which parts of this were already on the cart?

Here is a complete list of all the debug / beta content left on the cartridge which we showed off: 
   * Inventory editor
   * Arwing
   * Part of text patching system is for the N64DD expansion (URA Zelda)
   * Beta Kokiri head and body
   * Butterfly get item model and entry in one of the item tables
   * Arguable (i.e. hints of it, but not full thing): full Ocarina system
   * Arguable (i.e. hints of it, but not full thing): ability to change between child and adult without pulling the Master Sword
   * Giant magenta rupee (including its behavior of exploding when you touch it)
   * Arguable (i.e. hints of it, but not full thing): melting the ice in Zora’s Domain
   * Beta Great Fairy
   * Pedestal of the Ocarina
   * Triforce wipe animation before entering Triforce room
   * Beta Staff Roll cutscene flying through Kokiri Forest
Here is a complete list of all the alpha / beta content which was shown in official prerelease footage / screenshots / etc. So this is real alpha / beta content, but it was recreated by our team, not left on the cartridge. 
   * Equipping Medallions to C-buttons
   * Unicorn Fountain scene
   * Beta Great Fairy behavior
   * Triforce room scene
   * Triforce pieces model / behavior
   * Triforce light model
   * Triforce chest model / behavior
Here is a list of the “urban legends” which we brought to life in game. These are concepts which are not actually in OoT in any way, but which were made up by other people over the years. 
   * Beating the Running Man in the race
   * Overture of Sages
Everything else we showed was made up by the Triforce% team, including: 
   * All the dialogue
   * Lost Woods exit code
   * The concept of the Gerudo having created the Song of Time / Nabooru teaching it to child Link
   * Running Man boss battle
   * Sages’ Charm
   * Chamber of Sages sequence
   * All the content after obtaining the Triforce

Why did the team make Triforce%?

The short answer is, to finally get the Triforce in OoT and make the dreams of millions of players come true. Other answers include: 
   * to raise hundreds of thousands of dollars for MSF (Doctors Without Borders)
   * to explore the relationship between beta content and urban legends in OoT
   * to create a new type of video game related creative work, which has almost never been seen before (prior example was MrCheeze putting Mew under the truck with ACE)
   * to show off what can be technically achieved on a real N64 console

So is the Triforce actually obtainable in Ocarina of Time?

The answer to this question depends on your philosophy. Some good answers to this question would be: 
   * “It is now!”
   * “If you are a superhuman with 12 arms who can accurately press buttons on four controllers 480 times per second, then yes, you can get the Triforce!”
The scene where Link gets the Triforce shown during Triforce% was created by our team and injected into the game via controller input. None of the data comprising this scene (room model, cutscene, music, custom Link animation, etc.) is in the cartridge ROM.
However, the glitches which can be used to get arbitrary code execution ARE in Ocarina of Time. They were not put there intentionally by the developers of course, but they are as much a part of the game as any other part of the game’s programming. Since the player can use those glitches to get the Triforce, there is a sense in which getting the Triforce is indeed now part of the game. Of course this also means that anything else the player is able to create, program, and inject via controller input–in our case, Breath of the Wild Link and Zelda and all of the people in the Twitch chat–also become a part of the “““““canon””””” of Ocarina of Time.

Why does Breath of the Wild Link speak Japanese?

The primary reason is that the Japanese version of BotW gets closer to having Link speak than the English version does. Some have conjectured that the Japanese audience is more willing to accept Link speaking–being more of his own character rather than just an avatar–than the Western audience. And, after all, Link has had Japanese voice actors for his vocalizations since OoT, even though these have not included actual speech. So, it seemed more appropriate to give Link a Japanese voice than an English voice.

What was the significance of X element (e.g. Unicorn Fountain, Overture of Sages, etc.) shown in the run?

See SwankyBox’s video and Hard4Games’ video for more information on the lore.

Surely this is a romhack playing off of a flashcart?

No. The cartridge is a completely unmodified, original OoT V1.0 (US) cart. The N64 used during the SGDQ marathon has an RGB mod, but this is a passive mod to the video output circuit to get better video quality and does not affect the game contents in any way. All of the custom content shown during the project was injected into the game live through controller input.

I saw some references to a “shortcut” Triforce% ROM and a “romhack” Triforce% ROM. What are these?

Shortcut: This is a copy of OoT which has one modified actor, which gives arbitrary code execution (ACE) “for free” without needing any complicated setup. From there, the rest of the data is injected through controller input just like in the real run. This was used by the development team to test most of the contents of Triforce%, because no one on the team besides Savestate is actually skilled enough to do the ACE setup by hand. You can build this mod with our toolchain from an original 1.0 ROM. This was NOT used during the live event at SGDQ.
Romhack: The Triforce% build toolchain also outputs a romhack version of Triforce%, which is a modified ROM which contains all the custom contents built-in instead of being injected through controller inputs. This was used by the development team for testing purposes. You can build a smaller, partial version of this with our toolchain from an original 1.0 ROM. This was NOT used during the live event at SGDQ.

Where can I get a copy of the ROM / all the data injected?

For legal reasons, we cannot distribute any ROMs, and we also cannot distribute any data files which contain any copyrighted code or assets. All of the tools, code, and assets created by the Triforce% team are released here. Because this release does not include any copyrighted content, it is not possible to reproduce the full Triforce% run from this data. Besides, Triforce% was not created to be a fully playable “game”; if the player were to go outside the plot shown in the live performance, at best they would simply find the vanilla game, and at worst they would encounter crashes and other issues.

Was any of the beta content shown copied from the Gigaleak or the Spaceworld ‘97 overdump?

No. In fact, Triforce% was in progress before the Gigaleak was released and was mostly done before the Spaceworld ‘97 overdump was released. All of the content which was not already in the OoT cartridge was created by the Triforce% team based on the beta trailers and such.

What is TAS (in the context of this run)?

A TAS is a Tool-Assisted Speedrun. The tools used vary by game but often consist of using an emulator with features to help record inputs and retry things. Sometimes letting the player see game values, create savestates to go back to and retry a section, slow down the game, or even go frame by frame so specific inputs can be entered for every individual frame.
However, in Triforce%, the inputs TASBot plays into the game were not created this way at all. They were generated by scripts which convert various programs the team wrote into controller input. Emulators were only used when initially debugging these programs, but most of Triforce% was developed on a real N64, not using emulators at all.

What is TASBot doing?

TASBot the cute robot is a mascot for a team of people working towards getting TASs to actually run correctly on real consoles through the use of a TAS replay device. You can find this team at https://discord.tas.bot/

What is a TAS replay device then?

A TAS replay device is basically a controller adapter but better. It takes the TAS input file and actually feeds the necessary inputs to the console when the console needs it. This is the little box which TASBot holds on stage, which connects between a host computer and the game console’s controller ports. (See Shop.TAS.Bot for pre-built TAStm32 replay devices when they are in stock, although all replay devices made by the TASBot community are open hardware designs available for anyone to build for free.)

Where could I learn to make something like this?

Can I still contribute to N64 / OoT / TAS development if this all seems over my head right now?

You sure can! Hop into any community that seems interesting and ask around. Do consider trying to read into things if you can find things to read, but most speedrunning and adjacent communities are quite helpful to those seeking to give things a try. If you’re not sure where to head first any of the linked communities may be a good start.

How did you do all of this?

21 years of reverse engineering effort by the OoT community, and then an additional 2 and a half years of development effort by the Triforce% team. Over a thousand hours by more than 25 people: programmers, artists, musicians, voice actors…
For a technical explanation of how we use glitches to obtain ACE and then use ACE to get Total Control over the game, see the Retro Game Mechanics Explained video on Triforce%.

How did you get the Twitch messages in the game? Were they actual messages from the live chat?

Yes, during the SGDQ 2022 showcase they were real live chat messages. In any of the videos created in advance, you’ll see fake messages with usernames made of random characters which were generated by a script to simulate the Twitch chat.
As far as how it was done, some scripts on the computer controlling TASBot connect to the Twitch API and read the chat. Chat messages are filtered and metadata about them is encoded and sent to TASBot. TASBot injects them into the game via the controller ports, in a similar way to how all the other custom content was injected. A custom actor in the finale scene renders the messages in the sky, in a vaguely similar way to how they are rendered in a web browser.

How did you do the cel shading?

We hope to eventually make some sort of video explaining this, but the short answer is that it is using alpha compare and drawing each triangle once per cel level. A patch to the F3DZEX RSP microcode moves the lighting information from the shade color channels to the shade alpha channel. This was initially conceived by Sauraen, first implemented and tested by glank, and then integrated into fast64 by Sauraen. The fast64 display list generation code is here. Two other, completely different, implementations of cel shading on the N64 were created in January 2022, one by James Lambert and one by Wiseguy (no public video). Our implementation was made in October 2021 but was not made public and is different from both of these.

How did you make the Running Man boss fight?

Generally, the same way we made all the other custom content. As far as the Running Man boss himself, the original 3D model and skeleton was imported into Blender, IK rigged, and animated by rankaisija. rankaisija also wrote his AI and most of his other code, and Sauraen wrote a few small parts. The Running Man boss is a separate actor from the custom Running Man who meets us at the end of the race in the Lost Woods, which is also separate from the Running Man actors in the vanilla game.

How did you make the voice acting?

Again, generally, the same way as we made all the other custom content. Audio is just more data which was injected through controller input. But to spell it out in more detail: 
   * Our voice actors recorded the lines in their home studios and sent us the WAV files. (They are not the official voice actors from BotW / Nintendo, but they do both have professional voice acting experience.)
   * We used a tool to encode the audio files into N64 format.
   * The scripts on the computer converted this data to commands to the TAStm32 replay device (the box TASBot was holding) over USB.
   * The TAStm32 converted this into individual controller inputs and sent them to the console.
   * The hyperspeed loader converted the controller inputs back into data and placed it in the Expansion Pak RAM.
   * Patches to the game’s OS audio code (which is itself stored in RAM) allowed it to read samples from RAM instead of cartridge ROM.
   * Other patches replaced entries in the game’s lists of SFX for Ganon and other boss voice sound effects with pointers to our new SFX.
   * The BotW Link and Zelda actors’ custom code triggered playback of the Ganon / boss SFX like any other SFX, which caused the voice lines to be played in game.

The ROM is, by definition, read-only. So how did you change content within existing parts of OoT which were loaded from the cartridge?

There are three overall techniques which were used for patching the game. 
   * After the game loads anyfile from the cartridge, it checks a list of patches which have been loaded into memory via controller input. If this is a file we want to patch, it then applies our patch to that data, in the form of a list of differences (e.g. move to byte 100, write 2 bytes: 00 01, etc.). This happens before that data is “returned” to whatever code requested to load it. This method is used for making relatively small changes to things like scene files or actor overlays (code).
   * We can replace a whole file which is normally on the cartridge with a copy we uploaded into RAM. That is, we change the file list (which is stored in RAM) and overwrite the entry for that file from the ROM with a special entry telling it to load from our injected RAM data instead. This is used for things like fully custom scenes, or actors which needed larger changes like the Gerudo guards and Zora’s Domain ice/waterfall.
   * We can patch the code (stored in RAM) which asks to load the data from ROM, so that it doesn’t try to load from a ROM file at all in cases we want. This is used for types of data which are normally streamed in uncompressed format from the ROM, including music sequences, sound effects, and Link animations, and types of data which have tables separate from the file list, like completely custom actors and objects.

Credits

In-game credits

Director
  • SAURAEN
RTA Speedrunner
  • SAVESTATE
Scenario
  • SAURAEN
  • REBECCAETRIPP
  • TERUSTHEBIRD
  • DWANGOAC
Scene Design
  • CDI-FAILS
  • ZEL
  • SAURAEN
Music
  • SAURAEN
  • REBECCAETRIPP
Screen Text
  • KIM-SUKLEY
  • SAURAEN
Cinema Scenes
  • SAURAEN
  • DWANGOAC
  • DEFENESAM
BotW Model Conversion
  • ALI1234
  • SAURAEN
Animation
  • UNESAG
  • SAURAEN
  • RANKAISIJA
  • AEROARTWORK
Zelda (English Voice)
  • SAOIRSE
Link (Japanese Voice)
  • ZERO
Translator
  • YUKLOJ
Textures
  • CDI-FAILS
  • KIM-SUKLEY
  • ZEL
  • SAURAEN
Actor Program
  • SAURAEN
  • RANKAISIJA
  • Z64ME
  • MNGOLDENEAGLE
  • ZEL
Game / Actor Patches
  • SAURAEN
  • MNGOLDENEAGLE
System Patches
  • SAURAEN
  • ZEL
Cel Shading
  • SAURAEN
  • GLANK
Bootstrapper Chain
  • SAURAEN
Hyperspeed Loader
  • TERUSTHEBIRD
  • SAURAEN
Host Frontend
  • SAURAEN
  • THEMAS3212
TAStm32 Firmware
  • OWNASAURUS
  • SAURAEN
SRM / ACE Setup
  • MRCHEEZE
  • SAVESTATE
Build Toolchain
  • SAURAEN
  • Z64ME
Video Editing
  • MUSKET012
  • GRAVATOS
  • SAVESTATE
  • SAURAEN
Technical Support
  • ZEL
  • Z64ME
  • MZXRULES
  • THARO
  • WISEGUY
  • JACK WALKER
Special Thanks
  • KAZE EMANUAR
  • XDANIEL
  • ARIANA ALMANDOZ
Executive Producer
  • DWANGOAC
PRESENTED AT
  • Summer Games Done Quick 2022
  • Assets, Toolchain, and Performance Copyright (C) 2019-2022 The Triforce% Team
  • The Legend of ZELDA: Ocarina of Time Copyright (C) 1998 Nintendo

Partner credits

Partner Creators:
Partner Reactors:
OST Published By SiIvaGunner:

Press coverage

Samsara: I would say "judging" here, but that's not accurate to what I'll be doing. Instead, let's call it "figuring out how to accept this, and not stopping until I find a way".
Samsara: This is going to require some internal discussion and new site infrastructure, so I'm setting to Delayed until we can work it all out. We have a lot going on, so it may take some time, but it will happen.
Samsara: Hey, hi, it's me, Samsara. Just wanted to provide a quick update on this run that definitely hasn't been on the workbench for 8 months please don't remind me.
In short, the delay is 100% on me. The stuff we need is all drafted, the infrastructure stuff just needs to be properly condensed and passed off to the site devs, and I have admittedly been slacking on that. There's been a lot of pressing site matters to attend to and they've unfortunately all been pushing back what we need to be able to accept this. Again, it will happen, and it will remain on the workbench until it does, and I apologize for taking so long to be able to do so. Hopefully it'll happen before the one year anniversary of submission, at least!
Last Edited by fsvgm777 on 7/15/2023 5:32 PM
Page History Latest diff List referrers