Submission Text Full Submission Page

Final Fantasy "Console Verification"

The fm2 submitted is set up to run from the standard clean start. However, the true form of this run is a system that can create a TAS of the game for any starting SRAM state. All the code needed for this is available here: [dead link removed]
I got the idea for this after dwangoAC described how he and realtime FF runner Feasel attempted to run the existing Final Fantasy TAS on an NES in the practice room at AGDQ 2014. They were unsuccessful but I realized that I could work around both of the major problems. dwangoAC asked me if I would be interested in preparing this for AGDQ 2015. Even though it was determined to not be a good candidate for the event I decided I would finish it for console verification.
The first problem is that no bot can yet play back a run with resets. There are plans and projects, but nothing working yet. Since my runs on this game use many soft and hard resets, such a system would have to work reliably and consistently to make it through the current TAS.
To get around this I decided I would make a new TAS that did not use any resets or power cycles. At first I wasn't sure how much time this would add. It could be impossible to avoid a large number of unrunnable fights. Fighting those battles would take time and possibly require more healing items, getting more equipment or other setup. Or maybe I could avoid unrunnable fights but with a lot of pacing and extra random runnable battles in order to advance the encounter mechanism.
After some analysis I discovered that it wouldn't actually be that bad. This run has 184 random fights compared to 173 in any%. Some extra walking or sailing in circles is needed, but I discovered some other tricks to tweak things as well. There are only four unrunnable fights compared to two in any% and all four go pretty fast. So in the end this run is only about a minute and a half slower than the any% run.
A problem not unique to this game is that RAM is not initialized by the game at startup. Some versions of NES hardware will have cleared RAM after sitting powered down for a while. A more conclusive solution is the erase cart developed by true and Scrimpey that actively resets the RAM.
The next problem is that the seed for the battle RNG is stored in SRAM, the battery-backed memory used for the savegame. That means clearing the RAM will not get to same start state as the standard clean start state for TAS. In this case the hardware solution is much simpler: a cart eraser that deletes the SRAM memory. When I first started this did not exist, but true and others have since gotten it work.
The non-hardware solution to this is a little more complicated but pretty cool. Regardless of the SRAM state I can walk through the outer world until the first battle before desyncing. I do this and on starting that battle I hold down A, ordering my guys to attack the first enemy. That battle will consist of 3-5 imps. Four fighters beat them every time, but will take varying amounts of damage and time to do so. In the worst case it could take about 90 seconds. Anyway, by looking at what happened in that battle we can determine the value of the RNG counter. Specifically, I found that recording the final hp of everyone in the party would find the counter value about 90% of the time. Throwing in the number of enemies and the first and last party members to attack would be enough to find the value every time.
Once you know that value there are a few ways to proceed. You can hit the power and start all over with a run calibrated to that counter value. true and I were able to get the run going this way. My other idea was to let the bot know the counter value and have it tweak the run in real time. This is certainly possible and would have been a neat trick but it did require some hacking in to the bot's code and maybe some hardware tweaks to set up a port to get that info to the bot.
Continuing on from an unknown start has a few other complications. After that first fight my party members have an unknown amount of hp and in fact there is about a 5% chance that the first character dies! I deal with this variation by picking one of three different battles with first boss Garland and then going to the inn on first entering Pravoka. Still, if my first character died in the imp fight then the characters were in a different order on beating Garland and randomization of levelup stats means that my characters have different amounts of max HP. The variation is only 2-4 so I deal with it. It's only really an issue for the pirates fight.
In the end, the cart eraser is much more convenient than these tricks. But since I have everything set up I am going to release all the code needed to do this. The process involves using python scripts to generate lua that can then be used to record an fm2 tuned to a particular RNG value. As with the any% run I have also included the option to change the names used in the run. I have used lua to test that the run has synced and verified that my system can sync for any starting RNG value.

About the run

The run starts pretty close to the any% run. The first big change is skipping the treasure in Dwarf Cave and picking up some treasure in Earth Cave instead. That side trip in Earth Cave helps avoid unrunnable fights without just walking in circles. I pick up the Coral Sword for a better weapon and also a cabin for healing later. The Coral sword is used on Vampire and Lich instead of the Dragon sword. In the second trip to Lich I have my first unrunnable random fight against two Wizards. With a better weapon and a few levels compared to the fight in Marsh Cave I am able beat them much more easily.
After Earth Cave I proceed directly to Crescent Lake and Ice Cave with the exception of a short sailing excursion to prime the encounter list. This leads to no unrunnable encounters in Ice Cave. In Ice Cave I collect the Flame Sword and tent as in the any% run as well as a good chunk of gold to spend on the Bottle.
On the way to the airship is the longest set of extra pacing to set up encounters. I get on and off the ship because the ship moves faster and the encounter rate is higher on land. Even though the step on and off the ship does not increment encounters it is still worth it. This will also be the last place I need to move in circles just to set up encounters.
While using the Floater to reveal the airship I also equip the flame sword and use the cabin and tent to restore some hp.
The Sea Shrine and Waterfall are pretty straightforward. Still no unrunnable fights and about the same as any%. As usual, the robot is just where I need him to be and Dr Unne of Melmond moves down to talk to me.
The trip to Lefein is a little different because of two unrunnable ZomBull fights. ZomBulls are weak to flame and I have the flame sword. So even on a very low level I am able to beat them in one hit if I take some damage first. That was the purpose for restoring my hp earlier. This will be the last time I take damage in battle.
Moving on, I beat the Blue D without taking damage. Pick up Bane sword and use it on Tiamat as usual. Gurgu is a little different; instead of walking on lava to avoid encounters as much as possible I step on normal ground to run up the encounter list. This sets everything up for a fast trip through the final dungeon.
Temple of Fiends: Revisited actually goes quite a bit faster than in any% run. Because of how encounters fall, I can use Bane Sword on Phantom with no worries. The only and last unrunnable random encounter is one WORM that is quickly dispatched with Bane Sword. The rest of the dungeon is straightforward Bane Swording.
There were a couple spots where taking more damage would be convenient. But after looking into it, nothing was worth the time it would take to shop and use items or visit an inn. Unlike any% I don't need to use tents or cabins to save before reset, so I don't have any excuses to visit item shops or even open the menu.

Thanks

Special thanks to true for playing this on his bot to verify. He also verified that the RNG finding method worked on console.
Also thanks to dwangoAC for including this as an option for AGDQ though it wasn't accepted. Since it is so easy to set the character names I was hoping we could auction or raffle off the names in the verification movie to raise money for his AGDQ 2015 efforts. This will take some more planning and discussion though.

adelikat: I'll handle this one

adelikat: Nevermind, backing out of judging this.
Noxxa: I'll take this submission, then.
Noxxa: This submission is made with console verification in mind, by avoiding resets that require the console's RAM to be cleared when console verifying. The justification given is that, when console verifying the run, the resets would cost time, as the cart would have to be removed, the RAM-clearing cart inserted and run, and reinserting the game cart before the verification process can resume. This would, in the end, make this run faster on console than the currently published run. However, we don't account for this when timing runs; this is because it is impossible to time the cart-swapping, clearing etc. for TAS timing purposes. Therefore, this added time is disregarded. As such, as this run is slower than the currently published run and the run does not meaningfully add to the current published runs, rejecting.
As a side-note, given this run's goal to showcase a bot that has to figure out RNG on the fly from any SRAM by letting a fight run while holding a button, I have doubts whether this is an optimal run, compared to a proper TAS that would use savestates to know the RNG in advance.

Editor, Player (68)
Joined: 1/18/2008
Posts: 663
TheAxeMan wrote:
Nice! The explanation of the erasing process in the beginning is good. The controller visualization is nice. Looks like it holds the buttons through lag frames so it doesn't exactly match the fm2.
This matches exactly the FM2. It doesn't matter what is done with the button until it is polled for, so it is "pressed" before it is polled. The NES asks the controller what button is being pressed and the controller responds; the controller cannot interrupt the NES and force it data. As Final Fantasy polls so infrequently it looks to be "holding." It is completely accurate here.
TheAxeMan wrote:
Also, it looks like you are using the generated result (that we ran earlier right when I finished). I changed the submitted fm2 to run from the first fight, saving a couple minutes.
I ran the original posted movie. Did you change it after it was submitted?
true on twitch - lsnes windows builds 20230425 - the date this site is buried
Ambassador, Experienced player (697)
Joined: 7/17/2004
Posts: 985
Location: The FLOATING CASTLE
true wrote:
This matches exactly the FM2.
I guess I should have said that it doesn't match FCEUX. When you play a movie back on the emulator with the input display on it shows the input frame by frame regardless of controller polling. I don't question the accuracy and obviously the result is the same. It's just that the conversion from frame-based to poll-based input makes it look different.
true wrote:
Did you change it after it was submitted?
There have been no changes since I submitted but there have been since I sent you the first full fm2. I changed it so that I run from the first battle, saving a couple minutes. The encode in one of the first few posts has the right movie.
Joined: 11/17/2005
Posts: 278
Location: Massachusetts, USA
jlun2 wrote:
If this gets accepted, I don't think it should obsolete the current TAS due to reasons stated here. I know its the same author for this particular TAS, but this can potentially be applied to other games as well.
In the case of submission 3389 (Pokemon Yellow) the new submission was merely a resynch. Masterjun even said that the strategy was the same. He just re-executed the same run on a more accurate emulator. He did no original work. In this submission the route and strategy has changed. The core of the argument is that emulator accuracy has affected the route. Emulator-specific glitches have been banned from Super Metroid and Kirby Superstar where the glitch was only possible because of an emulator fault. This submission is passively saying that most reset button use is inaccurate or at least inaccurately timed. And that if the current TAS were subjected to fair resets that this run would beat it. (I don't know what a fair reset is, but magically zeroing out all the RAM in one frame is unfair.)
Editor, Player (68)
Joined: 1/18/2008
Posts: 663
TheAxeMan wrote:
true wrote:
This matches exactly the FM2.
I guess I should have said that it doesn't match FCEUX. When you play a movie back on the emulator with the input display on it shows the input frame by frame regardless of controller polling. I don't question the accuracy and obviously the result is the same. It's just that the conversion from frame-based to poll-based input makes it look different.
Not at all. I dumped by frames. The bot has no idea what a "frame" is and only knows what is polled. (My old bot tried to guess but did a shit job at it, so I dropped it.) Thus, it can't know there is deadtime unless I program in fake deadtime. But if I do that, then the button is hardly lit, which makes it less visible. So why do that? One could autofire in that time. It doesn't matter. The console never sees it. And I give a shit less about matching FCEUX; FCEUX should match the NES. If anything, consider anything shown on unpolled input frames a meaningless artistic show.
TheAxeMan wrote:
true wrote:
Did you change it after it was submitted?
There have been no changes since I submitted but there have been since I sent you the first full fm2. I changed it so that I run from the first battle, saving a couple minutes. The encode in one of the first few posts has the right movie.
I was pretty sure I downloaded this one. I guess my surety doesn't matter. Oh well, it can be redone and the video updated if necessary. This is >99.6% identical right now anyway.
Catastrophe wrote:
In the case of submission 3389 (Pokemon Yellow) the new submission was merely a resynch. Masterjun even said that the strategy was the same. He just re-executed the same run on a more accurate emulator. He did no original work.
That was an emulator-to-emulator resync. This is an emulator-to-console resync, which has a completely different purpose. After all, we try to TAS games here, not their emulators. Any further provided reasoning is less important than this reason alone, or serves to back up this point.
Catastrophe wrote:
This submission is passively saying that most reset button use is inaccurate or at least inaccurately timed.
This isn't saying this passively or actively. What is being said is that a run with resets cannot be run in real-time in the amount of time shown on-site. There is no exception or corner case. Once a bot supports resets, using them may or may not be deterministic. This is what we don't know right now.
Catastrophe wrote:
(I don't know what a fair reset is, but magically zeroing out all the RAM in one frame is unfair.)
I know you weren't saying this, but as it could be misread this way, let me state: Resets do not clear memory. Power cycle / POR will typically have most/all console memory cleared. Resets jump code to the reset vector. POR is also at the reset vector. Power cycle is instant in an emulator, but will take some seconds to hours on a console. Reset, done physically, is also likely to take some physical time to do, which can vary.
true on twitch - lsnes windows builds 20230425 - the date this site is buried
Site Admin, Skilled player (1237)
Joined: 4/17/2010
Posts: 11280
Location: RU
So... what's preventing judgment now?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3601)
Joined: 11/3/2004
Posts: 4739
Location: Tennessee
I'll be honest, I have no idea what to do with this submission :/
It's hard to look this good. My TAS projects
Noxxa
They/Them
Moderator, Expert player (4144)
Joined: 8/14/2009
Posts: 4083
Location: The Netherlands
Let me try to get straight what the deals are with this run (please correct me if I'm wrong anywhere):
  1. This run is really a showcase of a bot that clears the game from any RAM and SRAM state, although the run itself is of the game being run from cleared RAM state (matching FCEUX' default "cleared RAM" state) and erased SRAM state (so that the run would sync on emulator, as well as on console after clearing/erasing RAM and SRAM)
  2. This run avoids using resets for console verification purposes. If I am to understand this correctly, it's because the emulator rewrites RAM to its cleared state when resetting, and to console verify, one would have to put in the erase cart to clear RAM every time the game is reset, which takes time. (If there are other reasons resets are avoided, please let me know.)
This raises a few questions for me:
  1. This bot relies on holding A during the first battle, so that it figures out the RNG using the outcomes of that battle. A neat concept, sure, but is it optimal? It seems to me like it would be slower than a (manual) TAS that uses simple RAM watching and/or disassembly to figure out the RNG, without needing to do the first battle like this.
  2. Assuming I'm correct on point 2 above, let's say the emulator is fixed so that it does not reset RAM on power cycle when recording. Would console verification without using the clear cart mid-run for power cycles then possibly work? If not, what else could an emulator do to make reset recording work for console verification (without erase carts or other time-consuming processed)?
http://www.youtube.com/Noxxa <dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects. <Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits <adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
Joined: 7/2/2007
Posts: 3960
Mothrayas wrote:
This run avoids using resets for console verification purposes. If I am to understand this correctly, it's because the emulator rewrites RAM to its cleared state when resetting, and to console verify, one would have to put in the erase cart to clear RAM every time the game is reset, which takes time. (If there are other reasons resets are avoided, please let me know.)
My understanding agrees with yours. Without using a RAM clear cart or similar trick, RAM decay is nondeterministic, so games like FF1 that rely on initial RAM state would be impossible to console-verify.
This raises a few questions for me: ... Assuming I'm correct on point 2 above, let's say the emulator is fixed so that it does not reset RAM on power cycle when recording. Would console verification without using the clear cart mid-run for power cycles then possibly work? If not, what else could an emulator do to make reset recording work for console verification (without erase carts or other time-consuming processed)?
This runs into the "RAM decay is not deterministic" issue I mentioned earlier. When you turn the console off, its RAM state should eventually hit zero across the board (ignoring random bitflips from ionizing radiation and the like, which do happen, albeit rarely). As I understand it, this decay is broadly like radioactive decay -- there's a half-life, so after T amount of time, half the bits will (on average) be zero, after 2T 3/4ths of them should be zero, etc. But there's no guarantee that they'll ever all be zero, let alone that certain ones will be zero. Conversely, I don't know what the half-life of a bit in RAM is; I imagine it's pretty short. It's possible that even an extremely fast reset would still result in some of the RAM decaying. But I'm not an expert on this so I'll bow to people who actually know how the hardware behaves.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
TASVideosGrue
They/Them
Joined: 10/1/2008
Posts: 2739
Location: The dark corners of the TASVideos server
om, nom, nom... juicy!
Editor, Player (68)
Joined: 1/18/2008
Posts: 663
Not quite there. TheAxeMan said the submitted run is faster than the one I posted and doesn't have the input delay anymore (I think?) That said, I think this "delay" is the length of the longest possible battle? The hold-A run can be played back with the use of only erasing the console RAM, not requiring a particular and possibly invalid situation to erase the cart. Basically it will play back on any cart after preparing using the values determined from the first battle. It's a bit unique. If only the fastest is desired, again, I think that is what this run is now? Re: SRAM it won't decay when powered, and I think NES reset keeps it powered? (but maybe bus and data is floating and can do odd things? I don't know) Only once theory becomes practice and we know how it will work I think the question of how to proceed with resets should be answered. Really, the real bug here is the FCEUX startup state.
true on twitch - lsnes windows builds 20230425 - the date this site is buried
Skilled player (1708)
Joined: 9/17/2009
Posts: 4952
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"
I tried seeing the code but:
File was deleted from server. File was deleted by owner or exceeded maximum storage time (100 days from last download).
:/
Moderator, Senior Ambassador, Experienced player (900)
Joined: 9/14/2008
Posts: 1007
jlun2 wrote:
I tried seeing the code but:
File was deleted from server. File was deleted by owner or exceeded maximum storage time (100 days from last download).
:/
Hmm... I'll check in with TheAxeMan when I can; I know he is currently distracted with Real Life right now so it could be a bit. I should still have a copy of the code somewhere if nothing else. As a side note, please see the technical tier thread about a possible method to publish this TAS in some form.
I was laid off in May 2023 and could use support via Patreon or onetime donations as I work on TASBot Re: and TASBot HD. I'm dwangoAC, part of the senior staff of TASVideos as the Senior Ambassador and BDFL of the TASBot community; I post TAS content on YouTube.com/dwangoAC based on livestreams from Twitch.tv/dwangoAC.
Ambassador, Experienced player (697)
Joined: 7/17/2004
Posts: 985
Location: The FLOATING CASTLE
Ah, I'll have to fix that sometime. I also need to fix earlier downloads that were using google code and probably switch to public github. I now have a decent internet connection so I can do it if I find the file.
Joined: 1/27/2014
Posts: 181
Interesting. Read through the discussion. I would support this video being published and replacing the current faster video due to better emulation and console verification. We don't penalize optimal runs for inserting lag frames. I don't think a separate category is needed.