Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
CasualPokePlayer wrote:
Alyosha wrote:
That's what it does for WRAM, but what about HRAM (FF80 - FFFF)? I see in Gambatte initstate.cpp that it has a 'ffxxDump' array that it initializes from, but where is that from?
I don't think that's based on any actual testing, it seems to be just a padding value put in (not like it super matters anyways, the values would very likely be cleared out by the game, and also more accurate behavior would essentially be non-deterministic... which is bad for the purposes of a TAS).
With a name like 'ffxxDump' it seems like it must be dumped from somewhere. The BIOS uses about half of HRAM, and the values in that array match what is left in HRAM after the BIOS ends (for the ones it uses) so I'm guessing at some point in the past it was dumped from a GBC or GBA.
Anyway, I looked at some trace logs of what the game is doing with unitialized RAM:
So basically, as long as these 3 things are true, the game will sync. I made a much longer run with 0xFF state and it synced completely (~91000 frames.)
Actually, all of the above are true for Gambatte as well:
FFCD = 0x97
FFCC = 0x46
FFC9 = 0x6D
It must desync for a different reason, I'm trying to figure out what that is.
EDIT: there is at least one difference so far, reading from STAT in the transition from LY = 0 (line 153) to LY = 0 (line 0):
GBHawk:
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
TiKevin83 wrote:
We verified Kirby's Pinball Land last night. I'll link here when I have the GBI movie archives finished for the last few verifications
Cool, that will be a neat one to see.
CasualPokePlayer wrote:
It largely depends on where it's reading uninitialized RAM (I believe some places in RAM would have swaths of 00 data and then another place with swathes of FF data, with random bits not set in the FFs and such), and besides that, a lot of random bs in the real world.
That's what it does for WRAM, but what about HRAM (FF80 - FFFF)? I see in Gambatte initstate.cpp that it has a 'ffxxDump' array that it initializes from, but where is that from? And is it different for GBA then for GBC? GBA handles some memory related things differently from GBC, so it might be worth getting a dump from a GBP.
So far in my testing there is a consistent de-sync in gambatte with it's HRAM initial state and GBHawk with it's 0x00 initial state, and a consistent sync with 0xFF initial state.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I made test runs of Prehistorik Man (worked fine) and RC Pro AM (desynced due to reading from uninitialized RAM at FFCC-FFCD.) R.C. Pro AM also desynced with a run made in Gambatte. It's a bit disappointing to find such a high percentage of games that use uninitialized RAM even just in the limited number of tests I've done.
GBHawk initializes RAM to all zeroes, both WRAM and HRAM. I did test HRAM initialized with all 0xFF instead and that run did sync on 2 spereate tries. So maybe starting up with all 0xFF is accurate? At least for HRAM maybe, WRAM seems to be much more random in my testing with Dirty Racing.
On a side note, it's been 3 whole years since I started this thread, where'd all that time go! It seems like there's always more to do and less time to do it, I'm happy to still be making steady if slow progress though.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I did some follow up research on Warioland GEG. First I tried Double Dragon 3 in GBI since it is a rare example of a game that uses the controller interrupt to read the inputs, so I wanted to verify everything worked. It works fine for both normal play and when playing back a movie.
I still believe that control inputs should un-stop the system correctly though for the game end glitch run, so I figured something else must be up.
Looking at the trace log, it turns out the STOP was an incorrectly formatted STOP, STOP $04 instead of STOP $00. I didn't notice this before since the trace log just ignored the second byte and the Hex Editor displayed 10 00 which is STOP $00 but that is only true at the start of the frame. Since the GB hardware test ROMs indicate that STOP $## do not behave correctly, this seems to be the problem. I'm not sure if everything can be made to line up correctly for a STOP 00 with everything else correct as well, but I guess I'll try testing. hopefully I can get an instance of a STOP 00 that does something after un-stop so at least I can verify that it actually happens.
EDIT: actually it seems like STOP XX shouldbehave normally, so I'm not really sure what's wrong here.
EDIT2: Warioland II is another game that either needs SaveRAM flashed to FF or a movie that starts from default saveram that gets written on data clear. It won't sync otherwise.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
Cool, I'll get to LADX eventually but seems like it's not that strict.
The game Race Days does not verify on either Gambatte or GBHawk (if you pick Dirty Racing at least.) It reads from uninitialized RAM at C224, so this is probably the culprit.
EDIT: Race Days reads lots of memory locations before writing, it looks like it's testing if if needs to clear the regions, but anyway it desyncs in all kinds of ways with every reset, probably no hope for that one unless by chance you can reset right after it's done initializing and nothing changes between loading segments.
Wave Race seems to work, but you either have to flash SRAM before starting or use a movie that contains the game's default SRAM after clearing, otherwise it will desync.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I've started looking at gambatte's hardware tests to see if they could help at all with Men in black. It has a lot of useful HDMA tests, and it turns out that I was starting HDMA too early. After fixing that Men in Black behaves much closer to Gambatte, but still doesn't work on console due to STAT timing not being exactly right. The HDMA timing issue also effects single speed mode, so I went ahead and committed the fix, but hardly any games actually care about when HDMA starts, only how long it takes, so this didn't effect any other runs in my testing, even Pokemon TCG which requires perfect accuracy in a lot of other ways. It's a pretty lucky coincidence that Men in Black just happens to require the most complete accuracy of any game I've tested so far.
So far I have not found any deficiencies in single speed testing. I partially re-syncing the Metroid 2 select glitch run since it's very glitchy and it worked just fine. The resync process is quite horrible though as RNG does not work out at all. I managed to get up to the first 7 metroids but I don't think I'll go any further, it's just too much effort. I also recorded a quick movie of Micro Machines by hand for the first few tracks and it also worked fine. I'm going to keep testing with a few more games before trying to tackle double speed STAT timing.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I updated the control scheme based on my own testing. Barring something demonstrably wrong, this is the last time I'll update it, so anything made now should work with 2.5 when it's released.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I updated controls for Kirby Tilt n' Tumble to better match what I get on Console; it's pretty close now, but there are a lot of factors that I didn't account for to simplify things. The control scheme doesn't work so well with a joystick because it's pretty non-linear, but works pretty well with a mouse as long as you make the screen size large and remove dead zone. I'm not very good at either console or emulator control though so individual results may vary.
I also console verified this version of Men in Black the Series for Gambatte:
http://tasvideos.org/userfiles/info/65096124653759503
The run I posted previously did not sync on console, due to it using equal length frames and there was a case where I guess input showed up too close to a frame boundary. I expected this kind of thing to happen, it's definitely best not to use equal length frame mode when TASing with Gambatte.
The GBHawk version did not verify, looking at trace logs it looks like mode 3 timing in double speed mode is the culprit (indirectly anyway, it's actually HDMA occurring slightly too early but that is timed off of the end of mode 3.) This is a known issue and something I'll start addressing after 2.5 is released. For now I'm going to continue with single speed mode testing.
Also I confirmed that Star Trek 25th anniversary has a system dependent glitch in the PPU where a text box border can be aligned or misaligned. Old consoles (GB and SGB) have it aligned and at GBP has it misaligned. Not sure about any other details.
Still a lot of R&D to do.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
EZGames69 wrote:
There is one case I forgot to mention.
Mario Kart 64 is considered console verified, but because you cant press the reset button with TAS inputs, they had to split the movie file on those resets. So it was verified with multiple playback files, where as the emulation uses a single file.
Is it not considered verified if it’s not a single file being played back?
It's not ideal, but my opinion on this is basically that it's ok, since we aren't really concerned with time that the console spends being off. That time is invisible in terms of emulation (as long as no RTC is involved), so nothing was really added, just made convenient, as in just a series of runs strung together by SRAM. Not strictly the same, but I guess there has to be a line of practicality somewhere.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
http://tasvideos.org/userfiles/info/65037296378073579
Here is a console verified version of Zen for GBHawk. This run actually failed at first because of behaviour of writing to the LY register. The GB CPU manual states that writing here resets the counter, but even if this is true for the GB, it is definitely not true for GBC, as the run will fail to console verify if this is the case. I have committed a fix accordingly. So this process has already revealed some important information. (Zen is the only game I know of that tries to do this, and does so for no real reason, it writes zero but then just waits for VBL shortly after.)
I have also verified a version of the Donkey Kong Land 2 TAS I have been working on up to beating K. Rool for the first time, so that was cool to see as well.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
RetroEdit wrote:
But that leaves a question: We have gotten a movie file that represents more accurate emulation, namely the movie file that has been resynced to real-console. How do we proceed? How do we treat the original publication?
...
But waiting around for emulation to be "fixed" doesn't resolve this either. One reason being our movie rules don't allow movies with identical gameplay to obsolete prior movies (unless you disagree with that rule?)
If our goal is to actually improve emulation, it would be better to improve the ability to communicate the console-verification status of a movie, rather than obscure that by making console-verification completely black and white. Having the site index this is much more convenient than requiring independent efforts to keep track of "verified but somehow not verified" movies.
I think a dedicated forum thread or github is a better place for movies that don't quite work then attaching them to a publication. An issue tracker is a great format for compiling these cases.
I do disagree with the rules requiring gameplay improvements, and think that a console verified version should obsolete an older de-verified one (with the caveat that authorship remains with the original author.)
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
RetroEdit wrote:
But I also think there's a huge difference between being able to verify a movie by adding a few frames to adjust for certain transition screens and not being able to playback on console at all.
I don't think so. If you have to modify it, then you aren't verifying the movie. Even if it's only one frame, the solution is to fix the emulator so it works correctly. I disagree with the rules here, and think the emulation environment is an important part of a TAS. I think a movie that works on a NES should be considered superior to one that only works on Not-A-NES.
In the particular case of mario advance, having a movie that creates the SRAM I think would be an ok solution to avoid non-deterministic stuff (although that sounds like something that needs a technical explanation) as there are already standards for movies that start from SRAM, but that would still leave the 10 loading frames inconsistency, which I personally don't agree with excluding and calling it verified.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
http://tasvideos.org/userfiles/info/64991397408720138
Not a GBHawk movie, but here is a console verified version of Zen Intergalatic Ninja by EZGames69. I'll try to get a GBHawk version too. There is a lot of lag and sprites and scrolling all going on at the same time in this game, so it's a good one to verify.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
http://tasvideos.org/userfiles/info/64970777987927243
Battletoads by lobsterzelda.
Apparently GBI doesn't play the last input in it's file, so I had to extend one frame so the last input would be empty, otherwise the last boss is left with 2 health. Otherwise, easy resync.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
http://tasvideos.org/userfiles/info/64967031374165337
I can finally get GBHawk runs console verified now. you can run the above script with a GBHawk movie and get an input file that works with GBI.
In it's current form it only works with single speed mode movies, but I'll be expanding capabilities to double speed and subframe in the future.
I have verified this version of Kwirk with it already:
http://tasvideos.org/userfiles/info/64882123397930984
More to come as I figure out the process a bit better.
I don't have a capture device so no video right now though.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
EZGames69 wrote:
The problem is you can’t expect console to be 100% deterministic, in the case of SMA4, the entire save data screen had to be removed in order to work properly. The time it takes to create is not deterministic and would be impossible to get right. Not only that but if this was the requirement, then the verifications would have to be removed from plenty of other games, such as Sonic Advance, Sonic Advance 2, chip n dale 2 players and 1 player, Pokemon Yellow, anything listed here basically http://tasvideos.org/ConsoleVerificationTests.html. I dont think it should wait until emulation is 100% accurate, because like I said before, emulation will almost never be 100% accurate to console, as console has weird quirks that an emulator would not be able to emulate in a deterministic way, like the save data thing with SMA4.
So you think Sonic Advance should have it removed only because it needed one lag frame added, but not say the same thing about NES? I find it really difficult to understand this position.
We have, and we always will. Part of the process that I mentioned in Case 2 was to ALWAYS provide documentation on what failed and what was required to get consistent sync. But the goal of console verification should not be a test of how accurate the emulation is, but rather a test of how accurate the gameplay in the TAS is. Do the inputs work on console? Do they produce the same results as on console? It just seems unfair to exclude verification if the only thing is loading time, which can be easily adjusted.
I'd be fine with removing the flag from NES games where extra obscure steps are needed. But, I'm also fine with verification that is accomplished through a consistent process even if the underlying emulation isn't 100% correct. This is because the verification flag is attached to a publication, so it should represent what the movie (combined with the emulator it was made on) can do by themselves.
There are other ways to present console verified input that aren't attached to specific movies, maybe make a page specifically for them. But for the actual flag that gets attached to publications, little to no modifications should be allowed.
Also, loading data is an essential part of what consoles do, I consider it pretty important to get the timing of that right.
Chanoyu wrote:
Would it be helpful to have a tiered system?
At least in my opinion, the movie file is source from which the verification either works or it doesn't, so I don't think any kind of tier is needed.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I prefer strict rules regarding what movies can be considered console verified. If you have to add or subtract frames, then the movie is not console verified, only the input file is. Personally I think of console verification of a movie file as a standardized and ideally reversible process. Take your movie file, run this standard script, use the resulting input file on playback device. If you need to add additional steps in there, like manually editing in frames, or per-game hacks, then you do not have a verified movie.
In practice, this does lead to cases where a movie can be verified according to a standard process but still be emulated incorrectly, as is the case for NES games where only input latches are used, so if lag frames show up in the wrong places it doesn't matter (sometimes) since they are ignored anyway. I believe this is the case for SMB2 at least. But, since the standard process was followed, it can still be considered a verification. This doesn't work for GB/A though, where cycles and not latches are used, so cases like Sonic and Mario Advance which require manual editing, are not verified.
I do agree that there should be a process where if a movie file can be imported into a more accurate emulator where manual editing is not required for console verification then this movie can be placed alongside the original movie in the publication and be considered verified.
EDIT: also if you are able to detect emulator inaccuracies in loading times etc please report them so they can be addressed.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
http://tasvideos.org/userfiles/info/64926196486444191
Here is a resync of the old pokemon yellow ACE movie for Pi Day.
It works except that it writes to VRAM at times that it's not accessible, so the result is glitched in appearance. If I let VRAM writes always succeed (not console accurate) it does look the same as the original.
My ultimate goal is to use SubGBHawk to have a console verified version of MrWint's new Pokemon Yellow ACE run, this test was just a first step. I'm not sure exactly how that will work out yet though.
Also I'm getting into console verification with GBI myself, and I'm open to suggestions on what existing GB runs people might want to see resynced and console verified. I think any game not using double speed mode or using hard rests or uninitialized memory should be verifiable now. But, the more games that get tested the more edge cases might pop up, so I'm just going to be testing all kinds of games.
Editor, Experienced Forum User, Published Author, Expert player
(3560)
Joined: 11/30/2014
Posts: 2744
Location: US
I'm not sure what you mean, but the returned cycle count accurately keeps track of time at a resolution of 4 MHz. , regardless of what speed mode the system is in at what time. Just divide the returned cycle count by 4194304 to get the time in seconds.
Thanks for this tutorial, I followed all the steps but it failed to load the files from the SD card. I think my SD card has too high a capacity, the website says 4 GB limit, but mine has 64 GB. Probably good to mention that somewhere for the tutorial.
