Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I've made a lot of progress understanding Grunty's Revenge. The game doesn't use halt, and bases RNG off a timer it starts shortly after power on, so everything needs to be perfect to get correct RNG. It's a really good test case.
Debugging this was difficult because I was getting confused by 2 unrelated errors that very coincidentally cancelled each other out. What are the odds! One error was in multiplication timing (6 cycles too long) and the other was a new behaviour in the prefetcher that I just discovered thanks to this game (which ended up resulting in execution being 6 cycles too short.) On ROM boundaries of size 0x20000, the prefetcher completely stops. These boundaries are known to generate forced non-sequential accesses, but it wasn't known that this causes the prefetcher to halt. I guess the prefetcher can only do sequential accesses. There is apparently also an extra cycle penalty when this happens.
After fixing these I was still off a bit, so something else was wrong. It turns out that the prefetcher still runs even when execution is happening from RAM or BIOS. Originally I had this as disabled. This usually doesn't effect anything because execution in these regions is almost always longer than 16 cycles, which is the time it takes the prefetcher to fill its 8 byte buffer and halt. When execution returns to ROM it is either at a different point, or the address before the first one in the prefetcher (since the pipeline is two instructions long) triggering a reset. However Banjo Kazooie jumps to BIOS from ROM and runs only 2 instructions after the end of IRQ code, which is short enough that having the prefetcher still active triggers some bus access delays.
With all of that fixed, now I can get as far as initial loading with exact execution timing. I can do a couple more tests up to first input, but after that it will just be a matter of running the TAS and seeing if it works. Unfortunately I can't get good RNG to resync the current run yet, so I'm a bit stuck. Hopefully I can guess and check my way to success.
All of this also improved sync on Shrek 2, which started desyncing in mid-run around GBAHawk 2.0.1 due to IRQ changes. Console was known to sync further than this so something was up but I didn't know what. With these fixes it now goes back to desyncing at the next to last level, where it originally desynced. Unfortunately these improvements STILL didn't seem to fix it. The search for a solution continues.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Cool! How many such soft resets would there be in a complete run? 75 frames is an inexplicably long time, not sure what's causing that, but I guess it's manageable as long as it's consistent.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I've been testing Grunty's Revenge, and am getting desyncs that look like they could potentially lead to diagnosing some emulation issues. It appears I am getting bad RNG, and after g0goTBC helpfully told me where to look, I found that the RNG is initially tied to one of the GBA hardware timers for initialization. I confirmed that EEPROM is not an issue, by testing with both an original cart and with a dev cart that doesn't have EEPROM and I got the same results. This is very promising, as it makes this a perfect candidate for some more test ROM injecting. I'll be diving into this over the next couple of weeks, hopefully something interesting shakes out.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Link to video
I was able to properly verify Metroid Fusion, so that run is verified, finally completing one of my original goals. I am working on 100% as well.
The resync was done using RetroEdit's resyncing tools, which are incredible and saved me hours of effort. They should greatly speed up future verifications where simple adjustment for lag / loading times are all that is needed. EDIT: 100% done as well, cool!
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I finished up verifying Wario Land 4. All resyncs were pretty simple. 4/4 runs verified. I'm going to try Metroid Fusion again, then move on to Metroid Zero Mission.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I have released version 2.0.3 of GBAHawk. This version adds back in ffmpeg encoding, which was requested by several people. For emulation improvements, I added basic emulation of stop mode, which I suppose isn't very helpful for TAS, but is a usable hardware feature so is good to have implemented. In the process of adding it I realized that stop mode is disabled on GBP, so when I get around to Gamecube detection I will have to account for that. I also fixed the open bus issues and gamepad IRQ issues mentioned in the previous posts, along with some other minor things.
With this, the only known console verification issue is with Shrek 2. I have no leads on it and no way to test anything. I think I will have to simply set it aside for now and test other games. I recently went back to double check Bionicle, and it was indeed just an input timing issue, so nothing fruitful came of it.
There is a new test ROM available for Sprite VRAM access timing that GBAHawk currently doesn't pass. It looks like maybe I just have to cut off VRAM access a bit earlier when HBlank is used for VRAM access, but I'm not yet sure. Aside from that the only known failing test is due to decrementing DMAs in the ROM region, which I might fix for next release but isn't really important.
I will be focusing on console verifications now, starting with Metroid Zero Mission and associated ROM hacks. With little else to go on, I kind of have to hope to see some desyncs on console that look approachable.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I finally figured out Incredibles enough to get it to sync. The game initially turns on gamepad IRQs which makes playback difficult, but it turns them off again after the first input. So I found that having an extra start press while the game title is loading turns off the gamepad IRQs before they cause issues with the rest of the run. The code it runs at the IRQ seems like it is not entirely benign though and messes up sync for the rest of the existing movie file from RetroEdit. GBAHawk 2.0.2 had a bug where the interrupt wasn't firing at all, so the desync seems unavoidable. I resynced what I could manually, the video is below. It seems like, barring anything else being wrong, Incredibles should be console verifiable in GBAHawk 2.0.3.
Link to video
I also decided to test Super Monkey Ball Jr as the run is short and the 3D seemed like a good test of timing. The TAS desyncs very badly, but I got a test run of level 1 going and it made it through that. The game uses EEPROM, but doesn't appear to be too sensitive to timing. Comparing the video to emulator, it looks like the EEPROM is going somewhat faster on console, so the game spends a few extra frames on the title screen, but it doesn't effect the rest of the run:
EDIT: video removed, superceded below.
So good progress. I have a couple more things I want to do before releasing 2.0.3, but now that I finally figured out Incredibles I can start making progress again.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
So I finally managed to track down the source of desyncs on console for Incredibles, and it turns out to be something related to gamepad IRQs. I don't have all the details yet, but what I know so far is that if I modify the ROM to not enable them, the existing test run syncs just fine. Gamepad IRQs are tricky because they are triggered at different times on console compared to emulator, since the input resolution is only 4 scanlines. But it also seems clear that my current implementation is somehow wrong. in the end I may need proper sub-frame input timing so that I can match input to the emulator where it would be console to get sync. I need to do several tests first, at least the problem is known now.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Dimon12321 wrote:
Thank you very much for your effort for GBA emulation and verifications! A sceptic question, sorry. Did anyone else verified a movie which has been previously verified by you? I'm just worried that you may be developing GBAHawk with only your improvised verificator and ROM re-writer in mind.
I don't think so, but all my files are available here: https://github.com/alyosha-tas/GBA_replay_files and I use original carts except for homebrew.
So, anyone can check, but that kind of thing is kind of unavoidable with things this niche.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
It's a new year, and almost 3 years already since I started this thread, wow that time went fast. That time though saw big advances in emulation and console verification on a number of systems. Here's my synopsis of the current state of R&D.
NES: The biggest deficiency left for NES is reset behavior, but few runs use that anyway. Aside from that many / most TASes should be verifiable, even ones that rely on precise power up timing and DMC glitches. Console verification of NES TASes is now pretty standard, and it is far more rare to see one fail than succeed (as long as its being done on NESHawk.) There are some edge cases that are un-emulated in NESHawk, but they are unlikely to ever come up in practice, probably only if some homebrew dev used them for an emulator check. Implementing these would require a major rewrite to NESHawk, and I'm not sure they even effect all hardware revisions. Anyway, NES is in good shape.
SNES: Modern versions of BSNES are now in BizHawk, but even so console verification has limited success. Most desyncs are random, very likely due to the audio chip clock, a well known issue. There are also some deterministic desyncs, though with unknown cause. There have been a few successful verifications along the way, and recently I saw that DwangoAC managed to verify a WIP of SMW 96 exits. Surprisingly, there is even new emulation discoveries still happening, as very recently an un-emulated cpu quirk was found. It's hard to say what is needed to advance verification efforts for SNES, it might even take some kind of hardware mod to tame the SPC clock. Deterministic desyncs make me think that something is not right on the emulation side, although it could still be a bot issue, but where to even look?
GB/C: Nothing much new has come up that I'm aware of for GB/C, its pretty mature.
GBA: Lot's of progress has happened on GBA. Tests from Flevoriux have nailed down most ppu timings and emulation details. Myself and GhostRain0 have found a lot of details related to prefetcher emulation. I've also found a few of other minor emulation details for other system components. All of this effort on the emulation side has led to many new verifications, including of first party games. There is still a ways to go, with some known desyncs and emulation deficiencies. However I'mn hopeful (and pretty confident) that 2024 will see GBA emulation and verificaiton mature to the level of NES and GB/C.
N64: A great amount of emulation effort has taken place for N64 since my original post. The N64 core of the Ares emulator is maturing quickly and is increasingly accurate, with several talented N64 devs contributing. Even cache emulation is now close to cycle accurate, it's an impressive feat. I'm not aware of any actual console testing taking place yet, and perhaps it's still early as there are still many issues, but there are exciting prospects. Maybe I'll do some testing myself later this year.
DS: While it seems like a likely next target for work after GBA and along side N64, the DS is a very complicated system and much work still needs to be done.
Genesis: Bigbass made a new verification pipeline and tools for Genesis verifications and even made the first new verification for Genesis in a long time. This is impressive work and now there is a lot of room for more genesis testing and verification.
I believe that sums up all the major developments. Console verificaiton is overall much more mature than it was 3 years ago, in some cases it is even commonplace. Also compared to 3 years ago, a lot of the low hanging fruit is now gone.
So, what now? Personally I don't know what I will do once I finish up work on GBA. N64 looks like the only approachable system, though maybe its worth it to take another look at SNES. Then there are basic questions like what do we do about disc based systems? What do we do when clock speeds get too high to emulate systems in a cycle accurate way efficiently?
That about sums up my thoughts, anyone else have any thoughts or predictions?
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
i improved open bus emulation enough to fix my current tests. After that they pass on emulator the same as on console. So this leaves me with no leads as to why Incredibles doesn't work,. After initial loading takes place there is a lot of down time where the cpu is halted so there shouldn't be any timing sensitive errors, Only vblank and timer 1 IRQs occur, with audio DMA and not much else happening. So, I don't know, test rom injection was my only real strategy for isolating errors, back to the drawing board, I'll need more examples I guess.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I've been pretty busy lately and haven't been able to focus on emulation, but now I am back on the accuracy grind. Figuring out Incredibles is the first thing I have to do. I made some test ROMs in the same style as Flintstones, and so far I am about half way through the game load process with no errors found. Then I made another test towards the end of the loading process that had a very large error compared to console. It turns out though that this test inadvertently uses accesses to un-mapped memory, so it wasn't really testing game loading, or at least it was mixing game loading and other edge cases of open bus accesses. It should still work in emulator as in console of course, so the first thing I have to do now is overhaul my open bus emulation to make sure I have all the edge cases correct. Looking over the available documentation it seems I definitely missed at least some details.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
BigBoyAdvance wrote:
Found this new clip that would save 35+ seconds compared to Qwerty6000's TAS published on this website.
All you gotta do is jump in the correct spot and hold Up to climb through the wall. Sometimes you get stuck inside the wall lol.
Link to video
I'm not gonna TAS this game, so I'm leaving this here in case someone in the future wants to improve the current TAS.
Nice find!
I was coincidentally thinking of looking into TASing this game in the future, now I'm even more motivated.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Good progress in console verification recently. Sonic Advance Knuckles, Flintstones, and two runs of Connect 4 all verified. Additionally Perfection (which comes from the same cart as connect 4) was able to be verified directly from the BizHawk movie. So that's 5/5, good stuff.
I tried resyncing an older Sonic Advance run made on vba but wasn't able to get correct RNG, so the remaining 3 Sonic Advance runs will need more effort to get working.
I'm going to do my best to get to 20 verified runs by the end of the year.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
A resync was posted by GoddessMaria a few posts above. Loading times are longer because Flash is very slow (when emulated correctly.) It takes ~460000 cycle, about 2 frames, to clear a 4k sector, and then 325 cycles to write a single byte.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
https://tasvideos.org/UserFiles/Info/638362605045693618
I managed to find a manip that gives the same number of frames as Tompa's run, 8 frames faster than my original resync (and hence 8 frames faster than the accepted run.)
EDIT: Console verification video:
Link to video
