Topic by Alyosha

Posted: 9/24/2020 10:01 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

RetroEdit wrote:

Alyosha wrote:
Here is a file for gambatte (2.4.2) that desyncs on console on the second boss in a deterministic way.
Does that file sync in 2.5? Gambatte was updated in 2.5.

Not sure, I haven't gotten the chance to look at the new GBA BIOS stuff. EDIT: I hacked back in the fake GBA bios stuff and can confirm the run syncs on the current build of gambatte.

Posted: 9/25/2020 1:58 AM

Quote

CasualPokePlayer

Emulator Coder, Judge, Experienced player (608)

Joined: 2/26/2020
Posts: 698
Location: California

CasualPokePlayer wrote:

Playing through the movie in 2.5, it appears to desync, so there's probably a good chance the cause of the console desync is fixed in 2.5?

Ok this "desync" on 2.5 is just because there's just no input after frame 44621. There's just a ton of blank frames after that. I guess aloysha uploaded the wrong file?

Posted: 9/25/2020 11:52 AM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

Yeah the file I uploaded accidentally cut off the last couple of inputs. The console desync though happens around frame 12300. I think we need a sprites + window mode 3 test. The boss part of the screen is all window and I don't think we have a lot of details about timing in that situation. The game is checking for end of mode 3 each scanline. Not entirely sure this is it but at least need to rule it out.

Posted: 9/26/2020 12:30 AM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I updated GBHawk's interrupt processing to pass all of gambatte's interrupt precedence tests (except HDMA ones.) This is also brings behavior on Gensan 2 inline with Gambatte. As always please let me know if this breaks anything. There is still a bit of room for error here as the test coverage isn't 100%. I tested against there console verified runs I have and everything seems to still work as expected. Curiously gambatte does not pass some of the hdma tests either. I haven't had a chance to look at what these tests were doing yet, but it's pretty interesting since some of them aren't even in double speed mode.

Posted: 9/26/2020 4:18 PM

Quote

RetroEdit

Any

Editor, Reviewer, Player (165)

Joined: 8/8/2019
Posts: 131

Alyosha wrote:

I updated GBHawk's interrupt processing to pass all of gambatte's interrupt precedence tests (except HDMA ones.) This is also brings behavior on Gensan 2 inline with Gambatte. As always please let me know if this breaks anything. There is still a bit of room for error here as the test coverage isn't 100%. I tested against there console verified runs I have and everything seems to still work as expected.

So now GBHawk and Gambatte behave the same for this game? What does that mean for console-verification, I guess the run needs to be modified for the new emulation to see if it will verify on console, or are there still clear emulation problems that will prevent console-verification? Also, is the list of console-verified games publicly available anywhere? I'm aware it's possible to filter submission by which ones have the console-verified flag, but I'm not sure if you have something different.

Posted: 9/26/2020 6:00 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

RetroEdit wrote:

So now GBHawk and Gambatte behave the same for this game? What does that mean for console-verification, I guess the run needs to be modified for the new emulation to see if it will verify on console, or are there still clear emulation problems that will prevent console-verification? Also, is the list of console-verified games publicly available anywhere? I'm aware it's possible to filter submission by which ones have the console-verified flag, but I'm not sure if you have something different.

They behave the same in this particular case. I have another test run that works correctly in GBHawk (compared to console) but not in Gambatte do to incorrect mode 3 emulation of sprites + xscroll in Gambatte. This particular run avoids that though, and both Gambatte and GBHawk agree here cycle for cycle, but the run desyncs on console. So, there is something unknown missing in both GBHawk and Gambatte. My speculation is mode 3 sprites + window timing, but have no way to test yet. It could also be an edge case of IRQ source selection. All the complete runs in GBHawk that I have either personally verified or match cycle - for - cycle to runs already verified in Gambatte are linked in the opening post. I have a few other console verified test runs but are incomplete. The list of Gambatte verified runs is longer, and can be found here: https://runs.tas.bot/runs.html I think it's still missing some recent ones like mega man though. Eventually I will catch up but I've been busy plus stuck with Gensan 2. I have personally verified the recent Men in Black TAS but I don't have a capture device so it's not officially verified.

Posted: 9/28/2020 2:40 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

Still working on Gensan 2. I looked a little closer at the console replay, and it's actually desyncing at the end of level after the first boss, not on the second boss as I previously thought. This eliminates the window as the source of the desync, but that level has a lot of lag and sprites and it's hard to narrow down exactly where things are going wrong. I'll post another movie file when I have a minimum case of where console and emulator diverge. If I had to speculate it would be some edge case of mode 3 timing. The test roms only test xscroll up to 4 mod 8, so maybe we need 5,6 and 7 as well. Beyond that I'm not really sure what it could be. Changing anything too drastic causes immediately obvious desyncs, and I already checked for usage of uninitialized RAM and blocked VRAM accesses. I compared with video from my phone as best I could for a frame by frame analysis, and everything looks correct until the very end of the level, where a jump and attack happens slightly differently on console, allowing the level to end one frame earlier, which is what causes the later desync in the second boss. If I don't make any more progress I might need to set it aside and look for another game that desyncs but is a little more obvious what's going wrong.

Posted: 9/29/2020 4:23 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I tried working on some more of Gambatte's test ROMs to see if I was missing anything obvious that would effect Gensan 2. But, I ran into some trouble on some of the PPU tests. The issue is that this test: lycint152_ly153_3_dmg08_cgb04c_out00.gbc is incompatible with the results from AntonioND, gbc-hw-tests: mode1_disableint_gbc.gbc It appears that on later model GBCs (and presumably GBA and GBP) that LY timings are slightly different compared to older models and original GB. This effects other tests as well but this is the simplest case to see. I knew this was a problem, it causes at least one desync (Super R.C. Pro Am) on Gambatte, but I didn't connect the dots that Gambatte's test suite is also tuned to this expectation until now. So looks like I'm going to have to be more careful about how I use Gambatte's tests going forward. I had implicitly tuned GBHawk to the newer models since there were consistent results between AntonioND's tests and WilbertPol's tests. I have to make sure I don't bake in some contradictory behaviour by relying too heavily on Gambatte's test suite at the same time. I don't know what else besides this is effected. It can't be anything too major or there is no chance Men in Black or Pokemon TCG would have worked on console. It's worth noting though as it already cost me some time trying to sort out an inconsistency that is fundamentally caused by different models of console.

Posted: 9/30/2020 6:11 PM
(Edited: 10/5/2020 8:44 PM)

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

~~http://tasvideos.org/userfiles/info/66431762538224618~~ Here is a movie that demonstrates the desync in Gensan 2 as clearly as possible. On emulator, the character stops just short of beating the level before input ends. On console, the player beats the level, indicating that there is one less lag frame somewhere. I haven't narrowed down exactly where the desync happens. I played around with the inputs at the end, and there are some points where lag frames definitely line up differently compared to console, but it's hard to tell where this starts. I made a post about it in the console de-verified thread: http://tasvideos.org/forum/viewtopic.php?p=500074#500074 I'll probably move on for now since I've run out of ideas. Probably needs some new test ROMs around mode 3 but who knows for sure.

Posted: 10/1/2020 5:55 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I did some testing with Mortal Kombat today. This game is interesting because it uses the serial port extensively even when there is no connection. it also uses the window as the primary part of the display. The test run I made desyncs almost immediately in GBHawk. Gambatte is slightly better (despite some sprite compatibility issues on emulator), making it through the first fight before desyncing on the second. Still so much work to do even for single speed mode.

Posted: 10/4/2020 11:08 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I believe I finally figured out Gensan 2. There is a penalty when evaluating sprites at position 0 with non-zero x-scroll. I have previously added this penalty for all values of x-scroll, as this was consistent with existing tests of x-scroll up to values of 4 mod 8. However, the tile fetcher is only active for 6 cycles, so i speculated that maybe this penalty should not be added for x-scroll values 6 and 7. If I implement this, then I do indeed get the same de-sync as Gensan 2 as on console, and without any de-syncs in other console verified runs. When I adjust the run of gensan 2 to include this case, it syncs to the end of the game on console as well. I think this is highly likely to be correct, there isn't much else it could be to de-sync in this exact way, but to make sure I manually edited the existing sprite + scroll tests to use cases 6 and 7 and will get some dev carts to test on console to verify. Hopefully this will bring this case to a close and i can move on.

Posted: 10/5/2020 2:17 AM

Quote

CasualPokePlayer

Emulator Coder, Judge, Experienced player (608)

Joined: 2/26/2020
Posts: 698
Location: California

Alyosha wrote:

I manually edited the existing sprite + scroll tests to use cases 6 and 7 and will get some dev carts to test on console to verify.

By any chance could you post your modified tests? I happen to own a flash cart and would be willing to run the tests on my GBP.

Posted: 10/5/2020 2:26 AM
(Edited: 10/5/2020 5:36 PM)

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

scx = 5: https://www.dropbox.com/s/w4twsy5g6m0qphv/intr_2_mode0_timing_sprites_scx5_nops.gb?dl=0 scx = 6: ~~https://www.dropbox.com/s/4vtnpa23wwegfk6/intr_2_mode0_timing_sprites_scx6_nops.gb?dl=0~~ scx = 7: ~~https://www.dropbox.com/s/v1xvvym9kztazpg/intr_2_mode0_timing_sprites_scx7_nops.gb?dl=0~~ I think scx = 5 should pass on console. If what I'm thinking is correct, 6 and 7 should fail on console. All 3 pass on GBHawk and Sameboy currently. Gambatte does not pass any of the scroll tests, not sure why. Assuming 6 and 7 fail, I will make proper versions that pass. If they also pass on console in their current form, well then I'm pretty stumped.

Posted: 10/5/2020 5:09 AM

Quote

CasualPokePlayer

Emulator Coder, Judge, Experienced player (608)

Joined: 2/26/2020
Posts: 698
Location: California

scx = 5

scx = 6

scx = 7

Posted: 10/5/2020 1:47 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

Awesome, thanks for testing. Looks like I was almost right. Based on those results, it looks like the penalty doesn't go away completely, it just gets capped at 5. I just checked and this is indeed still enough to fix Gensan 2, so looks like this is definitely the solution. I'll make new versions of tests 6 and 7 later today that should actually work on console. In hindsight this should have been obvious, the normal penalty for non zero sprites goes 5,4,3,2,1,0,0,0. So, it makes sense that this one should go 0,1,2,3,4,5,5,5 and not 0,1,2,3,4,5,6,7 that I currently have implemented. That's what I get for extrapolating beyond the available data without thinking it through. At least the fix is super easy. Hopefully this can get fixed in Gambatte too, it's one of the only really obvious flaws it has. This is not some rare edge case so it probably impacts other games too. EDIT: Here are revised tests for scx6 and scx7 that i believe should work on console. https://www.dropbox.com/s/l9hfqvfn39vqmyc/intr_2_mode0_timing_sprites_scx6_nops.gb?dl=0 https://www.dropbox.com/s/4yvtv675szu9vpu/intr_2_mode0_timing_sprites_scx7_nops.gb?dl=0

Posted: 10/5/2020 7:46 PM

Quote

CasualPokePlayer

Emulator Coder, Judge, Experienced player (608)

Joined: 2/26/2020
Posts: 698
Location: California

Yep, both of the new tests pass on console.

Posted: 10/5/2020 8:37 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

Cool, thanks for your help. Fix committed! I'm glad despite this being a pretty obvious oversight that it led to something useful. Overall Gensan 2 helped improve GBHawk pretty substantially, and even helped gameboy emulation in general a little bit I guess. I'll defnitely be submitting a run of it next release. Now I can move on confidently to Mortal Kombat. My preliminary guess on that game is that there is a sprite + window timing issue somewhere. Hopefully it takes me less time to find then this one. I've also been cleaning up window emulation in general a bit. There are a lot more behaviours you can do with the window then I expected, and GBHawk isn't really built to handle all of them, but some of the more important issues for games at least are getting resolved.

Posted: 10/5/2020 8:41 PM

Quote

RetroEdit

Any

Editor, Reviewer, Player (165)

Joined: 8/8/2019
Posts: 131

Alyosha wrote:

Hopefully this can get fixed in Gambatte too, it's one of the only really obvious flaws it has. This is not some rare edge case so it probably impacts other games too.

This doesn't appear to be filed on Gambatte-Speedrun's issue tracker. I'm sure they would appreciate it if you filed the issue.

Posted: 10/6/2020 4:03 AM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

RetroEdit wrote:

This doesn't appear to be filed on Gambatte-Speedrun's issue tracker. I'm sure they would appreciate it if you filed the issue.

I think CasualPokePlayer could do that if he wanted one. I looked over what Mortal Kombat was doing and noticed a pretty serious error in how I was dealing with sprites + window. Turns out I was not resetting the alignment for sprite evaluation when the window was activated. Obviously this would yield incorrect results for mode 3 timing. When I fixed this oversight, Mortal Kombat synced fine. This error has been present since almost the inception of GBHawk. Sprites interact with the window rarely enough that this simply didn't come up often enough to notice. I only did the basic case though, that is when sprites are comfortably in the middle of the window. Edge cases where sprites span across the edge where the window is activated or around x=0 are not accounted for at all, and certainly need their own comprehensive test ROMs to do correctly. What I have currently is enough for Mortal Kombat though. I'll do a few more tests with Mortal Kombat, then I'll try to find another game that does some non-standard stuff and see what comes up. If things work out it might be time to move to double speed mode finally. EDIT: Nope, Mortal Kombat is still broken, the run I made just got lucky, looks like it's time to make some window+ sprites tests.

Posted: 10/12/2020 5:32 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I recently got a development cart to run hardware tests with. The first thing I did was run lycint152_ly153_3_dmg08_cgb04c_out00.gbc and I can confirm it gives 99 on a GBP instead of zero. I'll be working on window + sprite timing tests now. I'll post them when I have ones that work on GBP. Hopefully this will solve Mortal Kombat and be the last major piece of the single speed puzzle. EDIT: Here is the first test which already revealed an error in GBHawk. It also doesn't work in Gambatte but fails in a weird way, I don't really get it. It worked on GBP. This is the simplest case of a window mid screen and sprites in that region are sufficiently spread out to avoid overlap. It failed in GBHawk because sprites were being evaluated according to the pre-window offsets when window and sprite started on the same pixel. Fixing this actually made the rendering code more symmetric in terms of when different things were getting evaluated, so it is an accuracy fix and a good clean up too. https://www.dropbox.com/s/f8sdl76ew8u37qr/intr_2_mode0_timing_sprites_win26_nops.gb?dl=0 There are a lot more tests to go. I'll do a few more of the basic case, then a bunch with overlapping sprites, then one test for each window value 0-7 where I expect weird stuff might happen.

Posted: 10/14/2020 10:07 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

Made a lot of improvements to window + sprite timing and made a bunch of new tests. Everything actually worked out pretty cleanly. The cases around 0 weren't really that bad, although I have a few more left for complete coverage. I'll be putting all the tests I put together here: https://github.com/alyosha-tas/Misc.-GB-Tests This does not seem to have fixed Mortal Kombat though. I think I'll have to look closer at the serial port.

Posted: 10/23/2020 11:40 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

So I think the problem with Mortal Kombat is just input resolution of GBI. On the ladder screen, the game is just spamming checks to the control register. If I change the offset built into the input script by 1 incrament, with equates to 1024 cycles at 4MHz, then Mortal Kombat syncs just fine. This also explains why TAS input syncs just fine, since I was pressing start on the ladder on the first available frame, the input was already there before the spamming started. For now I'm not going to change the script, since I'm not entirely sure what the correct fix is (it could be only off ~200 cycles and still break the sync), but it's something I'll keep in mind.

Posted: 10/25/2020 12:22 AM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I tried improving my Gensan 2 run before submitting it but when I did I ran into another desync on console. This game sure is working out the bugs in GBHawk. This one though only took a few minutes to track down (at this point there really aren't many things left that can be changed that it could be.) ~~The problem was in the timing of clearing interrupt flags. There's still a little room for variation in the setup I have now, but this case reduces the uncertainty down to 1 cycle.~~ Actually the above was wrong, the real solution is that when clearing an IRQ flag at the same time it is set in GBA mode sets it instead of clears it. This test: tc00_irq_late_retrigger_2_dmg08_outE4_cgb04c_outE0.gbc gives E4 on GBA. Before I was using the GBC04 behaviour of getting E0, which is evidently wrong at least for GBA (and likely GBC E.) I'll still need to look over other tests to see if this breaks anything else before committing anything. Looks like Gensan 2 will wait for yet another BizHawk release.

Posted: 10/25/2020 8:53 PM

Quote

Alyosha

He/Him

Editor, Expert player (3532)

Joined: 11/30/2014
Posts: 2729
Location: US

I'm pretty confident in single speed mode at this point. There are some edge cases to clean up still but it's time to start looking ahead to double speed mode. In preparation for that I re-synced the run of Spirou Robot Invasion to the latest dev build. To my surprise it worked on console: http://tasvideos.org/userfiles/info/66989349617966603 The run contains some adjustments, there have been a great number of emulation changes since it was first made, but it's still almost entirely faithful to the original. This gives me another run to check against along side pokemon TCG when I start working seriously on double speed mode pretty soon.

Posted: 10/27/2020 7:27 PM

Quote

jlun2

Skilled player (1706)

Joined: 9/17/2009
Posts: 4952
Location: ̶C̶a̶n̶a̶d̶a̶ "Kanatah"

Sorry to bother, but on Wario Land 2 GB, there's a new desync on console. https://www.twitch.tv/videos/777978631 It starts at 21 minutes in on stage 1-2 (ID 1). This is how the stage looks like if it syncs, with lua overlay https://www.youtube.com/watch?v=YMmODgtmDso This is the input file https://cdn.discordapp.com/attachments/280806848909541376/765267063206969424/Wario_Land_II_GB_100.bk2 I have no idea if your changes fixes it, but just letting you know.

LLCoolDave wrote:

This seems to be competing for the "Worst looking TAS of 2011" Award.