Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Link to video
Console verified.
(Like an idiot I got myself locked out of my youtube channel since I couldn't log back into my google account since I had my verification set up to an old e-mail address that I deleted, so this video is on a new channel.)
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
cirote3 wrote:
Congrats on beating it! I never thought anyone would be able to pass it :)
However, there's still some missing scanlines when rendering the 3D pink train. Not sure if you try to be as accurate with the PPU as you are with the CPU, but well xD
Thanks! Oh I see, there are a couple sprite scanlines that aren't displayed correctly there. I have to do some other general fixes to the ppu, so I imagine that will be resolved at the same time.
Out of curiosity, did you do anything to purposefully make the test mode difficult, or did it just end up that way by virtue of all the complicated effects you used?
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Link to video
I tried Kaizo Mario 3 and it worked, so at least some of the Mario 3 hacks can work. I'll try the remaining ones eventually.
Recently 100th coin made a pretty comprehensive test ROM for the NES, and NESHawk fails several related to DMC, so it's likely Batman and the other desyncs are related to that. I haven't looked at NES emulation in quite a while at this point, and I don't think I'll come back to it, but there is a hint for the right direction at least for any future work, so that's pretty cool. Here is a link:
https://github.com/100thCoin/AccuracyCoin
I'm in the process of putting together a few more dev boards to verify a lot of the remaining hacks / homebrews. slowly grinding towards 500 verified TASes.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
I fixed all the easily fixable tests from png183. The remaining ones are pretty pathological, I made separate issues for them but honestly I probably won't bother fixing them as they are all edge case invalid encodings. I also fixed the remaining invalid thumb encodings that previously threw errors, they weren't that bad to fix.
I have also finished cleaning up and uploading all my test roms. A lot of them have poor to no documentation, but I think each one tests something specific and interesting enough to have available. I think my GBA research efforts are pretty much done now. Unless something comes up in console verification, I don't really have any motivation to endlessly test edge cases and weird interactions. Fixing Cirotemark was the capstone for me.
Now my focus will be on code cleanup and serial port emulation.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
GBAHawk 2.3.1 is now released. This is a bug fix release that fixes a bug where execution state was not being correctly cleared when ARM conditional execution check fails.
I had earlier fixed a very similar bug with a different state variable. This bug followed the same logic, so basically it has always been there. Apparently this case had previously just never come up, I guess being sufficiently rare that it was unlikely to actually happen. Still though I decided since it could potentially cause game breaking behavior, I'd release a new patched version.
I found this while looking through some of png183's new cpu tests, which test a wide range of cpu edge cases (although the bug itself is not an edge case and can happen in normal execution.) I fixed a few of these tests in the process.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
GBAHawk v2.3.0 is now released.
This release mainly focuses on accuracy improvements as well as fixing several bugs in the ppu:
- LDM^ glitch is fully implemented.
- Advance Wars dialogue now renders correctly.
- Invalid sprite shapes are now handled correctly.
- Hades tests now pass
- Cirotemark now passes
- Improves timing of sprite OAM fetches with additional hardware tests.
- Correct ppu fetches when target is out of bounds
- Fix implementation of IRQ stalling on DMA
- BIOS dumping via open bus now works
Overall this is a significant accuracy improvement compared to 2.2.0, but issues remain. This release also required significant hardware testing, which is increasingly tedious as the remaining bugs become harder to pin down. I'll be slowing down now that the big push to get Cirotemark working is done. There's a lot of research to catch up on and edge cases to clean up. I expect 2.3.1 to be an incremental accuracy improvement of relatively minor things.
EDIT: I fixed a regression pointed out by sczther and backported the fix to be included in 2.3.0.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
The fight against Cirotemark is over and I am finally at the correct cycle count:
The final issue was with prefetcher behavior at ROM 0x20000 boundaries. I previously thought what happens is that the prefetcher stops and behaves as though it is full. But what actually happens is that the prefetcher continues to try and read from the boundary address, but always fails, thus the prefetcher address never advances. This distinction is important because if the prefetcher is one cycle away from finishing a read (even a failed attempt as here) it will hold the bus from other accesses to complete. This can occasionally add 1 cycle delays to whatever is trying to read ROM. Fixing this brought total count to the correct value. This is done in the current dev build.
I uploaded several tests to the repo that test this.
Now that this is done I'll put together an official release, after I do some regression testing.
There are still a bunch of smaller issues to deal with, but most of them are comparatively simple. The biggest remaining mystery is why that Snood run crashes on hardware.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Dimon12321 wrote:
Wow, GBAHawk participates in GBA tests on gametechwiki!
https://emulation.gametechwiki.com/index.php/GBA_Tests
Looks awesome. I didn't expect to see Mesen there, I thought it was just a NES emulator.
CasualPokePlayer made some tests, wow!
Good luck, Alyosha! I keep reading your posts
Thanks! Yeah there is a lot of activity on GBA dev / emulation right now. Things are pretty well figured out now and GBAHawk, Mesen and Ares are all kind of converging. I don't think it will be too long before GBA research is more or less in a done state at least for the core system.
I finally got my debug no-prefetcher version of Cirotemark working:
The basic problem was that while I had previously found that IRQ clocking is delayed during DMA, this is not the case for internal CPU cycles, which would seem to be obvious given that they still run while DMA is running but I still somehow got it wrong. I need to do some more testing before I commit the fix to make sure I don't break anything else and have all the edge cases correct, but the basic idea is there.
Unfortunately, even with this the original test still fails by 235 cycles. So now I need to basically repeat the entire process with the prefetcher enabled. But making progress though!
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
My save state driven testing is slowly but surely moving me in the right direction on Cirotemark. On the pre-fetcher disabled version, I can now get over half way through the test with a perfect count, and overall I'm off by only 46 cycles. On the original test I'm at +295. I imagine I'm only one or two details away from getting everything right. One of the things that turned up during testing is that Halt exit timing is not effected by DMA (even though IRQ timing is.) Additionally, my recent refactors of sprite processing had a bug where processing continued past the usual endpoint at scanline 159. Fixing these two things moved be pretty far into the test. The next hurdle was that I wasn't doing BG2 accesses in mode 1 when the tile was out of bounds (even though nba documentation clearly says that this should still happen.) Fixing those things brought me up to where I am now. I just hope that when the non-prefetcher version is fully working, that also brings the prefetcher version up to fully working as well.
Performance has taken a hit from the recent accuracy improvements though. Not sure what I can realistically do about it. For comparison Mesen is approaching comparable accuracy but is ~4 times faster. This just shows how deep a hole I dig myself with my one-cycle-at-a-time methodology in terms of performance.
Currently Ares is making a lot of accuracy with emulating cpu edge case details, and it seems the multiplication algorithm (for setting the carry flag) is figured out now, though I haven't had time to look into the details. So overall GBA R&D is quite active a the moment.
Anyway, my next post should be with a working Cirotemark, hopefully it doesn't take too long.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Making substantial progress on various emulation details.
- Fixed open bus emulation so that the BIOS dumper that relies on open bus emulation and correct BIOS locking behavior now works.
- Fixed how rotation and scaling parameters are latched into the BG layers, which fixes display in Pinball Tycoon.
- Fixed invalid sprite mode 3 which should behave like sprite mode 0, which effects a few games.
The above were all pointed out by Sour in the Cirotemark issue, but none of them actually effect that test. I have been making some progress there though. I used my save state loading technique to try to narrow down where errors occur. Right now things seem to go awry in an area where HBL DMA's are happening to OAM. These occasionally collide with sprite evaluation, leading to stalls in dma execution. It turns out my conditions for disabling sprite evaluation when HBL free is set were slightly wrong. I was stopping OAM reads right at the end of allowed evaluation time, when what should happen is that all OAM reads for a sprite are completed once started.
With that I'm now at -1508 on the test (I said I was at -906 above but it turns out that iteration had some bugs.) Now that I can break into the code wherever I want it's really only a matter of time until I narrow down what the other problems are, it's just a really laborious process.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
I used the remainder of the go-fund-me money to finally get an MBC1 dev board, and used it to verify Powa.
Link to video
I'll be working through the rest of the mbc1 homebrew, which will finally get us to 450 verified runs.
Also I saw this cool video by Malleo recently about running gamecube TASes back for Mario Kart, pretty cool and impressive to see:
Link to video
Still cool stuff being done out there, even if only for special cases.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
I'm starting to make progress again here, and there are several new discoveries that have cropped up recently to direct work towards.
One of the things that needed fixing is how / when BG scroll parameters are updated in modes 0 and 1. This is most prominent in Advance Wars where the text boxes don't display correctly if you latch the values. This is a common emulator issue, also recently fixed in Ares by png183, who provided some helpful notes about the fix. So I fixed this, and made some other miscellaneous updates thanks to a new venison of Flevoriux's ram access timing test and some helpful comments from Sour (Mesen author also now working on GBA.) I still have a few other ppu related details to work out, nothing too major though.
Right now the big driver for accuracy improvements is an (NSFW) test rom called CiroteMark. This test basically times how many cycles it takes to run an intro cutscene that has various complicated ppu effects. It then gives you a count of how long it took compared to how long it takes on console. Current best is Mesen, with +133. GBAHawk 2.2.0 was -1650. As of now I am at -906. I suspect ppu details account for all of the error.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Been busy lately, but i did get the new SMW unintended exits TAS verified:
Link to video
I resynced to v115 but it didn't really change anything except for some loading lag frames.
At some point I'll get some dev boards to test things like the sound clock and maybe try some of the SMW rom hacks, but not sure when.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Following up on DBZ Advance adventure.
The run desyncs at about 30 seconds in on GBAHawk. and in the same spot and same way on console. There are no lag frame differences to account for, so there was no resyncing, it just desyncs presumably due to some accuracy issues. The run also desyncs in current versions of BizHawk, but at a different spot further along.
Link to video
So it seems a run would have a chance of syncing if made on GBAHawk, but I don't know how to fix the current one or if it is fixable.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Coincidentally I just attempted SMB3Mix and SMB3 Ridley X Hack 6 earlier today. Both desynced after several minutes. Think the former reached about 8 minutes in.
Ahhh, too bad. I was kind of hoping to just be able to knock all these SMB3 hacks out easily, kinda disappointing.
Back to the testing grind.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
I haven't had much luck with Mario 3 rom hacks so far. Super Orb Bros desyncs in NESHawk, and I can't seem to fix it. Super Sheffy Bros desyncs on console, in a consistent way at the start of the pyramid stage.
So, out of curiosity I put Batman on the dev boarrd, to see if it would desync in the same spot as the original cart. It did in fact desync in the same way. The original uses MMC3A, and my dev board uses MMC3B, so this confirms that it's not a revision dependent thing.
Currently I have no ideas what the issue is, and I can't imagine what edge case Sheffy Bros encounters that much longer SMB3 runs do not.
Maybe I'll take another look at MMC3 clocking.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
GJTASer2018 wrote:
Alyosha wrote:
The first game I tried to verify was actually Extra Mario Bros, but that game doesn't work on console because it doesn't set the mmc3 bank before jumping to 0x8000.
Has this been reported to the hack creator so that they have a chance to fix it on their end instead of you having to jury-rig something to get a console verification?
Well I can try to contact ATA on romhacking.net, but at any rate there is a convenient row of 0xFF in the last block that I can edit to easily fix the banks.
Superfast Mario 2:
Link to video
I'll try some Mario 3 hacks next.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Link to video
After many failed attempts at soldering the mmc3 chip, I finally got it working and verified my first mmc3 game, Indivisible. The first game I tried to verify was actually Extra Mario Bros, but that game doesn't work on console because it doesn't set the mmc3 bank before jumping to 0x8000. Maybe I will hack the hack to get it to work properly on console. This took me a lot of time to figure out because I was also struggling with the mmc3 soldering at the same time. I also toasted one of my mmc3 dev boards in the process, but I'll count that as climbing the learning curve, oh well. In the end though it's cool to have a mmc3 dev board working, now I can verify a bunch of mario rom hacks (assuming they don't make the same mistake Extra Mario Bros does.)
I also put together an MMC1 dev board, but the only game I found that actually uses it, SMB 35th anniversary edition, desyncs on console due to uninitialized RAM. (Mario gets hit by a cheap-cheap in 3-2.)
I think for NES now the biggest capability I am missing is mapper 30 (NESMaker) but I'm not sure when I'll get to it.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
After a lot of additional testing, I believe I finally have a complete implementation of the LDM^ glitch. Implementing this involved basically having an extra copy of the entire ARM mode code path for the cpu to execute when the glitch is activated to avoid performance penalties, but aside from that it wasn't that bad. The glitch is somewhat simpler than originally thought, as it really just amounts to OR'ing regs together in certain instances, there are no cases of both banked and user mode reg's being written to at the same time. Also the STM* version does not seem to trigger the glitch.
With that done I can start thinking about how to properly fix the mGBA suite background test.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
Link to video
I made a recording of a test rom that checks the power on ram state of the nes. My console is pretty consistent in terms of the overall pattern of 00's and FF's, but is pretty noisy between trails and many bits change between 0 and 1 randomly.
My results don't seem to match what others have mentioned ex. on nesdev, but that's not surprising.
EDIT: rom from here: https://forums.nesdev.org/viewtopic.php?t=13334
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
GJTASer2018 wrote:
Alyosha wrote:
In the next few weeks I'm hoping to have MMC1 and MMC3 dev boards up and running. A lot of recent homebrew use those.
Is there a particular reason why those dev boards seem to be in fashion for homebrewers these days?
I think it's just a combination of rom hackers / homebrewers becoming more skilled and having access to modern tools and techniques that make it easier to make full use of the chips. People come up with a lot of cool ideas and there's only so much you can do with the 32k prg and 8k chr that nrom gives you. MMC3 gives you 512 prg and 256k chr and a scanline interrupt with minimal overhead.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
I'm slowly building dev boards to continue hardware research and console verification efforts. I get my boards from mouse bite labs: https://mousebitelabs.com/ pretty cool stuff!
I now have a cnrom board, so I verified Bob-omb Mario by HappyLee:
Link to video
This was simple in terms of verification but it's the first new capability I've added to my verification tools for NES in a long time.
In the next few weeks I'm hoping to have MMC1 and MMC3 dev boards up and running. A lot of recent homebrew use those. Some interesting new test ROMs have come up recently that use boards other than nrom as well.
Still looking for something to point me in the right direction on Batman, more verifications means more chances to find something new.
Editor, Emulator Coder, Experienced Forum User, Published Author, Expert player
(4081)
Joined: 11/30/2014
Posts: 2884
Location: US
So the input I missed adjusting was around frame 8300 at the wiggling back and forth part while waiting to get dropped down the vent. Evidently this doesn't change RNG all the way up until the room where the enemy drops bombs to drop through the floor at the bottom of the stairs at frame 21710. At least for small changes.
I noticed that this enemy has a pause in between dropping bombs (drops 2, pauses, then drops another 2) where as in my original run he drops 4 bombs straight in a row. Maybe it's possible to use the waiting part at the vents to manipulate out the best bomb dropping RNG?
The final issue was with prefetcher behavior at ROM 0x20000 boundaries. I previously thought what happens is that the prefetcher stops and behaves as though it is full. But what actually happens is that the prefetcher continues to try and read from the boundary address, but always fails, thus the prefetcher address never advances. This distinction is important because if the prefetcher is one cycle away from finishing a read (even a failed attempt as here) it will hold the bus from other accesses to complete. This can occasionally add 1 cycle delays to whatever is trying to read ROM. Fixing this brought total count to the correct value. This is done in the current dev build.
I uploaded several tests to the repo that test this.
Now that this is done I'll put together an official release, after I do some regression testing.
There are still a bunch of smaller issues to deal with, but most of them are comparatively simple. The biggest remaining mystery is why that Snood run crashes on hardware.
The basic problem was that while I had previously found that IRQ clocking is delayed during DMA, this is not the case for internal CPU cycles, which would seem to be obvious given that they still run while DMA is running but I still somehow got it wrong. I need to do some more testing before I commit the fix to make sure I don't break anything else and have all the edge cases correct, but the basic idea is there.
Unfortunately, even with this the original test still fails by 235 cycles. So now I need to basically repeat the entire process with the prefetcher enabled. But making progress though!