Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Occasionally I look around for any homebrew or tech demos that I missed testing, and I recently came across this Yoshi's Island tech demo that I hadn't seen before and was broken in GBAHawk: It turns out I was not doing rotation and scaling correctly in this case, so I went back and redid that code. With a better understanding how it was supposed to work, the rewrite was more accurate and also quite a bit faster since I got rid of the floating point arithmetic. It gives a couple fps boost to games that use mode 7 stuff. Now the demo works correctly: I'll probably go back and do the same clean up for rotation and scaling of sprites, might get a small performance boost from that too. Now that I finally have a Metroid Zero Mission cart I will be testing that game and associated ROM hacks.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Testing with Metroid Zero Mission is going well. I already verified a couple of the ROM hack runs and the 100% run. Two of the hacks require a 32MB card which I don't have, so I'll to get one before I can do them. On the baseline run I actually got a desync. Haven't figured out the cause yet, but I can see the desync point pretty clearly and the error would have to be fairly large (~50 cycles) to get a desync in that spot. I would think such a large error at this point can only be caused by something with audio DMA, but nothing obvious sticks out at me so far. Zero Mission also revealed an emulation error in the ppu blending effect. This was an an easy fix but pretty surprising this case didn't come up before. Banjo Kazooie continues to reveal bugs as well. It turns out I missed a case of dma pausing another dma resetting access timing to be non-sequential. This happens to occur in Banjo Kazooie and was causing failures in my testing. After fixing that I can finally get past loading and the intro with correct timing. I tested up to about 4 minutes into the run and it seems RNG is still good. So maybe that game is good now, but I can't get correct enemy movement so far trying to manipulate RNG with jumping so not sure yet.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
GBAHawk version 2.1.0 is released! This version comes with numerous accuracy improvements along with a small but noticeable performance improvement. With this release I was able to add console verified check marks to Shrek 2 and the baseline run of Metroid Zero Mission. I should be able to do the low % zero mission TAS this week along with the pending 6 scrolls ROM hack. I tried resyncing Fire Emblem with RetroEdit's script but it desynced at the same point as when I tried manually, so I guess RNG for that game will prevent verification of the current run as is. I also tried Turok Evolution but desynced due to RNG as well. RNG is also preventing me from making progress on Banjo Kazooie. I'm going to focus on fixing some of those RNG issues as well as making a few test TASes of different games that I haven't previously used for testing. Still a lot of work and verifications to do, with Mega Man Zero and Battle Network Series, various Castlevanias, Zelda Minish Cap and Mario Super Star Saga. I'm hoping the most tricky and time consuming emulation issues are behind me so I can focus more on the TASes.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
I haven't had any luck with RNG and resyncing runs, so I'm working on a few more small accuracy issues. One of these is a known error in sprite rendering where too many pixels are rendered. I made a test for this and also a similar thing that happens when hblank interval is free: Fixing it was pretty simple, but it's possible I'll need to redo sprite rendering in the future if there are other side effects. The next thing I'll do is fix the weird behavior involving DMA in ROM area.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
GBAHawk v2.1.1 is released! This version adds a sync setting for EEPROM timings, along with various bug fixes. This allows me to add a check mark to DKC 101%. I also fixed a bug in reading from BIOS while not executing in BIOS region, which is supposed to return the last fetched instruction from BIOS. I was not properly reading the correct byte when only byte sized data was asked for. I found this out when trying to figure out a desync in Fire Emblem. It turns out this game has a bug in calculating enemy movement when enemies are on the edge of the map and attacking characters also close to the edge of the map. In this case the game ends up reading from an incorrect address (in this case 0x00000007) which is in BIOS region. After fixing this I was able to get a resync of Fire Emblem to work on console. All the actual moves are the same with only small RNG adjustments needed, so I'll go ahead and give that one a check mark too. I'm half way through the same process for Sacred Stones as well, so that one will be next up for verification. It's pretty satisfying to get the Fire Emblem runs working, as when I first started testing it I couldn't even get past the first start press. Good Progress!
Dimon12321
He/Him
Editor, Reviewer, Experienced player (596)
Joined: 4/5/2014
Posts: 1223
Location: Romania
I wish I could have such a strong motivation to deal with my stuff! Great work once again! I went through your repository and the console-verified GBA publications and, I guess it makes sense to clarify the things. I assume a gbmv input file recorded in an elder version of GBAHawk (say, 2.06) will playback in the latest version, so there's is no point in marking them in the repository. However, you mentioned RNG manipulations for some of the games to become verified, like you had to add some blank frames here and there to make the movie successfully play on a real GBA. Does it make sense to store these input files as well, for example, in "gbmv files verified" folder? Even if this final step of input adjustment has to be done for each verifying device independently, this at least will give verifiers the right whereabouts where to add or remove blank frames.
TASing is like making a film: only the best takes are shown in the final movie.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Dimon12321 wrote:
I wish I could have such a strong motivation to deal with my stuff! Great work once again! I went through your repository and the console-verified GBA publications and, I guess it makes sense to clarify the things. I assume a gbmv input file recorded in an elder version of GBAHawk (say, 2.06) will playback in the latest version, so there's is no point in marking them in the repository. However, you mentioned RNG manipulations for some of the games to become verified, like you had to add some blank frames here and there to make the movie successfully play on a real GBA. Does it make sense to store these input files as well, for example, in "gbmv files verified" folder? Even if this final step of input adjustment has to be done for each verifying device independently, this at least will give verifiers the right whereabouts where to add or remove blank frames.
The verified movie files are played back as is, there is nothing added to adjust for console after the fact. By RNG adjustment I mean compared to the original move file on either VBA / BizHawk. The only exception to this is that occasionally I need to adjust the time stamp on the input file that GBI uses, but this is a limitation of the resolution of GBI not an accuracy issue. I do check some previously console verified movies with each new version release to make sure I didn't accidently break anything, so yeah movie files should all be forward compatible. Your post reminded me that there is one exception to this though. Feline currently desyncs in current versions. At first this didn't make much sense to me since it seemed to work on console just fine. But it turns out I dumped the movie with an old version of the GBI input script that didn't a include start up offset. By some weird coincidence of error cancellation this let it work on console. When I dump it with the current script it desyncs the same as on emulator. So at some point I need to find some frames to save to get a new movie and remedy this.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
I have updated the release of GBAHawk 2.1.1 to fix a save state error when using EEPROM or FLash mappers. This error has apparently been present all along, and happens when loading a state where saving to cart is happening. So, I strongly recommend updating to the the current release of 2.1.1 to avoid sync errors.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
RetroEdit pointed out a serious bug in v2.1.1 which prevented using the default core without a pre-existing config file. I fixed this issue and with a couple other minor fixes released 2.1.2. I also backported the fixes to 2.1.1, to avoid having to simply delete it altogether.
Dimon12321
He/Him
Editor, Reviewer, Experienced player (596)
Joined: 4/5/2014
Posts: 1223
Location: Romania
In scope of TAS movies provision to test emulation accuracy, I'd like to provide two movies (following previously posted Duke Nukem Advance movie): - Serious Sam Advance: User movie #638512881522132559 - Doom: User movie #638513118358303271 Speaking of Doom. For some reason, GBAHawk doesn't make an in-game save. I don't know if I should create an issue on GitHub or maybe I missed some warning on this regard, but since battery-backed saves were mentioned in previous verifications, I think that means the emulator must be capable of doing saves. For comparison, here is my BizHawk 2.6.1 TAS, where game is being saved just fine: User movie #74380673068812668
TASing is like making a film: only the best takes are shown in the final movie.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Dimon12321 wrote:
In scope of TAS movies provision to test emulation accuracy, I'd like to provide two movies (following previously posted Duke Nukem Advance movie): - Serious Sam Advance: User movie #638512881522132559 - Doom: User movie #638513118358303271 Speaking of Doom. For some reason, GBAHawk doesn't make an in-game save. I don't know if I should create an issue on GitHub or maybe I missed some warning on this regard, but since battery-backed saves were mentioned in previous verifications, I think that means the emulator must be capable of doing saves. For comparison, here is my BizHawk 2.6.1 TAS, where game is being saved just fine: User movie #74380673068812668
Thanks for the movies! I'll look into them as soon as I get the carts. Doom probably just needs a gamedb entry. One way or another I'll have it fixed for v2.1.3.
EZGames69
He/They
Publisher, Reviewer, Expert player (4462)
Joined: 5/29/2017
Posts: 2762
So I noticed there's some audio mixing issues that I first noticed for games like https://tasvideos.org/9040S in GBAHawk. In the first level's track, there's a guitar like instrument that comes in (might be a different instrument idk what it would be) that is mixed pretty quietly in comparison to the console footage in that submission. I make a video highlighting the differences using mGBA as a comparison: Link to video In the TAS itself, this part of the track kicks in at around 1:06.
[14:15] <feos> WinDOES what DOSn't 12:33:44 PM <Mothrayas> "I got an oof with my game!" Mothrayas Today at 12:22: <Colin> thank you for supporting noble causes such as my feet MemoryTAS Today at 11:55 AM: you wouldn't know beauty if it slapped you in the face with a giant fish [Today at 4:51 PM] Mothrayas: although if you like your own tweets that's the online equivalent of sniffing your own farts and probably tells a lot about you as a person MemoryTAS Today at 7:01 PM: But I exert big staff energy honestly lol Samsara Today at 1:20 PM: wouldn't ACE in a real life TAS just stand for Actually Cease Existing
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
EZGames69 wrote:
So I noticed there's some audio mixing issues that I first noticed for games like https://tasvideos.org/9040S in GBAHawk. In the first level's track, there's a guitar like instrument that comes in (might be a different instrument idk what it would be) that is mixed pretty quietly in comparison to the console footage in that submission. I make a video highlighting the differences using mGBA as a comparison: Link to video In the TAS itself, this part of the track kicks in at around 1:06.
Wow, not sure how you were able to notice that but I agree it does sound a little too quite. I'll open a github issue for audio balance for 2.1.3.
EZGames69
He/They
Publisher, Reviewer, Expert player (4462)
Joined: 5/29/2017
Posts: 2762
Would it be possible for that movie to sync on 2.1.3? I’d like to have it be published with the better sounding audio whenever that will be available.
Alyosha wrote:
Wow, not sure how you were able to notice that
I tend to compare audio differences between emulators on occasion whenever I’m publishing. It’s easier to do when you’re going through footage in virtual dub while looking at previous submissions. It’s made me pretty hyper aware of anything that sounds “off”.
[14:15] <feos> WinDOES what DOSn't 12:33:44 PM <Mothrayas> "I got an oof with my game!" Mothrayas Today at 12:22: <Colin> thank you for supporting noble causes such as my feet MemoryTAS Today at 11:55 AM: you wouldn't know beauty if it slapped you in the face with a giant fish [Today at 4:51 PM] Mothrayas: although if you like your own tweets that's the online equivalent of sniffing your own farts and probably tells a lot about you as a person MemoryTAS Today at 7:01 PM: But I exert big staff energy honestly lol Samsara Today at 1:20 PM: wouldn't ACE in a real life TAS just stand for Actually Cease Existing
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
EZGames69 wrote:
Would it be possible for that movie to sync on 2.1.3? I’d like to have it be published with the better sounding audio whenever that will be available.
Yes, all console verified movies should be forward compatible. I have no eta for a next release, but yeah it can wait.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
GBAHawk v2.1.3 is now released. This is a bug fix release which reworks EEPROM size detection (fixing probably ~1000 incorrectly entries based on MAME DB.) I probably should have done this from the start but didn't realize how bad the state of affairs was. This also re-balances audio between FIFO channels and GB style channels. Next release will have manual Flash timings, which I wanted for this release but don't currently have the time to sort out properly, as there are a lot of variables. I haven't encountered any core accuracy problems in a while which is promising. I know some of some very edgy edge cases that could cause problems, but they haven't popped up yet, so for now I am content to keep grinding out console verifications. EDIT: There was am bug in audio mixing, I fixed it and updated the 2.1.3 executable / downloads.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Link to video I was working on the 'time attack' run of Snood when I came across this weird crash on console. It happens deterministically, though the stuff after the crash isn't quite deterministic. Doesnt happen in emulator currently. Nothing super interesting appears to be happening here, its the same as any spot in the game. I even put the ROM on a dev cart to make sure my real cart wasn't the issue and same result. Something very weird must be happening, I'll dig into it. This another excellent example that you truly never know where interesting edge cases for emulation are going to pop up. Especially since the first Snood run had no issues.
YoshiRulz
Any
Editor, Emulator Coder
Joined: 8/30/2020
Posts: 106
Location: Sydney, Australia
I'm not a fan of how you've forked the BizHawk repo—or to be more precise, how you did not. It makes it much harder to track differences and share fixes, and it turned what could have been a simple task of building GBAHawk with Nix into an hours-long slog. (Final result here.) And was deleting every OSTailoredCode.IsUnixHost check really necessary?
I contribute to BizHawk as Linux/cross-platform lead, testing and automation lead, and UI designer. This year, I'm experimenting with streaming BizHawk development on Twitch. nope Links to find me elsewhere and to some of my side projects are on my personal site. I will respond on Discord faster than to PMs on this site.
Hey look buddy, I'm an engineer. That means I solve problems. Not problems like "What is software," because that would fall within the purview of your conundrums of philosophy. I solve practical problems. For instance, how am I gonna stop some high-wattage thread-ripping monster of a CPU dead in its tracks? The answer: use code. And if that don't work? Use more code.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
YoshiRulz wrote:
I'm not a fan of how you've forked the BizHawk repo—or to be more precise, how you did not. It makes it much harder to track differences and share fixes, and it turned what could have been a simple task of building GBAHawk with Nix into an hours-long slog. (Final result here.) And was deleting every OSTailoredCode.IsUnixHost check really necessary?
Certainly not necessary, I just deleted as much as I possibly could while maintaining the functionality I wanted. Actually I originally planned on deleting much more to reduce and simplify the front end code, but I never got around to doing the necessary refactoring as I wanted to focus on core emulation work. Anyway thanks for the build!
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
I started working on some new tests for a strange hardware glitch involving accessing banked registers after loading user mode registers through LDM^. https://github.com/alyosha-tas/gba-tests/blob/master/LDM/LDM_ALU.gba The basic effect, described here: https://github.com/mgba-emu/mgba/issues/2942 , is that reading OR's user and banked registers together and writes write to both sets of registers. Unsurprisingly, this is extremely annoying to implement. It basically requires an entire separate execution path when the processor is in this state. It's still a work in progress, there are a lot of cases to work through. There is also a test for this in the most recent AGBEEG tests. But these tests were tested on 3DS hardware which evidently behaves differently as they don't work on GBP. So far I don't see any games even use LDM^, so this might have low utility, but its a good thing to be aware of (I did check if Snood perhaps encounters this but it doesn't.) EDIT: Additionally, when I compiled the recent AGBEEG tests the cartridge tests fail on hardware. It looks like I get a different binary than the release version when i compile from source at that commit, so my tool chain must be a little different. Anyway the test passes on emulator but fails on console. I'm pretty sure this is the result of changing wait states, as when I change the order they are checked in, the fails happen in a different way. This is unexpected, and itself another major headache to implement if my guess is correct. But, one complicated edge case at a time.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
I found an edge case of prefetcher behavior not accurately emulated in GBAHawk so far. It concerns what happens when an 0x20000 boundary is reached. Previously I discovered that the prefetcher becomes disabled at these boundaries. and that there is sometimes an extra cycle penalty as a result. The conditions for the 'sometimes' are not currently correct in GBAHawk as new testing shows. It seems I need to be more careful in defining states that the prefetcher can be in to get this correct. I may take this opportunity to reorganize the prefetcher code better to sort all this out. This is the first time in a while I found a new inaccuracy in execution timing. I don't think it will effect DKC 2 as I already tried the run with the extra cycle penalty both on and off and nothing changed, but maybe in reviewing my code again something new will become apparent.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
I found a pretty fundamental bug in IRQ emulation while working on why Super Metroid desyncs. The bug concerns what happens with IRQs when a DMA is occurring. Currently I have IRQs go through the pipeline normally during DMA, but current testing indicates that the IRQ pipeline is stalled during DMA. This is at least responsible for the desync in Super Metroid, and probably DKC 2 as well. I haven't worked out the details exactly yet, but hopefully fixing this will be pretty straight forward. Pretty surprising this bug went undetected for so long. Finding this was difficult as the desync occurred 30 minutes into a run. I had to devise a way to load the game state on the desired frame and then inject this into the game code and branch back and forth to run timing tests. This is the technique I will probably have to use to diagnose Snood.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
After a great deal of testing I finally figured out what was wrong with Donkey Kong Country 2. The problem is here:
0801431A:      4778  Bx (R15)
0801431E:  00000000  (fail)  AND (no flags) R00, R00, (R00 << 00)
That second instruction is an ARM instruction, but it is not word aligned. This is fine in general, but evidently it is not fine for the prefetcher. It seems in this situation the prefetcher continues to fetch, but the comparison to check if it has the correct value always fails. Implementing this, DKC 2 now desyncs in the same way as on console. One I have tested sufficiently thoroughly, I will resync the existing run and test on console again. This is another one of those edge cases that I just wouldn't have thought to check on my own. Luckily my testing methodology points me in the right direction pretty reliably.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
Now that DKC2 is finally figured out, I can turn my attention to other things not accuracy related. I still want to include manual Flash timings for next release, and one of the games I want to test for that is SMB Advance: SMB 3. Coincidentally though, this game has GBP detection. So, I'm going to try to get that working at the same time. I implemented rumble support already about a month ago along with the screen detection algorithm, but the GBP rumble commands seem to be more complicated than indicated on GBA TEK. More work needed there. I also want to have a pretty comprehensive implementation of the LDM^ glitch, but I'm willing to let that one wait for a future version since it's not critical and I can turn off emulation of it by commenting out a single line. Overall GBA is starting to feel like its in pretty good shape, time to start thinking about what's next.
Alyosha
He/Him
Editor, Emulator Coder, Expert player (3822)
Joined: 11/30/2014
Posts: 2832
Location: US
GBAHawk 2.2.0 is released, with implementation of Flash chip type selection and manual timings. This also includes the accuracy improvements for DKC 2 and Super Metroid GBA. Next thing to work on is GBP detection / emulation, which is disabled in this release, but all the wiring is done, I just need to understand the commands. Aside from that the only major upgrade I want to make is finishing implementing the LDM^ glitch. Just bug fixing after that.