Link to video
I put together a mapper30 board (this one) and started verifying some of the related homebrews. So far 3/3 worked. I'll be punching out the remaining ones this weekend. This should put us around 470 verified TASes. I'm still trying to push for 500 by year end.
Mapper30 testing was pretty successful, only 1 game I tried didn't work (nightmare on elm street) due to RNG desyncs. I don't really have the time right now to dig into the code and figure out why, so I just set it aside and moved on. After catching up on a few recent publications, Mighty Bomb Jack and SMB max score, we are now at 471 verified TASes. Right now the biggest source of quick verifications are all the Mega Man ROM hacks, but I only have Mega Man 6 (U) currently which doesn't help.
Other notes:
Vs. Air Man - graphical glitches on my dev board, is CNROM so maybe missing bus conflicts but haven't looked into it.
Astral World - needs FME7, which is available in Return of the Joker but needs a custom dev board
Rohrleitung Gate - needs VRC7 (and a Famicom)
EDIT:
Wordle - relies on uninitialized RAM, doesn't work on my console
I think that's all for now. I'm hopeful steady progress can continue at least up to 480 verifications.
I was able to console verify Famidash:
Link to video
This one is interesting because this game uses MMC3 with 4 screen mirroring. There isn't an existing dev board for this, so I decided to design it myself (with a little help from lidnariq at nesdev) and after a few iterations it works!
I used easy EDA to design the pcb, which is pretty easy to use and has a library of parts including for the NES (even though one of them had an error which I didn't catch and caused glitches in my first board.) It's aslo pretty cheap to get the pcb's made, I was able to get them for less than the cost of a typical NES or GBA game, we really live in the future!
I'll hopefully verify the other Famidash runs this week, though I won't leave my NES on for 10 days for the all levels one.
Now that this experiment is a success, I plan to make a few other boards as well. The first one is SXROM, which is needed for Nova the Squirrel.
I verified most of the Famidash runs, and I got in my SXROM board and was able to verify the 2 runs of Nova the Squirrel:
Link to video
My board doesn't have a battery so I had to do the World 6 run from a rest to keep the SRAM set, but good enough.
I have an MMC2 board as well so once I have time to put it together I can try the Punch Out ROM hacks.
486 runs verified, 14 to go to 500.
Over the past coupe of months I've been slowly working on a new C++ version of NESHawk, and I'm finally starting to see some results. The above screenshot is from the Castlevania III TAS I console verified a while ago working in the new core. The movie file that goes with it doesn't actually sync in any existing BizHawk version, and since I deleted my old research builds a while ago, it was actually orphaned with nothing that could run it. I had tried a C++ port some years ago, but it was limited and wasn't really in a workable state. This new version incorporates what I learned from GBAHawk development, so is much better organized. All the basics work like controller selection, audio, and save states, as well as other tools like hex editor and PPU viewer.
A lot of things are also removed to simplify the process, like PAL, FDS, famicom and unisystem specific code, and cheats, but I consider this purely for my own console verification work so it really only needs to replicate my personal NES.
The goal is to completely reorganize everything so that all operations are internally consistent and consist with NESDev documentation. I already fixed the ppu core, which was off by 1 in both cycle and scanline definition. I have read and write execution on the cpu side almost fixed, but the existing code is quite messy and it is a painstaking process.
Once the groundwork is done I can hopefully finally fix Batman as well as OneHundredthCoin's tests and a few other outstanding issues.
Making slow progress here still. I finally got a proper phase outcome in the reset state detect test, without breaking anything else in the process. I don't plan to try to emulate all 4 different phases, so this one is done. Now I can move on to the next phase of ppu work which is overhauling the VRAM address bus code to match NESDev documentation.
Link to video
My TAStm32 broke recently (and I broke it even worse trying to fix it) but thanks to DwangoAC I have a new one and can verify NES / SNES again. I gave it a test run with this simple Mario hack that was published recently to make sure it works with my existing process and seems good, so now I can try some Super Mario World ROM Hacks.
I also tried the game Cybercoaster but it crashes on my cart, not sure why though. EDIT: It crashes because the game only works with MMC3 chips with new style IRQ's but evidently the one in my cart is old style.
I'm making decent progress in my NES core rewrite attempt. I completely overhauled the main execution loop and rewrote most of the CPU core to be able to track the data bus better. With a bit of work I now pass 128/131 accuracy coin tests, with the remaining ones being relatively new ppu information I haven't implemented yet. So all the tricky DMC tests now work and match my console, and the console verified test cases I use still work, so so far I'm not aware of any regressions. Also the new core is much cleaner and more orderly, and I'm no longer chasing off by one errors around which is nice.
I was hopeful that all this work would move the needle on some of the desyncs like Batman and SMB3 ROM hacks, but to my bewilderment there is no improvement. The poll dump of Super Sheffy Bros 3 is the exact same. The one for Batman is different but only towards the end of the run, not at the desync point.
So, despite the considerable amount of work I have put into this rewrite, I'm somehow no closer to figuring out what the problem is.
I'll implement the remaining ppu cases to at least get to 131/131. My console actually fails 2 tests but I can just make sync settings for revision specific behavior.
Back to thinking about edge cases I guess.
Follow up from MMC3 testing. Since I realized that my MMC3B was an old style IRQ chip, it invalidated the previous testing I did with Batman. I had a game that did have an MMC3C chip, and I had a remaining unpopulated MMC3 dev board, so I put it in there and ran Cybercoaster. The game worked, so this is a new style IRQ chip, and I tried Batman and it did desync in the same place as all the other tries. So, nothing new learned but this cleaned up an incorrect analysis.
I tried once again to find the desync in Batman and Super Sheffy Bros but without success. At this point I don't think I can make any progress without tracking what's happening at the desync points with an oscilloscope, which I don't have. Maybe I will try some different games and see if anything pops up.
Otherwise I'm making progress on implementing new info found by 100th Coin and Fiskbit, the only thing I have left to do is edge cases of background rendering enable / disable. With that I will have a passing Accuracy Coin test. I also have subframe stuff working and SMB3 and SMB Bad Apple both work as expected.
At this point I don't think I can make any progress without tracking what's happening at the desync points with an oscilloscope, which I don't have.
So ask "Santa" to give you one for Christmas, then. :P
If only it were so easy! Really I should be able to correctly guess what the problem is by now, it's pretty frustrating to not be able to. I could potentially hack in a save state loader like I did for GBA, but I don't know where exactly things go wrong so I don't know what the exact state should be. Being able to watch M2 would basically immediately reveal what the problem is. Someday I will figure it out!
After another considerable rewrite of ppu emulation, I am able to pass the current AccuracyCoin test suite (without breaking anything else in the process.) All the 1's and 2's match my console too.
So, that's good. I also managed to verify Mike Tyson's Punch Out with my current setup. This was already verified but I couldn't work with this game previously but now I got it figured out. I also of course tried Batman but no change.
Since 100th Coin's emulator TriCNES let's you play .bk2 files on it, I decided to try Low G Man on it, which is known to desync seemingly needing extra polls inserted into the replay to work. TriCNES also let's you choose power up clock alignment. It turns out that some clock alignments sync the TAS, and some desync exactly as I see on console. So, no mysterious extra polls are happening, it seems my console just only powers up into the set of alignments that don't work.
I'm not sure what I need to change yet to adjust the cpu clock alignment on my end, but that's another mystery solved.
I'm not sure what I need to change yet to adjust the cpu clock alignment on my end, but that's another mystery solved.
You can't, sorta. The clock alignment is a result of the clock dividers inside the CPU (the APU actually) and PPU starting in a "random" state. The dividers are essentially just counters that increment on the master clock and roll over at 12 and 4 respectively (or 16 and 5 for PAL.) The only way to affect these dividers is to reset the CPU/PPU chips (either by powering the system off and back on, or using soft-reset).
I say they start in a "random" state in that the counters are implemented as a sequence of transistor gates that have an non-deterministic initial state. There may be a tendency for a particular alignment for each console due to age and manufacturing differences within each die, but it's effectively random.
So you can't force a specific alignment, but what you can do is repeatedly reset until you get a desired alignment. I suspect it may be possible to automate this process akin to how we can initialize RAM using a custom ROM. Couple differences though. I believe 100thCoin has come up with a technique for software to detect which alignment the system is in. Instead of (or in addition to) initializing RAM, you'd need to use that technique to detect the alignment, and if it's not the desired one, reset the system and try again. Once reached, hold the system in a loop awaiting a controller signal (instead of reset), then manually "reset" the system to boot into the game.
There would ideally be some form of communication between the software and the replay device, but that can be done via the controller.
So you can't force a specific alignment, but what you can do is repeatedly reset until you get a desired alignment.
Well, I reset lots of times but I only ever get the same desync, so I guess my console is very stable in startup state in terms of cpu alignment. Specifically in terms of 100th coin's numbering, the desync I see is for alignments 5-8, and the TAS works for alignment 9-0.
I guess I will adjust the emulator to match 5-8, I just need to figure out exactly what the effect is.
Follow up about Low G Man. This game is somewhat unique in that it reads from $2002 in a loop at times besides power on. This is important because it does so while NMI is on, and controllers are polled in the NMI loop. Reading from $2002 clears VBL flag and thus cancel NMI. So, what's happening in Low G Man is that there actually ARE extra controller polls happening, due to an NMI happening on console that isn't happening on emulator.
So, why? I don't have any ideas at the moment, but this happens early enough that I can investigate it much easier, maybe I can scroll through trace logs long enough to get an idea.
Joined: 1/24/2018
Posts: 416
Location: Stafford, NY
Alyosha wrote:
So, why? I don't have any ideas at the moment, but this happens early enough that I can investigate it much easier, maybe I can scroll through trace logs long enough to get an idea.
It seems your comment is quite prophetic. Low G Man reads from WRAM despite disabling it (and none existing for this game anyway.) see: https://www.nesdev.org/wiki/Tricky-to-emulate_games
This returns open bus, but it turns out open bus is somewhat noisy, in fact emulating this noise is needed to verify Castlevania 3 and Battletoads game end glitch.
Low G Man reads open bus and tests its value for a branch. If this doesn't match emulator every time it is read, which is very often right before it desyncs in the Ice level, there is a chance it mis-aligns the $2002 reads, which can cause the extra polls I see on console.
This is pretty pathological, and probably prevents this game from ever being console verified, but at least the mystery is solved.
Link to video
I finally figured out what was wrong with Batman. In the end it was a simple problem. Taken branches only process interrupts on the second cycle, but what happens when RDY is low during that cycle? Well it turns out interrupts should only be processed at the start of the cycle, not the entire time RDY is low, which is what I was doing before. A simple fix but it eluded me for a long time, and I am happy to FINALLY solve this riddle.
EDIT: This may only be true for NMI, more testing is needed.
A simple fix but it eluded me for a long time, and I am happy to FINALLY solve this riddle.
Congratulations from me!
Thanks!
This also fixed Super Sheffy bros 3:
Link to video
However for this one I needed to only make the above change for NMI, if I also make it for IRQ it desyncs in a different way. So I think I don't know all the details here yet. But anyway the answer is in there, it just needs some careful testing to suss out the details.
EDIT: Also fixes the Junkyard Challenge Batman rom hack.
Here is the repo I have been working in, for anyone who wants to replay the associated movies:
https://github.com/alyosha-tas/BizHawk_Research
It's not as polished as original neshawk and does some non-standard things like respects the prg ram size byte in the ines header, but it plays the movie files associated with the new verifications.
I retested Bad Dudes, and indeed with the recent improvements there is another controller poll where there wasn't before, solving the last riddle from my previous round of testing.
I think that wraps up the NES work I wanted to do for now. The new core still needs a lot of work, but it is now relatively simple stuff I can tinker with rather than big rewrites and intractable edge cases.
Over the past coupe of months I've been slowly working on a new C++ version of NESHawk, and I'm finally starting to see some results. The above screenshot is from the Castlevania III TAS I console verified a while ago working in the new core. The movie file that goes with it doesn't actually sync in any existing BizHawk version, and since I deleted my old research builds a while ago, it was actually orphaned with nothing that could run it. I had tried a C++ port some years ago, but it was limited and wasn't really in a workable state. This new version incorporates what I learned from GBAHawk development, so is much better organized. All the basics work like controller selection, audio, and save states, as well as other tools like hex editor and PPU viewer.
A lot of things are also removed to simplify the process, like PAL, FDS, famicom and unisystem specific code, and cheats, but I consider this purely for my own console verification work so it really only needs to replicate my personal NES.
The goal is to completely reorganize everything so that all operations are internally consistent and consist with NESDev documentation. I already fixed the ppu core, which was off by 1 in both cycle and scanline definition. I have read and write execution on the cpu side almost fixed, but the existing code is quite messy and it is a painstaking process.
Once the groundwork is done I can hopefully finally fix Batman as well as OneHundredthCoin's tests and a few other outstanding issues.
Making slow progress here still. I finally got a proper phase outcome in the reset state detect test, without breaking anything else in the process. I don't plan to try to emulate all 4 different phases, so this one is done. Now I can move on to the next phase of ppu work which is overhauling the VRAM address bus code to match NESDev documentation.
I'm making decent progress in my NES core rewrite attempt. I completely overhauled the main execution loop and rewrote most of the CPU core to be able to track the data bus better. With a bit of work I now pass 128/131 accuracy coin tests, with the remaining ones being relatively new ppu information I haven't implemented yet. So all the tricky DMC tests now work and match my console, and the console verified test cases I use still work, so so far I'm not aware of any regressions. Also the new core is much cleaner and more orderly, and I'm no longer chasing off by one errors around which is nice.
I was hopeful that all this work would move the needle on some of the desyncs like Batman and SMB3 ROM hacks, but to my bewilderment there is no improvement. The poll dump of Super Sheffy Bros 3 is the exact same. The one for Batman is different but only towards the end of the run, not at the desync point.
So, despite the considerable amount of work I have put into this rewrite, I'm somehow no closer to figuring out what the problem is.
I'll implement the remaining ppu cases to at least get to 131/131. My console actually fails 2 tests but I can just make sync settings for revision specific behavior.
Back to thinking about edge cases I guess.
After another considerable rewrite of ppu emulation, I am able to pass the current AccuracyCoin test suite (without breaking anything else in the process.) All the 1's and 2's match my console too.
So, that's good. I also managed to verify Mike Tyson's Punch Out with my current setup. This was already verified but I couldn't work with this game previously but now I got it figured out. I also of course tried Batman but no change.
