It's a new year, and almost 3 years already since I started this thread, wow that time went fast. That time though saw big advances in emulation and console verification on a number of systems. Here's my synopsis of the current state of R&D.
NES: The biggest deficiency left for NES is reset behavior, but few runs use that anyway. Aside from that many / most TASes should be verifiable, even ones that rely on precise power up timing and DMC glitches. Console verification of NES TASes is now pretty standard, and it is far more rare to see one fail than succeed (as long as its being done on NESHawk.) There are some edge cases that are un-emulated in NESHawk, but they are unlikely to ever come up in practice, probably only if some homebrew dev used them for an emulator check. Implementing these would require a major rewrite to NESHawk, and I'm not sure they even effect all hardware revisions. Anyway, NES is in good shape.
SNES: Modern versions of BSNES are now in BizHawk, but even so console verification has limited success. Most desyncs are random, very likely due to the audio chip clock, a well known issue. There are also some deterministic desyncs, though with unknown cause. There have been a few successful verifications along the way, and recently I saw that DwangoAC managed to verify a WIP of SMW 96 exits. Surprisingly, there is even new emulation discoveries still happening, as very recently an un-emulated cpu quirk was found. It's hard to say what is needed to advance verification efforts for SNES, it might even take some kind of hardware mod to tame the SPC clock. Deterministic desyncs make me think that something is not right on the emulation side, although it could still be a bot issue, but where to even look?
GB/C: Nothing much new has come up that I'm aware of for GB/C, its pretty mature.
GBA: Lot's of progress has happened on GBA. Tests from Flevoriux have nailed down most ppu timings and emulation details. Myself and GhostRain0 have found a lot of details related to prefetcher emulation. I've also found a few of other minor emulation details for other system components. All of this effort on the emulation side has led to many new verifications, including of first party games. There is still a ways to go, with some known desyncs and emulation deficiencies. However I'mn hopeful (and pretty confident) that 2024 will see GBA emulation and verificaiton mature to the level of NES and GB/C.
N64: A great amount of emulation effort has taken place for N64 since my original post. The N64 core of the Ares emulator is maturing quickly and is increasingly accurate, with several talented N64 devs contributing. Even cache emulation is now close to cycle accurate, it's an impressive feat. I'm not aware of any actual console testing taking place yet, and perhaps it's still early as there are still many issues, but there are exciting prospects. Maybe I'll do some testing myself later this year.
DS: While it seems like a likely next target for work after GBA and along side N64, the DS is a very complicated system and much work still needs to be done.
Genesis: Bigbass made a new verification pipeline and tools for Genesis verifications and even made the first new verification for Genesis in a long time. This is impressive work and now there is a lot of room for more genesis testing and verification.
I believe that sums up all the major developments. Console verificaiton is overall much more mature than it was 3 years ago, in some cases it is even commonplace. Also compared to 3 years ago, a lot of the low hanging fruit is now gone.
So, what now? Personally I don't know what I will do once I finish up work on GBA. N64 looks like the only approachable system, though maybe its worth it to take another look at SNES. Then there are basic questions like what do we do about disc based systems? What do we do when clock speeds get too high to emulate systems in a cycle accurate way efficiently?
That about sums up my thoughts, anyone else have any thoughts or predictions?
