Posts for Alyosha
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Yes resyncing was necessary, as I get organized I am putting files here: https://github.com/alyosha-tas/NES_replay_files I mean things that got sent to lua scripts were slightly wrong (lag detection, input poll event), actual execution didn't change at all (except right now with the start up state update, but that doesn't matter for bionic commando.) Bionic Commando occasionally encounters situations where input is polled, but then not read because something else interrupts it. Before fixing things up this would result in bad lag detection and mess up the dump. Additionally, the all items run runs into a case where input is polled twice in one frame (it only happens once in the entire run!) and this case was missed by the frame dump script, so it needs the poll dump script. Link to video
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
The state is updated with the -4 value since it has the nice bonus of the Battletoads runs still working. I think the next big thing to focus testing on is the timing of the DMC glitch. Hopefully it's not random. Figuring this out will allow runs for games like Ninja Turtles. I also need to find some other MMC3 games that don't use DMC to have more confidence that Nightshade wasn't just a fluke,
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
http://tasvideos.org/userfiles/info/72385648792277409 Here is a console verified run that will work in dev build or 2.6.3 when it is released. It gets all the items in the right places, but I didn't put much effort into manipulation after that point so it ended up slower then the published run by ~100 frames. So, if anyone wants to use this to get a faster run that can be console verified feel free. EDIT: I found an improvement and submitted a new run.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I have updated NEShawk's startup state to match console testing I have been doing. Now several more timing sensitive games console verify that didn't work before. Donkey Kong, Nightshade, Roger Rabbit all sync on console through full runs. I nice side effect of the state I chose (out of 3 more or less equally reasonable options) is that the runs of Battletoads still sync, with very minor adjustment for initial lag frames. I have console verified the current 2 player warps run to verify this, will post videos when I organize everything. I've also made some other small updates to how input is polled so that games like Bionic Commando which hit some edge cases now console verify as well. this game always gave trouble before, but the emulation was never wrong, only the communication between emulator and console. I'm very confident in the new state, and am hopeful many more games can be verified now, and maybe progress can be made on DMC timing (assuming it's not random.) As a final side note, the Streemerz runs, which have never worked on console, still work in the new state, that game is a real mystery! Link to video
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Patashu wrote:
Out of curiosity, is NES startup state the kind of thing that tends to be similar between different people's NESes or different, or do you only know about your own for now?
I only know about my own, but while there is probably some variation amongst individual units it's probably not by much. I have a standard front loader nes with G model ppu (common.) I expect these results to more or less hold for similar models maybe with slightly more or less overall likelyhood.The proposed state gives a result in read2004 of 8, which True reports as pretty common. The only caveat is that I would not necessarily expect this to apply to top leaders. It could be, but needs testing.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I did some investigating of power on timing using the numerous playback attempts of Roger Rabbit I recorded. As it turns out it wasn't that hard to nail down exactly what the valid power on states were. In the following results, ofst is the offset from cycle zero, scanline 0 where the ppu starts that the cpu starts executing cycles, vbl is the state of the vbl flag at power on, and even_odd is the parity of the frame (one cycle skipped every other frame for NTSC consoles.) ofst = -1, vbl = true, even_odd = false; gives no match to console. ofst = -2, vbl = true, even_odd = false; gives exact match to console. Gives correct items. ofst = -3, vbl = true, even_odd = false; gives exact match to console. ofst = -4, vbl = true, even_odd = false; gives exact match to console. ofst = -5, vbl = true, even_odd = false; gives exact match to console. ofst = -6, vbl = true, even_odd = false; gives no match to console. ofst = -1, vbl = true, even_odd = true; gives no match to console. ofst = -2, vbl = true, even_odd = true; gives no match to console. Gives correct items. ofst = -3, vbl = true, even_odd = true; gives no match to console. ofst = -4, vbl = true, even_odd = true; gives no match to console. ofst = -5, vbl = true, even_odd = true; gives no match to console. ofst = -5, vbl = false, even_odd = true; gives no match to console. ofst = -4, vbl = false, even_odd = true; gives no match to console. ofst = -3, vbl = false, even_odd = true; gives no match to console. ofst = -2, vbl = false, even_odd = true; gives no match to console. ofst = -1, vbl = false, even_odd = true; gives no match to console. ofst = -5, vbl = false, even_odd = false; gives no match to console. ofst = -4, vbl = false, even_odd = false; gives no match to console. ofst = -3, vbl = false, even_odd = false; gives no match to console. ofst = -2, vbl = false, even_odd = false; gives no match to console. ofst = -1, vbl = false, even_odd = false; gives no match to console. ofst = 0, vbl = false, even_odd = false; gives no match to console. The results are pretty definitive. No valid states result from even_odd being true or from VBL being false. The documentation says that VBl is expected to be on some of the time, but I couldn't find any cases where it was off that matched playback. I originally expected valid ofst values clustered around 0, but I guess the CPU takes an extra cycle to reset, so -3 is the next most likely one, which is what we see here. values of -5 and -2 are possible but of low probability. (Mesen appears to match -5 in 0.9.9.) I then tested nightshade, and got a syncing run using -2, but only with very low probability ~1/20 or less. Testing with -3 resulted in a desync, not sure why, maybe something to do with MMC3 clocking. -4 did result in a syncing run though with much higher probability (worked on the third try.) A run of Donkey Kong on the -4 setting also synced to console. So with all of that, it seems very likely that the new startup state will be ofst = -4, vbl = true, and even_odd = false. I'm going to get complete runs of Nightshade and Roger Rabbit tested on these setting, and then try Battletoads, if everything works I will update NESHawk settings in master.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Game: Who Framed Roger Rabbit Emulator: BizHawk: NESHawk, local build with new startup state Console Verification Device: TAStm32 Movie:http://tasvideos.org/userfiles/info/72277515094486081 Description of Desync: Enemies are not in the correct locations after game start Research: Items do (occasionally, ~1/10 chance) appear in the right place on console. I have confirmed this by stopping playback and then manually checking the locations they should be in, but the enemies are in the wrong place, in a seemingly deterministic fashion. Un-initialized RAM is not the cause as I played the movie several times with random power up RAM state with no desyncs. This is curious as getting the right item locations requires accuracy to the ppu level already, but for some reason after that it doesn't work. I have looked at some trace logs, and the only interesting thing happening is that the game starts to use sprite zero hit once gameplay starts, after item location is chosen, maybe this hints at something wrong with sprite zero hit, but it passes all the relevent tests so I don't know. I tried the same movie on Mesen 0.9.9 but it desyncs in a different way, also not observed on console. Possible Next Steps: The game also encounters the OAM address glitch, but resets $2003 properly. Not emulating it all does not cause a desync anyway. Never the less thinking what might be wrong with sprite zero hit seems like the most likely way forward. Status: Open. Closed. resolution: Testing of power on state revealed the correct settings that give the observed enemy positions and correct item locations.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I updated NESHawk's timing of the oninputpoll event to match actual polling and made an appropriate dump script, so now I can test games that do more then one poll per frame. http://tasvideos.org/userfiles/info/72268404788366985 With this I was able to verify Balloon Fight: Link to video I also verified the all items run of Bionic Commando as well (although that run is outdated as it doesn't use the skips in the last level.) I'll get some videos up later today and organize .r08 files.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
CasualPokePlayer wrote:
So by chance you can effectively enter into several different "odd modes" for the NES. At least for GB/C that can only happen for GBC and that's with multiple speed switches...
Essentially yes, it makes tracking down emulation issues especially difficult, because if you have a run that desyncs in 3 different ways, how do you know which one is the emulation issue and which ones are from the wrong alignment? This is why I have anchored NESHawk to a particular run of Battletoads that was shown to work on console for so long, changing it without a lot of evidence that a different state would be better was just too risky. Anyway, I figured out what is wrong with Bionic Commando, incompatible lag definition between emulator and replay device, and having fixed that it now works. I'll be streaming the results a bit later if anyone is interested. I also updated the dump script for NESHawk which works better with the new Frame-increments-after-emulation rule. http://tasvideos.org/userfiles/info/72231161716346055 Unfortunately a lot of runs I try use uninitialized RAM, so progress is pretty slow on actually verifying new things, eventually I'll need to make a RAM clearing cart, there's a lot to do.
Post subject: TASes effected by uninitialized RAM
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I am encountering a large number of runs in my NES testing that rely in some way on uninitialized RAM, so I thought I would make a thread for such runs here so the knowledge that they need special RAM clearing for further testing is written down. [717] NES Monopoly "4 CPUs" by FractalFusion in 01:07.67 [4104] NES Monopoly by adelikat in 00:29.53 [2847] NES Rad Racer by FatRatKnight in 20:32.89 [2120] NES Silver Surfer by Skaad & HardCoreMangeur in 29:45.41 [4349] NES Marble Madness by Aglar & LeKukie in 02:42.07 I don't have a good way to do RAM clearing right now, but hopefully I'll be able to come back to these eventually.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Bigbass wrote:
Alyosha wrote:
Donkey Kong syncs with minor adjustments to the current run, but only with 1/4 probability due to start up variability.
Curious, what is the variability from? Uninitialized RAM affecting something?
I mean this hardware effect: http://forums.nesdev.com/viewtopic.php?t=6186 For games requiring careful timing, that use most ppu features, this basically gives them a 1/4 chance of working from power on. Also there is probably some additional variability of the ppu and cpu seeing the end of reset at slightly different times. If you look earlier in this thread, you can see some results from True when running the read2004.nes test rom on hardware. He got more then 4 different values, meaning something else is giving some timing variability. I think this is what is keeping Nightshade from working (or possibly working but with very low probability.) Almost all NES test roms test relative timing between events, so we don't know very much about absolute timing from power on. I'm not even entirely sure the test rom suite is internally consistent in always relying on the same power up alignment. NES is far messier then GB/C in this sense. Mesen 0.9.9 emulates the alignment variability (options->emulation->advanced-> randomize power on/reset cpu/ppu alignment) if you are interested in tinkering with it. Link to video
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Mixed results in testing today. Donkey Kong syncs with minor adjustments to the current run, but only with 1/4 probability due to start up variability. T&C surf designs syncs with the OAM glitch implemented properly. Silver Surfer needs RAM cleared, I think it would work with that, but doesn't with just hard resets. Nightshade doesn't work, not sure why, but it's MMC3 so might be related to that. Bionic Commando desyncs in the same spot as described by Bigbass, very strange as nothing of note appears to be happening at that point.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Got my TAStm32 board working and am streaming some Donkey Kong TAS verification attempts now. Current NESHawk settings resulted in no syncs. Mesen 0.9.8 settings also resulted in no syncs. Currently trying alternate settings that work for SMB3 glitch run, with some success.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
please try the dev build, I think this issue is fixed already: https://ci.appveyor.com/project/zeromus/bizhawk-udexo/builds/39575136/artifacts
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Please remove me from this and the other Princess Rescue submission, my contribution is insignificant, no need for name clutter.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Started doing some preliminary work. I confirmed that the OAM glitch occurs on console and that if I implement it in NESHawk I can match sync with Mesen 0.9.8. I found several other games that encounter the new OAM glitch, apparently it's not too uncommon to turn off the screen during rendering. They reset $2003 with manual writes though, so are unaffected. Looked at Monopoly, confirmed it uses uninitialized RAM, but is otherwise not too timing sensitive, those runs should work on console with RAM cleared to 0.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
That doesn't sound right, non-sync settings shouldn't be part of the state or related to movie settings, I'll take a look at it.\ EDIT: fixed
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Dusting off this thread as I prepare for renewed NES console testing. Surprisingly there are some notable changes since I last looked at this. Probably one of the first important things to look at is this: https://forums.nesdev.com/viewtopic.php?f=3&t=18414 This glitch (currently unimplemented in NESHawk, but is in Mesen) actually effects a published TAS: http://tasvideos.org/3314M.html . It also occurs in Streemerz (but I can't tell yet if it effects sync.) The internal workings of Mesen timing have changed significantly as well. I used to be able to match execution between NESHawk and Mesen but as of 0.9.9 I am unable to do so. There isn't much testing here yet, could be interesting, maybe just using current Mesen will solve all the problems. Unfortunately, Sour has since left emulation, so if something doesn't work, it's not really clear what to do. I have several games lined up for testing: Donkey Kong: No agreement between NESHawk, Mesen 0.9.8, and 0.9.9, will be interesting to see what console shows. Nightshade: Again no agreement Bionic Commando: NESHawk and both versions of Mesen agree that this run should work on console, but it never has. Lolo 3: shows interesting power up timing variability between emulators Lots of interesting stuff to work on. My goal (still) is to see Streemerz work on console.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
EZGames69 wrote:
Just as a note, I did not add the console verification flag to this movie because the current video is of the old run, not the more current one. I can set it as console verified once a new video is posted.
New console verification video added to the submission.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Here are 2 runs I made, one to unlock 800cc and one to beat 800cc and unlock world circuit. I guess a real run should be done on world cup, but the 800cc one is pretty well optimized itself. Unlock 800cc: http://tasvideos.org/userfiles/info/69190303673171301 Unlock World Circuit: http://tasvideos.org/userfiles/info/71807288799326461
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Ok, thanks for your help anyway. HandsomeRudy removed as co-author and submission notes updated. I think this is a good place to wrap this one up anyway.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Did some testing of castlevania adventure in the dev build with updated gambatte. Execution is now identical up to a desync due to mode 3 timing.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
The changes were made two days ago.
movie.getinput(emu.framecount() - 1)
is what would make it break. Now it should just be
movie.getinput(emu.framecount())
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
After comparing video again, I realized that something wasn't right with how input was getting sent to the console. As it turns out, the recent fixes I made to fix the BizHawk issue with autofire broke the dump script (because it was decrementing the frame count from core, which it no longer needs to do.) After fixing that (updated script posted to OP), everything syncs just fine, there is no mystery here, only me not keeping track of stuff carefully. @CasualPokePlayer: this might effect gambatte too, might want to check input dumps in dev build. I did have to delay input in one spot because input was polled right on a frame boundary, nothing I can do about that one. It also syncs with or without the window timing fix, so I'll have to investigate that one on it's own some more. Gambatte will still need the mode 3 fixes and the end-of-VBL fix to work, but those are already known issues. So, I'm calling this one closed.
Alyosha
He/Him
Editor, Experienced Forum User, Published Author, Expert player (3602)
Joined: 11/30/2014
Posts: 2749
Location: US
CasualPokePlayer wrote:
Here are the tests the hacky "fix" made fail: https://cdn.discordapp.com/attachments/842947491435118622/845859569195417610/failed_tests.zip
Thanks, looks like GBHawk still fails about a dozen of those, including some single speed ones. The last time I worked on screen timing stuff was quite painful, hopefully nothing is too seriously wrong. I took a closer look at where things go wrong compared to console, and it's actually much earlier then I thought. The game turns on the screen mid way through a frame when it's ready to start gameplay, and this gives a helpful visual queue of how things are syncing. Here is what GBHawk and Gambatte show: and here is console: So something is wrong in between pressing start and game loading, I have no idea what, I don't think there are any sprites involved at this point. If I make GBHawk's VBL behaviour match Gambatte, then I do get the same screen as Gambatte, so it seems like there might be some new behaviour at play here.