@ true: http://tasvideos.org/userfiles/info/34108792710373640
Ok I tested the most recent Kirby's Adventure run. I had to wait one extra frame before hitting start on the title screen, but other then that it synced through world 1. I truncated the movie to where it desynced in 2-1, so if it makes it there I guess that is a good start.
I also tested Battletoads, 2 players warps, but it de-syncs at the life stealing enemies mid way through level 3. This one I traced back to the title screen, where there is an extra lag frame in BizHawk not present in FCUEX which seems to throw off RNG, changing where the enemies in level 3 go. This seems to come from OAM DMA, which is too short in FCEUX, and the extra cycles it saves over the first 1000 or so frames convert into a saved lag(loading) frame.
Let me know if there are any other games you think it would be useful to test.
I also realized that Ninja Gaiden is effected by the initial conditions of the DMC channel clock and bit counter. The game does nothing special to initialize them, and I had previously only guessed at initial conditions. Since these are constantly going at all times, there can be radically different sync results depending on the iniital state there. But after some tinkering the furthest I got was mid way through the game. another variable to look at though.
EDIT: i also set BizHawk start up back to matching FCEUX again, so now there is no need to add or subtract any frames from an fm2 converted to bk2, this should make testing more convenient and remove a potential source of error. I tested Zelda with this build at least up to getting the step ladder and it synced just fine.
Hey, you're looking for games that are console verified but desync in BizHawk right? I just tried playing back [1014] NES Journey to Silius by klmz in 09:32.03 since I had the ROM, but it desynced on the first stage on 1.11.7.
On 1.11.6 it desyncs on the 2nd stage instead. No idea why.
Oh, and I had to convert it to a fm2 file first before playing it on BizHawk, or else it doesn't even play past the title screen.
Ok, I've got a lot of new info, hopefully this will help make some progress.
Let's start with monopoly since it's the easiest. Monopoly appears to rely on uninitialized RAM. I ran acmlm's run on NESHawk with default RAM settings (the usual FF-00 pattern) and the run does sync. Then I ran it with all start up values set to AE and it desynced after the menus. I haven't tracked down which addresses exactly are at play, but that's what's going on with that one.
I'll try Journey to Silius next.
EDIT: nevermind all that other stuff that was here before about battletoads, turns out my movie file was corrupted somehow. Back to the drawing board
EDIT2: Silius syncs after adjusting for lag, no change of input needed. Will check Ninja Turtles next
EDIT 3: Ninja Turtles does not get the ninja star drop in the Dam level
You can probably make it sync just by adjusting the RNG, which is done by adding random button presses before the frame in question.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I also tested Battletoads, 2 players warps, but it de-syncs at the life stealing enemies mid way through level 3. This one I traced back to the title screen, where there is an extra lag frame in BizHawk not present in FCUEX which seems to throw off RNG, changing where the enemies in level 3 go. This seems to come from OAM DMA, which is too short in FCEUX, and the extra cycles it saves over the first 1000 or so frames convert into a saved lag(loading) frame.
Let me know if there are any other games you think it would be useful to test.
http://tasvideos.org/1920M.html this syncs way farther on console from FCEUX than it seems you got.
Didn't notice Kirby until now; I'll test in a bit.
- EDIT: tested, but it desyncs in level 1?? Doesn't get the UFO glitch...should I be running a newer build than 7727a70?
- EDIT2: Newer build syncs as you described on the emulator
- EDIT3: Console synced very similar to build 7727a70... anything I should continue testing here? I don't have this build anymore though.
hegyak wrote:
Try these runs:
Monopoly (Either)
Snake Rattle and Roll (Either)
Teenage Mutant Ninja Turtles
I would prefer testing games which have been shown to sync on console already. Also, games without the DCM hit would be nice to check.
Aloysha wrote:
EDIT 3: Ninja Turtles does not get the ninja star drop in the Dam level
@True: The Kirby run has a reset on frame 6, how does the bot deal with that? The only difference between the builds is how many lag frames there are at startup I believe. Kirby also used the DCM channel but seems to be more careful then NG about how it uses it.
I'm stumped on Battletoads, need some new info before I can figure out how to fix that one.
EDIT: Ok, so I finally finally finally figured out Battltoads. It turns out this was breaking on SPRDMA timing. It takes either 513 or 514 cycles to complete an even or odd DMA depending starting cycle..
As it turns out, I had my check backwards. Normally this wouldn't matter since you might expect each DMA to roughly randomly take place on an odd or even cycle. However, it turns out most of Battletoads' DMAs were taking place on the same type, which happened to be 514, when this should have been 513. The accumulation of extra cycles eventually added a lag frame at the menus which changes RNG and causes the desync at level 3. Setting the DMA timing correctly gives the right result and the run syncs through level 3.
As a bonus, this fix makes the mysterious behaviour in the second test of scanline/scanline appear correctly. So that's a 2 for 1.
Be the first as usual! Happy Dragon's Year to all of our TASers!
By the way, why isn't it on the Front Page yet? Something changed in the submission system?
There is a 5 minute cache on the front page.
Also VERY impressive movie! Some surprising new tricks. You've managed to make SMB TASes so much more entertaining over the years. Good job. (Yes vote, obviously)
EDIT:
It turns out this movie desyncs with the new PPU on 3-1, the trick in the screenshot actually causes lag and makes the trick fail. I would suspect the same thing would happen if console verified (DarkKobold, care to try?)
enoent wrote:
I just console verified this on the first try with (a slightly modified version of) micro500's bot. Maybe FCEUX improvements in the last few years have made it possible? I was using version 2.2.1 for Linux to strip the lag frames. What are the steps for getting this marked as console-verified?
Full disclosure: the second time I ran the movie it desynced on 2-1, and the third time it desynced 3-3. I think that was just from ESD or insufficient decoupling or something silly like that.
It doesn't sync on new PPU on FCEUX, and on console it syncs inconsistently.
By adjusting for lag, I mean that if there is a lag frame in BizHawk that is not present in FCEUX, I add in a frame at that point in BizHawk, this isn't necessarily a blank frame, it could have button presses in it. The idea is that every valid input frame in FCEUX will also be an input frame in BizHawk, even there are lag differences. I don't know how the RNG works, but it could be that it only updates on input frames.
I don't know enough about new ppu vvs old ppu to comment on why that Mario run desyncs though.
@True: Here is a Battletoads run (2-p warps) that syncs on the latest BizHawk build through the level 8 boss. I think it's possible that it will sync on console where FCEUX fails. It is identical in terms of input up that boss.
http://tasvideos.org/userfiles/info/34147851839238345
Also, Battletoads does some initializing with this code:
LDA $#0
LDX $#17
STA $4017,X
DEX
.
.repeat until X=0
.
BizHawk and FCEUX report this as an input frame (frame 3) since it is a write to $4016 (also a read from $4016 since it's indexed and the CPU reads before it writes.) But since a 1 was never written, no controller data is actually latched. Does the bot see this as an input frame as well?
@True: The Kirby run has a reset on frame 6, how does the bot deal with that? The only difference between the builds is how many lag frames there are at startup I believe. Kirby also used the DCM channel but seems to be more careful then NG about how it uses it.
It doesn't.
Kirby resets as soon as the game inits RAM and the game knows it has booted once. In my case, I power on the console, hold reset then start the bot. It effectively does the same thing in this case.
Alyosha wrote:
EDIT: Ok, so I finally finally finally figured out Battltoads. It turns out this was breaking on SPRDMA timing. It takes either 513 or 514 cycles to complete an even or odd DMA depending starting cycle..
Good news. Awaiting a testrun / something to test on hardware. (EDIT: you posted it in another post. I'll test when I get home.)
Alyosha wrote:
BizHawk and FCEUX report this as an input frame (frame 3) since it is a write to $4016 (also a read from $4016 since it's indexed and the CPU reads before it writes.) But since a 1 was never written, no controller data is actually latched. Does the bot see this as an input frame as well?
Every write to $4016 will cause a latch, and yes, it will be interpreted as an input by the bot, because the latch line was strobed. Now if I need to pad the input at the beginning to compensate for an additional "controller read", I can do that (and do this as a matter of course if a game doesn't sync immediately; most games sync at default settings). I will keep this in mind while testing your testrun.
It doesn't sync on new PPU on FCEUX, and on console it syncs inconsistently.
Modern robots are not inconsistent with sync. I think the stated run synced just fine on console (in fact, every Zelda or SMB run I have tried has synced).
@True: Here is a Battletoads run (2-p warps) that syncs on the latest BizHawk build through the level 8 boss. I think it's possible that it will sync on console where FCEUX fails. It is identical in terms of input up that boss.
http://tasvideos.org/userfiles/info/34147851839238345
Tried it, desyncs in Turbo Tunnel at the life stealing enemies. One goes to the right and steal's Toad 2's life.
desyncs on console in Turbo Tunnels? That is where it was desyncing in emulator before I swapped the parity of SPRDMA.
That run contains the exact same input frames as the FCEUX version up to the level 8 boss. The console verification page says the run makes it up that far 5/5 times.
Something must be off here. Can you try the FCEUX version and see what happens to that one? Or compare the input log to see if there are discrepencies?
That is a very strange result.
desyncs on console in Turbo Tunnels? That is where it was desyncing in emulator before I swapped the parity of SPRDMA.
That run contains the exact same input frames as the FCEUX version up to the level 8 boss. The console verification page says the run makes it up that far 5/5 times.
Something must be off here. Can you try the FCEUX version and see what happens to that one?
I dumped the FCEUX inputs and compared, and yup they are identical up to the boss.
Well, I'm stumped. 8D
Any ideas? Maybe the run just doesn't sync as reliably as I had hoped.
Any suggestions for other easier / shorter runs to try?
I feel like I'm really close but missing a piece of the puzzle.
If the inputs are the same, it should sync the same...
Dealing with some data recovery at the moment; let me check this tomorrow. I don't think I did a RAM clear to get this to sync, but I did not try starting from reset.
I'll edit this post with what I find.
EDIT: Having the same desync problem with FCEUX input. I'm not sure I changed anything with my old robot code? But I don't know when I did these tests. Tried every form of reset I could. My last guess is memory init, but I'm assuming NESHawk doesn't use the bad FCEUX init? ....I finally found my other robot, tested, the FCEUX ran desynced at the snake level just before the warp (I think I saw frame ~7900), and trying again...it is desyncing in Turbo Tunnel. So it looks like, maybe, I need to init RAM before I test this? Can you find out if you think this may be required?
I can say, nearly every run desyncs in the same way, with the life stealing enemy at the bottom not going left, but going right.
Battletoads initializaes all RAM to 00 except for FD and FE. It does this almost immediately after the first VBlank it sees.
FD checks whether the intro has been played through once. If it has, then you can skip it if you hit reset. (The value it uses to check this is 0x28)
FE is not used in any way as far as I can tell.
RNG is initialized with a set seed.
As far as I can tell, the only thing resetting (before the intro plays) does is possibly change the even/odd alignment of the DMA unit relative to code execution, which is known to effect timing enough to change the enemy direction in level 3. Initializing RAM in any way has no effect.
Your present tests seem to indicate that the original alignment was correct. Actually I was scrounging around NESDEV and found some tests that also indicate it was correct before. So I think I'll revert that change.
My only guess is that in your previous tests you happened to do a reset at a point where it would change the DMA alignment, or that on startup there is a small chance of it being changed (since one of the above synced to snakes.)
I'll make a new run with the enemy in level 3 going to the right and see how far it can go. ACtually, it turns out you can get the enemies going back to the left simply by putting one blank frame before the second 'Start' press right at the beginning of the game. So I'll do that and see if it works on console too.
EDIT: http://tasvideos.org/userfiles/info/34230566350048461
Ok here it is. (Don't forget to update your BizHawk build.) This one should get the enemy going back to the left on console. If it does, then I think we are making progress and I'll get the run back up to level 8 or maybe try the new game end glitch run.
EDIT 2: just checked and the newest game end glitch urn syncs on BizHawk without modification, so might want to check that one too since it's short.
Thanks for the followup. Your explanation makes sense to me.
It may have synced before because I may have been using the toploader NES instead of the frontloader NES. Perhaps they're acting differently somehow (manufacturing tolerance or something). I wish I wrote down more details of the failures.
Anyway, good news! This is my first NESHawk successful sync. It synced just as shown in the emulator.
I couldn't get the glitch run to sync in NESHawk (removed 1 frame, added up to 2 frames). When running the FCEUX dump, the same problem with the life stealing enemy happens.
Wow actual progress alright! Of course presumably many RNG values get the enemy going in the desired direction and there is no way to know if we are really matching console, but for now I'll take what I can get!
Well then might as well be bold and try to beat that level 8 boss.
http://tasvideos.org/userfiles/info/34263027362025815
Here is a movie file that does so. If a miracle happens and it syncs through the boss, it should desync at the first checkpoint in level 8. If it doesn't sync, can you maybe get a video clip of the boss fight? It would be good to know what attacks the toads are doing. (Or even just a description would be helpful.)
ALso, my mistake on the game end glitch run. It did require some adjustment to work in BizHawk, just not in terms of input frames. It did need some lag adjustment though. But since the input frames were the same, I'm a bit surprised it desynced on console at that enemy since it syncs there in both emulators. Oh well, I guess there is a bit more to figure out here.
Wow actual progress alright! Of course presumably many RNG values get the enemy going in the desired direction and there is no way to know if we are really matching console, but for now I'll take what I can get!
Well then might as well be bold and try to beat that level 8 boss.
http://tasvideos.org/userfiles/info/34263027362025815
Toad test syncs successfully. (this is the first time we've gotten past this boss on 2p warps console verification.)
First try it did NOT sync with live stealing enemy desync, so this may be console-biased. But still, it did sync through second time. Good work, keep going for it :)
EDIT: After Battletoads, another good / easy candidate for sync would be SMB3. The long runs do not sync on console. SMB3 is a workable choice however as we know it has DMC hits that affect sync. I could test against your implementation of this by dumping inputs on a per-poll basis. If the console run syncs in this mode, this would be proof of accuracy greater than FCEUX.
1) Does event.oninputpoll poll for input any time latch happens? Or just once per frame?
2) Will it be possible to use/read/get subframe input in the future? Not important for these runs, but already shown to be important for some ACE runs that have no emulator with working native support.
Woah it actually synced? Awesome!
Ok let's try the whole run then:
http://tasvideos.org/userfiles/info/34284092070056771
I guess it's no longer the same run since inputs are changed, but it beats the game with both players, so that seems like a pretty good test of accuracy to me.
I think the times it doesn't sync are probably due to different PPU-CPU alignment, which is known to be pretty much random at poweron and can influence timing. This would make sense with how random the desyncs seem to be.
With this in mind it seems like the game end glitch run should sync, maybe try it a few more times to see if you get good luck? Here is a BizHawk file just in case for comparison, but input frames should match:
http://tasvideos.org/userfiles/info/34284149904082436
I'll try SMB 3 next. The last time I tried all stages it desynced the same way as on console in world 2, so I'm a bit optimistic about that one. But going per poll would be interesting.
1) Does event.oninputpoll poll for input any time latch happens? Or just once per frame?
I'll look into it. EDIT: It's every poll. More specifically, it's called everytime a button is read, so for Ninja Gaiden for example it is 16 times per frame, 17 if there is a DMC hit.
2) Will it be possible to use/read/get subframe input in the future? Not important for these runs, but already shown to be important for some ACE runs that have no emulator with working native support.
Possible? Sure easily. I could hack it in there now as an alternate input method. But it would basically be equivalent to the scripting method used to do the SMB3 run. Natively supported as in working at the same level as input does now, with rerecording and TASStudio and such? I don't think this will happen any time soon.
I'll try SMB 3 next. The last time I tried all stages it desynced the same way as on console in world 2, so I'm a bit optimistic about that one. But going per poll would be interesting.
I think the run is on PRG0. If you can do this against PRG1 that would be preferred as I only have the PRG1 version. If not I will try to come up with a donor. Also, it got farther than world 2 IIRC - I remember desyncing in water just after coming out of a pipe, but can't remember the level. The world 2 thing was using PRG1 vs PRG0 on an emulator.
Alyosha wrote:
It's every poll. More specifically, it's called everytime a button is read, so for Ninja Gaiden for example it is 16 times per frame, 17 if there is a DMC hit.
Wow, that's annoying. Any way to fix this, or to add an onlatch and onclock event to lua similar to LSNES? The amount of reads isn't the only important piece of information - knowing when the data is actually latched is very (and more) important.
Alyosha wrote:
Possible? Sure easily. I could hack it in there now as an alternate input method. But it would basically be equivalent to the scripting method used to do the SMB3 run. Natively supported as in working at the same level as input does now, with rerecording and TASStudio and such? I don't think this will happen any time soon.
Yes, I am talking about native support to decouple input frames from video frames. They are two different things but opinions expressed before (from memory, I could be wrong) showed no interest / care in this fact.
----
2P Warps: Synced no problem.
Glitched: Took several attempts before the life stealing enemy cooperated. After that, it synced. If these are the same inputs that would be made from FCEUX then this run can be marked as console verified.
Re: SMB3, I guess I'll need to write a dumpscript that counts polls per frame then repeats the input that many times div 8 until a proper latch/clock event can be implemented once you have something to test.
Wow, that's annoying. Any way to fix this, or to add an onlatch and onclock event to lua similar to LSNES? The amount of reads isn't the only important piece of information - knowing when the data is actually latched is very (and more) important.
I have no idea. That's more of an adelikat or zeromus level question and not really an Alyosha level question. 8D
Ok I'll see what I can put together with prg1. Hopefully by tomorrow I will have something worth testing.
Wow great Battletoads synced! Time to make some Battletoads runs on NESHawk to get some console verifications! Thanks for your testing efforts!
I don't really know what the standards are for adding the console verified tag. Technically the FCEUX run is wrong and obtains the correct result in part from blind luck. But on the other hand all the actual input frames are indeed the same, so it does contain the correct stream of inputs, although every actual frame is not the same.
I don't really know what the standards are for adding the console verified tag. Technically the FCEUX run is wrong and obtains the correct result in part from blind luck. But on the other hand all the actual input frames are indeed the same, so it does contain the correct stream of inputs, although every actual frame is not the same.
I will add this tag then.
As long as the inputs result in the same outcome, I don't think we're all that caring of how it gets there right now. Though of course the prime goal (at least mine) is better emulation, and through better emulation, preservation of history.
small edit: You can see these sync at http://twitch.tv/is_true; look under previous broadcasts. Not too important so they'll be gone whenever they time out, but if you are interested in seeing it and didn't see it live, it's there for now.
http://tasvideos.org/userfiles/info/34305774732879677
Here is a world 1 run of SMB3 on PRG1, it cmes directly from the current warpless run (which syncs on PRG1 up to the boss.)
Here are the frames in the first 3 levels that NESHawk says a DMC conflict occurs on:
623 , 1000 , 2197 , 2256 , 2327 , 2411 , 2736 , 2747 , 2748 , 2779 , 2791 , 2827 , 3583 , 4345
So if you play this back purely per poll, it should desync very quickly if any hits are different.
And here is another interesting test to try:
http://tasvideos.org/userfiles/info/34305974613284573
This is a ninja Gaiden movie that does nothing but alternater pressing left and right (after moving to roughly the center of the screen. )
On NESHawk this ultimately moves to the right and hits the first enemy at 111 on the timer. It would be very interesting to see what the behaviour is on console.
EDIT: True, I think I know another way to test to DMC behaviour that would be really helpful. The test ROM read_errors_fast prints on screen when there are DMC hits.:
https://github.com/christopherpow/nes-test-roms/tree/master/read_joy3
The problem is that it is inconsistent when using things like everdrive because it doesn't sync to the timer in any way (as demonstrated by feos a couple pages back.) But if it can be dumped to real cart, I think it should have consistent behaviour from power on. Are you able to do this?
