Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Ok I just commited a fix to MMC3 IRQ clocking that fixes Recca, Burai Fighter, and Mickey's Safari in Letterland. All 3 of these games are known to require precise IRQ clocking behaviour and didn't work properly before this patch.
Most of the relevent info can be found on this thread:
https://forums.nesdev.com/viewtopic.php?f=3&t=11361
However, there seems to be some disagreement over exactly what is happening here on a hardware level and different emulators seem to be dealing with the edge cases a little differently. New hardware tests would likely be needed to resolve them.
All the MMC3 test ROMs still pass with this patch though, and I have noticed no deficiencies with any other MMC3 games. (The changes would be invisible to most games anyway as they only really effect edge cases.)
Still I would greatly appreciate anyone willing to test their favorite MMC3 game and let me know if anything is wrong.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I can certainly see where the initial vector for initializing RAM I used is non-physical based on what True said, so I will re-sync this run to one where all of RAM is 0 except the 3 bits I purposely set (well it's actually 2 bits that are set to 1)
RNG is usually just a matter of pressing select until you get what you want, so it should be done later today. I'm a bit surprised the run de-synced since I was quite certain I looked at RNG before and concluded it wasn't impacted by uninitialized RAM, but it's possible I just made a mistake since I didn't have the tools back then to easily check like now.
Anyway I guess my opinion would be that it's fine to deviate a bit from 'ideal' NES behaviour so long as it doesn't venture into the realm of statistical impossibility. Where to draw the line? Only a certain number of bits can be set maybe? Maybe that can only be decided when more such runs are submitted.
EDIT: And here it is:
http://tasvideos.org/userfiles/info/34501518718338775
Please replace the movie file with this one. As stated it only sets 0x523 and 0x5FB to nonzero values. I also saved a few frames along the way with another death abuse, so that's good too.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
@True:
http://tasvideos.org/userfiles/info/34498057663840024
Here is a new NG run that syncs on 1.11.8 up to the first ladder in level 2. 1.11.8 should have a start up state that is very close to a real console, given that it is anchored by your previous Battletoads tests and conforms with NESDEV info about VBlank timing. The only variable left SHOULD be the state of the DMC timer at power on, which I don't have any info on but I should be able to infer from where the run (hopefully eventually) desyncs on console.
@fsvgm777:
I tried the game on FCEUX but I couldn't get the level select feature to work in the debug screen. It worked just fine in BizHawk so not sure what's happening. Do you have a savestate or movie file for FCEUX that gets to the title screen in new game plus mode?
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Derakon wrote:
Having played this game before, I had no idea it had music! How bizarre. The version I'm used to just plays a droning sound wave. If I understand correctly, music is only enabled if an uninitialized value in memory happens to be set to 1?
Well you can enable music manually by pausing with Start and then toggling between music and sound effects with A. But yes the game never writes to the address before reading from it to chekc what mode it's in, so if you didn't know you can change the setting you'll be stuck with the droning sound (if that's what your console favors.)
I just tried to initialize only the 2 variables I initially mentioned and set everything else to 0, but the run then de-synced and I don't know why. So, looks like there are other uninitialized factors at play here that I wasn't aware of.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
What are the actual values written in this run?
The initializing values are:
FF,FF,FF,02,FF,FF,FF,FF,FF,FF,FF,01,FF,FF,FF,FF
Repeated across all of RAM.
The other bits at the actual addresses at issue, 0x05FB and 0x0523 don't matter since the game explicity ANDs them with 1 and 3, so this run is basically setting 3 bits.
Now if you say I should be using 00 instead of FF I can certainly do that too, or even set all of RAM to 00 except the 2 addresses I explicitly need to, maybe that would be less controversial.
Right, I remember discussing this with adelikat on IRC some time ago, when he was about to add this feature, and his opinion (and mine too) is that such runs shouldn't be obsoleting the ones that don't manipulate startup RAM. Because it won't be a fair competition, after more than a whole decade of not using this in TASes.
Why would it not obsolete the existing run? Start up RAM is already manipulated with the default RAM setting, it's just that that manipulation was consistent until now. To me it seems like just adding another variable to optimize (while still being physically reasonable of course, as True points out.)
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Looking into it it seems like an IRQ timing issue. I was supposed to look into this with Burai Fighter too but never got around to it, looks like I'll have to move it up on the priority list though if more games are effected.
This severe graphical bug in Recca's title screen after you beat the game (or use a debug/stage select feature to start from e.g. Area 1) still persists:
Still? Was this brought up somewhere before? Are there other bugs you are aware of?
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
No worries take your time :)
Also I have made some progress on reconciling start up state as tested by blargg and others with how start up happens in emulator.
When I corrected start up timing to match NESDEV wiki, (first VBlank ~ 27384 cycles after power ) and sync the warps run it turns out that the game end glitch run also changes. Instead of looking more like the FCEUX run, it almost completely matches what you posted in your console verification. This is good because I was a bit puzzled by why the two results were so different.
The only problem is that it now fails to end the game. So it seems I still have a bit more research to do. Even a single PPU clock changes results quite a bit, so if I can match the console verification test it will be really strong evidence that start up state is correct.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Huh? I must be misunderstanding what you said in your previous post then. If the NG test run hit the enemy, doesn't that mean you are already accounting for the DMC glitch?
My understanding of the dump script is that it pulls inputs from the movie file and uses them if there is no lag frame. But it's not the movie file that contains the glitched reads, they only show up on playback. So shouldn't it not matter what the source of the input file is, whether it be FCEUX or BizHawk?
Well either way, here is an NG test run, for me Ryu jumps onto a wall early in level 1 and just kind of sits there. I'll be interested to see what happens on console.
http://tasvideos.org/userfiles/info/34390976874363545
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Aww so the test rom didn't work after all, too bad, I had high hopes for that one. Thanks for trying anyway. Maybe I will try porting it if it seems simple enough.
Hmmm so the bot was working around the glitch after all. Can you try the published run again now and see if it still syncs? My guess is that it won't but it would be really really helpful to know where it desyncs and if it is consistent. I think we're making progress here!
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
http://tasvideos.org/userfiles/info/34305774732879677
Here is a world 1 run of SMB3 on PRG1, it cmes directly from the current warpless run (which syncs on PRG1 up to the boss.)
Here are the frames in the first 3 levels that NESHawk says a DMC conflict occurs on:
623 , 1000 , 2197 , 2256 , 2327 , 2411 , 2736 , 2747 , 2748 , 2779 , 2791 , 2827 , 3583 , 4345
So if you play this back purely per poll, it should desync very quickly if any hits are different.
And here is another interesting test to try:
http://tasvideos.org/userfiles/info/34305974613284573
This is a ninja Gaiden movie that does nothing but alternater pressing left and right (after moving to roughly the center of the screen. )
On NESHawk this ultimately moves to the right and hits the first enemy at 111 on the timer. It would be very interesting to see what the behaviour is on console.
EDIT: True, I think I know another way to test to DMC behaviour that would be really helpful. The test ROM read_errors_fast prints on screen when there are DMC hits.:
https://github.com/christopherpow/nes-test-roms/tree/master/read_joy3
The problem is that it is inconsistent when using things like everdrive because it doesn't sync to the timer in any way (as demonstrated by feos a couple pages back.) But if it can be dumped to real cart, I think it should have consistent behaviour from power on. Are you able to do this?
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Wow, that's annoying. Any way to fix this, or to add an onlatch and onclock event to lua similar to LSNES? The amount of reads isn't the only important piece of information - knowing when the data is actually latched is very (and more) important.
I have no idea. That's more of an adelikat or zeromus level question and not really an Alyosha level question. 8D
Ok I'll see what I can put together with prg1. Hopefully by tomorrow I will have something worth testing.
Wow great Battletoads synced! Time to make some Battletoads runs on NESHawk to get some console verifications! Thanks for your testing efforts!
I don't really know what the standards are for adding the console verified tag. Technically the FCEUX run is wrong and obtains the correct result in part from blind luck. But on the other hand all the actual input frames are indeed the same, so it does contain the correct stream of inputs, although every actual frame is not the same.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Woah it actually synced? Awesome!
Ok let's try the whole run then:
http://tasvideos.org/userfiles/info/34284092070056771
I guess it's no longer the same run since inputs are changed, but it beats the game with both players, so that seems like a pretty good test of accuracy to me.
I think the times it doesn't sync are probably due to different PPU-CPU alignment, which is known to be pretty much random at poweron and can influence timing. This would make sense with how random the desyncs seem to be.
With this in mind it seems like the game end glitch run should sync, maybe try it a few more times to see if you get good luck? Here is a BizHawk file just in case for comparison, but input frames should match:
http://tasvideos.org/userfiles/info/34284149904082436
I'll try SMB 3 next. The last time I tried all stages it desynced the same way as on console in world 2, so I'm a bit optimistic about that one. But going per poll would be interesting.
1) Does event.oninputpoll poll for input any time latch happens? Or just once per frame?
I'll look into it. EDIT: It's every poll. More specifically, it's called everytime a button is read, so for Ninja Gaiden for example it is 16 times per frame, 17 if there is a DMC hit.
2) Will it be possible to use/read/get subframe input in the future? Not important for these runs, but already shown to be important for some ACE runs that have no emulator with working native support.
Possible? Sure easily. I could hack it in there now as an alternate input method. But it would basically be equivalent to the scripting method used to do the SMB3 run. Natively supported as in working at the same level as input does now, with rerecording and TASStudio and such? I don't think this will happen any time soon.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Wow actual progress alright! Of course presumably many RNG values get the enemy going in the desired direction and there is no way to know if we are really matching console, but for now I'll take what I can get!
Well then might as well be bold and try to beat that level 8 boss.
http://tasvideos.org/userfiles/info/34263027362025815
Here is a movie file that does so. If a miracle happens and it syncs through the boss, it should desync at the first checkpoint in level 8. If it doesn't sync, can you maybe get a video clip of the boss fight? It would be good to know what attacks the toads are doing. (Or even just a description would be helpful.)
ALso, my mistake on the game end glitch run. It did require some adjustment to work in BizHawk, just not in terms of input frames. It did need some lag adjustment though. But since the input frames were the same, I'm a bit surprised it desynced on console at that enemy since it syncs there in both emulators. Oh well, I guess there is a bit more to figure out here.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Battletoads initializaes all RAM to 00 except for FD and FE. It does this almost immediately after the first VBlank it sees.
FD checks whether the intro has been played through once. If it has, then you can skip it if you hit reset. (The value it uses to check this is 0x28)
FE is not used in any way as far as I can tell.
RNG is initialized with a set seed.
As far as I can tell, the only thing resetting (before the intro plays) does is possibly change the even/odd alignment of the DMA unit relative to code execution, which is known to effect timing enough to change the enemy direction in level 3. Initializing RAM in any way has no effect.
Your present tests seem to indicate that the original alignment was correct. Actually I was scrounging around NESDEV and found some tests that also indicate it was correct before. So I think I'll revert that change.
My only guess is that in your previous tests you happened to do a reset at a point where it would change the DMA alignment, or that on startup there is a small chance of it being changed (since one of the above synced to snakes.)
I'll make a new run with the enemy in level 3 going to the right and see how far it can go. ACtually, it turns out you can get the enemies going back to the left simply by putting one blank frame before the second 'Start' press right at the beginning of the game. So I'll do that and see if it works on console too.
EDIT: http://tasvideos.org/userfiles/info/34230566350048461
Ok here it is. (Don't forget to update your BizHawk build.) This one should get the enemy going back to the left on console. If it does, then I think we are making progress and I'll get the run back up to level 8 or maybe try the new game end glitch run.
EDIT 2: just checked and the newest game end glitch urn syncs on BizHawk without modification, so might want to check that one too since it's short.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I found some minor improvements while working on 100%, will post an updated movie file later. So please set this to delayed for now.
http://tasvideos.org/userfiles/info/34236752933750740
please replace the movie file with this one. Thank You.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
I dumped the FCEUX inputs and compared, and yup they are identical up to the boss.
Well, I'm stumped. 8D
Any ideas? Maybe the run just doesn't sync as reliably as I had hoped.
Any suggestions for other easier / shorter runs to try?
I feel like I'm really close but missing a piece of the puzzle.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
desyncs on console in Turbo Tunnels? That is where it was desyncing in emulator before I swapped the parity of SPRDMA.
That run contains the exact same input frames as the FCEUX version up to the level 8 boss. The console verification page says the run makes it up that far 5/5 times.
Something must be off here. Can you try the FCEUX version and see what happens to that one? Or compare the input log to see if there are discrepencies?
That is a very strange result.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Well, sure it's possible. But I don't know enough of the other parts of BizHawk to do it, and am busy with other things. So I'll add it to the list and save it for later.
(By the way, I tried that chinese translation of TMNT 3 but couldn't make it work. I tried the obvious extension to CHR ROM size but nothing happened, so the game must be expecting something else for CHR mapping but I don't know what.)
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Nice improvement! This looks much smoother then the previous version (in the parts I watched marked as big improvement.) I still didn't see any corner boosting but admittedly it would only save a few frames over an 1.5 hour movie.
I'm also wonder about the big speed boost you can get while jumping off an enemy while going down hill (as in the published run), was there a reason it wasn't possible in 1-1?
Anyway it seems much better played then before voting yes.
Finally, what is a 'Dolphin Frame'?
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
By adjusting for lag, I mean that if there is a lag frame in BizHawk that is not present in FCEUX, I add in a frame at that point in BizHawk, this isn't necessarily a blank frame, it could have button presses in it. The idea is that every valid input frame in FCEUX will also be an input frame in BizHawk, even there are lag differences. I don't know how the RNG works, but it could be that it only updates on input frames.
I don't know enough about new ppu vvs old ppu to comment on why that Mario run desyncs though.
@True: Here is a Battletoads run (2-p warps) that syncs on the latest BizHawk build through the level 8 boss. I think it's possible that it will sync on console where FCEUX fails. It is identical in terms of input up that boss.
http://tasvideos.org/userfiles/info/34147851839238345
Also, Battletoads does some initializing with this code:
LDA $#0
LDX $#17
STA $4017,X
DEX
.
.repeat until X=0
.
BizHawk and FCEUX report this as an input frame (frame 3) since it is a write to $4016 (also a read from $4016 since it's indexed and the CPU reads before it writes.) But since a 1 was never written, no controller data is actually latched. Does the bot see this as an input frame as well?
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
@True: The Kirby run has a reset on frame 6, how does the bot deal with that? The only difference between the builds is how many lag frames there are at startup I believe. Kirby also used the DCM channel but seems to be more careful then NG about how it uses it.
I'm stumped on Battletoads, need some new info before I can figure out how to fix that one.
EDIT: Ok, so I finally finally finally figured out Battltoads. It turns out this was breaking on SPRDMA timing. It takes either 513 or 514 cycles to complete an even or odd DMA depending starting cycle..
As it turns out, I had my check backwards. Normally this wouldn't matter since you might expect each DMA to roughly randomly take place on an odd or even cycle. However, it turns out most of Battletoads' DMAs were taking place on the same type, which happened to be 514, when this should have been 513. The accumulation of extra cycles eventually added a lag frame at the menus which changes RNG and causes the desync at level 3. Setting the DMA timing correctly gives the right result and the run syncs through level 3.
As a bonus, this fix makes the mysterious behaviour in the second test of scanline/scanline appear correctly. So that's a 2 for 1.
Editor, Experienced Forum User, Published Author, Expert player
(3602)
Joined: 11/30/2014
Posts: 2749
Location: US
Ok, I've got a lot of new info, hopefully this will help make some progress.
Let's start with monopoly since it's the easiest. Monopoly appears to rely on uninitialized RAM. I ran acmlm's run on NESHawk with default RAM settings (the usual FF-00 pattern) and the run does sync. Then I ran it with all start up values set to AE and it desynced after the menus. I haven't tracked down which addresses exactly are at play, but that's what's going on with that one.
I'll try Journey to Silius next.
EDIT: nevermind all that other stuff that was here before about battletoads, turns out my movie file was corrupted somehow. Back to the drawing board
EDIT2: Silius syncs after adjusting for lag, no change of input needed. Will check Ninja Turtles next
EDIT 3: Ninja Turtles does not get the ninja star drop in the Dam level
