Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I dumped the FCEUX inputs and compared, and yup they are identical up to the boss.
Well, I'm stumped. 8D
Any ideas? Maybe the run just doesn't sync as reliably as I had hoped.
Any suggestions for other easier / shorter runs to try?
I feel like I'm really close but missing a piece of the puzzle.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
desyncs on console in Turbo Tunnels? That is where it was desyncing in emulator before I swapped the parity of SPRDMA.
That run contains the exact same input frames as the FCEUX version up to the level 8 boss. The console verification page says the run makes it up that far 5/5 times.
Something must be off here. Can you try the FCEUX version and see what happens to that one? Or compare the input log to see if there are discrepencies?
That is a very strange result.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Well, sure it's possible. But I don't know enough of the other parts of BizHawk to do it, and am busy with other things. So I'll add it to the list and save it for later.
(By the way, I tried that chinese translation of TMNT 3 but couldn't make it work. I tried the obvious extension to CHR ROM size but nothing happened, so the game must be expecting something else for CHR mapping but I don't know what.)
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Nice improvement! This looks much smoother then the previous version (in the parts I watched marked as big improvement.) I still didn't see any corner boosting but admittedly it would only save a few frames over an 1.5 hour movie.
I'm also wonder about the big speed boost you can get while jumping off an enemy while going down hill (as in the published run), was there a reason it wasn't possible in 1-1?
Anyway it seems much better played then before voting yes.
Finally, what is a 'Dolphin Frame'?
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
By adjusting for lag, I mean that if there is a lag frame in BizHawk that is not present in FCEUX, I add in a frame at that point in BizHawk, this isn't necessarily a blank frame, it could have button presses in it. The idea is that every valid input frame in FCEUX will also be an input frame in BizHawk, even there are lag differences. I don't know how the RNG works, but it could be that it only updates on input frames.
I don't know enough about new ppu vvs old ppu to comment on why that Mario run desyncs though.
@True: Here is a Battletoads run (2-p warps) that syncs on the latest BizHawk build through the level 8 boss. I think it's possible that it will sync on console where FCEUX fails. It is identical in terms of input up that boss.
http://tasvideos.org/userfiles/info/34147851839238345
Also, Battletoads does some initializing with this code:
LDA $#0
LDX $#17
STA $4017,X
DEX
.
.repeat until X=0
.
BizHawk and FCEUX report this as an input frame (frame 3) since it is a write to $4016 (also a read from $4016 since it's indexed and the CPU reads before it writes.) But since a 1 was never written, no controller data is actually latched. Does the bot see this as an input frame as well?
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
@True: The Kirby run has a reset on frame 6, how does the bot deal with that? The only difference between the builds is how many lag frames there are at startup I believe. Kirby also used the DCM channel but seems to be more careful then NG about how it uses it.
I'm stumped on Battletoads, need some new info before I can figure out how to fix that one.
EDIT: Ok, so I finally finally finally figured out Battltoads. It turns out this was breaking on SPRDMA timing. It takes either 513 or 514 cycles to complete an even or odd DMA depending starting cycle..
As it turns out, I had my check backwards. Normally this wouldn't matter since you might expect each DMA to roughly randomly take place on an odd or even cycle. However, it turns out most of Battletoads' DMAs were taking place on the same type, which happened to be 514, when this should have been 513. The accumulation of extra cycles eventually added a lag frame at the menus which changes RNG and causes the desync at level 3. Setting the DMA timing correctly gives the right result and the run syncs through level 3.
As a bonus, this fix makes the mysterious behaviour in the second test of scanline/scanline appear correctly. So that's a 2 for 1.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Ok, I've got a lot of new info, hopefully this will help make some progress.
Let's start with monopoly since it's the easiest. Monopoly appears to rely on uninitialized RAM. I ran acmlm's run on NESHawk with default RAM settings (the usual FF-00 pattern) and the run does sync. Then I ran it with all start up values set to AE and it desynced after the menus. I haven't tracked down which addresses exactly are at play, but that's what's going on with that one.
I'll try Journey to Silius next.
EDIT: nevermind all that other stuff that was here before about battletoads, turns out my movie file was corrupted somehow. Back to the drawing board
EDIT2: Silius syncs after adjusting for lag, no change of input needed. Will check Ninja Turtles next
EDIT 3: Ninja Turtles does not get the ninja star drop in the Dam level
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
@ true: http://tasvideos.org/userfiles/info/34108792710373640
Ok I tested the most recent Kirby's Adventure run. I had to wait one extra frame before hitting start on the title screen, but other then that it synced through world 1. I truncated the movie to where it desynced in 2-1, so if it makes it there I guess that is a good start.
I also tested Battletoads, 2 players warps, but it de-syncs at the life stealing enemies mid way through level 3. This one I traced back to the title screen, where there is an extra lag frame in BizHawk not present in FCUEX which seems to throw off RNG, changing where the enemies in level 3 go. This seems to come from OAM DMA, which is too short in FCEUX, and the extra cycles it saves over the first 1000 or so frames convert into a saved lag(loading) frame.
Let me know if there are any other games you think it would be useful to test.
I also realized that Ninja Gaiden is effected by the initial conditions of the DMC channel clock and bit counter. The game does nothing special to initialize them, and I had previously only guessed at initial conditions. Since these are constantly going at all times, there can be radically different sync results depending on the iniital state there. But after some tinkering the furthest I got was mid way through the game. another variable to look at though.
EDIT: i also set BizHawk start up back to matching FCEUX again, so now there is no need to add or subtract any frames from an fm2 converted to bk2, this should make testing more convenient and remove a potential source of error. I tested Zelda with this build at least up to getting the step ladder and it synced just fine.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
^ oh , good timing.
DMC is tested to grueling levels with blargg's test suite, so if something is still wrong I don't know what it is.
The DMC re-read happens in 3 CPU cycles, so maybe it's possible it's not getting picked up by the bot?
Yeah if you are available to do something on IRC let me know where to go, because I do have several things I need clarified before I can make any real headway.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
@true: Thanks for your efforts so far. That the run syncs at all is baffling. FCEUX doesn't emulate DMC conflicts , so a run made on it for a game with seemingly common ones and weak error correction seems to have little chance of syncing , yet it somehow makes it. Are you able to check ninja gaiden 2? That would be another good test case .
Also I have a suggestion if you have the time . DMC conflicts seem to be pretty common , but would be undetectable if it happened on a frame where ryu is just running left or right with no other inputs. So would it be possible to make the bot stop replaying the file at the first DMC hit? I am very curious where it happens and would be very useful in debugging.
And if there are none at all, well then it's back to the drawing board since I would have no idea what's happening.
EDIT:
for reference, here are all the frames that BizHawk says a DMC conflict occurs on up to desync in level 2. There are about 50, but only 1 happens in a fatal place. So , about 1% of frames have a conflict on them, more or less.
90 , 305 , 497 , 696 , 714 , 733 , 736 , 754 , 767 , 952 , 989
1075 , 1167 , 1267 , 1282 , 1551 , 1606 , 1645 , 1799 , 1816
1983 , 2039 , 2141 , 2190 , 2208 , 2243 , 2252 , 2291 , 2300
2304 , 2352 , 2353 , 2449 , 2457 , 2462 , 3265 , 3336 , 3371
3462 , 3492 , 3563 , 3683 , 3785 , 3804 , 3996 , 4031 , 4139
4189 , 4200 , 4219 , 4265
EDIT2:
true wrote:
I'll write a script to verify the fceux-dumped input stream against bizhawk, and dump the desyncing run from bizhawk and verify against console. I'm not sure what will happen for the former, but I am assuming the run will not (de)sync in the same manner for the latter.
Can you go into more detail on this and how you do this? How is the input stream different from the per-frame inputs? Are you just cataloging what the emulator says $4016 gives at each poll?
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Most likely that is referring to how NesHawk handled illegal opcodes in previous versions. Many were poorly emulated, and at least one that should have crashed the system was emulated as basically a NOP and let code execution continue.
So as described in that post, this might allow things on emulator not possible on console, since console would crash but emulator wouldn't.
Any of those such problems should be cleared up by now though, all opcodes are emulated correctly, and NESHawk is much more accurate in basically every respect.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
@sergunov: Oh cool thanks, yup that is same result BizHawk gives. Also the third party controller hangs becuase it returns zeroes instead of 1's after the 8th read (this is what they say on the wiki anyway.)
@Potato Stomper: That's interesting. I will try to run the currently published any%run and see if it works when I have a bit more time. Maybe we can clear that up, thanks for pointing it out.
EDIT: Quickly tested it and it worked just fine. Ended the game in the exact same way. I should also point out that the current 100% run desyncs in the exact same way on console as on BizHawk, so I think it's quite accurate for SMB3 at this point.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
That was amazing! I really liked the Bowser in the Fire Sea fight.
Hard work, endless determination, and a bit of genius sure does make some great things happen!
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Dolphin has matured a lot since Soig's NSMB Wii run was published a year ago. Have you thought about trying to improve that one? I'm sure you can learn a lot from it and maybe save some frames with a more stable Dolphin.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I removed the extra DMC read and both the new and old Ninja Gaiden runs sync just fine on NESHawk without it (after removing one frame at the beginning as before), so that is definitely the only problem. So I guess we just need to wait for results from True's console testing to see what happens.
Also I'm still making progress on mappers, so hopefully soon the only left to tackle will be the VS stuff.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Looking good Tompa! Back to Croctopus Chase now!
I don't think there is really that much more to do there honestly, my last WIP was already pretty close to breaking.
I can definitely help if you want me to look at anything. If you plan on starting from scratch don't forget to turn on the radar, it's not perfect but way better then TASing blind.
Also I'm about 80% sure that everything in the level has to be loaded in order, and things break when they aren't, so this might be another to check for things, if you can identify all the objects that is.
Anyway, good luck! This is the last major hurdle so don't give up now!
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
@Scumtron: Short Answer: Yes, button mashing can resolve the conflict.
Long Answer: What's happening here is the DMC unit is changing the address from the controller port to wherever it's reading the DMC byte from. Once it's done, it puts the address back to the controller port. An unfortunate side effect of this is that this causes the controller port to see this as an extra read, and pushes the next key press onto the shift register.
Normally the port is polled 8 times for the 8 keys, but this extra poll from the DMC means one button is lost. The last button read is 'right' and the behaviour is that any reads after the 8th returns '1'='the key is pressed' so normally in this situation you see an extra right press and one lost key. Ninja Gaiden and other DMC using games compensate for this by polling the entire controller twice. What games do after this differs though. Some games, like SMB3, if they detect a conflict will do the whole routine again, and repeat until there is no conflict (which is where the possibliity of the 2 second SMB3 run from SGDQ comes from.)
Other games though, like Ninja Gaiden here, if they detect a conflict, just give up and fall back to the previous frame's input. This is fine for casual play where you mostly wouldn't notice, but under TAS conditions this is obviously fatal.
Mashing buttons simply makes the game think that both reads match (by hitting all the buttons after the extra poll took place, it looks like nothing is wrong.)
@feos: Something that would immediately help with determining if BizHawk's implementation of this is correct is running dmc_dma_during_read4/dma_4016_read with different buttons held down on the controller. Of particular interest would be the results from holding down A,B, A+B. This would at least confirm that the basics are all correct.
Beyond this I'm not sure what besides hardware testing can shed any more light on this. I tried adding and subtracting a few CPU or PPU cycles at startup see if I can get different sync behaviour. At best I can make it to about half way through level 3, but it seems inevitable that there will be a DMC hit somewhere. I'm not sure how the original run survived, could be a result of CPU - PPU alignment but I don't really know.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Due to different power up behaviour, adding frames is no longer the way to sync FCEUX runs. The correct thing to do is remove 1 frame.
I did so and the run syncs correctly through the first level. It then desyncs at the ladder in level 2. Currently looking into why it happens, looks like it misses a 'right' press.
Will edit this post if I get any more info.
EDIT: nevermind problem is already obvious, it's a DMC hit. Hmmm, this is troublesome, it's certainly possible that this same hit would happen on console, but it's also possible it wouldn't. It would be nice if there were a way to add a method into the emulator to avoid this probabalistic behaviour but at the same time this is cycle accurate and correct, so fatal hits like this just come up now and then.
Anyone have any thoughts?
Also, I would be really really interested to see what happens on console. If code execution is tied closely enough Vblank, this hit might be deterministic, or at least happen with a high probability.
EDIT2: the old run desyncs in the exact same place, but that one is known to sync on console. We need more information, such as if the old run always syncs or only a certain percentage of the time.
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
Well I do have some suggestions maybe you can consider.
1. The published run of NSMB Wii uses a corner boosting technique extensively. You can probably save some time here and there with this technique.
2. It looks like if you jump off a goomba while going down a slope you get a pretty good speed boost, see the published run at 1:51 to see what I mean. It looks like you can do the same thing here in level 1-1 right before the pipe to get in it quicker.
3. It looks like you don't need to get the fire flower in 1-1? You trade it off later for the helicopter and it doesn't look like it saves any time between the two points. Maybe it's for showing off or I'm missing something.
4. In 1-7 it looks like you lose penguin slide speed after a while, seems like it can be maintained a bit better, although I didn't test.
You don't want to keep getting rejected runs, then you will end up with 'limited user' status and not be able to submit at all. D:
Editor, Experienced Forum User, Published Author, Expert player
(3536)
Joined: 11/30/2014
Posts: 2733
Location: US
I still like this hack and this run, and am still voting yes, but I think it would be better served if you take more time to really go over each level and really optimize each bit of it. A TAS of this length would really take at least a month to do , and I bet you can save much more time by really studying the game.
I do want to see this hack published, but it's going to take a seriously optimized and thought out run to get it there, hacks are generally given pretty harsh scrutiny. You can create a thread for this game over in the forums : http://tasvideos.org/forum/viewforum.php?f=40 EDIT: there is already a thread for this games hacks here: http://tasvideos.org/forum/viewtopic.php?t=14861 as was kindly pointed out to me 8D
You can also post videos and input files of your progress there for feedback. In the mean time I might recommend cancelling this run, as Spikestuff points out some of your new times are slower then your old ones, which doesn't bode well.
Still a good start for a first TAS though, keep improving!
