Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Instead of watching the encode, do you want to watch a live TAS attempt?
I will be running this on console at about 7pm PST today - in a little less than an hour and a half. It probably won't sync - the last warpless desynced in world 8 - but I will try...
Edit: The run went well. It desynced after 2-1, just as PRG1 did on emulator. This is both a good thing and a bad thing. I need SMB3 PRG0 or a donor to continue, but it may verify yet. I'll see what I can do.
Edit 2: I ran PRG0 SMB3 on console. It desycned as PRG1 did on console and emulator. This might (or might not?) be a helpful test in improving emulation.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I tried verifying this, but like previous MM4 runs, it doesn't sync. We can get to the first boss, but the patterns don't match. Different results were observed with P2 plugged in and unplugged. Emulation probably needs work...
This run also acts wildly different on QuickNes and NesHawk cores. NesHawk is nothing like console, whereas QuickNes acts like console when it is off by a frame. If I remove a leading frame, it only kills the first enemy - another problem I had on console. Still isn't acting like console though...
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Does the multitap actually have a switch on it for control? I don't remember this, maybe my memory is flawed...
If so, then yes, it shouldn't be an impossible change, and I will need to update my playback device to have any hope of replaying an SoM movie.
If not, is it considered valid user input to unplug and replug controllers? Particularly, in mtap case, for fastest playback this would probably imply unplugging mtap and instantly plugging in a controller. I'm not sure that is a valid input case, at least as far as TAS is concerned. (Are we TASing the game, or the emulator? What I mean is, if an instant swap could not be reproduced, then would it be valid? I would say no...)
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Zowayix wrote:
dwangoAC wrote:
for the Pokemon Plays Twitch run we used the first bit of two bytes to signal whether what followed was 3 5-bit bytes or 2 7-bit bytes with an extra flag left over for signaling things RED said. I can dig up the details but long story short it gave us one extra character to play with, which was very helpful.
Things like this are why I was so disappointed that the AGDQ 2015 TASBot runs never became actual submissions here - details like this are what I'm really interested in. Even for the SMB in SMW run it looked like Masterjun used some glitches that have never been seen before in any other SMW submission. For example it doesn't look like Yoshi eats anything in the last frame before the ACE begins - what triggers everything then?
I ran / tested a lot of the previous exploits but had no involvement in this one so even I don't know what is going on...and I've brought it up since I wanted to test a faster console-viable run (the current ace to credits run does not sync on console), but got no response other than that it's slightly slower...
So you're not the only one thinking this. However there doesn't seem to be any momentum to accept any non-fastest run unless it has some hint of nostalgia attached. Frankly the runs are too short to induce the nostalgia judge trigger. I was going to make a tierless run aggregation list that looked at runs submitted here and elsewhere - is this something I should finally find time for?
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
If it does gameover at 0, that's interesting, since if it is actually set to -1 and not 600 at timer start, that would mean that you should also gameover at 0xffffffff - 600. But still probably useless.
What I mean by check on read is to break when the variable is read, not written. How to do it in your emulator -I don't know. If there is a RMW operation that has a decremented result then ignore that one and see where else it is read. The results of these discoveries, if relevant, should really be what is discussed here. Not to discourage learning, but there's 29 pages already, do we need more pages of someone's learning efforts rather than discussions of useful information?
If dealing with RAM search you will want to learn about variable types, signed vs unsigned, ... for this game you probably don't need to deal with BCD, or anything strange...
I understand documenting observations but if we all documented everything we observed then we'd be busy observing the pile of observations for buried useful information...
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
filwac wrote:
Escape Timer: 09D268 (4 bytes, unsigned)
Just saw that the escape timer starts at 4294967295 which is like 136 years.
Well maybe because said timer isn't unsigned, but signed...
Didn't 0xFFFFFFFF look a bit suspicious to you?
Based on your description this likely isn't the counter per se, but rather the offset applied in the counter calculation. The variable was likely declared as an (unsigned) int in the code and most compilers targeting MIPS specify int as 32bit.
Thus the counter starts and doesn't stop as it's just a few instructions in whatever part of the code handles decrementing this. Who cares if it stops? It isn't used anymore past the 10 minute mark.
Perhaps you should check for read, not write of this variable to confirm this if you care. You can also force this variable closer to 0 and I would not be surprised to see nothing happen when it hits 0, since I doubt this variable is checked directly. This game wasn't written like an NES game.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
DiscoRico wrote:
Let me try to explain a little better.
The fact that the console-verified TAS exists means that its inputs are of high-enough quality to be input to the real hardware and execute to completion.
If you take the output from the Input Logger and play it back on an emulator, you can detect inconsistencies between the real hardware and the emulator, or errors in the Input Logger (should there be any).
This isn't helpful in any way though. All it would say is that for some millions of instructions, there is some inconsistency at that exact point.
We can already see these states when doing console verification. If it plays one way, it will almost certainly play in the other unless other bugs are found. (An example: SMB3 is not emulated well enough. Whether recorded on console or via the emulator, the runs will cross-sync. The bones/wrenches are the only thing I have found that does not sync up. Any interaction with these may cause a desync.)
DiscoRico wrote:
It stands to reason that if you:
1) take a known control input file that was "generated" via emulator
2) input it to the console
3) use Input Logger to record that same input
What you are basically saying is "copy the input." The input bot would see what the output bot puts out. It would be 100% identical in all cases, without exception. I can already copy a file on a computer, but this would be a pointless task in this case.
This really makes no sense. Maybe you lack understanding of how input works on NES/SNES? If so then as I've explained, this probably doesn't work the way you think it does. If not then I call BS.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I don't know what you mean by taking a "console-verified TAS" and "resyncing it on the console" - it has already been synced on the console, which is why it is console-verified.
If movies don't sync on NES, it is because emulation quality is poor. The solution is not to compensate inputs, it is to fix emulation.
If movies don't sync on SNES it is likely because the code being executed relies on bus timings that can't be met in hardware without extremely accurate clocks. The solution is not to compensate inputs, it is to inject clocks or otherwise consider the game unsyncable.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
endrift wrote:
And here I was assuming the game was unsyncable due to variance in how the hardware starts and runs. I guess if you modify the clocks you can do it, but at what point does it start being too intrusive into a hardware mod? Perhaps I'm getting to philosophical though.
For verification purpose, clock injection isn't too severe - emulators have precise clock timings and sources and those of the console aren't anywhere close to accurate. The emulator clocks are (close to - see below) the specified frequencies of the oscillators they replace. I am basically accurizing the console to induce determinism. I don't change any characteristics of how it runs or of user input. If I had to, say, snoop the bus to inject or modify instructions, change how controller polling worked by forcing polling to take place, run my own software on the console etc. then that's a bit far, but making the console more accurate isn't.
Obviously I would like NOT to need to need to do this, but for many games on SNES there will be no way around this. The end product of this modification isn't as severe as it seems - the console contains clocked devices and I am simply clocking them at near-nominal specs.
Note: I have concerns over frequencies chosen by byuu. The cpu frequency is not nominal specified frequency, but it is at least well within spec. The chosen frequency is 2hz higher, well within a high specced 10ppm crystal's accuracy tolerance; this selection is only out about ~93ppb out. I can't find a datasheet for this product but his frequency _could_ be the most common standard deviation of accuracy at a specified temperature...though I am guessing not. The APU is even farther out though. On the console it is clocked by a cermamic resonator (at least in rev1 SNES) and is a comparatively inaccurate and drifty clock; that said, it probably won't be the ~1250ppm out that the emulator's clock is. The nominal frequency of the resonator is 24.576MHz but byuu has chosen 24.607104MHz. This is WAY out of spec, and should be fixed in the emulator, but as long as I sync the console to this frequency, game sync should occur.
I would suggest that the clocks by set to main:21477270 apu:24576000 or some similar values with a shorter phase sync period. (unless someone has better information as to why these values were chosen for accuracy?) These values are in line with the actual oscillators used and specified for use in the SNES. Ilari has modified lsnes to support changing APU but I don't know if the main oscillator can be changed with how the emulator is designed, or if it can, if it will have serious performance consequences...
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
There are a few issues right now.
1) I don't know the state of the clockdividers in the CPU/PPU/SPU at power-on. Each chip has its own dividers built in, and I don't know if they power-up reset or if they reset when reset is asserted. Most divider implementations do not reset...
2) I cannot start both clocks at the same time, and I have no control of phase. This can be solved with a better clock generator or an external circuit. This circuit design would depend on how the dividers work and if they will reset if reset is asserted.
Why I desync now? I don't know, but my theory is a combination of: source clock jitter, SPU clock and CPU clock divider offset, initial start phase mismatch. It could also be emulation inaccuracy or emulation board rev mismatch (I have been doing this on a rev1 board) but I need to rule out these other things first. I don't have equipment to test all of them but I can test some things still, like divided clocks to know the reset state of the on-clip clock dividers.
But what's really interesting is that I moved the CPU clock injection from the crystal (where it goes through buffers) to directly at the CPU/PPU with a 10k pulldown (to have a known start state and to possibly dissipate reflections) it had a drastic affect. Before the move, we were deterministically getting to a point in the runs, and after we were getting to a different point. Since phase and dividers don't match, why would this one change do anything? It's baffling. My guess right now is that the pulldown alone is responsible for this - but phase doesn't match at start anyway, and the SPU is clocking before CPU, so why would it matter?
Lots of work left, this is only the beginning. I need time, motivation, and probably money for better tools, in that order. I'll try to get the first two because that's affordable :)
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Just to note here since discussions have primarily been on IRC, I have been working on Super Metroid sync. dwangoAC sent me a broken SNES and I have modified it to use clocks injected from my signal generator.
We're getting close to syncability (can get about ~10 minutes in) but I think my sig gen, though a good unit, has two major issues preventing sync. I also need to confirm how the clock dividers work on the CPU, PPU and SPU and perhaps even try a different SNES board rev.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
arandomgameTASer wrote:
True wrote:
Oh, another one of these overly-talky platformer Mario games.
Sorry, couldn't watch past the first level, but I am certain it would have been assisted suicide.
Wow way to be as rude as possible.
I thought it was a well made run. Once you get past the talking, there's a lot to it. Yes vote.
I said nothing about the run. The game is from the modern Mario era and I didn't realize there was a GBA game with low quality Mario talky effects. The game choice is awful. I have a pretty bad migraine now from just the time it took to get to the first level. Every second it was getting worse.
Really man, your personal hatred against me and anything I say is getting old. If you have an issue with me, take it up with me.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Oh, another one of these overly-talky platformer Mario games.
Sorry, couldn't watch past the first level, but I am certain it would have been assisted suicide.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Actually, I thought of something else this could be useful for.
Right now the fast SMW run does not verify on console. We don't know for sure what is going on but we're probably hitting a register that isn't being emulated properly. Looking at the controller on the LA I see that we're getting past some controller register so the main exploit is working - we're getting stuck some time past that.
Perhaps some work could be done so we can figure out what is going on in this state (see PC, etc) and improve / fix emulation.
This may require a combination of a replay robot and this board. I was originally going to hook up a logic analyzer to A BUS but maybe this could be used instead?
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Yes, it very much is Italian. The author is Italian.
Perhaps you are referring to code comments and some constants in Italian? I am working on translating / clarifying these as well as clarifying small bits of code, but nothing substantial. Even then, the comments are mostly useless. They are used at best as sugar to visually separate things.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
feos wrote:
This emulator that's already the most accurate NES emu known to mankind is now open source.
https://github.com/punesemu/puNES
Now idea about how free it is, but still, it happened! And yeah, cute Spanish commit notes (and some comments).
COPYING shows GPLv2.
Anyone have plans for this yet?
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I played the DOS version as a kid. Never was very good at it; this is a hard game. This version is quite faithful to the DOS version.
Need to get my sick Lynx working somehow...
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Actually, this could be useful potentially for Super Metroid sync if my clock inject method doesn't work, or perhaps in addition to that. Want to discuss on IRC?
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I tried syncing this.
I had to write a dumpscript for bizhawk. That emulator is crap for getting input for these games. There is an input event callback but it provides no useful information, or at least it isn't documented. Even then it is doing it by clockout, not by latch. lsnes is immensely better here.
Luckily it looks like DKC is a standard autopoll or 1-poll game and doesn't do anything weird so I was able to dump input. I've also verified that I seem to have a v1.2 cart so I can't verify this game :(
However it played just like v1.2 so chances are good that this would sync on console. I just need a v1.0 cart to try it.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Wow, thanks for the memories :) I tried building my first robot while basically homeless and dealing with near death experiences regularly. Interesting times. I've learned a lot since then.
There have been discussions for a long time... I think they started more seriously when I was showing progress on my PIC-based NES robot. micro500 was working on his Arduino bot but wasn't really talking about it publicly (or at least here?). He started his robot after I started mine but finished it and got it working properly first.
One common theme after NES Mario was verified were doubts that other more complex NES games would and could not be verified, both due to poor emulation and inconsistent console startup state. The first game I worked on other than Mario was Duck Tales and this was one of those games. But I verified it and continued to verify other supposedly "difficult" games after that.
Later the SNES robots were built. I think GhostSonic was the first one to do this, or if not at least the first one that got 2P support working. It also had very restricted / finicky multitap which was fixed output and not really to specs. Might find this in either the SMW thread or in the Console Verification thread. Not bad for being on a full breadboard. But by this time I was already working on my next generation robot that would support full 8-player multitap and do it (somewhat) within specs, without all the discrete components. This was the robot used for NES / SNES at AGDQ.
Between then and now micro500 made a Propeller-based N64 robot which was shown at AGDQ. notaz has made a working Genesis robot (although the Genesis itself will have issues with sync and it isn't true verification yet) - you can find those posts in the Console Verification thread. endrift made a working GBA GBP robot that was shown at SGDQ. I've made a robot that has very flexible hardware and currently does NES/SNES, has SMS/Genesis in progress, and will also eventually do GB/GBA and N64 - just need time to write code.
A robot currently being made now is micro500's DS bot. I think this is FPGA based with a Prop controlling it. I am considering doing a similar bot but porting it to run on a $4 PSoC prototyping board.
I am in the planning stages of my next generation multi-console robot which will probably use a PSoC5 MCU (has built-in PLDs) and possibly an external CPLD or FPGA. This will finally realize the idea of having a robot perform to the exact spec of an official controller...although in practice it doesn't matter much :) It may also simplify firmware.
