Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I'll test the Ninja Gaiden run the next time I have some time, probably in a week, or maybe throughout the week if I have an early day at work. (
I have a TKROM board, so if you have either a romhack or a test ROM that will work on TKROM/TLROM/TSROM, and an inputfile with test input, I can check what you need.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
This is not how current policies here are formed. Besides, the memory set on the run before, as arbitrarily specified, is statistically impossible. Because it is just statistics, and not hard impossibility, if such reasoning was followed, then are we truly TASing the game then as it is expected to be run, or just some imaginary arbitrary setup of it?
Yes, it is possible. But here's some differences from simple dirty save RAM:
- In dirty save RAM, the same game that is being played is also used in verification. In this case, completely different code is run. The code created is arbitrary and created by the user or verifier.
- As the code is different, this implies carts have been swapped. Are cart swaps now allowed? If so, should the emulator support this?
- Such a ROM can do far more than set memory. It's rather arbitrary to draw the line at just setting memory. Do we only allow certain instruction sequences to accommodate for an emulator's settings?
- If our goal is to TAS games, and not specific emulations of games, then why would the actual game on console as originally released come second to the game emulated?
This doesn't make sense, based on everything else written; can you clarify? Because for what we've written, on a console, ROM switching would be required to match an unlikely starting state arbitrarily specified in the emulator.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Every one of these examples may not be achievable with stock hardware, true. But some are. Even if they were not, every one of these examples uses a user-accessible interface. Working RAM in an NES has no such interface for the end user.
This is a thread about an NES movie. No "teensy metal pointer" is needed for arbitrary human input on this console. Let's keep the discussion with NES in mind.
First, yes, this is an arbitrary change.
Second, no, we can't put in a "setter cart" unless we use a toploader, or unless we modify our console. Front-loading NES has 10NES which will prevent a cart swap, as doing so will put the console into 1Hz reset.
What you are proposing would either require modifying the game or running arbitrary code before the game. In this case, shouldn't we TAS the modified game? Or, should the TAS instead start with the setter cart and do a swap? And wouldn't _either of these situations_ then be against site rules?
Runs that sync on console are already not allowed to be submitted here if they don't also run in an emulator. If you really want to go the setter cart route, then the emulator must support this as well. A movie utilizing this technique should consist of running multiple ROMs and allow cart swaps within the movie file.
Simply setting arbitrary RAM is not the same as your setter cart idea. I can't arbitrary set RAM on an NES unless I run my own code. The setter cart, as I have used to sync some NES games, is a workaround to flaws in emulation, such as FCEUX's completely invalid startup memory state.
The controller is using a standard protocol. In the case of NES, there isn't much to it. This discussion is about NES.
The cartridge slot, on the other hand, is a direct connection to the CPU and PPU buses, and to 10NES in the front-loader. This usually isn't user-accessible during play. One puts in a cart, powers the console, and keeps the cart in during the entire play session. 100% of NES runs submitted to this site make this assumption. Also, this slot isn't hot-swap safe. I've made it work on a toploader by holding the console in reset during the swap, but even then it isn't reliable.
This is exactly what I was arguing. The only sane choice would be running against an "ideal" NES, with case-by-case exceptions. It is these exceptions that need to be worked out and have not yet been.
An "ideal" NES will have no bits in working SRAM set at power on.
Why should this allowed, but dirty save RAM is not?
If we're talking of a memory setter cart, why stop at just setting memory? That seems awfully arbitrary?
Most movies published on TASVideos are runs against commercial games. If the game was not commercial, it typically must be notable or "entertaining" to be published.
If we went with the setter cart idea, would the setter cart ROM need to be entertaining or notable? Would TASing commercial titles requiring a homebrew memory setter, running arbitrary code not related to the title being run, seem fair? That's what this setter cart thing is getting to.
I'm OK with accepting input files that sync on console to the site. If I make a thread discussing that and others agree, does that make it submittable?
Some of the technical information in the thread you linked does not apply to NES, such as information about DRAM or small process node SRAM startup and decay states. Some of this incorrect information was posted by a user who shares my view. This shouldn't be a crowd-sourced decision, as it is a very technical one and the correct information should be presented before reaching any consensus or conclusion.
feos, this specific run uses a runner-chosen initialization value that is, in all likelihood, _statistically impossible_ to achieve on console. Where is the line drawn? Perhaps that's the more important question.
Are we moving from tool-assistance (tools are used to create a run) to tool-dependance (tools are required to play the run)?
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
No, not all TASes. This is about NES specifically.
SNES, for example, uses a PSRAM. The contents at power-up are generally random. Specific patterns may be characterizable to a console, but I don't know this for sure, and anyway, each power-up will still be sufficiently random that start-state may be able to be manipulated and we could consider it acceptable.
However, keep in mind that such an input is still not within the realm of what a (super)human can do given a controller and buttons on the console. So what should be acceptable in that case? I don't know...
Back to NES, FCEUX has an alternating pattern involving 0x00 and 0xFF which has been responsible for desyncs during verification. I believe it uses this pattern as a workaround for a game. An accurate emulator would be all 0.
Here's an alternate view: if a game uses uninit RAM to initialize an RNG seed, should we allow bit setting then? If so, which ones? Most NES will have some bits set... but no guarantee as to which ones or how many. And it's possible (and ideal per the technology in use) to have none set. Dumps I have seen usually have the most random bits set near the beginning of the range. Do we only allow it around there? Do we test consoles to see what has been seen "in the wild" and replicate that? We're then back to not TASing a game, but a specific game+console... and again, the ideal console has none set. Do we ignore the ideal then, as we ignore inputs that actually sync on console but don't have an emulator equivalent?
I would argue, to have a solid foundation that closely resembles the ideal console for which we emulate, that NES runs should have cleared RAM at start. This should be the norm for which exceptions are raised, rather than a more likely unrealistic scenario of per-TAS startup RAM configuration. Such a rule would not be dissimilar to dirty save RAM - we don't allow dirty save RAM unless there is an exception (like a setup movie).
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
On console, it syncs as it should, and Ryu gets to the next section. It does not match NESHawk.
Left+Right test confirms I am in the correct mode. I can't rule out the robot as the culprit but it is unlikely.
Streamed on twitch if you want to see how it looks.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Tests are showing that it does seem to be more accurate.
FCEUX startup state seemed to resemble console in most cases I have timed (can't think of any outliers that were definitively shown not to be measurement error), but I don't know about Balloon Fight specifically.
NESHawk is undergoing substantial sync-unstable changes. If your goal is to TAS with it, I would suggest picking a specific version and going with it or wait until it's stable. If you run with 1.11.8, your run will be more likely to be console verifiable than if you ran with an older version.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Most SRAM is typically 0x00 when powered. NES is like this.
The bits that are set usually aren't random - a strongly associated per-console characterization can be performed. Certain consoles will have certain bits that they like to set, and probabilities of being set dependent on temperature, noise on the power supply, and so on.
As such, even if you changed just those three bits to being set (which isn't an unreasonable count), it is still TASing the game on a specific "characterization" of a specific console. Rather than running the spirit of the game, as it would play with the cart, you're running the game as if it were played in one very specific console that just happened to have the correct luck-of-the-draw from the fab. This luck, unlike luck in games, isn't one within our control _at all._ Unless you want it to be in an emulator, but good luck probing your SRAM with a teensy metal pointer in your console. (Is that valid input?)
This is my opinion: I believe is, as a site that puts emulators on a pedestal above actual console input, and as a site that favors perfect play, we really should TAS against an "ideal" (as close to perfect) emulation of a console as we can. Also, the typical goal of an emulator is to emulate a system as perfectly as possible, within resource bounds. This view is confirmed by this site by the obsoletion of emulators with poor emulation quality as new, more ideal emulators have been written.
But I do care, because an NES is biased to having its SRAM completely cleared. An NES with a perfect SRAM chip should, at power-on, have zero bits set high.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Please keep in mind, that as an SRAM, NES memory will have almost all bits cleared at power-on. On real-console tests, those that are set are usually at the beginning of the memory for some reason. Obviously, others can change per Alyosha's comment, but it's rare for more than two bits to be set.
Playing with memory is a sort of meta-gaming, doesn't involve user input, and in general, shouldn't be allowed as a published movie per the goals of this site. But emulators need to improve as well - the default FCEUX memory values are completely inconsistent with what a console will actually have. This is a neat experiment, regardless.
The fact that this movie initializes all of RAM with a pattern, and a real SRAM wouldn't work that way, should be reason enough not to accept. This movie is akin to using a game shark code, that affects all of memory and only before startup, I guess.
Now, should preset / uninit values be allowed? Perhaps, but I would suggest that if so, only those that have actually been observed.
In the case of consoles with PSRAM or DRAM, or with the default state of CPU registers on some architectures, it isn't so clear. But for NES, it is.
What are the actual values written in this run?
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
That's not the problem, the testing and conversion is. I've already tried with another movie you said was converted and it didn't work. I've been working 12-14 hour days and while I am willing to test, I don't have time to play with inputfiles to figure out how to convert them across emulators.
I'll test NG this weekend, or at least probably will; I've been working nearly all waking hours this week.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
The published run is an FCEUX run. Post files you specifically want me to test.
Also, if you want me to test with DPCM glitch, it'll be at earliest this weekend when I write the dumper for that.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I will test SMB3 and Ninja Gaiden after I wake up.
I should be able to run the test on an NROM-128 that I have with some modification - namely the test is using CNROM for some reason but the actual code size is quite small and only exists in the second half. If I duplicate this, it'll probably work.
Edit: Got stuck being busy, couldn't test until now.
* Re: SMB3 I still need to write the dumper. This won't take long but give me until this weekend.
* Re: the test rom, it doesn't work on NROM after all, so the test ROM is probably using features of the mapper. I can't run this right now unless a version is made that runs on NROM, TKROM/TLROM, SLROM, AOROM, UNROM or whatever mapper Rockman 2 uses as those are the boards I have now.
* Re: Ninja Gaiden, with both old and new robots, ninja never inched forward at all. I will need to see how my robot is responding to see if it is a timing issue. Maybe I am clearing my clock interrupt late rather than early, but I think I am clearing it right away so DPCM glitch can trigger it again. .....wait, my mistake! I implemented a DPCM glitch workaround on the new robot (and I think implicitly have on on the old robot), though I don't remember the details - haven't looked at the code in a long while. I imagine it is just shifting when this clock interrupt re-enable happens. I just disabled this mode and it synced with the enemy hitting at 111 on the timer as in the movie.
What this means of course is many NES runs syncing (including Ninja Gaiden) have a caveat that the controller is acting like a very slow to respond register (compared to normal CD4021) in order to work around emulator deficiencies. If emulation is accurate enough, NESHawk verifications should trump FCEUX verifications.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I will add this tag then.
As long as the inputs result in the same outcome, I don't think we're all that caring of how it gets there right now. Though of course the prime goal (at least mine) is better emulation, and through better emulation, preservation of history.
small edit: You can see these sync at http://twitch.tv/is_true; look under previous broadcasts. Not too important so they'll be gone whenever they time out, but if you are interested in seeing it and didn't see it live, it's there for now.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I think the run is on PRG0. If you can do this against PRG1 that would be preferred as I only have the PRG1 version. If not I will try to come up with a donor. Also, it got farther than world 2 IIRC - I remember desyncing in water just after coming out of a pipe, but can't remember the level. The world 2 thing was using PRG1 vs PRG0 on an emulator.
Wow, that's annoying. Any way to fix this, or to add an onlatch and onclock event to lua similar to LSNES? The amount of reads isn't the only important piece of information - knowing when the data is actually latched is very (and more) important.
Yes, I am talking about native support to decouple input frames from video frames. They are two different things but opinions expressed before (from memory, I could be wrong) showed no interest / care in this fact.
----
2P Warps: Synced no problem.
Glitched: Took several attempts before the life stealing enemy cooperated. After that, it synced. If these are the same inputs that would be made from FCEUX then this run can be marked as console verified.
Re: SMB3, I guess I'll need to write a dumpscript that counts polls per frame then repeats the input that many times div 8 until a proper latch/clock event can be implemented once you have something to test.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Toad test syncs successfully. (this is the first time we've gotten past this boss on 2p warps console verification.)
First try it did NOT sync with live stealing enemy desync, so this may be console-biased. But still, it did sync through second time. Good work, keep going for it :)
EDIT: After Battletoads, another good / easy candidate for sync would be SMB3. The long runs do not sync on console. SMB3 is a workable choice however as we know it has DMC hits that affect sync. I could test against your implementation of this by dumping inputs on a per-poll basis. If the console run syncs in this mode, this would be proof of accuracy greater than FCEUX.
1) Does event.oninputpoll poll for input any time latch happens? Or just once per frame?
2) Will it be possible to use/read/get subframe input in the future? Not important for these runs, but already shown to be important for some ACE runs that have no emulator with working native support.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
Thanks for the followup. Your explanation makes sense to me.
It may have synced before because I may have been using the toploader NES instead of the frontloader NES. Perhaps they're acting differently somehow (manufacturing tolerance or something). I wish I wrote down more details of the failures.
Anyway, good news! This is my first NESHawk successful sync. It synced just as shown in the emulator.
I couldn't get the glitch run to sync in NESHawk (removed 1 frame, added up to 2 frames). When running the FCEUX dump, the same problem with the life stealing enemy happens.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
If the inputs are the same, it should sync the same...
Dealing with some data recovery at the moment; let me check this tomorrow. I don't think I did a RAM clear to get this to sync, but I did not try starting from reset.
I'll edit this post with what I find.
EDIT: Having the same desync problem with FCEUX input. I'm not sure I changed anything with my old robot code? But I don't know when I did these tests. Tried every form of reset I could. My last guess is memory init, but I'm assuming NESHawk doesn't use the bad FCEUX init? ....I finally found my other robot, tested, the FCEUX ran desynced at the snake level just before the warp (I think I saw frame ~7900), and trying again...it is desyncing in Turbo Tunnel. So it looks like, maybe, I need to init RAM before I test this? Can you find out if you think this may be required?
I can say, nearly every run desyncs in the same way, with the life stealing enemy at the bottom not going left, but going right.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
re: SMB:
Modern robots are not inconsistent with sync. I think the stated run synced just fine on console (in fact, every Zelda or SMB run I have tried has synced).
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
It doesn't.
Kirby resets as soon as the game inits RAM and the game knows it has booted once. In my case, I power on the console, hold reset then start the bot. It effectively does the same thing in this case.
Good news. Awaiting a testrun / something to test on hardware. (EDIT: you posted it in another post. I'll test when I get home.)
Every write to $4016 will cause a latch, and yes, it will be interpreted as an input by the bot, because the latch line was strobed. Now if I need to pad the input at the beginning to compensate for an additional "controller read", I can do that (and do this as a matter of course if a game doesn't sync immediately; most games sync at default settings). I will keep this in mind while testing your testrun.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
http://tasvideos.org/1920M.html this syncs way farther on console from FCEUX than it seems you got.
Didn't notice Kirby until now; I'll test in a bit.
- EDIT: tested, but it desyncs in level 1?? Doesn't get the UFO glitch...should I be running a newer build than 7727a70?
- EDIT2: Newer build syncs as you described on the emulator
- EDIT3: Console synced very similar to build 7727a70... anything I should continue testing here? I don't have this build anymore though.
I would prefer testing games which have been shown to sync on console already. Also, games without the DCM hit would be nice to check.
Desyncs here on console as well.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
freenode #tasvideos or #tasbot is fine.
In the case of Zelda above, the dumped run matches an uncorrected FM2. Correcting the FM2 does sync. It looks like I messed something up when I was testing, because I implemented skip/inserting blank frames in the script, and setting skip=1 has it syncing now...at least until about frame 3800, where Link hits an enemy... Corrected FM2 does get past this point.
Editor, Experienced Forum User, Published Author, Player
(68)
Joined: 1/18/2008
Posts: 663
I don't have Ninja Gaiden II. It is a TLROM board and I'm not sure I have a donor for that. Actually, I do have a TKROM board, that might work. Give me some time, work has me very busy.
Keep in mind our bots don't play back the exact input stream on NES specifically because of this glitch, and partially because of emulation quality. They play back in a "windowed" mode where an input has a window in which it can be repeated. If we had a hit, the game has error correction, and it isn't processing enough data to affect future frames, then the game has a chance to sync.
Probably, but it isn't programmed to look for this. I would need to change the code, but honestly the DMC hit might be too fast that I might still be stuck in a read routine. TASLink might be the better check for this until I get my new bot done.
Re: input stream, it is exactly the "frame" inputs. The script I wrote will replay the only-valid-inputframes data, ignoring "lag" frames as reported by the emulator. If this is glitchy or buggy in any way, or if the emulation quality is poor, the game won't sync.
Because bird emulator only seems to support single frame input, or at least because I haven't implemented anything for it it if it actually does support it, my script effectively mimicks the window mode of a robot on actual hardware. If the input syncs on an NES, but doesn't in the emulator, the emulation probably has a problem.
Re: register access, bird emulator has poor way of detecting this (as far as I know right now - am I wrong?) compared to say lsnes, which has extensive callbacks for controller polling operations. FCEUX also doesn't have a way to do this, so I just dump input by frame on NES console. This is necessary anyway because of this bug.
----
Testing hardware-validated per-frame input data (dumped from FCEUX runs) against 7727a70 NESHawk, I get the following results:
- Ninja Gaiden doesn't even start. The second start press is eaten or something, so we get the intro graphic after starting the game.
- Legend of Zelda shows promise, as it syncs to Level-3. However Link can't get through the first door to the left, he doesn't go up high enough.
I think there's more at fault than DMC hits with NESHawk. This matches my expectations based on prior experience with NESHawk, so it's possible something is still off in emulation here. I am open to more tests. Perhaps we can discuss on IRC?