Hypothetical situation, by Zeupar.
You are creating a TAS for NES game X, which has never had a published TAS. You are also working with a person who has an NESbot. Now, your run will console verify if you waste say.... 100 frames throughout the run. However, it is now 100 frames slower than the input file could be, if just played in FCEUX.
Which would you (the TAS community) rather see? The fastest file in FCEUX, or the fastest sync-able TAS on an NES?
I'm on the fence, myself.
(Note, I set up this scenario to specifically not cover a movie that is already published, as that is a whole can of worms I don't want to argue.)
Sage advice from a friend of Jim: So put your tinfoil hat back in the closet, open your eyes to the truth, and realize that the government is in fact causing austismal cancer with it's 9/11 fluoride vaccinations of your water supply.
Emulator Coder, Site Developer, Site Owner, Expert player
(3584)
Joined: 11/3/2004
Posts: 4738
Location: Tennessee
If there weren't so many unknowns I'd lean toward's syncable. But a run that desyncs on a console hardly proves anything at this point. There are so many unkowns, maybe the run COULD sync on the console given the right randomized DMA for instance.
I'm going to go with fastest because of what adelikat said, and the fact that we're just making the fastest run with the tools that we are given, meaning, we shouldn't have to edit our files to make it work for something else.
(I didn't word that well at all, but I hope you understand what I am getting at.)
I don’t know all the technical stuff but… my belief of TASes has always been that everything done in a TAS can be done on the real console, otherwise it would just be cheating
so I’m for syncable
I'm for syncable. It is my understanding that we want to see how fast these games could theoretically be played (entertainment is important too, of course). I realize that randomness in consoles can mess things up in some cases and that our emulators aren't completely accurate which compounds the problem, but if for example a more accurate emulator ends up being laggier I would still TAS using the more accurate emulator if possible *points at his ActRaiser Special Mode TAS*. A TAS made using the more accurate emulator may turn out to still not sync on a real console but if it is closer to what could be done on a real console, great.
I think adelikat is right, and I'll expand.
There are too many variables within a real console, even the room temperature or a slight variation in the voltage supplied can alter the behaviour and the outcome of the game. Not to mention that different revisions of actual console hardware can exist, such as the various Mega Drives / Genesises.
The nesbot itself has been seen to take a few attempts to sync even the runs that ended up to sync, so it's not an exact science.
While I'd love to see many TASes on real consoles as much as you do, I think it's not going to happen especially when luck manipolation takes place. For trivial TASes you can "waste" 10, 100, 1000 frames in an attempt for it to sync, but I still think it's not worth it.
The occasional syncable movie every now and then will still be welcome.
(by the way the poll should have a third, "indifferent"/"neutral" option, in my opinion)
Emulator Coder, Site Developer, Site Owner, Expert player
(3584)
Joined: 11/3/2004
Posts: 4738
Location: Tennessee
Yeah, a lot can go wrong with on the bot's side of things. For instance, my bot's plug into the controller port is somewhat loose and it looses connection from time to time. Based on our current technology a desync doesn't really leave you with any indication that this tas can not be done on real hardware.
I think TASers should be striving to make the fastest run on the best versions of emulators we have available. And devs should the ones worrying about the accuracy of the emulator. I don't like the idea that tasers should worry about what the nesbot is going to say about their TAS. If anything it should be evidence that the coder uses, not the taser.
Looking at this question from the standpoint of right now, if there is a TAS A that is longer but console verified, and there is TAS B that is shorter but lots of bot attempts have failed, we will be publishing TAS B, no question.
However, TAS A is worth noting somewhere, and that even brings up an interesting concept. What if I made a TAS that avoids randomness whenever possible, does everything possible to avoid near deaths and other desync prone situations and beats all known real time speed runs. I would have the console-verified record. And what if someone else manages to make a tas that beats mine? That could be an interesting evolution, one that we have no precedent for displaying on the site.
Soon we will have sda, tasvideos and nesbotvideos:-D
I think having "console-verified records" is a good idea. You dont even have to create different branches. If a faster version doesn't sync then write something about that in the information text of the slower, console-verified movie (in the console-verified section).
How should I know what I think before I read what I post?
Yeah, a lot can go wrong with on the bot's side of things. For instance, my bot's plug into the controller port is somewhat loose and it looses connection from time to time. Based on our current technology a desync doesn't really leave you with any indication that this tas can not be done on real hardware.
This said, there are quite a few runs that desync 100% of the time at a very specific spot, and the runs could be modified to sync (Mega Man 5, Bionic Commando, Little Mermaid, Duck Tales ... I just realized, these are all Capcom titles...) This is more the situation I'm referring to. This isn't for complete lack of sync-age, more just 'close to, but not quite syncing'
This is clearly not a problem with the connection, or the way the NESbot is made, as far as I can tell. It is a problem between the emulator and the console. This is the situation where I'd prefer the run sync on the console.
Also, I was in the process of modifying Bionic Commando's NESbot file to sync. I'm 100% sure with enough time and effort, it could be made to sync.
Sage advice from a friend of Jim: So put your tinfoil hat back in the closet, open your eyes to the truth, and realize that the government is in fact causing austismal cancer with it's 9/11 fluoride vaccinations of your water supply.
I would have to know the reason why it's not syncing on the console before forming an opinion. If it's not syncing because the emulator is not emulating the hardware properly, then I think the emulator should be fixed. If it's not syncing because, somehow, the game is getting some truly non-deterministic input from somewhere that could in theory be proper for the run to sync, then go for the fastest (because the TAS is, after all, representing what could be possible on the console given the right circumstances).
Noob Irdoh wrote:
even the room temperature or a slight variation in the voltage supplied can alter the behaviour and the outcome of the game.
Could you elaborate? Usually computers are by definition deterministic and non-determinism is undesirable and would cause erratic behavior and constant malfunction.
What about games that use uninitialized RAM for random numbers (or other quirks of behaviour)? Is there a tool for nesbot that cleans RAM before starting so it'll be like in the TAS?
I remember being told that TASvideos isn't purely a records site, so since faster run is presumably more entertaining, it's more valuable than "indisputably legitimate record" run. I voted "Fastest".
Patashu wrote:
What about games that use uninitialized RAM for random numbers (or other quirks of behaviour)? Is there a tool for nesbot that cleans RAM before starting so it'll be like in the TAS?
Honestly, this is such a speculation - about using uninitialized RAM.
Do you recall any single example of NES game using uninitialized RAM as a seed for RNG? I don't. And I know why it wouldn't be practical.
Uninitialized RAM has little entropy, thus it is not good source for RNG seed. When console is switched on, its RAM is cleared/set to FF, and even if there are some differences between consoles from different regions, a value in any particular memory cell is not random enough to serve as a seed for RNG. Player's input is much more reliable for such purpose (truly random).
Warp wrote:
Noob Irdoh wrote:
even the room temperature or a slight variation in the voltage supplied can alter the behaviour and the outcome of the game.
Could you elaborate? Usually computers are by definition deterministic and non-determinism is undesirable and would cause erratic behavior and constant malfunction.
Indeed. This room temperature and stuff is another groundless speculation.
As the second person worldwide to have a NESBot built by micro500, I can say with absolute certainty that the technology is far too inaccurate to prove that a run is unsyncable. A run should obsolete another one if it beats the previous run and uses a the same emulator version or a newer one. If the new emulator causes lag / other areas of frame loss, those parts should be accounted for. Emulator coders should continue striving for accuracy to avoid problems like this in the first place, and the NESBot should only be treated as a tool to prove that a run is verifiable, not that one is not.
Joined: 11/22/2004
Posts: 1468
Location: Rotterdam, The Netherlands
Some games simply aren't sync-stable on the console. There's very little you can do in that case.
Console verification should only really be used in extremely rare cases, such as the race condition in Rockman that causes the end-of-level code to be called.
I would think that if a movie doesn't sync on the console, it would be a problem with NESBot (which is still in its early stages), and not the movie. Therefore, I would prefer the "fastest file in FCEUX".
Fastest.
A run not syncing using NESbot may indicate a problem with NESbot as well as with the emulator. Until we have the whole input transmission methodology and technology perfect (or close enough to it), relying on the bot for accuracy is pointless.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Fastest. I would use the situation as a way to press for any known but unimplemented improvements to the emulator, though.
Dwedit wrote:
I'm all for fixing FCEUX to make it match real hardware better, old TASs be damned.
I agree with Dwedit
<offtopic>In light of recent events, however, I might have some fun by putting up an encode of the longer movie on YT, waiting for someone to submit a version with replicated input, and then submitting the shorter version just before that other submission got published (:P). To be fair, I would include some vague mention of possible improvements so that any potential thief may have the opportunity to arrive at the shorter version himself.</offtopic>
Fastest, because otherwise we'd have to delete all our runs that aren't done on a NES. Runs should only be accepted if we have reason to believe that they could be syncable with future improvements to NESbot and similar projects (i.e. don't exploit emulator bugs). That's quite different from syncable with current technology...
Fastest, because otherwise we'd have to delete all our runs that aren't done on a NES. Runs should only be accepted if we have reason to believe that they could be syncable with future improvements to NESbot and similar projects (i.e. don't exploit emulator bugs). That's quite different from syncable with current technology...
This is a total straw-man. I didn't say that we'd only accept syncable runs, just that syncable runs would be preferred.
Also, everyone is saying how inaccurate NESbot is, without really looking at the code or understanding how the thing works. There honestly is not much behind the NESbot, and most desyncs I'm sure are emulator/console inaccuracies.
C'mon, all the thing has to do is get a latch and write the proper controls. It isn't rocket science.
Sage advice from a friend of Jim: So put your tinfoil hat back in the closet, open your eyes to the truth, and realize that the government is in fact causing austismal cancer with it's 9/11 fluoride vaccinations of your water supply.
This is a total straw-man. I didn't say that we'd only accept syncable runs, just that syncable runs would be preferred.
Even so, I think a run that's syncable with future technology (and is faster/more entertaining) is more worthy of acceptance than a run that can be synced.
Also, everyone is saying how inaccurate NESbot is, without really looking at the code or understanding how the thing works. There honestly is not much behind the NESbot, and most desyncs I'm sure are emulator/console inaccuracies.
C'mon, all the thing has to do is get a latch and write the proper controls. It isn't rocket science.
I'm quite aware of what NESbot does. It isn't controlling anything but the controller input. It should be regulating the temperature of the NES, leaving it disconnected for days so that the RAM settles down to a known value, starting the NES with a voltage ramp-up that gets its oscillators synchronized in the right way, etc. It's very hard to do all that on the actual hardware, yet necessary for consistent results.
I need only point out that NESbot sometimes syncs and sometimes desyncs to prove that it's much more inaccurate than most TAS emulators. If it's getting different results from the same input, it's very inaccurate compared to most emulators, simple as that.
I'm quite aware of what NESbot does. It isn't controlling anything but the controller input. It should be regulating the temperature of the NES, leaving it disconnected for days so that the RAM settles down to a known value, starting the NES with a voltage ramp-up that gets its oscillators synchronized in the right way, etc. It's very hard to do all that on the actual hardware, yet necessary for consistent results.
I need only point out that NESbot sometimes syncs and sometimes desyncs to prove that it's much more inaccurate than most TAS emulators. If it's getting different results from the same input, it's very inaccurate compared to most emulators, simple as that.
The temperature thing is crap - if it were true, the NES would act differently every time. I'll agree with the start-up issue, that is why it desyncs when starting so often - I thought about adding a debounce to the NES power button. However, once a game starts to sync, it syncs. (See my major fail at SGDQ).
However, in this list Wiki: ConsoleVerificationTests you'll see a list of games with partial sync. Games in this category clearly desync due to emulator-NES inaccuracies. Mainly because they desync consistently in the same spot. That is the group I'd prefer to see be 'syncable' over fastest. TASers could easily work around the emu inaccuracies by testing.
Sage advice from a friend of Jim: So put your tinfoil hat back in the closet, open your eyes to the truth, and realize that the government is in fact causing austismal cancer with it's 9/11 fluoride vaccinations of your water supply.
However, in this list Wiki: ConsoleVerificationTests you'll see a list of games with partial sync. Games in this category clearly desync due to emulator-NES inaccuracies. Mainly because they desync consistently in the same spot. That is the group I'd prefer to see be 'syncable' over fastest. TASers could easily work around the emu inaccuracies by testing.
It all depends on what games are doing. I can imagine, on various consoles, games using things that are effectively unstabilisable as RNG sources (Pokémon Black/White, for instance, use the difference between two oscillators as one of the ingredients in the seed; I've noticed temperature-dependent behaviour there but it may have been coincidence). In the case of consistent desyncs at the same point, I agree that it's probably an emulator issue, in which case the run shouldn't be submitted at all until the emulator is fixed; if it's syncable, it's most likely for a different reason on the emulator as on the console (which are both interpreting the run in their own way, and it just happens to be correct for both), and is so likely suboptimal on both emulator and console.
However, in this list Wiki: ConsoleVerificationTests you'll see a list of games with partial sync. Games in this category clearly desync due to emulator-NES inaccuracies. Mainly because they desync consistently in the same spot. That is the group I'd prefer to see be 'syncable' over fastest. TASers could easily work around the emu inaccuracies by testing.
I still think that if it's determined that the desyncing happens because the emulator is not accurately emulating the hardware, the emulator should be fixed. Otherwise the run is, basically, relying on an emulator "bug", which is very frowned upon.
Settling for a TAS that syncs on both the emulator and the console might produce a suboptimal TAS. One which has compromises to make it work in both, rather than one which is the fastest possible on the console (and hence a properly-working emulator).
And, as I said, if the reason for the desync is something along the lines of the game reading uninitialized RAM, which may have nondeterministic values in it, but otherwise the emulator is doing a proper job at emulating the hardware, then there's no problem in accepting the TAS. As long as it represents what could be possible on the console if those RAM values happened to be set appropriately it's ok.
(Edit: Oh, and of course there's the third very real possibility: That the nesbot device itself is not being accurate enough. In that case it's the device that should be improved.
I'm actually wondering if some of the desyncs could be caused by the game reading input between frames or otherwise too often at some points, which confuses the nesbot. After all, the keypress file only has data for each frame, no more.)
Hmm.
If the run deterministicly desynchs, then either the bot or the emulator is at fault, and it needs correcting.
If the run is inconsistent on the real console then it's either the fault fo the console itself or the bot.
Unitialized ram does NOT set itself to zero or FF automatically, unless it's a part of the consoles boot routine. Depending on how long it's been without power, it can either be near the values it had last (if you just pressed the reset button or flicked the power switch off for only a second), or random bit noise if you've left it off a while.
I can't speak for the NES, but this was an issue on the commodore 64 computer and other 8 bit machines, where as piracy protection the randomness of uninitialized ram was tested, and the game crashed if it saw a pattern placed there to help the freezer cart recognize what needed compressing.
