One reason why people use old emulators like Snes9x is because it lags less than newer, more accurate ones like lsnes. This creates runs that cannot be beaten with more accurate emulators (since they lag more), causing people to abuse older emulators more and the cycle repeats. This cycle hinders emulator development, so in order to break this trend, older emulators should (hopefully) be disallowed.
It goes without saying that a more accurate emulator should always be preferred over a less accurate one, and that a run being faster (even if it's just by one frame) because of emulation inaccuracy is unacceptable.
That being said, the expression "keypresses may be performed on the original hardware" is a big vague and misleading (especially since the word "may" can be interpreted in different ways).
Some games in some consoles behave in a completely deterministic way, always reacting the same way when certain buttons are pressed at certain times. In these cases the run can be replicated in the original console if you have the means to supply the properly timed button input to the console (which usually means you need a device to do that).
However, not all games in all consoles behave in a completely deterministic way. There are many sources of nondeterminacy in many consoles (as a rule of thumb, the more modern the console, the more likely it is that games do not behave in a deterministic way). In fact, AFAIK, this can go as far back as the NES, where a game can in theory act in a non-deterministic way if it so chooses (by reading the RAM immediately after power on, since IIRC the RAM is not zeroed or reset in any way by the console when it boots up, and can contain random values which cannot be affected from the outside). In more modern consoles there are more plentiful sources of nondeterminacy (such as reading a data CD).
In these cases a TAS cannot be reliably replicated on the console. However, does that mean that the TAS is cheating? Not really.
In the case of consoles/games with nondetermistic sources of randomness that cannot be affected, what the TAS is doing is a hypothetical run in one example case (assuming that the emulator is otherwise very accurate): In other words, if the console happened to have that precise set of circumstances for all of its sources of nondeterministic randomness, how the perfect run would look like. The exact situation may be practically impossible to replicate on the console, but that doesn't mean that the run wouldn't look exactly like that on the console assuming that it wasn't.
It also means that a hypothetical perfect player, who can see the entire state of the console at each moment and react to everything with perfect timing, could make a run that looks extremely similar to the TAS (or could even be a bit faster, depending on how the randomness happens to behave).
In these cases a TAS cannot be reliably replicated on the console. However, does that mean that the TAS is cheating? Not really.
An emulator which can't emulate lag correctly would not be able to sync up (relaibly) even if the games were 100% deterministic.
Also, I thought about it, and I think banning older emulators will do more good than harm since (some) people are abusing older emulator's flaws to make faster runs.
An emulator which can't emulate lag correctly would not be able to sync up (relaibly) even if the games were 100% deterministic.
As I said, a more accurate emulator should always be preferred over a less accurate one. The rest of my post was a comment on the reasoning you mentioned for this, ie. that it's because the TAS should be replicable on the console, which isn't actually the case in practice. However, even though the TAS may not be replicable on the console, that doesn't mean the emulator isn't accurate.
In other words, the reason for banning an emulator cannot be "its keypress files cannot be replicated on the console". That's an impossible demand to make for consoles with nondeterministic sources of random timing.
In other words, the reason for banning an emulator cannot be "its keypress files cannot be replicated on the console". That's an impossible demand to make for consoles with nondeterministic sources of random timing.
Alright. I changed the original reason to another one that also bothers me.
Are you suggesting any concrete emulators or is this just an unspecific rant?
In the case of SMW or SM, it's not just a question of saving frames, but also a question of comparing to existing runs. If you're working on an improvement, and you're losing a frame - was it you, or was it the emulator? How will you find out?
Same for the final submission. If you're switching emulators, you'll need to explain how many frames were gained or lost due to the change.
This is additional work, it complicates things, and thus switching to a different version when improving runs isn't as easy as just using the same emu. This is especially true for games with highly optimized runs, where changes due to emulator differences will dwarf any actual improvements.
That should never be an excuse to start a run for a new game on an inaccurate emu, but I can understand if it's done when improving existing runs.
You should also consider that newer emulators are often less mature, less stable (as in, the run may not sync on future versions), may have unknown bugs (as in, eats your TAS and laughs at you), and sometimes lack useful secondary features (like lua-support, memory watch etc). Or sometimes TASers are simply more comfortable with the emulator they've always been using.
Since the goal of this site is to produce entertaining runs, perfect console compatibility is a nice feature to have, but by no means required. And to address the issue you raised in the edit of the initial post: the goal of this site isn't to expedite emulator developement, either.
I agree with Warp on this:
It goes without saying that a more accurate emulator should always be preferred over a less accurate one
In other words, the reason for banning an emulator cannot be "its keypress files cannot be replicated on the console". That's an impossible demand to make for consoles with nondeterministic sources of random timing.
True.
Taking a specific example though (SNES); there are better reasons than that. byuu made quite a few test roms and ran them all on the original console plus available snes emulators. You can say without a doubt that bsnes is more accurate than any snes9x, even though there isn't a SNESbot at all.
Also, replicating keypress files on the console doesn't show much about accuracy either, especially when the bot doesn't force matching the emulator's timing (droid64 or whatever they call it).
NES also has accuracy tests, which have been tallied here: http://tasvideos.org/EmulatorResources/NESAccuracyTests.htmlTub wrote:
In the case of SMW or SM, it's not just a question of saving frames, but also a question of comparing to existing runs. If you're working on an improvement, and you're losing a frame - was it you, or was it the emulator? How will you find out?
Same for the final submission. If you're switching emulators, you'll need to explain how many frames were gained or lost due to the change.
Yes, it's extra work; but considering that the original run you're comparing with was never "right" in the first place, the switchover seems like a no-brainer to me.
Tub wrote:
Since the goal of this site is to produce entertaining runs, perfect console compatibility is a nice feature to have, but by no means required. And to address the issue you raised in the edit of the initial post: the goal of this site isn't to expedite emulator development, either.
I think the goal of this site ought to be to produce runs that are verifiable and authentic, first and foremost. But people don't agree with that, so oh well I guess.
In any event, nothing is going to happen. There was discussion about obsoleting 1.43 when 1.51 came out, but that didn't happen, and it's been how many years now? TASers only switch to new emulators when they don't like the old ones (eg Famtasia); the community at large doesn't care whether they're TASing the console or TASing the console with speed-change cheat codes.
Are you suggesting any concrete emulators or is this just an unspecific rant?
Past VBA revisions, Snes9x, and just basically any old versions of emulators.
In the case of SMW or SM, it's not just a question of saving frames, but also a question of comparing to existing runs. If you're working on an improvement, and you're losing a frame - was it you, or was it the emulator? How will you find out?
Same for the final submission. If you're switching emulators, you'll need to explain how many frames were gained or lost due to the change.
This is additional work, it complicates things, and thus switching to a different version when improving runs isn't as easy as just using the same emu. This is especially true for games with highly optimized runs, where changes due to emulator differences will dwarf any actual improvements.
That should never be an excuse to start a run for a new game on an inaccurate emu, but I can understand if it's done when improving existing runs.
Yea, but since there's always a time when someone improving a run, there must be a deadline to stop using old emulators, or people would keep using them.
You should also consider that newer emulators are often less mature, less stable (as in, the run may not sync on future versions), may have unknown bugs (as in, eats your TAS and laughs at you), and sometimes lack useful secondary features (like lua-support, memory watch etc). Or sometimes TASers are simply more comfortable with the emulator they've always been using.
Actually, even when Snes9x 1.5x was considered good, people kept using Snes9x 1.43. Being comfortable isn't a (very) good reason to stick with older revisions.
Since the goal of this site is to produce entertaining runs, perfect console compatibility is a nice feature to have, but by no means required. And to address the issue you raised in the edit of the initial post: the goal of this site isn't to expedite emulator developement, either.
I agree with Warp on this:
It goes without saying that a more accurate emulator should always be preferred over a less accurate one
With the emphasis on "should" instead of "must".
The entertainment factor doesn't mean the site should use older emulators. What if someone posted a very entertaining "glitched" playaround that abuses errors that a console couldn't do due to emulation flaws?
Anyway, how about at least banning the emulators which are 2 revisions/versions behind? Like Snes9x 1.43 at least?
Past VBA revisions
...
how about at least banning the emulators which are 2 revisions/versions behind?
I definitely agree. If I remember correctly, VBA 20 GBx movies sync on VBA 24-M, since they have the same GBx core. There are some runs, though, like the Pokémon Green TAS, that require VBA 20, so that revision could stay in use.
However, VBA 19, 21, and 22 movies should all sync perfectly on the latest VBA 23, so I don't see any reason for runs on these older versions to still be accepted.
We're all in agreement that the most accurate emulator should be used if possible. The question is, what're we gonna do when a TAS on an older emulator is submitted? Instant-reject?
With all the work and policy changes towards making TASing more open to new players and to TASes from other communities, such draconic restrictions won't do us any good.
And don't forget that "newer" doesn't alway mean "more accurate". I'd have to look up the details, but I remember a couple of emulator updates that did quite the opposite to certain games.
Having a strict deadline of, say, no emulator older than 1 year would also prevent anyone from submitting a TAS that has been worked on for longer than a year. That does actually happen, when the TASer doesn't have much spare time, or needs to drop TASing for a while due to RL stuff, or is generally working on ten projects at a time and rarely gets any finished. Any policy that'd prevent these TASes from being submitted is bad. "2 revisions behind" is actually worse, since you newer know how many revisions are going to be published and when.
Now you might say that we count the time when the TAS was started. But that's impossible to verify, making any strict bans unenforcable as long as the submitter is smart enough to lie about it.
jlun2 wrote:
What if someone posted a very entertaining "glitched" playaround that abuses errors that a console couldn't do due to emulation flaws?
Then it shouldn't be published. Gamebreaking glitches are always subject to more scrutiny than usual runs.
But verifying the glitch is more work than looking at the emulator's release date. Ideally, it's checked against the console itself. Lacking that, against multiple emulators of the same system. There's no guarantee that the most recent emulator gets it right, nor that the older emulator gets it wrong.
CoolKirby wrote:
However, VBA 19, 21, and 22 movies should all sync perfectly on the latest VBA 23, so I don't see any reason for runs on these older versions to still be accepted.
If 23 isn't more accurate than 19, 21 or 22, what's the point in banning the older ones?
Please, let's stick with the recommendation instead of "banning" things.
However, VBA 19, 21, and 22 movies should all sync perfectly on the latest VBA 23, so I don't see any reason for runs on these older versions to still be accepted.
If 23 isn't more accurate than 19, 21 or 22, what's the point in banning the older ones?
Isn't 23 more accurate? I thought it was more accurate in addition to being backwards-compatible with movies from previous revisions. Even if it's not more accurate, what's the point of using old revisions if those movies sync on this one?
I think only the latest revisions of the two VBA branches should be accepted: VBA 23.5 (backwards-compatible except VBA 20), and the more accurate VBA24-M (and VBA 20 for rare cases). Why would we need to accept a run from any other VBA revision?
Note that even though an emulator may fail some of those tests, it may still emulate most games accurately (for the simple reason that those games never trigger the things that those tests are testing). AFAIK many of those tests deal with things like running undocumented opcodes and other such unusual behavior.
We're all in agreement that the most accurate emulator should be used if possible. The question is, what're we gonna do when a TAS on an older emulator is submitted? Instant-reject?
I would say 'yes' to instant-reject, with a small caveat.
Ideally, it should go along these lines: an announcement has to be made at a certain date clearly stating that, starting from that day, only the new emulator version will be accepted or the movie will be rejected. The announcement will also grant one exception -- everyone that was already working on a TAS on the older version may get grandfathered if they meet certain conditions.
For example, the conditions could be:
At least one recent public WIP from before the announcement;
said WIP must be of good quality (publication quality, or very close to it) and it must comprise of either at least half of the game (for shorter games), or it must be at least (say) 10-20 minutes (for longer games such as RPGs);
the TAS must be finished within a year of the cutoff announcement.
Failing to meet the conditions will lead to an instant-reject. Allowing a TASer to show a private WIP to an admin might also be grounds for an exception, but that would be up to the admins.
aren’t we banning them already? older versions of DeSmuME aren’t accepted anymore, I thought… but I guess there, you have bigger changes than simple lag :/
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
What about the case of Dolphin where it's being updated literally constantly?
don't make it a rule for ALL emulators,just the ones whose emulation is well stablished already.Basically all but hourglass,psx,64,dolphin,right?
PCSX-rr isn't updated anymore and PSXjin is updated only once in a while (like VBA), but I would think older versions of those emulators would be less accurate and should be banned. Mupen-rr hasn't changed in over 6 years, so since old versions don't exist, it doesn't fit this rule. I agree that the emulators that are getting new revisions often (Hourglass and Dolphin) should be exempted from a ban.
getting new revisions often (Hourglass and Dolphin) should be exempted from a ban.
Hourglass isn’t being developed
It's not? Weren't there lots of new revisions for a while? I haven't checked in two months, but I thought there were still new revisions being worked on.
Well, in Dolphin's case, I think that the accepted revisions for submitted TASes should be limited to the revisions after the big save state fix four months ago. There shouldn't be any reason to use the old, desync-prone revisions from before the fix.
The save fix doesn't matter. It only affects recording, not playback. You'd have to be an idiot to use anything else (actually i think i'm going to go ask skidau to merge it since it seems nitsuja isn't coming back any time soon..), but there's no reason not to accept it.
