I've been gathering info on the details, and here's the actual situation we're facing.
Authenticity of the game
We always enforced this. We want the image to be good, we disallow hacking it to make it easier (or to otherwise insignificantly change it). We disallow cheat codes. We disallow games that are poorly emulated, as well as in-game tricks and glitches resulting from poor emulation.
When a game is not available at all in its original form, we allow its modified version that persisted as long as it doesn't glitch out over unintended console settings (like region). If a game can't be TASed due to various reasons, we may allow tweaking some of its files as long as it doesn't affect gameplay.
Authenticity of the environment
We also require our emulators to stay true to the original consoles they emulate. This feels implied, because consoles have locked and known once and for all specs, with exception for cases when some quirk was undocumented or documented wrongly. We don't allow emulator settings that don't correspond to something the original consoles had.
It's important to note than in cases when a console behaves non-deterministically, we don't require emulators to inherit this aspect, quite the opposite: we enforce determinism even if it didn't exist on the actual console! With that one exception, we don't allow to emulate a hacked console, including its hardware and software it's been officially shipped with.
Authenticity of their communication
We don't allow NTSC games for CRT TV based consoles to be ran on PAL consoles. We don't allow using BIOS from the wrong region. The type of consoles the game version was released for is preserved while TASing.
Exceptions are cases when different console version results in identical gameplay or seemingly improves it, like GBC or SGB do to GB games.
Sometimes authenticity is not even remotely possible
This is about PC specs. When architecture is open and anyone can release hardware and software for a machine, and when anyone can build a machine that fits their taste and budget, it's impossible to demand authenticity. Because there is no
spherical original machine in vacuum, everything is infinite variations. Consoles have locked and sealed specs that may vary a bit, but otherwise it's possible to take a game, run it on original console and compare to how it's emulated. It's impossible to take
a PC and check how well a game for it is being emulated to run on PC in general, because there's no
the PC. You can only emulate one of the countless components.
Then, you may manually overclock your hardware or throttle it as you like and can, to force the games do what you want.
On top of that, some operating systems allow modifying games, even DOS could do that easily, and some games explicitly react to file modifications by enabling or disabling some features, or ignore them completely, loading levels in the wrong order or blindly using utterly irrelevant data as legit.
So does that mean that for PC games, we should allow anything?
Of course not. Probably by chance, but it happened so that over the years we've developed our rules to allow entertaining arbitrary decisions on one hand, and strictly defined, purely speed oriented, boring content on the other. When it comes to something inherently arbitrary, we do our best to disallow it from fastest completion and full completion, from the Vault tier (because those categories are supposed to be strict and obviously legitimate to vast majority of people). When it's hugely entertaining, we may allow non-standard approaches that may look odd or questionable, but we still try to forbid blatant misuse.
For example, we allow in-game glitches that actually exist in the original software, but we don't allow injecting unintended game images before or during play. We don't allow pre-setting startup memory to something not known to be possible on authentic hardware. We don't allow Game Genie that goes between the game and the console, intercepting the game code. We don't allow unintended console region.
Play can only feel superhuman if we enforce all the original limitations human had from the outside, and overcome our internal human limitations: planning in advance, perfect reaction and absolute precision. Breaking integrity and authenticity breaks the challenge we've been having with the game, as well as the challenge we're having while TASing. It'd also defeat the purpose of watching a TAS: who cares if I can kick my console and jump straight to ending? But we care how perfectly the game
software can be played.
We limit ourselves to working with software in isolation. Hardware this software is running on we take as is. All the conditions are authentic, only the human that faces them is no more mere human.
So what do we do to PC games given unlimited specs variations?
Just like with picking the intended console region or the intended BIOS, we can pick intended conditions the games were supposed to be played in. The easiest way to find them out is reading the official docs on recommended PC specs. If that's not enough or not available, we can use modes the games explicitly supply, for example several pre-defined speed modes. If even that is not enough, we may look at some less public resources like game code and deduce intended settings from that.
Sticking to intended specs is required, because some games may completely glitch out on things they were never designed for, exactly like some NTSC games glitch out in PAL mode.
Some games may be skipping levels over unsupported CPU, others would allow time saving glitches if you overclock the CPU, no one plays 3D games on GPUs that aren't meant to support them and break the gameplay. Breaking the authenticity just because it allows glitches is very very shaky ground.
Come on already, should we ban that or not?
No one is convinced such stuff should be outright banned. After all, maybe someone can make an entertaining TAS with noclip. But the arbitrary broken nature of unsupported environment doesn't really sound like it should be allowed for Vault.
Also, when someone makes a TAS of Doom with overclocked CPU resulting in just 1 minute of very entertaining gameplay, such a setting would compromise the legitimacy of a movie to some people, just like there are people disliking major skip glitches or glitches in general. So if we limit such movies to Moons, we will need to add some labels and classes to them that would prevent any confusion. Just like we always mark movies that start from a save file and play nawgame+.
The range of possible scenarios such classes and labels would need to reflect may be huge, so just throw out crazy ideas, brainstorm if you feel like it, we'll participate and review the suggestions.
And of course tell me what you think about this post.
Movie.
