So something that I have been wanting for a long time is support for single game/series/engine tasing tools. On that note, I felt one of the first steps towards this would be a framework of sorts for submitting them to be looked at, and bare minimum rules behind what can be acceptable. Now there is the Laws of TAS page, but that feels a lot more philosophical and more aimed at emulators.
So what I think at the bare minimum should be required for a TASing tool to be acceptable:
Open source
Able to be played back between different people
Movie format that allows for determining run time in some fashion. This can be achieved in a number of ways.
Documentation on how to run the tool.
One staff member running a test movie and parser support (done on our end, but try to make this as easy as possible).
Things that I feel need discussion
AV Dumping support. Should internal dumping be a requirement? Should compatibility with a lossless capture program like KKapture be required? Or are we open to recording lossily?
How deterministic do we want it to be at minimum? Would it be acceptable if you need a specific video card sometimes like dolphin?
A distinct file extension. I'm pretty sure about this one but I'd like confirmation from adelikat on how this works.
Changes to games should be fairly minimal and mainly for the sake of creating consistent rerecording.
There might be something else I'm missing. Feel free to discuss.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
The only kind of input this custom movie format supports should be the input that the game directly supports and provides to the player. What regular player can not do, should not be possible in the input format either.
For example, in some games, movie format may allow to set character coordinates or velocity directly, without having to hit input keys to get there gradually. We don't want to allow that. TAS environment should be as close as possible to running the game in an emulator of whatever device it runs on.
Scepheo wrote:
feos wrote:
Elaborate on the differences between these 2, and on (again) how emulating Doom altogether is better than making an internal movie for it (have fun expanding that principle to Hourglass too).
As you yourself said; maintaining a strict policy for which game engine movies we accept is impossible, and this will eventually go wrong. This is the first reason I'm against it. The second reason is that game engine replays aren't the same thing as TASes. In the case of Half Life 2, replay files contain actor positions rather than input, which means that completely impossible movements (like going through walls) is possible by hex editing these. In the case of Doom, it is impossible to control the menu, while this is very much part of the game. If a menu related glitch was found, a TAS would be allowed to (and in fact should) exploit: something that wouldn't be possible with demo files.
Should internal dumping be a requirement?
Lack of it uses to lead to complicated setups that are not always consistent. I had to dump my latest DOOM publication on several versions of kkapture and PrBoom, with all sorts of config tweaks, to figure out why something mysteriously breaks in every scenario I try. Not only this makes it hard to learn for regular encoders/publishers (we don't want to depend on having PhD people on staff), it also makes encoders hate the complicated setup, dream of getting rid of it, and try to dodge having to deal with it. This is why we dropped support for mupen64 and pcsx. Their encoding was absolute nightmare.
Should compatibility with a lossless capture program like KKapture be required?
kkapture can do lossy encoding too, but overall, if there's no way to record AV internally, kkapture has to be supported since it captures AV on API level, which is more stable and leads to less possibility of info loss. Other tools are usually much worse at this. And as I said, kkapture may also be a hell to handle.
Or are we open to recording lossily?
It looks terrible at low resolutions. And even more terrible after you upscale that for youtube.
How deterministic do we want it to be at minimum? Would it be acceptable if you need a specific video card sometimes like dolphin?
Poor determinism leads to another level of nightmares and blackouts, for tasers, judges, and publishers. If we don't have anyone available with the magic PC spec, the movie will end up forever in the queue or rejected anyway. We don't want TASing to depend on real world luck.
Overall here's the list of dumping needs that I compiled after having dealt with all sorts of complicated encoding setups at tasvideos: http://tasvideos.org/LawsOfTAS/OnVideoDumping.html The farther a potential new emulator or framework is from those, the less I will want to have it accepted. We also have 2 worthwhile pages by the previous senior publisher: http://tasvideos.org/Natt/SyncHallOfShame.html and http://tasvideos.org/Natt/CaptureHallOfShame.html
Agreed on all the rest.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I personally feel this is an ideal, not a hard requirement. However, as feos noted, a lack of it can complicate the workflow by a fair amount. (Dega is actually the notable exception that plays nice with .kkapture)
Should compatibility with a lossless capture program like KKapture be required? Or are we open to recording lossily?
I'm willing to allow any external capture software that allows to capture audio and video without any loss of usable information (so no frame loss and no audio sample loss), up to and including lossy codecs. I just don't want a workflow that's literally impossible to work with.
Joined: 11/13/2006
Posts: 2792
Location: Northern California
I'm just going to go on record and say that if "Publishers don't feel like dealing with it" is an actual open reason for not accepting these frameworks, then our publication standards need to be overhauled.
This is a discussion we've had a few times in the past as staff, and to me this actually feels like the biggest thing holding back custom TAS frameworks. The idea that a custom framework absolutely needs to have built-in lossless AVI capture for us to accept it is kind of an absurd thing to ask of people. With our publication standards as they are currently, the argument is really just a them or us situation regarding the heavy amounts of work that goes into our current quality standards for publication. Either we have to spend days or possibly weeks figuring out how to encode something to our insanely perfectionist standards, or we have to force every custom framework dev to implement features that are specifically only for us.
This should not be "someone has to work hard and I don't want it to be me", it should be "nobody has to work hard at all".
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
I'd like to note my questions don't necessarily indicate I'm leaning one way or another, but should be answered by the community at large. My one hard rule though is that a staff member be able to run the tool and encode a test run with it.
[16:36:31] <Mothrayas> I have to say this argument about robot drug usage is a lot more fun than whatever else we have been doing in the past two+ hours
[16:08:10] <BenLubar> a TAS is just the limit of a segmented speedrun as the segment length approaches zero
The idea that a custom framework absolutely needs to have built-in lossless AVI capture for us to accept it is kind of an absurd thing to ask of people. With our publication standards as they are currently, the argument is really just a them or us situation regarding the heavy amounts of work that goes into our current quality standards for publication. Either we have to spend days or possibly weeks figuring out how to encode something to our insanely perfectionist standards, or we have to force every custom framework dev to implement features that are specifically only for us.
It's always a scale.
There are things that are absolutely necessary (ability to rerecord and replay movies), things that are extremely good to have (determinism, ability to dump audio-video at all, AV sync), things that are nice (ability to capture on-screen display into video), and things that are only for uber geeks (debuggers).
If it's feasible to have the most good things, we're happy. If things are limited, but reliable, we like it. If basic needs are hard to satisfy, we don't like it, but still do it (accept and process). If the most basic needs are impossible to meet, we don't accept.
For any candidate tool, there would be this list (possibly with priorities assigned), and some agreement on minimal acceptable cutoff. I'm fine with lowering the standards.
The most painful publications I remember were still not about lossless encoding of every frame being hard, they were about no way to capture syncing AV while also syncing the movie itself. Sounds like the only painless solution for those is capturing them using some kind of an external screengrabber. Which is probably worth discussing as an option.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I'm willing to allow any external capture software that allows to capture audio and video without any loss of usable information (so no frame loss and no audio sample loss), up to and including lossy codecs
Gonna amend this post and say I'm willing to allow a minimal amount of dropped frames here and there. Basically the same deal with dropped samples.
My personal thoughts, without much reflection behind them:
I geeeeenerally don't think A/V dumping needs to be a hard requirement anymore. Of course, it's always a huge bonus, but unlike back in the day, people actually have the ability to screen record at an acceptable resolution and bitrate now. It still takes some powerful hardware, and doesn't lead to optimal results, but for most people watching, it will be good enough.
What's much more important to me is determinism, and the ability for people to play back movie files on their own system without too much hassle. I'm in particular concerned about future proofing and bit rot. A TAS is not much good if the last person who was able to get it to sync successfully left the site 5 years ago.
I would also want to warn against being too enthusiastic about accepting new formats before they're ready. While it gets movies on the site faster, which is a good thing, it will create much more problems down the line. See famtasia, yabause, mupen, pcsx, dega, and the whole assortment of other junk emulators we got rid of, and the legacy of hard to deal-with movies they leave behind.
There's also this earlier thread. The thoughts in my post in it still apply.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I think there are so many questions that need answering that any progress on this is unlikely to happen as it is. If someone approached TASVideos with a custom TAS tool, an input file, playback instructions, and the best way to record video, then maybe it could go from there.
One other concern I have is with TAS tools that are direct modifications of the game, and how you would go about distributing that tool without also distributing the game.
Moderator, Senior Ambassador, Experienced player
(898)
Joined: 9/14/2008
Posts: 1007
Randomno wrote:
If someone approached TASVideos with a custom TAS tool, an input file, playback instructions, and the best way to record video, then maybe it could go from there.
https://github.com/EverestAPI/CelesteTAS-EverestInterop
We already have a perfect example of what should be supported that has multiple examples of everything you asked for. If we build it, they may come... But right now the barrier of entry remains gated. My take remains that we should pave the way for Celeste TAS content we *know* is both important to their own community as well as to viewers (a Celeste TAS was just shown at SGDQ 2023). Once we have this one example in place, others will have a blueprint to follow. Without a path to publication there's a lot less drive to try (ask me how I know...); making this Celeste example will immediately make a significant trove of content available for submissions, assuming the community is still interested by that point.
Which steps do we need to go through to make CelesteTAS accepted?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 11/13/2006
Posts: 2792
Location: Northern California
dwangoAC wrote:
If we build it, they may come... But right now the barrier of entry remains gated. My take remains that we should pave the way for Celeste TAS content we *know* is both important to their own community as well as to viewers (a Celeste TAS was just shown at SGDQ 2023). Once we have this one example in place, others will have a blueprint to follow. Without a path to publication there's a lot less drive to try (ask me how I know...); making this Celeste example will immediately make a significant trove of content available for submissions, assuming the community is still interested by that point.
I don't think we can just jump in and start building support for any of these tools without having a conversation with their respective communities first. We want to support these tools and these communities, but there's no way we can reasonably do so if we don't know what they are, how they work, or if those communities are even interested in working with us. That's why this thread exists: It's meant to be an open invitation to developers of these tools to reach out to us and either show interest or let us know what we could be doing better. We haven't had anyone reach out yet to the best of my knowledge, which to me means that either nobody knows this is an option or that nobody is interested in working with us. It could be anything, really, but the solution to anything it could be is the same: We need to be talking to these communities directly, and if they're not reaching out to us, then we should be reaching out to them.
Is the Celeste TAS community interested? I was under the impression it was the opposite, actually, in that they still had concerns with TASVideos and were hesitant to reach out. In that case, just building the parser likely isn't going to help bring them over here, and I feel like they could even see it as hostile. I'd be more than happy to talk to them about it and hopefully fix those concerns, but I would need to know where they are, when they're available, and whether or not they're willing to talk in the first place. Without that conversation, more specifically and importantly their explicit support and approval, I wouldn't be comfortable with going ahead and implementing a CelesteTAS parser, and that also holds true with any tool from any other community.
TASvideos Admin and acting Senior Judge 💙 | CohostwarmCabin wrote:
You shouldn't need a degree in computer science to get into this hobby.
Moderator, Senior Ambassador, Experienced player
(898)
Joined: 9/14/2008
Posts: 1007
I'd like to start with a brief apology, the way my tone came off in text was not ideal and I sounded way more harsh than I intended. I've had a chance to chat with Samsara and agree that there are some other aspects that need to be tackled all while there's a lot of other things that are asking for admin attention.
With that aside, Vamp reached out to me and it led to a discussion with a couple of other folks on the Celeste TAS side. There is in fact interest in making this happen from them as well, and Vamp may make an account here to join the discussion. One major challenge that will need to be solved on the Everest / Celeste TAS framework side is that their distribution method doesn't currently involve a movie file in the way TASVideos would expect; it relies on a series of git-controlled individual level files with a .tas extension that are tied together by a main input file that references them. Their development methods are somewhat similar to how the Super Mario 64 120 star TAS efforts tackle things with individual levels that then get stitched together later. There's a discussion going on now about the best way to collect individual levels into a run such as 100% and zip them up in a single file similar to how BizHawk or libTAS works along with a metadata file containing everything the TASVideos parser expects to see including author information and a rerecord count field (that will likely be a wild guesstimate of the sum of all individual levels combined, but I digress). That work will have to take place first but it does look like there's a viable path forward here.
The reservations noted by Samsara in particular are reasonable and real and I appreciate the willingness to speak up. There's nothing urgent here and we can take our time working out the best way to do this but it definitely seems like it's a good idea to at least try.
I read through this discussion a while ago, and I just today came across a submission that appears to have been rejected purely because the sound effects couldn't be captured properly in the encode: #5317: ThunderAxe31's Windows Unknown Game in 00:59.42. I'm not sure if that was due to an issue with how Hourglass encodes, or if Hourglass just didn't support the sound effects for this game.
But I'm sort of unhappy if runs like this can't be accepted, although I guess that might be a separate discussion of where we draw the line for emulation accuracy.
I read through this discussion a while ago, and I just today came across a submission that appears to have been rejected purely because the sound effects couldn't be captured properly in the encode: #5317: ThunderAxe31's Windows Unknown Game in 00:59.42. I'm not sure if that was due to an issue with how Hourglass encodes, or if Hourglass just didn't support the sound effects for this game.
But I'm sort of unhappy if runs like this can't be accepted, although I guess that might be a separate discussion of where we draw the line for emulation accuracy.
Since then we agreed that if nothing helps, a screengrab could work, assuming it's done well.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
For me, I think the current most pressing concern is one of preservation, plain and simple. TAS recordings using specific engines, custom mods, or anything else that are presently impossible to publish could become accepted in the future as these tools either gain broader acceptance, gain necessary features like AVI dumping, or anything else, but that's dependent on the recording files actually surviving to that day, which is anything BUT guaranteed. Many of these demofiles and tools are hosted in very obscure locations, unreliable filehosts, or etc that could cause them to just become lost before they ever get their fair shot. I think a good stopgap solution, one way less strenuous than overhauling the submissions system, would be expanding Userfiles so that it can accept additional movie file types that aren't supported by the site now, but could be in the future; my understanding is that adding new accepted extensions for Userfiles is as simple as editing the relevant System page, but it might be good to add a third system page besides just 'SupportedMovieTypes' and 'SupplementalUserFileExtensions', possibly named 'ExtraMovieTypes' or something, for better organization.
Every new movie format requires a movie parser implemented on the site to display the metadata correctly.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I stopped by the Hollow Knight Discord server and had a chat with several of the established folks there. They're using libTAS in their efforts, but they have to use it in a very different way than most of us are used to, and some of those ways may... sound somewhat alien at first, but I think we can find a path forward to publish Hollow Knight TAS runs. Here are the considerations at play:
Hollow Knight TAS runs will not require a new movie format as they are using standard libTAS movie files
Most runs are based on an altered initial state, as in, runs would require a verification movie to be run first
Nearly all runs significantly change the FPS setting within a band of 50 to 500 FPS to mirror RTA rules
Variable FPS means Load Removed Timing (LRT) is used instead of TASVideos style power-on to last input rules
Many runs are segmented or rely on other runs; think everything from simple NG+ runs to individual level runs
The Hollow Knight TAS community enjoys a wide variety of branches, many of them amusing meme categories
Hollow Knight is a Unity game which currently introduces significant non-deterministic behavior on playback
Savestates in libTAS fail across different play sessions; saved runs must be replayed from the beginning
Encodes can take over 10 hours for a single run due to the nature of software rendering
Encodes nearly always have to be done segmented due to desyncs on playback
The desyncs are accepted as a part of TASing a Unity game as an unalterable annoyance
Nearly all Hollow Knight TAS runs are consumed via video encode playback due to the above issues
So let's just address the big issues right off the bat - the TASVideos community isn't used to having to put up with desyncs on playback as a part of life, and the concept of having to do a segmented encode attempt seems ridiculous. Investigations into how Unity handles RNG have led folks to the conclusion this problem may never be solvable due to the sheer complexity of it, so unfortunately this isn't a matter of waiting for emulation or tooling to improve. We the TASVideos community have absolutely rejected runs that couldn't sync in a single setting, but that's just the norm for the Hollow Knight community. The differences continue from there, most every TASVideos run is locked to something like 60 FPS but Hollow Knight RTA runners use a number of various tricks that significantly affect the FPS the game runs at and the TAS community mirrors the range that RTA runners can achieve. (Hollow Knight TAS folks reading this, please feel free to register an account and comment on things I got wrong, misremembered , or simply explained and/or represented badly.)
Publishing Hollow Knight TAS's on TASVideos raises many challenges and resolving those issues is almost certainly going to require compromise, moreso on the TASVideos side in the form of having to accept segmented encoding and the resulting requirement to make the encode itself the primary consumable. Compromises would also be needed from the Hollow Knight TAS community. For instance, I made it clear that we'd need reproducible steps such as a verification movie to get a game configuration into the correct state to allow a judge to recreate their run, painful though it may be. They'd also have to.. um... potentially rename some of their meme categories to meet site decency requirements. Maybe. One of their categories is Bench Out-Of-Bounds, and that one has a comparatively tame acronym. I'll let you be the judge. :)
One other topic that came up was the Hollow Knight TAS community currently lacks a home to place their knowledge. They have a few scattered documents and links to resources such as a Pastebin of abbreviations and this Hollow Knight enemy health analysis document but there's no central repository. A game resources page would be ideal for them.
So how about it? Can we find a way to work past the somewhat foreign requirements and chart a path to accepting Hollow Knight TAS runs here on TASVideos? I, for one, think we should do what we can to accommodate them in some fashion but I'm usually overly optimistic so let's see how shocked folks are at what I've said and take it from there. :)
So let's just address the big issues right off the bat - the TASVideos community isn't used to having to put up with desyncs on playback as a part of life, and the concept of having to do a segmented encode attempt seems ridiculous. Investigations into how Unity handles RNG have led folks to the conclusion this problem may never be solvable due to the sheer complexity of it, so unfortunately this isn't a matter of waiting for emulation or tooling to improve. We the TASVideos community have absolutely rejected runs that couldn't sync in a single setting, but that's just the norm for the Hollow Knight community.
On the topic of desyncs with HK tases which we didn't mention yet is that movies sometimes don't sync on other devices and sometimes even don't when switching between WSL2, VM and dual boot on the same system. Also the latest interim build of libTAS has caused existing TAS's to desync, likely because the stack smashing issue with loading save states that was fixed.
Also on segmented encodes, yes sometimes we do encode in segments because of the desync issues. But there are also quite a few who segment the whole tas into multiple libTAS movie files because of the desyncs. This is especially done on patch 1.4.3.2, and on patch 1.2.2.1 it is done as well, but less segmented than 1.4.3.2.
And yes the desync issues are patch specific, likely because of the difference in Unity versions. Patch 1.0.2.8-1.2.2.1 are on Unity 5.4 and 1.4.3.2 is on Unity 2017.4.
dwangoAC wrote:
They'd also have to.. um... potentially rename some of their meme categories to meet site decency requirements. Maybe. One of their categories is Bench Out-Of-Bounds, and that one has a comparatively tame acronym. I'll let you be the judge. :)
Bench Out-Of-Bounds is a glitch, not a category. The only category that would be indecent is All Stag Stations.
On the topic of desyncs with HK tases which we didn't mention yet is that movies sometimes don't sync on other devices and sometimes even don't when switching between WSL2, VM and dual boot on the same system. Also the latest interim build of libTAS has caused existing TAS's to desync, likely because the stack smashing issue with loading save states that was fixed.
Yeah, the interim build causes many issues when it comes to syncing LTMs from the 1.4.4 release, I haven't even been able to figure out why. On top of that, on the main 1.4.4 release, for some people save states don't work, but others don't get the stack smashing error (the error that was fixed in 1.4.4). I TASed for months without savestates and it sucked, especially on a game like HK where LTMs can easily get in the hundreds of thousands of frames and it takes a while to get to the right spot, even fast forwarding and without force software rendering on.
As you very well know Peeka,
Hollow knight TASes also sync horribly over changed settings.
Changing the menu background leads to different particles and different rng.
Changing resolution or language from the originals used to create the TAS have similar effects.
This game is a very time consuming and difficult venture to optimize properly. We're still discovering new things even 6+ years after the game's release and 2 1/2 years after the formation of the HollowKnight TAS discord server.
In conclusion, I'd like to see this game displayed for a wider audience in the TAS community, but I don't think altering rules or how we TAS is in the spirit of that, since we face much different adversities in our TASing than others.
dwangoAC stopped by the Hollow Knight TAS Discord and encouraged us to take part in the discussion regarding TASVideos submissions, so I figured I'd add my two cents. I'd first like to clarify that I don't speak for our entire community; there are a lot of different HK TASers and we all have different goals and priorities.
For me, at least, the fundamental thing that's dissuaded me from attempting to submit to TASVideos is more philosophical than technical. HK's long encode times and need to deal with nondeterministic desyncs as a fact of life definitely offer technical barriers to any sort of verification, but, more than that, I've just never been particularly enamored with the 'serious' side of speedrunning and the idea of applying that semi-competitive flavor to TASing especially doesn't appeal to me. I appreciate TASVideos as a resource for discoverability, but the idea of having a centralized authority make my TAS 'verified' based on some standardized set of rules that are a very awkward fit for HK isn't something I'd care to pursue.
To be clear, I have no objection to those who want to do the whole 'serious business: speedrunning' thing, but if that's the dominating and centrally enforced atmosphere for TASVideos in particular, then it's not someplace I feel welcome. And that's fine, if that's what your community values, but if you are interested in being inclusive to folks like myself, then I figured it's worth letting you know the current vibe I perceive when looking through the submission rules.
As a further aside, I will say that most of my TASes could in principle be replicated by others from the movie files given compatible hardware and using modified game code for some of the gimmicks (hacking in a starting charm loadout, giving lantern for better viewing, tracking grounded frames based on game state, etc.), and enough time and patience. Times on the order of 20+ hours to encode an ~1 hour TAS, usually a few times to recover from desyncs (with fast forward to the desync point, so not having to reencode the entire thing in one go), then splice together the resulting videos. Some of my TASes are so unstable though that even I couldn't replicate them again from the original movie files; my Sundered game TAS was so desync-prone that the only way I was able to complete the video was to just never close libTAS and encode it over 19 segments as I went, using savestates to keep things going and praying that libTAS didn't crash before I was done. That TAS will never be able to be replicated again and the final video is the only resulting artifact of the hundred hours or so that went into making it. A lot of more complicated PC games are like this and reliable determinism is a luxury we're not afforded.
I agree completely with Jarlyk's statement to be honest, I like the hollow knight TAS community and seeing that turn into something competitive or having to "verify" our TASes just does not sound fun. I don't really understand how TASVideos works or what its rules for TASes are seeing as I joined here specifically to comment on this thread, but just with what Jarlyk's said, I'd prefer not to have that type of community. I've experienced the "pray it won't crash" multiple times while TASing modded bosses on hollow knight patch 1.4.3.2 and I know what Jarlyk means when he talks about it
