If you don't want to read the wall of text, just answer the poll question please :D
I hate avisynth!
I've been maintaining the
TAS Encoding Package since
2012, and from having had all this avisynth experience, I can say that
I absolutely hate it. And the most complex thing in the package has always been the subtitles. We do 3 different types of encodes, and with every one of them, subtitles needs special treatment.
For SD encodes we need pixelated font, because default avisynth subtitles (as well as .ass via plugin) have hardcoded antialiasing that makes them blurred as hell at font sizes like 10. Having huge font will be very hard to make it fit into the screen (especially for lowres consoles like GameBoy or A2600). So we use a thing created by natt called FreeSub, it relies on bitmap fonts, so we can make them tiny and pixelated, and they remain readable.
For HD encodes we have to use ng_bighalo, because default avisynth subtitle halo is also hardcoded, and it looks tiny in hires encodes where font size is huge. And while ng_bighalo matches avisynth subtitle functionality, FreeSub does not, so every time we need to do some custom tweaking, we have to tweak 2 separate codepaths and then make sure to remember to tweak them back.
This is further complicated by primary mkv encode, because it needs to be done at native resolution without any stretching in the encode. We stretch it in the video player via certain arguments. What makes this a nightmare is trying to match how subtitles appear in all the encodes. Consistency is easier to maintain as a user, but it's now completely impossible to maintain it as a developer. Because in order to make the subtitles appear in the same place, we have to guess exactly how much we will resize other encodes for aspect ratio correction, and account for it in the primary. Not only positioning is hard to do consistently, but also dimensions of the subtitle font, because in some encodes font will be too wide or too narrow, depending on which side of the video we have to stretch to apply ARC, and how much (sometimes twice!) Add to that consoles with tiny resolution that need to be stretched (A2600), and it becomes completely impossible to make the subs look decent no matter what you do... while also maintaining automation!
There's also VirtualBoy that has 2 screens and we need to make them
anaglyth, including the subtitles.
Perfect consistency and automation would be possible if we switched to some subtitle method that works equally well in all encodes, but that method either doesn't exist or is a hell to set up and configure.
There's also an option to just drop primary mkv encode...
Most people watched on Youtube
last time I asked. And while youtube is unique in that it offers space for indefinite amount of videos of indefinite resolution, it's doing weird things with bitrate, and its compression ratio has increased over the years for things like 720p. So the better video you want to see, the higher resolution it has to be. Right now we're kinda overshooting and making 4K encodes (some do 8K), because those look great, but it may be non-trivial to watch them (and upload size is also
gigantic). My guess is that most people watch in 1080p.
Also there's a thing called takedown. Having everything on one channel is incredibly handy, but we need to be very careful and not accidentally get too many strikes. It happened once some years ago, and it's hard to predict how well youtube keeps behaving in the future.
Also for years youtube has had capped framerate, and sometimes it was a real issue, and we had to use workarounds like ng_deblink (or even tasblend). Even today you can't get more than 60fps on youtube, and some systems do need to go higher.
So what about having an actual streaming alternative that's not youtube, doesn't have to be 4K to look good, can have any framerate, and has good and consistent subtitles?
I'm talking about turning compatibility mp4 encode (512kb) into that!
Most of our systems are TV based. TV based systems (consoles and computers) generate 240p type of signal, and it gets doubled vertically on a TV and always occupies the same
field every frame (as opposed to 480i footage that occupies alternating fields). It's why some people saw the scanline effect: the second field was missing, so there were gaps on high
TVL sets.
But currently, due to limitations in soft aspect ratio correction, we can't force height to double and then adjust width to something specific. We can only do that before encoding: pre-resizing can only be done in compatibility mp4 and in youtube HD. Primary HQ mkv just makes the video player stretch height by uneven factor, which makes things look very weird.
To be honest, the entire aspect ratio correction thing looks very bad on native resolution video! For anything that's single pixel in width, you get jumpy artifacts when the screen scrolls, and pixels are constantly unequal. And it's never an integer resizing factor either! If you don't use pixel filter in your player, it may not look as jumpy, but then once you try to zoom in to be able to actually watch it, the whole thing becomes very blurred.
Which is why I want our new primary encode (compatibility mp4 stream) to be point-resized to 2x native resolution, and then aspect ratio correction to be applied with lanczos filter.
If we do that, subtitles don't have to be pixelated anymore, because now resolution also matches what subtitle developers had in mind, and hardcoded antialiasing will look nice. And we could use the same exact functions for SD and HD encoding, for example .ass, which is much more professional and adjustable than bugged avisynth subtitles. No problem with hardcoded halo width anymore either. There could probably even be some cool movement/masking subtitle effects in SD encodes too!
Legacy
15 years ago it was important to compress encodes like crazy, because youtube was not a thing, other streaming media sites were limited (they still are!), and people were actually paying for every extra megabyte they'd download. This is the only reason we remove duplicate frames from primary encodes, and apply aspect ratio correction to them via video player (so we don't have to resize them, making them bigger).
But back then, subtitle business was also much, much, MUCH simpler. People would just put a giant black box covering half the screen, including the main character, and write movie info inside that box. Sometimes way before gameplay even starts.
Then 10-bit 4:4:4 encoding appeared. It allowed us to get rid of chroma subsampling that we had to allow in regular encodes. Because compatibility mp4 that we used for streaming sites would be impossible to stream if it had those fancy encoding methods, as well as frame deduplication and soft-ARC. But guess what? 10bit444 encodes are heavier than compatibility mp4! There's more data to process, so in a whole bunch of cases those things meant for minimal size were even bigger than 8-bit 4:2:0 mp4 encodes!
Somewhere along the way, filesize stopped playing the huge role it used to play. So to actually maintain some kind of encoding standard, I'm personally ready to reevaluate what we need to encode and how, and I'll be interested in seeing how we can compress 480p compatibility mp4 to make it a good youtube alternative while also feasible for downloads.
If a publisher actually wants to do some extra encode (for example for .ass subtitles? or maybe lossless? or extra hires?), I would absolutely not mind, but I'd like to get primary HQ mkv off the official encoding scripts!
For publications, subtitles need to be placed in a way that doesn't obscure anything relevant to the game or the TAS, i.e they can't cover the character or UI elements. MKV stretches out subtitles horizontally, pretty significantly in fact:
This is something you have to actively check during the encoding process, too: For our encoding package, we preview in VirtualDub, and the VirtualDub preview shows the subtitles as they pertain to the MP4 encode. You have to manually check whether or not the MKV subtitles line up in the way you want. This is often not the case, meaning you either have to find a new placement altogether or re-encode MKV with different placement, as shown in the first screenshot with Batman. This is something that new encoders generally find out the hard way, and something that even experienced encoders often overlook, requiring further re-encoding and more time wasted, slowing down publication for a team that's already understaffed and overworked as it is. And the thing is, this problem is inherent to the MKV encode. The YouTube HD stream is essentially just an upscaled version of the MP4 in terms of sub placements, so finding a good placement for MP4 means you've found a good placement for YouTube HD.
Dropping support for MKV would mean significantly streamlining the encoding process. TASvideos is trending towards accepting more movies and games as well, meaning in the future publishers will have to work even harder than they already do. No MKV means that encoding would take much less time (both in pure encode time and in prep time), and the overall process would become less daunting for new publishers, possibly enticing new people to help out. In my eyes, there is nothing bad about this change whatsoever, and I 100% support it.
