Post subject: Old TAS videos that need a 4K remaster
Player (35)
Joined: 4/21/2021
Posts: 5
N64 Tony Hawk's Pro Skater 3 (USA) in 14:35.22 by emu. https://www.youtube.com/watch?v=5RknPHKYTqc http://tasvideos.org/1029M.html This video only goes up to 240p.
Spikestuff
They/Them
Editor, Publisher, Expert player (2292)
Joined: 10/12/2011
Posts: 6337
Location: The land down under.
We know. This applies to basically a lot of TASes pre-2015 publication date.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Noxxa
They/Them
Moderator, Expert player (4139)
Joined: 8/14/2009
Posts: 4083
Location: The Netherlands
Even so, it wouldn't be a bad idea to have a list of those to keep track of (for active publications, anyway). Or for those that are in particular demand by viewers, like the one that this topic was made for.
http://www.youtube.com/Noxxa <dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects. <Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits <adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
Spikestuff
They/Them
Editor, Publisher, Expert player (2292)
Joined: 10/12/2011
Posts: 6337
Location: The land down under.
I disagree that complaining about something not being updated to modern resolutions (like majority of the publications) compared to something like melted encodes is counterproductive, especially when people are actually doing reencodes to begin with other TASes. If they want to complain about it they might as well learn to be an encoder and learn how to encode every publication pre-2015. This is not something that we should be condoning. It's almost like bumping publications going "hey new encode please".
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
EZGames69
He/They
Publisher, Reviewer, Expert player (3968)
Joined: 5/29/2017
Posts: 2707
Location: Michigan
The list is way too big to keep track of it.
[14:15] <feos> WinDOES what DOSn't 12:33:44 PM <Mothrayas> "I got an oof with my game!" Mothrayas Today at 12:22: <Colin> thank you for supporting noble causes such as my feet MemoryTAS Today at 11:55 AM: you wouldn't know beauty if it slapped you in the face with a giant fish [Today at 4:51 PM] Mothrayas: although if you like your own tweets that's the online equivalent of sniffing your own farts and probably tells a lot about you as a person MemoryTAS Today at 7:01 PM: But I exert big staff energy honestly lol Samsara Today at 1:20 PM: wouldn't ACE in a real life TAS just stand for Actually Cease Existing
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
If someone has a spare day for scripting, yt-dl can obtain all the info about all the streams.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
greysondn
They/Them
Joined: 4/29/2018
Posts: 44
Can anyone give me the format code(s) expected for current encodes? see for example: https://gist.github.com/sidneys/7095afe4da4ae58694d128b1034e01e2
Editor, Player (67)
Joined: 6/22/2005
Posts: 1041
315 seems to be the most common code for 2160p60 when running youtube-dl -F on several recent publications, but there was at least one that also had 401.
Current Projects: TAS: Wizards & Warriors III.
greysondn
They/Them
Joined: 4/29/2018
Posts: 44
Dacicus wrote:
315 seems to be the most common code for 2160p60 when running youtube-dl -F on several recent publications, but there was at least one that also had 401.
Would imply 2160p60 by yt formats - those are: 305, 315, 337, 401 cf: https://gist.github.com/AgentOak/34d47c65b1d28829bb17c24c04a0096f (different list from before)
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
For future purposes, I think it'd be more useful to print details about the best available stream (not what YT marks as "best" tho). yt-dl doesn't print info on just bestvideo, so I guess one would have to parse everything from -F and then pick the highest note (or list them all?), for example 2160p60.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
greysondn
They/Them
Joined: 4/29/2018
Posts: 44
As it stands, I'm just posting ones that don't match a target. I think I have the script figured. It will take some time to run. https://gist.github.com/greysondn/ed8167c273b3617099c813cabac2901d
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
2160p-non-60fps may be a legitimate and desired format for games that run at 30fps. And for some runs it's a 30fps encode for actual 60fps game, even if it's 2160p too. Those 2 types of encodes are hard to automatically distinguish, so it'd be helpful to print whatever is available for ones that needsUpdate.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
greysondn
They/Them
Joined: 4/29/2018
Posts: 44
Code monkey not crazy, just proud. Let me show you what I can see here from a single video. I'll just pick on myself for this task.
greysondn@LAPTOP-27D4Q1U4:/mnt/c/Users/Dorian Greyson$ python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import youtube_dl
>>> ytdl = youtube_dl.YoutubeDL()
>>> out = ytdl.extract_info("https://www.youtube.com/watch?v=FNGcikv7jx0", download=False)
[youtube] FNGcikv7jx0: Downloading webpage
[youtube] FNGcikv7jx0: Downloading MPD manifest
>>> # ---------------------------------------
... # All data for a given video that I can see
... # --------------------------------------
...
>>> out.keys()
dict_keys(['id', 'title', 'formats', 'thumbnails', 'description', 'upload_date', 'uploader', 'uploader_id', 'uploader_url', 'channel_id', 'channel_url', 'duration', 'view_count', 'average_rating', 'age_limit', 'webpage_url', 'categories', 'tags', 'is_live', 'automatic_captions', 'subtitles', 'like_count', 'dislike_count', 'channel', 'extractor', 'webpage_url_basename', 'extractor_key', 'playlist', 'playlist_index', 'thumbnail', 'display_id', 'requested_subtitles', 'requested_formats', 'format', 'format_id', 'width', 'height', 'resolution', 'fps', 'vcodec', 'vbr', 'stretched_ratio', 'acodec', 'abr', 'ext'])
>>>
>>> # -----------------------------
... # dat about formats that I can see
... # ----------------------------
...
>>> type(out["formats"])
<class>
>>> type(out["formats"][0])
<class>
>>> out["formats"][0].keys()
dict_keys(['format_id', 'manifest_url', 'ext', 'width', 'height', 'tbr', 'asr', 'fps', 'language', 'format_note', 'filesize', 'container', 'vcodec', 'acodec', 'url', 'fragment_base_url', 'fragments', 'protocol', 'format', 'http_headers'])
>>>
>>> # -------------------------------------
... # for example...
... # -------------------------------------
...
>>> out["formats"][0]["format_id"]
'139'
>>> out["formats"][0]["width"]
>>> out["formats"][0]["height"]
>>> out["formats"][0]["fps"]
>>> out["formats"][0]["format_note"]
'DASH audio'
>>> out["formats"][1]["format_note"]
'tiny'
>>> out["formats"][2]["format_note"]
'tiny'
>>> out["formats"][3]["format_note"]
'DASH video'
>>> out["formats"][4]["format_note"]
'DASH video'
>>> out["formats"][5]["format_note"]
'240p'
>>> out["formats"][6]["format_note"]
'DASH video'
>>> out["formats"][7]["format_note"]
'240p'
>>> out["formats"][8]["format_note"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> out["formats"][7]["format_note"]
'240p'
>>> out["formats"][7]["width"]
256
>>> out["formats"][7]["height"]
224
>>> out["formats"][7]["fps"]
30
>>> out["formats"][7]["format_id"]
'18'
>>>
Essentially, at this point, it makes more sense to fight with it yourself to get specific data. The best I can offer is a massive headache of a json file dump of all formats and what they say, because the data isn't exactly straightforward. What is "best" is a bit subjective here, too, once we stray from "absolute best". In other words, what you're asking for ends up being a JSON dump of the data for the entire channel, sooner or later, as opposed to focused data; sifting it becomes not-my-problem. That can be done. I can do that. All I need is the "yes, quit being a jerk and just dump all the formats with their data into the output json". Edit: There is a middle ground of organizing the format ids in a preferred order, then listing the first of those (or if none at all exist). In other words, I'd need some semblance of youtube's format IDs in a preferred order. I know we all want to say "2160p60 or p30" and be done with it, but sometimes - and this is no joke - that stream is literally silent video and there's not much warning without rifling through all the data or knowing their format ID. It's also of note that a large part of the problem comes from how Youtube tags these things, which is perhaps unfair to be quite so snippy about. (Well, snippy at anyone but YouTube, anyway.)
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Well if you have all the links to needsUpdate in a file, then someone can run a more detailed script on those rather than the entire channel. Also there are other channels tasvideos encodes are scattered around: http://tasvideos.org/Uploaders-Site-YouTube.html
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
greysondn
They/Them
Joined: 4/29/2018
Posts: 44
Was a small error in that file for what it is. Should be fixed now. The whole thing is - if you'd give me something that is deterministic and robust enough, I can write that code. What appears to me is that it is not as simple as throwing a figure out, and it may not be as simple as searching for that figure. I wrote assuming the solution requirements to be: * There is some target format(s) for videos, some don't match that * Videos should be on the TASVideos Youtube Channel * Anything not matching that format needs an update What I believe I am following from you is * There are some target format(s) for videos, some don't match that * However, my list isn't necessarily correct to the target formats * And beyond this, for some reason, there's a need for the "best" format to still be listed? Instead of checking again later against whatever the new target formats are? * Videos aren't necessarily on the TASVideos Youtube Channel * Anything not matching the format thing needs an update This however doesn't seem to be making anything clear. Beyond this, much other data can conceivably be dumped, if there's some use for it. It was omitted because there's not an apparent use for that I understand. Likewise, for example, if there's some keyword that distinguishes games that can be ignored, that wouldn't be impossible to work with. (Occurred to me while wondering if obsoletes actually get reencoded.)
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11268
Location: RU
Resolution seems to always go third, so getting the height should be reliable. Note varies too much indeed, but if it contains "fps" we could use that info. The end result would only contain what the biggest video height is there, and what the highest framerate is, well and the link itself. Does this make sense?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
greysondn
They/Them
Joined: 4/29/2018
Posts: 44
So some notes. I did talk directly to feos via Discord as I waited on this to wrap up; this is largely just a summary of what was said there. To begin with, to answer feos' question:
feos wrote:
Does this make sense?
Certainly. I wound up rewriting this to have two files output. The first is still just the videos it thinks needs updated. The second is all videos, regardless of whether it thinks they need updated. In both cases, the script gives the following data: * title * url * highest height amongst all formats * highest width amongst all formats * highest fps amongst all formats * whether it believed (based on the script) that it needs updated WARNING! The three features for which the highest is given are not necessarily from the same format - in effect, not necessarily from the same version on Youtube. The source file has been updated in the gist. Here it is again, anyway, just in case the address is needed: https://gist.github.com/greysondn/ed8167c273b3617099c813cabac2901d I wanted to staple outputs from running it right now to that gist but - no joke - they're too large. So I've put them in two other gists instead. Here is the json for all the videos, which you'll have to pull the raw to see all of: https://gist.github.com/greysondn/41463a7b7b912e39b2f94d897993b332 And here is the json for just the ones the script thinks need updated: https://gist.github.com/greysondn/227ba283bbf88a67d97acd310e2e5cc9 Just one more aside here that is in feos message that I noticed last minute:
feos wrote:
Resolution seems to always go third, so getting the height should be reliable.
The line feos links to here appears to be related to youtube-dl's console output. However, it is possible to embed youtube_dl as a library in python as I've done here. Why interpret the console output if you can ask the library directly? feos did already answer this: it wasn't known that it could be used as a library, and in feos's defense outside a comparatively brief note in the readme it's not immediately obvious how to do so. (I had to keep using python's help() function to get data on the objects in use - I didn't see any precompiled documentation, even if it is all well documented in the source.)
Acumenium
He/Him
Banned User
Joined: 6/11/2020
Posts: 73
Spikestuff wrote:
I disagree that complaining about something not being updated to modern resolutions (like majority of the publications) compared to something like melted encodes is counterproductive, especially when people are actually doing reencodes to begin with other TASes. If they want to complain about it they might as well learn to be an encoder and learn how to encode every publication pre-2015. This is not something that we should be condoning. It's almost like bumping publications going "hey new encode please".
Do you know what complaints are or are you taking offense to the notion of ever-progressing video technology relative to what you did a decade ago? I'm not sure why you're getting so absurdly hostile here. There is nothing wrong with a list of TASes that could use a remaster. No one is telling you to do it or insinuating you did a bad job years ago, or whatever.
Spikestuff
They/Them
Editor, Publisher, Expert player (2292)
Joined: 10/12/2011
Posts: 6337
Location: The land down under.
Hey Acumenium are you actually bothering to read what I wrote or are you just skimming over the points again? Heck, I'm not even the only one in this thread with issues about this thread's existence:
EZGames69 wrote:
The list is way too big to keep track of it.
I'm just more in-depth about it. Also "hostility" seriously? Either make a valid point here about why you're for or against a massive list or just don't participate in the thread in the first place. You're not a Publisher or an Encoder, you seriously lack the information.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Banned User, Former player
Joined: 3/10/2004
Posts: 7698
Location: Finland
Spikestuff wrote:
Also "hostility" seriously? Either make a valid point here about why you're for or against a massive list or just don't participate in the thread in the first place. You're not a Publisher or an Encoder, you seriously lack the information.
That sounds like a rather hostile attitude to me. Just calm down.