Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Introducing one of my new projects MediaSynth!
The goal of this project is to create a very capable encoding program which can facilitate all of TASVideos' encoding needs.
Right now it doesn't feature some TASVideos requirements like a logo and subtitles, but it can generate every kind of video needed. At this stage, it's good for testing, or if you want a personal encode. I do need feedback to ensure this is on the right track. Once I know that the videos being generated are meeting expectations, I'll add on more features.
Some nice things about this application:
If you have a 64-bit OS, video encoding and various audio encoding tools will automatically become 64-bit, which may run faster, and allows much larger RAM usage.
Color matrices which differ on video resolution are properly selected. No more needing to fiddle these between different consoles.
Supports AVI input and more.
All-in-one, you don't need to download anything else to encode, not even AVISynth (although having the right video codecs and filters installed is important).
The MP4 "Compatible" option was highly tested for embedded video, flash, Android, IOS, and more, to ensure it really is compatible all over the place.
MediaSynth Video Encoder v1.0 Copyright (C) Nach 2013
Usage: mediasynth <input> [options...]
Options: [type 0] <output 0> [type 1] <output 1> ... [type N] <output N>
Types:
mp4 MP4 container with broadly compatible AVC video and AAC audio
youtube MKV container with YouTube compatible H.264 video and Vorbis audio
hq_opus MKV container with 10-bit 4:4:4 H.264 video and Opus audio
hq_vorbis MKV container with 10-bit 4:4:4 H.264 video and Vorbis audio
hq_aac MKV container with 10-bit 4:4:4 H.264 video and AAC audio
hq_flac MKV container with 10-bit 4:4:4 H.264 video and FLAC audio
hq_wav MKV container with 10-bit 4:4:4 H.264 video and PCM audio
std_opus MKV container with standard H.264 video and Opus audio
std_vorbis MKV container with standard H.264 video and Vorbis audio
std_aac MKV container with standard H.264 video and AAC audio
std_flac MKV container with standard H.264 video and FLAC audio
std_wav MKV container with standard H.264 video and PCM audio
Example:
mediasynth video.avi mp4 video.mp4 youtube video_yt.mkv
This would use video.avi as input, and output two files
If you have a video.wmv which you'd like to convert to our standard encodes, you'd run: "mediasynth video.wmv mp4 video.mp4 youtube video_yt.mkv hq_opus video.mkv". Now, video.mp4 would be the "compatible" video and can be placed on Archive.org as _512kb.mp4, video.mkv would be the standard 10-bit 4:4:4 to upload, and video_yt.mkv should work nicely for YouTube.
So please test, share results, and thoughts. Thanks!
Edit:
Spikestuff discovered there's some issues with Windows 8. I don't have Windows 8 handy, so it's hard to debug. Please test on other versions of Windows (or inform if it happens to work on Windows 8 by you).
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
In some cases, it seems to be point resizing, in some Lanczos resizing. Why the difference?
Also, can someone clarify what sizes are actually needed for various cases?
At what point does a video need to be downscaled?
What sizes are we looking at for 512kb?
What kind of up-scaling is needed for YouTube? When is upscaling not needed?
I'm looking to understand what width and height number are important. I want to create an algorithm that solely works with width and heights, not concepts of handhelds and approximations.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
In the current encoding scripts, I see the following:
<snip>
In some cases, it seems to be point resizing, in some Lanczos resizing. Why the difference?
Point resize for HD, Lanczos for 512.
Nach wrote:
Also, can someone clarify what sizes are actually needed for various cases?
At what point does a video need to be downscaled?
What sizes are we looking at for 512kb?
What kind of up-scaling is needed for YouTube? When is upscaling not needed?
Primary is never downscaled nor upscaled as whole (but there may be scaling to common resolution).
512kb shouldn't exceed 640x480/768x432, because playing such is too heavy for some players (esp. Flash).
512kb encodes also should be resized to correct for aspect ratio (square pixels).
Youtube needs at least 1922 width OR 1082 height (or native 1080p). The upscaling SHOULD be point upscaling (Nearest Neighbor) by even factor.
Nach wrote:
I'm looking to understand what width and height number are important. I want to create an algorithm that solely works with width and heights, not concepts of handhelds and approximations.
For producing 512kb (and to lesser extent primary) correctly, you need to know the PAR too.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Downscaling (with LanczosResize) is done when the dump was captured at higher resolution from 3D games, which does antialiasing. So no matter what the dump resolution is, for SD it is downscaled to 320x240. For HD it is upscaled with PointResize to be above 1080p (twice for 720p and 960p).
Nach wrote:
II'm looking to understand what width and height number are important. I want to create an algorithm that solely works with width and heights, not concepts of handhelds and approximations.
If you want to obsolete manually specifying handheld/TV console, I don't think you will succeed, because it can only be done so that you list resolutions and it knows which is for what console type, but Windows, Arcade, and maybe some Computer games may have resolution matching the ones listed as TV consoles, but you don't want to apply aspect ratio correction to those. Is there any other way to automate detection of ARC need? Or did you mean something different at all?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
If you want to obsolete manually specifying handheld/TV console, I don't think you will succeed, because it can only be done so that you list resolutions and it knows which is for what console type, but Windows, Arcade, and maybe some Computer games may have resolution matching the ones listed as TV consoles, but you don't want to apply aspect ratio correction to those. Is there any other way to automate detection of ARC need? Or did you mean something different at all?
I'm currently not at the stage of looking into aspect ratios and per system needs. I'm still at the purely generic stage. I must understand the requirements of the different domains and how to best meet them. Only once all the generic pieces are developed as best as they can will I look into specialty needs. I want to start from a clean base, not the muddled state things are in.
Ilari, you said downscaling works best with Lanczos and two taps. Is there any important algorithms to use in figuring out how much to downscale? Should it always be a whole number? An even number? Should width or height be preferred when maximizing the the area of the video?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Okay, I made a program to figure out what to downscale a large resolution to. Thoughts on the algorithm? Suggestions? Improvement ideas?
Download adjust.c
Language: c
#include <stdio.h>
#include <stdbool.h>
static const unsigned int target_size = 331776;
static unsigned int round_to(unsigned int x, unsigned int scale)
{
return(((x+scale-1)/scale)*scale);
}
static bool is_mult_of(unsigned int x, unsigned int mult)
{
unsigned int x2 = round_to(x, mult);
return(x == x2);
}
static inline void shrink(unsigned int *w, unsigned int *h, float ratio, unsigned int multiple)
{
for (*w = 800; *w; *w -= multiple)
{
float height = *w / ratio;
*h = round_to(height, multiple);
if ((height == *h) && (*w * *h <= target_size)) { break; }
}
}
static inline void shrinkwrap(unsigned int *w, unsigned int *h)
{
float ratio = (float)*w / (float)*h;
shrink(w, h, ratio, 16);
if (!*w) { shrink(w, h, ratio, 4); }
}
static void adjust(unsigned int width, unsigned int height)
{
unsigned int w = width, h = height;
if (width*height > target_size)
{
shrinkwrap(&w, &h);
if (!w)
{
printf("!!!");
if (!is_mult_of(width, 4)){ w = round_to(width, 4); h = height; }
else if (!is_mult_of(height, 4)) { w = width; h = round_to(height, 4); }
shrinkwrap(&w, &h);
if (!w)
{
printf("!!");
if (!is_mult_of(width, 4)){ w = round_to(width, 16); h = height; }
else if (!is_mult_of(height, 4)) { w = width; h = round_to(height, 16); }
shrinkwrap(&w, &h);
}
}
}
printf("In: %ux%u Out: %ux%u = %u 16: %d\n", width, height, w, h, w*h, is_mult_of(w, 16) && is_mult_of(h, 16));
}
int main()
{
adjust(640, 480);
adjust(800, 600);
adjust(1024, 768);
adjust(1280, 1024);
adjust(720, 404);
adjust(1280, 800);
adjust(1024, 1024);
adjust(1920, 1080);
adjust(1440, 1050);
adjust(1440, 900);
adjust(1440, 960);
adjust(1440, 1024);
adjust(1440, 1080);
adjust(1366, 768);
adjust(1024, 770);
return(0);
}
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
MediaSynth v1.1 is now available.
What's New:
Windows' exec*(), spawn*(), and system() now completely replaced with custom versions. The replacements should now properly handle files and paths that contain spaces and other interesting characters. "run.exe" should no longer pause on occasion upon completion.
Windows' temporary file generator replaced with a custom version. This custom version now creates files in the current directory, instead of some random location which you might not even have write access to. The custom version also has better conflict management, and ensure many temporary files can be created without issues. Temporary files now also bear the correct extensions.
Encoding options changed. Now there's many more presets to choose from.
Existing encoding presets tweaked for better compression and higher quality.
If the specified input file is inaccessible, you'll now be told immediately.
Improved self contained input support for MP4 and MKV.
Inability to create MKVs with certain audio outputs from FLAC and raw PCM fixed.
You are now told if you use a bogus "type".
An improved implementation of the above sizing algorithm now utilized for automatic downscaling of MP4 videos intended for streaming.
I want to thank Spikestuff for a lot of testing, and Ilari and turska for video encoding information.
Current options:
MediaSynth Video Encoder v1.1 Copyright (C) Nach 2013
Usage: mediasynth <input> [options...]
Options: [type 0] <output 0> [type 1] <output 1> ... [type N] <output N>
Types:
youtube MKV container with YouTube optimized H.264 video and Vorbis audio
mp4_cmax MP4 container with maximum compatible H.264 video and AAC audio
mp4_ch MP4 container with highly compatible H.264 video and AAC audio
mp4_cb MP4 container with broadly compatible H.264 video and AAC audio
mp4_std MP4 container with standard H.264 video and AAC audio
hq_opus MKV container with 10-bit 4:4:4 H.264 video and Opus audio
hq_vorbis MKV container with 10-bit 4:4:4 H.264 video and Vorbis audio
hq_aac MKV container with 10-bit 4:4:4 H.264 video and AAC audio
hq_flac MKV container with 10-bit 4:4:4 H.264 video and FLAC audio
hq_wav MKV container with 10-bit 4:4:4 H.264 video and PCM audio
std_opus MKV container with standard H.264 video and Opus audio
std_vorbis MKV container with standard H.264 video and Vorbis audio
std_aac MKV container with standard H.264 video and AAC audio
std_flac MKV container with standard H.264 video and FLAC audio
std_wav MKV container with standard H.264 video and PCM audio
Example:
mediasynth video.avi mp4_cb video.mp4 youtube video_yt.mkv
This would use video.avi as input, and output two files
Regarding the various MP4 options, mp4_cmax creates a 512 KB bitstream akin to what Archive.org automatically creates (with hopefully significantly better quality). This output option has also improved since v1.0 (named simply "mp4" there).
mp4_std is sort of like the compatibility MP4 we currently put out.
mp4_cb is a more compatible option than std, with downscaling, and even did a decent job on Braid.
mp4_ch is an in between for cmax and cb, where the former is too low quality, and the latter not compatible enough.
Essentially, mp4_cmax should work just about everywhere. mp4_ch should work almost everywhere, but very old toys, or highly constrained internet connections will have issues. mp4_cb should work on everything modern, or high-end toys from previous years.
Please test, report, and discuss!
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Ilari wrote:
Youtube needs at least 1922 width OR 1082 height (or native 1080p). The upscaling SHOULD be point upscaling (Nearest Neighbor) by even factor.
Are there cases where a larger even factor is preferable to a smaller even factor, even though they both exceed the needed resolution?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Ilari wrote:
Nach wrote:
Are there cases where a larger even factor is preferable to a smaller even factor, even though they both exceed the needed resolution?
Factor of 8x usually gives smaller files (albeit with longer encode time) than 6x.
Is there a known reason for this? Perhaps solid color 8x8 macro blocks? Does it depend on the resolution of the input to be true?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
I think we should test then if x4 is better than by x2, and if x16 is better than x10, x12, x14.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I think we should test then if x4 is better than by x2, and if x16 is better than x10, x12, x14.
x2 obviously is faster/smaller than x4 (but may not have enough resolution).
Also, the only systems that need more than 8x are Lynx (which needs at least 12x) and TI83 (which needs at least 18x). I should have one Lynx dump (Gordo106 by adelikat)...
12x:
encoded 17935 frames, 84.56 fps, 697.67 kb/s
1120.01user 23.70system 3:32.17elapsed 539%CPU (0avgtext+0avgdata 685692maxresident)k
16inputs+102088outputs (0major+158802minor)pagefaults 0swaps
16x:
encoded 17935 frames, 54.62 fps, 419.27 kb/s
1719.42user 37.76system 5:28.46elapsed 534%CPU (0avgtext+0avgdata 1194412maxresident)k
8inputs+61464outputs (0major+286683minor)pagefaults 0swaps
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Are there any cases that need to be handled that may not be handled here?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
So I ran some tests regarding multipliers for conversion.
When testing, it's important to use actual YouTube upscaling parameters, so tests can be representative of real world scenarios. It's also important to tweak the parameters to match each case (including multiplier), so useless steps aren't performed (something which MediaSynth contains the logic for).
For tests to be statistically correct, they really need to be run multiple times and averaged. But I only ran these once and in a VM with a dedicated CPU assigned to it.
Going for a multiple of 8 as opposed to 6 offers a tremendous speed boost, not to mention space savings. Really, it's logical, as x6, x10, and x14 don't allow for proper blocking. Turns out x12 is really good (x8+x4). One might think x16 would be better than x12 based on H.264 characteristics, but there is also a significant jump in the amount of blocks per frame which seems to outweigh the simpler partitioning possible.
Looks like I need to update the above algorithm, and also add some more heuristics to MediaSynth's parameter generation.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Bump for future reference. We might need this when we kill avisynth workflow.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Part of what was holding up further releases was some clunkiness of the underlying libraries I was using. However all that has been resolved. So going forward if we want this, fixing some of the drawbacks should now be possible.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Going for a multiple of 8 as opposed to 6 offers a tremendous speed boost, not to mention space savings. Really, it's logical, as x6, x10, and x14 don't allow for proper blocking. Turns out x12 is really good (x8+x4). One might think x16 would be better than x12 based on H.264 characteristics, but there is also a significant jump in the amount of blocks per frame which seems to outweigh the simpler partitioning possible.
Looks like I need to update the above algorithm, and also add some more heuristics to MediaSynth's parameter generation.