Posted: 2/27/2011 2:32 PM
(Edited: 2/27/2011 3:51 PM)
Post subject: Reducing the size for HD encodes
Quote
feos
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
For my connection isn't as stable as I wish, I sometimes need the filesize to be less, but keep it still lossless, of course. I need some additions to the main encoding command for x264, taking into account YouTube's reencoding peculiarity.
moozooh wrote:
feos: you won't be able to shrink it much further, but try including --subme 9 (or 10), --me umh --merange 32. also, --ref 6 or more should help.
My current command looks like that: 
x264 --qp 0 -b 0 --threads 2 --keyint 60 --fullrange on
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 2/27/2011 3:49 PM
Post subject: Re: Reducing the size for HD encodes
Quote
Nahoc
Editor, Experienced player (860)
Joined: 8/12/2008
Posts: 845
Location: Québec, Canada
TASer of 2012 3D Platforms TASer of 2012 TAS of 2012 Speedy TAS of 2012 3D Platforms TAS of 2012
feos wrote:
it still lossless, of course.
Since when CRF-20 is lossless?
Posted: 2/27/2011 3:51 PM
Quote
feos
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
woops, it was a mistake. fixed
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 2/27/2011 3:52 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
yea, feos, your command line is all screwed up. I would guess lucky x264 takes --qp 0 over --crf 20. To answer your question change --qp 0 to --qp 10 is one way. Also remove --crf 20. Adding --keyint infinite would also help. If --keyint infinite doesn't work, update your x264.
Posted: 2/27/2011 4:04 PM
Quote
feos
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
Thnaks, I'll try. --crf 20 tiptoed here by accident, I don't actually use it.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 2/27/2011 4:11 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
On a side note, no idea why you have --threads 2. You are just limiting your encode speed.
Posted: 2/28/2011 4:46 AM
Quote
Flygon
Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria 
start /wait x264.exe "consoleencode.avs" --versioninfo --ssim --crf 0 --keyint 300 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree --weightb --direct auto --subme 10 --trellis 2 --partitions all --me umh --merange 64 --rc-lookahead 250 --fullrange on --8x8dct --no-dct-decimate --output "encodedboth.mp4"
Yes, this is seriously my HD encoding script.
Posted: 2/28/2011 4:48 AM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
Flygon wrote:
start /wait x264.exe "consoleencode.avs" --versioninfo --ssim --crf 0 --keyint 300 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree --weightb --direct auto --subme 10 --trellis 2 --partitions all --me umh --merange 64 --rc-lookahead 250 --fullrange on --8x8dct --no-dct-decimate --output "encodedboth.mp4"
Yes, this is seriously my HD encoding script.
And it is not very useful =p
Posted: 2/28/2011 4:51 AM
Quote
Flygon
Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria
Hey, it cuts down the size. :P
Posted: 3/5/2011 4:50 PM
Quote
feos
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
Updating an x264 gave me a grey video for youtube problem. I've downloaded the build posted there & it works, but wothout --keyint infinite. Though, using finite (300) keyint gives me just a few more MBs, than infinite did..
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 3/5/2011 4:57 PM
Quote
Aktan
Publisher
Joined: 4/23/2009
Posts: 1283
grey video problem is related to the MKVToolNix versio you are using. Update to the latest MKVMerge, and you should be fine.
Posted: 3/9/2012 1:45 AM
Post subject: Reducing file size for YouTube encodes.
Quote
Guga
He/Him
Joined: 1/17/2012
Posts: 838
Location: Chile
I've doing some encodes and all of them at least have a size of 1GB, and for don't having a good connection, they take too much time to upload. So here is my question: Any possible way to reduce the size? I will put the command lines I use now: 
x264.exe --crf 12 --keyint infinite --min-keyint 1 --range tv --colormatrix bt709 --input-range tv --output "output" "input"
Posted: 3/9/2012 2:41 AM
Quote
natt
Editor, Emulator Coder, Site Developer
Joined: 5/11/2011
Posts: 1108
Location: Murka
changing --preset can help somewhat
Posted: 3/9/2012 3:24 AM
(Edited: 3/31/2012 10:50 AM)
Quote
Lex
Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada
Encode from your RGB source to lossless RGB ZMBV using ffmpeg. YouTube handles ZMBV perfectly and ZMBV is a very good lossless format. For this, download a static build of ffmpeg from Zeranoe's ffmpeg builds page (32-bit or 64-bit; doesn't matter; the exe is in a "bin" subdirectory somewhere in the archive), then try the following ffmpeg command, which should result in a pretty small lossless RGB ZMBV with lossless FLAC audio, all of which YouTube will handle properly. 
ffmpeg -i input.avs -c:a flac -compression_level 8 -c:v zmbv output.mkv
If you must use h264, you should give x264 RGB data because it now does the color conversion properly. You should be using bt470bg color matrix for everything to accurately reproduce colors when converting from RGB to YUV and back to RGB. YouTube will do the conversion to RGB properly with a tv-range YUV 4:2:0 video using bt470bg color matrix. The quality should be lossless so you don't get YouTube transcoding lossy to lossy. For x264, the following is as good as you can make it without losing quality and without spending too much time ("placebo" is usually many times slower than "veryslow" without much file size reduction). 
x264 --qp 0 --preset veryslow --keyint infinite --output-csp i420 --range tv --colormatrix bt470bg --output x264output.mkv input.avs
To help others, it would be useful to try both commands, compare file sizes, and post your findings here. Edit: Note that the x264 command line doesn't handle audio at all. You'll need to mux audio after you use it if you want to make an accurate comparison. You can do this with ffmpeg. Simply add these lines after your x264 line in your batch script, making sure to modify the input and output filenames as appropriate, like this. 
x264 --qp 0 --preset veryslow --keyint infinite --output-csp i420 --range tv --colormatrix bt470bg --output x264output.mkv input.avs

ffmpeg -i input.avs -c:a flac -compression_level 8 -vn audio.flac
ffmpeg -i x264output.mkv -c:v copy -i audio.flac -c:a copy finaloutput.mkv
Posted: 3/9/2012 1:47 PM
(Edited: 3/9/2012 1:57 PM)
Quote
Nahoc
Editor, Experienced player (860)
Joined: 8/12/2008
Posts: 845
Location: Québec, Canada
TASer of 2012 3D Platforms TASer of 2012 TAS of 2012 Speedy TAS of 2012 3D Platforms TAS of 2012
Please take some of your time to check and post in previous threads instead of always creating new ones: http://tasvideos.org/forum/viewtopic.php?t=10813
Posted: 3/9/2012 1:54 PM
Quote
Lex
Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada
To be fair, the fact that he posted this thread caused me to want to help. I don't think I would have chipped in so much information without this thread. Also, that thread doesn't mention ZMBV at all.
Posted: 3/9/2012 4:12 PM
Quote
feos
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
Let the mod merge these 2 please.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 3/9/2012 4:15 PM
Quote
natt
Editor, Emulator Coder, Site Developer
Joined: 5/11/2011
Posts: 1108
Location: Murka
Lex wrote:
If you must use h264, you should give x264 RGB data because it now does the color conversion properly. You should be using bt470bg color matrix for everything to accurately reproduce colors when converting from RGB to YUV and back to RGB. YouTube will do the conversion to RGB properly with a full-range YUV 4:2:0 video using bt470bg color matrix. The quality should be lossless so you don't get YouTube transcoding lossy to lossy.
Youtube can NOT handle full range YUV input properly. The same test clip as was used in the JWplayer test, uploaded to youtube. In this case the levels are wrong no matter what the flash acceleration level is, so youtube broke the levels in their transcode process. http://www.youtube.com/watch?v=S0Eh2fseKec
Posted: 3/9/2012 4:16 PM
Quote
Lex
Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada
Ah. All the more reason to just use RGB ZMBV, which YouTube does handle properly. I'll fix my x264 line then, and my post. Done.
Posted: 3/9/2012 5:01 PM
Quote
creaothceann
He/Him
Editor
Joined: 4/7/2005
Posts: 1874
Location: Germany
Lex wrote:
ffmpeg -c:a flac -compression_level 8 -c:v zmbv -i input.avs output.mkv
I get errors with that: 
ffmpeg version N-38622-g1eabd71 Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar  7 2012 00:18:03 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 63.100 /  2. 63.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, avs, from 'tmp.avs':
  Duration: 00:39:02.36, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: zmbv, rgb24, 640x480, 589824 kb/s, 60 tbr, 60 tbn, 60 tb
c
    Stream #0:1: Audio: flac ([1][0][0][0] / 0x0001), 44100 Hz, 2 channels, s16,
 1411 kb/s
Incompatible pixel format 'rgb24' for codec 'libx264', auto-selecting format 'yu
v420p'
[buffer @ 01C146E0] w:640 h:480 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01C1CEE0] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 01C1D180] w:640 h:480 fmt:rgb24 -> w:640 h:480 fmt:yuv420p flags:0x4
Incompatible sample format 's16' for codec 'libvorbis', auto-selecting format 'f
lt'
[libx264 @ 01C14280] using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisali
gn LZCNT
[libx264 @ 01C14280] profile High, level 3.1
[libx264 @ 01C14280] 264 - core 120 r2146 bcd41db - H.264/MPEG-4 AVC codec - Cop
yleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deb
lock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 m
e_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chro
ma_qp_offset=-2 threads=9 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_c
ompat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 we
ightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=
0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4
ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'tmp.mkv':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: h264, yuv420p, 640x480, q=-1--1, 1k tbn, 60 tbc
    Stream #0:1: Audio: vorbis, 44100 Hz, 2 channels, flt
Stream mapping:
  Stream #0:0 -> #0:0 (zmbv -> libx264)
  Stream #0:1 -> #0:1 (flac -> libvorbis)
Press [q] to stop, [?] for help
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
...
Input=Avisynth, colorspace=rgb32, latest ffmpeg build from that site.
Posted: 3/9/2012 6:28 PM
Quote
Lex
Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada
Oh, my mistake. It was just a matter of ffmpeg command line arg ordering. The working line is this:
ffmpeg -i input.avs -c:a flac -compression_level 8 -c:v zmbv output.mkv
WTF. Never mind. Apparently, ffmpeg's ZMBV encoder can only encode 8-bit paletted ("pal8" pixel format in ffmpeg) video. I know ZMBV supports higher bit depths and works wonderfully with YouTube (example 1080p video I uploaded as ZMBV), but ffmpeg's ZMBV encoder fails.
Posted: 3/9/2012 11:01 PM
Quote
natt
Editor, Emulator Coder, Site Developer
Joined: 5/11/2011
Posts: 1108
Location: Murka
Looks like it hasn't been mentioned in this thread, but a huge spacesaver (and quality increaser, for the "original" transcode) is to pointresize to 8x8 and let youtube handle the aspect correction at playtime. More details here: http://tasvideos.org/forum/viewtopic.php?p=303545#303545