Post by natt

Posted: 2/27/2011 2:32 PM
(Edited: 2/27/2011 3:51 PM)

Post subject: Reducing the size for HD encodes

Quote

feos

Site Admin, Skilled player (1236)

Joined: 4/17/2010
Posts: 11274
Location: RU

For my connection isn't as stable as I wish, I sometimes need the filesize to be less, but keep it still lossless, of course. I need some additions to the main encoding command for x264, taking into account YouTube's reencoding peculiarity.

moozooh wrote:

feos: you won't be able to shrink it much further, but try including --subme 9 (or 10), --me umh --merange 32. also, --ref 6 or more should help.

My current command looks like that:

x264 --qp 0 -b 0 --threads 2 --keyint 60 --fullrange on

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 2/27/2011 3:49 PM

Post subject: Re: Reducing the size for HD encodes

Quote

Nahoc

Editor, Experienced player (852)

Joined: 8/12/2008
Posts: 845
Location: Québec, Canada

feos wrote:

it still lossless, of course.

Since when CRF-20 is lossless?

Posted: 2/27/2011 3:51 PM

Quote

feos

Site Admin, Skilled player (1236)

Joined: 4/17/2010
Posts: 11274
Location: RU

woops, it was a mistake. fixed

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 2/27/2011 3:52 PM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

yea, feos, your command line is all screwed up. I would guess lucky x264 takes --qp 0 over --crf 20. To answer your question change --qp 0 to --qp 10 is one way. Also remove --crf 20. Adding --keyint infinite would also help. If --keyint infinite doesn't work, update your x264.

Posted: 2/27/2011 4:04 PM

Quote

feos

Site Admin, Skilled player (1236)

Joined: 4/17/2010
Posts: 11274
Location: RU

Thnaks, I'll try. --crf 20 tiptoed here by accident, I don't actually use it.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 2/27/2011 4:11 PM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

On a side note, no idea why you have --threads 2. You are just limiting your encode speed.

Posted: 2/28/2011 4:46 AM

Quote

Flygon

Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria

start /wait x264.exe "consoleencode.avs" --versioninfo --ssim --crf 0 --keyint 300 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree --weightb --direct auto --subme 10 --trellis 2 --partitions all --me umh --merange 64 --rc-lookahead 250 --fullrange on --8x8dct --no-dct-decimate --output "encodedboth.mp4"

Yes, this is seriously my HD encoding script.

Posted: 2/28/2011 4:48 AM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

Flygon wrote:

start /wait x264.exe "consoleencode.avs" --versioninfo --ssim --crf 0 --keyint 300 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-adapt 2 --mbtree --weightb --direct auto --subme 10 --trellis 2 --partitions all --me umh --merange 64 --rc-lookahead 250 --fullrange on --8x8dct --no-dct-decimate --output "encodedboth.mp4"

Yes, this is seriously my HD encoding script.

And it is not very useful =p

Posted: 2/28/2011 4:51 AM

Quote

Flygon

Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria

Hey, it cuts down the size. :P

Posted: 3/5/2011 4:50 PM

Quote

feos

Site Admin, Skilled player (1236)

Joined: 4/17/2010
Posts: 11274
Location: RU

Updating an x264 gave me a grey video for youtube problem. I've downloaded the build posted there & it works, but wothout --keyint infinite. Though, using finite (300) keyint gives me just a few more MBs, than infinite did..

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 3/5/2011 4:57 PM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

grey video problem is related to the MKVToolNix versio you are using. Update to the latest MKVMerge, and you should be fine.

Posted: 3/9/2012 1:45 AM

Post subject: Reducing file size for YouTube encodes.

Quote

Guga

He/Him

Joined: 1/17/2012
Posts: 838
Location: Chile

I've doing some encodes and all of them at least have a size of 1GB, and for don't having a good connection, they take too much time to upload. So here is my question: Any possible way to reduce the size? I will put the command lines I use now:

x264.exe --crf 12 --keyint infinite --min-keyint 1 --range tv --colormatrix bt709 --input-range tv --output "output" "input"

Posted: 3/9/2012 3:24 AM
(Edited: 3/31/2012 10:50 AM)

Quote

Lex

Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada

Encode from your RGB source to lossless RGB ZMBV using ffmpeg. YouTube handles ZMBV perfectly and ZMBV is a very good lossless format. For this, download a static build of ffmpeg from Zeranoe's ffmpeg builds page (32-bit or 64-bit; doesn't matter; the exe is in a "bin" subdirectory somewhere in the archive), then try the following ffmpeg command, which should result in a pretty small lossless RGB ZMBV with lossless FLAC audio, all of which YouTube will handle properly.

ffmpeg -i input.avs -c:a flac -compression_level 8 -c:v zmbv output.mkv

If you must use h264, you should give x264 RGB data because it now does the color conversion properly. You should be using bt470bg color matrix for everything to accurately reproduce colors when converting from RGB to YUV and back to RGB. YouTube will do the conversion to RGB properly with a tv-range YUV 4:2:0 video using bt470bg color matrix. The quality should be lossless so you don't get YouTube transcoding lossy to lossy. For x264, the following is as good as you can make it without losing quality and without spending too much time ("placebo" is usually many times slower than "veryslow" without much file size reduction).

x264 --qp 0 --preset veryslow --keyint infinite --output-csp i420 --range tv --colormatrix bt470bg --output x264output.mkv input.avs

To help others, it would be useful to try both commands, compare file sizes, and post your findings here. Edit: Note that the x264 command line doesn't handle audio at all. You'll need to mux audio after you use it if you want to make an accurate comparison. You can do this with ffmpeg. Simply add these lines after your x264 line in your batch script, making sure to modify the input and output filenames as appropriate, like this.

x264 --qp 0 --preset veryslow --keyint infinite --output-csp i420 --range tv --colormatrix bt470bg --output x264output.mkv input.avs

ffmpeg -i input.avs -c:a flac -compression_level 8 -vn audio.flac
ffmpeg -i x264output.mkv -c:v copy -i audio.flac -c:a copy finaloutput.mkv

Posted: 3/9/2012 1:47 PM
(Edited: 3/9/2012 1:57 PM)

Quote

Nahoc

Editor, Experienced player (852)

Joined: 8/12/2008
Posts: 845
Location: Québec, Canada

Please take some of your time to check and post in previous threads instead of always creating new ones: http://tasvideos.org/forum/viewtopic.php?t=10813

Posted: 3/9/2012 1:54 PM

Quote

Lex

Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada

To be fair, the fact that he posted this thread caused me to want to help. I don't think I would have chipped in so much information without this thread. Also, that thread doesn't mention ZMBV at all.

Posted: 3/9/2012 4:12 PM

Quote

feos

Site Admin, Skilled player (1236)

Joined: 4/17/2010
Posts: 11274
Location: RU

Let the mod merge these 2 please.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 3/9/2012 4:15 PM

Quote

natt

Editor, Emulator Coder, Site Developer

Joined: 5/11/2011
Posts: 1108
Location: Murka

Lex wrote:

If you must use h264, you should give x264 RGB data because it now does the color conversion properly. You should be using bt470bg color matrix for everything to accurately reproduce colors when converting from RGB to YUV and back to RGB. YouTube will do the conversion to RGB properly with a full-range YUV 4:2:0 video using bt470bg color matrix. The quality should be lossless so you don't get YouTube transcoding lossy to lossy.

Youtube can NOT handle full range YUV input properly. The same test clip as was used in the JWplayer test, uploaded to youtube. In this case the levels are wrong no matter what the flash acceleration level is, so youtube broke the levels in their transcode process. http://www.youtube.com/watch?v=S0Eh2fseKec

Posted: 3/9/2012 4:16 PM

Quote

Lex

Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada

Ah. All the more reason to just use RGB ZMBV, which YouTube does handle properly. I'll fix my x264 line then, and my post. Done.

Posted: 3/9/2012 5:01 PM

Quote

creaothceann

He/Him

Editor

Joined: 4/7/2005
Posts: 1874
Location: Germany

Lex wrote:

ffmpeg -c:a flac -compression_level 8 -c:v zmbv -i input.avs output.mkv

I get errors with that:

ffmpeg version N-38622-g1eabd71 Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar  7 2012 00:18:03 with gcc 4.6.2
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-ru
ntime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --enable-libope
ncore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --en
able-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger -
-enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwben
c --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-
libxvid --enable-zlib
  libavutil      51. 42.100 / 51. 42.100
  libavcodec     54. 10.100 / 54. 10.100
  libavformat    54.  2.100 / 54.  2.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 63.100 /  2. 63.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
Input #0, avs, from 'tmp.avs':
  Duration: 00:39:02.36, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: zmbv, rgb24, 640x480, 589824 kb/s, 60 tbr, 60 tbn, 60 tb
c
    Stream #0:1: Audio: flac ([1][0][0][0] / 0x0001), 44100 Hz, 2 channels, s16,
 1411 kb/s
Incompatible pixel format 'rgb24' for codec 'libx264', auto-selecting format 'yu
v420p'
[buffer @ 01C146E0] w:640 h:480 pixfmt:rgb24 tb:1/1000000 sar:0/1 sws_param:
[buffersink @ 01C1CEE0] auto-inserting filter 'auto-inserted scale 0' between th
e filter 'src' and the filter 'out'
[scale @ 01C1D180] w:640 h:480 fmt:rgb24 -> w:640 h:480 fmt:yuv420p flags:0x4
Incompatible sample format 's16' for codec 'libvorbis', auto-selecting format 'f
lt'
[libx264 @ 01C14280] using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisali
gn LZCNT
[libx264 @ 01C14280] profile High, level 3.1
[libx264 @ 01C14280] 264 - core 120 r2146 bcd41db - H.264/MPEG-4 AVC codec - Cop
yleft 2003-2011 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deb
lock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 m
e_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chro
ma_qp_offset=-2 threads=9 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_c
ompat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 we
ightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=
0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4
ip_ratio=1.40 aq=1:1.00
Output #0, matroska, to 'tmp.mkv':
  Metadata:
    encoder         : Lavf54.2.100
    Stream #0:0: Video: h264, yuv420p, 640x480, q=-1--1, 1k tbn, 60 tbc
    Stream #0:1: Audio: vorbis, 44100 Hz, 2 channels, flt
Stream mapping:
  Stream #0:0 -> #0:0 (zmbv -> libx264)
  Stream #0:1 -> #0:1 (flac -> libvorbis)
Press [q] to stop, [?] for help
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
[zmbv @ 0003A980] Error! Got no format or no keyframe!
Error while decoding stream #0:0
[flac @ 01BF7EC0] invalid sync code
[flac @ 01BF7EC0] invalid frame header
[flac @ 01BF7EC0] decode_frame() failed
Error while decoding stream #0:1
...

Input=Avisynth, colorspace=rgb32, latest ffmpeg build from that site.

Posted: 3/9/2012 6:28 PM

Quote

Lex

Joined: 6/25/2007
Posts: 732
Location: Vancouver, British Columbia, Canada

Oh, my mistake. It was just a matter of ffmpeg command line arg ordering. The working line is this:

ffmpeg -i input.avs -c:a flac -compression_level 8 -c:v zmbv output.mkv

WTF. Never mind. Apparently, ffmpeg's ZMBV encoder can only encode 8-bit paletted ("pal8" pixel format in ffmpeg) video. I know ZMBV supports higher bit depths and works wonderfully with YouTube (example 1080p video I uploaded as ZMBV), but ffmpeg's ZMBV encoder fails.

Posted: 3/9/2012 11:01 PM

Quote

natt

Editor, Emulator Coder, Site Developer

Joined: 5/11/2011
Posts: 1108
Location: Murka

Looks like it hasn't been mentioned in this thread, but a huge spacesaver (and quality increaser, for the "original" transcode) is to pointresize to 8x8 and let youtube handle the aspect correction at playtime. More details here: http://tasvideos.org/forum/viewtopic.php?p=303545#303545