Post subject: Encoders: Compressing JPEG
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
I've noticed that we haven't been doing much to compress our JPEGs. Therefore I created a new application, Crushed JPEG, which can convert various images to Crushed JPEGs. Usage is: crushedjpeg input.image output.jpg Or Usage is: crushedjpeg input.image output.jpg 85 The last parameter here is quality to preserve (0 - 100). On default settings, I cut several of our JPEGs in half, and didn't notice any significant quality loss. Please play with this and post comments. Edit: While this can compress from JPEG, it's always best to compress directly from a lossless source (yes, from PNGs is supported). Edit 2: I'm not compatible with 3D, so I don't understand what I'm looking at, so I'm not the best judge on quality here, but with default settings: 2443M.jpeg.jpg: 43827 2443M.jpeg2.jpg: 9816 Difference: -34011 I can't tell the difference, and that's a huge size savings. (Which would work better if done from a lossless original source) (If you can tell the difference, you're free to tweak the quality setting.) Edit 3: Another test with lossy source using default settings (would be nice if someone can test with lossless source). 2376M.png.jpg: 43702 2376M.png2.jpg: 17759 Difference: -25943 Edit 4: Same deal. 2587M.jpg: 43548 2587M2.jpg: 8336 Difference: -35212 2562M.jpg: 39181 2562M2.jpg: 22351 Difference: -16830 2501M.jpg: 44861 2501M2.jpg: 12611 Difference: -32250 2089M.png.jpg: 43029 2089M.png2.jpg: 15128 Difference: -27901 2009M.jpg: 37315 2009M2.jpg: 14375 Difference: -22940
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
At least in the Luigi screenshot there's a quite remarkable visual difference (the more compressed one is clearly more blurred). Not that anybody would notice in practice...
AntyMew
It/Its
Encoder, Player (35)
Joined: 10/22/2014
Posts: 425
Comparisons between lossless, 85, 95, and normal jpeg: EDIT: Fuck dropbox normal jpeg, lossless, 95, 85 lossless, 95, normal jpeg, 85 85, 95, lossless, normal jpeg
Just a Mew! 〜 It/She ΘΔ 〜
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
Those images are not showing up (at least here).
fsvgm777
She/Her
Senior Publisher, Player (226)
Joined: 5/28/2009
Posts: 1213
Location: Luxembourg
I've tested a bit with Metroid Prime Pinball (specifically, the publication screenshot). Lossless PNG (for reference): Now, for any of the cases below, left is regular, right is compressed. 100 quality JPG: (regular is 125749 bytes, compressed is 124178 bytes, 1571 bytes saved) 82 quality JPG: (regular is 49980 bytes, compressed is 43674 bytes, 6306 bytes saved) 77 quality JPG: (regular is 44322 bytes, compressed is 38110 bytes, 6212 bytes saved) The input was always the reference PNG.
Steam Community page - Bluesky profile Oh, I'm just a concerned observer.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Looking good I can't tell the difference. We may even be able to take the number lower.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Default quality is 75, and here's notes from the manual:
       The -quality switch lets you trade off  compressed  file  size  against
       quality of the reconstructed image: the higher the quality setting, the
       larger the JPEG file, and the closer the output image will  be  to  the
       original  input.   Normally  you want to use the lowest quality setting
       (smallest file) that decompresses  into  something  visually  indistin-
       guishable  from  the original image.  For this purpose the quality set-
       ting should be between 50 and 95; the default  of  75  is  often  about
       right.  If you see defects at -quality 75, then go up 5 or 10 counts at
       a time until you are happy with the output image.  (The optimal setting
       will vary from one image to another.)

       -quality  100 will generate a quantization table of all 1’s, minimizing
       loss in the quantization step (but there is still information  loss  in
       subsampling,  as  well  as  roundoff error).  This setting is mainly of
       interest for experimental purposes.  Quality values above about 95  are
       not  recommended  for normal use; the compressed file size goes up dra-
       matically for hardly any gain in output image quality.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
Woah! Gotta spend some time on testing. But I already have a usage suggestion: can you make it work by drag'n'dropping the file on a program icon? For quality input, it could just open a command prompt and ask for user numbers. This way using it will take seconds, and tons of images could be processed in a row with various settings (which would be needed to determine the best quality to size ratio for a particular screenshot). Anty-Lemon: please put spaces between images, it unbreaks the page for narrow browser windows.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Or you could do it yourself. Download crushed85.bat
Language: batch

crushedjpeg "%1" output85.jpeg 85
Download crushed80.bat
Language: batch

crushedjpeg "%1" output80.jpeg 80
Download crushed75.bat
Language: batch

crushedjpeg "%1" output75.jpeg
Download crushed70.bat
Language: batch

crushedjpeg "%1" output70.jpeg 70
Edit, or a single combined script: Download crushedmany.bat
Language: batch

crushedjpeg "%1" output85.jpeg 85 crushedjpeg "%1" output80.jpeg 80 crushedjpeg "%1" output75.jpeg crushedjpeg "%1" output70.jpeg 70
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
That says to me "crushedjpeg is not a command, program, or a batch file"... Yes, I'm running it just near crashed.exe.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
That says to me "crushedjpeg is not a command, program, or a batch file"...
You have to put it in the same directory as crushedjpeg.exe and run it from there. I'll look into adding a massive batch program too.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Okay feos, you can redownload http://nach.tasvideos.org/crushedjpeg.7z It includes batch.exe which you can drag and drop multiple images onto, and it will create for each image compressions in intervals of 5 from 60 to 95.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
Does cjpeg have an option to set subsampling to 4:4:4? Without it no matter how I try, the loss is visible against GIMP, even if the file size/quality is greater than GIMP's.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Player (36)
Joined: 9/11/2004
Posts: 2630
Build a man a fire, warm him for a day, Set a man on fire, warm him for the rest of his life.
Experienced player (690)
Joined: 11/23/2013
Posts: 2233
Location: Guatemala
Noticed many differences, just because it's me. What I do to compress JPEGs is to use Gimp and save as a JPEG and then other saving settings and stuff.
Here, my YouTube channel: http://www.youtube.com/user/dekutony
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos wrote:
Does cjpeg have an option to set subsampling to 4:4:4? Without it no matter how I try, the loss is visible against GIMP, even if the file size/quality is greater than GIMP's.
That should be an option for convert, part of ImageMagick, not cjpeg. If you can find the correct parameter for it, I'll add it. Edit: Actually now that I think about it, cjpeg should be the one to set it, not sure if there's an option, we'll have to look for it.
OmnipotentEntity wrote:
Have we tested this against MozJpeg? https://blog.mozilla.org/research/2014/07/15/mozilla-advances-jpeg-encoding-with-mozjpeg-2-0/
If you'd look at what this does and what MozJpeg is, you'll see your question makes no sense.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
New version uploaded. This now uses DCT float and 4:4:4. I'm noticing significantly higher visual quality at lower quality settings. I also fixed a bug in cjpeg misparsing the dct setting. Idiots left out {} in one place. Listen carefully programmers, even though {} is optional in most languages, NEVER LEAVE THEM OUT. Because of this, batch.exe now also outputs from 50 and up. fsvgm777: Want to look at Metroid Prime Pinball again? May get even higher quality at smaller size.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Using Anty-Lemon's PNG's and "regular JPEG" from above, and new version of Crushed JPEG: Order is PNG, Crushed JPEG, "Regular JPEG": 26,904 -> 18,105 31,118 -> 20,537
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
Download test.bat
Language: batch

:: Supposed to be ran by itself, no drag'n'dropping :: Hardcoded source name is 0.png, change it if you wish @echo off : start echo Type 0-99 for the desired quality echo Type r to roll by quality step echo Type q to quit set /p quality= if "%quality%"=="q" goto end if "%quality%"=="r" goto bystep goto normal : bystep echo Type the step size set /p step= echo Type 0-99 for where to stop set /p stop= set /a quality=99 : loop crushedjpeg 0.png %quality%.jpeg "%quality%" set /a quality-=%step% if %quality% GTR %stop% goto loop : normal crushedjpeg 0.png %quality%.jpeg "%quality%" goto start : end
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
Did some real testings now. For games that make screenshots go over 45kb, it's hardly possible to get a jpeg with no visible loss under the site's limit. If it's not some exceptional case (like png being smaller than the best sub 45kb jpeg, or best jpeg being 40kb), it'd be best to just pick the first one that's below the limit, since in most cases there will already be compression artifacts there. However, to know which one fits best, you'd need to create jpegs with quality step of 1 (down to some 70). Here are the images that I would use for publication. Format: Game: quality / jpeg size in bytes. Comment PNG JPEG images Abe: 94 / 44,685. "0" at the monitor starts fading at lower qualities. F-Zero: 92 / 48,624. Artifacts at the road stripes are already present on the first sub 50kb jpeg. Pinball: 76 / 50,578 (woops). No comments. Luigi 1: 98 / 40,649. Artifacts appear around the ligntning if you go under 98, but 98 is already small enough. Luigi 2: 97 / 42,458. BG on the right gets blurred and blocked under 97. Crash: 94 / 47,341. Hard to see, but artifacts around Crash and on the sky. So here are the steps that I'd be using when making a publication screenshot for such a game:
    1. Check the compressed PNG size. If sub 45kb, publish it. If it's bigger: 2. Make a bunch of jpegs from 99 to 70, with quality step of 1. Use the above batch for that. 3. Throw away everything above 45kb. 4. Compare the best one of the rest with the source. If there are artifacts already, publish. Otherwise: 5. Go through all the smaller jpegs to see where artifacts start appearing. 6. Pick the one right before the first artifacted one. Publish.
EDIT: Well, the limit is actually 45kb. Whatever.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
feos, in your opinion, are these images better quality than the ones we are using for publication? (Is the quality per size ratio higher?)
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Site Admin, Skilled player (1254)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
Since we don't have the lossless sources of them anymore, I dunno how to make an educated judgment about it. But looking at how gimp compared to crashjpeg with proper parameters, we might be able to save a few kilobytes over the published jpegs (while not losing quality), because it seems they were compressed with gimp already (praise Ilari).
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Okay, good to hear. Also, the various tools in Crushed JPEG are slated to come out with new releases in the near future with significant improvements (seeing chatter of 5-10% improvements). So if this is a minor improvement now, it'll be a significant improvement then.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
New version out. Uses significantly improved cjpeg, some improvements to jpegoptim, and some tweaks to crushedjpeg wrapper. This should provide more quality at lower file sizes.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
feos wrote:
Does cjpeg have an option to set subsampling to 4:4:4?
Using that subsampling mode indeed increases quality (because it retains more color information). Note, however, that it will probably be useful mostly if the source image is lossless. Re-compressing a 4:2:2 jpeg to a 4:4:4 one is probably not going to increase the quality (because the original color information has already been lost.)