Post by Nahoc

Posted: 5/21/2011 2:06 AM
(Edited: 5/21/2011 2:46 AM)

Post subject: Lossless vs Lossy

Quote

GabCM

He/Him

Joined: 5/5/2009
Posts: 901
Location: QC, Canada

I didn't make this thread to change my way to encode audio for TASVideos. It's my own personal research. I'll still use FLAC as the audio codec for my YouTube encodes, because YouTube compresses it anyway. This thread is for everyone. I'd like opinions from anyone, no matter if you're experienced in audio encoding or not at all. If you paid over 9000 dollars on headphones, or if you're using these earbuds that come with an iPod, I'd also like your opinion. There are people who pretend that they can hear a difference between lossless audio and lossy audio. What's the difference between them? Lossless audio is encoded from an uncompressed audio, without losing any important data, so you can hear absolutely everything from a sound. The file size is freaking big, but you miss nothing at all. Examples of lossless audio are FLAC and ALAC (Apple Lossless). Lossy audio is encoded from an uncompressed audio, by losing data. It makes waaaaaaaaaay smaller files than lossless audio, but the quality is lowered. But why are lossy audio so much used? It's because (I think that) the average Joe can't hear a difference between lossy and lossless when it comes to simply listening to music. Examples of lossy audio are MP3, OGG and AAC. ---- TL;DR. Let's get to the main point. Today I've found some of my favorite tunes in WAV format. I was like "Yay! Finally! I can hear why FLAC maniacs can hear a difference between lossless and lossy!". Then I've encoded a part of that tune in FLAC, MP3 (320 kbps) and MP3 (128 kbps). I've compared all three of them, and even if I tried to hear a difference, I couldn't. I've tested them in VLC media player, using a Logitech G35 headset. So I got the idea to share these pieces of music with you, so you can compare them and tell me if you hear a difference between these three audio files I've encoded. But they are all encoded in FLAC. Two of them were encoded in MP3 (one is 320, and the other is 128) before they were converted back to FLAC. It's like a blind test, if you wish. Audio 1 Audio 2 Audio 3 I seriously can't hear a difference between these three files. I'd use FLAC for editing and producing, but I don't see the point of using it just to listen to music. It's like 5 to 6 times bigger than a good lossy audio file. Anyway, I'm still storing these tunes in ALAC format in my iPod Touch because it's 64 GB big and it has too much room for my music, videos, photos and apps. So, do you hear a difference between them?

Posted: 5/21/2011 2:08 AM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

You should have made it a blind test, and have people guessing.

Posted: 5/21/2011 2:09 AM

Quote

GabCM

He/Him

Joined: 5/5/2009
Posts: 901
Location: QC, Canada

Aktan wrote:

You should have made it a blind test, and have people guessing.

Nice idea! Let's randomize the stuff. EDIT: I don't think I can rename the files on MediaFire without removing the extension. People would still know which files are encoded in which format.

Posted: 5/21/2011 2:18 AM

Quote

Aktan

Publisher

Joined: 4/23/2009
Posts: 1283

Reconvert them all to FLAC. Aka convert the lossy MP3 to FLAC.

Posted: 5/21/2011 2:19 AM

Quote

Bobo_the_King

Player (79)

Joined: 8/5/2007
Posts: 865

Just re-encode them all in a lossless format. In other words do FLAC -> MP3 -> FLAC and give it a random name. I don't claim to know how to do it, though. You're the encoder, after all. Edit: What Aktan said.

Posted: 5/21/2011 2:22 AM

Quote

adelikat

He/Him

Emulator Coder, Site Developer, Site Owner, Expert player (3598)

Joined: 11/3/2004
Posts: 4738
Location: Tennessee

Yes, please do a blind test. I'd be curious if I could tell the difference

It's hard to look this good. My TAS projects

Posted: 5/21/2011 2:36 AM

Quote

Weatherton

Experienced player (601)

Joined: 10/23/2004
Posts: 706

You could present everything in FLAC but have some be made lossy and then convert back.

Current Project: - Mario Kart 64

Posted: 5/21/2011 2:37 AM

Quote

GabCM

He/Him

Joined: 5/5/2009
Posts: 901
Location: QC, Canada

Ok ok I'll convert the whole stuff in FLAC. EDIT: The blind test is ready.

Posted: 5/21/2011 3:11 AM

Quote

DarkKobold

Skilled player (1637)

Joined: 11/15/2004
Posts: 2202
Location: Killjoy

So, after listening to all three, I put them in a VLC playlist, closed my eyes, and pushed the jump ahead button. I was able to pick the one I thought was best Audio 2 seemed to have the best bass, and most dynamic range, but I could just be hearing things To be honest, I think your music selection of techno for this test wasn't the best. I'd like you to rerun the test on something like classical music.

Sage advice from a friend of Jim: So put your tinfoil hat back in the closet, open your eyes to the truth, and realize that the government is in fact causing austismal cancer with it's 9/11 fluoride vaccinations of your water supply.

Posted: 5/21/2011 3:33 AM

Quote

GabCM

He/Him

Joined: 5/5/2009
Posts: 901
Location: QC, Canada

DarkKobold wrote:

To be honest, I think your music selection of techno for this test wasn't the best. I'd like you to rerun the test on something like classical music.

I'll wait for a couple votes on this one, and then I'll start another blind test with classical music. Thanks for the suggestion.

Posted: 5/21/2011 5:46 AM

Quote

feos

Site Admin, Skilled player (1236)

Joined: 4/17/2010
Posts: 11268
Location: RU

No way! This is distorted sound, how can we hear the difference? You shall get some instrumental (perfectly mixed & mastered, with clever balance of all frequencies) music to compare. Your example is just like taking Raw Black Metal for this test lol. EDIT: BTW, for generated console sound you have no need in lossless or even in 320. V0 is enough, we compared different bitrates for gameripping music.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 5/21/2011 9:52 AM

Quote

Warp

Banned User, Former player

Joined: 3/10/2004
Posts: 7698
Location: Finland

I think that the ability of distinguishing any loss in quality will depend heavily on what kind of audio is being compressed (or not compressed). As feos commented, this particular sample you chose might be a really poor choice because any loss of quality might be indistinguishable from the distortion that is already there in the original. In other words, the compression might not make any audible difference with this particular sample, even though it might make with a different, cleaner sample. Also, a bit like JPEG is optimized for photographs, these audio codecs may be optimized for natural sounds, such as music and speech. They might be not so optimized for very primitive artificial sounds such as chip music. This could potentially mean that compression might not audibly hurt normal music, while it makes an audible difference in chip music. Given the nature of the website, this might be quite relevant.

Posted: 5/21/2011 10:29 AM

Quote

Bobo_the_King

Player (79)

Joined: 8/5/2007
Posts: 865

I have a suggestion for your next test. Post ten randomly named audio files, five identical pairs. For each of the five, use a different lossy (or lossless) encoding. In this case, the goal wouldn't strictly be to evaluate which is "best" among them, but rather to see if someone can distinguish any difference between them. Listeners would be asked to pair up the identical recordings. For the lossiest format(s), I expect they'll be able to do it, but I think the lossless and moderately lossy formats will be functionally indistinguishable. I also recommend that listeners post what speakers or headphones they're using. Although I haven't taken the test yet, I'm using a set of Harman/Kardon speakers that came with my HP zd7260us laptop.

Posted: 5/21/2011 10:32 AM

Quote

Flygon

Joined: 11/4/2007
Posts: 1772
Location: Australia, Victoria

Just as a recommendation, instead of just compressing MP3 files, can I please suggest compressing OGG and AAC files? I've found that OGG in particular ends up having better audio quality at lower bitrates (eg. less background noise) and would be a very fine candidate for such a test.

Posted: 5/21/2011 12:34 PM

Quote

Weatherton

Experienced player (601)

Joined: 10/23/2004
Posts: 706

I listened to these first on a set of Klipsch computer speakers and then on a Sennheiser HD 280 Pro headphones. I'm not accustomed to this type of music so it took a lot of listening to even be familiar enough with the song to know what to look for. I tried singling out the highs and listening for breadth and variety in that. To me, there was not much difference, but I did feel that 1 and 2 were more similar than 3 and that the things that made 3 appear different was greater variety in the highs. The difference, if they were even there, were subtle. Quick note: Based on my own listening experience, 192kbps mp3 is "good enough" to avoid extremely noticeable distortions. However, I rip all of my CD's to FLAC to allow for lossless transcoding into whatever lossy format I need. Also, if I ever edit the songs, I know the end result will be just as good as the start. To me, that's the main benefit of lossless audio. Generally, the quality of the sound has a LOT more to do with the speakers / headphones in use than the quality of the audio file. Though bad speakers may make one make bad assessments / decisions about compression quality...

Current Project: - Mario Kart 64

Posted: 5/21/2011 4:48 PM

Quote

Dada

Joined: 11/22/2004
Posts: 1468
Location: Rotterdam, The Netherlands

Without having read the post itself (not sure if you're giving away the answers), I can definitely say that 1 sounds the worst. 2 and 3 sound pretty much exactly the same. I think 2 might have sounded slightly better, but whatever differences I might have detected are negligible. The most notable difference in 1 is that the high-pitched instrument that you occasionally hear on the background just doesn't sound as good.

Posted: 5/21/2011 5:01 PM

Quote

Kuwaga

Joined: 10/20/2006
Posts: 1248

I think with this sample there's hardly any difference at all. Who's to say that mp3 quality loss couldn't make this particular sample sound better? Personally, I generally find VBR to sound better than strict 320 and I'm not a big fan of flac, as the quality improves only marginally with today's music (where most of the dynamic range is already lost in the studio just to make the music sound louder), but at a much larger file size. For mp3s, there's also a giant difference as to whether you use Joint Stereo, or not. In any case, I'd rank them in 1<3<2, 1 seems to have lost some stereo information and the hi-hat sounds strange, in 2 the bass sounds most dynamic and the panning-heavy part sounds most crispy.

Posted: 5/21/2011 5:58 PM

Quote

Sir_VG

He/Him

Player (39)

Joined: 10/9/2004
Posts: 1911
Location: Floating Tower

In my opinion, there's virtually no difference between 2 and 3. 1 to me seems to be a little off on the high hat compared to the other two.

Taking over the world, one game at a time. Currently TASing: Nothing

Posted: 5/21/2011 7:04 PM

Quote

GabCM

He/Him

Joined: 5/5/2009
Posts: 901
Location: QC, Canada

Thank you for your ideas and your answers. Looks like you prefer Audio 2, but by reading your replies, it seems that the differences are really small. The 2nd most answered option is "I can't tell". So yeah, the differences are small. I think the best use of FLAC is for storage and editing. It can also be useful if you want to upload a video to a streaming site like YouTube, since it compresses the audio. Now, these audio files are being unmasked. Audio 1 = WAV -> 128 kbps MP3 -> FLAC Audio 2 = WAV -> FLAC Audio 3 = WAV -> 320 kbps MP3 -> FLAC I'll make a new blind test of this kind later, using your suggestions and ideas. I'll use another music type, since you're telling me that techno isn't the kind of music to use for this kind of test, because the sounds are synthesized. I don't think Rock can do the trick, since the electric guitar is distorted. So I'll go with DarkKobold's suggestion, which is Classical music, since all of the sounds are recorded authentically. Also, I'll go with Flygon and Bobo the King's idea to encode more files using more audio codecs. Plus, I'll make CBR and VBR audio files, and I'll encode lossy tunes into lossy tunes. So expect more FLAC to download for the next test. I'll make a new thread for the new test.

Posted: 5/21/2011 8:08 PM

Quote

sgrunt

He/Him

Emulator Coder, Former player

Joined: 10/28/2007
Posts: 1360
Location: The dark horror in the back of your mind

Before reading the other posts: I would rank them Audios 2, 3, 1 in that order; I can hear the high frequency notes most clearly in 2. So, I'm going to guess those are FLAC, 320kbps, and 128kbps respectively. EDIT: Seems I was right on the money.

Posted: 5/21/2011 9:34 PM

Quote

Warp

Banned User, Former player

Joined: 3/10/2004
Posts: 7698
Location: Finland

Mister Epic wrote:

I'll use another music type, since you're telling me that techno isn't the kind of music to use for this kind of test, because the sounds are synthesized.

The objection was that this sample in particular uses some distortion/overdrive effects which might be masking the loss in quality.