Posted: 5/22/2011 4:03 AM
Post subject: Lossless & Lossy Blind Test #2
Quote
GabCM
He/Him
Joined: 5/5/2009
Posts: 901
Location: QC, Canada
As I said, I've made another Blind Test. I've encoded the same part of a classical music that even your unborn baby knows. That tune was taken from a classical music compilation CD. It was ripped in WAV format. Then, I've taken one minute of it, and I did the stuff with it. Click here to download all 15 FLAC files in a ZIP. Each file is encoded differently before it was encoded into FLAC, except one, which is simply encoded in FLAC directly from the WAV. If you're interested, you can listen to each of these FLAC files, and vote on the one that sounds the best for you. If you can't hear a difference between each of these files, there's an option for that as well. I'll reveal the list of all the process behind each FLAC files later, in a spoiler. If you're going to try this, thank you.
Posted: 5/22/2011 6:35 AM
(Edited: 5/22/2011 7:55 AM)
Quote
Bobo_the_King
Player (79)
Joined: 8/5/2007
Posts: 865
I mentioned this in the other topic, but I'm using Harmon/Kardon speakers in my HP zd7260us laptop. They're nice speakers, but I probably would do better with headphones. I'm not much of an audiophile and my speakers aren't the greatest, but I gave it my best shot. I homed in on 00:40 in the piece, when the violin nears its highest pitch and volume. My reasoning was that the higher overtones tend to be underrepresented in the lossier formats. Therefore, in the lossless format, this section should be almost painful to listen to with the volume high enough. With this method, I had some success, but I admit it was mostly guesswork. Track 8 struck me as the worst (I hope I don't eat those words later) and tracks 6 and 14 were close to the best. I ultimately chose track 14 as the best among them, and I'd be interested to hear if track 8 is indeed one of the lossier formats. Honestly, though, this was very difficult for me and I think you could use whichever format has the highest compression ratio for encoding. Now I'd like to open these up in Audacity and subtract them from each other to see exactly where the differences lie. I expect this won't work well because the human ear isn't sensitive to phase differences, so the lossy formats won't necessarily be in phase with each other or the lossless format. In fact, from my understanding of signal processing, the clearest giveaway should be very sudden changes-- delta functions and finite impulses. I know it's a little soon to ask for another test, but for the next one, you might try a "song" with a lot of clicks in it. Even a track with a single click should illuminate very interesting differences between formats. Edit: (In reply to sgrunt.) Haha, I should have just picked "I can't tell". I'm no good at this.
Posted: 5/22/2011 7:24 AM
Quote
sgrunt
He/Him
Emulator Coder, Former player
Joined: 10/28/2007
Posts: 1360
Location: The dark horror in the back of your mind
So, here's my take on this: There's a bit of a spread in quality - probably the worst are tracks 11 and 15, which you can hear in the fanfare starting at 0:15, as the individual instruments are harder to picker, and the high-pitched work starting at 0:31 sound decidedly quieter and subdued, not to mention less clear. At the top end I'd have to say tracks 1, 4, 5, 6, and 8 sound the clearest, with probably 8 getting the nod for top spot, though there's really not much of a difference between those five. EDIT: having read the above post (note: I did this *after* my own tests), I'm even more curious to see what Track 8 turns out to be, and if it proves to be divisive.
Posted: 5/22/2011 2:23 PM
Quote
juef
He/Him
Player (131)
Joined: 1/29/2007
Posts: 205
Location: Québec, Canada
I'd be talking out of my behind if I said I could hear a difference between the 15 files. Somehow though, I've managed to in other blind tests that feature the kind of music I listen to more frequently.
Posted: 5/24/2011 7:17 PM
Quote
GabCM
He/Him
Joined: 5/5/2009
Posts: 901
Location: QC, Canada
Well this one doesn't have a lot of feedback. I guess 15 audio files was a bit too much. Anyway, here is the encoding process behind each file. Audio 1.flac = WAV -> OGG 320 CBR -> FLAC Audio 2.flac = WAV -> OGG 128 VBR -> FLAC Audio 3.flac = WAV -> MP3 320 CBR -> FLAC Audio 4.flac = WAV -> AAC 320 CBR -> FLAC Audio 5.flac = WAV -> AAC 320 VBR -> FLAC Audio 6.flac = WAV -> FLAC Audio 7.flac = WAV -> OGG 128 CBR -> FLAC Audio 8.flac = WAV -> OGG 320 VBR -> FLAC Audio 9.flac = WAV -> OGG 320 CBR -> AAC 256 CBR -> MP3 192 CBR -> FLAC Audio 10.flac = WAV -> AAC 128 VBR -> FLAC Audio 11.flac = WAV -> MP3 128 VBR -> FLAC Audio 12.flac = WAV -> MP3 320 VBR -> FLAC Audio 13.flac = WAV -> AAC 256 CBR -> MP3 192 CBR -> FLAC Audio 14.flac = WAV -> AAC 128 CBR -> FLAC Audio 15.flac = WAV -> MP3 128 CBR -> FLAC I like how nobody chose Audio 6. I guess this proves my point. People wouldn't hear the differences between lossless and lossy. It may depend on the device used to listen as well. But three answers isn't much. I guess I'll put this blind test on my YouTube channel.
Posted: 5/24/2011 11:05 PM
Quote
Bobo_the_King
Player (79)
Joined: 8/5/2007
Posts: 865
I don't know if I should feel better that I narrowed it down to 6 and 14, one of which was the lossless version or worse that 14 had a lower bitrate and 8 had a relatively high bitrate. Ultimately, I just couldn't tell the difference between all 15.
Posted: 5/24/2011 11:20 PM
Quote
sgrunt
He/Him
Emulator Coder, Former player
Joined: 10/28/2007
Posts: 1360
Location: The dark horror in the back of your mind
Well, I lost on the 1-in-5 pick, but at least I picked five that were all high quality.
Posted: 5/25/2011 3:55 AM
Quote
feos
Site Admin, Skilled player (1236)
Joined: 4/17/2010
Posts: 11274
Location: RU
NES TAS of 2011
Audio 4.flac = WAV -> AAC 320 CBR -> FLAC Audio 5.flac = WAV -> AAC 320 VBR -> FLAC Audio 6.flac = WAV -> FLAC Even the spectral view didn't show any difference between these 3. Which are the best by eye. But my ear didn't allow me to compare closely so much files at once.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.