Mega Lossy v Lossless Test

[jonesi]

I look forward to listening to the next selection of tunes from DJ Holt

 

[Mescalito]

Because it's not a subjective matter: one is better.
Because that's what learning is: training our faculties to discern differences.
Because learning is intrinsically a good thing.

Because many listeners don't use revealing systems, record companies have become negligent about the quality of our recordings, which are often blatantly and cynically degraded from the studio masters.

And because promoting the patent untruth that the difference between compressed and lossy recordings is negligible, or doesn't matter, literally damages music.

It's bad science.

It implies we shouldn't care about recording quality.

It fosters a climate of indifference that has delayed the release of 24-bit recordings for decades.

It makes people buy more cruddy MP3 players.

It sets people down a path of compromise: why not compress the files? Why not play them back on an AV amp, or a phone, or through a sock? They're all convenient, too . . . that way lies musically unsatisfying experiences.

It's a pretty dumb question, ultimately: storage is now SO cheap, why compress the files anyway? Does anyone here believe that compression - in theory, or in practice, will IMPROVE quality? It should sound worse; it does sound worse: the question is: do we care?

Most of what you say is true. However, your argument is predicated on the assumption that compression is an intrinsically bad thing. It is not.

The actual quality difference between, say 256Kb compressed & wav is actually vanishingly small. Yes, if you really concentrate, you can hear compression artifacts. But if you just get down & boogie to the music, which, let's face it, is what it's all about, and stop listening to the sound, which is definitely NOT what it's all about, 256Kb compressed is perfectly acceptable.

I have about 10500 CDs ripped to my NAS. Probably 95% of them are ripped to flac. Thother 5% are 256-320Kbs. And it's the same old story. If I remember that the track is compressed, I tend to stop listening to the music and start listening to the hi-fi. If I don't recall that it is a compressed track, I just enjoy the music.

Regards,

Chris

 

[Cav]

Because it's not a subjective matter: one is better.

Of course it is subjective unless you can define (and then get everyone to agree) what "better" is. If the difference between lossy and lossless is as difficult to identify as this test would suggest, what is your problem.

 

[Coda II]

Here's the issue for me:

I didn't do this batch, didn't get myself sorted really - but did the first.

I don't like listening for artifacts, for evidence of compression etc as I'm not particularly good at it and I find it starts to creep in to my everyday listening.

But, I find my home system very satisfying at present and that's largely down to adjustments to the digital side of things over a period of several years.

I know that I prefer the way it sounds now to the way it sounded two years ago, but I couldn't tell you why.

If you were to play me a single track (in a quality such as the one's that have been used here) I'm pretty sure I couldn't tell you whether it was compressed or not. But listening to two tracks side by side I preferred one to the other.

If I can take a variable out of the listening equation (ie. listen to uncompressed files) then it's another thing not to be bothered about; in the greater scheme of things there are other factors that have a greater impact* but I'd at least like to start with lossless files - having now heard for myself that the differences are there.

*for the car I've started putting mp3s on CD, given the relative effects of road noise vs lossy files, sticking six CDs on one disc wins.

 

[bottleneck]

And because promoting the patent untruth that the difference between compressed and lossy recordings is negligible,

If the average audiophile with expensive audio equipment cannot tell the difference between compresed lossy recordings - then perhaps the difference IS negligable.

I don't have an opinion on the matter, tests will bear out whether people can hear a difference, .... or not.

 

[hubsand]

Of course it is subjective unless you can define (and then get everyone to agree) what "better" is. If the difference between lossy and lossless is as difficult to identify as this test would suggest, what is your problem.

How we identify the differences - that's (partly) subjective. Also room- and system-dependent.
The differences themselves are not: they are simple, objective facts, literally embedded in the data.
It's important not to confuse the two.

AAC coding is 'lossy'. John Atkinson's excellent Stereophile article from March 2008 demonstrates precisely what this means. QED.

Moving on, the only ground for debate is WHY some can hear the difference, and some cannot. Bottom line: half the participants in this test are not using revealing enough equipment in a controlled environment.

If the two files didn't sound sufficiently differentiated to you, upgrade something until they do. When you hear the difference, you've passed a milestone en-route to a much better system.

Or, if you're sick of upgrading stuff, and just want to boogie, don't torment yourself with this kind of poll!

 

[hubsand]

If the average audiophile with expensive audio equipment cannot tell the difference between compresed lossy recordings - then perhaps the difference IS negligable.

I don't have an opinion on the matter, tests will bear out whether people can hear a difference, .... or not.

I'm not sure expense is the issue: obviously some people have flat out super hearing (nature or nurture: debate!), but we have to bear in mind that lots of expensive equipment is a bit useless at being accurate.

I'd like to think that this test, above all, filters those with relatively 'correct' equipment and listening environments, and not those with dud ears. I played both files (effectively double blind) on a pair of Adam A8X, via a DAT2 and a modded Benchmark DAC1: adults, children and pets heard easily identifiable differences.

 

[hubsand]

But if you just get down & boogie to the music, which, let's face it, is what it's all about, and stop listening to the sound, which is definitely NOT what it's all about, 256Kb compressed is perfectly acceptable.

That's reasonable, and sums all there is to say about compressed formats: if you stop listening, they're fine.

They're psycho-acoustically tuned to deliver maybe 80% of the boogie in 20% of the space. Which is quite a lot (but not all) of the boogie.

 

[Cav]

The differences themselves are not: they are simple, objective facts, literally embedded in the data.
It's important not to confuse the two.

Yes, I know that the data is different but you said one was "better" than the other - that is, in the absence of an agreed definition of "better", a subjective opinion. It's a bit like saying Mahler is better than Bach.

 

[RobHolt]

Titian asked me recently to provide some classical pieces of known high quality production.
These are the tracks if others wish to download and listen.
Fantastic quality and wide dynamics.

Format as before - version 1 and 2 with one of them being 256k AAC:

Rutter: Requiem - Pie Jesu . Timothy Seelig; Turtle Creek Chorale: Dallas Womens Chorus. Arabesque Classical:

http://www.mediafire.com/file/ar1t1raw0a97fmf/Requiem 1.wav

http://www.mediafire.com/file/rl7d68gd4o90316/Requiem 2.wav



Danse Macabre. Eiji Oue cond. The Minnesota Orch. Arabesque Classical:

http://www.mediafire.com/file/g3sndxnlezalobr/Danse Macabre 1 .wav

http://www.mediafire.com/file/5ljbjw4ayblw4jz/Danse Macabre 2.wav

 

[hubsand]

Yes, I know that the data is different but you said one was "better" than the other - that is, in the absence of an agreed definition of "better", a subjective opinion. It's a bit like saying Mahler is better than Bach.

Mahler is different to Bach; lossless is - empirically, theoretically, practically, indisputably and absolutely by definition - BETTER than lossy.

If you admit 'I can't hear the difference' the question is: why? Probably because your system isn't revealing enough. There's no mystery here.

 

[RobHolt]

Mahler is different to Bach; lossless is - empirically, theoretically, practically, indisputably and absolutely by definition - BETTER than lossy.

If you admit 'I can't hear the difference' the question is: why? Probably because your system isn't revealing enough. There's no mystery here.

Your first assertion is unquestionably correct.

The second has a range of possible answers and you conveniently omit the most obvious - that the difference might in most cases be vanishingly small enough to pass unnoticed, on any system.

Better only matters is it results in audible benefit.
All else being equal, an amplifier with 1% distortion will sound better than one with 10%.
The same probably won't apply when comparing 0.001% with 0.01% despite what is on paper a huge difference.

 

[hubsand]

Before voting, can we compile a list of things to listen for?
Extracting previously helpful comments from this thread . . .

"I focus entirely on transients. Drums, cymbals, surface noise. I don't know if that's just because I'm very sensitive to the sounds of transients in hi-fi reproduction in general, or because transients are the most complex waveforms, or because the design of the codecs did not place much emphasis on dynamics . . .

Same poster also said: "It seems a number voted based on computer speakers or cheap headphones. Those votes are useless--they poison the pool."

"I prefered the Menu 1 because I had the feeling that the instruments were more spread out, especially the trombones and tuba . . . This gave also the effect of orchestral fullness.
Also I found Menu 2 very compact less "airy" going in the direction of congested. I had the feeling the instruments were crunched in the centre.
Saying this the image of some instruments (flute oboe and clarinet) were more consistent and "real" with Menu 2."

"With MP3s, AACs etc it is the file size that is being compressed, not the dynamic range. The loud bits are just as loud, the quiet bits just as quiet, the thing that is missing is certain detail that has been thrown away in order to make a smaller file . . . lossy compression loses low-level detail . . . to my ears it mainly screws up reverb / depth / space and becomes confused when the going gets tough, i.e. it becomes harder to identify instruments or recording acoustic in busy or noisy passages.

The Jimmy Smith. I really struggled with this one to be honest. I felt there was a little more acoustic space around the applause, the cymbals just a little better defined, a little more sense of things occurring in a plausible acoustic, but it was the best preserved of the tracks I listened to by far, and based upon this track alone I'm not sure I'd have committed and may well have hit the 'don't know' option.

The Jane Monheit. The vocal reverb gave a coherent sense of three dimensional space on the wav and sounded flatter, crunched, and more like a tacky FX unit on the AAC. The piano also sounded more real / better recorded / and in a more believable acoustic space on the wav. I was certain of my choice by now and I'd have happily committed to option 2 without listening to anything else.

RATM. Lossy compression is always pretty crap with heavily distorted guitars, it just loses that 'thing' that makes a well mic'd overdriven Marshall stack sound so much better than a 50 quid distortion pedal. It's also a lot easier to hear the vocal quality and kit metalwork. This was the easiest IMO, the wav just sounds a lot better, cleaner, better separated, more real, the AAC kind of blurs it altogether into one homogenised sound.

Reverb, acoustic space, the 'air' around instruments, all the ultra-low level 'round-earth' stuff as it is this which tends to be mangled or missing in action once the file has been compressed. Concentrate on the vocal reverb on the Jane Monheit track, all the info needed to make a choice is in that acoustic space, then listen to the guitars, vocal and drum kit metalwork on the RATM track, listen to how separated the instruments are, try and figure out how many guitar tracks there are, listen to how the voice fairs when the guitars are really loud, what happens to the cymbals at this point.

That's it! No magic, no golden ears! Just zone in on what the technology buggers up and you'll spot it assuming the replay equipment is up to it."

 

[RobHolt]

No, some people (a small minority as this test and others demonstrate) can spot lossy effects on some tracks on just about any equipment if they specifically listen for it. In other words, unless they can directly A/B they are not aware that the file is lossy.

The majority hear no difference, or do not identify the lossy file as worse.

Equipment 'up to it' is a standard laptop audio out and a set of £50 headphones.

 

[hubsand]

Your first assertion is unquestionably correct.

The second has a range of possible answers and you conveniently omit the most obvious - that the difference might in most cases be vanishingly small enough to pass unnoticed, on any system.

Better only matters is it results in audible benefit.
All else being equal, an amplifier with 1% distortion will sound better than one with 10%.
The same probably won't apply when comparing 0.001% with 0.01% despite what is on paper a huge difference.

I'm sorry but this kind of thinking makes me feel OCD twitchy: it's flat-out wrong. There is no question at all - really - that lossless is in every way 'better' than lossy, unless the criterion we're applying is purely that of file size. That's why the format is called 'lossy'. QED.

It's fair to ask 'HOW lossy?' But we might as well ask: 'do all Chinese and Japanese people look the same?' If your perception isn't well trained, perhaps. If you're Chinese, the question is ridiculous and offensive: the differences are gross and impossible to miss.

Do all red wines taste the same? Very roughly, yes. But are those differences trivial to an oenologist? No: they would claim there is a world of difference between wines that might seem very similar to the rest of us.

Even if we discount the audio equipment and listening room as a factor, there's no way to quantify how important these differences are to people: to some they will be subtle, to others overwhelmingly different; to a few, it's all the same.

I've always tried to follow listeners with good ears, and learn how they listen; what they listen for. Here's hoping this thread makes us all more perceptive listeners!

 

[RobHolt]

I'm sorry but this kind of thinking makes me feel OCD twitchy: it's flat-out wrong. There is no question at all - really - that lossless is in every way 'better' than lossy, unless the criterion we're applying is purely that of file size. That's why the format is called 'lossy'. QED.

It's fair to ask 'HOW lossy?' But we might as well ask: 'do all Chinese and Japanese people look the same?' If your perception isn't well trained, perhaps. If you're Chinese, the question is ridiculous and offensive: the differences are gross and impossible to miss.

Do all red wines taste the same? Very roughly, yes. But are those differences trivial to an oenologist? No: they would claim there is a world of difference between wines that might seem very similar to the rest of us.

Even if we discount the audio equipment and listening room as a factor, there's no way to quantify how important these differences are to people: to some they will be subtle, to others overwhelmingly different; to a few, it's all the same.

I've always tried to follow listeners with good ears, and learn how they listen; what they listen for. Here's hoping this thread makes us all more perceptive listeners!

You are arguing with yourself.

Nobody is disputing that lossless is better than lossy if the sole criteria is to replicate the original file perfectly.

What is certainly up for discussion, particularly in light of blind testing, is just how far you can use lossy compression before the effects become audible. Tests so far indicate that most people have extreme difficulty, and that most fail to identify at the level we've used in these tests. You can draw many conclusions from this but I find this kind of thing fascinating, because audiophiles as a breed will have you believe that they can detect the sound of a gnat farting at the end of the garden

But to go back to your red wine analogy, the oenologist may fail to detect say the reduction of tannin if the quantity reduced is very small. At some point it will become noticeable - and that is all that counts.
The same applies to audio. You can go on refining technical performance chasing ever better specification but at some point the benefit sinks way below audibility.
Our ears are noisy, the dynamic range is poor, the distortion is high and the frequency response highly variable. There comes a point where good enough really is good enough

 

[titian]

Our ears are noisy, the dynamic range is poor, the distortion is high and the frequency response highly variable. There comes a point where good enough really is good enough

Good enough is a personal matter. It's good reminding people about their limits, to make them question about their goals and the way they choose to archieve them.
To impose them what is enough or not goes behond the line.

 

[RobHolt]

Good enough is a personal matter. It's good reminding people about their limits, to make them question about their goals and the way they choose to archieve them.
To impose them what is enough or not goes behond the line.

Good enough can be tested and measured, and i don't mean test equipment but with groups of real live specimens.
When you repeatedly test people and they consistently fail to hear differences beyond a certain point, there is no point going further.

Encouraging and suggesting that people can hear what testing proves they cannot is definitely going beyond the line - in fact it gets to the hear of the audiophile disease.

What tests such at the this (ongoing) help to demonstrate is where the limits of audibility might be.

 

[titian]

Good enough can be tested and measured, and i don't mean test equipment but with groups of real live specimens.
When you repeatedly test people and they consistently fail to hear differences beyond a certain point, there is no point going further.

First it has to be scientifically prooved that the tests are relevant. Then the conclusions out of any test or statistics is personal.

 

[Cav]

I'm sorry but this kind of thinking makes me feel OCD twitchy: it's flat-out wrong. There is no question at all - really - that lossless is in every way 'better' than lossy, unless the criterion we're applying is purely that of file size. That's why the format is called 'lossy'. QED.

If the definition is that one file contains less data than another then the larger will be ââ'¬Å"betterââ'¬Â. But that is irrelevant if, when listening, there is no discernable difference.