Mega Lossy v Lossless Test

Which menu contains AAC lossy encoded music?

  • Menu 1 contains AAC music

    Votes: 8 47.1%

  • Menu 2 contains AAC music

    Votes: 3 17.6%

  • Too close to call

    Votes: 6 35.3%
  • Total voters
    17
UK Duty Paid said:
the fact they required a close listen comparison says it all
Click to expand...

That is i think the message that comes from the result and is more important than the division of voters between the answers here.

Some listeners can consistently identify lossy files and the reasons behind that are next on the agenda for exploration.
 
I'm not sure I agree with some peoples interpretation of the results.

I think all this test, and previous ones, show is that a group of 20 people can't tell the difference between lossy and lossless, in this test.

For any one individual it's still a 50/50 choice, where a preference can be expressed, and that is still the same as the luck of the draw. It doesn't demonstrate any individual's ability to tell the difference imo.

What needs to be done is to give each individual, a series of twenty tests, and then see how many they get right. Only that will show whether any individuals can out-perform the group, which as a whole can't tell the difference.

So, no surprises so far, imo. We've known for a long time that on the whole, listeners can't reliably tell the difference between lossy and lossless, in spite of what some claim.

JC
 
RobHolt said:
Some listeners can consistently identify lossy files and the reasons behind that are next on the agenda for exploration.
Click to expand...

Yes, as I get it wrong every time, I am keen to know what to listen for. Tony when can we start the Jedi mp3 identification training?
 
jonesi said:
... as I get it wrong every time, ..
Click to expand...

It isn't really 'getting it wrong' ..... you are presumably, merely expressing a preference.

That isn't 'wrong' ...... plenty of people use AAC or mp3 all the time :)

JC
 
I heard more on the second files, like the way a string was rubbed, so that had more detail & was the reason I voted for 2. Opening bars were enough.
listening via headphones old sennheiser 475's through a dell computer with standard sound card. So nothing special in the equipment.
 
jcbrum said:
So, no surprises so far, imo. We've known for a long time that on the whole, listeners can't reliably tell the difference between lossy and lossless, in spite of what some claim.

JC
Click to expand...

Sort of, yes, but then that is the purpose of the test.
You can devise something more complex - and please do if you you wish, I'll certainly take part - but such tests limit the number of participants quite dramatically IMO. We got 46 doing this test on three sites which gives you some idea of the task you face.

If lossless is 'clearly' superior to 256 AAC you would expect a high number of correct votes.

This test and others tell me that the performance between the two is quite close for most people and that shows two things.
Firstly, those steadfastly refusing to listen to MP3/AAC or whatever at a decent bitrate on a hi-fi systems (as opposed to iPod) are out of touch and dare I say it displaying some audiophile snobbery.

Secondly, it demonstrates that lossy compression technology is now very good and advancing all the time.

It doesn't tell me that these files sound identical or that differences don't exist.
A handful of people get it right every time regardless of the music of codec, but even they will tell you that sometimes it can be close call.
In other words, in this blind test I'm convinced that it proves a differences exists - it proves a positive and not because of the headline result. That is useful to me and the poll and test have served several functions and will open several parallel discussions in due course.
 
jcbrum said:
It isn't really 'getting it wrong' ..... you are presumably, merely expressing a preference.

That isn't 'wrong' ...... plenty of people use AAC or mp3 all the time :)

JC
Click to expand...

If you read the poll options you'd realise that Rob was asking us to identify the AAC from lossless, not which one we preferred.

Regarding the validity of the poll, the way I look at it, most of us were not able to identify lossy from lossless because the differences aren't enough. Although, some listeners are able to repeatedly and reliably able to identify lossy from lossless. As Rob said the lossy compression technology now provides very good results. Also note how many people viewed the thread without voting, I assumed they didn't participate because they were embarrassed about getting it wrong. This is how I interpret the results, no one need agree with me:).
 
Dev said:
Also note how many people viewed the thread without voting, I assumed they didn't participate because they were embarrassed about getting it wrong. This is how I interpret the results, no one need agree with me:).
Click to expand...

5253

2217 on ZG
3036 on PFM
 
RobHolt said:
5153

2117 on ZG
3036 on PFM
Click to expand...


That's amazing.

Some would not participate because of the embarrassment of being wrong, some because of the prejudice that they know which is best and it doesn't need proving, and the inaction of many will be down to sloth.
 
jcbrum said:
I'm not sure I agree with some peoples interpretation of the results.

I think all this test, and previous ones, show is that a group of 20 people can't tell the difference between lossy and lossless, in this test.

For any one individual it's still a 50/50 choice, where a preference can be expressed, and that is still the same as the luck of the draw. It doesn't demonstrate any individual's ability to tell the difference imo.

What needs to be done is to give each individual, a series of twenty tests, and then see how many they get right. Only that will show whether any individuals can out-perform the group, which as a whole can't tell the difference.

So, no surprises so far, imo. We've known for a long time that on the whole, listeners can't reliably tell the difference between lossy and lossless, in spite of what some claim.

JC
Click to expand...


exactly so, very good post. so good I wanted to repeat it with a quote.
 
RobHolt said:
5253

2217 on ZG
3036 on PFM
Click to expand...
I'm not sure about this total Rob. A question for Mark/Tony, if the thread is viewed multiple times, does the count increment? I know it doesn't increment if no one replies, but if someone replies and then members view the thread again I think it does.
 
Dev said:
I'm not sure about this total Rob. A question for Mark/Tony, if the thread is viewed multiple times, does the count increment? I know it doesn't increment if no one replies, but if someone replies and then members view the thread again I think it does.
Click to expand...

The figure is for how many views the thread has received, not the number of unique users who have viewed it, i.e. if a user goes back to view a new post the number is incremented. There is to my knowledge no way of obtaining the unique user figure for a specific thread.

Tony.
 
Lets be very conservative and assume 10 views per person.

Gives us around 500 which is probably closer to a true figure.

Participation will always be reduced in these things because of the need to download.
My point to JC is that the more complicated the test, the more downloading and listener segmentation that is required, the fewer take part.

I hope we got the balance here about right.
 
Very very interesting test and results - another big thank you to Rob for taking the trouble to set it up. I am very interested in this subject, and I look forward to Tony's "How To Spot Lossy Files 101".
(BTW I voted "Too close to call", on PFM though.)
 

Latest posts

Back
Top