Manufacturers on the web: a double edged sword?

[Fnuckle]

Of course it is and clock references are an extremely poor analogy IMO.

If real differences, eg something you've measured and know to exist aren't identifiable on a blind test, that tells you that the difference while real isn't audible, probably doesn't matter and you can ignore it.

I can (and have) shown for example that small adjustments to EQ of a db or two are clearly and repeatedly identified on a blind test. Drop the variation to 0.25db and it becomes harder, and often impossible to reliably detect.
That simply tells me that we've established where a real difference either matters or doesn't.

Do the same thing sighted and the difference - no matter how small - is always heard becasue the listeners are fully aware of what you've done.

Now, neither test is perfect but I know which I'd being more inclined to rely on.

Trouble is, the test you have described is scientifically pointless. It's laden with potential biases and errors. If you ran this test under lab conditions with certain material, you'd find panels fail to find differences even with 6dB changes in an EQ curve, but with other material, 3dB would be the threshold. Anything less than that is fantasy land, from a scientific perspective.

Play different EQ curves in one order and you get one set of results, play them again in another order and you'll get a completely different set of results. Play more than A-B and you bias listeners toward the middle presentations.

So your 'real difference' is no such thing. It can confidently be ignored too, even though it appeared under blind conditions, because those conditions weren't blind enough. In other words, it's no more or less robust than someone being shown what they are listening to before they listen to it.

Worse, given the huge range of variation in tests depending on how you stage the test, how can you be sure that if something fails to pass your test that it isn't that your test is throwing out a false negative? You are making a huge assumption on the validity of your test, based purely on the test results. That way lies all manner of random results.

From my experiments, no one methodology is overarching. The only way to test things is to do use as broad a spread of tests as possible and weight their scores according to their reliability.

But this is a topic that is probably best broken out of this thread, as it has no relevance to the OP.

 

[RobHolt]

Trouble is, the test you have described is scientifically pointless. It's laden with potential biases and errors. If you ran this test under lab conditions with certain material, you'd find panels fail to find differences even with 6dB changes in an EQ curve, but with other material, 3dB would be the threshold. Anything less than that is fantasy land, from a scientific perspective.

Play different EQ curves in one order and you get one set of results, play them again in another order and you'll get a completely different set of results. Play more than A-B and you bias listeners toward the middle presentations.

So your 'real difference' is no such thing. It can confidently be ignored too, even though it appeared under blind conditions, because those conditions weren't blind enough. In other words, it's no more or less robust than someone being shown what they are listening to before they listen to it.

Worse, given the huge range of variation in tests depending on how you stage the test, how can you be sure that if something fails to pass your test that it isn't that your test is throwing out a false negative? You are making a huge assumption on the validity of your test, based purely on the test results. That way lies all manner of random results.

From my experiments, no one methodology is overarching. The only way to test things is to do use as broad a spread of tests as possible and weight their scores according to their reliability.

But this is a topic that is probably best broken out of this thread, as it has no relevance to the OP.

That's all well and good, and we can discuss different degrees of 'blindness' in testing methods in order to hone a test, but that is beside the point.

I am arguing that even a pretty basic blind test is preferable to a sighted one - not that it is perfect. We don't have the luxury of perfection.
But you are over-complicating the issue IMO.

We have discussed this before at length - I've just sussed *who you really are - but by all means get a thread going on the subject.



* and it stays private so don't worry!

 

[Dave Simpson]

Legit question for those of you who believe current scientific testing is the absolute truth when it comes to what we hear....

What happens when your test tells you can't be hearing what you're hearing? Do you tell yourself it's a delusion every time you hear the phenomenon thereafter (and for the rest of your life?)

regards,

dave

 

[dudywoxer]

Manufacturers on the web: a double edged sword,

if the previous 8 pages are indication of the industry on forums, its a single edge sword, all pointing inwards.
________
WHITE WIDOW SEEDS

 

[RobHolt]

Legit question for those of you who believe current scientific testing is the absolute truth when it comes to what we hear....

What happens when your test tells you can't be hearing what you're hearing? Do you tell yourself it's a delusion every time you hear the phenomenon thereafter (and for the rest of your life?)

regards,

dave

Not had the experience yet, but were it to happen I'd just widen the test regime and keep looking.

People commonly test using a very rigid and basic regime.

So, if you get say two dacs with 0.5% THD they could still sound different, because within that headline figure you can drill down and examine the distribution of the distortion. And what does the IMD look like for example, and how does the op amp on the output react to the pre amp load?
How often in dac, phono stage, tuner or pre amp tests do you see testing into different load conditions - rarely if ever.
For example, some op amps will ring into capacitive interconnects - others won't

Similarly, you might have two amplifiers which on test have the same frequency response, distortion spectrum, current and voltage capability, noise levels etc.
You connect them to your speakers and discover that one sounds different to other. So you dig a bit deeper and discover that one has an inductor in series with the load and that your spealers fall from a nominal impedance of 8 ohms to 3 ohms at HF. The inductor will give the amplifier high output impedance at HF and this will cause the response to fall at HF. Typically, labs express output impedance as damping factor and measure it a low freqencies.......
So the test is looking in the wrong place in this instance.

IME you can always find the answer - but you have to look in the right places.

 

[Dave Simpson]

Great point Rob.

I suspect much of it is looking in the wrong place but how frequently is that recogonized vs declaring something heard is a delusion? I'll bet the ability to do this is what separates the most promising designs from the unfullfilling ones.

regards,

dave

 

[Richard Dunn]

Great point Rob.

I suspect much of it is looking in the wrong place but how frequently is that recogonized vs declaring something heard is a delusion? I'll bet the ability to do this is what separates the most promising designs from the unfullfilling ones.

regards,

dave

Dave I have been looking for over 30 years for genuine correllation between specs and music, and I really wanted to find them. They don't exist apart from on a gross level. Nothing of any subtlety is shown, only the ears can do that. Test gear tells me something is working as designed or has been repaired correctly. To actually discover with it why the differences occur is like chasing your shadow. Now we are making advances in digital simulation and there may be a point where we can get nearer but at the moment not. You want to know what good hi-fi is then you listen to it as music and the better music represents the better hi-fi, it is that simple.

I can show it in a simple test - two identical amplifiers one with 300va transformer and one with 600va, now the 300va is already way over top for the electrical requirements of the amplifier and yet the amp with the 600va is clearly musically better - yet they spec and measure identically - objectivist should explain that. AND I can quote similar things with every other part of the audio chain including cables. In fact tommorrow I will show it blind with digital interconnects at the bake-off.

 

[RobHolt]

You cannot prove 'musically better' - it is a purely subjective term.
All you can hope to do with those two amplifiers is present them blind and perform multiple A/B listening tests and see if you get any pattern emerging when detecting a difference.

Differences between digital interconnects are measurable. I can show you a pair of optical cables and demonstrate a difference, depending on the dac.
What a dac does with the signal received varies hugely depending on how it deals with the data, and particularly poor data. For example, some re-clock and some don't. Some comprehensively process the data with DSP and others don't.

 

[Richard Dunn]

You cannot prove 'musically better' - it is a purely subjective term.
All you can hope to do with those two amplifiers is present them blind and perform multiple A/B listening tests and see if you get any pattern emerging when detecting a difference.

Differences between digital interconnects are measurable. I can show you a pair of optical cables and demonstrate a difference, depending on the dac.
What a dac does with the signal received varies hugely depending on how it deals with the data, and particularly poor data. For example, some re-clock and some don't. Some comprehensively process the data with DSP and others don't.

This is what you don't seem to understand, I don't need to verify it to anyone else, they either hear it or they don't. I couldn't give a toss about proving anything. I hear it and anyone with a pair of ears and no agenda will hear it on my system, it is so simple.

You may have discovered some tests to satisfy you ego and desire, good for you, I don't need them as I can perceive reality very easily. I do it at concerts and I do it with my hi-fi.

 

[RobHolt]

If you don't give a toss that's fine.

I and plenty of others definitely do give a toss, and we've been around the block enough times to have seen all of the tricks.

'They hear it or they don't' just isn't good enough in my book.
People can be encouraged and persuaded to hear all sorts. Unless that opinion had been arrived at solely using their ears I will always question it.

Note I said 'question' and not dismiss.

 

[Richard Dunn]

If you don't give a toss that's fine.

I and plenty of others definitely do give a toss, and we've been around the block enough times to have seen all of the tricks.

'They hear it or they don't' just isn't good enough in my book.
People can be encouraged and persuaded to hear all sorts. Unless that opinion had been arrived at solely using their ears I will always question it.

Note I said 'question' and not dismiss.

Here we go again "the tricks", you must suffer with some sort of audio paranoir, always looking for people who are playing tricks on you, that is very sad. I don't need to play tricks on anyone and I never have, because I don't give a toss. I build equipment for me, if others like it they buy it, if they don't so what, they buy something else, they are happy. All this anally retentive intellectual masturbation with boxes with dials and probes just gets in the way of reality, which is the music, all people need to do is listen to music and decide which is the best music. You people are the death of real hi-fi wanting to turn it into some sort of train spotting. I am afraid I just want to design equipment for listening to music and that is what I do with it, once that is done the test gear can sit and gather dust in the corner as far as I am concerned.

 

[RobHolt]

Richard, this isn't about you - 'tricks' refers in this discussion to methods used by many in the industry to sell and as far as I'm concerned this is a general discussion. And yes, sadly it is very common.
It needn't even be deliberate. Simply talking enthusiastically about a product before demonstrating it can influence what is heard.

It actually doesn't bother me a great deal because I've had enough experience of it to see it a mile off but that took a long, long time. It does bother me when I see others sucked in to some of this nonsense though, because it represents a diversion away from the things that really matter onto trivia.

You know all this, its old ground.

 

[Richard Dunn]

Richard, this isn't about you - 'tricks' refers in this discussion to methods used by many in the industry to sell and as far as I'm concerned this is a general discussion. And yes, sadly it is very common.
It needn't even be deliberate. Simply talking enthusiastically about a product before demonstrating it can influence what is heard.

It actually doesn't bother me a great deal because I've had enough experience of it to see it a mile off but that took a long, long time. It does bother me when I see others sucked in to some of this nonsense though, because it represents a diversion away from the things that really matter onto trivia.

You know all this, its old ground.

Simply talking and enthusing about your test gear and how complicted it is and how expensive and how clever you are in knowing how to use it, and on top of that the blind testing, is enough to trick a lot of people into *not* hearing it.

 

[RobHolt]

That sounds awfully complicated.
I just disguise/conceal the product identity and play it.

 

[Richard Dunn]

That sounds awfully complicated.
I just disguise/conceal the product identity and play it.

I don't think you read the post correctly, or else you don't realise that is what *you* are doing.

 

[nando]

i thought emerdale and all were bad enoigh, but this beats the lot,
nando.

 

[flatpopely]

i thought emerdale and all were bad enoigh, but this beats the lot,
nando.

You calling my pint a puff? Outside now :JOEL:

 

[Dave Simpson]

I suspect the best designers research the cause when measurements contradict what's heard including Richard. I also have a feeling few will deny their ears if they must choose between best sound vs measurements. The uninspired who consistently produce unremarkable products just for a living are the exception of course.

 

[RobHolt]

Dave, I usually use a Quad 306 for the main amplifier.
There are many reasons, but they boil down to it being small, stable, a good visual match for the system and of course I think it sounds fine.
But sometimes I need more volts for less sensitive speakers and so a 405/2 gets substituted. Now as you probably know, all sorts has been written about this amplifier and very little of it particularly complimentary.
Now I could leave it in the system permanently as it drives all speakers I own and there would be no need to keep swapping.
But whenever I use the 405/2 I have this nagging feeling that it isn't quite right. I hear some of the supposed 'defects' in the way it amplifies, yet on paper it should be near as dammit the same as a 306.

So a while ago I had both powered up and got a friend to swap input and speaker connections while I left the room. You think I could pick them apart now? - absolutely not!

So even when you think you might have developed some immunity from expectation bias, it still has influence if powerful enough.
Clearly I'm not arguing that folk should go to elaborate lengths to set up complex ABX test regimes every time they buy some hi-fi, but it also isn't unreasonable to alert them to the pitfalls of reading press reviews, listening to forum opinions and attending dealer dems.
Where discussion with manufacturers on the subject are concerned there should be no constraints, because where claims are made that appear illogical or unlikely, they deserve the closest of scrutiny and questioning IMO, and if that means arranging ABX testing and listening panels so be it.

When all said and done, those interested will read and/or participate while those who couldn't care less can go talk about something else, or watch Emerdale, Nando

 

[Richard Dunn]

Dave, I usually use a Quad 306 for the main amplifier.
There are many reasons, but they boil down to it being small, stable, a good visual match for the system and of course I think it sounds fine.
But sometimes I need more volts for less sensitive speakers and so a 405/2 gets substituted. Now as you probably know, all sorts has been written about this amplifier and very little of it particularly complimentary.
Now I could leave it in the system permanently as it drives all speakers I own and there would be no need to keep swapping.
But whenever I use the 405/2 I have this nagging feeling that it isn't quite right. I hear some of the supposed 'defects' in the way it amplifies, yet on paper it should be near as dammit the same as a 306.

So a while ago I had both powered up and got a friend to swap input and speaker connections while I left the room. You think I could pick them apart now? - absolutely not!

So even when you think you might have developed some immunity from expectation bias, it still has influence if powerful enough.
Clearly I'm not arguing that folk should go to elaborate lengths to set up complex ABX test regimes every time they buy some hi-fi, but it also isn't unreasonable to alert them to the pitfalls of reading press reviews, listening to forum opinions and attending dealer dems.
Where discussion with manufacturers on the subject are concerned there should be no constraints, because where claims are made that appear illogical or unlikely, they deserve the closest of scrutiny and questioning IMO, and if that means arranging ABX testing and listening panels so be it.

Where is the pleasure of listening to music in all this!