anyone in LA want to help me do a blind test?

Discussion in 'High End Audio' started by Michael Mossey, Mar 22, 2005.

  1. I live in Pasadena, CA. I'm interested in doing blind tests on
    interconnect cables, using a long-listening protocol rather than a
    quick switching protocol. I've already done 12 trials and scored 9/12
    correct- not statistically significant yet, but I'm learning under what
    conditions I perform better, so there is still a chance I could reach a
    statistically significant positive result. Unfortunately these tests
    take a while and I've lost the help of my prior assistant. If you want
    to help, I'd also help you with blind tests in trade. Anyone
    interested?

    If so, don't reply to this email. It is a free account to receive
    spam. Reply to

    mpm
    At
    alumni
    Dot
    caltech
    Dot
    edu

    -Mike

    PS. or post here
     
    Michael Mossey, Mar 22, 2005
    #1
    1. Advertisements

  2. Michael Mossey

    Bernie Guest

    I live in Pasadena also. Sounds like a fun project--fraught with the
    possibility of frustration. Oh, boy! Feel free to e-mail me at
    .

    Cheers, Bernie
     
    Bernie, Mar 23, 2005
    #2
    1. Advertisements

  3. Michael Mossey

    ---MIKE--- Guest

    I'm not in LA so I can't participate in your test. I fail to see how a
    long term test can show anything. I notice differences in the sound of
    my system from day to day without making any changes. Variables such as
    temperature, humidity, atmospheric pressure, whether I blew my nose too
    hard, my level of fatigue, mood, etc. make greater differences than any
    change of cables would (unless one of the cables was defective).


    ---MIKE---
     
    ---MIKE---, Mar 24, 2005
    #3
  4. You might be right, but I need to do the test to find out.

    -Other Mike
     
    Michael Mossey, Mar 24, 2005
    #4
  5. The atmospheric pressure, humidity, state of your sinuses, etc., should all
    be uncorrelated
    with which of the two cables being compared is in use. So those random
    variables should
    not invalidate a long enough series of tests with a significant deviation
    from chance.

    I'm not a "believer" in high end cable myself but I'm all in favor of tests
    like this being done.
    After all, I've been wrong about things before.
     
    John P. Green, Mar 24, 2005
    #5
  6. Michael Mossey

    Scott Guest

    While Michael does say he uses a "long-listening protocol" rather than a
    "quick switching protocol", he does not define these terms. It isn't
    possible to know if he only listens to one pair of cables per hour, per
    day, or per week. As such, we can't assume environmental or internal
    variables would significantly alter the outcome. Even switching cables
    once per hour can "take a while" if one is trying to complete exhaustive
    tests.

    Even *if* environmental or internal changes alter perceptions of sound,
    these variables are constantly changing and could impact equally upon
    "quick switch" protocols provided the research took place across a
    period of days, weeks or months. In fact, changes in mood or blowing
    one's nose too hard can occur during research taking place in the course
    of a single afternoon.

    The key issue isn't whether there are variables that can alter
    outcome/value of the dependent variable. The issue is how those
    variables are distributed through the study (i.e. randomly vs.
    non-randomly). When environmental and internal variables are viewed in
    that light, the results of even the fabled quick-switch/double-blind
    model can be tainted.

    -Scott
     
    Scott, Mar 25, 2005
    #6
  7. I'm not sure myself about high-end cables. I tend to doubt they do
    anything. However, a few years ago I listened carefully and thought
    they were doing something. Was I imagining this? I did an informal
    blind test with the help of a friend, which involved several five
    minute listening sessions to cables. Every four sessions was either
    ABAB or ABBA, unknown to me. We did 16 sessions which gave me four
    chances to guess the order ABAB or ABBA. I guessed right all four
    times. Not only that, but I was quite confident that I knew what I was
    hearing.

    But four trials is not enough to be statiscally sound. Last year I had
    another friend help me in a similar test. I did eight trials. This
    time I got five right and three wrong, which isn't very promising.
    Furthermore, in the last trial I was absolutely confident I knew which
    cable I was hearing, and I was wrong. It showed me how strong
    expectation bias can be.

    But I still wonder if I was just tired. After all, each trial involved
    four listening sessions, so that was 32 sessions, or about five or six
    hours of real time to get through all that.

    So I want to do some more tests.

    -Mike
     
    Michael Mossey, Mar 25, 2005
    #7
  8. That might make more sense if the listeners claimed to be able to hear the
    difference on some days and not others. But in most if not all ABX/DBT,
    the listener *does* claim to hear a difernece *during the test*. IN an
    ABX, the listener basically *has* to report 'hearing' a difference between
    A and B -- that is, during the 'sighted' portion of the test; otherwise
    the test is meaningless -- it's just guessing in the literal sense.
     
    Steven Sullivan, Mar 25, 2005
    #8
  9. As a kind of similar point, I've also thought that the kind of quick
    switch test that switches *while the music is in progress* is nonsense.
    The idea is that you are supposed to tell if you hear a change right
    at the moment you switch--well of course you hear a change, because the
    music itself is changing!

    To clarify my intended protocol (and the same one I've used in past
    tests) : I listen for about five minutes to each cable, enough time to
    settle in and hear it at music. I don't try to compare cables
    directly--I'm not comparing "what I hear now" to my memory--but instead
    I'm just taking notes on my current musical experience without trying
    to think too much about which cable it is. I ask that my assistant do
    four trials, hooking up the cables in the order ABAB, ABBA, BABA, or
    BAAB. This means that I'm getting a good sense of contrast through the
    four trials--at least twice the cable switches to the other one. My
    job after the four trials is to guess whether the middle two were the
    same or different.

    I've done twelve trials overall, but in two separate periods with
    vastly different equipment and setup. 4/4 in the first set, and 5/8 in
    the second set.

    "Quick switch" tests should be also called "contrast tests" because
    they emphasize how one piece of equipment sounds "placed right next"
    (the sound placed in time, that is) to another one. But this doesn't
    represent how people actually listen to music, or how they enjoy it.
    After all, if a piece of audio equipment is beautiful-sounding, (or a
    violin is, or a piano is, or a string quartet is), then it sounds
    beautiful "on its own." You don't need to put it right next to
    something else to be able to hear the beauty. Likewise, if a piece of
    audio equipment has some sort of character--if it is either ugly or
    beautiful--it should sound that way after you've settled in with it.
    It shouldn't be necessary to compare the sound to something else to
    determine this, and it fact I think that contrast tests obscure
    differences because they prevent the listener from settling in and
    hearing a piece of music, over its natural span of time, hearing it as
    actual *music*.

    -Mike
     
    Michael Mossey, Mar 26, 2005
    #9
  10. Michael Mossey

    nabob33 Guest

    Ah, then your earlier assertion that you'd gotten 9 out of 12 correct
    was statistically irrelevant. You cannot just add up the results of
    different tests as you go along. You generally need to decide in
    advance how many trials you are going to do (and how many correct you
    are shooting for).

    Besides, you described this as "a similar test." If it was not exacly
    the same test--same cables, system, room, protocol--there would be no
    comparison at all between the two.
    Yeah, ain't it the truth.
    I would think fatigue would make you less confident about what you were
    hearing, not more, but it's possible. Next time, don't try to do it all
    in a single day.
    Some questions:
    --what cables (and length)?
    --what system?
    --did you level-match, and how?
    --did you also try doing a quick-switching test for comparison?

    bob
     
    nabob33, Mar 26, 2005
    #10
  11. Even if you can tell the difference, if it requires this amount of careful
    test listening to discern differences, is it worth the cost? When I listen
    to music I like to forget about everything else and simply enjoy the trip.
    Tests of the kind you wish to conduct appear interesting enough (for young
    folks with young hearing) engaging in the audio hobby trip, but not the
    enjoying the music trip.
     
    Norman M. Schwartz, Mar 26, 2005
    #11
  12. Michael Mossey

    Chung Guest

    As a control, can you also ask your assistant to use a sequence that is
    different than the 4 you have shown? You only want to find out if the
    middle two are the same or different, so I do not see why you need to
    limit the number of possible sequences to those 4.
     
    Chung, Mar 26, 2005
    #12
  13. Michael Mossey

    nabob33 Guest

    nonsense.

    You are entitled to your opinion. But there is good scientific evidence
    that you are wrong.
    A case for using minimalist drone in DBTs.
    This protocol doesn't make a lot of sense, actually. What you are doing
    is a standard same-different test, with two extra segments that are
    useless (since you already know they are different from the segments
    adjacent to them). I would suggest instead that you simply do a
    same-different test, AA or AB (you could alternate or randomize which
    cable is A), and allow yourself to listen more than once to each. This
    should improve the sensitivity of your test.

    One thing that's very important in a same-different test, by the way,
    is that you have an equal number of same and different trials. That's
    another reason you need to decide on the number of trials in advance.
    And yet you described your test above as giving you "a good sense of
    contrast." You are playing with words here. Contrast is exactly what we
    are trying to determine.
    True, but you are not trying to measure enjoyment. You are trying to
    measure contrast--as you yourself said!
    Such is your hypothesis. All available evidence suggests you are wrong,
    but if you think otherwise you should definitely test it.

    bob
     
    nabob33, Mar 26, 2005
    #13
  14. Look, I never claimed I had achieved anything that was statiscally
    sound, nor was my earlier assertion submitted to a peer-reviewed
    refereed journal. I'm an interested as you in doing this test
    properly, and I certainly plan to do as you say, decide the number of
    trials in advance. I was simply suggesting that I thought it was worth
    continuing the exploration. However, I still don't have an assistant
    so it will have to wait until someone wants to help. (I can't find any
    audiophile clubs in LA... anyone know of one?)
    Yes, and let me say that I think expectation bias exists alongside real
    differences in cables--or at least that's my hypothesis. So the test
    has to be carefully designed. And in the end I might just think that
    cables aren't worth it, if they can only be heard under highly
    specialized conditions.
    To me, fatigue makes it harder to be in touch with my reaction to
    music. Easier to imagine something.
    Cables: 2M Radio Shack basic gold plated vs. the cheapest Transparent
    cable (the one with no network boxes), also two meters.

    Marantz CD player into Calfornia Audio Labs tube DAC into SP-6
    preamplifier into B&K EX442 power amp into headphones. (Yes I'm using
    a 200 wpc power amplifier to drive 600 ohm headphones - it sounded okay
    to me). The cable between preamplifier and power amplifier was
    swapped.

    However, when I do this again I think I'm going to skip the
    preamplifier and run the DAC straight into Antique Sound Lab tube
    headphone amplifer.

    I didn't level match but I was assuming that 2M of bare interconnect
    cable doesn't have much effect, espcially into a 100 KOhm load (the
    next test). That could be a flawed assumption.

    Haven't tried a quick-switching test with this setup, although if I can
    find any audiophiles in LA who can loan me an ABX box I'll try it.

    Best,
    Mike
     
    Michael Mossey, Mar 27, 2005
    #14
  15. Well what do you think the truth is? Do you think that it doesn't
    matter that you use a changing signal during a quick switch test? Why
    not?
    Okay, you might be right.
    Not playing with words, Bob. I just didn't word this carefully.
    Actually these concepts I'm trying to describe are pretty deep. That's
    one thing I've noticed about "objectivists" like you--the world is much
    more black-and-white to you. If you find an apparent contradiction in
    my words, you assume that's a flaw to the core, instead of thinking
    carefully about what I might mean.

    One possible way to describe this is "conscious contrast" versus
    "unconscious contrast." A quick-switch test uses conscious
    contrast--the listener is actually trying to hear a difference, or even
    if not, they can't escape noticing the moment of switch. I was
    attempting to introduce unconscious contrast. That is, I was listening
    by noting what I heard, taking each experience on its own--so no
    emphasis on contrast in the listening. But I still wanted to have a
    test that varied the conditions frequently, to "clear the palete" as it
    were. There was no need for me to focus on changes, but let them work
    in the background to change what came to my attention.

    Anyway, the world isn't black-and-white. Contrast isn't evil. It can
    be simultaneously true that a quick-switch test is doomed by its
    emphasis on contrast, while contrast is still important.
    I never said I was trying to "measure" contrast. I was always trying
    to note my reactions as objectively as possible, including the reaction
    of enjoying the music. Typically I would note *how* or *what* was
    enjoyable. My hypothesis is that cables may differ is what aspects of
    the music they bring to conscious enjoyment.

    I'm trying to devise a test that is closer to natural listening. But
    it's a terribly difficult job. I would absolute agree with you that we
    can't reproduce a natural listening environment and simultaneously test
    people. That's the whole problem! But quick-switch tests are about
    the furthest thing from musical enjoyment I can imagine.
    wrong,

    Wow, pretty damning statement there.
    Yup, we have to have objective evidence one way or another, I'll agree
    with that.

    -Mike
    Mike
     
    Michael Mossey, Mar 27, 2005
    #15
  16. I was trying to have a balanced variety of conditions. But as Bob
    says, it may be better to do a same/different test.

    -Mike
     
    Michael Mossey, Mar 27, 2005
    #16
  17. Hi Norman,

    You are speaking right to the dilemma. We want to test as a way of
    learning what's true. What equipment brings maximum enjoyment? We
    want objective evidence of that. But testing is usually an unnatural
    environment. I'm trying to find a way to reconcile this. The
    "objectivists" on rahe as far as I can see don't attempt to reconcile
    it at all.

    -Mike
     
    Michael Mossey, Mar 27, 2005
    #17
  18. Michael Mossey

    nabob33 Guest

    Only parts of it, and we are careful to keep the discrete and the
    continuous separate. I am suggesting that you are failing to do this,
    and that your fuzzy language was symptomatic of that.
    I'm pretty sure I know what you mean. It's a concept that's been batted
    around here before. I just think you're wrong.
    Surely you would agree that there is something counterintuitive about
    the claim that we are more likely to hear differences between things if
    we do not try to hear differences between them. And before you object
    tht this isn't what you said, I will agree that it isn't what you said.
    But I would argue that what you actually said reduces to this.

    Nonetheless, I have already agreed that while I think your "unconscious
    contrast" hypothesis is implausible, it's at least thoughtfully
    rendered, and I hope you are successful in testing it.
    You used the word "contrast," and measuring it is exactly what you are
    doing. Specifically, you are measuring the contrast between these two
    interconnects against the threshold below which humans cannot detect
    sonic contrasts.
    Well, of course, but that's not what they're designed for. And I'm not
    sure that a protocol designed to enhance musical enjoyment would be the
    most effective means of determining sonic differences. That's where we
    differ.

    bob
     
    nabob33, Mar 27, 2005
    #18
  19. Michael Mossey

    nabob33 Guest

    That's because we see this as two separate questions, with detection of
    differences coming first. After all, if your ears can't deliver
    different signals to your brain, there's no way for you to have
    different levels of musical enjoyment, right?

    It's your hyothesis that our ears really are delivering different
    signals to our brain, but that if we try to focus consciously on those
    differences, we won't detect them. Whereas if we focus instead on our
    enjoyment of the music, we will become conscious of a difference.

    I said in another post that the evidence is stacked up against you. It
    is. The evidence suggests that ABX tests and similar DBTs are capable
    of identifying sonic differences very close to the physical limits of
    what the ear can detect. The evidence further suggests that the kinds
    of differences that interconnects, say, can be responsible for are ones
    for which we have a very short memory, so your protocol is likely to be
    less sensitive than a quick-switch test.

    But that's the existing evidence. You are welcome to throw new evidence
    on the table, and you have my respect for your willingness to do so.

    bob
     
    nabob33, Mar 28, 2005
    #19
  20. It is counterintuitive, and paradoxical. But human perception and
    performance is full of such paradoxes. I'm an amateur musician and
    I've talked to a lot of musicians, and it is pretty common for a
    musician to say that "trying too hard" interferes with their
    performance. I've also heard martial artists say that. It is VERY
    common for a musician to say that focusing too much on the details
    destroys a balanced sense of the whole (notice the relevance to
    quick-switch testing that focuses on a very small moment of
    experience).

    It is easy enough to observe that paying attention to some part of
    your experience changes it, so that it is hard to observe your own
    natural responses. For example, try to be aware of your eyeblinks
    and count the number in one minute, but *without* any sense that
    you are changing your natural pattern. Or note that bodyworkers can
    ask a person to pay attention to their breathing, and can
    see immediately that the depth or rate of their breathing changes
    with this attention.

    My own ideas for testing protocols aren't free from these effects.
    If there is any truth to the idea that trying too hard, or conscious
    focus on contrast, can interfere with listening, quick-switch
    testing is going to be susceptible to these effects even more.

    If we can get a grant of about a million dollars, we could pipe music
    into subjects while they lie in a PET scanner. Then they wouldn't
    have to consciously try to do anything at all. Of course PET
    scanners create an enormous background noise.
    Well, thank you.
    Okay, that's true, I am looking for differences. However, I'm not
    necessarily looking for the conscious perception of contrast.

    Once an audio engineer described to me a blind listening test on
    cartridges he did with a panel. The members of the panel rated
    the sound quality after each listen. They also chatted with each other
    between sessions. The engineer said he noticed informally that
    sometimes
    the panel chatted about the music they had just heard, while sometimes
    they chatted about the sound of the cartridge. In his opinion, the
    better cartridges inspired people to talk about the music.

    Now, never mind that this test was not scientific or controlled. I'm
    only using it to suggest a possibility--that outside observers could
    see that a person reacts differently to different things, while at the
    same time the person doesn't have to be aware of the difference. Or
    the person could be aware of the difference but attribute it to the
    wrong thing.

    I've had enough life experience to realize that other people are
    observing my emotions and reactions to things, sometimes seeing stuff
    that I'm not aware of.

    Maybe we could do some kind of audio test where somebody observes
    the subject, rather than having the subject give their own
    observations.
    But that's tricky. For now, what I'm trying to explain is this:

    I my own tests I tried to function as an observer of
    my own musical enjoyment, rather than an observer of sound.
    This is actually a difficult or maybe
    impossible thing to do. But I tried. Sometimes I found myself
    listening to a cable and just really enjoying it or noticing all
    sorts of pleasant musical details. Sometimes I found myself not
    enjoying the music. I wrote these observations down. And more than
    half the time, my observations fit a pattern: more enjoyment with
    the Transparent cable compared to the Radio Shack. I couldn't
    consciously point to the differences in sound while I was listening.
    So that's why my test was different than quick switch.
    It could be (this is probably what you believe) that all the details in
    a musical signal that the ear can actually hear are available in the
    act of short-term comparison of sound. If this were true, then I would
    agree that quick-switch testing would be the gold standard of
    comparison.

    But how do we know this is true? Can we determine this is true from
    quick-switch testing? If all the tests we do involve the same mode
    of listening, can we infer things about other modes of listening?

    I'm seriously posing these questions in case you or anyone else wants
    to
    take a shot at them.

    Best,
    Mike
     
    Michael Mossey, Mar 28, 2005
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.