Blind listening test!

Discussion in 'High End Audio' started by Michael Mossey, Apr 7, 2005.

  1. I found someone to help me. We are starting tonight. I'm going to do
    some training of my ears first. I would appreciate any suggestions
    that people can give me for improving the test.

    Here's the system: I'm connecting a Marantz CD player to a CAL DAC to
    an Antique Sound Lab MG Head OTL Mk III headphone amplifier to AKG K501
    headphones.

    The cables to test will run between the DAC and the headphone
    amplifier. They are:

    Rat Shack 2M gold-plated, about $5
    Transparent 2M, the one without network boxes, about $75

    Making it blind
    ------------------

    I will hang a sheet over most of the system. I still need to access
    the CD player to start and stop things and put them in and out. The CD
    player is at the bottom of my rack, so I will tuck the sheet into the
    component above it, blocking everything else from my view. I will make
    sure that there is no way I can see the cables themselves, but I will
    have to sit in front of the rack.

    My friend will hook up cables following my instructions, either A, B or
    X as I instruct. He will determine X with a coin toss and write it
    down. I will leave the room, he will hook up the cable and call me
    back into the room. We will need to have some interaction as he will
    call me, and we will pass each other coming in and out of the room. I
    will keep my eyes closed so at least I won't pick up any verbal clues.
    Hey it occurs to me that I can have him knock on something instead of
    calling me so I can't pick up on any voice cues.

    Level matching
    ------------------

    I don't have a good way to control levels.. I don't have a CD with a
    test tone. Hmm, it occurs to me that I can burn one. I have a basic
    voltmeter. Okay, I will do some basic measurements on a test tone. Is
    a 1Khz test tone good? Also, it is a good assumption that two straight
    wires into a 100K ohm load aren't going to differ by more than 0.1 dB?
    But in any case I will leave the amplifier's volume control in one
    position and not touch it during each trial.

    Documentation
    ---------------

    I type everything that happens in time order into my laptop and publish
    it on the web.

    Protocol
    --------------

    Basic ABX, but not with a switcher box-- instead I will ask my friend
    to hook up one of the cables each time. This will take a while,
    obviously, and I won't be able to do many auditions of each cable, nor
    will I be able to switch quickly.

    Also, if I get tired or confused, I will simply not give an answer for
    the current trial, and start with that trial again later or another
    day. Does that sound acceptable? It seems to me that there's a small
    chance this could be a way to defeat the test; i.e. refuse to answer
    when I'm not sure, implying that I'm only sure when I've picked up the
    answer by non-sonic means. However, I will write down everything that
    happens including the times that I refuse to answer so people can
    review it later.

    Training
    ---------------

    For the initial tests, I will ask my friend whether I guessed right
    after each trial. When I'm confident that I've learned well the sound
    of each cable, then the "test proper" will start. I will determine a
    number of trials and write down my answer after each trial, but not
    compare answers to the answer sheet until the proper number of trials
    has been reached.


    -Mike
     
    Michael Mossey, Apr 7, 2005
    #1
    1. Advertisements

  2. Michael Mossey

    nabob33 Guest

    Is that a tube amp? I'd want somebody to confirm that there aren't
    impedance problems here that might create FR anomalies.
    dB?

    Hey, you're the engineer. You tell us. But if you can level-match, I'd
    check it at 100 and 10k as well.
    Uh, no. You should make your best judgment, and move on to the next
    trial. I'd also set a limit in advance: Do eight trials today, or do as
    many trials as you can fit in two hours, or something. (You migth want
    to do a dry run to see how many trials you can reasonably do.) The
    temptation to finagle when the numbers aren't going your way can be
    strong, so you want fixed rules going in.
    This really isn't necessary for training. All you need to do is listen
    to each enough that you feel you can tell the two apart. If you wait
    until you get a string of correct answers, you may never start!
     
    nabob33, Apr 8, 2005
    #2
    1. Advertisements

  3. It's almost certainly the case that you won't need to worry about
    level-matching for this test, but if you are going to do it, then a
    15kHz tests tone is better than 1kHz (any differences will most likely
    be due to cable capacitance, which will show up at high frequencies).
    You will need to do at least ten trials, and preferably twenty, to get
    statistical significance. The usual standard around here is fifteen
    out of twenty for a 'success'.
    Best to document everything, including 'don't knows'.
    That's fine.
    That is important, although you if you plan a twenty trial test, you
    might want to check your results after ten trials, to see if there's
    any point in continuing. Get more than four out of ten wrong, you
    might as well save your time and grab a beer! :)
     
    Stewart Pinkerton, Apr 8, 2005
    #3
  4. Michael Mossey

    Guest Guest

    That sounds like a good test, Mike. I would only suggest that you make sure
    your assistant disconnects the cables after every trial, and BEFORE he flips
    the coin. There's a tendency to just leave the cables in place if there's
    no change from the previous trial. This is bad news.

    Also, I would be surprised if there was any significant level change between
    cables. If there is, I'd investigate the setup thoroughly; there shouldn't
    be.

    Norm Strong
     
    Guest, Apr 8, 2005
    #4
  5. Follow up to this:

    I did some blind listening to cables last night and did not feel
    confident that I could tell the difference. I also observed some facts
    about blind listening.

    I agree with the objectivists that expectation during sighted listening
    can affect what we hear. I observed that even in blind listening, I
    formed "expectations." For example, on the basis of an initial
    impression I might immediately make a conclusion about the sound as a
    whole, and it was extremely difficult to separate the sound from those
    expectations.

    At the moment I'm probably not going to follow up with a more extended
    blind test because I don't feel confident I can tell the difference
    between cables. I may get some ideas about how to make the test
    conditions more sensitive, and maybe I'll reconsider. I already use my
    cheapest interconnect in my main system, anyway.

    -Mike
     
    Michael Mossey, Apr 8, 2005
    #5
  6. Yes, I had my assistant disconnect and reconnect the cables on every
    trial.

    My initial impressions of the sound did not correlate with the cables.
    I.e., my friend hooked up cables a few times, and I just listened to
    them blind and wrote down observations. Then he revealed the order, I
    checked my observations against that, and there was no match. I did
    not find any aspect of the sound that would let me tell the cables
    apart. Except at the very beginning. I had strong opinions about the
    first three trials and I was right about them. Easily a chance
    phenomenon, but it is interesting to hypothesize that my ear was less
    discrinimating after listening to many things--which is something I
    observe, that the more stuff I listen to in an hour, the more it all
    sounds the same. Of course that could also be because it is all the
    same.

    -Mike
     
    Michael Mossey, Apr 8, 2005
    #6
  7. Michael Mossey

    nabob33 Guest

    Aww, does this mean you're not going to tell us how you scored? We were
    really looking forward to that, you know.
    That's how pernicious it is. You don't have to know anything more than
    that the two things you are listening to are different (not how, or
    what they are), and they can sound different.
    A reason why longer listening might not help. Our brains can be so
    stubborn.
    Keep in mind that you did not use a particularly sensitive test
    protocol. I know you don't want to believe that quick switching is the
    best way to do these things, but it is. If there were an audible
    difference between these interconnects, that would be the only way to
    determine it.

    bob
     
    nabob33, Apr 8, 2005
    #7
  8. Fine, so *test* your new hypothesis. Do say three trials a day for a
    week, *then* check your score for the 21 trials.
     
    Stewart Pinkerton, Apr 9, 2005
    #8
  9. Of course I want to test it. Why would you think otherwise? Actually
    I have something I want to do first. My plan is to have my friend hook
    up one of two cables at random. I will listen with different
    selections of music, and in different ways. And I will write down my
    impressions on a scale. For example, "harshness" vs. "smooth" on a
    scale of 1 to 5. Sometimes I will do long listening. Sometimes I will
    make an instant impression.

    Then my friend will reveal the order to me. I will look for any
    correlation in my data. Does there seem to be a pattern in long
    listening tests, but not in quick impressions? Or vice-versa? Is
    "harshness" a good place to listen, or is something else better?

    Basically I'm looking for the best listening strategy. If there
    appears to be no way to tell the cables apart after this experiment it
    is pointless to continue.

    However, if I find a pattern, then I will set up the 20 blind ABX
    trials.

    I only have one friend who is only occasionally free to help me with
    this, so it's going to be a while.

    -Mike
     
    Michael Mossey, Apr 10, 2005
    #9
  10. Michael Mossey

    Guest Guest

    What you're proposing here is an "investigation." That's a good idea, but
    why not start by replicating some of the experiments already run by others,
    using a variable that's easily controlled. I would advise changing nothing
    but the volume level. Can you tell the difference between 2 signals that
    differ by 1db, and nothing else? Try it using music, sine waves of
    different frequencies, white noise and pink noise. My guess is that you'll
    have difficulty hearing a difference with music as a source; less difficulty
    using a steady signal, either sine or noise.

    Once you find the most sensitive signal, change the reference level. Try
    60/61db, then 80/81db, then 100/101db. After you've found the signal that
    makes a 1db level difference most obvious, reduce the difference from 1db to
    0.5db, etc. etc. What I'm suggesting here is that you investigate your own
    hearing ability before proceeding to something that is entirely terra
    incognita. At least that's the way I'd approach the problem.

    Cheers,

    Norm Strong
     
    Guest, Apr 10, 2005
    #10
  11. Michael Mossey

    Buster Mudd Guest


    Isn't that always the case with us audio geeks?
     
    Buster Mudd, Apr 12, 2005
    #11
  12. Michael Mossey

    Buster Mudd Guest


    I thought the most compelling defense against the Subjectivist Camp's
    insistence that quick switching wasn't practical, realistic, or
    ultimately revealing of sonic differences was that you DIDN'T have to
    do quick switching!

    Haven't DBT proponents (including the many on RAHE) repeatedly argued
    that quick switching is NOT a requirement of these tests? That's what
    makes "Audio Objectivism" so appealing to me: theoretically, Mr.
    Audiophile can use his own equipment in his own room playing his own
    material at his own pace on his own schedule...he can take hours, days,
    weeks, months to reach his conclusion...the only difference between his
    normal music listening activities & his participation in this
    hypothetical double-blind ABX test are A) he can't ever know whether A
    or B is in place at any given time, and B) at some point he does have
    to make a guess as to whether X is A or B.

    If that's not the case, where's the fun? :)

    (And besides, regardless of whether or not quick switching is
    demonstrably better for identifying audible differences, wouldn't slow
    switching be just as good for identifying the inability to identify
    audible differences?)
     
    Buster Mudd, Apr 14, 2005
    #12
  13. Michael Mossey

    nabob33 Guest

    You don't. But if you don't, there will be some audible differences
    that you miss. OTOH, for interconnects it really won't make a
    difference either way.

    bob
     
    nabob33, Apr 15, 2005
    #13
  14. No, because extant psychoacoustic data suggest that 'slow switching'
    (long intervals between sound samples) is a less sensitive means for detecting
    subtle differences between the samples, than 'quick switching'. Ditto
    long samples versus short samples. So an 'inability' identified in this
    way might simply be due to the insensitivity of the protocol. That needs
    to be addressed by trying the more sensitive protocol.

    So, yes, Mr. Audiophile is free to do the test any way he wants, as long as
    A and B are adhered to. But if Mr. A. *fails* to detect difference using his
    own switching protocols (e.g., 'slow switching' using long samples),
    Mr. Objectivist will suggest he try protocols considered to be more sensitive,
    that involve short samples and quick switching. He might even suggest some
    progressive ABX 'ear training' for differences. In doing so Mr. O is
    trying to *help* Mr. A substantiate the differences he wants so badly to
    hear, not hinder him. ;>
     
    Steven Sullivan, Apr 15, 2005
    #14
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.