The Benefits of Blind Listening
The other day, Jeff Fritz, our managing
editor, phoned to tell me that, in an effort to compare the sound quality of two digital
sources, he’d done something he’d never done before: participated in his first
blind listening test.
A blind test is one in which the identities of the
components being evaluated are concealed, to eliminate any bias on the listener’s
part about brand, price, technology, appearance, etc.
Jeff took part in a single-blind test -- the
products’ identities were concealed from him, but were known to the person conducting
the test. In a more rigorous, double-blind test, measures are taken to conceal the
identity from both parties. Single-blind testing, though, still has considerable benefit.
According to Jeff, he heard no consistent differences
between the two components. Nor, later, did he have any reason to dispute the results --
they still sounded identical when he listened to them while knowing exactly which was
playing. So why did he go through the inconvenience of a blind test when he could have
listened to them sighted in the first place? In this case, one source costs twice the
price of the other, and Jeff knows that bias can easily accompany the knowledge of such a
difference in price; it’s too easy to assume that a product costing so much more must
be better in some way.
After Jeff told me about his experience, I told him about
my own first blind-listening test, 15 years ago. A friend who’d been an audiophile
for 20 years challenged me to do it because he thought I’d learn from the results. He
was right; in fact, it forever changed my opinion about how to evaluate audio equipment.
Like Jeff’s, my first blind test involved two digital
sources, one of them a transport-DAC combo I’d purchased a month before for $3000,
after reading reviews of it and hearing about the company’s reputation. I even drove
three hours to seek these components out. I still vividly remember the first time I saw
them in the store, and how impressed I was by their sturdy, all-metal chassis. They were
far better built, and more impressive looking, than any of the small, plastic CD players
I’d owned before -- and much more expensive. I listened to them, too, but only after
being in awe for some time. Furthermore, all my listening was done sighted, as is
customary at dealers. At the end of the day, I was convinced I’d picked the
best-sounding digital front end I could for that kind of money -- and for that kind of
money, it should be very good indeed.
The opponent was an all-in-one CD player owned by a friend,
and which sold for $600 -- one of those plasticky CD players like the ones I was used to.
Furthermore, I was already familiar with this player; I’d auditioned it at stores a
few times, including the one where I bought my $3000 transport-DAC, and neither its looks
nor its sound had impressed me. So I said to my friend, "Bring it on!" And my
first blind test began.
Unlike Jeff, I heard profound differences between the two.
With one source, the bass was fuller, the midrange cleaner, the highs tidier --
impressions that remained consistent regardless of the music we played. Furthermore, these
weren’t mere differences; one player sounded considerably better, and I
was sure I knew which one it was.
When my friend revealed that the source that I’d
consistently preferred was the $600 player, I was stunned. A few seconds later, I was
angry. And not long after that, I came to the conclusion that there must be something
wrong with the test. So my friend let me listen again, again under blind conditions.
No question -- the $600 player sounded better, no matter
how much music I played through it and the $3000 combo. I wished I could have done that in
the store.
It’s hardly surprising that some of the best audio
companies use blind listening in their R&D process. I’ve visited the factories of
Paradigm, Audio Products International, and Axiom Audio, and each company has a specially
built room in which it conducts blind tests to critique its own designs. After all,
everyone -- even accomplished audio designers -- can be influenced by bias. Furthermore,
Canada’s National Research Council (NRC), where the SoundStage! Network tests
loudspeakers, has a special room set up specifically for blind listening tests, and in
recent years I’ve taken part in a number of such tests there. Blind tests are
standard practice for the NRC experts, as they are in many other disciplines.
However, that’s not to say that every audio-reviewing
professional agrees that blind testing has merit. In fact, some vehemently disagree.
Surprisingly, for one reason or another, the practice is largely dismissed by much of the
audio-reviewing press. However, I think the reason most reviewers don’t understand
the tests well is that they haven’t been privy to well-conducted blind tests (not an
easy thing to do). Either that, or they’re simply scared to critique the sound of a
product unless they first know the make, model, and price.
In my opinion, though, blind listening has tremendous
benefits, and Jeff obviously thinks so too. Furthermore, consumers can benefit
tremendously by auditioning components they might purchase under these sorts of
conditions. The problem is, no store I know of is set up for blind tests.
But even if you can’t audition something in an ideal
blind setup, you can incorporate elements of blind testing into your evaluations to reduce
your own biases and thus make better buying decisions: Before you evaluate a component,
don’t ask its price. Before you listen, ask the salesman to not give you his
or her opinion about what a component sounds like. Some salespeople can be very convincing
-- it’s their job, after all -- in leading you to think that even poor-sounding
equipment sounds great. Then, when listening, close your eyes so that you can’t see
the component, and try as hard as you can to forget everything you might already know
about it. Rather, focus exclusively on what you hear, not what the component looks
like or how much it costs.
Understanding the principles behind and the benefits of
blind listening is important; once you know them, they can help make you a better listener
overall.
…Doug Schneider
E-mail comments to the editor@goodsound.com.
|
