What is a “Blind” Test in AV?

One common refrain from those arguing on AV forums is that a person could never hear a difference in a “blind test.” We’ve all heard about blind tests and double-blind tests. But do people really do that in real life? Do they test AV gear with blindfolds on? Let’s discuss!

What is a Blind Test?

At its most basic, a blind test is one where the participants don’t know which of the options they are experiencing at any given moment. In the medical field, they are given one of two pills. One is the actual medicine and the other is usually a sugar pill (placebo). In AV, a blind test is one where the participants don’t know which piece of gear (usually out of two but sometimes more) is playing.

You sometimes hear people talk about double-blind tests. This is where neither the participants nor the administrator knows which option is being administered at any given time. This is considered the “gold standard” of testing because it eliminates most sources of bias. But it is hard or impossible to achieve in most tests. That’s why most tests are single-blind (what most people would just call blind) where only the participants are left in the dark.

How are Blind Tests Done in AV?

The easiest way to set up a blind test in AV is to put the equipment behind an acoustically transparent curtain. The participants won’t be able to see the gear and the tests can be run normally. There are devices and programs that can automate some tests (search for ABX boxes or programs) but these are expensive.

Are Blind Tests Needed?

The dirty little secret of blind tests is that they are often unneeded. In medicine, it is important to be able to know not only that a drug or treatment had an effect, but the size of that effect. Face it, AV just isn’t that important. What people really want to know is if one thing is worth the extra money over another thing. This is not a life or death question.

On top of that, blind tests are really only important when the effects are small. Do you need to be blindfolded to tell that a Ferrari sounds different than a Toyota? Often in audio, those are the differences we experience. One speaker or subwoofer is obviously better than the other. When the differences are small, the price difference usually is the deciding factor.

In AV the blind test is only really needed to combat expectancy bias in the participants. If the participants believe that one product should be superior (usually because of price or marketing), they’ll “hear” those differences even if they don’t exist. But this is only if the differences are small or (more often in the case of amps, wires, and most electronics) non-existent.

Why Do People Care About Blind Tests?

They don’t, for the most part. There are two times people make claims about blind tests. The first is to discount any review or claim with which they don’t agree. Someone says that the speaker that another person owns isn’t as good as a different speaker. The easy out is to reject the claim because the test wasn’t done blind. In fact, any data that doesn’t mesh with a person’s worldview is met with the same retort. If the test wasn’t done blind, it doesn’t count.

The second time you are likely to hear about blind tests in AV is when someone makes a claim when the two products have no measurable differences. When someone says one Blu-ray player is better than another, a speaker or digital cable makes a difference, or similar, people will wonder how the test was conducted. If there is no measurable difference, then the subjective review has to be more rigorous. Because if the reviewer didn’t take care to eliminate expectancy bias, then their claims are suspect. Did they actually hear a difference or did they just want to? If you can’t back up your claims with objective data, you need something else. And sighted listening tests often aren’t enough unless the differences are large (and therefore would be measurable).

Take Away

Are blind tests necessary to hear differences between AV equipment? Most of the time the answer is no. If there is a large chance for expectancy or other bias, or if the differences are expected to be small, then a blind test will help ensure your results are more accurate. Otherwise, you can listen to them side-by-side. If there are any major differences, you’ll hear them. If not, get the less expensive one.

Have you participated in a blind listening test? Let us know about your experiences in the comments below or on our Facebook page!