Three Reasons Why Measurements Matter

The debate about the importance of product measurements has raged for decades among audiophiles -- there always seem to be plenty of people willing to take extreme positions in arguing about it. But after all this time, we seem to have come no closer to any sort of agreement about just how much measurements matter.

At one extreme are those who claim that measurements don’t matter at all, that your ears alone can tell you far more. At the other extreme are those who think that measurements mean everything, that you don’t even need to listen to a product to know how it will sound -- the graphs will tell you.

Here at the SoundStage! Network, we feel that the truth lies somewhere in between. Because we believe that measurements have some validity, we have a strong measurement program that has resulted in some of the most accurate measurements yet published of loudspeakers, amplifiers, and preamplifiers. You can find our measurements on our sister site SoundStage! A/V, or by clicking on one of the following links: www.amplifiermeasurements.com, www.preamplifiermeasurements.com, or www.speakermeasurements.com.

To measure components properly is expensive and takes time. Obviously, we think it’s worth doing. However, the point of this article is not to tell you that measurements mean everything, or anything close to that.

Instead, I’ve written this piece to present to you the three main reasons we feel measurements matter, and why we do them -- and, we hope, to put to rest the notion that they don’t matter at all, and possibly even convince some naysayers that measurements have some worth. What we hear can, at least to some extent, be quantified by measurements, and if you know how to read the resulting graphs, it can tell you something about how a product will sound. In fact, if you know how to read such charts, you can weed out the worst products by first looking at their graphs, and thus save yourself a lot of time.

1) Measurements can help to tell you what a product will sound like. While no set of measurements will tell you everything about a product’s sound, a few key measurements can tell some of the story.

A frequency-response chart can tell you many things, including the frequencies that a product can accurately reproduce. Whether it’s an amplifier, preamplifier, or loudspeaker, the product’s frequency-response chart will show you how low its output will extend into the bass and how high into the higher frequencies. Of course, you can hear this in listening tests, but measuring equipment can quantify it with a high degree of accuracy. The linearity of the frequency-response curve will also tell you how evenly the product reproduces the frequencies between its extremes of high and low. This is important for knowing how "neutral" the product’s output is. Again, some listeners can learn some of this by listening, but measurements can give you data that are accurate to within a fraction of a decibel, a level of precision I doubt anyone can quantify by ear.

2) Measurements can show if a product meets the specifications claimed by its manufacturer. Almost all manufacturers publish lists of product specifications that are the results of measurements, but you’ll be able to verify those measurements only if you conduct tests of your own. Some manufacturers fudge their numbers, and publish misleading specs in order to make their products look better on paper than they actually are. Someone has to check up on them, and that’s why we double-check their work with our own measuring regimen.

For example, higher amplifier power is often more desirable than lower power because higher-powered amplifiers are compatible with a wider variety of speakers. It’s like having a car with more horsepower -- you have less of a chance of running out of steam. Therefore, to make an amplifier look better on the spec sheet, many manufacturers try to claim as high a power rating as possible. (Power ratings are usually stated in watts, usually into an 8-ohm load.) A 150W amp will seem more impressive than a 100W model selling at the same price, but only if the former actually does output 150W -- and the only way to know is to measure it.

Makers of power amps aren’t the only ones who might be tempted to embellish their figures. Manufacturers of speakers, preamplifiers, CD players, you name it -- every one of them wants to publish the most impressive-looking specs they can, and none wants to look bad and thus lose a sale.

Manufacturers can juggle numbers in various ways -- everything from playing around with how the test is performed to outright lying. Suffice it to say that there’s plenty of incentive -- higher sales -- to publish misleading figures, and that measurements such as the ones we perform are a good way to test such claims.

3) Measurements can tell you whether or not a product has been competently designed. This isn’t often mentioned in debates about the validity of audio measurements, but it should be. Serious design flaws can show up in the kinds of tests we perform, and often do.

A couple years ago, a manufacturer sent us samples of a brand-new speaker model for review. It had an attractive cabinet, appeared to be a conventional three-way design, and, by the look of it, should have performed competently in the lab -- there seemed to be nothing strange about it. But when we measured the speaker’s frequency response, we found it to be very uneven -- i.e., nonlinear -- and therefore not very neutral. Nor was it particularly extended in the bass and treble, as you’d expect such a design to be. Worse yet, when we measured the speaker’s impedance, we found that, at two frequencies between 20Hz and 20kHz, the impedance was less than 1 ohm -- that’s this close (imagine my thumb and forefinger squeezed together) to presenting the power amplifier with a damaging electrical short. This speaker wasn’t just poorly designed; it was unsafe to use.

We learned that the review sample was not faulty, nor was there a quality-control issue. Instead, it appeared to be a matter of designer incompetence. Talking to the manufacturer revealed little understanding on his part of loudspeaker technology, and next to no knowledge of relevant measurements or techniques. In fact, from what I could gather, before we got our hands on it this speaker had never been measured, or even been near any test equipment of any kind. It seemed to be the result of guesswork, not proper engineering. And when we listened to it, sure enough, the same problems that had shown up in the lab measurements were audible in the listening room. The frequency-response aberrations were easy to hear, and it was obvious that the amplifier was having difficulty with the load.

If someone doesn’t want to look at a product’s measurements to get some idea of what it will sound like, or doesn’t care whether or not it actually meets the specifications claimed for it, that’s fine. But I think everyone will agree that measuring a component to see if it’s been competently designed and, more important, is safe to use, are sensible things to do.

Measurements matter

Measurements may not tell you everything about a product, but the three points made above will tell you some things. For these reasons alone, we believe measurements are worth taking, and will prove valuable to you when you’re shopping for a new component. I encourage you to visit SoundStage! A/V, or use the links mentioned above, to take a look at our measurements. And you can look forward to some upcoming articles in the "Guide" section of GoodSound! that will tell you more about how to read and use these graphs. Measurements matter -- more than many think.

. . . Doug Schneider
editor@goodsound.com