Behind the Scenes at Secrets – Number 1 – June, 2008

Since a quarterly newsletter seems to be the norm for large websites, the Secrets staff has decided that it is time for us to join the trend.

The problem I see with most newsletters is that they don’t deliver anything that the website doesn’t deliver in one way or the other.

Secrets was the first A/V magazine to be published only on the Internet.  We have never been a trend copier. We have always done things our own way. Maybe a few mistakes here and there, but mostly, we have set standards such as the Secrets DVD Player Benchmark that is now accepted worldwide as the most comprehensive set of criteria for the evaluation of DVD players.

What we want to do here is give you info that is not available on our website, and that information we think you will find useful. Perhaps eventually, these pages will be placed in our Primer section or somewhere else, but for now, you can only get it here in the newsletter.

OK, to start, we want to explain why and how we do our bench tests. There is too much to put in one newsletter, so it will be spread out over the entire 2008 year. That way, you can send us an e-mail requesting further info on some particular aspect that you would like to see more explanation.

So, why do we perform bench tests?

The short answer is that I (John Johnson, Editor-in-Chief) have a scientific background in the field of Neuroscience. Science requires objective measurements where possible and I routinely used very sensitive instruments to obtain objective measurement.

The long answer is that listening to music through a product being reviewed and stating that the sound has a lot of air, grunge, grit, or whatever other words one wants to use which don’t really mean a lot when referring to audio, does not necessarily convey the same thing to everyone who reads them, nor even to the entire staff at the magazine. If you handed that product to a different reviewer, you would very likely end up with a completely different set of words, because the field of audio is filled with subjective words to describe the sound.

There is nothing wrong with such descriptions as long as there is something objective to go along with them. That does not mean that absolutely every product has to have some objective bench tests, but at least some of them do, in order to put numbers to the way of thinking in that organization.

So, we bench test, and we do these tests on CD players, DVD players, preamplifiers, SSPs, power amplifiers, speakers, subwoofers, cables, and some accessories like earphones. In general, all the tests are performed in the room where the products are listened to. This is a matter of convenience, but for speakers in particular, it gives real world numbers.

At Secrets, we use several instruments to get our numbers. These include hardware and software.

AudioPrecision is the name that all engineers think of when bench testing audio equipment. They have been around for decades. Their first instruments were analog, and the most recent ones have both analog and digital capabilities.

Last year, we purchased an Audio Precision SYS-2722. It consists of a hardware chassis that interfaces with a PC through a card or USB port. Software controls the chassis and displays the results, but all the number crunching is performed within the hardware chassis. The 2722 is not cheap, at $25,000. But, it is a must-have component for bench testing if you want certain tests that smaller software based analysis systems just can’t perform.

Here is a photo of a recent test I performed on some speakers, using the AP SYS-2722.

In order to set the test up, one has to configure all the screens such that the frequency range and type of signal being delivered are standardized. There are a lot of options, so it is easy to mess things up, but as long as you remember to make sure the output level is very low when you fire up the test, so that you don’t put too much voltage through the test circuit, nothing will be damaged. But, in any case, there is a whole lot of trying this and trying that at first, before the final test procedure is set. Then, it is just a matter of opening a basic template file the next time you want to run that test on a new product in for review.

If one is testing speakers, obviously a microphone has to be used, and it needs not only to be a very high quality one, but it has to be calibrated because different microphones output different amounts of millivolts at any one SPL that is fed to it.

Calibration is accomplished by using an output of 1 kHz at known value and setting whatever millivolts that turns out to be incoming to the test instrument to that value. We do this by using a microphone calibrator that has a small opening where you stick the tip of the microphone, and setting the calibrator to play a 1 kHz tone at either 94 dB or 104 dB. Another way is to play 1 kHz through the speaker, and using a Radio Shack (RS) SPL meter – which is accurate at 1 kHz – one places the Radio Shack meter microphone tip at the same place as the microphone in need of calibration, adjusting the SPL to read 104 dB on the RS meter, and then setting the voltage incoming to the test instrument from the microphone needing calibration to be 104 dB as well.

So, after the microphone is calibrated, you need to make sure you use a standardized distance from the speaker, because SPL decreases exponentially as you move farther from the speaker. So, if you measure 0.5% THD+N (Total Harmonic Distortion plus Noise) at 100 dB and 1 meter, but the next speaker you put the microphone at 4 feet, you would have to turn up the volume in order to get that 100 dB. Even if you used the same speaker, the distortion reading would be higher, because every speaker produces higher distortion as the output is increased. So, in order to be fair to all the speaker manufacturers, not to mention making the results reproducible, you need to standardize the distance. In our case, we use 1 foot, 1 meter, and 2 meters for various tests.

In the next installment, I will discuss the other test systems that we use (primarily software-based) and go through an actual measurement, with details on all the parameters that have to be paid attention to during the tests.