Introduction to the Benchmark Tests

On May 13 and 14, 2000, Secrets hosted its first DVD player Benchmark Event (Shootout I) in Redmond, Washington, where we tested numerous DVD players as to their video, audio, functional, and usability qualities. Audio Precision and Tektronix instrumentation were employed to gather extremely accurate data. The information you will see here is much more extensive than you will ever find in any other A/V magazine, on or off the web.

DVD is the highest quality consumer video format we have ever had. At the same time, it is also one of the most complex and sometimes confusing formats. Because of this complexity, we get e-mails asking us all kinds of questions. It's because of these questions that we are very nitpicky in our Benchmark standards.

We will mention Toshiba over and over again as we complain, so we need to qualify that right now. Toshiba has been at the forefront of DVD technology, as they had the first players on the scene. They were the first with component video outputs, and they were the first with progressive DVD players. We are very thankful for their strong commitment, but their marketing department has also done a lot to confuse the consumer. More on this later.

So what is our DVD Benchmark? We are going to tell you just how good or bad the audio and video quality of each player is, based on objective measurements. We will also do our best to explain how each measurement relates to the audio and video quality. We are going to report on how each player handles the various features of DVD, including seamless branching, multiple angles, parental control, and much more.

Before beginning, we will say that with the exception of two DVD players, they all looked pretty good in Benchmark I. They are all an improvement over VHS, DSS, and even laserdisc, which was the top dog for a long time. The players we looked at range from a mere $179 up to $14,500.

We have a series of six Benchmark articles that describe the technology we tested and the types of tests we performed. Part 1 discusses the video quality of the DVD players. Part 2 covers the audio side of the DVD players. Part 3 covers the functionality of the DVD players. Part 4 illustrates the usability study that was conducted on the DVD players. Part 5 deals with progressive DVD players. And finally, Part 6 covers DVD Audio and MLP. You can refer back to Parts 1 through 6 to see exactly what the test results mean. A list of the players we have tested is shown at the bottom of this introduction.

The following people made it happen:

Stacey Spears
Stacey is our Video Editor (and video guru). He organized the whole thing.

Guy Kuo
Head of Ovation Software, Author of Avia, Guy manned the Tektronix scopes and deserves the most thanks, because if it were not for him, we would still be making the video measurements. Guy also took the time to answer any technical questions we had during the tests.

Andrew Rosen
Andrew is Technical Director of the Microsoft WHQL DVD Test Annex. He gave up a weekend to offer his expert technical experience with not only testing the VM700T but also walking us through the WHQL DVD. He also dissected the Toshiba and Panasonic progressive DVD players. Andrew is also the person responsible for our having the use of an Audio Precision System 2.

John E. Johnson, Jr.
JJ is Editor-in-Chief: Secrets of Home Theater & High Fidelity.

Colin Miller
Colin is our Digital Audio Editor. He ate two BBQ sandwiches! He also spent about 50 hours behind the Audio Precision measuring those DVD players so I guess he deserved that second sandwich.

Evan Upchurch
As a Contributing Writer Secrets of Home Theater & High Fidelity, Evan has several years experience as a usability engineer. He orchestrated the entire DVD usability study and also put together the on-line survey that we conducted.

Brian Florian
Although Brian did not attend Benchmark I, he did contribute with all of the excellent graphics and animation work. He also wrote a nice essay on the 3-2 frame rate conversion process in Part 5.

Don Munsil
Don is a contributing writer for Secrets and is responsible for the Part 5 re-write to explain how progressive DVD works. He also documented all of the tests we performed on the various progressive DVD players. Don's brother took the chroma upsampling error photos in Part 5. Don also helped out with Part 6.

John Kotches
John did the brunt of the work on the part 6. John also helped out with subjective listening.

Kris Deering
Kris helped the 3rd time out with running our deinterlacing tests.

Paul Carleton
Paul is Administrator of the Home Theater Spot, www.hometheaterspot.com. He was kind enough to bring over his Panasonic H1000 progressive DVD player for us to critique. Paul also videotaped the entire event with his digital camcorder, hopefully to put a video of our testing activities on his website.

Jim Taylor
Author of DVD Demystified and DVD FAQ, www.dvddemystified.com, Jim was only able to come by on Saturday. Along with Andrew and Paul, Jim took an in-depth look at the progressive players. If there was something that we did not fully understand about DVD, he was always happy to answer the question.

Joe Kane
President of Joe Kane Productions, and designer of Video Essentials, Joe helped out with video measurement questions. He also suggested many of the DVD titles that we used to test the players functionality.

Keith Jack
Keith is the author of Video Demystified, www.video-demystified.com. Keith continually answered every video specific question that we had. He also provided the component color bar numbers of the various CAV formats.

Brad Marcus

President of Better Cables LLC, www.bettercables.com, Brad provided all of the Canare video cables and 75-ohm terminators for the Benchmarks.

Bill Bean and Al Gaddis
Bill and Al are VM700T gurus for Tektronix. They also helped out with various CAV and VM700T questions. Bill also arranged the loaner 1735HD with the local Tektronix rep.

So, we had plenty of people and plenty of players, and all the heavy duty testing equipment we needed. The results will be posted in sections, with links below that will go live as we publish the various parts. If you have any questions about the description of terms or techniques, please send us an e-mail, and we will clarify them in updates. Enjoy!

Special Reports
Chroma Upsampling Error in DVD Players