DVD Benchmark - Part 1 - Video - September, 2000 (Updated March, 2005)
Let's start with a little technical information on DVD. I am never really sure how much to cover and how technical to get. I will do my best to make things as easy as possible for everyone to understand.
DVD is encoded as MPEG 2 video and stored on DVD following the ITU-R BT.601 standard. ITU-R stands for the International Telecommunication Union, Radio Communication Sector. BT.601 is the standard for digital component video.
BT.601 specifies how the signal is transcoded from R'G'B' to Y'Cb'Cr', the digital filters used, sampling rate, and the horizontal resolution. While NTSC (SDTV) is based on BT.601, HDTV is based on BT.709. Y is the luma channel and 'Cb'Cr' are the chroma or color-difference channels.
There are several different sampling formats for Y'Cb'Cr', namely 4:4:4, 4:2:2, 4:1:1, and 4:2:0. BT.601 defined 4:4:4 and 4:2:2, MPEG added the 4:2:0 format because it requires less storage. We are really only concerned about the last one, 4:2:0, because that is the format that DVD uses. I should also mention that there is more than one version of 4:2:0. What 4:2:0 means is that not only are the chroma channels reduced 2:1 horizontally compared to the luma channel, but they are also reduced 2:1 vertically.
The best would have been 4:4:4, where there is no reduction for chroma and luma, but that takes up too much space on the DVD.
The sampling rate of luma (the black and white, brightness, portion of the signal) is 13.5 MHz, which provides 720 active samples per line. The sampling rate of chroma (the color portion of the signal) is 6.75 MHz. The Nyquist theorem states that the sampling rate must be twice that of the highest frequency so that aliasing will not occur. This limits the maximum luma bandwidth to 6.75 MHz (half of 13.5) and the maximum chroma bandwidth to 3.375 MHz (half of 6.75).
We generally speak in terms of resolution, not bandwidth. Each MHz represents 80 horizontal lines of resolution or television lines (TVL). (This only applies to 4:3 TVs.) That equates to DVD having a maximum TVL of 540. I have only seen a couple of DVDs that actually contain information out to 540 TVL, and those are Avia and the MS Test Annex. (WHQL)
Recently, a couple of DVD player manufacturers have begun advertising that their players are capable of upsampling the chroma channels. The fact is that 4:2:0 must be converted to 4:4:4 before it can be displayed. Interpolation is used to generate the new color difference samples. How those samples are generated can vary from processor to processor, and some algorithms are better than others. The DVDO Pure Progressive™ engine uses an excellent technique to perform the "upsampling."
Y'Cb'Cr' data for DVD are encoded with 8-bit precision. While a couple of first generation DVD players used 8-bit video DACs, 10-bit video DACs are now the standard. These extra bits of resolution provide headroom. Some companies are now moving to 12-bit and 14-bit DACs. Most of the DACs run at 27 MHz or 2x oversampling. (That's 2x the sample rate of luma's 13.5 MHz.) This oversampling helps the video reconstruction filters.
Because the DACs are running at twice the sample rate, more interpolation is done and additional samples are created resulting in 8:8:8 Y'Cb'Cr' with 1440 active samples per line. Remember that this is not real information but interpolated information. Who do you suppose the first manufacturer will be that makes that claim?
Some newer players have DACs that are running at 54 MHz. If the DVD player is outputting a progressive image, it has to generate twice the number of scan lines in the same amount of time as an interlaced player. That requires the DACs to be at least 27 MHz. Through oversampling, and by going to 54 MHz, the DACs once again have 2x oversampling on the progressive output. I guess that makes it 4x oversampling on the interlaced output. As high-end players begin to offer video scaling (converting 480p to 600p, 720p, 768p, 960p, etc.), they will need to utilize even faster DACs.
Once the MPEG 2 video has been decoded and converted to analog, it should no longer be referred to as Y'Cb'Cr'. The correct label is Y'Pb'Pr', which is the proper designation for Component Analog Video (CAV). The CAV labeling of DVD players is not consistent, and many use the improper term Y'Cb'Cr'. Note that CAV in the video realm (Component Analog Video) is different than CAV for the mechanical movement of laser beams on tracks of the discs, where it means Constant Angular Velocity.
Results of all our Benchmark Tests are located HERE.
Toshiba has come up with a clever name for CAV called Color Stream. That's fine, except that no one else uses that term, and we constantly get questions asking if the Color Stream outputs on their DVD player will work with the component video inputs on their TV. Its gets even better, since they call progressive DVD outputs Color Stream Pro. And if that wasn't enough, their DVD players with 54 MHz DACs are Super Color Stream Pro. I have no problem with the concept of marketing, but we are talking about using some clever terms that actually refer to standards.
One more thing I would like to mention at this point is the connector used on the CAV outputs. Since this format was adopted from the commercial TV broadcast industry, it only makes sense that the DVD players would use the same BNC connectors that industry uses. Nope! There was a chance to do something right from the beginning, but the first ones on the block with CAV outputs chose RCA connectors. You will only find BNC connectors on the high-end DVD players. The importance of the jack stems from a need for 75 Ohm connections, due to the high frequency of the video signal. BNCs are 75 Ohms, but RCA jacks are often much lower than that, say 25 Ohms, which will degrade signal transmission through reflections at the connection point.
There is a new type of output that is being offered on at least one brand of player, and it is called Serial Digital Interface (SDI). SDI itself is not new, it is defined in BT.656 and SMPTE 259M. Theta, who offers the output, is calling it the 4:2:2 digital output. This output is interlaced and does not contain any of the MPEG flags that can aid in the de-interlacing process. A few digital display devices offer SDI inputs along with two video processors, Miranda and Snell & Wilcox.
CAV (Component Analog Video)
If implemented properly at the source, video processing and display, CAV offers the best video quality from DVD and should be used whenever possible.
In the broadcast world there are several different versions of CAV, which include SMPTE/EBU N10, MII, and Betacam®. So which one did they adopt for DVD? How about none of them! In fact, up until recently there was no standard at all for consumer CAV. There are now three CAV formats for consumer devices, and they are all part of the EIA 770 standard. EIA 770.1 and 770.2 are for both interlaced and progressive SDTV, while 770.3 is for HDTV. EIA 770.2 looks just like SMPTE/EBU N10, so I will not list it. I also want to mention that the Y portion of EIA 770.1 looks like Betacam® Y, while the 'Pb'Pr' portions look like SMPTE/EBU 'Pb'Pr'.
Since we measured the output levels of each player, I am going to list which CAV format a player most closely resembles. Before I can do that, I think it is best to actually tell you about the various CAV formats. Armed with lots of data and Microsoft Excel, I have put together some graphs to visually show the level differences and I will include the actual values too.
What makes things even more interesting is that the devices you are going to plug the DVD players into, like your TV or video processor, are expecting a certain CAV format, but which one? Faroudja, for example, calibrates all of their video processors for the Betacam® format. Not a single DVD player that we measured outputs Betacam® levels, but as long as the TV or processor you buy has user adjustable controls like brightness, contrast, and color controls, you can calibrate the levels. Some devices even let you adjust tint on the CAV format. Normally this is not possible because chroma phase is not part of the CAV format. However, it can be done by mixing 'Pb' and 'Pr' proportionately. Toshiba TVs do allow you to adjust tint on the CAV input.
While we are on the subject of Toshiba TVs, it would make sense that if a manufacturer makes both DVD players and TVs, and that if you connected the two together, they would be calibrated to work without any adjustments. Not so! You even have to adjust tint on the CAV inputs to get them to match.
If you are unable to utilize the CAV outputs on your DVD player, each player also comes equipped with S-Video and Composite outputs. The NTSC standard for the composite/YC format is SMPTE 170M. With one format in NTSC land, you would think that they could get those output levels correct, but manufacturers have failed to ensure that their outputs conform to this standard. Most devices have brightness, contrast, color, tint, and sharpness controls so that you can calibrate the levels.
I feel it is important to mention the lack of standards, because if you are evaluating a DVD player, it MUST be calibrated to the display device you are viewing it on so that it's a fair test. This is especially important when comparing CAV to S-Video or Manufacturer A to Manufacturer B.
There is one product, the DVDO iScan Plus, which does not have any user adjustable controls. They brought this groundbreaking product to market at an unbelievably low price. They made an assumption that the DVD players would output the appropriate levels. They lucked out on tint, if S-Video is used, because every single player we tested had perfect chroma phase (tint). However, when it came to the chroma level (color), the players were all over the board.
Due to limited time and the fact that the Composite output has too many compromises, we only measured the S-Video (YC) output of each player.
DVD Player Video Measurements
I am not aware of any standard measuring practice in regards to DVD players and their video performance. Most of the reviews that I have read have all been virtually subjective. There is nothing wrong with subjective reviews, but sometimes taking a closer objective look is important.
We paid a great deal of attention to detail in our measurements right down to the smallest item. We did our best to ensure that every player was tested on an even playing field.
We chose the Tektronix VM700T as the test and measurement device. It is highly regarded as one of the best, if not the best around. The VM700T was equipped with option 1 for NTSC Composite/YC testing and option 30 for CAV testing. The VM700T provides 3 channels, which was enough to measure both the YC (2-channels) and CAV (3-channels) outputs. The VM700T is only useful for interlaced video. We had to use a Tektronix 1735HD to look at the outputs of the progressive players. We were not able to get the detailed information with the 1735HD as we did with the VM700T, but we got what we could.
The VM700T is designed for use in a video broadcast environment and has a video input and output for each channel. At the end of every video chain, the signal must be properly terminated. In our case, the VM700T was at the end, so we had to terminate the outputs in order to get accurate measurements. Since the quality of the terminators will affect the measurements, it is very important to use 75-Ohm terminators with very little tolerance for error. For our DVD benchmark, we chose Canare terminators. Prior to getting the Canare terminators, I had a handful that I picked up from the local electronics shop. I could actually touch them and the measurements would change. The Canare terminators exhibit no such behavior.
Since we were so particular about the quality of the terminators, you can imagine that we also followed the same methodology for video cables. The most important thing about cables to me is having the proper connectors. The VM700T uses BNC connectors. Some DVD players use RCA, while other use BNC. I did not want to use converters anywhere in the chain. I had three 1-meter cables made for the tests, each cable having the necessary connectors: 1 BNC->BNC, 1 BNC->RCA, and 1 RCA->RCA, all of which having Canare terminators. The specific Canare cable is V3-5CFB.
The VM700T does not have a YC input, so a special box was required that would convert the single Din plug into two BNC cables. For this I used a TecNec SV4-YC breakout box. I could have had a special cable made with an S-Video female on one end and two BNCs on the other, but I did not.
For lab grade power, we went with the PS Audio P300 Power Plant AC regenerator. It does not matter what the AC coming out of the wall looks like because the P300 completely regenerates it. This guaranteed that all players had equal power. Colin made some noise floor measurements on the audio side, with each player plugged into the P300 and then into the wall. Did the P300 make a difference? You can read the audio portion to find out. I would also have liked to feed each player really bad power to see how each power supply performed, but we ran out of time.
Aside from the actual players, the only other necessary part was the test patterns. We used two discs (DVDs) for our video measurements, Avia and Video Essentials. One of the most interesting problems with test patterns and DVD is knowing whether or not the pattern you are using is accurate. The MPEG 2 encoder can do some pretty wicked things to test patterns. There were actually a couple of measurements that we did not make because we felt the patterns themselves were not reliable enough. We also discovered that we did not have access to all of the test patterns we really needed.
While on the subject of accuracy, just because we had the VM700T did not mean it was accurate. The VM700T is a precision device, and Federal Express delivered it to my house. We used a Tektronix TSG-170a test pattern generator to verify the accuracy of the VM700T. From our quick test we can safely say that the VM700T was accurate within +/- 0.04 dB.
A series of measurements were first made on the CAV outputs, followed by the YC outputs. Below are the test patterns we used and the various measurements we made. I tried to do my best to completely document how the measurements were made so that they could be reproduced by anyone with the right equipment.
We also used the Extron CVC-200 CAV-to-RGBHV transcoder, so that we could look at the progressive image on a PC monitor, which only accepts RGBHV. We only evaluated a player's progressive performance through the transcoder.
Black and White Levels . . . Setup
The first thing we measured were the black and white output levels of each player. Before I can explain that I guess I had better explain the term setup.
Setup is the offset used to separate the active video from the blanking level. That offset is 7.5 IRE. If setup is not used, then the black level and the blanking level are at the same level, which is 0 IRE. Setup may also be referred to as pedestal. Setup is only used in NTSC. As I mentioned previously, some players give you the ability to turn off setup. I actually encountered a couple of players that had no setup on the CAV output and no way to add it. All players should at least have setup on by default and then offer the ability to turn it off. The Dwin TranScanner video processor assumes that the black level is at 7.5 IRE and clips everything below that.
All of our video measurements where made with setup, and this choice was made because I live in NTSC land (USA). Given more time I would also liked to have made measurements without setup.
Note: The graphics shown below are clickable to see larger versions for more detail.
We used the blues bars (Title 4, Chapter 1) pattern on Avia to measure the black and white levels. The measurement was made around line 220 near the bottom of the pattern where you see the 100 IRE white bar on the scope. Breezeway is taken as zero. We measured white at the top of the white bar deflection and black at base of white bar.
The values are in IRE, and a perfect player would have white at 100 IRE and black at 7.5 IRE. Some of the players we measured had white above 100 IRE, which can be bad. The Dwin TranScanner clips (removes) everything above 100 IRE, and other video processors may do the same.
If you are comparing two players with each other, and their black and white levels do not match, you can use the contrast and brightness controls to match the players.
One last thing that I want to mention is that, because of these output differences, the gray scale must be adjusted for the player one is using. If you change out the player, or even switch from YC to CAV, you will have to recalibrate your gray scale.
The change to the gray scale may not be major, but the fact that you had the gray scale calibrated in the first place means that you do care about accuracy.
Using the same blue bars pattern, we measured the color levels. Here is where you can pinpoint which CAV standard each player is outputting. This measurement was made on line 50. Both option 1 and option 30 automatically calculated the color bar values and provided the results.
For the YC output, Luminance Level, Chrominance Level, and Chrominance Phase were presented in IRE. Each player will be presented in graph format compared against SMPTE 170M.
For CAV output, Y, Pb, and Pr were given in mV. Each player will be presented in a graph compared to SMPTE/EBU N10, MII, Betacam®, and EIA 770.1.
The color and tint (for YC only) controls can be used to match the DVD player with the display device.
We also looked at the color bars using the vectorscope display. This display is used to quickly tell you if the signal is within tolerance. These are used throughout a production studio to ensure that the video signals are not being decimated somewhere in the chain.
Whenever a manufacturer provides you with their resolution specs, they always seem to say that their player outputs >500 lines or 540 lines, but what does that really tell you? There is more to the equation than just a number.
First, I need to explain the term amplitude. Amplitude represents the size of the signal. For our measurements, amplitude will be expressed in dB.
When we measure resolution, what we are really measuring is the frequency response. Perhaps it is more correct to say we measure the amplitude of various frequencies within the range of the device, in this case the DVD player. Remember from the video basic discussion that DVD players are able to reproduce frequencies up to 6.75 MHz.
The two most common test patterns used to measure the frequency response are the sweep and multiburst. A sweep signal will start at a specific frequency and will continually increase in frequency until a specific frequency is reached. The Avia sweep pattern ends at 5 MHz, while VE continues to 5.5 MHz. A multiburst pattern gives your discrete frequencies packets. The Avia multiburst contains 7 frequency packets. The downside of the multiburst is that you do not know what the frequencies look like between the discrete packets.
What we measured was the peak-to-peak amplitude of each packet. If the amplitude deviates from zero either +/- dB it results in linear distortion. We made our frequency response measurements on line 50. The VM700T is capable of measuring 6 packets at once. The test pattern on Avia contains 7 packets, so we skipped the very first packet, which is 0.5 MHz. We made our measurements at 1 MHz, 2 MHz, 3 MHz, 3.58 MHz, 4.18 MHz, and 5.0 MHz. The 1.0 MHz packet was used as the reference frequency.
Measurements were made on both the YC and CAV outputs. I am not aware of any reason why there should be any measurable difference between the two outputs, but in many cases there were. I suppose if a difference did exist, I would hope that the CAV output is better. Unfortunately this was not always the case.
We also looked at the sweep pattern on every player to verify that nothing funny was happening between the multiburst frequencies. They were all ok in that regard.
I said earlier that if the amplitude deviates from zero, this would result in a linear distortion. If it's a + dB distortion, then it may show up as ringing (halos at edges) in the picture. If it is a - dB distortion, then fine detail may appear softer, blurred, or not at all if the roll-off is steep enough.
What is ringing? Ringing and often called overshoot shows up as a faint artificial line around an object's edge.
The sharpness (Title 3, Chapter 28) pattern on Avia was used as a visual verification of ringing in a DVD player. You need to follow the instructions that accompany the sharpness pattern to properly set the sharpness control on your display device. Once that is done, you can look at the test pattern to see if any ringing exists.
On a few players, there will always be ringing no matter where you set your sharpness control. The fault often lies with the filters used in the DVD player. If you are evaluating the picture quality of DVD movies, you need to be aware if the ringing you see in the picture is in the transfer or an artifact of your DVD player. Another source of ringing can come from your display device. A nasty feature that many TVs employ, and which results in ringing, is called Scan Velocity Modulation.
A third pattern that is very useful, but was not available to us, is a multipulse test pattern. Along with measuring the frequency response of the player, you can also measure the group delay.
What is group delay? It's a distortion that happens when some frequency components of a signal are delayed more than others. What does this look like in terms of picture quality? It shows up as a lack of vertical line sharpness.
On a few players, we did look at group delay even though we did not have a multipulse or 2T pulse pattern, which are needed to easily make such a measurement. We were able to indirectly test for severe group delay anomalies by measuring the reproduction of the resolution wedge on the WHQL DVD. We captured line 129, field one of the "Weave Freeze Frame" test slide on the WHQL DVD. We also measured the original DigitalBeta from which that disc was made and used those values as our reference.
The Pixel Cropping (Title 5, Chapter 89) pattern on Avia was used to see how much of the DVD image the player is delivering. To measure the pixel cropping of a DVD player, a display device is needed that has no overscan. This is a visual verification test.
In the NTSC system, it takes 52.65 µs (microseconds) to draw a scan line. It just so happens that it takes approximately 0.07407 µs to draw a single pixel. If you multiply 0.07407 µs by 720 pixels you end up with 53.3304 µs, which is 0.6804 µs more than the time it takes to draw a scan line. If you divide 0.6804 µs by 0.07407 µs, you end up with 9.19 pixels less than 720 pixels, or approximately 711 pixels.
Those extra pixels will be cropped from the sides, and there is no hard and fast rule where they come from. They can all come from one side or the other, or even be cropped from both sides. The degree of cropping will vary from player to player, and some players are even cropping from the top and bottom.
It is also possible for a DVD player to output all of the pixels. Computer DVD players usually do this because they do not have to conform to NTSC timing. This also would not be a problem if they followed BT.601 vs. SMPTE 170M timing.
We measured the cropping from all four sides of each player. We only counted the pixels using the YC output, and we made the assumption that both the YC and CAV outputs were the same.
We used the Red Field (Title 4, Chapter 18) and Magenta Field (Title 4, Chapter 19) on Avia to measure the signal-to-noise ratios of each player. What is signal-to-noise? It's the ratio of the amplitude of the signal at full output to the amplitude of the noise floor.
The red field was used for the YC output, and the magenta for the CAV output. I did go through all of the color fields on two DVD players, and the noise measurements did not really change.
Manufactures usually provide the number 65 dB when they tell you the signal to noise ratio of their player. I am not sure really what that number means, but they might be referring to the composite output of the player. The YC and CAV actually have more than one output, and will have more than one number; perhaps the 65 dB is an average.
The measurements were made on line 50 of each pattern, and are are slightly different for the YC and the CAV outputs. The YC consists of two measurements, Chroma AM (Amplitude) and Chroma PM (Phase) noise. Both of these numbers are generated by the AM/PM noise measurement in the VM700T option 1. The CAV measurements are generated by option 30 and consist of Y, Pb, and Pr noise levels.
Noise levels are also referred to in dB or -dB where, the lower the number, the better. When I say lower I mean -65 dB is better than -35 dB.
The larger your display device (such as with a front projector and 6 foot screen), the more you will appreciate the better signal-to-noise ratio on a DVD player.
Component Channel Timing
We used the bowtie pattern to measure the channel timing (delay) between the CAV channels. It is very important for the three signals to be time aligned. Delays can cause vertical edges, and fine detail to be soft, fuzzy, and even improperly colored.
A delay of 10 ns (nanoseconds) is noticeable, and will cause color smearing near the edge of objects. It takes a delay of 74 ns for a complete pixel to be delayed. I will note here that the worst delays were from two THX certified DVD players.
I mentioned earlier that we did not have all the test patterns we would have liked. A multipulse pattern would have been useful to measure group delay. A chroma frequency response pattern would have been useful to see how the chroma channels matched the luma channels in terms of frequency response.
Another set of measurements we could have made was differential phase (DP) and differential gain (DG), because test patterns are available on both Avia and Video Essentials. However, we felt that the inherent noise in the test patterns on both discs was so high that they would have produced inaccurate results. They would have shown the relative difference among the players, but we wanted more. It is very unlikely that you will experience any DP and DG problems because the video is generated digitally.
DP and DG occur when the chroma phase and gain levels are not processed equally at all luma levels. How does this affect the picture quality? Phase affects the hue or tint while the gain affects the chroma or color. You get changes in the hue or chroma as the brightness in the picture changes. Brighter portions of the picture may appear discolored.
We will be gathering more test discs and procedures as time goes on.
I have provided you with a lot of information on the video side of DVD. Please don't be confused by all of the color space information that I threw at you. It is meant merely as a reference and to show just how important it is to adjust your TV to match your DVD player in order to get the most out of DVD.
Standards are in issue in any industry. The problem with the consumer electronics industry is that no one adheres to all the standards. Now that you know what they are, you can demand more from the manufacturers. Just look at what the ISF has done on the display side. A few years ago, no one cared, but that has all changed. Manufacturers, like Toshiba, have listened, at least a little bit, and we hope others will too.
- Stacey Spears -
Avia images Copyright 1999, Ovation Software
Video Essentials images Copyright 1997, Joe Kane Productions
DVD Benchmark - Technical Article Index
Part 1 - Video Part 2 - Audio Part 3 - Functionality Part 4 - Usability Part 5 - Progressive Scan Part 6 - DVD Audio
© Copyright 2000 Secrets of Home Theater & High Fidelity
Return to Table of Contents for this Issue.