Articles

The Secrets Blu-ray Player HDMI Benchmark - Part I

ARTICLE INDEX

Introduction

It's hard to believe that we are already into the second decade of the 21st Century, and all future Blu-ray players will have to deliver their HD content digitally over HDMI only, with analog component video outputs limited to 480 lines of resolution (480p). Since all of the content will be transferred digitally to your display, this should result in perfect picture quality with no issues since "bits are bits", right?  Unfortunately the reality is far more complex than this, and there are many factors that can come into play.

This article is an introduction to the way we will be changing how we benchmark Blu-ray players, since we will be taking measurements of the digital signals through the HDMI output instead of analog data measurements through the component video outputs. There is a lot to discuss, so start reading now, and you won't be confused when we begin publishing Blu-ray Player Benchmark Reviews with our new methods shortly.

In the current article, we will lay out what these factors are, how we are going to test for them, and provide results showing what this data will look like. The reason that "bits are bits" doesn't apply for Blu-ray is that the storage medium and our display devices work in different languages. HDMI will transfer all of this data between the devices correctly, but how the devices and displays handle that data is what we are investigating.

All current display devices use RGB to display color on the screen. White is the sum of all those colors, and black is the absence of all that color, with the remaining spectrum coming in-between. The easy way to store this on Blu-ray would be in RGB as well, but unfortunately Blu-ray isn't close to large enough to save raw RGB data for a whole film to a disc. Because of this, data is stored on Blu-ray in YCbCr format using compression.

The compression that Blu-ray uses is very effective because it values information that we can observe more easily – black and white (luma) over color information (chroma), which the eye isn't as sensitive to. Blu-ray saves all of the luma data while disregarding much of the chroma information to save space, and then compressing the data. Additional information on this storage format can be found here: http://en.wikipedia.org/wiki/Chroma_subsampling. This difference in how data is stored compared to how it is are displayed leads to the two main issues that we can run into with Blu-ray players and that we are going to investigate and report on.

Color Conversion: As no consumer display can handle the YCbCr 4:2:0 data that is stored on a Blu-ray disc, they must first be converted to another colorspace. The colorspaces that you will see on consumer equipment are 4:2:2, 4:4:4, and RGB. While in theory this should be a conversion that everyone should do correctly, there is the ability to make a mistake and have data remap incorrectly in the process.

Color Space Handling: If you have looked at our recent receiver and processor reviews, you have seen that we have started to grade them on how they handle certain color spaces. While ideally every component would be able to handle 4:2:2, 4:4:4 and RGB without issue, some components cannot and lose color data or resolution along the way. Additionally, we have found that some consumer displays might only display one or two of the three formats correctly, and so unless all of your source components can display that format, you will never get the full resolution out of your sources.

Because of these color space issues, we believe that all Blu-ray players should offer the option for the user to select between 4:2:2, 4:4:4, or RGB color spaces for output and that it should do the color conversion to these from 4:2:0 correctly.

Additionally, players should offer a Source Direct mode that outputs the data as closely as possible to how it is displayed on the disc. In Source Direct, the only alteration made to the data is a conversion from 4:2:0 color space to 4:2:2, but no adjustment in resolution or anything else. In this case, as the video output is bypassing the image processing of the Blu-ray player, any color errors introduced are often the fault of a hardware component and are often not able to be fixed in a software update since it is a hardware flaw.


Why This Matters

Your first question might be, "Why does it matter if the color conversion isn't quite perfect?" or, "I don't know how to choose a color space, so why would this affect me?"

These are very valid questions, so let us look at what could happen. From the players we have analyzed so far, if your player does incorrect color conversion, you can suffer from many issues including:- A loss of dynamic range in your image. Outdoor scenes won't be as bright as they should be, and night scenes won't be as dark. Image information will be squeezed closer together and not appear as it should be.

- Colors will appear incorrectly from your Blu-ray player compared to HDTV. The color of red from your Blu-ray player won't be the same as your color of red from other HD sources, even if your display has been calibrated.

- You can lose resolution in color or black and white that should be there. Artifacts might appear that shouldn't be there, blue skies can suffer from banding instead of being smooth, and color transitions won't happen correctly, resulting in abrupt changes instead of being gradual.

- Data that you should be receiving might be clipped off incorrectly, or data that you shouldn't get might be visible when they should be hidden.

In short, you will never get the full performance from your system, no matter what you do, if the color conversion and color space selection is incorrect. Note, however, that most Blu-ray players deliver a picture that is good, in general. The color errors that we will report on in our Benchmark reviews are not defects that jump off the screen. Otherwise, the manufacturer would have noticed them and fixed the problems. The function of our Benchmark is to show not only our readers, but the manufacturers, where errors are found, so that they can be corrected in future models.


What We Will Report

Now that we know the issues that exist, how are we going to report on them to our readership? A Spears and Munsil Blu-ray test disc has a set of test patterns that let us evaluate the color conversion of players, either in RGB or YCbCr color spaces. Using a Quantum Data HDMI Analyzer, we can read these values directly and compare them to those that were encoded on the disc. 

There is no guessing based on visual observation, but instead we have the actual data to report on. We will look at multiple intensities of the primary colors (R, G, B) or the YCbCr components (Y, Cb, Cr) and compare those to their reference values. We will look at 0%, 25%, 50%, 75% and 100% amplitude levels for each to see how they compare.

Finally, we will take these values and compute the Delta Error, or dE, for these conversions using the CIE 1994 formula. If you have looked at a display review, the dE is the difference between what the color should be and what the color actually is. With a display, a dE of less than 1 implies a non-visible difference, and this is what you aim for with a display calibration. A value of over 10 is high, and the color errors are noticeable by a viewer. In most cases, you would see a dE less than 3 from a player, which isn't visible, so why would this be important?

Almost any display that you purchase will have a dE for all colors, even after calibration. You might be able to reduce that value with calibration, but it will still be present. If you are introducing a certain amount of dE from your source component as well, then it is going to be far harder to get those values below the visible threshold of 3 than if your source had a dE of 0. Additionally, you are going to want to use a source with a dE as close to 0 as possible to calibrate a display, since otherwise you will be applying the correction for that error to every component connected to your display, introducing it into devices that didn't have the error.

So, your ideal source will have output values that match the reference values on the disc, a dE of 0, and support all available color spaces: 4:2:2, 4:4:4, RGB and Source Direct. Players that don't support all these modes, or introduce errors, will no longer be able to be fully recommended by us, as it would be impossible to guarantee that they would work in a different system correctly.


Data Examples

For our examples here, we went to players we have reviewed here previously: The Sony BDP-S570, and a set of OPPO players (BDP-83SE, BDP-93, and BDP-95). All of these players were very well regarded during our prior looks at them and have garnered year-end awards from us as well.

How would they hold up to more advanced testing that wasn't possible before?

Let's begin with the OPPO players. They support all of the color spaces we want, as well as a Source Direct mode. Additionally, the 93 and 95 have dual HDMI outputs on them, so we need to test both of them to see if one offers better performance than the other for video. Numbers in green mean the player passed the test, and if in red, the player failed the test.

Looking at the Oppo BDP-83SE, the data are spot-on. The dE in the output stays near 0, with an average dE of 0.1 for RGB. 4:2:2 and 4:4:4 come across perfectly when compared to their reference values. The OPPO does have a Source Direct mode, but we did not test that on the players this time. The Oppo behaves perfectly on the tests and would work well as a reference point for your video system, and we could easily use it to calibrate a display.

Moving onto the newer 93 and 95 models from Oppo, these have the same video chipset in them, so we expected the same video performance, and that is what we got. The only errors were over HDMI 2, and only at RGB levels 0 and 255, which were 1 and 254 respectively. All other RGB values between these were perfect, and on HDMI 1 they were all correct. As OPPO recommends using HDMI 2 for audio or a secondary display, we would suggest the same; though the amount of error here was nothing we would ever think twice about.

Similarly, 4:2:2 and 4:4:4 data looked perfect over both HDMI 1 and HDMI 2. The OPPO handled the color space conversions perfectly, and output all data formats correctly, so once again this player could be introduced into any consumer system and work ideally. Other aspects of the OPPO performance have been covered in our reviews of the players, so we will stick to the HDMI data alone, and on that, all of the OPPO players were fantastic.

Next, we looked at the Sony player. It also supports 4:2:2, 4:4:4 and RGB color spaces, but does not have a Source Direct mode that the OPPO has. It also only has a single HDMI output, so there are much less data to look over than with the recent OPPO players. Let's look at the chart of the Sony data below, and then the implications of those results:

Unlike the Oppo players, we have a lot of data here that is different that the reference values. Lets go over some of the issues and why they are problems.

- The Y value for 4:4:4 and 4:2:2 color space is incorrect. In this case, the Luma value, which is the black and white image and responsible for the majority of the detail you perceive, is being radically adjusted by the Sony.

- WTW (Whiter than White) and BTB (Blacker than Black) data, which exist on Blu-ray discs, but you will typically not see much of, will be visible in the image when they should be hidden on a properly calibrated display. The BTB data will be correct here, but values over 235 that are WTW would typically be hidden after a calibration. With this player, the highest Y value output by the Sony is 232. All values above that have been adjusted down below the WTW mark, and a value that would typically be bright white (235) has been adjusted down to 224, or a very light gray instead of white.

- In addition to passing all this WTW data, the dynamic range of the image has been altered. Take the peak white value of 235. A value of 225 should be a full 10 steps below that, but on the Sony, the 235 renders it as 224, and the 225 comes out at 219. A dynamic range of 10 has been shrunk down to 6. The overall dynamic range we would expect from the Sony has gone from 220 values (16-235) down to 209. That's a loss of 5% of your dynamic range.

- Looking at the RGB values, you can see the dE numbers there for the five values for each color, and an average of the 25%, 50%, 75%, and 100% values (0 has been omitted as it's less important and less visible). Red has a dE of 0.9, and Blue has a small dE of 0.1, but Green has a dE of 4.7. Because of the error in green, there is no way to calibrate a display to not have an error in how it displays green that is visible to the end user without introducing other errors.

- Finally, these issues are present in all the color spaces, not just RGB. Because of this, you can't select a different mode to get around this issue.

We were quite surprised by these data, as it seemed that everyone, including ourselves, had loved the Sony player. Comparing the output of the Sony with an OPPO player on a calibrated reference projector, and using the Spears and Munsil HD Benchmark Blu-ray test disc, it was possible to see the problems. Gradients were perfectly smooth on the OPPO but were blocky and showed the effects of the lack of dynamic range on the Sony. With the WTW patterns, it was possible to see all of the white blocks since they were adjusted down below where the contrast was calibrated, but on the OPPO, the WTW blocks correctly blended into the white background.

With these issues, I can no longer recommend the Sony player, because its output is incorrect and will compromise the performance of the rest of your system. The OPPO players might be more expensive than the El-Cheapos at the bargain counter, but they offer every single color space that you might need, Source Direct mode, and have perfect color conversion with a dE of 0, so it won't be holding back your system at all in image quality.


Conclusions and Industry Feedback

When Secrets of Home Theater and High Fidelity began publishing the DVD Benchmark, issues like Chroma Upsampling Error (CUE) and improper cadence detection were common, but now those issues have been mostly eliminated, since manufacturers have a reference to achieve. We hope that our new Blu-ray Player Benchmark testing, using the HDMI output, can help to lead manufacturers to the same level.

Following, are comments from industry representatives:

SpectraCal, makers of CalMan calibration software:

"Over the last couple of years display manufacturers have started listening to our requests for better control over their displays to attain greater color accuracy. For the ones that are working with us the results have been impressive; however, if the sources are unable to produce the same performance it becomes a major bottleneck in the video system.

Our customers frequently contact us regarding poor source decoding. It typically happens when they are relying on their optical player as a reference source to calibrate their display's grayscale and gamut instead of using a reference pattern generator. This is a major issue as many of our customers' primary content is watched from their optical player and because color accuracy is most critical to their experience. What we end up telling our customers is, "if you rely on your optical player (Blu-ray or DVD) as your primary source and you use it to generate reference test patterns from a pattern disc to calibrate your display(s), there is no guarantee you will end up seeing the content as it was originally intended. This is due to the possibility that you will have improper source decoding and perceivable deltaE in output levels. If this occurs, your entire video chain including the display will not be calibrated properly and the quality of the original content will be lost." This advanced testing will help consumers to avoid these issues and get the most out of their system."

HDMI.org:

"The HDMI Specification defines a clear data protocol that allows test equipment makers and product reviewers to get detailed, accurate information on how a product behaves", said Jim Chase, Director of Technology at HDMI Licensing, LLC. "This feedback allows product manufacturers to achieve a greater variety of product options and a higher level of performance and quality than previously attainable."

Tom Huffman, ISF and THX Certified Calibrator, maker of ChromaPure calibration software:

"The testing that Secrets has done provides consumers with significantly more information than was previously available about the performance of Blu-ray players. It is important to see how well color information is transmitted to the display at different levels of stimulus as this better reflects real-world performance."

We want to acknowledge the direction and contributions of Stacey Spears and Stephen Hornbrook, developing and executing the new SECRETS Blu-ray HDMI Benchmark approach.

The SECRETS review results for Blu-ray players utilizing this new Blu-ray Player HDMI Benchmark as integral to the reviews, begins in June.

FEATURED COMMENTS

What Quantum Data equipment do you use
Written by Kim Fredtoft , May 23, 2011

Very impressive, and something I've long wanted to do myself, but newer found the equipment (and budget) to do.

What piece of Quantum Data equipment do you use to monitor the actual values on the HDMI output of the players, and what is the approximate cost?

All the best

____________________________________________________________________________________________

Quantum Data
Written by ChrisHeinonen , May 23, 2011

We used a Quantum Data 882 HDMI Analyzer and I don't have the exact cost, but it's in the neighborhood of $16-18,000. I looked into as many other analyzers as I could and you lose either the ability to look at HDCP data, or you lose the ability to analyze all colorspaces and so they weren't able to work for us.

Just as important, however, is having access to the test patterns that allow you to know what data you are looking for as otherwise you're just guessing as to the values of the original source data.

____________________________________________________________________________________________

Please, bench PS3!!!
Written by Onesolo , May 23, 2011

Are you going to bech Sony PS3 (fat and skinny) ?!?! Hope so...

  • CLICK TO VIEW MORE COMMENTS

    PS3 Bench
    Written by ChrisHeinonen , May 24, 2011

    On our next batch of player reviews, we will try to bench the PS3 as well for reference. I'm not sure when this will be, but we will try to get it included as we know that's of interest to people. I believe I have access to both an original and updated version as well, so we can do that.

    ____________________________________________________________________________________________

    What about the Pioneer BDP 51?
    Written by Paul , May 24, 2011

    Please test this player under your new "testing guidelines". I, as well as many other Pioneer owners would be very curious to know what/if any, delta errors this players has.

    Thanks

    ____________________________________________________________________________________________

    Pioneer BDP-51
    Written by ChrisHeinonen , May 24, 2011

    Paul,

    I reviewed the Pioneer a long time ago, but I don't have one around now to use for testing, and Pioneer wouldn't have any samples to send us at this point for testing either. We do have a couple of Pioneer players coming up for review soon so you can keep on the lookout for those, as they do use this new testing format. The only other player that I have easy access to beyond the PS3s at the moment that we haven't reviewed is an LG 570, which I will try to bench next time for reference. Thanks.

    ____________________________________________________________________________________________

    Yes, please benchmark the PS3
    Written by Helen , May 24, 2011

    Yes, please benchmark the PS3. I have the skinny one and am very curious how it measures.

    ____________________________________________________________________________________________

    Ah gheez..
    Written by Kevon Manuel , May 24, 2011

    I really wish I didn't see this article. I just got over my excessive tweaking stage. Now every bluray movie I watch I'll have to pause and tweak..thanks, dude... thanks.

    ____________________________________________________________________________________________

    Confused
    Written by Paul S , May 24, 2011

    I'm a little confused with the numbers. You state for the Sony, Cb Reference and Actual as 1 and then state a deviation of 3.7?

    How does that work?

    ____________________________________________________________________________________________

    Typo or misprint or ?
    Written by larry , May 25, 2011

    The chart for the -83SE shows no errors for both YCbCr formats and RGB. The text below the chart says "Looking at the Oppo BDP-83SE, the data are spot-on. The dE in the output stays near 0, with an average dE of 0.1 for RGB.

    ____________________________________________________________________________________________

    Answers to some questions
    Written by ChrisHeinonen , May 25, 2011

    Some questions have come up here, or at other sites, so here is some clarity.

    Stacey Spears Involvement: Stacey Spears and Don Munsil created a new series of test patterns that display the full range of RGB and YCbCr data in 1 step increments. These patterns aren't generally released yet, but I believe they will be coming later this year from them. We have a set of them on a burned BD-R, and actually we couldn't test one player since it couldn't read a BD-R disc.

    The testing was done at Stacey Spears house, with him assisting us as he knows the QD machine much better than we did at that point. For data that seemed very strange (like the Sony player), we ran the tests multiple times and had the same data output every single time, so we could be certain that what we were seeing was correct. Stacey didn't do the data analysis, but after we found what the best and worse performing players were, we did use his theater (Joe Kane certified Samsung Projector and Screen, totally light controlled) to compare them more, and see what differences we could spot.

    Source Direct mode: The reason for wanting source direct mode with 4:2:2 output is simple: It's the least processing you can do to the 4:2:0 data, and then you can use an external scaler from Lumagen or DVDO to scale all content to your desired resolution and colorspace. If you don't have source direct, you run the risk of introducing more errors before the scaler, and defeating the purpose of the scaler as well.

    Numbers that don't match in the chart. This is the harder one. For example, if we are looking at the Red chart for RGB, while the Red value is what we are focused on, there are still G and B components to that, we are just less concerned with them. However, you can have a value where you are looking for Red 0 and get Red 0, but Green should be 15 and you get 14, or Blue should be 8 and you get 9. In this case the Red value is correct, which is our primary focus, but something else is a little off so you can get the reference value, but still have a very small dE 1994 introduced.

    With the Sony Cb and Cr data, since you have to use the Y data to convert from YCbCr to RGB, and then be able to calculate the dE 1994 value. Since the Y value on the Sony is so far off, even if the Cb and Cr values are decoded correctly, the resulting data is still far off due to the Y value. This is confusing in the chart but I didn't have a better way to display it without having far too many numbers, but I will try to come up with something. I felt that since dE is a concept that readers are familiar with from display and projector reviews, presenting the data as a dE value would be easier to understand at a glance.

    As far as hard data versus personal viewing, both have a purpose in the world. Visual errors are harder to pick up on Blu-ray compared to DVD, and to most people they will look almost identical among players. However, once you start moving up to larger and larger screen sizes, you can start to notice them more than before, or you can have other items in your chain that can cause issues that you couldn't see before. If I have two players and one is outputting the correct numbers and one is outputting the incorrect data, wether I think one looks better or not is unimportant at that point I feel. I can adjust my display to make things look how I want, but if my source is incorrect then I'm at the mercy of that component for how everything downstream from it looks. I can't recommend something that works incorrectly to people and feel good about it.

    The reason we had a Sony S570 available for testing is that I bought myself one for my bedroom last year. It's only been used with a 32" TV and so I never compared it to my Oppo, but I can't recommend that player to people anymore, and some of our reviewers that own it can't use it for testing the video processing of a receiver or display anymore, since the output from it is so far off that you can't tell if the receiver is clipping WTW.

    I will try to check in a few times over the next few days and answer other questions that come up about this. I'm also working on a better way to present some of this data in the future. As each colorspace we test produces 768 data points, and some players can offer 8 sets of data (2 HDMI outputs, 3 colorspaces and Source Direct) to test, I'm writing some software to better present all the data than trying to manipulate it in Excel. Also, we can't really go back to old reviews to update them as vendors no longer have those models available for us to use, but our plan is to use it on reviews going forward. Thanks!

    ____________________________________________________________________________________________

    Sony S570
    Written by Terry Jensen , May 25, 2011

    I have used your reviews for years to make informed purchases. Needless to say I was more than a little disappointed to learn of the Sony S570 players flaws. Before I purchased it I read several reviews(good & bad)& went with your recommendation. What responsiblity do you have as reviewers when you make false claims. You have lost a good deal of crediblity with me! Terry.

    ____________________________________________________________________________________________

    ...
    Written by Tyler , May 25, 2011

    This is the kind of work that made me start coming to this site many years ago. Great stuff so far.

    ____________________________________________________________________________________________

    A flawed gem...
    Written by Jim M , May 25, 2011

    In spite of the "not so stellar" performance of the Sony S570...I still enjoy it and am glad I made that purchase. It is quick, reliable and plays SACDs wonderfully. Besides, how else can one justify another upgrade in the near future?

    Hopefully, this bench testing will motivate ALL mfgs. to really ramp up the quality of their products. This is good news for all consumers!

    ____________________________________________________________________________________________

    Oppo players
    Written by Ron W , May 25, 2011

    I have owned an Oppo BDP83 for quite some time now(soon to buy a "95")and it has been interesting to play around with the color space settings to see if there is any difference between them. The "Auto" function "seems" to choose the 4:4:4 color space, however, when switching to 4:2:2, as compared to 4:4:4, the overall picture seems ever so slightly brighter with a small emphasis toward the red. The choice of the RGB color space emphasizes red even a little more.

    Any thoughts on this and which of these settings might represent a more accurate representation of the picture, especially, when using a calibration disc for monitor grayscale and color setup?

    ____________________________________________________________________________________________

    Terry
    Written by stephenhornbrook , May 25, 2011

    We did not have this information available to us when the original review of the Sony S570 was Written. Based on the benchmarks that were our previous standard, the Sony performed very well and was easy to recommend, especially given the price. While these test results are very important, the flaws present in the Sony players are only visible under certain conditions and for the most part, our eyes cannot detect the luma and color errors on NORMAL viewing material. Where this is a major problem, is when using the player as a source for calibrating a display.

    I'm sad to hear we have lost credibility with you, but I feel like, if anything, we should have gained more credibility with these test results.

    ____________________________________________________________________________________________

    Sony S570 and Colorspace
    Written by ChrisHeinonen , May 25, 2011

    Terry, I too bought a Sony S570 for myself and that it the one that we tested. We certainly weren't alone in the department of recommending it, but going forward we are going to use this type of testing. HDMI performance has been a black box up until now and we're trying to open that box up to see what's actually going on.

    For choosing a color space, the Spears and Munsil disc is very useful for this I find. There are a couple of patterns, the chroma multiburst and plate, that I come back to a lot for testing. If there is an error with those colorspaces in your display chain usually the high frequency detail will be missing, or muted in color. Additionally they have a needle pattern on there that is designed to help you choose the correct colorspace. If you don't have their disc, I recommend picking it up and their website has details on using it to choose the correct colorspace.

    ____________________________________________________________________________________________

    Error in player vs. error in display.
    Written by Jeremy Anderson , May 26, 2011

    ChrisHeinonen's post raises a question that I was never able to get a concrete answer on. I'm running an Oppo BDP-83 on an Epson 8500UB projector. When displaying the Spears & Munsil disc's chroma multiburst, the only color space on the Oppo that reproduces the high frequency detail completely is RGB. So assuming the Oppo is sending each color space correctly, does this essentially mean that the Epson is losing chroma resolution when it does its conversion from 4:2:2 or 4:4:4 to RGB to send to the LCD panels? I'm assuming that feeding it RGB essentially bypasses any color conversion in the Epson, which could be why it only fully resolves the chroma multiburst when fed RGB.

    I had actually e-mailed the fine folks at S&M about this quite some time ago and they said they were going to run it by Epson, but I never got a response.

    ____________________________________________________________________________________________

    Chroma Multiburst
    Written by ChrisHeinonen , May 26, 2011

    Jeremy,

    In this case yes, the Epson is most likely doing an incorrect colorspace conversion and losing chroma detail when going from 4:2:2 or 4:4:4 to RGB. Is this signal going through a receiver before the projector? If it is, then it could be the receiver that is doing the incorrect conversion, as virtually every machine I've tested can do that.

    The fact that displays and receivers can do this is yet another reason that Blu-ray players should offer all colorspace options, as with your Epson you would only want to use one that uses the RGB colorspace, and can do the conversion correctly. The Spears and Munsil website has a nice guide to using their HD Benchmark Blu-ray disc showing the different things that can happen to the multiburst pattern, and what the causes of that are.

    ____________________________________________________________________________________________

    Arcam Flaship Bluray Player
    Written by silvertone , May 26, 2011

    Jeremy,

    Arcam seems to have taken a purist approach to their flaship bluray player, it'd an interesting comparison agains the Oppo. They compete in the same price range.

    ____________________________________________________________________________________________

    Arcam BDP-100
    Written by das , May 27, 2011

    Re the Arcam comment. Having observed this player first hand I believe that it manipulates the image in ways that cannot be bypassed i.e. sharpening, noise reduction etc. This has also been reported by some magazine reviews and owners.

    Arcam tell me the BPD-100 uses a Broadcom solution design that appears similar if not identical to the solution used in the Anthem Blu-ray player.

    Do they compete in the same price range ? Here the Arcam is twice the MRSP of the Oppo 93 yet lacks many of its features.

    I look forward to the Secrets review of the Arcam.

    ____________________________________________________________________________________________

    ...
    Written by JJ , May 27, 2011

    There were no false claims with the Sony S570. We were simply not able to perform the HDMI Benchmark tests in the original review, so were not able to detect its color errors which were only picked up recently when we obtained access to the Quantum Data instrument and its available tests. Some of us also purchased the Sony S570 based on the original review, so we have the same player with the same color errors that you have.

    ____________________________________________________________________________________________

    Sony S570
    Written by Nick , May 27, 2011

    I too purchased the S570 based on your great review. I am happy with the player, it is quick and fully featured for my needs. I am disappointed that SONY has introduced flaws into the player and I hope that this review comes to their attention and they may be able to resolve it with a firmware update, if that's possible. I thank you for including this player in your new testing and for sharing your results with us. Knowledge is Power!

    ____________________________________________________________________________________________

    Sony BDP-S770
    Written by p5browne , May 27, 2011

    Has there been a comparison between the 570 and 770? I understand there are differences in the electronics, and possibly the setups between the 2 Models?

    ____________________________________________________________________________________________

    Chroma Multiburst
    Written by Jeremy Anderson , May 27, 2011

    ChrisHeinonen,

    I'm running my BDP-83 through an Onkyo TX-SR1007, but with it set to THROUGH and with video processing disabled completely in the hidden menu. The AVR shouldn't be doing any conversion whatsoever, so I can only assume the Reon chipset in the Epson 8500 isn't handling color space conversion well. Then again, it seems very odd that this would be the case, considering how widely lauded the Reon chipset is. Fortunately, my BDP-83 and Xbox 360 both let me use RGB, but my digital cable box doesn't have that option.

    ____________________________________________________________________________________________

    What about AVRs?
    Written by Gotchaa , May 28, 2011

    ChrisHeinonen,

    Most ppl are switching thru AVRs and some of these that claim not to touch the video signal acutally do. Denon as an example allows you to turn on video conversion for analog sources only, which allows you to get the Denon Volume "Gui" overlayed even on digital sources like HDMI. I found that this introduced chroma errors using one of the Spears Munsil test discs. I am hoping you guys will be taking this up in your reviews and making recommendations for those that do not direct connect. We'd like to see this applied to the entire video chain. It would also be good to address the colorspace mode options the player, receiver, and display offer and what changing one option in the chain will do to the signal. I've seen ppl turn on various modes that are not consistent throughout the video chain, need to educate on this as well. Thanks for the good work

    ____________________________________________________________________________________________

    Pana 110/210/310 testing
    Written by Mike , May 28, 2011

    ChrisHeinonen,

    What an eye-opener. I have the BX37 (equivalent to the S370) and have been happy with it.

    Thanks for the review.

    Are you planning on testing any of the Panasonic DMP-BDT110/210/310 models

    ____________________________________________________________________________________________

    AVRs and Color Space
    Written by ChrisHeinonen , May 29, 2011

    Gotchaa,

    If you look at my review for the Marantz AV7005 you will see that we introduced a colorspace test for AVRs and Processors that we test now showing what they can handle. We are trying to get it into as many reviews as possible, but we actually had to pull most of the data from one upcoming review since the author had the Sony S570 player and as this testing discovered, we couldn't use that as a source and be certain of the results! Rest assured we will try to test AVRs and Processors for these same issues as they come through our writers systems.

    I believe we will also be able to do the same colorspace testing with displays and projectors as well. As Jeremy found, and as another one of our writers found, not all displays can handle all colorspaces correctly. My Samsung plasma does, but the one released a year later does not. I'm glad it does as otherwise I might be out buying a new display to be able to do all of this testing! The fact of the matter that as devices stand today, it has me seriously considering buying an external video processor that I know will do this all correctly, and only using my processor to handle audio in the future.

    As far as future models we will test, whatever we can get our hands on we will test. The Sony 770 has been replaced by a new model at this point so that won't be tested, but current model Sony players could show up for review sooner or later.

    ____________________________________________________________________________________________

    Still getting S570
    Written by Bee , May 29, 2011

    numbers are just numbers..Even the expert struggling to tell the difference under normal viewing material. why would i pay extra 300-400 for something that hardly there? peace of mind? not me..

    ____________________________________________________________________________________________

    Further impacts
    Written by Mark Vignola , May 31, 2011

    Bee -

    While I think there are many reasons to spend extra $$ for a player like the Oppo, this data isn't just splitting hairs - these errors can have very real and noticeable impacts on your content...especially if you use a test disc through a player like the Sony 570 to do even basic display calibration like brightness and contrast. Since most people aren't using a pattern generator, test discs through your disc player are by far the most common method used to get patterns to a display. Chris posted the following on AVS Forum highlighting why we need to care about this information:

    As far as noticeable, it didn't just off the screen with the Sony, but the WTW test result is just really bad when we know that the data is bad. It looks like the Sony is properly passing WTW data, but since all the WTW data has been squished below the WTW level, it's not actually passing it correctly. If you tried to set your contrast using this player, you'd run into two issues:

    - You'd have to really, really push the contrast level up to get the WTW boxes on a test disc to disappear, since they're so far below the normal level.

    - All other sources would look completely blown out because of it. Your Blu-ray data would be passable, but all highlights would probably not exist from your HDTV source and others..

    ____________________________________________________________________________________________

    Additional Comments
    Written by Stacey Spears , June 01, 2011

    The QuantumData used was the 882EA.

    The Sony players are compressing the full 1-254 range into 16-235, which is throwing away bitdepth / dynamic range. This also introduces visible banding. On top of the dynamic range compression, they are also applying an S-Curve. it is really bizare.

    The original reason for developing the RGB monotonicity test pattern was because of Pixar. Pixar noticed that their content was off by one level on a particular player. Our RGB monotonicity pattern is the first of its kind. Its not the first time someone has tried to build one, but the first time someone has actually built one that works.

    If anyone has followed the DVDO clipping problem, this pattern help identify the actual cause. It turned out to be the same problem that Pixar pointed out using one of their movies.

    Panasonic's RGB output on the DMP-BDT300 has an issue and should not be used. I have not verified the issue is still present with their latest firmware update, but their engineering team tells me it should be fixed in the generation after the BDT300.

    As far as Epison, my contact is no longer involved with them and they have not responded to my request. It is possible you are seeing what we are showing in our sample image (Chroma Multiburst - Multiple Conversions) in the Choosing a Colo Space article..

    ____________________________________________________________________________________________

    Just my 2 cents
    Written by Bee , June 01, 2011

    Mark

    Completely agree with you. Afterall people who would find/read this kind of website are more than just enthusiast i believe.

    I'm just saying here that people should less worry about their equipment and enjoy the movies more. Sometimes they do forget. With the kind of equipment we're talking about here you cant go wrong by far.

    ____________________________________________________________________________________________

    'bout time
    Written by ws , June 04, 2011

    Looks like I have a reason to start coming to your site more often. I used to come here for the leading-edge measurements of DVD players and Subs. Don't get me wrong, but the reviews of interconnects were totally opposite to that approach. Now it seems you're benchmarking again.

    Too bad it wasn't done 3 years ago. What took you so long?

    Congrats!.

    ____________________________________________________________________________________________

    Double Damm
    Written by AllanG , June 05, 2011

    Just bought the S570 about six weeks ago myself. Well! just will just have to save up for the Oppo-93.

    To the secrets team. Please keep testing for these errors on all new equipment. My faith is still with you. Looking at a Panasonic TCP-ST30 for Xmas. So don't you dare post without testing it. As a matter of fact the VT30 as well.

    Thanks again..

    ____________________________________________________________________________________________

    Anthem D2v test?
    Written by David , June 09, 2011

    I wish you go back and test high-end popular AVP's such as Anthems D2v or 50V with 8 HDMI input ports for color-space conversion accuracy to see how well they perform on the bench..

    ____________________________________________________________________________________________

    Processor Testing
    Written by ChrisHeinonen , June 09, 2011

    Next go-around, I will put my Marantz AV7005 on the bench to see how it does, and if we have anything else around we will try that out as well. I'm hopeful that we won't find any issues aside from the current colorspace conversion issues that we now chart for reviews, but will will test to see..

    ____________________________________________________________________________________________

    Signal tests are great, what about compatibility tests?
    Written by Michael C. , June 16, 2011

    People shoot their home videos on AVCHD and HDV camcorders, then they transfer these videos onto recordable DVDs (making what is known as "AVCHD disc"), on Blu-ray discs or even on any external USB media, and watch them on their BD player. I believe that all BD players have USB inputs, and some players have SD card or Memory Stick slots. So the question of compatibility arises.

    In particular, my Panasonic BD80 plays native 720p30 from an AVCHD disc despite that this frame rate is not legal for BD or AVCHD. It even plays 1080p60 from my camcorder, but when played from DVD it stutters. I know that the Sony S570 can play a bunch of formats including AVCHD clips and non-standard 1080p60 clips from variety of media including DVD, BD and USB-connected devices. And for those who look for playback of various formats, the S570 is a terrific device. It does not even require full AVCHD structure on media, it can read a single file.

    What I am trying to say is that your tests are great, but they are targeted strictly on audience that watches BD discs authored by big studious. This folk is in minority now. I bet that most people would prefer broader compatibility to pixel-perfect picture. So maybe this is what you have to look into for your next batch of tests..

    ____________________________________________________________________________________________

    Pioneer Elite BDP-09FD
    Written by BLUTRIG , June 17, 2011

    Would you like me to send my Pioneer Elite BDP-09FD for testing,it has 2 HDMI output,source direct,YCbCr,4:22,4:4:4,no SACD and I can send Pioneer Elite BDP-23FD.

    ____________________________________________________________________________________________

    Thanks for this
    Written by LW , June 18, 2011

    As an amateur calibrator, who relies on his optical player, you just made up my mind for buying an Oppo to go into my system. No need calibrating if the source is skewing your data.

    One question, and not to get too off topic, would calibrating with and viewing Blu-ray be best recommended at Source Direct, 4:2:2, or 4:4:4. Id assume Source Direct would be most true, but the lack of error in all modes has me wondering.

    Thanks..

    ____________________________________________________________________________________________

    Recommended Output
    Written by Brad , June 18, 2011

    Which output method would then be recommended for a Sony Sx70 series player,4:2:2,4:4:4,or RGB?

    ____________________________________________________________________________________________

    Sony S570 mapping?
    Written by Ron Jones , June 19, 2011

    I does look like the S570 is more-or-less mapping the 0-to-255 range into a 0-to-23x range. I might be interesting to see some detail as to what it is doing with the blacker-than-black range (0-to-15). As the yet untested PS3, it offers a mapping function when "RGB Full" is selected as the HDMI output format. In this case Sony says that the full RGB range of 0-to-255 is remapped into the video range of 16-to-235 (while the "RGB Limited" setting is supposed to be accurately outputting in the 16-to-235 range without any remapping). The YCbCr setting on the PS3 is not supposed to be doing any remapping over the 16-to-235 range and with "super white" option turned on, the WTW range of 236-to-255 is supposed to be correctly output. It will be interesting to learn if this is being done correctly..

    ____________________________________________________________________________________________

    Secrets Benchmark
    Written by Peter , June 25, 2011

    Years ago I purchased a Panasonic S97 based on your excellent review.

    It is still in my HT and seems to do well with calibration pattern DVD's.

    A bit obsolete now that I have an iScan Duo and ChromaPure with autocal and a Chroma C5..

    ____________________________________________________________________________________________

    Choosing a Color Space
    Written by Stacey Spears , June 29, 2011

    One question, and not to get too off topic, would calibrating with and viewing Blu-ray be best recommended at Source Direct, 4:2:2, or 4:4:4.

    >Which output method would then be recommended for a Sony Sx70 series player,4:2:2,4:4:4,or RGB?

    The color space output you set your player for is based on the signal path. You need to test all options and find out which one works best in your situation. Here is an article that helps you decide which is best: http://www.spearsandmunsil.com...space.html

    I can provide two different examples:

    1. Samsung SP-A800B and Samsung SP-A900B. This display requires 4:4:4 input from an OPPO. If you send in 4:2:2, chroma is shifted half a pixel away from luma.

    2. You want to send 4:2:2 into a DVDO, or Lumagen, processor. Both of these convert the incoming inputs into 4:2:2. If you send in 4:4:4, a low pass filter will be applied to the chroma channel. The Duo and Edge are not that bad, but the VP50 and 50Pro really filter the chroma channels in this scenario..

    ____________________________________________________________________________________________

    regarding sony player
    Written by Rao , July 01, 2011

    This article is eye opening. Thank you for this article.

    I have questions regarding the testing of sony player. You haven't mentioned which video option you have chosen. Standard , Bright room, theater etc.

    After reading your test I went and did test on my player and found theater setting is out putting the WTW correctly. Any setting other than Theater are suffering the phenomena this article mentioned about.

    I kindly request you to redo the testing with Theater mode setting for sony players and post your findings..

    ____________________________________________________________________________________________

    regarding PS3 WTW
    Written by Rao , July 05, 2011

    "As the yet untested PS3, it offers a mapping function when "RGB Full" is selected as the HDMI output format. In this case Sony says that the full RGB range of 0-to-255 is remapped into the video range of 16-to-235 (while the "RGB Limited" setting is supposed to be accurately outputting in the 16-to-235 range without any remapping). The YCbCr setting on the PS3 is not supposed to be doing any remapping over the 16-to-235 range and with "super white" option turned on, the WTW range of 236-to-255 is supposed to be correctly output. It will be interesting to learn if this is being done correctly.

    In response to above question :-)

    My observation are as below for PS3.

    RGB(full range) and YCbCr(superwhite) are not extendig WTW all the way to 255 in PS3. It was going to 242 or so.

    If I turn off RGB(full range) and YCbCr(superwhite) WTW is set to 232..

    ____________________________________________________________________________________________

    10 years and many movies later....
    Written by nbmformula , July 30, 2011

    I will now be searching for the BD equivalent to the 2000/2001 Diamonds-in-the-rough Panasonic DVD-RP56 and 82 that were so sought after following the first DVD Benchmarks.

    Thanks for all of your hard work in the last 10+ years..

    ____________________________________________________________________________________________

    Picture Quality Mode on Sony
    Written by Andy M , August 04, 2011

    The point that Rao made on July 1st is very important. The Picture quality mode (options button during playback) adjusts the picture for different lighting conditions, which setting was used for this test?.

    ____________________________________________________________________________________________

    Sony Picture Mode
    Written by ChrisHeinonen , August 04, 2011

    The Sony was reset to factory defaults for the test, so any special picture modes wouldn't have been used, as that's the way it is most likely to be used by consumers. If we have time in the future, I will attempt to test both modes and have results for it. The Sony image adjustments lead to a higher gamma factor, and therefore a darker image, so if they have a Theater mode that's neutral, that would actually produce a brighter image in a darkened theater than the normal mode, which seems to be a strange choice..

    ____________________________________________________________________________________________

    So, the ideal situation should be...
    Written by Victor , August 12, 2011

    Having the Blu-Ray Player and the projector or TV set to RGB, 4:2:0, 4:2:2, 4:4:4 or RGB?

    Regarding the OPPO players used with a Panasonic Plasma VT20 for instance?

    And connected with a PS3, what do you recommend?

    If my projector also has the same options as the OPPO, the best match would be RGB?.

    ____________________________________________________________________________________________

    ...
    Written by Rob , November 21, 2011

    I can provide two different examples:
    1. Samsung SP-A800B and Samsung SP-A900B. This display requires 4:4:4 input from an OPPO. If you send in 4:2:2, chroma is shifted half a pixel away from luma.

    2. You want to send 4:2:2 into a DVDO, or Lumagen, processor. Both of these convert the incoming inputs into 4:2:2. If you send in 4:4:4, a low pass filter will be applied to the chroma channel. The Duo and Edge are not that bad, but the VP50 and 50Pro really filter the chroma channels in this scenario."

    If all displays display in RGB, and you know your Oppo or other player outputs a perfect conversion to RGB, then what happens if you just stick to RGB output?.

    ____________________________________________________________________________________________

    RGB Conversion
    Written by ChrisHeinonen , November 24, 2011

    "If all displays display in RGB, and you know your Oppo or other player outputs a perfect conversion to RGB, then what happens if you just stick to RGB output?"

    While all displays do use RGB at the end of the chain, they all might process the signal differently. Some might take RGB and convert it to 4:2:2 or 4:4:4 first, then apply any color or tint adjustments at that stage, and then reconvert to RGB. That introduces two potential stages for a bad conversion, and feeding it 4:2:2 or 4:4:4 could remove one of those.

    If you have a player that outputs all colorspaces correctly, you can then use test patterns (Spears and Munsil has a few that do this) to determine if your display handles them correctly, and which it handles best. Once you determine what it handles best, then you want to use that if possible from your source components..

    ____________________________________________________________________________________________

    ...
    Written by Chris Eberle , November 25, 2011

    Very few displays will process an RGB signal without conversion to YPbPr. The way to tell is if your Color and Tint controls are grayed out when you input RGB. Those adjustments are only available for YPbPr signals. If you feed RGB and you can still adjust Color and Tint, then the display is converting twice as Chris H. describes above..

    ____________________________________________________________________________________________

    This is soooo cool, thanks!
    Written by Head_Unit , January 26, 2012

    So now we now "bits are bits" except when they aren't! I didn't realize about the conversion from the data format on the disc itself. Maybe to fit enough data, the industry could introduce 12" Laserdisc II :-D

    Great work, appreciate it very much..

    ____________________________________________________________________________________________

    exceptional job
    Written by winston churchill , April 14, 2012

    So refreshing to see a site devoted to facts and getting information out to assist manufacturers and to consumers. Very powerful information. I used to use Avsforum but find it useless these days. Great work. I will look forward to your coming reviews and very well Written and easy to understand articles. This is not the easiest stuff to make understandable and you do an incredibly great job at it.