Posted Tue Sep 25, 2018 at 09:00 AM PDT by Steven Cohen
Yes, it's that time of year again. The time to get holed up in a dark room for hours on end along with the industry's best TVs as they're pitted against each other in the ultimate 4K HDR cage match. And we got a front row seat.
This past weekend, High-Def Digest was invited to participate on the panel of judges for the 2018 Value Electronics TV Shootout held in Scarsdale, New York. The annual event placed the year's top displays side-by-side in order to choose the new "King of TV."
The lineup of competing displays in this year's competition included four flagship premium 4K HDR TVs:
In order to evaluate the displays, the Shootout mostly relied on real-world content and a couple of test patterns specially selected by the organizers to highlight various aspects of picture quality. Specific attributes were then scored by a panel of eight judges from 1 to 10 across three categories to determine the individual winners for each section. Likewise, the display with the most overall total points was then named the "King of TV." All of the TVs were visually matched to and evaluated against a Sony BMV-X300 30-inch OLED Reference Monitor used for professional color grading applications.
And now that the dust has settled, here's a rundown of the winners in each category based on all the judges' scores:
Below, I'll offer full details on the tests conducted in each category, along with my own impressions on how each TV fared during the competition and my general feelings on the winners. Let the shootout begin!
(Note: Pictures included are from various demos throughout the event and do not necessarily correlate with the specific section heading)
This category was designed to determine which display had the best SDR daytime viewing and processing for a living room setting with a high amount of ambient light. The judging criteria included scores for Motion Handling, Off-Axis Performance, Upscaling, and Peak Brightness. Three real-world demos were then used to evaluate all of these attributes. All three were played from a DirecTV HD feed and included clips from a football game, a hockey game, and a local news broadcast. All of the TVs were set to their brightest calibrated SDR mode with motion interpolation activated. Likewise, the blackout curtains were pulled back and the lights were turned on.
Motion Handling - Though sports are typically a good showcase for motion, I didn't really feel like any of these clips were substantial enough to offer a particularly good example of each TV's motion handling. In fact, a specific test clip of a woman swinging on a hammock demoed after the judging was completed, provided a much better look at each panel's approach to motion. And even then, each TV offers a lot of leeway to fine-tune its motion handling, meaning that more fiddling with the settings would be necessary to see each at their best. With that in mind, based on the earlier clips this was mostly a toss-up between the four displays for me.
Off-Axis Performance - As expected, the two OLED models fared much better than the two LCDs in this regard, with the LG E8 and Sony A9F OLEDs demonstrating relatively minor shifts in color at extreme angles. Meanwhile, the Sony Z9F LCD had a more pronounced shift in color and contrast, and the Samsung Q9FN had the most trouble maintaining accuracy from off-angle viewing.
Upscaling - Due to the heavily compressed nature of the testing material, this section was also a bit difficult to judge precisely. In general, I didn't really think any of the TVs did a particularly good job with the low-grade content, and any differences were quite small. With that said, I did notice slightly smoother edges with less visible pixelation around text on the LG E8 OLED compared to the other TVs.
Peak Brightness - Here's where the LCDs got a chance to really shine... literally. Though the two OLEDs had solid brightness in the well-lit room (and were essentially equal), the LCDs demonstrated a notable increase in light output -- with the Samsung Q9 edging out the Sony Z9F as the clear leader in this spot.
TV Shootout Winner - Sony XBR-65A9F OLED (based on all the judges' scores)
My Winner - LG OLED65E8PUA OLED (based on my scores)
Final Thoughts - Based on the "Day Mode" designation of this category, I'm a bit torn about the results here (both the overall and my own). Off-Axis Viewing is what really ended up skewing the scores toward the OLEDs instead of the brighter LCDs. Of course, being able to have a good view from any seat on the couch is an important factor for a "Living Room TV" -- I'm just not sure it trumps higher luminance for bright room viewing.
For this category, the focus was on which display had the best SDR dark room performance for a home theater. The judging criteria included scores for Black Level/Perceived Contrast, Shadow Detail, Color Accuracy/Skin Tones, and Near Black Uniformity. A mixture of real-world clips and test patterns were used for this section, and the scores were determined based on how well a display matched the same content on the professional BMV-X300 Reference Monitor. The blackout curtains were let back down and all of the lights were turned off in the store. Likewise, all of the TVs were calibrated to the SDR standard.
Black Level/Perceived Contrast - A clip from the Bu-ray edition of Gravity was used for this section, with the disc being paused on an image of an astronaut against a large field of stars. The LG E8 OLED demonstrated the deepest blacks (they essentially disappeared in the darkened room) while still maintaining contrast in the bright whites of the astronaut's suit and the stars. Meanwhile, the blacks on the Sony A9F OLED were just a hair lighter while offering similar contrast. Thanks to its local dimming, the Sony Z9F LCD maintained solid blacks and contrast as well, but the astronaut appeared flatter and dimmer against the backdrop of space, and the blacks were not quite as deep as the OLEDs making the letterbox bars more visible. Finally, the Samsung Q9 LCD and its aggressive local dimming actually produced black levels that were just as deep and inky as the LG OLED, making the letterbox bars completely vanish while offering nice contrast between the astronaut and space. As a side effect, however, the algorithm made the starfield in the back look comparatively patchy, revealing the dimming zones while causing many stars that were visible on the other models to disappear.
Shadow Detail - Here's where things might get a little controversial. For this section, the competition used a clip from the Blu-ray edition of Harry Potter and the Deathly Hallows – Part 2 where Voldemort and his followers invade Hogwarts at night. The sequence is very dimly lit and serves as a torture test for shadow delineation. The LG E8 OLED performed admirably, maintaining solid detail in the darkest portions of the picture, though I noticed that the top left corner of the screen actually tended to crush more than the Sony A9F OLED. Meanwhile, I thought that the Sony Z9F LCD actually did the best job of preserving small details in the dark portions of the screen, making it the closest to the Reference Monitor.
Unfortunately, the Samsung Q9 LCD was an entirely different story. This sequence essentially broke the TV's local dimming algorithm, causing the contrast to fluctuate and vignette to the point that the panel was almost completely off during some of the scene, basically making it unwatchable. With that said, I reviewed this TV myself and I didn't encounter anything even close to this on anything I watched during my nearly three months with the display (though I never demoed this particular movie). It's clear that this was a weird anomaly rather than a good representation of how the Q9 typically performs. Of course, it's an anomaly that Samsung absolutely has to address and hopefully fix with a firmware update, but I still think that a different scene should have been used. In fact, the Gravity scene used for the black level section was a much more realistic depiction of the Q9's flaws in this regard.
Color Accuracy/Skin Tones - A shot from the Blu-ray of Kingsman was paused on the screens for this section. The image featured the cast lined up side by side, offering a fairly neutral assortment of flesh-tones with a few more saturated hues on some of the characters' clothing. In general, I found that both OLEDs exhibited a cooler push while the LCDs had a warmer tinge. With that said, the OLEDs matched the color characteristics of the Reference Monitor more closely. The differences between the LG E8 and Sony A9F were very small, but I ultimately thought that the slight teal bias of the LG actually mirrored the Reference Monitor better.
Near Black Uniformity - Banding in darker tones is a known issue for many OLED panels, and this test made that especially clear. A near black screen was displayed on all four panels and both of the OLEDs featured faint banding lines throughout the entire screen. In contrast, the Sony Z9F LCD and Samsung Q9 LCD were both much smoother and consistent, with the Z9F offering the most impressive uniformity of the bunch.
TV Shootout Winner - Sony XBR-65A9F OLED
My Winner - LG OLED65E8PUA OLED and Sony XBR-65Z9F LCD (tie)
Final Thoughts - I can't really argue with the Sony OLED winning since it was essentially neck and neck with the LG and they both matched the Reference Monitor very closely. With that said, I actually think the Sony Z9F LCD did a better job with shadow detail and uniformity. I was also quite surprised by the amount of banding visible on the near black pattern on the OLEDs, though it's important to note that I never saw anything like that during the real world clips. And though I still think the Harry Potter scene was a clear local dimming anomaly rather than a realistic depiction of the Samsung Q9's shadow detail, the test did reveal a very serious problem for that display -- essentially disqualifying it in this category and basically pushing it out of the running.
This category focused on which display had the best 4K HDR10 dark room performance for a home theater. The judging criteria included scores for Color Accuracy/Skin Tones, 4,000 Nit Tone Mapping, Wide Color Gamut, High Brightness HDR, and Perceived Sharpness. Several real-world clips from 4K Ultra HD Blu-ray discs were used for this section, and the scores were determined based on how well a display matched the same content on the professional Reference Monitor (except for one section). Again, the room was completely dark and the sets were calibrated for HDR WCG playback.
Color Accuracy/Skin Tones - The same clip from Kingsman used for the SDR test was used for this section, but this time it was from the 4K Ultra HD Blu-ray edition. Even in HDR WCG, the results were essentially the same, with the OLEDs matching the Reference Monitor better. Again, the differences between the LG E8 and Sony A9F were very small, but I ultimately thought the LG mirrored the Reference Monitor ever so slightly better.
4,000 Nit Tone Mapping - For this section, a shot from Pan was paused on all four TVs with an image of the sky and the sun partially covered by some clouds. The sequence was graded to 4,000 nits and the highlights actually clipped on the Reference Monitor. With that in mind, we were supposed to judge how well the displays maintained detail and luminance in the sun. I actually thought the LG E8 OLED did the best job of offering specular luminance while maintaining the best detail, allowing one to make out the complete shape of the sun and the nuances in the surrounding clouds. The Samsung Q9 LCD also did a very good job in this regard. In contrast, I thought a little more detail in the clouds and white of the sun were clipped away on the Sony A9F OLED, and a lot of the detail was clipped away on the Sony Z9F LCD, causing the sun to look more overexposed while making some of the clouds disappear.
Wide Color Gamut - An extremely colorful scene from The Lego Batman Movie was used to highlight wide color gamut support. Though all of the displays looked vibrant, I found that the OLEDs were not quite as bold as the two LCD models which could better match the extreme reds, blues, and greens of the Reference Monitor.
High Brightness HDR - For this section, we paused on a shot from The Lego Ninjago Movie where the characters sit on a beach with a volcano in the background. Again, while all of the displays looked great, the OLEDs' HDR performance was a little dimmer than the Reference Monitor. Meanwhile, though the Samsung Q9 LCD actually had the punchiest HDR output, it actually looked a little too bright compared to the Reference Monitor. Ultimately, the Sony Z9F LCD best matched the BMV-X300 best.
Perceived Sharpness - A shot from the 4K Ultra HD Bu-ray of The Revenant was used here, focusing on the details in a close-up of one character's face. To be honest, I found this to be one of the closest tests, and I didn't really see any truly notable differences in detail between the panels. With that said, the Sony Z9F LCD looked just a hair less sharp to me than the other displays.
TV Shootout Winner - Sony XBR-65A9F OLED and LG OLED65E8PUA OLED (tie)
My Winner - LG OLED65E8PUA OLED
Final Thoughts - While the LCDs did offer higher HDR brightness, the OLEDs matched the overall look of the Reference Monitor more closely. To be honest, though, I'm not sure I'd really say the OLEDs are "better" at HDR performance. It's really more about what picture quality attributes a user prioritizes.
The 2018 Value Electronics TV Shootout was an eye-opening and sometimes surprising competition. Getting to see these flagship sets side-by-side is the only real way to judge their individual pros and cons. And though I had a couple of issues with some of the choices made, in general, I thought that the evaluation process was executed fairly.
Here's a ranking of the final total points for each TV based on all of the judges' scores:
Though I can't argue with the results, the Shootout did further reinforce to me just how hard it is becoming to really crown a definitive "King of TV." As you can see by the totals, the competition was really close. In fact, scoring the differences between the competitors was often like splitting hairs, and each model revealed its own specific pros and cons that will vary in importance for different users. Based on the criteria provided here, however, the Sony XBR-65A9F OLED was definitely one of the closest to matching the BMV-X300 Reference Monitor for the majority of the tests.
We'd like to extend a big thanks to Robert Zohn (President of Value Electronics) and the rest of the Shootout's dedicated team for hosting the event, inviting us to participate on the judges' panel, and for setting up all of the tests!
Finally, for those interested in more details about the competition's specific process and equipment used, here's a complete rundown of the testing and calibration methodology used for the Shootout:
See what people are saying about this story in our forums area, or check out other recent discussions.
The latest news on all things 4K Ultra HD, blu-ray and Gear.
