Reviews & Articles

How We Score 3D Quality

We're sometimes asked how we determine the 3D Review Score used in our PS3 and Xbox 360 3D games lists so we have decided to publish the specific criteria as follows.

We have developed the criterial over the last 18 months through the process of scoring the 3D quality of the available stereoscopic 3D console games, changing and tuning the criteria many times. The criteria are fairly brutal – as you can see from the 3D games lists in the links above only one game is rated above an 8 out of 10 and many games score only 5 or 6. Our review scores are focused on 3D – we don't take into account general video game review criteria such as gameplay, sound, presentation (other than graphics) or value for money. There are plenty of sites that cover those aspects and you should use a combination of sources to determine the games that are right for you.

So what does 3D quality mean exactly? It's not just about the depth or wow factor. Implementing stereoscopic 3D can require up to twice the graphical processing depending on the technique used. Adding 3D to existing game engines or where graphical quality is high and the system pushed close to its limits, often requires certain compromises to be made, such as lowering the resolution, frame rate, or graphical detail.

A high quality 3D game will maintain graphical quality, have good depth, be comfortable to view for extended periods, and perhaps provide additional features specifically for stereoscopic 3D.

If you have any feedback on our rating criteria or methodology then please comment below or contact us directly.

3D Review Score Rating Criteria

Criteria Points Description
Resolution 10 A matrix of the resolution of the rendered graphics and anti-aliasing determines this score, with a perfect 10 being 1920×0180 (FullHD) with good anti-aliasing and as low as 0.5 for sub-SD resolution with no anti-aliasing. No current games have top or bottom scores.
Current average: 7.5, represented by 720p with no AA.
Frame Rate 10 A matrix of rendered refresh rate and screen tearing, this criteria evaluates the throughput rate of the games graphical fidelity at its output resolution. Poor frame rates result in poor viewing comfort. A 60fps game without tearing scores 10 points and sub-24fps with constant tearing a 0. While there are a couple of games at the top end thankfully there are few scoring less than 5 points.
Current average: 8.0, represented by locked 30fps with no tearing.
Graphical Detail 10 It's easy to hit high resolutions and frame rates if there isn't much to display. This criteria evaluates the on-screen visuals such as object and texture detail, lighting and shadows and particle and other effects, and combined with depth, resolution and frame rate, is linked to the level of immersion that can be achieved. High scoring games include Gran Turismo 5, Uncharted 3, Datura and Crysis 2.
Current average: 7.4
Depth 20 An important factor in stereoscopic 3D is the depth displayed in objects both in and out of the screen. Without depth there is no 3D and this criteria contributes more to the final score than other criteria as a result. Pop-in (distant) depth and pop-out (near) depth are rated separately with two-thirds of the weight given to pop-in as pop-out has more wow factor but increases eye-strain and should be used carefully. High scoring games include Puzzle Dimension, Happy Feet 2, de Blob 2: The Underground, Gran Turismo 5 and MotorStorm Apocalypse.
Current average: 12.4, representing 7.5/10 for pop-in and 3.8/10 for pop-out.
Settings 5 We evaluate the 3D settings provided in the game, such as the ability to set the amount of parallax (depth), convergence (pop-in/pop-out point), the ability to fine tune these, and whether or not a live preview is available. All 3D games should provide a live preview, and a convergence setting where appropriate, but sadly many don't and some games do not even provide the ability to adjust the depth.
Current average: 2.7
Extra Features 5 Some games go the extra mile to provide additional 3D features such as 3D game replays, the ability to take in-game 3D photos, 3D split-screen or support for 2DTV's in addition to 3DTV's by using stereoscopic anaglyph output (colored glasses). Unfortunately very few games provide additional 3D features.
Current average: 0.5
Projection -5 Points are deducted for poor 3D viewpoint implementation such as depth artefacts (mostly where the depth buffer is used to create the frame for the alternate eye), incorrect perspective and window violations (objects at the edge of the viewing prisms that appear in one eye but not the other). These all lead to eye-strain and reduce comfortable viewing time. Projection is evaluated on a scale of 1-10 and up to 5 points deducted from the total score.
Current average: -0.8
Crosstalk -5

Crosstalk, or ghosting as it is sometimes called, is evident on virtually all 3D displays but thanks to improvements in screens and glasses it is decreasing with successive generations of hardware. It refers to the image for one eye being partially visible to the opposite eye, mostly with high-contrast scenes such as bright objects on a dark background and is largely a function of poor light filtering in the 3D glasses and image retention in the display.

Crosstalk can be minimised through careful choice of object and background colors, deliberate blurring of forground and background objects (which is also useful to keep the players focal point near the screen plane) and limiting depth (and providing settings to allow the player to adjust the parallax and convergence) though some game engines are particularly susceptible. Excessive crosstalk leads to eye-strain and a poor viewing experience.

Crosstalk is evaluated on a scale of 1-10 and up to 5 points deducted from the total score based on the level of crosstalk found. The best crosstalk free experience currently available is with the Sony HMZ-T1 personal 3D viewer where separate screens provide the images for each eye. Once crosstalk is far less of an issue on the majority of 3D displays this criteria will be removed from the ratings.
Current average: -1.3

3D Quality Marks

When a game is evaluated against the criteria above up to 60 points are awarded. The final 3D Review Score is then calculated on a scale of 1 to 10 in increments of 0.5. 3D quality marks are awarded for high quality games as follows:

8.5 or greater 3D Tested Rating: Excellent
7.5 or greater 3D Tested Rating: Very Good
7.0 or greater 3D Tested Rating: Good

Achieving Top Scores

Using our criteria top scores can be achieved as follows:

  • A 720p/60fps game could achieve a score of 10 with perfect scores in all other criteria
  • A 1080p/24fps game could achieve a score of 9.5 with perfect scores in all other criteria (1080p 3D is limited to 24fps with most current displays and the implementation of HDMI 1.4 3D on consoles)
  • A half-resolution 1080p/60fps game (960×1080 side-by-side) with good anti-aliasing could achieve a score of 10 with a near perfect score in all other criteria (up to half a point could be dropped).

Clearly, we're not likely to see the perfect 3D game any time soon.

Add a Comment

*