Image quality

All evaluations of picture quality are performed after the photos are transferred to the PC. Unless otherwise stated the cameras are set to full automatic function – including exposure, white balance and focus – with maximum resolution and image quality setting in JPEG . Test charts are adjusted to fill the full monitor display of the camera. If not stated otherwise a tripod is used.

Resolution

Resolution is determined by the evaluation of a black and white test pattern. This is a visual appraisal of the pattern in the middle and at the corners of the test chart. Additionally, there is a score for aberration and colour problems at the edges of the image, with comments where appropriate.

The pictures are taken in maximum and minimum optical zoom setting at ISO200 (high-end cameras take pictures at ISO1600 and ISO3200 and in both JPEG and RAW formats where possible). The test chart is evenly illuminated by 2 light sources with D65 light characteristic (daylight), illumination level is at 2000 cd/m².

Colour reproduction

Colour reproduction is tested using an IT8 colour test chart. The lighting is similar to resolution (above) and the camera is set to ISO200 and Auto or its equivalent. For high-end cameras the test is repeated at ISO1600.

There's a subjective score of the colour balance for the IT8 test chart for luminance, contrast, colour saturation and definition.

The pure white field of the test chart is evaluated for colour balance of the primary colours. This is done using a software calculation (histogram) of the test chart differences in brightness for the 3 primary colours in the white field (ideally they should be equal).

Low light performance

Picture brightness in very low light conditions is measured by the electronic evaluation of pictures taken of a plain screen with a dimmable white light source. The pictures are taken at a very low luminance setting of 1 cd/m². The sensitivity of the camera is evaluated by measuring the average brightness and the standard deviation of the evaluated pixels in the central part of the digital image.

Resolution in low light conditions is tested at the same light level (1 cd/m²) using the resolution test chart. This test chart is photographed at the maximum wide position of the camera's lens and scores are given for sharpness and image noise.

Low light performance is evaluated with a visual inspection of an image of a test chart taken in a low light environment. The test chart is placed on a turntable which rotates at 45 seconds per rotation. Photos are taken at an illumination level of 11 lux and evaluated for loss of sharpness and image noise.

Image noise (high-end models)
This is a measurement of the luminance and colour noise (red, green, blue) in the central grey field of the test chart at ISO 1600 and ISO3200 (JPEG and raw format). The camera is set to its minimum focal length and white balance is done manually.

Distortion

Measures vertical and horizontal grid lines on a target at 1/3 and at 2/3 image height and width in both JPEG and RAW format at ISO 200. The lens is set to minimum and maximum focal length.

Saturated colour reproduction (high-end cameras only)

We generate a continuous, optical spectrum from a white light source on a screen and add a bright invisible infrared light source. We take pictures at the highest JPEG quality and in sRGB colour space. These pictures are displayed on a profiled monitor and evaluated for their respective differences to the original spectrum.

This is done at ISO 200 and ISO 1600 and always with manual white balance (measured at the projection screen without dispersive element and infrared source) for all high-end cameras. Scores are given for the colour characteristics at low and high frequencies (infrared to ultraviolet) and the colour continuity (steps or holes in the spectrum).

Veiling glare (high-end cameras only)

This measures the amount of stray light in a lens caused by reflections of light between the surface of the lens element and the inside of the barrel of the lens. A lens that performs poorly will often show up as producing images with a washed-out look over most of the photo. This is often less noticeable than other types of glare which may leave a distinctive light spot on an image. Veiling glare can result in a reduction in the contrast of the image as well as the colour saturation.

Measured according to ISO 9358:1994 at maximum and minimum aperture and at normal focal length with ISO 200 setting and using manual white balance.

Reflections (high-end cameras only)

An image of a dark screen with a bright light source at 10° right or left of the maximum horizontal image angle is taken at maximum aperture (minimum F-number) and minimum and maximum focal length . We do not use a lens hood, even if one is provided.

Practical pictures / viewing tests

Practical pictures are performed using 5 images:

• Images 1 and 2 are portraits under medium light conditions (D65 daylight, about 140 lux) in a room with a dark background, both with and without the use of the integrated flash. The tests are performed with the camera set to auto or its equivalent. The subject is 2 metres away and the focal length is adjusted to fill the screen with the subject. Scores are given for skin tone quality, colour quality in general, exposure level, sharpness and image noise.
• The 3rd test picture is an outdoor landscape picture with sky, a building and a natural background of lawn and trees, taken in bright sunshine. The camera is focused at infinity. Scores are given for detail in highlights and shadows, colour quality, exposure level, sharpness and image noise.
• The 4th test picture is an indoor image of different items such as skin, clothes, toys with artificial colours, high contrasts and a heat source at 450°C (with almost invisible infra red emission). Scores are given for detail in highlights and shadows, colour quality, exposure level, sharpness and image noise. The subject is 2 metres away and the focal length is adjusted to fill the screen with the subject.
• The 5th test picture is for the evaluation of the face detection function. An image is taken of 2 people left and right of the image centre, in front of a bright white screen with some high contrast objects. The image is taken as close as possible to normal viewing angle (47°). The people are at a distance of 1 and 3 meters and the rear screen is at a distance of 5 metres. Scores are given for sharpness and the exposure of the faces, with preference given to the person closest to the camera.