How the lab assessed the 40 action cams in our test.
All testing was done with the cameras set to their highest resolution; best encoding mode (using the highest possible constant bit rate); with automatic shutter speed; automatic aperture or exposure; automatic white-balance setting; automatic focus; image stabilisation enabled; and, any other automatic detection mode enabled (potentially with object and face detection). The recordings are made using 16:9 aspect ratio, when possible.
We record a scene featuring colourful stuffed animals, realistic artificial plants, and mannequins wearing colourful clothes with fine details. The camera is mounted on a tripod. The scene is then recorded in 3 different simulated lighting conditions: simulated daylight, artificial light, and low lighting. Each lighting situation is filmed three times: with everything still; with mannequins moving and fans blowing; and, with the camera being continuously panned from the tripod.
The recordings are judged by a viewing panel. The panellists compare the recordings to the actual scene.
Note that this is the same test and rating scale used for our DSLR tests. As such, the image quality scores are quite harsh for action cams. We have kept the numerical scores the same to enable comparison to DSLR performance, but altered our scale for “good”, “poor” etc, to more closely reflect how the action cameras score against each other.
For each camera 6 recordings are made. Each camera is hand-held and 10-second clips are recorded: while standing motionless; while standing and panning; and, while walking. Each version is recorded twice, once with image stabilisation on and once with image stabilisation off.
The panellists judge the recordings on how well the image stabilisation worked, and also what the effect, if any, was on the image quality.
Our durability test has two components: the drop test and water resistance.
We place each camera into a “tumbling barrel”, which simulates a drop of 50cm on to a “normal” floor surface. The camera is fully powered and set to standby mode before testing starts. The camera is tested for loose parts, visible damage and correct performance after 20, 40 and 50 drops.
Note that the unit used for this test in not used in the image tests for obvious reasons.
If the camera has a water-resistant or waterproof claim then we test it. We immerse the camera to a depth of 0.91m (3 feet) for 30 minutes. If the camera comes with a protective case, we will use that as per the manufacturer’s instructions. After 30 minutes we remove the camera and check for water intrusion and if the camera is still operable.
Ease of use, portability and versatility
Ease of use scores are the average of three users evaluating the physical usage of the cameras.
The portability score is calculated from a formula determined from the camcorder’s weight, overall dimensions (“bulk”), and presence of a secure belt-type hand strap.
Versatility is based on the number of functions, both physical and digital, offered by the camera.