Renowned camera tester Alan Roberts has released his EBU assessment of the Panasonic GH5s. The EBU – Tech 3335 assessment has long been a method of measuring the imaging performance of television cameras.
So what do the EBU tests measure and why are they important?
Camera Measurements
To help maintain acceptable levels of quality in European (U)HDTV programme-making, the EBU has specified a series of measurements that will reveal a camera’s ability to produce high-quality television pictures.
The measurements and advice on how they may best be performed are published in EBU Tech 3335. This document augments previous EBU publications concerning camera quality measurements.
Knowing the performance of your (U)HD cameras is one thing, but understanding how this performance relates to a camera’s suitability for a given programme genre is quite another. This is where tiering comes in.
Camera Tiering
EBU Recommendation EBU R 118 uses the published technical specifications and measured performances (according to EBU Tech 3335) of cameras to categorize them into a set of levels, or tiers.
Once a camera has been allocated to a tier, its suitability for a production may quickly be determined, which will simplify production budgeting and facilities procurement.
Technically, the criteria used to identify which tier a camera falls into concern its:
– Resolution
– Sensitivity
– Noise
– Aliasing
– Exposure Range
– Codec used (see EBU R 132)
Panasonic GH5s Results
According to Alan, the camera performed rather well under test. Tests were made according to EBU Tech.3335, and the results establish that the camera belongs in HD Tier 1 and could arguably belong in UHD Tier 2.
All measurements were made on frames captured in the camera onto SDHC card using 10-bit 4:2:2. Live viewing was done on a 50” HD/UHD LCD television display (Panasonic 50DX750). Clips were ingested into Edius 8.52 and images for this document were extracted as BMP files. In all cases, the project resolution was set to match the clip resolution thus avoiding any unnecessary scaling. The lens used for the tests was the Lumix 12~35mm (F/2.8) zoom.
Colour performance
To check the colour performance of the GH5s a standard ColorChecker chart was exposed, using tungsten illumination. The GH5s has many ‘modes’ which affect colour rendition, some of these were also tested. Alan said, “It would seem to be wrong to compare these without including the post-processing and display system for which each may be intended, but there is no reason to expect the resulting colour performance to be affected by these settings, provided the correct post-production operations are done on them.”
Above you can see the results for the seven picture profile modes. Like709 is the most accurate reproduction (Knee was turned off for this test), but the orange patches are a little on the blue side. There is little or no difference when switching between ISO400 and ISO2500. Also, there is little to choose between Like709, Natural and Standard, and the two Cine modes both show significant desaturation as expected. HLG looks right when viewed on a display with HLG decoding, and V-LogL looked right but dim using standard display gamma.
According to the report, there was no evidence of any Infra Red response; there must be an IR filter.
Above you can see the curves for four of the modes with the data plotted with logarithmic axes. Alan noted that all the curves are very similar except V-LogL, which he found surprising. In most cases, the dynamic range appears to be about 10,000:1, or about 13 stops. However, the lines are still curving gently at the lower end, so the dynamic range may actually be greater (or less), subject to noise levels.
Resolution and aliasing
The GH5s has HD (1920×1080), UHD (3840×2160) and 4K (4096×2160) recording modes. 4K is normally known as DCI, Digital Cinema Image format. Only HD and UHD were recorded.
Alan did these tests using the usual zone-plate method fully framed for HD, framed to fill exactly half the image height for UHD.
Above you can see that the image is quite clean, with very little aliasing and resolution well-handled up to near the limits of HD. According to Alan, there is very little evidence of aliasing beyond the limits (in the smaller quadrant which explores the limits of UHD) – this shows that the down-scaling to HD is done very well.
He found that the sharpness control has a nicely limited range. Presumably, the minimum setting (-5) implies no enhancement, so Fig.4a shows the native performance of the camera. Even with the maximum setting (Fig.4c, +5) performance is not bad, and the default value is a very good compromise. It is rare to see downscaling done so well in a camera.
So how does the GH5s perform when it comes to resolution and aliasing when in UHD? To check this, the HD zone plate chart was reframed so that it filled exactly half the frame height.
The above image shows one luma quadrant when the camera was tested with the Lumix 12~35mm (F/2.8) zoom lens.
Alan found that there was significant aliasing both horizontally and vertically because the image appears to be taken from an area of the sensor which matches the UHD resolution. Thus the red and blue channel resolutions are only HD (1920×1080), whilst the green channel resolution is confused by the camera’s improved manner of decoding luma information from a larger-than-normal number of neighbouring photo- sites. There appears to be an optical low-pass filter, preventing resolution beyond the limits of UHD from reaching the sensor – the evidence is in the smaller quadrant which now reaches the limits of UHD2 (7860×4320) where there is very little content outside UHD (3840×2160).
Alan found these results highly encouraging, and although the sharpness control appears to be working a little too hard (the default value produces excessive aliasing without improving the resolution), the images look surprisingly clean for a camera using a Bayer patterned sensor with the same photo-site count as the image format.
Overall, Alan said that the UHD performance is reasonable for a Bayer-patterned sensor, but not really good enough for high-end television production.
Dynamic range
So how much dynamic range does the GH5s actually have?
According to Alan’s testing, the GH5s dynamic range is about 14.6 stops in HLG, which is said was a creditable performance. CineLike-D is about 13.9 stops and Like709 about 10.3 stops with the Knee switched off. However, all of this is using the 4:2:2 10-bit recording modes. Using any of the 8-bit recording modes will automatically reduce the dynamic range by at least a stop. However, the dynamic range of Like709 can be extended by using the Knee function. Previous tests, not published, showed that the knee can capture about 2.5 times normal exposure (+1.3 stops), so the dynamic range for Like709 can be extended to about 11.6 stops – even with 8-bit recording.
Alan, however, notes that these dynamic range figures are a little speculative and subjective. He stresses that it is up to the user to decide whether the performance at the extreme ends of the dynamic range are acceptable, thus there is an uncertainty of about 1 stops in all these results. As mentioned earlier it’s important to note that if the camera is used in 8-bit recording, the dynamic ranges will all be lower because the quantum changes are bigger, so at least one stop will be lost in the blacks. This is an important piece of information to remember if you are looking to get the most amount of dynamic range out of your GH5s. As Alan clearly mentions in the report, these dynamic range figures are a little speculative and subjective. There is a big difference between actual dynamic range and useable dynamic range.
It is also important for users to know that they can obtain more dynamic range shooting in the HLG or CineLike-D picture profiles than V-LogL, but in saying that, V-LogL and HLG have a lot more dynamic range in the stops allocated above middle grey than CineLike-D. The amount of stops isn’t as important as where they are located above or below middle grey. Just because Cinelike-D has more DR than V-LogL doesn’t mean it’s a better picture profile to shoot in.
Noise levels
The exposures of the ColorChecker were used to measure noise levels in the grey scale. All measurements were made in HD mode (1920×1080). Noise levels in UHD and 4K modes ought to be similar.
According to the report, Photon noise (properly known as Shot Noise) must increase with signal level, but this can be countered by analogue gamma correction in which noise levels rise towards black. Alan found that all the picture profiles except CineLike-D appear to have noise levels rising near black, which seems to imply that the gamma-correction is being done in analogue before the colour matrix. Whilst unconventional, this can result in pictures which look less noisy than they actually are because noise levels near mid-grey can be lower. The down-side of such processing is that colour performance may not be constant with changes in signal level.
Alan found that the result for V-LogL shows a much more pronounced rise towards black. At about 50% signal level, the noise level is about -48dB for Like709 mode and -52dB for HLG and CineD modes.
Noise reduction
So what happens if you start using the cameras inbuilt noise reduction? Above you can see how the resolution changes with noise reduction. Alan found it somewhat surprising that far from reducing the effective resolution, noise reduction appeared to improve it. According to Alan, there appears to be little reason to avoid it, and the higher settings may well be acceptable. This is encouraging, if rather surprising.
Rolling Shutter
The GH5s uses a CMOS sensor, which can, in theory, be read either by scanning or by instantaneous transfer into a readout store. Scanning produces the so-called ‘rolling shutter’ effect. To test for rolling shutter Alan did a simple using a small desk fan.
The above image shows a still frame, in HD with the shutter set to 1/1000. The fan blades filled about 75% of the image height when they were above and below the spindle, a fairly critical condition. Although the right- hand (falling) blade is widened and the left-hand (rising) blade is narrowed, the effect is not serious. Evidently, the sensor is not read out at video speed but much faster. Alan found that the result was acceptable for this class of camera.
Conclusion
Here are Alan’s conclusions from testing the GH5s:
The sensor has about 11 megapixels according to the specification (11.92 if you look on the Panasonic site) and is a Micro Four Thirds format (sensor size 17.3x13mm which means 4:3 aspect ratio). The camera does have multi-aspect sensor and can be used in 4:3, 17:9, 16:9, and 3:2 aspect ratios.
The specification says that only “10.28 Megapixels are active”, making the resolution about 3704×2778 if it’s 4:3. So the individual photo-sites must be about 4.7μm square, about the same as a conventional 3-sensor HDTV camera. Alan found that neither of these sets of dimensions was enough for 4096×2160 recording unless his assumptions are over-simplifications – which seems likely from the measurements above. To be clear Alan is only commenting here on the GH5s 4:3 mode and not in regards to the 16:9 (4016×2256) or 17:9 (DCI) (4096×2160) modes. Alan mentioned at the top of his report that he didn’t test the camera in the 17:9 4K DCI mode.
According to Alan “There’s some deep misunderstanding here. The ‘multi-aspect-ratio’ claim does not come from elastic photo-sties – it can only be from the use of several different shapes within the singly-shaped sensor. Given the dimensions of the sensor, true DCI is impossible. Ask Panasonic for an explanation, not me.”
Colour performance is good and there was no evidence of any Infra-Red response.
The sensor is evidently scanned, the so-called ‘rolling shutter’ effect. Rotating motion is not ideal but is quite acceptable.
Resolution at HD is very good, with very little spatial aliasing. At UHD (and therefore at 4K as well) resolution is good but not as good as at HD, and there is inevitable coloured aliasing horizontally and vertically, and diagonal luminance aliasing, but neither is at a high level. This confirms that there is a good optical low-pass filter in the camera and that the down-scaling to 1920×1080 HD is done with a good algorithm.
The dynamic range is about 14.6 stops in HLG, a creditable performance. CineLike-D is about 13.9 stops and Like709 about 10.3 stops with the Knee switched off. However, all of this is using the 4:2:2 10-bit recording modes. Using any of the 8-bit recording modes will automatically reduce the dynamic range by at least a stop. However, the dynamic range of Like709 can be extended by using the Knee function. Previous tests, not published, showed that the knee can capture about 2.5 times normal exposure (+1.3 stops), so the dynamic range for Like709 can be extended to about 11.6 stops – even with 8-bit recording.
Noise levels are acceptable. The EBU R.118 noise-level target at mid-grey for HD Tier 2 is -44dB, which the camera meets comfortably in all modes. The target for Tier 1 HD is -48dB which is also met using all modes. Best performance appears to be at ISO 2,500, with noise deteriorating at both higher and lower speeds. For UHD, the noise target of -50dB can be met at some ISO settings.
The camera achieves HD Tier 1 because of the sensor size. The camera cannot achieve Tier 1 or 2 for UHD broadcast or cinema because the sensor resolution is inadequate. However, the performance at UHD is probably adequate for use in productions provided it is not the prime camera.
Alan’s tests clearly show that the GH5s is a very capable camera, despite it not being able to meet EBU Tier 1 or 2 UHD broadcast or cinema standards. It’s important to note that these tests and subsequent results are only really relevant if you need meet EBU standards. Tests and standards are important, but you don’t have to take them as gospel. We don’t shoot charts for a living and ultimately if you don’t have to meet certain delivery standards (not everyone does) then you have little to worry about.
As the GH5s can certainly record in 16:9 (4016×2256) or 17:9 (DCI) (4096×2160) modes you may be wondering how it didn’t get a Tier 1 or Tier 2 UHD rating. I too was a little confused by this. It seems that there are many factors that determine these Tier ratings. It’s not just about being able to record a certain resolution. Sensitivity, noise, aliasing, exposure range and codec also have to be looked at. I’ll attempt to contact the EBU to find out what the primary reason was behind the camera not receiving a Tier 1 or 2 UHD rating.
For anyone wanting answers to information they feel is wrong, Alan Roberts has given me answers that I have added to the comments section.
To be crystal clear with our readers, these aren’t our tests nor did we have anything to do with them. These results have already been published by the EBU and are public record. Feel free to agree or disagree with the findings. Erik reviewed the GH5s earlier this year and you read that review here.
Do you own a GH5s? Do any of these results surprise you? Let us know in the comments section below.
