Panasonic S1H Lab Test – Dynamic Range and Rolling Shutter Results

October 28th, 2019
Panasonic S1H Lab Test - Dynamic Range and Rolling Shutter Results

Finally, we were able to put the Panasonic S1H to the test in our cinema5D lab. Looking at the various frame rates and resolutions, as well as the full frame and Super 35 modes of this camera, this turned out to be quite an endeavour. Ready for a lab marathon? Read on…

Panasonic S1H – Image credit: cinema5D

I have long waited for the day to come when I would be able to put the Panasonic S1H through our lab test procedures. I sold my Sony a7S II a while ago expecting a successor to be announced soon, now finding myself impatient in the waiting position for my next camera purchase … will this cam be it?

Spoiler alert: lab results are very impressive – the Panasonic S1H is the first camera in our tests, that shows nearly identical dynamic range results for the two ISO settings of the dual gain sensor! I will share most of the results in detail with you, so this is going to be a long read. Sorry about that…

As described by my colleagues Nino, Jeff, and Johnnie in detail here, the Panasonic S1H is designed with a “video first” mindset by Panasonic and ticks a lot of my personal boxes. I was even hoping for an internal ProRes recording option, but dream on my friends (just to be clear, Panasonic and Atomos are working together in delivering 5.9K ProRes RAW data output to Atomos Ninja V recorder. More info can be found here).

Nevertheless, the specs are impressive as is the price – so will it live up to the high expectations?

To help you read through this post, it is organized as follows:

  • ISO640 V-Log dynamic range results for 6K, 5.9K, and 4K DCI full frame
  • ISO4000 V-Log dynamic range results for 6K, 5.9K, and 4K DCI full frame
  • Rolling shutter results for 6K, 5.9K, and 4K DCI full frame as well as Super 35
  • Summary table of all measurements

So let’s dive into the results.

Dynamic Range Measurement of the Panasonic S1H

If you are not familiar with how we test dynamic range in our lab, have a look here first.

As mentioned at the beginning of the article, the list of resolutions/codecs/ frame rates/crop options on the Panasonic S1H is endless, and bearing in mind the fact that this camera offers two native ISO’s (640 and 4000 in V-Log), I had to focus at these key areas: 6K 3:2, 5.9K 16:9, 4K DCI full frame and Super 35 in 24p, 30p and 60p.

ISO 640 V-Log 6K 3:2 Mode (5952×3968) and 5.9K 16:9 Mode (5888×3312) Dynamic Range Results:

First positive surprise: in 6K 3:2 mode (5952×3968) and 5.9K 16:9 mode (5888×3312) the dynamic range measured by IMATEST at a signal to noise ratio (SNR) of 2 is 12.7 stops! This is the second-best result that we have ever measured, right after the ARRI Alexa (14 stops at SNR=2).

IMATEST dynamic range result Panasonic S1H in 5952×3968 (full frame 3:2 mode) V-Log ISO640. 12.7 stops are calculated for SNR = 2

 

Waveform plot of the Xyla 21 step chart Panasonic S1H 5952×3968 Full frame V-Log ISO640 – about 13 stops can be identified above the noise floor.

Looking at the waveform plot of the step chart the IMATEST result is confirmed – about 13 stops above the noise floor can be identified, see the plot above. As a video codec, H265 10bit 4:2:0 (HEVC) at around 200Mbit/s is the default for the 6K 3:2 and 5.9K 16:9 modes (I measured around 180Mbit/s for the step chart video files).

ISO 640 V-Log 4K DCI (4096×2160) Full Frame Dynamic Range Results:

Next up is 4K DCI (4096×2160) in full frame mode. Quite interestingly the dynamic range drops to 12.3 stops. This comes unexpectedly, as I was assuming that the 6K sensor width would be scaled to 4K hence the noise and therefore dynamic range should slightly improve.

IMATEST dynamic range result Panasonic S1H 4096×2160 Full frame V-Log ISO640. 12.3 stops are calculated for SNR = 2

Looking at the noise results in IMATEST (lowest graph of the three in the image above) and waveform plot for 4K DCI full frame mode below, it becomes obvious that the noise floor is more pronounced – simply more noise is present.

A major difference to 6K and 5.9K is the used codec: H264 10bit 4:2:2 is the setting for 4K DCI, and the step chart video files showed about 420 Mbit/s bitrate.

Waveform plot of the Xyla 21 step chart Panasonic S1H at 4K DCI Full frame V-Loog ISO640 showing 12 stops above the noise floor.

Now we are left to speculate what is happening here – why are we getting 0.4 stops less?

First, in 6K and 5.9K, the measured H265 bitrate of about 180Mbit/s is low for 10bit 6K. If we compare 5.9K 16:9 (19.5 Megapixel) to 4K DCI (8.8 Megapixel) we have about 2.2 times the pixels to encode – it equals to about 80Mbit/s H265 4:2:0 compared to 420Mbit/s H264 4:2:2 for 4K. Is this enough to encode the noise in the lower stops properly? I could not see any macro blocking in the shadows so it does looks robust to my eye.

Second, we can see that the image in 6K and 5.9K at ISO640 is less noisy compared to 4K DCI. Again, pure speculation – but is some sort of internal noise reduction happening here to help encode the image at this low H265 bitrate? Is the full frame 4K DCI image much closer to the “raw” image of the sensor?

Third, the spec sheet mentions a low pass filter that is used to reduce Moiré. Could it be that the 6K image is a bit soft on the pixel level due to the low pass filter, and therefore the noise is blurred to an extent? As will be seen below looking at the ISO4000 results, to an extent (about 0.1 stop) this seems to be true but cannot explain the 0.4 stops difference that we are seeing.

For your info, comparing the Panasonic S1H 4K DCI dynamic range results to the Panasonic S1 UHD results, we are getting almost equal numbers: 12.3 stops for the S1H, 12.2 stops for the S1 at SNR = 2.

Now let’s see what happens if we increase the ISO to the second gain value, namely 4000.

ISO4000 V-Log 6K 3:2 Mode (5952×3968) and 5.9K 16:9 Mode (5888×3312) Dynamic Range Results:

The next surprising result – at ISO4000 we see only a slight drop of 0.4 stops in dynamic range, see the IMATEST result below. While we got 12.7 stops at ISO640, we are now seeing 12.3 stops at ISO4000 for 6K and 5.9K full frame mode.

IMATEST dynamic range result Panasonic S1H in 5952×3968 (full frame 3:2 mode) V-Log ISO4000. 12.3 stops are calculated for SNR = 2.

Looking at the waveform below, this result is confirmed. More than 12 stops can be identified at ISO4000 (even the 13th stop looks still quite solid to my eye).

Waveform plot of the Xyla 21 step chart Panasonic S1H 5952×3968 Full frame V-Log ISO4000. More than 12 stops can be identified above the noise floor.

ISO 4000 V-Log 4K DCI (4096×2160) Full frame Dynamic Range Results:

Another surprise – in 4K DCI mode almost the same result as for 6K and 5.9K is now achieved – 12.2 stops. Even more surprising is the fact, that there is almost no difference between ISO640 and ISO4000 in 4K DCI mode – only 0.1 stops of dynamic range are lost switching to the higher ISO value – very impressive!

IMATEST dynamic range result Panasonic S1H 4096×2160 Full frame V-Log ISO4000. 12.2 stops are calculated for SNR = 2. Notice that we are getting almost exactly the same result as with ISO 640!

 

Waveform plot of the Xyla 21 step chart Panasonic S1H at 4K DCI Full frame V-Log ISO4000. 12 stops can be identified above the noise floor.

Another interesting observation: at ISO4000, the 6K and 5.9K V-Log results are now very similar to the 4K DCI full frame results – we see only 0.1 stops difference. Seems that noise reduction (speculation #2 above) is not so present here. The 0.1 stops could be due to the low pass filter blurring some noise on the pixel level (as mentioned earlier, see speculation #3).

Rolling Shutter Measurements of the Panasonic S1H

We are using a strobe light at 300Hz which creates a pattern of black and grey bars due to the nature of read out from top to bottom of CMOS sensors. Each set of black/grey stripes is equal to 3.3 [ms] of rolling shutter.

Let’s start with the worst scenario: the 6K 3:2 mode should have the highest rolling shutter, as all the vertical 3968 pixels from top to bottom have to be captured for each frame. In this mode at 23.98 frames per second, the Panasonic S1H has a rolling shutter of 29.7 [ms].

This is a very high value – the rolling shutter is partially mitigated thanks to the stabilized sensor so handheld shooting is no problem but takes it slow on pans.

For 5.9K 16:9 mode at 23.98fps, a rolling shutter of 24.8 [ms] is measured, for 29.98fps the result is better at 21 [ms]. For a full frame sensor, those values are average. Looking at the 23.98 fps result, the Panasonic S1 was a tad better – at 25fps a value of 22 [ms] capturing UHD was measured for the S1.

A rolling shutter of 29.7 [ms] is measured for the Panasonic S1H in 6K full frame 3:2 mode at 23.98 fps.

The situation improves slightly at full frame 4K DCI settings. A rolling shutter of 24.2 [ms] is measured, see the result below.

 

A rolling shutter of 24.2 [ms] is measured for the Panasonic S1H in 4K DCI full frame mode at 23.98 fps.

In the Super 35 mode, a result of 13.9 [ms] is achieved for 4K DCI 23.98 as well as 59.94 fps. This now is a good result – but again the Panasonic S1 was better at 10.4 [ms] at 25 fps.

 

A rolling shutter of 13.9 [ms] is measured for the Panasonic S1H in 4K DCI Super 35 mode at 59.94 fps.

Summary of Test Results for the Panasonic S1H

Wow, you made it until this point? Great, so here is my summary table for all the different modes and ISO’s.

Summary of cinema5D lab results for the Panasonic S1H.

All in all, I am impressed – the dynamic range results are exceptional. Panasonic claims this cam to be a 14+ stops camera, which is close to the signal to noise ratio = 1 result (13.8 stops) – as mentioned earlier we are a little more strict here and use the SNR = 2 result of 12.7 stops as the official cinema5D rating (following ARRI’s logic).

Furthermore, the fact that the dual gain sensor shows very similar dynamic range results (in 4K DCI) at both ISO640 and ISO4000 settings is phenomenal. You can use the higher ISO without any impact on your final image.

However, where there is light you also find shadows: compared to the Panasonic S1, the rolling shutter in all modes is higher – in full frame depending on the fps setting around 13%, in Super 35 mode about 33%.

So, is the Panasonic S1H worth the 4000 USD versus the 2500USD Panasonic S1 (+200USD for the V-Log upgrade)? Certainly if you are looking for a Netflix certified camera (Full information can be found by clicking here and here). My colleague Johnnie might also help us with this decision. His Panasonic S1H review and the short documentary he made as part of this review are now online and can be watched by clicking here.

And yes, I have not decided on this subject myself either…

Are you considering the Panasonic S1H as your next full frame camera? Let us know in the comments below.

Advertisement

55
Leave a Reply

avatar
14 Comment threads
44 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
15 Comment authors
RolfInterceptor121Paul CurtisAndy Greenhalgh Recent comment authors
1
1
- 2
Filter:
all
Sort by:
most voted
Alex
Alex
GuestOctober 29th, 2019

Amazing DR findings. Wow, second to the Arri.

Noah Yuan-Vogel
MemberOctober 29th, 2019

Second to the Arri in a lineup that doesn’t include any of Arri’s competitors, No REDs, no Venice, no Varicam, no C700 etc.

That said, the methodology seems odd here, arent you comparing to Alexa which was only tested at 2K? And we know that downscaling increases DR numbers? Seems like the only number that really matters is DR at 1080p downscaled from various sensor/recording modes since that is the only data that can be directly compared as some other cameras where only tested at 1080p and we know that testing at 1080p gives an advantage in this measurement. Seems very odd not to include downscaled 1080p analysis and to compare 6K/4K against Alexa 2k/1080p downscaled output…

Also I have been looking at h265 encoding lately and am finding there is something a bit odd as it naturally applies a lot of noise reduction and shifts shadow contrast somewhat achieve its greater efficiency. This is a big problem for using it as a capture format, especially for log recording. I’m seeing this in some Zcam h265 sample recordings from their new cameras and from the slashcam S1h test which includes h265. I’d be more inclined to trust the h264 4k numbers for this reason, but likely the s1h would gain some back in an HD downscale.

 Dennis Schmitz
Dennis Schmitz
MemberOctober 29th, 2019

H265 doesn’t really apply noise reduction. Using X265 I can compress grainy 4K footage down to 20Mbit with preserving all the grain.

What you see on the S1H is heavy temporal noise reduction applied to the 5.9 und 6K modes and the reason for their higher dynamic range.
Also the reason why these modes have many issues with any types of motion, especially with low contrast (objects out of focus, similar hues etc.). Smearing and sometimes strong ghosting artifacts occur quite often.

 Luke Wen
Luke Wen
MemberOctober 29th, 2019

There are two important encoding parameters that affect image quality in H.265/HEVC codec, called SAO (Sample Adaptive Offset) and SIS (Strong Intra Smoothing). They are usually switched on by default and must be explicitly turned off.

 Dennis Schmitz
Dennis Schmitz
MemberOctober 30th, 2019

Problem is, footage recorded externally in ProRes HQ shows the same issues.

Noise reduction messing up motion.

 Dennis Schmitz
Dennis Schmitz
MemberOctober 29th, 2019

The Zcam has strong temporal noise reduction as well creating the exact same artifacts lol

Noah Yuan-Vogel
MemberOctober 29th, 2019

I’m not suggesting it’s necessarily noise reduction by design. But by necessity all image compression techniques have noise reduction side effects as noise is by nature impossible to compress. Are you sure the temporal noise reduction you’re seeing isn’t just a side effect of aggressive compression?

 Dennis Schmitz
Dennis Schmitz
MemberOctober 30th, 2019

Record externally and the issue is still there. And yes, it is an issue created by badly implemented motion compensation used to reduce noise -> temporal noise reduction.

The image looks great, maybe a bit thin, until you have areas with low contrast detail in motion and it gets worse the higher the ISO or the darker the subject is.

 Dennis Schmitz
Dennis Schmitz
MemberOctober 29th, 2019

Great DR, but cheated by using strong temporal noise reduction which is just horrible in the 6K/5.9K modes and still pretty strong in the 4K/UHD modes.

6K/5.9K modes are pretty much not usable at the moment. Ghosting and smearing artifacts even appear at native ISO settings, but these artifacts even do appear in 4K/UHD.

Really hoping for a fix soon, even if that means less impressive synthetic DR measurements I don’t really care about. I actually want to use the camera without the image looking horrendous wherever there’s movement with low contrast detail.

 Wayne S
Wayne S
MemberOctober 29th, 2019

What happened with that 16 stop firmware update?

Bill
Bill
GuestOctober 29th, 2019

Hello Gunther and all here. Thanks for your very informative tests and reviews of this camera. (I just purchased) So far, after years of m43 and before that 35mm, I moved to this system and am liquidating tons of 43 stuff.
And yes, there is something strange about the high rez modes of this camera. And the HFR stuff. I thought it was my monitor or brain or something. Without getting too tech, maybe this can be rectified somehow? I have never shot full frame in anything other than 35mm film, so I thought it had to do with the large frame limitations, like Imax film.
But noise reduction is gross looking and I’d much rather have noise than weird computing going on. Imagine what that would do to the cat video business?!
Anyway, I’m going to take as close of a look as I can and will report back here if I have any findings.
I like the camera so far, and especially love that they power on and roll with no boot time. And they shed water… Thanks.

 Ross Fairgrieve
Ross Fairgrieve
MemberOctober 29th, 2019

Thanks for this Gunther. Do you guys have a library of lab test results anywhere on the site so that we can compare results from different cameras?

 Ross Fairgrieve
Ross Fairgrieve
MemberOctober 29th, 2019

I should have said, I am aware of the ‘Test Lab’ menu under ‘Reviews’ but I mean a page where you can actually see, for example, all of the dynamic range tests that you’ve carried out side-by-side, as opposed to just all of the separate posts that are categorized as reviews and may or may not have tech tests included. Thanks!

Geoffrey Reacher
Geoffrey Reacher
GuestOctober 31st, 2019

I don’t understand the use of compression, let alone varying compression, to do a test like this.

Why was this done?

 Paul Curtis
Paul Curtis
MemberNovember 30th, 2019

Do we know what sensor is in the S1H? Hearsay says it’s the Sony IMX451.

If this is the case then there’s no sensor mode that does 6k at 14 bit. It’s 6k at 12 bit so there’s no way to get 12.7 stops of range… Unless… the S1H is doing highlight reconstruction before baking a movie format.

Is there a RAW movie mode with it yet?

I ask out of curiosity, having been diving deep into sigma fp tests and seeing very clearly the range in the RAW for cine vs stills. The S1H has a chunk of processing before you see the results, and a wedge test is an easy one to benefit from highlight reconstruction (in fact i’d have thought you’d see 13+ stops)

cheers
Paul

Interceptor121
Interceptor121
GuestMay 12th, 2020

Your conclusion is incorrect. Log compression is done in such a way that you can pack more DR in a smaller space so the outcome will be higher than the readout if the sensor can manage the extra DR. Where this camera will hit the limit is the fact that the V-log implementation is 10 bits not 12 and therefore there is additional scaling applied. As cameras do not resolve 10 bit colour there will still be headroom for greys that are monochromatic. So it is possible for this implementation to produce 14 stops DR with around 24 bits colour which is fine for practical use. The 13.8 SNR=1 result here is actually underestimated as log is not decompressed so it is likely that in an HDR space this camera has 14+ stops

Interceptor121
Interceptor121
GuestMay 12th, 2020

I believe you guys make a lot of efforts to do those tests but I have sometimes the impression that among you there is a lot of passion but not much engineering knowledge.
On this an other posts on this website you allude that when you take footage at higher resolution or lower resolution on the same readout SNR should improve. This is not the case unless the ratio of pixels is 4:1 as pixel binning goes with a 2×2 matrix. So it is not surprising that there is not improvement but when you compare 4K and 6K un-cropped and actually a drop as you are eliminating signal. You seem also to suggest elsewhere that shooting 4K and then using results at HD in an editor brings benefit compared to shooting HD in camera again this is incorrect. NLE does not perform resampling and averaging of pixels they simply trash pixels. If you had to resample each pixel in the frame would have been pixel binned and outputted real time and this is not something any NLE or photography application for that matter supports. Only printing a scaled image performs this process and this is why it takes time. If instead a camera does a full read out and has 4x pixels there will be 1 stop benefit of SNR because of pixel binning real time done by the DSP. But this cannot be generalised without knowing the readouts modes so you should really make an effort to stop spreading those beliefs that generated work downstream to those taking your findings at face value.
Finally your tests of DR are plagued by a number of other issues that are far more relevant than the DR of the sensor. You use a ffmpeg framegrab on an 8 bit video file to assess dynamic range on a log curve. Because log curves are compressed this will not actually show the DR capability of the camera but just the ability of your limited toolset to deal with the output produced. You should consider performing tests using an HDR colour space after a transformation LUT unless you use RAW as SNR is impacted by log compression in the darks. As manufacturers use slightly different log compression those that are closer to your test model will have better results than others, again those log formats are not normalised and tests should be done after normalisation in the output space not before because this is not how you are going to use it. There are other things that are useful to people like SNR and color accuracy that are easier to test perhaps you could try to include those instead of trying on DR that is almost impossible to measure in an accurate way due to the number of variables at play

Rolf
Rolf
GuestMay 27th, 2020

I would love to see an update with firmware 2.0 and the NR set to -1. I am interested to see if that affects the dynamic range in just 6k or across the board.

 Luke Wen
Luke Wen
MemberOctober 29th, 2019

Still can’t match ARRI in terms of highlight DR, which is more than 7.5 stops above 18% gray. Sony sensors typically manage 6.2 stops.

 Renny Hurst
Renny Hurst
MemberOctober 31st, 2019

I’ll stick with my GH5 for now…Maybe ill get a firmware update

1
1
- 2
Filter:
all
Sort by:
most voted

Take part in the CineD community experience

mode_comment55