Blackmagic RAW – Dynamic Range on the URSA Mini Pro 4.6 Tested

December 12th, 2018
Blackmagic RAW - Dynamic Range on the URSA Mini Pro 4.6 Tested

EDIT: Based on multiple comments made by our audience suggesting that the original results of the URSA Mini Pro 4.6 Dynamic Range tests may be flawed since we did not execute camera black shading prior to the tests, we decided to rerun it. Our workflow was as follows: Let the UMP 4.6 warm up for about 15min, perform black shading in the menu with the lens cap on and then repeat the tests at 5600k WB.

As can be seen from the above IMATEST screenshot, we are getting exactly the same results as before (SNR = 2): shooting ISO 800 BRAW 4.6k (scaled to UHD) gives us 12.6 stops of dynamic range at a signal to noise ratio of 2 (13.7 for a signal to noise ratio of 1 without further post processing like highlight recovery or noise reduction in DaVinci Resolve 15.2. Shooting ProRes also gave the same results as in the initial test.
One thing however did change after black shading – the shadows that were a bit blueish in the original tests became black. (Thanks to doing black shading).

Nevertheless, thank you all for your suggestions, keep them coming in the future as we all can learn from each other!

Original Article:

When Blackmagic RAW (BRAW) was announced a while ago, I was curious to get my hands on a BMD URSA Mini Pro and test it’s dynamic range with this new codec. The combination of having the full flexibility of 12bit RAW and smaller file sizes at the same time is a real innovation of BMD. How does it fare on the UMP 4.6?

A while ago BMD was so kind to send us the URSA Mini Pro 4.6 but unfortunately, we were all out traveling yet there was time to install the new firmware 6.0 and test the dynamic range at ISO 800 and 3200 using our test setup described here.

One note upfront: I was very surprised to see the playback performance of BRAW in DaVinci Resolve Studio 15.2. I am using an I7-4720HQ Laptop with 16GB RAM and a GTX980M graphics card, and 4.6k ProRes XQ resulted in ~7 frames / second playback (fps), 4.6k RAW DNG (lossless) resulted in ~11 fps, whereas BRAW had smooth 24 fps playback in a UHD timeline! Hence, not only smaller filesizes with BRAW but also vastly improved playback – a real winner combination.

4.6k ProRes XQ ISO 800 dynamic range results

As a baseline, we started first with our standardized testing procedure analyzing the 4.6k ProRes XQ dynamic range. Here, at ISO 800 the UMP 4.6 does not disappoint and comes in with a strong 12.5 stops of dynamic range at a signal to noise ratio of 2 according to our IMATEST analysis (13.6 stops at a signal to noise ratio of 1). I also tested ISO 400, which yielded slightly lower results. This is the best result in our recent 2018 testing, coming in half a stop higher than the Sony FS7. However, quite far from the 15 stops that Blackmagic Design claim. Anyways, we are going to test BRAW subsequently, so let’s see. The noise floor looks very clean, looks like some internal noise reduction is going on here.

Fig.1: Step chart waveform plot of the UMP 4.6k ProRes XQ at ISO 800: a little more than 12 stops can be discerned above the noise floor (code values scaled to 8bit).

4.6k BRAW dynamic range results

Now, there is a little issue with BRAW and our standardized testing workflow, as we usually try to avoid any 3rd party software when extracting keyframes for our IMATEST analysis (other than FFmpeg) to avoid any kind of manipulation/influence of this software to the image. With BRAW this philosophy doesn’t work, as we have to use DaVinci Resolve to develop the files – the latest version 15.2 Studio in our case.

ISO 800 BRAW constant bitrate 3:1 4.6k scaled to UHD

Now, using ISO 800 yielded again the best results, with the dynamic range coming in at an even stronger 12.6 stops at a signal to noise ratio of 2 (13.7 stops for SNR = 1). However, this slight difference to ProRes might be also a result of scaling the 4.6k BRAW file to UHD (3840×2160) – as there is no 4.6k timeline preset in DaVinci Resolve (for exporting keyframes to IMATEST).

Fig.2: Step chart waveform plot of the UMP 4.6k ISO 800 BRAW. Again, a bit more than 12 stops can be identified visually above the noise floor.

NOTE: DaVinci Resolve offers a “highlight recovery” option for BRAW, which can reconstruct the RGB channels even if one channel is already clipped. Together with the noise reduction features, you can heavily post-process the BRAW files – I just tried around a little bit and was able to reach around 13 stops for SNR = 2 (14 stops for SNR = 1). However we don’t count this result as our intention is to test the untouched files straight off the camera – this is not a post-processing contest, it is about a standardized result using a standardized workflow for all the cameras we test.

ISO 3200 BRAW constant bitrate 3:1 4.6k scaled to UHD

For ISO 3200 we counted 11.7 stops for an SNR = 2 (12.9 stops for SNR = 1), again without post-processing. Playing around with the highlight recovery option and noise reduction, I was able to get around 12 stops for SNR = 2 (13 stops for SNR = 1).
Noise kicks in, hence about one stop is lost vs ISO 800. This is still a very good result.

Fig.3: Step chart waveform plot of the UMP 4.6k at ISO 3200: noise kicks in, and about one stop is lost vs. ISO 800, leading to around 12 stops of dynamic range.

4.6k RAW DNG (lossless) dynamic range results

Similar to BRAW, we have to develop the RAW DNG files in DaVinci Resolve in order to export keyframes for analysis in the IMATEST software.
Interestingly, the dynamic range results of the Cinema DNG files without postprocessing are lower than with ProRes and also BRAW, coming in slightly above 11 stops untouched (SNR = 2). The DNG files exhibit much more noise than BRAW and ProRes, hence leading to a lower dynamic range reading in IMATEST. This is a similar phenomenon as we observed with the BMPCC 4k. Using postprocessing like the highlight recovery feature in Resolve along with noise reduction lifts it above ProRes and BRAW (I played around and was able to get more than 13 stops at an SNR = 2, and almost 14 stops for SNR = 1).

Fig. 4: dynamic range comparison of the UMP 4.6k (BRAW) ISO 800 result with other cameras. It fares very well, coming in one stop higher than the BMPCC4k.

Conclusion

When testing the various codecs on the UMP 4.6k there are clear signs of internal noise reduction going on in ProRes and BRAW, very similar to what we observed with the BMPCC4k (please watch particularly the end of our BMPCC4K review for an insight into our testing procedure for dynamic range).
Speaking of the BMPCC4k, the UMP4.6k overall comes in one stop higher with regards to the dynamic range. Impressive. However, I cannot reproduce the 15 stops dynamic range that Blackmagic Design are claiming, even at a signal to noise ratio SNR = 1 the maximum I was able to squeeze out by heavy post-processing was about 14 stops. In contrast to this, I was able to confirm the 13 stops for the BMPCC4k at SNR = 1, thereby confirming Blackmagic’s statements.
Therefore the untouched BRAW ISO 800 SNR = 2 (or 1/SNR = 0.5) dynamic range result will be the cinema5D reference for future dynamic range comparisons with other cameras.

We know that many of you are waiting for an overview chart of all our dynamic range and other technical tests, and please stay tuned for a bit, we are still working on it. There’s a lot of parameters to think about, and also it’s not so easy to display elegantly on our site – however it won’t be long until we will share it with you!

You can find Nino’s Ursa Mini Pro Hands-On video review by clicking here and Johnnie’s review and “real world footage” by clicking here.

Did you already work with the URSA Mini Pro 4.6 in RAW recording mode? If yes, What is your impression? Please share your experience with us in the comments below.

58
Leave a reply

guest
1
1
- 2
Filter:
all
Sort by:
latest
Scotty
Scotty
Guest
January 31st, 2019

I’m not a tecspurt – just an operator..I used the UMPro4.6k on a 6 week shoot Nov 2018, I decided to shoot BMRaw @ 4k 16×9 (constant BR and floated between 8:1 and 12:1)the files are solid and beautiful to work with, struggled a bit in super low light at 1600 ( basically ran out of light during that horrible storm in Italy last Nov ) but we did scrape enough out of the files to get usable footage – very happy with the new firmware upgrade @3200 – looking forward to BMRaw in the BMPCC. ( would love a proxie option btw)

Autokroma
Guest
January 31st, 2019

You can now import BRAW Files on Adobe Premiere Pro with BRAW Studio, our new plug-in. It’s only for Windows but there will be soon a version for OSX !
Of course, with BRAW Studio you can import AND tweek the RAW parameters inside Premiere.
Check this out : https://autokroma.com/ (With Free Trial Demo !)
If you like it, send us some feedbacks !

 Gellért Ficzere
Gellért Ficzere
Member
January 10th, 2019

I think you missed the ‘Sensor Calibration’ in the Setup menu since there’s a lot of blue in the shadows.
Sensor calibration can improve the image quality and noise performance a lot (relatively). It is neccessary to do after firmware updates and when you had calibrated in a totally different temperature environment before. (my experience)

Gunther Machu
Guest
February 7th, 2019

Hi Gellert, see the edit section at the top of the article – we retested doing black shading and got exactly the same results for the DR, but yes, the shadows are less blueish now.

Betursus
Betursus
Guest
December 20th, 2018

Hello. A simple question… BRaw is available on Ursa Mini 4k?

Alexandre Lauret
Guest
December 13th, 2018

Looking forward to its implementation on the regular 4.6K…someday…I hope… ?

Guest
December 13th, 2018

Why are you ignoring the 2 stops available to the left that clearly show gradation ?

Guest
December 13th, 2018
Reply to  Gunther Machu

thanks for that. in that article you measure from what would be the 2nd from the left patch as just under clipped value. I can see that cam makers would count both of those and come out with numbers that are inline with what you measured. Even discounting the pure clipped leftmost patch, counting should start at the 2nd then since there is usuable picture info. if the difference between the pure white clip and next stop down was less than a full stop let it go, but if its about an honest stop it should be counted as thats usable image info.

andi
andi
Guest
December 13th, 2018

how many DR on prores record ?

Member
December 13th, 2018

I feel like your notion that you should test the footage before any adjustments does not make sense since the codec itself does not clip highlights but rather the log curve baked into the codec. The raw codec has log curve meta data applied for display purposes only and a ton of highlights can be recovered from them as opposed to the non raw codes. Why not test after highlight recovery and simple noise reduction since any serious project would apply these to the footage anyway.

Member
December 13th, 2018
Reply to  Gunther Machu

But its not directly off the sensor, a curve is applied thats the point. BlackMagic made up a log curve they felt was a balance of looking ok in camera and dynamic range. However its not the dynamic range of the sensor. Further more you are comparing footage with no noise reduction against say FS7 footage with internal noise reduction so again its not a fair comparison. I feel like you should compare the best of both rather then the default of both.

Jamie LeJeune
Jamie LeJeune
Member
December 12th, 2018

Did you run a black balance calibration after updating the firmware to v6.0 and before running the test? I’ve found that running the black balance can really improve the noise performance on Ursa Mini Pro, and it’s absolutely necessary to run it after every firmware update.

Jamie LeJeune
Jamie LeJeune
Member
December 12th, 2018
Reply to  Gunther Machu

In my experience when the black balance is out it can make a 1 stop difference in noise, and running your test without running a proper black balance first quite likely explains why your results for the 4.6K are much lower than BMD’s own claimed stats. It also makes your current test results unreliable. I fully appreciate that it must be tough to work with unfamiliar cameras, but for every camera model out there, you have readers/followers who are owner operators of those cameras. For future tests, it would be a good idea to reach out and get some detailed knowledge from skilled owner operators of each camera model before you run tests.

Jamie LeJeune
Jamie LeJeune
Member
December 13th, 2018
Reply to  Gunther Machu

I can only report what I’ve seen in my own extensive use of the camera ¯\_(ツ)_/¯ It does make sense, however, that when the black balance is out, noise will go up because noise on every digital sensor is most prominent in the blue channel, so if that blue channel is raised in the shadows by an uncalibrated black balance, the bottom of the signal will be disproportionately noisy. And, that is exactly my experience whenever I’ve updated the firmware on the BMD Ursa Mini Pro and viewed (or accidentally shot) images before recalibrating. If you find it hard to believe, you’ve got the gear, so I respectfully suggest running a comparative test to see for yourself.

Jamie LeJeune
Jamie LeJeune
Member
December 14th, 2018
Reply to  Gunther Machu

Glad to know you are retesting. You should see some higher numbers in the results after calibration.

The procedure for black shading is covered on p115 of the current Ursa Mini manual.

How are you processing the BRAW signal in Resolve? There are 3 options in the raw tab — film, video, and extended video. And beyond that there are sliders to affect the contrast curve and the highlight and shadow rolloff. These settings all affect the distribution of the available stops, and I would assume affect your test as well. Or are you doing your own expansion from log using the curves control or LGG?

The Kelvin setting of the camera also affects noise performance in the shadows — tungsten balance at 3200K has more noise than daylight balance at 5600K. This isn’t unique to the Ursa Mini Pro (Art Adams describes the effect here: http://www.dvinfo.net/article/optical-science/sensorcolorbalance.html), but it is more noticeable on cameras that don’t have any noise reduction in the internal processing like the URSA Mini Pro, compared to cameras like the C300 and FS7 that do add noise reduction to the processing.

In your article you stated that BMD is applying noise reduction to the internal processing, but my understanding is that this is not the case. The new firmware does have a new debayer designed to reduce false color/detail relative to the previous debayer, and the result is likely cleaner than the old debayer, but it’s not noise reduction per se. I’d suggest reaching out to Captain Hook at BMD for confirmation on that point.

Good luck with the new test!

Gunther Machu
Guest
February 7th, 2019
Reply to  Jamie LeJeune

Hi Jamie, as you can see at the top of the article in the edit section we retested the UMP 4.6 by doing black balancing (black shading as it is called by Blackmagic) and got exactly the same results as before.

Jamie LeJeune
Guest
February 12th, 2019
Reply to  Gunther Machu

All good science requires repeatable results : )
I applaud you for running the tests again to check. The better color balance in the blacks that you saw after balancing does help post recovery of underlit or underexposed shots. I’d have thought it would also lead to better test results as well, but software is more objective than our eyes and perhaps those significant differences I’ve seen after balancing are solely aesthetic and subjective. Thank you for all the great work to provide us all with solid objective information!

Gunther Machu
Guest
February 12th, 2019
Reply to  Jamie LeJeune

Hi Jamie, thanks for your kind words – very encouraging!

Guest
December 13th, 2018
Reply to  Gunther Machu

Doing a black balance is standard procedure on most cameras. certainly after a firmware update. others like canon will even say after ISO change and certainly if the camera has taken a major in change in tempeture – the hot and cold kind. So honestly that kind of killed the accuracy of the tests.

Gunther Machu
Guest
February 7th, 2019
Reply to  Steve Oakley

Hi Steve, see the edit section at the top of the article – we retested doing black shading and got exactly the same results.

Chris Niswonger
Chris Niswonger
Guest
December 12th, 2018

You guys ever put the C200 RAW into the lab for dynamic range? I would love to know your stop findings.

 Keith Mullin
Member
December 12th, 2018

OK, maybe silly question here, but I am really new to the URSA Mini Pro. When shooting RAW, does your ISO setting make a difference? Hear me out. I did a test recently, where I shot BRAW, and ramped up the ISO between each shot (200, 400, 800, etc), without adjusting anything else (IRSI, etc). In resolve I went into the RAW controls for each clip and reset the ISO to the same setting and applied the same built in LUT and the clips are absolutely indistinguishable from each other. The way I interpreted the result was that the sensor has the same actual sensitivity no matter what ISO setting you are using, and the ISO functions similarly to the EI on Sony cameras, in that it will let you over or under expose your footage easily.

Does the ISO actually change anything about what the sensor is doing in RAW? Or are the differences yielded in the above test a result of under exposing the footage when rating the camera at 3200?

 Keith Mullin
Member
December 12th, 2018
Reply to  Gunther Machu

I think we are not communicating the same thing. My question is about whether or not changing the ISO when shooting RAW doesn’t do anything at all to what is being captured, but rather, the changes made to the IRIS push the image higher or lower into the range of what is being captured. In my test the only thing I changed was the ISO, and when I corrected for that in Resolve the clips are exactly the same, no difference in noise or roll off, and no over exposure, even though I exposed for 200 ISO and went all the way to 3200, which should have blown out something, if it actually changed the behavior of the sensor.

Andrée
Andrée
Guest
December 13th, 2018
Reply to  Keith Mullin

Keith, it’s correct that ISO is “just meta data” when shooting raw. But ISO is still an important tool for monitoring, since it might influence your choice of aperture and shutter. Both if you DON’T change those, the sensor gets the same amount of light and you can then push and pull your image in post if needed.

 Keith Mullin
Member
December 13th, 2018
Reply to  Gunther Machu

Hi Gunther, I am following what you are saying. And I am all too familiar with how confusing it is when talking about “rating” a sensor vs. having an ISO setting that changes the behavior of the sensor. I have been working with the FS7 for about 4 years now, usually in CineEI mode where the sensitivity of the sensor is locked, but you can “rate” the sensor as more or less sensitive for viewing and exposure purposes. Your tests are very useful for understanding what is possible when overrating the sensor by 1 or 2 stops, which you might want or need to do depending on lighting or the depth of field you want to use.

I wonder of the results you are seeing are due to the gamma curve that is being used by the camera, since the URSA uses a Log RAW instead of Linear RAW. Knowing that the sensor is not becoming more sensitive by rating it at 3200 instead of 800, I would expect you to lose 2 stops of DR in the test as you under exposed, but instead you only lost 1.

 Keith Mullin
Member
December 13th, 2018
Reply to  Gunther Machu

But where does that “extra” stop come from? Since rating the sensor at 3200 doesn’t change anything that is happening at the sensor, just changing how much light is hitting it as you close down the iris?

Crimson Son
Crimson Son
Guest
December 12th, 2018

I don’t like Fig 4. It seems more of subjective chart presented as an objective chart. You should replace it with SNR rating between stops. Or a difference displacement image.

Joseph Moore
Member
December 12th, 2018

I appreciate the test, and that you are keeping it consistent, but I don’t know why one would expect to find a meaningful difference in DR between codecs if shooting with BM film. BM tunes BM Film for each camera, and all codecs have more than enough room to contain it. The only small difference – given your testing methodology – comes from what you consider the noise floor.

Given how you test, it’s totally understandable that you would consider the CDNG to have less usable DR if you don’t process it. RAW files are expected to be processed – in fact must be processed – in post, therefore it is reasonable to expect that you should do some amount of NR is done in post in order to compare DR against a codec that already has it baked in. Otherwise you’re testing apples against oranges.

It’s very similar to how people complain that RAW shadows are noisy. Of course they are, the camera hasn’t pre-smooshed them. ;-)

Clayton Burkhart
Clayton Burkhart
Guest
December 12th, 2018

I would like to see you do a test with an F3 in SLOG 444 mode. Frankly Abelcine did a similar test 7 years ago and found the camera had 13.5 stops. Interesting that even after all this time that camera still stands up well, if not better than the current batch of cameras here (including the URSA 4.6) in terms of dynamic range.

Clayton Burkhart
Clayton Burkhart
Guest
December 12th, 2018
Reply to  Gunther Machu
Clayton Burkhart
Clayton Burkhart
Guest
December 12th, 2018
Reply to  Gunther Machu

Hi Gunther, are you forgetting to count middle grey ? (5 over and 7 under + middle grey = 13).

Andrée
Andrée
Guest
December 13th, 2018

Clayton, freeze the video at any point where you get a full screen of the waveform. Start at +5 which is zero and walk down the stairs to +4 (one stop), +3 (2 stops) and so on. You get to 11 stops and then the next one is already buried in noise. Based on that waveform, 11.5 feels generous.

Agree?

Clayton Burkhart
Clayton Burkhart
Guest
December 13th, 2018
Reply to  Andrée

Hi Gunther, I have worked with the F3 for 7 years. I know it is way beyond 11.5 stops. It certainly is far better than the FS7 in terms of dynamic range for instance. I also know from experience that it has about 1-1.5 more stops than the Red Epic as well. Which I believe someone rated at around 12 stops. I have often shot the two of them together in the past. From a methodology point of view, it might be a good idea to check will Abelcine. Not sure what chart they used compared to yours. Also, from what I have seen most count the patches not the space in between them, because those patches represent visible values on the spectrum, as opposed to the transition between them.

Dmiriy Sakharov
Dmiriy Sakharov
Guest
December 12th, 2018

Objectively. Fair. Thank!

Patrick Murray
Guest
December 12th, 2018

Wait…..Ursa doesn’t use the same CInemaDNG format as the BLack Magic Production Cameras?

Joseph Moore
Member
December 12th, 2018
Reply to  Gunther Machu

Yes, it does, but it also has a new BRAW codec that does a portion of the debayering (including noise reduction) in camera. It is comparable to ProRes RAW.

Brandon
Brandon
Guest
December 12th, 2018
Reply to  Patrick Murray

It does. It shoots both CinemaDNG, as well as the new BMRAW format.

Nathan Porter
Nathan Porter
Guest
December 12th, 2018
Reply to  Patrick Murray

It can, but BRAW is a new codec and i’m guessing they are just putting it through it’s paces.

1
1
- 2
Filter:
all
Sort by:
latest

Take part in the CineD community experience