I’m new to Blender development and thought I’d say Hi, so Hi!
I’m currently working on a little bugbear of mine: to get ProRes support for my renders.
.I have currently got ProRes 422 and 4444 with transparency tentatively working however it is limited to 8-bit depth due to limitations with the rendering module currently.
I wonder if anyone knows a good place to start with regards to supporting higher bit-depths?
Best regards
David.
There’s an issue about higher bit depth here:
This is a good chat channel to discuss things:
https://blender.chat/channel/sequencer-module
Thanks for the pointers. I’ll be sure to check them out later.
Quick update on my ProRes support modification.
Per the above issue, Blender currently passes a frame of RGBA 32-bit pixels from the byte buffer to ffmpeg, which handles the heavy lifting of encoding the video. This means video codecs are currently limited to a maximum 8-bits per channel.
Most video codecs do not accept data in RGB format as they are usually designed to work with YUV (or technically YCbCr) component data. In order to satisfy this requirement, Blender uses libswscale (ffmpeg’s software scaler) to do the colour format conversion from Blender’s internal format (RGBA32) to whichever format the select codec prefers.
In order to support higher than 8-bit per channel codecs, we need to use the float buffer as the pixel source. Unfortunately, libswscale will not translate from pixels in packed 32-bit float RGBA format (at least I couldn’t get it to work). There does appear to be support for planar 32-bit float and packed 16-bit half-float formats, however.
So the scheme I am currently using is:
10-bit per channel vs 8-bit per channel comparison., After an identical severe colour grade, the 8-bit video shows signs of dithering. [Ffmpeg prores_ks encoder currently only supports 10-bit]
I now intend to work out how to create a fork and push my changes so others can see/comment on my work thus far.
Isn’t ProRes a propriatery codec of Apple?
Considering how not even Davinci Resolve is supporting this codec on windows i am really curious about how your effort in developing this is going to work for Windows and Linux,
It’s an Apple codec, but not limited to Apple software.
Premier can certainly export in prores, and DaVinci can do it with some aftermarket support. After Effects, I honestly cannot remember. It’s not a codec I use very often.
I read some concerns about the quality of the FFmpeg encoder which was reverse engineered, but I have no idea if that’s a big problem in practice or FUD. Either way, the changes for this will also make it easier to support other high bit depth encoders like AV1 and DNxHD.
It is not limited if you license it. You can read the codec but not encode in it unless you license it.
Adobe is licensing it for Windows. So that is what makes me wonder how it will work in Blender.
Per the above issue, Blender currently passes a frame of RGBA 32-bit pixels from the byte buffer to ffmpeg, which handles the heavy lifting of encoding the video. This means video codecs are currently limited to a maximum 8-bits per channel.
Since #118559 - Cleanup: Video: change bMovieHandle function to take ImBuf as input - blender - Blender Projects the changes to make FFmpeg to use 32bit per channel float images should be relatively limited to the writeffmpeg.cc. Hopefully for someone with proficiency in FFmpeg C-API it would not be huge deal to pass higher bit depths buffers to FFmpef.
The requirement of licensing is incompatible with GPL.
What I have in (WIP) 10-bit/HDR PR (that I should get back to someday), is going the floating point route. See this PR: #120033 - WIP: Video: 10/12 BPP and HDR video output support - blender - Blender Projects and basically there if the input data is floating point, I convert from packed to planar format (since libswscale can’t convert packed RGBA floats into YUV), and then convert into AV_PIX_FMT_YUV420P10LE or similar.
Largest questions wrt all that I had, was actually in the “how to present the settings in the user interface”. Both because I’m not good at Python, and also how to present things like “8 or 10 bits/pixel” only for scenarios that are actually supported. Since not every ffmpeg codec supports higher than 8bpp data.
First of all, I am developing on Windows. I don’t have access to Mac and I don’t have the energy to use Linux on the desktop.
Most of the common video codecs are proprietary, in the sense that they are potentially protected by patents. This is true of AVC (H.264) and HEVC (H.265). It’s probably also true of older codecs such as MPEG-2, DivX and WMV, though their patents have likely now expired. Even AV1, which Google claims to be ‘royalty free’ could still be affected by 3rd party submarine patents.
Davinci resolve DOES support reading the codec on Windows. This is kind of the point, I want to be able to take intermediate format footage into 3rd party tools for editing and grading.
I hear you. It does read but doesn’t write. And this is excatly my point as well. How do you plan to circumvent Apple’s licensing restrictions on Windows?
Let me reasure you that i am not arguing against your plan. It does sound interesting even though i would prefer your efforts to go towards a better codec than ProRes and the gamma shift drama it brings with it.
The Blender Foundation will have to make a determination if it’s ok to include the FFmpeg ProRes implementation in Blender. This kind of legal thing is not something we can discuss here though.
But note that this is not the same as licensing Apple’s proprietary ProRes implementation.
First, let me thank you for your comment and constructive criticism.
Whilst it is, of course, within Apple’s remit to license their implementation to whomever they choose, they do not have a right to restrict other implementations except as granted by law. Since you can’t copyright a bitstream format, it is mainly an area of patent law.
It’s not clear to me that Apple owns any patents on ProRes. If they do, then ProRes is only in exactly the same situation as all the other codecs that are protected by known patents. This affects most modern codecs with the exception of VP9, AV1 and any that are now too old to be afforded patent protections.
In Apple’s white paper, they talk about “authorized” use vs “unauthorized” use. It is worth noting however, the use of the term “unauthorized” does not imply “infringing” nor “illegal”. Nor do they allege any wrongdoing on the part of FFMpeg - merely that their competing product may be less performant, etc.
In the same way, Apple has “authorized” certain repair companies but in no way does it infringe upon any of Apple’s rights if you take your Apple product to an “unauthorized” repair shop.
Apple ProRes is a codec technology developed for high-quality,
high‑performance editing in Final Cut Pro. Apple has licensed ProRes
to select companies for use in specific products and workflows.
In some instances, unauthorized codec implementations have been used
in third-party software and hardware products. Using any unauthorized
implementation (like the FFmpeg and derivative implementations) may
lead to decoding errors, performance degradation, incompatibility,
and instability.
I think its right that the Blender Foundation ultimately forms the policy on which codecs are implemented. Further, I would never wish (wittingly or unwittingly) to expose the Blender Foundation to any sort of risk of expensive and drawn out litigation. However, I urge the BF to at least make the policy consistent. If proprietary codecs are forbidden, they should remove all proprietary codecs from the product.
I agree ProRes isn’t perfect, however it is one of the most popular professional intermediate format codecs. Intermediate codecs are used for editing and other post processing rather than content delivery. They typically support intra frame coding, high bit-depth, high or no chroma subsampling and alpha channels. Other notable intermediate codecs are Avid’s DNxHD/DNxHR, Panasonic’s AVC-Intra and GoPro’s Cineform. DNxHR would be a viable alternative except FFMpeg doesn’t support alpha. I have yet to investigate Cineform.
Yes, I have reached the same conclusion. Ideally, we would be able to choose a profile, chroma sub-sampling level and bit-depth for all codecs that support them.
