Hardware accelerated video encoding/decoding for VSE
Name
José Vicente Barrachina Bosch
Contact
- e-mail: [email protected]
- chat: @jvteb:blender.org
- Blender repo
Synopsis
This project aims to incorporate into Blender’s VSE the necessary functionality to take advantage
of the hardware-accelerated video encoding/decoding mechanisms currently provided by the Ffmpeg libraries.
Benefits
The main and most notable benefits of using hardware acceleration for video encoding and/or decoding are the increased speed and performance. Additionally, using dedicated hardware frees up the CPU for processing other tasks to a large extent and allows for lower power consumption, improving performance in laptops.
These advantages can make Blender’s VSE more attractive to users who need to perform video encoding, decoding and transcoding, thus allowing them to incorporate it into their workflow.
Deliverables
-
The code required to implement hardware-accelerated video encoding and decoding in VSE depending on the user’s platform and configuration, including changes to the UI necessary to customize the process:
- Updated
Ffmpegintegration inBlenderto support hardware acceleration, taking into account licensing compatibility. - Implementation of available hardware detection mechanisms.
- Functions required to manage efficient data transfer between GPU and CPU.
- Updated Blender’s user interface and added new options to enable or disable hardware acceleration. Added options to select appropriated decodification/codification settings and display performance or status information.
- Updated
-
Updates to the user documentation required to explain the usage of the new functionality.
-
Appropriate tests and benchmarks.
Project Details
The VSE (Video Sequence Editor) is one of Blender’s video edition tools, allowing the user to integrate this workflow with others such as 3D modeling. Thus, VSE permits users to avoid the need for external tools.
Currently, VSE uses the Ffmpeg libraries to encode and decode video through the CPU. However, Ffmpeg provides also some cross-platform support to take advantage of specialized hardware for these tasks.
If you look at the Ffmpeg documentation, ongoing platform support can be summarized as follows:
| Linux | Windows | Apple | ||||||
|---|---|---|---|---|---|---|---|---|
| AMD | Intel | Nvidia | AMD | Intel | Nvidia | macOS | iOS | |
| NVENC/NVDEC/CUVID | F | F | ||||||
| Direct3D 11 | F | F | F | |||||
| Direct3D 9 | F | F | F | |||||
| Media Foundation | F | F | F | |||||
| OpenCL | F | F | F | F | F | F | F | |
| VAAPI | P | F | P | |||||
| VDPAU | P | F | ||||||
| VideoToolbox | F | F | ||||||
| Vulkan | F | F | F | F | F | F | ||
where P indicates partial support and F means full support.
Ultimately, the main objective of this project is to give Blender users the option to take advantage of the hardware-accelerated video decoding/encoding options offered by Ffmpeg, depending on the user’s platform and availability, by implementing the required architecture.
Due to the large number of options and time constraints, the scope of the project should be limited to achieving minimum support for as many platforms as possible.
In addition, there is access to two machines running Linux and Windows 11, one with a dual Intel/Nvidia graphics card an one with AMD, so the project should, for the time being, be limited to this configuration.
The hardware decoding process does not affect the final result, generating an output equivalent to that produced by a software decoder, beyond reducing the number of features supported in more complex codecs. Therefore, it should be used by default if the user’s platform allows it.
However, hardware encoders, despite the obvious advantages in terms of processing speed, tend to produce slightly inferior results to their software counterparts. Therefore, whether or not to use them should be the user’s choice at the time of use, so it will be necessary to modify the VSE UI to allow its activation and configuration according to the available options.
Project Schedule
- May, 8 - June, 1 (3 weeks): contact with mentor(s) and documentation period.
- June, 2 - Aug, 10 (10 weeks): coding, test and benchmarching.
- Aug, 11 - Aug, 24 (2 weeks): project documentation, last adjustments and bugfixing.
- Aug, 25 - Aug, 31 (1 week): final adjustments, if needed.
Bio
I am what you might call an atypical Spanish computer engineering student. I am currently 49 years old and, although I have worked all my life in other sectors not related to computer science, the year before the COVID pandemic I decided to start studying a subject I have been passionate about since I was a child.
The confinement, and a certain weariness with my job at that time, led me to make the decision to dedicate my time exclusively to finishing my degree, in order to change profession. On last year I finished last degree’s course and I am about to hand in the final project.
Unlike what I have observed in other contributors, I am not a regular user of Blender, but I know of its existence for a long time since I am passionate about computer graphics and video game development. Learning how to use it is yet another pending task in a long list.
I have been an advocate of free knowledge and free and open source software for as long as I have known of its existence and, in this regard, Blender seems to me a clear example of how this kind of projects can be viable and compete with proprietary solutions.
To this day I have collaborated with Blender less than I would like to, largely because of the respect it gives me as a beginner. Last year, in view of participating in the GSoC and to acquire some knowledge of the code, I sent my first pull request #119117. I have collaborated with other free and open source projects but only contributing translations or minor corrections.
I don’t have any experience using the Ffmpeg libraries either, so I have started a small personal project to learn how it works.
As far as languages are concerned, I am quite fluent with C/C++ and I can handle Python and Lua.