Hardware accelerated video encoding/decoding for VSE
Name
José Vicente Barrachina Bosch
Contact
- e-mail: [email protected]
- chat: @jvteb:blender.org
- Blender repo
Synopsis
This project aims to incorporate into Blender’s VSE the necessary functionality to take advantage
of the hardware-accelerated video encoding/decoding mechanisms currently provided by the Ffmpeg libraries.
Benefits
The main and most notable benefits of using hardware acceleration for video encoding and/or decoding are the increased speed and performance. Additionally, using dedicated hardware frees up the CPU for processing other tasks to a large extent.
These advantages can make Blender’s VSE more attractive to users who need to perform video encoding, decoding and transcoding, thus allowing them to incorporate it into their workflow.
Deliverables
-
The code required to implement hardware-accelerated video encoding and decoding in VSE depending on the user’s platform and configuration, including changes to the UI necessary to customize the process.
-
Updates to the user documentation required to explain the usage of the new functionality.
-
Appropriate tests and benchmarks.
Project Details
The VSE (Video Sequence Editor) is one of Blender’s video edition tools, allowing the user to integrate this workflow with others such as 3D modeling. Thus, VSE allows users to avoid the need for external tools.
Currently, VSE uses the Ffmpeg libraries to encode and decode video through the CPU. However, Ffmpeg provides also some cross-platform support to take advantage of specialized hardware for these tasks.
If you look at the Ffmpeg documentation, ongoing platform support can be summarized as follows:
| Linux | Windows | Apple | ||||||
|---|---|---|---|---|---|---|---|---|
| AMD | Intel | Nvidia | AMD | Intel | Nvidia | macOS | iOS | |
| NVENC/NVDEC/CUVID | F | F | ||||||
| Direct3D 11 | F | F | F | |||||
| Direct3D 9 | F | F | F | |||||
| Media Foundation | F | F | F | |||||
| OpenCL | F | F | F | F | F | F | F | |
| VAAPI | P | F | P | |||||
| VDPAU | P | F | ||||||
| VideoToolbox | F | F | ||||||
| Vulkan | F | F | F | F | F | F | ||
where P indicates partial support and F means full support.
Ultimately, the main objective of this project is to give Blender users the option to take advantage of the hardware-accelerated video decoding/encoding options offered by Ffmpeg, depending on the user’s platform and availability, by implementing the required architecture.
Project Schedule
- May, 8 - June, 1 (3 weeks): contact with mentor(s) and documentation period.
- June, 2 - Aug, 10 (10 weeks): coding, test and benchmarching.
- Aug, 11 - Aug, 24 (2 weeks): project documentation, last adjustments and bugfixing.
- Aug, 25 - Aug, 31 (1 week): final adjustments, if needed.
Bio
I am what you might call an atypical Spanish computer engineering student. I am currently 49 years old and, although I have worked all my life in other sectors not related to computer science, the year before the COVID pandemic I decided to start studying a subject I have been passionate about since I was a child.
The confinement, and a certain weariness with my job at that time, led me to make the decision to dedicate my time exclusively to finishing my degree, in order to change profession. On last year I finished last degree’s course and I am about to hand in the final project.
Unlike what I have observed in other contributors, I am not a regular user of Blender, but I know of its existence for a long time since I am passionate about computer graphics and video game development. Learning how to use it is yet another pending task in a long list.
I have been an advocate of free knowledge and free and open source software for as long as I have known of its existence and, in this regard, Blender seems to me a clear example of how this kind of projects can be viable and compete with proprietary solutions.
To this day I have collaborated with Blender less than I would like to, largely because of the respect it gives me as a beginner. Last year, in view of participating in the GSoC and to acquire some knowledge of the code, I sent my first pull request #119117. I have collaborated with other
free and open source projects but only contributing translations or minor corrections.
I don’t have any experience using the Ffmpeg libraries either, so I have started a small personal project to learn how it works.
As far as languages are concerned, I am quite fluent with C/C++ and I can handle Python and Lua.