GSoC 2024: Improve Distributed Rendering & Task Execution

Hi everyone!

My name is David Zhang, and I’ll be contributing to Flamenco over the summer, with improvements including allowing jobs to be paused and introducing sample-based distributed rendering of single images, which brings about better resource management and more flexible job scheduling.

For more details of implementation, see my original proposal

Synopsis
The objective of this project is to enhance the distributed rendering and task execution capabilities within Blender through several key improvements.
Firstly, we introduce the ability to pause jobs and submit them in a paused state, providing users with increased control over their rendering workflow and resource allocation. This feature will be particularly advantageous during peak usage periods or when prioritizing specific tasks.
Furthermore, we address the challenge of distributed rendering of single images by adopting a sample-based rendering approach. This method ensures more efficient utilization of computational resources across nodes, minimizing memory usage and avoiding artifacts caused by boundary dependencies.

Benefits
The benefits of these improvements to Blender and its community of artists are manifold. Artists will experience enhanced rendering efficiency and flexibility, enabling them to focus more on creativity and less on managing technical constraints. The introduction of job pausing and the ability to submit jobs in a paused state will allow for better resource management, reducing wait times and optimizing the use of available computational resources. The distributed rendering improvements will directly benefit artists working on complex scenes by reducing rendering times and improving image quality, without the need for extensive technical adjustments. These developments will also support future Blender enhancements by providing a more robust and flexible framework for distributed task execution and rendering.

Deliverables
The final deliverables of this project include

  1. New buttons and options for pausing and submitting jobs in a paused state in both the Manager web interface and the Flamenco Blender add-on. Pausing tasks are supported in the meantime.
  2. Improvements in distributed rendering that allows for distributed rendering of single images with minimized memory usage encapsulated in custom JOB_TYPE definitions and a Python merge script for efficient image processing.
8 Likes