Real-time Compositor: Feedback and discussion

talking about the realtime feedback from grading with the compositor. With my window setup with composite nodes on top and viewport on the bottom; I can manipulate the image with realtime feedback. The feedback from the viewport compositor slows down when I enable gpu execution mode instead of tile or fullframe. I am not yet doing a final render at this point.

Before I noticed the performance hit when using the the image editor with (view node) but had not yet realized it slowed down the viewport compositor as well (not navigation but composite updates from modifying nodes) . The performance hit is not a huge hit, but a noticeable one. It is difference between realtime grading and not.

Your suggestions sound like like they will help the issue ; I just wanted to clarify.

uploaded a video for reference. might be hard to see but pay attention to the mouse after I change to gpu and start changing colors.

2 Likes

I can confirm this. Every setting on every node is laggy when it is adjusted. This happens in the viewport in GPU mode, not in Tiled or Fullframe either. It seems like you can’t just slide and adjust the settings of the node in real-time. Also, if you are rendering a high-resolution image, go to the Composite tab with the backdrop enabled, and trying to adjust the node settings gets REALLY worse, too much lag, practically unusable.

1 Like

@OmarEmaraDev Maybe it’s because the GPU compositor wants to update the image on the fly, even when you still have the mouse button pressed and are still adjusting the slide settings (like the color picker, for example). Perhaps, to remove the lag, the GPU compositor should wait for the mouse click to be released before applying the changes, allowing for the completion of adjustments.

There are two problems here.

First, the compositor superfluously executes even when its result is not viewed or used, as mentioned by Brecht above. This is not a problem for the CPU compositors because they execute in a different thread which has no effect on Blender drawing the UI. It is a problem for the GPU compositor because while it also executes in a different thread, we switch to a separate GPU context and submit work to that, so the UI drawing code and the compositor are fighting over the GPU. More on this in the second problem.

I will submit a fix for this problem soon.

Second, as I mentioned, the UI drawing code and the GPU compositor are fighting over the GPU. To fix this, there are two things we can do:

  • First, we can try to eliminate IO bottlenecks that makes this fighting worse. We already started doing that last week by introducing a complete cache for images. The same issue still exist at the output, where the compositor output is written to the viewer/composite images, so we still need to improve this. Further, some operations like OIDN denoising can make this worse, which I guess can be improved by using one of the GPU OIDN devices, but that seems unlikely for now.
  • Second, we can try to divide the compositor work into smaller chunks, which would allow GPU drivers to interleave the work better and make the UI stuttering less of an issue. Though this is just a theoretical idea that I have not confirmed or investigated yet.


@Emi_100 The idea you mentioned could be done, but it would make interactivity worst, which seems integral to compositing work.

7 Likes

@OmarEmaraDev It would be very nice if there was an option way to keep the Grease Pencil objects from being processed in the realtime viewport compositor. The lineart generally needs to keep its integrity throughout the compositing, although that is not always the case. For instance using kuwahara totally kills all the lineart GP look in the viewport.

I generally render the lineart as separate render passes, so it is not that hard to do in the actual compositing, but the realtime compositor does not handle render passes or aovs.

2 Likes

noticed this difference when using the laplace node between viewport comp and main comp (gpu exec). I figure it must be a precision difference , but brought it up just incase. While there differences between fullframe and gpu execution mode (for main comp), the difference is minimal compared to viewport comp

@OmarEmaraDev Disabling/deleting kuwahara in the compositor (Full Frame) crashes Blender in certain cases.

I was not able to produce the crash with a simple scene and I can’t share my project files but here is the crash log.

# Blender 4.1.0, Commit date: 2023-12-20 13:27, Hash 408bc52bedb7
bpy.ops.node.select(deselect_all=True, select_passthrough=True, location=(494, 771))  # Operator
bpy.ops.node.mute_toggle()  # Operator

# backtrace
Exception Record:

ExceptionCode         : EXCEPTION_ACCESS_VIOLATION
Exception Address     : 0x00007FF74D8B19A5
Exception Module      : blender.exe
Exception Flags       : 0x00000000
Exception Parameters  : 0x2
	Parameters[0] : 0x0000000000000000
	Parameters[1] : 0x0000024D2CBAB730


Stack trace:
blender.exe         :0x00007FF74D8B12E0  blender::compositor::convolve
blender.exe         :0x00007FF74D8B3D90  blender::compositor::GlareFogGlowOperation::generate_glare
blender.exe         :0x00007FF74D8B7CD0  blender::compositor::GlareBaseOperation::update_memory_buffer
blender.exe         :0x00007FF74D8A52B0  blender::compositor::NodeOperation::render
blender.exe         :0x00007FF74D8BC260  blender::compositor::ConstantFolder::fold_operation
blender.exe         :0x00007FF74D8BCB90  blender::compositor::ConstantFolder::try_fold_operations
blender.exe         :0x00007FF74D8BC530  blender::compositor::ConstantFolder::fold_operations
blender.exe         :0x00007FF74D8A1D70  blender::compositor::NodeOperationBuilder::convert_to_operations
blender.exe         :0x00007FF74D8A68A0  blender::compositor::ExecutionSystem::ExecutionSystem
blender.exe         :0x00007FF74D89CB00  COM_execute
blender.exe         :0x00007FF74DA97570  ntreeCompositExecTree
blender.exe         :0x00007FF74E0021D0  blender::ed::space_node::compo_startjob
blender.exe         :0x00007FF74D3BE160  do_job_thread
blender.exe         :0x00007FF74D3E6BB0  _ptw32_threadStart
ucrtbase.dll        :0x00007FFD824D6BB0  recalloc
KERNEL32.DLL        :0x00007FFD83EC53D0  BaseThreadInitThunk
ntdll.dll           :0x00007FFD85024830  RtlUserThreadStart


I can try making you a test build for now. Since I am still not sure about the side effects of this option or whether we want to have it at this point.

1 Like

Thanks for the report. I can reproduce the crash. Though it seems to happen for unconnected Glare node instead of Kuwahara. Probably related to @izo recent work on constant folding.

1 Like

Can you share this image?

Testing_Imagery/blue_bar_709.exr at main · sobotka/Testing_Imagery · GitHub

from troy’s repo

@kurk The crash should be fixed now.

@kurk @frogstomp I published test builds with an option to exempt grease pencil from from compositing. Should be available at Blender Builds - blender.org once the build-bot finishes creating the builds.

@dracoroot7 @Emi_100 The issue about slow updates when GPU execution is selected even when backdrop is disabled should be solved. Though if the backdrop is enabled, the same issue remains, and we will make a fix for that in the future.

@dracoroot7 I can’t reproduce the issue with the Laplacian, maybe try the latest daily version and if the issue still persists, report a bug.

2 Likes

Found the culprit . It seems to be the scale node set to rendersize and crop, but the fit and stretch does the same depending on the image.

here is the bug report #116438 - laplace diff between viewport comp and main comp - blender - Blender Projects

Hi,

Thanks for the implementation. That works with my initial tests. I am not seeing additional perf impact but needs further testing, obviously. I would be personally happy to have this option during the 4.1 testing period which I use regularly, and most of my recent works are all stylized stuff which rely on the realtime compositor and the GP tools/drawings.

The stack trace was helpful, thanks. I was able to reproduce the crash with this setup:


I’ll fix it within the next few days.

2 Likes

The updates are slow on grease pencil? They see to be fine on my end

image

@AdamEarle I can’t contextualize your comment, can you clarify?

Could someone please add something about the control of the focal point to the defocus node documentation?
Using Blender 4.0 on an ancient AMD CPU, if I select “Use Z-Buffer” it assumes that the focal point is at 10m. To adjust the focal point it is necessary to multiply the depth information that is connected to the Z input by a suitable constant, e.g. if a focal point at 2m is required use 0.2.
cheers Matt

The focus distance is defined by the Depth Of Field > Focus Distance in the active camera in the scene selected in the node. Camera objects have a default focus distance of 10m, that’s why you noted that.
See the documentation here:

https://docs.blender.org/manual/en/latest/compositing/types/filter/blur/defocus.html#camera-settings