Process data with 25/50fps not working

Hey there.
We use Blender in our Virtual Production workflow to generate the CG Background. In order to generate a preview image, we send data with UDP (network) to Blender. The data contains e.g. transforms for objects.

The script is working properly and Blender is updating the scene, as it should be.

Our problem is the frame rate. We are not able to receive stable 25fps. We also need 50fps, but for now 25fps is the target.

I placed some timers down and monitored the behavior. As it turns out, TestRecvStart.modal is called every 0.0003s on our system. Thats more enough calls to update the scene in time. Though, as soon as we update an object with the received data, the update rate breaks down horribly.

e.g. bpy.data.objects["Cube"].location.x = 100 is enough to kill the update rate.

Here is a timing example when running the script:

processing data 0.0003105000005234615
processing data 0.0003127999998469022
processing data 0.0002902999995058053
processing data 0.00029910000012023374
processing data 0.00029240000003483146
processing data 0.00030640000022685854
processing data 0.0002868999999918742
processing data 0.000303199999507342
processing data 0.0002948000001197215
processing data 0.00030900000001565786
processing data 0.000330500000018219
processing data 0.00032869999995455146
processing data 0.00029400000039458973
processing data 0.0002950999996755854
processing data 0.00028620000011869706
processing data 0.0002916000003096997
processing data 0.0002936999999292311
processing data 0.00031229999967763433
processing data 0.0002851999997801613
processing data 0.00028870000005554175
processing data 0.0003265000004830654
processing data 0.0003274000000601518
processing data 0.00030869999955029925
processing data 0.0003105000005234615
processing data 0.0002986999998029205
processing data 0.0003057000003536814
processing data 0.00031399999988934724
processing data 0.0003116999996564118
processing data 0.0003219999998691492
processing data 0.00030900000001565786
processing data 0.0003600999998525367
processing data 0.00038580000000365544
processing data 0.00029830000039510196
Recieved Data 0
processing data 0.0064485000002605375        // BAD
processing data 0.000461399999949208
processing data 0.0016207999997277511       // BAD
processing data 0.000382299999728275
processing data 0.005071999999927357         // BAD
processing data 0.0004670000007536146   
processing data 0.017185099999551312         // BAD
processing data 0.0005203999999139342
processing data 0.01629789999969944             // BAD
Recieved Data 1
processing data 0.0008887000003596768
processing data 0.019568599999729486           // BAD
processing data 0.0005049000001235981
processing data 0.011451000000306522            // BAD
processing data 0.000482200000078592
processing data 0.01673879999998462              // BAD

Here is a reduzed version of the script:

bl_info = {
    "name": "Receiver",
    "description": "",
    "author": "",
    "version": (1, 0, 0),
    "blender": (2, 80, 0),
    "location": "View3D > Properties Panel > Testing",
    "category": "Object",
    'wiki_url': '',
    'tracker_url': ''
    }

import bpy
import time

from bpy.types import Operator

counter = 1

import queue
import threading

execution_queue = queue.Queue()

# This function can savely be called in another thread.
# The function will be executed when the timer runs the next time.
def push_data_to_main_thread(data):
    execution_queue.put(data)

def process_queued_data():
    while not execution_queue.empty():
        data = execution_queue.get()
        print("Recieved Data", data)
        bpy.data.objects["Camera"].location.x = data /10

def thread_function():
    counter = 0
    while(counter < 20):
        time.sleep(0.04)
        push_data_to_main_thread(counter)
        counter += 1 
        #print("Thread func data", counter)

class TestReceiver():
    timelast1 = time.time()
    timelast = time.time()
    jsonProcessingStart = time.time()

    def __init__(self):
        self.thread = threading.Thread(target=thread_function)
        self.thread.start()

    def __del__(self):
        self.thread.join()

    def run(self):
        timeCurrent = time.perf_counter()
        timeDelta = timeCurrent - self.timelast
        self.timelast = timeCurrent
        print("processing data", timeDelta)

        process_queued_data()
     

# create UI and controls
class TestRecvStart(bpy.types.Operator):
    bl_idname = "wm.testing_start"
    bl_label = "Testing Start"
    bl_description = ""
    bl_options = {'REGISTER'}

    enabled = False
    receiver = None
    timer = None

    def modal(self, context, event):
        if not __class__.enabled:
            return self.cancel(context)

        if event.type == 'TIMER':
            self.receiver.run()

        return {'PASS_THROUGH'}


    def execute(self, context):
        __class__.enabled = True
        self.receiver = TestReceiver()

        context.window_manager.modal_handler_add(self)
        self.timer = context.window_manager.event_timer_add(
                                                      1/10000,
                                                     window=context.window)

        return {'RUNNING_MODAL'}

    def cancel(self, context):
        __class__.enabled = False
        context.window_manager.event_timer_remove(self.timer)

        del self.receiver

        return {'CANCELLED'}

    @classmethod
    def disable(cls):
        cls.enabled = False

class TestRecvStop(bpy.types.Operator):
    bl_idname = "wm.testing_stop"
    bl_label = "Testing Stop"
    bl_description = ""
    bl_options = {'REGISTER'}

    def execute(self, context):
        TestRecvStart.disable()
        return {'FINISHED'}


class VIEW3D_PT_TestPanel(bpy.types.Panel):
    bl_space_type = "VIEW_3D"
    bl_region_type = "UI"
    bl_label = "TestingUi"
    bl_category = "TestingCat"

    def draw(self, context):
        layout = self.layout
        if(TestRecvStart.enabled):
            layout.operator("wm.testing_stop", text="Stop", icon='PAUSE')
        else:
            layout.operator("wm.testing_start", text="Start", icon='PLAY')


registerClasses = [
    TestRecvStop,
    TestRecvStart,
    VIEW3D_PT_TestPanel
]

def register():
    for cls in registerClasses:
        bpy.utils.register_class(cls)

def unregister():
    for cls in reversed(registerClasses):
        bpy.utils.unregister_class(cls)

if __name__ == "__main__":
    register()

Setting the update time to 1/1000 already causes weird timings, even without receiving data.

I also tried to use bpy.app.timers to create the update loop in the main thread, but those timers min time is 16ms (probably bound to monitor refresh rate)

Any idea on how to get a proper update loop on the main thread?

1 Like

Interesting… In couple of days I am going to write something similar in my little project, I am also going to update object by receiving some data from TCP connection. In first attempt I am going to do in loop request- answer approch: send me next frame pls - here it is. If it is going to be very slow I will try sending batches of frames,
I will let you know If I have same issues.

do you think it the issue can be that you are competing for main thread time?:

1. blender calls you timer - 1ms
2. blender updates scene - many more ms
3. blender calls you timer again but much later that specified

In literature of real time computing this property is called hard or soft

BTW, are you using any sensible UDP, TCP framework? So far I am using pure sockets, it is not ideal…

bwt2, just checked. I wonder how much will it improve performance:

>>> timeit.timeit(stmt='x', setup='x=1', number=100_000)
0.0017934000006789574

>>> timeit.timeit(stmt='x=2', setup='x=1', number=100_000)
0.0018512000006012386

>>> timeit.timeit(stmt='bpy.data.objects["Cube"].location.x', setup='import bpy', number=100_000)
0.05304510000132723

>>> timeit.timeit(stmt='bpy.data.objects["Cube"].location.x = 100', setup='import bpy', number=100_000)
0.12399229999937234

>>> timeit.timeit(stmt='c.location.x', setup='import bpy; c= bpy.data.objects["Cube"]', number=100_000)
0.0207615999988775

>>> timeit.timeit(stmt='c.location.x = 100', setup='import bpy; c= bpy.data.objects["Cube"]', number=100_000)
0.10802150000017718

>>> timeit.timeit(stmt='loc.x', setup='import bpy; loc= bpy.data.objects["Cube"].location', number=100_000)
0.006847599999673548

>>> timeit.timeit(stmt='loc.x = 100', setup='import bpy; loc= bpy.data.objects["Cube"].location', number=100_000)
0.06716720000076748

In a meantime you can try looking at GitHub - ubisoft/mixer: Add-on for real-time collaboration in Blender.
They also need to have some kind of synchronization, but I counld not find it easily.

We use default python sockets to receive the data. That is not the problem.
Updating the location of an object is no problem as well (0.023ms on our system). Keep in mind that timeit returns the accumulated time, NOT the average (the docs do not mention that…).

The problem is the timer function we use on the main thread that breaks after an object was changed.
The timer function itself seems to be broken. When setting other time values for the timer (not 1/10000), the result is a mess.

I also don’t think that the Mixer plugin is any help here. Its only for realtime collaboration when building the scene. If the frame is 5ms off, nobody cares. We need it for realtime compositing and thus require a steady framerate.

So here is what I am doing. I am using frames as my step counter. I am managing ~40 frames with this crude first implementation. Node that on the end of this video I am doing playback only only from cache, the is no newtork communication.

def on_frame_change():
    if curren_frame in cache:
        update_objects()
   else:
       request_more_frames() # request n next frames
       playback_stop()

def handle_data_timer(): # every 0.1 seconds
    if incoming_data:
        cache += incoming data
        playback_resume()

I am not sure how much it will help in your case. I am doing mainly mesh operations with vertices. But right now I need to take a step back and think about my design.

We can’t use the frame_change handlers, as we can’t have a running animation

Would this help: run Blender in batch (“background”) mode, with a Python script in control of the main thread, telling Blender what to do and when.