Plugin Hot-Reload by Cleaning sys.modules?

Archive Python API
2

I also ran into issues with the “standard” way to reload addons in the past. My vscode extension (which I don’t have the time to support anymore unfortunately…) uses the approach you describe here. It seems to work well in practice, at least I did not have any issues with it.

See blender_vscode/addon_update.py at master · JacquesLucke/blender_vscode · GitHub.

3 Likes
3

Thanks for the info

If this technique is not causing issues at first glance
then we should probably cleanse sys.modules within PREFERENCES_OT_addon_disable

4

One thing I find is it’s easier if your addon is all in one file. I have seen addons split across, say, half a dozen files with only a few hundred lines in each, and I feel this is excessive. I see no big deal with a single addon file being, say, a few thousand lines in size – it’s still faster to load than multiple split files that total to the same size

And this way, Blender can reload the addon more reliably without quitting.

1 Like
5

Simplified version of cleanse module function, works great so far tested on multiple projects

def cleanse_modules():
    """search for your plugin modules in blender python sys.modules and remove them"""

    for module_name in sorted(sys.modules.keys()):
        if module_name.startswith(__name__):
            del sys.modules[module_name]
6

Carreful, modules in sys might not be in the correct order, it might create dependencies issues when deleting them, i remember i had issues… that’s why i sorted them in the more “complex” function

1 Like
7

Ah right :man_facepalming:, I have to edit it

8

Related insights (Re)Importing in python - don't touch sys.modules

1 Like
9

Thanks for that link.

The moral of the story is, there is no fully reliable way to dynamically reload a module in Python. importlib.reload() is the least bad way, since that updates the module object in place, but

  • names which have disappeared from the module will not be deleted.
  • references of the form from module import name will not have those name references updated.

Seems like the first point is a bug that could be fixed, but the second one is a pretty fundamental consequence of the design of Python itself: every name is a variable, and every form of name definition (including import statements) is just another kind of variable assignment.

1 Like
10

Right, something fundamental to Python itself

11

I think maybe we can think about this from another direction
and prevent the issue from happening in first place

that is we ditch the import statement and load the file ourselves
can help: https://stackoverflow.com/a/67692/8094047

I still have to try this later

12

Question: why the cleansing is not done upon unreg automatically by blender?

that is we ditch the import statement and load the file ourselves

IMO, that’s like re-inventing the wheel
cleansing is much cleaner

1 Like
13

My caveman solution: Restart Blender and recover the session.
I put this in an operator and bound it to F5, so I can restart Blender quickly with one button press.

Of course you lose all addon state that’s not saved in properties, but for me it achieves the goal to continue from where I was with updated addon code. E.g. reproduce a bug, fix it, press F5 to load the new code, test again to confirm the fix is working.

You can find the code here: https://github.com/LuxCoreRender/BlendLuxCore/blob/d435660c798451c0de3f023d07b0721ddbeccdf5/operators/debug.py#L23

class LUXCORE_OT_debug_restart(bpy.types.Operator):
    bl_idname = "luxcore.debug_restart"
    bl_label = "LuxCore Debug Restart"
    bl_description = "Restart Blender and recover session"

    def execute(self, context):
        blender_exe = bpy.app.binary_path
        import subprocess
        subprocess.Popen([blender_exe, "-con", "--python-expr", "import bpy; bpy.ops.wm.recover_last_session()"])
        bpy.ops.wm.quit_blender()
        return {"FINISHED"}
3 Likes
14

right, cleansing is much simpler

15

interesting solution :ok_hand:

16

I have to admit I was wrong,
messing with sys.modules is not a good idea,
it can cause crashes and makes blender test suite fail (if you get your plugin added to Blender),
a better solution might be to use importlib.reload

17

This require to reload modules one by one
too much work depending on how large your project is IMO

it can cause crashes and makes blender test suite fail (if you get your plugin added to Blender),

Got ±9 thousand users that have been using a plugin with this hot reload implementation and never got any crash report :thinking: did you had issues?

18

I have been using sys modules my self for a long time too,
the problem is that messing with sys.modules is basically undefined behavior,
and it fails the blender test suite, there are so many things wrong about it,

I agree with this answer now Plugin Hot-Reload by Cleaning sys.modules? - #9 by ldo

we can automate the import lib reload just like we do a for loop for modifying sys modules

19

also a bit more info, it seems the “correct” way to modify sys.modules is already provided by

bpy.utils.register_submodule_factory

    # Snippet
    def unregister():
        from sys import modules
        for mod in reversed(submodules):
            mod.unregister()
            name = mod.__name__
            delattr(module, name.partition(".")[2])
            del modules[name]
        submodules.clear()

    return register, unregister
20

Hmmm
and how do you make sure that the modules are reloaded in the correct order?

see the initial post

But this method can cause issues as if the modules we want to reload contains other modules, those modules are not reloaded recursively, so depending on the exact code you can wind up in a rather broken state

1 Like
21

Ok my bad, i was deleting the modules twice one with my own code and one time with blender’s helper function, good to know that blender’s helper function already does this

1 Like