Failing to build blender as python module

Archive Python API
1

I am trying to build blender as python module within Docker container running ubuntu 20.04 with python3.8.2. The build finishes successfully. However, once I try to load bpy from python console I get the following error:

/lib/x86_64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block

I am running the build with make bpy and WITH_MEM_JEMALLOC is set to OFF. I even modified the bpy_module.cmake file to set it to off outside of the ifs related to what operating system is runnig.

I would appreciate any help.

2

This doesn’t make sense as the only link to jemalloc is:

  if(WITH_MEM_JEMALLOC)
    target_link_libraries(${target} ${JEMALLOC_LIBRARIES})
  endif()

First check if libjemalloc.so.2 is linked to the bpy.so directly (perhaps it’s an indirect link causing the issue).

Run:

objdump -x bpy.so | grep NEEDED

If it’s not included, you’ll need to check indirect links.

If it’s included, do a verbose build, logging all commands to a file to see which command is linking jemalloc, then trace this back to relevant CMake file.

3

If your bpy module is dynamically linked to a library using jemalloc (perhaps openvdb) then you can get that error. Perhaps relevant: Problem with running blender as a python module

1 Like
4

I try the solution you have provided in the post you linked, however it crashed with another error.

Color management: using fallback mode for management
Color management: Error could not find role data role.
Color management: scene view "Filmic" not found, setting default "Standard".
blf_load_font_default: 'fonts' data path not found for 'droidsans.ttf', will not be able to display text
blf_load_font_default: 'fonts' data path not found for 'bmonofont-i18n.ttf', will not be able to display text
blf_load_font_default: 'fonts' data path not found for 'bmonofont-i18n.ttf', will not be able to display text
bpy: couldn't find 'scripts/modules', blender probably wont start.
Freestyle: couldn't find 'scripts/freestyle/modules', Freestyle won't work properly.
ModuleNotFoundError: No module named 'bpy_types'
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /root/blender/blender-2.83.1/source/blender/python/intern/bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /root/blender/blender-2.83.1/source/blender/python/intern/bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /root/blender/blender-2.83.1/source/blender/python/intern/bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /root/blender/blender-2.83.1/source/blender/python/intern/bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
ModuleNotFoundError: No module named 'bpy_types'
ERROR (bpy.rna): /root/blender/blender-2.83.1/source/blender/python/intern/bpy_rna.c:7263 pyrna_srna_ExternalType: failed to find 'bpy_types' module
F0701 13:26:38.103008  2121 utilities.cc:322] Check failed: !IsGoogleLoggingInitialized() You called InitGoogleLogging() twice!
*** Check failure stack trace: ***
    @     0x7fa533298933  google::LogMessage::Fail()
    @     0x7fa53329d6ec  google::LogMessage::SendToLog()
    @     0x7fa533298611  google::LogMessage::Flush()
    @     0x7fa533298e4f  google::LogMessageFatal::~LogMessageFatal()
    @     0x7fa5332a0a92  google::glog_internal_namespace_::InitGoogleLoggingUtilities()
    @     0x7fa532fafb19  libmv_initLogging
    @     0x7fa53261cc2e  main_python_enter
    @     0x7fa532af61a4  bpy_module_delay_init
    @     0x7fa532af6211  dealloc_obj_dealloc
    @           0x5c9b4e  PyDict_SetItemString
    @           0x548201  PyModule_AddObject
    @           0x67dbcf  _PyImport_LoadDynamicModuleWithSpec
    @           0x67f00d  (unknown)
    @           0x5c075c  (unknown)
    @           0x5f1344  PyVectorcall_Call
    @           0x56d0fd  _PyEval_EvalFrameDefault
    @           0x5654d2  _PyEval_EvalCodeWithName
    @           0x5f1bc5  _PyFunction_Vectorcall
    @           0x56c056  _PyEval_EvalFrameDefault
    @           0x5f19cb  _PyFunction_Vectorcall
    @           0x567325  _PyEval_EvalFrameDefault
    @           0x5f19cb  _PyFunction_Vectorcall
    @           0x5671fd  _PyEval_EvalFrameDefault
    @           0x5f19cb  _PyFunction_Vectorcall
    @           0x5671fd  _PyEval_EvalFrameDefault
    @           0x5f19cb  _PyFunction_Vectorcall
    @           0x5671fd  _PyEval_EvalFrameDefault
    @           0x5f19cb  _PyFunction_Vectorcall
    @           0x5ef1f1  (unknown)
    @           0x5ef674  _PyObject_CallMethodIdObjArgs
    @           0x54ee23  PyImport_ImportModuleLevelObject
    @           0x67e029  PyImport_ImportModuleLevel
Aborted (core dumped)

What is interesting is that in previous version, whenever I build blender there is the bpy.so alongside a blender folder with scripts and datafiles. However not there is just the bpy.so. Is it some change or am I missing something with the build? Thanks for the help, I really appreciate it :slight_smile:

5

If I run directly make bpy it is included. I will try to build it verbose and see where it is linked. If I remove the ifs and directly set WITH_MEM_JEMALLOC to OFF it is not present, but still got the same error. As I am not really familiar with cmake and all of that how should I check for indirect links? Thanks.

6

I managed to build it with

export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.2 python3

and

cmake ../blender-2.83.1     -DWITH_PYTHON_INSTALL=OFF     -DWITH_PYTHON_MODULE=ON -DWITH_AUDASPACE=OFF

Thanks both for the help!

7

Is the root problem down to a mix of memory allocators being used? Without that LD_PRELOAD, your system Python is using the standard libc malloc, while your Blender Python module is using jemalloc?

I think I’m hitting something similar in my builds of the Blender executable. Again, I am linking against the system Python, rather than one embedded in the executable as in the official Linux builds. In my case, it manifests itself when I try to use OSL: soon as I try to compile an OSL script, Blender segfaults. I fixed it by building without jemalloc.