Does Blender use jemalloc and/or TBB?

Archive Blender Development
1

Hi all, hope everyone is staying safe and sound! I was wondering if Blender uses jemalloc or not? Doing a search in the project, I see this in a CMakeList.txt file:

# disable for now, but plan to support on all platforms eventually
option(WITH_MEM_JEMALLOC   "Enable malloc replacement (http://www.canonware.com/jemalloc)" ON)
mark_as_advanced(WITH_MEM_JEMALLOC)

and while the comment does say “disable for now”, the option looks like it’s enabled. However, I don’t see it referred to anywhere (not even as a linked library). So does it get used?

I have a similar question with TBB, in a CMakeList, I see:

# TBB malloc is only supported on for windows currently
if(WIN32)
  option(WITH_TBB_MALLOC_PROXY "Enable the TBB malloc replacement" ON)
endif()

However, I don’t see that being used anywhere either. Maybe I’m misunderstanding the syntax here?

Thanks in advance.

PS: That canonware.com URL is a dead link (and not the official homepage for jemalloc anyway).

2

I can only speak for windows given that is my platform, the WITH_MEM_JEMALLOC is a no-op for windows, jemalloc can’t hook into the CRT to capture the regular malloc calls and therefore is not used.

WITH_TBB_MALLOC_PROXY will use tbbmalloc only if WITH_TBB is enabled as well (given it needs the libs from the tbb folder)

Afaik windows is the only platform that supports tbbmalloc currently hence it being in a if(WIN32)

3

When building bpy as a python module on Debian -DWITH_MEM_JEMALLOC=OFF must be set. Not sure if that helps or not!

4

depends on the platform, like i said on windows that will make no difference on/off it’s ignored completely

5

Yeah, I’m on Windows. I believe TBB works on Linux and MacOS as well, I know that Maya and Houdini use it on those platforms. Can you explain what you mean about jemalloc not hooking into the CRT?

6

TBB works everywhere, and blender uses it on all platforms we ship, tbbmalloc however is currently used for blender on windows only.

to swap out a memory allocator (on windows, linux offers extra option that make this easier, there’s two options

  1. recompile all your code to call the new allocator rather than the standard allocator.

  2. All code calls malloc/free as usual, and at startup a hotpatch is made at the address of malloc to call the new allocator instead (whether that is tbbmalloc of jmemalloc)

1 is problematic since you cannot always rebuild all code, calling strdup for instance will internally call malloc, and you may or may not always have access to the sources, so recompilation is between tricky and impossible.

2 is an option, but jmemalloc doesn’t support this, tbbmalloc does

7

I probably need to study this some more, but my impression with jemalloc was that it was a drop in replacement and there wouldn’t need to be any code changes (in theory). I guess it isn’t that simple…

8

it’s drop in on linux, it’s quite a different story on windows

9

Curious, as on Arch I can build the bpy module with jemalloc with no problem.

10

At some point we might switch to using TBB malloc on all platforms. The differences (and outdated comments) are mainly due to historical reasons, not because one or the other is much better.

For shared libraries like the bpy module it’s best to disable custom allocators, since they can conflict with the allocator used by the host application. This type of problem would only happen at runtime and not necessarily in every application.

11

So what does that mean for dependencies of Blender? OpenVDB by default seems to use jemalloc on Linux. For example the Arch packaged binaries for OpenVDB are linked with it. Or is Blender linking statically to OpenVDB?

12

Blender official builds on Linux statically link both OpenVDB and jemalloc, and practically every library it uses. What Linux distributions do is up to them.