Build pyopenvdb as part of make deps

Patch D8123 aims to enable building of pyopenvdb with make deps. I created this thread for discussion about the implementation.

This is a first step towards integration of pyopenvdb into Blender, see design task T73201. Aim is to be able to import pyopenvdb in Blender Python Console. This does not include ability to modify Blender OpenVDB grids, only to have access to out-of-box pyopenvdb.

Info about pyopenvdb: https://www.openvdb.org/documentation/doxygen/python.html

More info in D8123.

Hi @LazyDodo, regarding your comment https://developer.blender.org/D8123#198025 while waiting comments from @brecht:

  • Splitting the Boost python build change from OpenVDB build change sounds good idea, I can split the patch this week. Also, I’ll try to add that python lib jam entry and see if it works.
  • I wonder if switching to dynamic build of OpenVDB requires more than -DOPENVDB_CORE_SHARED=ON …? Well, I’ll try and see, although I’m no expert here…

It does require a bit more, but not all that much, it’s pretty much update the builder to build dynamically and resolve any issues building/deploying blender (generally some small cmake changes on the blender side), don’t worry about it for windows, i’ll take care of it.

as for boost python i have found a nicer way to make the bjam user config file, if you don’t mind, i’ll take the initial stab at boost, since i think most of it will carry over nicely to the other platforms (while yours doesn’t necessarily transfer over to windows all that nicely)

@LazyDodo OK you go ahead with a new Boost patch.

From @brecht’s comments I get the feeling that moving to shared libs on unix side is not straightforward, so I’m not sure how to continue now.

What is it that you want to achieve exactly?

If it’s just to have the option to build pyopenvdb yourself and use it, then linking everything statically into the Python module will work fine.

If it’s to help us get towards shipping pyopenvdb with Blender, putting the changes behind a CMake option is fine and then we can finish remaining working including dynamic libraries at a later point.

We can also not use dynamic libraries, and statically link the Python module into Blender.

Personally, given you implied you wanted to eventually ship this, but are short on time i’d like to get it into a place where IF you start to work on this, it’s ready to go and you don’t have to mess about with getting the damn thing to build.

boost python isn’t that hard, but there’s a bunch of little details you have to get right or it’ll just silently not build it (same for boost::numpy) I have a patch ready for this that hides it behind a flag and seems to build for both linux and windows (can’t test on mac)

as far the ‘right’ solution for pyopenvdb i’d have to take a close look, i like the idea of linking it into blender though.

@brecht thanks for clarifying. Shipping pyopenvdb in Blender is my ultimate goal here. I was just confused about this static/dynamic aspect and if it needs to be resolved now or not.

@LazyDodo please link to the Boost patch once you have it somewhere. Then when Boost patch is ready, we can modify D8123 to include only changes for openvdb build (or is it better to create a new patch and discard D8123?). OK?

Ok, well any help getting this closer to shippable is welcome.

I think we will probably switch to dynamic linking on Linux and macOS at some point, also to help reduce compile times. But that’s a bigger project that I think would be more for 2.92, when I expect we do the next big libraries upgrade.

@LazyDodo Thanks for the Boost Python patch D8212. Great detective work with Boost, making it compile the Python modules, it is tricky! I got this patch to compile right after hardcoding -DWITH_BOOST_PYTHON=ON to cmake command for make deps in GNUmakefile (make deps does not use $(CMAKE_CONFIG_ARGS)).

Adding a switch defaulting it to off, and then hard-coding it to on somewhere else, seems… strange…

just turn it on in your CMakeCache.txt in the deps build folder if you want it on

Yes, but I was trying to make it work in one go.

I’ve updated D8123 for linux, will you continue from here with the windows side, or should I first fix the commented issues?

Current status summary, this work is on hold for now:

Thanks @LazyDodo and @brecht for your work! :smiley:

Current status: @LazyDodo has landed D8212 and D8282, and modified D8123 for Windows, so I suppose this patch is now about ready and waiting for @brecht. Thanks!

Hi,
Will pyopenvdb be by default in buildbot and official releases?
Or a user will need to compile his own build?

it’s on the module plans to at one point ship it, but afaik no-one is currently working on that, so i would not count on shipping this by default any time soon.