Possible bug when building headless mode in Blender 3.2.0 Alpha - With temporary solution

All,

I had a working headless Blender system running a beta version of 3.0.0 on Ubuntu. I was attempting to document how I got headless Blender installed and running, so I began the process on another Ubuntu system using the version available today, 2022-04-26:

Blender 3.2.0 Alpha
        build date: 2022-04-26
        build time: 04:26:18
        build commit date: 2022-04-25
        build commit time: 23:48
        build hash: 98ad294d1719
        build platform: Linux
        build type: Release

Working on the second computer, I was able to get the Blender source downloaded from git, linux_centos7_x86_64 from svn, ran make update, and everything was fine.

However, when I tried to run make headless, I would get (in part) the following errors:

Scanning dependencies of target blender
[100%] Building C object source/creator/CMakeFiles/blender.dir/buildinfo.c.o
[100%] Linking CXX executable ../../bin/blender
/usr/src/blender-git/lib/linux_centos7_x86_64/usd/lib/libusd_usd_m.a(glPlatformDebugWindowGLX.cpp.o):glPlatformDebugWindowGLX.cpp:function usdBlender__pxrReserved__::Garch_GLPlatformDebugWindow::Init(char const*, int, int, int): error: undefined reference to 'XOpenDisplay'
/usr/src/blender-git/lib/linux_centos7_x86_64/usd/lib/libusd_usd_m.a(glPlatformDebugWindowGLX.cpp.o):glPlatformDebugWindowGLX.cpp:function usdBlender__pxrReserved__::Garch_GLPlatformDebugWindow::Init(char const*, int, int, int): error: undefined reference to 'XCreateColormap'
/usr/src/blender-git/lib/linux_centos7_x86_64/usd/lib/libusd_usd_m.a(glPlatformDebugWindowGLX.cpp.o):glPlatformDebugWindowGLX.cpp:function usdBlender__pxrReserved__::Garch_GLPlatformDebugWindow::Init(char const*, int, int, int): error: undefined reference to 'XCreateWindow'
/usr/src/blender-git/lib/linux_centos7_x86_64/usd/lib/libusd_usd_m.a(glPlatformDebugWindowGLX.cpp.o):glPlatformDebugWindowGLX.cpp:function usdBlender__pxrReserved__::Garch_GLPlatformDebugWindow::Init(char const*, int, int, int): error: undefined reference to 'XStoreName'
...
/usr/src/blender-git/lib/linux_centos7_x86_64/usd/lib/libusd_usd_m.a(testGLContext.cpp.o):testGLContext.cpp:function usdBlender__pxrReserved__::Glf_TestGLContextPrivate::Glf_TestGLContextPrivate(usdBlender__pxrReserved__::Glf_TestGLContextPrivate const*): error: undefined reference to 'XCreateWindow'
collect2: error: ld returned 1 exit status
make[3]: *** [source/creator/CMakeFiles/blender.dir/build.make:518: bin/blender] Error 1
make[2]: *** [CMakeFiles/Makefile2:7867: source/creator/CMakeFiles/blender.dir/all] Error 2
make[1]: *** [Makefile:163: all] Error 2
make: *** [GNUmakefile:327: all] Error 2

After trying to redo the process a few times, I decided to wipe out my /usr/src/blender-git directory from the first (working, beta 3.0) computer and try again, assuming I had some weird configuration errors on the second system. Running through the process, I got the same errors on the previously working first computer.

I was able to run make and make bpy on both systems without issue. Looking at the log differences between make and make headless, I found that make headless had the following line in its output:
-- Disabling Ghost Wayland, X11, Input IME, and OpenXR

I tracked this text to blender/CMakeLists.txt line 839. Here there are the following set commands:

if(WITH_GHOST_SDL OR WITH_HEADLESS)
  message(STATUS "Disabling Ghost Wayland, X11, Input IME, and OpenXR")
  set(WITH_GHOST_WAYLAND OFF)
  set(WITH_GHOST_X11     OFF)
  set(WITH_X11_XINPUT    OFF)
  set(WITH_X11_XF86VMODE OFF)
  set(WITH_X11_XFIXES    OFF)
  set(WITH_X11_ALPHA     OFF)
  set(WITH_GHOST_XDND    OFF)
  set(WITH_INPUT_IME     OFF)
  set(WITH_XR_OPENXR     OFF)  
endif()

On a whim, I decided to try this (making a liar out of the message statement):

if(WITH_GHOST_SDL OR WITH_HEADLESS)
  message(STATUS "Disabling Ghost Wayland, X11, Input IME, and OpenXR")
  set(WITH_GHOST_WAYLAND ON)
  set(WITH_GHOST_X11     ON)
  set(WITH_X11_XINPUT    ON)
  set(WITH_X11_XF86VMODE ON)
  set(WITH_X11_XFIXES    ON)
  set(WITH_X11_ALPHA     ON)
  set(WITH_GHOST_XDND    OFF)
  set(WITH_INPUT_IME     OFF)
  set(WITH_XR_OPENXR     OFF)
endif()

This worked and did not require a display. The scripts I wrote with v3.0.0 worked (aside from the new requirement to use absolute paths when saving .blend files for some reason). This also worked on the new system.

I’m not sure if this is a bug, but it is weird. Has anyone else run into anything like this? I know this isn’t a permanent solution, since headless systems are unlikely to have X libraries. It seems like there is some part of the headless build process which (incorrectly?) requires those libraries even though that CMakeLists file disables them.

The problem is that we added Hydra support to the USD libraries, which also brings in X11.

@sybren I think it would make some sense to compile the USD libraries without X11, we don’t really need that debug window and various tests that use it. Being able to have a headless or Wayland only build is useful.

We could disable PXR_ENABLE_GL_SUPPORT, but OpenGL support would be useful if someone wants to work on a Hydra viewport in Blender. Perhaps ideally we patch in a new PXR_ENABLE_X11_SUPPORT and contribute it upstream.

1 Like

I agree compiling USD without X11 is a good idea. I see your point about having a Hydra viewport in Blender, and I agree this would be useful. I’d propose disabling PXR_ENABLE_GL_SUPPORT now, and patch in a new PXR_ENABLE_X11_SUPPORT when someone actually needs OpenGL without X11.

1 Like

This is handled in D14792: CMake: Reduce dependencies of USD to what’s actually needed by Blender.

Awesome, thanks everyone!