Blender 2.8: “Unable to open a display” by the rendering on the background (CYCLES)

[PaulMelis]

Ah! Just tried the latest official binaries (2.80.75) and with those I also get the error you’re seeing! So it indeed appears that when the compositing step starts the text overlay is rendered (directly after rendering is Finished) an attempt is made to open a display, even when running with -b.

Quick guess after looking at the file: there’s two scenes in there, Scene and TEXT overlay. Both have their own setting of the renderer to use (set to Eevee in the file). When you use -E CYCLES from the command line it only sets the renderer on the current scene, leaving the other set to Eevee, which won’t be able to render headless as its directly tied to OpenGL.

Haven’t tried a workaround, but the above looks to be the issue.

[Raimund58]

Hi Paul,
Coming from the Sheepit Renderfarm we get the same error. Unable to open a display
Also with CYCLES projects and also with Blender 2.83.3.
I couldn’t find the settings of “the renderer to use” that you described. Could you please describe how you found it? Because we need to detect it, so we can block these kind of projects (that somehow use eevee) on headless machines.
Also sorry for reviving such an old thread but I detected these errors quite a lot recently.

@Devs: Could you please add something to the log output to see what is causing the Unable to open a display? I mean, the frame is finished and for saving Blender needs EEVEE? I’ve run --debug-all but I have still no idea why it is happening because Blender doesn’t tell anything. See the appended log
log_2.90.txt (1.1 MB) log_2.83.3.txt (1.1 MB)
Maybe somebody else has an idea?

[Raimund58]

I now use the workaround with xvfb.
sudo apt-get install xvfb
xvfb-run -a "my command"
But the core problem persists. Blender still seems to use EEVEE somehow after the frame is finished with CYCLES. And I can’t find the reason…
@brecht Should I open a bug report for that?

EDIT: The workaround is working for the strange cycles/eevee thing but is ■■■■■■■ up normal eevee projects. A frame that takes me locally 2 seconds takes 4,5 minutes on Colab.

[Raimund58]

You can test a headless machine really fast in Colab if you want to. https://colab.research.google.com/gist/Raimund58/88061f785ea2563d345914d196c15b87/blender_script_for_google_colab_using_the_gpu.ipynb

At the bottom is also the mentioned Unable to open a display

[PaulMelis]

@Raimund58

I couldn’t find the settings of “the renderer to use” that you described. Could you please describe how you found it? Because we need to detect it, so we can block these kind of projects (that somehow use eevee) on headless machines.

That would be bpy.data.scenes['Scene'].render.engine. Note that the value can differ for different scenes in the same Blender file.

@Raimund58

You can test a headless machine really fast in Colab if you want to. https://colab.research.google.com/gist/Raimund58/88061f785ea2563d345914d196c15b87/blender_script_for_google_colab_using_the_gpu.ipynb

At the bottom is also the mentioned Unable to open a display

Well, I can’t open the scene file through Google Drive it seems. Do you have it available separately?

[Raimund58]

@PaulMelis

That would be bpy.data.scenes['Scene'].render.engine . Note that the value can differ for different scenes in the same Blender file.

Sorry, I’m less of beginner level of a programmer. How do I list the scenes? I only get:

import bpy
bpy.data.scenes[‘Scene’].render.engine
‘CYCLES’

@PaulMelis

Well, I can’t open the scene file through Google Drive it seems. Do you have it available separately?

Printer_Animatic_szene1_200724_1530.blend

[PaulMelis]

@Raimund58

@PaulMelis

That would be bpy.data.scenes['Scene'].render.engine . Note that the value can differ for different scenes in the same Blender file.

Sorry, I’m less of beginner level of a programmer. How do I list the scenes? I only get:

import bpy
bpy.data.scenes[‘Scene’].render.engine
‘CYCLES’

>>> for scene in bpy.data.scenes: print(scene, scene.render.engine)
<bpy_struct, Scene("Scene")> CYCLES

[Raimund58]

So, we only got one scene and that is using CYCLES. So, it is a bug then?

[PaulMelis]

I’m not sure. I can’t reproduce the issue here with 2.83.3 running under Xvfb and with your setgpu.py script:

$ Xvfb :1 -screen 0 1920x1080x24 &
$ cat setgpu.py 
import re
import bpy
scene = bpy.context.scene
scene.cycles.device = 'GPU'
prefs = bpy.context.preferences
prefs.addons['cycles'].preferences.get_devices()
cprefs = prefs.addons['cycles'].preferences
print(cprefs)
# Attempt to set GPU device types if available
for compute_device_type in ('CUDA', 'OPENCL', 'NONE'):
    try:
        cprefs.compute_device_type = compute_device_type
        print('Device found',compute_device_type)
        break
    except TypeError:
        pass
#for scene in bpy.data.scenes:
#    scene.render.tile_x = 64
#    scene.render.tile_y = 64
# Enable all CPU and GPU devices
for device in cprefs.devices:
    if not re.match('intel', device.name, re.I):
        print('Activating',device)
        device.use = True
    else:
        device.use = True
$ DISPLAY=:1 blender -b ~/Printer_Animatic_szene1_200724_1530.blend -P setgpu.py -o doh.png -f 1
Blender 2.83.3 (hash 353e5bd7493e built 2020-07-27 00:47:14)
Read prefs: /home/paulm/.config/blender/2.83/config/userpref.blend
Read blend: /home/paulm/Printer_Animatic_szene1_200724_1530.blend
<bpy_struct, CyclesPreferences at 0x7f40831dbb28>
Device found CUDA
Activating <bpy_struct, CyclesDeviceSettings("GeForce GTX TITAN (Display)")>
...
Saved: 'doh.png0001.png'
 Time: 00:48.16 (Saving: 00:02.77)

Blender quit

[Raimund58]

@PaulMelis
Of cause you can’t reproduce it. With Xvfb your machine is not headless. But this im my whole point.
My machine is headless and should be fine rendering CYCLES. But it is not…

[brecht]

If the scene contains grease pencil objects, does require OpenGL for rendering even if they are composited over a Cycles render.

[Raimund58]

First of all, thank you for your reply.

@brecht

If the scene contains grease pencil objects, does require OpenGL for rendering even if they are composited over a Cycles render.

Okay, so how can we detect all the things that don’t work on headless machines?
Because as I said, I’m coming from a renderfarm and need to stop all unrenderable projects.

Also, for my project I couldn’t find any grease pencil things but I don’t exactly know what to look for.
I wish I were better at Blender and coding

[PaulMelis]

@Raimund58

Okay, so how can we detect all the things that don’t work on headless machines?

The term “headless” is a bit overloaded, so what exactly do you mean by that?

• A machine without a GPU
• A machine with a GPU but without a display attached
• A machine with a GPU, but that does not (or cannot) run an X server

Instead of blocking Blender scenes that contain display-related items, why not set up the renderfarm nodes to provide a software OpenGL implementation (i.e. Mesa) and X server? There used to be a blender-softwaregl binary included in the official binaries, precisely for the former purpose, but I don’t know how well it interacts with EEVEE (although I suspect Mesa supports a new enough OpenGL profile).

[Raimund58]

@PaulMelis

The term “headless” is a bit overloaded, so what exactly do you mean by that?

• A machine with a GPU but without a display attached
  That is exactly what I meant

@PaulMelis

A machine with a GPU, but that does not (or cannot) run an X server

Actually I never heard about an X server. Something I need to research…

@PaulMelis

why not set up the renderfarm nodes to provide a software OpenGL implementation (i.e. Mesa)?

Well, our “renderfarm nodes” are other peoples PCs. https://www.sheepit-renderfarm.com/
I’m not sure about things we need to install on their system. But we could put it in our tutorial section.
Right now is Xvfb causing a few problems. EEVEE frames that takes 2 seconds locally takes 4,5 minutes on Colab (on a Nvidia P100)

[Raimund58]

xvfb is also producing faulty/wrong frames. Any idea what could produce better results?

[PaulMelis]

@Raimund58

xvfb is also producing faulty/wrong frames. Any idea what could produce better results?

Looks like there are some issues specifically when rendering with a P100 GPU. And the Radeon RX 580. I’m surprised you even get consistent results over different brands of GPUs, as it will mix CUDA and OpenCL renders in the same animation, where the render implementations differ AFAIK. But I’m no expert in how reproducable Blender GPU renders are anyway over the set of all CPU model + GPU model + GPU drivers + OS + … combinations. It’s certainly not the easiest task do consistent rendering using a set of heterogeneous nodes with GPUs.

[PaulMelis]

@Raimund58

EEVEE frames that takes 2 seconds locally takes 4,5 minutes on Colab (on a Nvidia P100)

That could easily be caused by software OpenGL rendering being used, instead of the hardware-acceleration of the GPU.

[Raimund58]

@PaulMelis

That could easily be caused by software OpenGL rendering being used, instead of the hardware-acceleration of the GPU.

That could explain it. (the CPU is pretty weak)
But what would be another option? If Xvfb is only using the CPU, how can I use the GPU?
Another software?

[PaulMelis]

EEVEE is based on the OpenGL API for rendering. The only way to do hardware-accelerated OpenGL rendering of the type that Blender currently needs on an NVIDIA GPU (on Linux) is by having an X server running. There simply is no other way to get the required hardware-accelerated OpenGL context.

The only way to do accelerated OpenGL rendering without an X server (or other windowing system) is through EGL. There are some developments in this area for Blender (e.g. https://developer.blender.org/T54638 and https://developer.blender.org/D6585) but it doesn’t seem production ready yet.

So for EEVEE there currently is no good workaround other than using software-based OpenGL rendering.

[Raimund58]

Anyway, thank you for your hints