[SOLVED] 2.90 headless rendering ignoring script selecting GPUs. Falls back on the CPU

Archive User Feedback
1

Hey guys, I’m at my wit’s end so if anyone has an idea, you would be a life saver. I am trying to render a scene using a Dockerized Blender in headless mode using the following command:

docker run --gpus all --ipc=host -v /mnt/vol_b/blender_input:/volume/blender_input -v /mnt/vol_b/blender_output:/volume/output nytimes/blender:2.90-gpu-ubuntu18.04 blender -b /volume/blender_input/water_sim.blend -E CYCLES -P /volume/blender_input/device_select.py -s 110 -e 380 -a --verbose -1 --log-level -1

The issue faced here is that Blender seems to ignore the script “device_select.py” which selects all of the GPUs and deactivate the CPUs.

The script device_select.py

import re
import bpy
import time
from datetime import datetime
scene = bpy.context.scene
scene.cycles.device = 'GPU'
print("~~~~~> scene device: {}".format(scene.cycles.device))
prefs = bpy.context.preferences
cprefs = prefs.addons['cycles'].preferences
cuda, opencl = cprefs.get_devices()
print("~~~~~> cprefs: {}".format(cprefs))
print("~~~~~> cuda devices: {}".format(cuda))
for compute_device_type in ('CUDA'):
  try:
    cprefs.compute_device_type = compute_device_type
    print('~~~~~> compute device found type set', compute_device_type)
    break
  except TypeError:
    pass
for device in cuda:
    print('~~~~~> Activating', device)
    device.use = (device.type != 'CPU')
    print("~~~~~> device {} usage status: {}".format(device, device.use))

for device in opencl:
    print('~~~~~> Deactivating', device)
    device.use = (device.type != 'CPU')
    print("~~~~~> device {} usage status: {}".format(device, device.use))

Here’s all the things I verified and confirmed are:

  • The cuda drivers are properly loaded in the docker container
  • I tested with a different project (simple default cube project) and I get the same results
  • I loaded and tried different versions of blender using the following containers nytimes/blender:2.83-gpu-ubuntu18.04, and nytimes/blender:2.91-gpu-ubuntu18.04 with the same problem
  • Blender runs fine otherwise and renders properly on CPU
  • The script print the modifications made before launching the animation render and the parameters are as expected

If you when to take a look at the dockerfile, you can find it at rd-blender-docker/Dockerfile at master · nytimes/rd-blender-docker · GitHub

Any ideas, even small ones would be appreciated.

Cheers,
Oli

2

Is your script doing what you think it’s doing? Testing in the python console, lets consider the first for loop:

>>> for compute_device_type in ('CUDA'):
...     cprefs.compute_device_type = compute_device_type
...     
Traceback (most recent call last):
  File "<blender_console>", line 2, in <module>
TypeError: bpy_struct: item.attr = val: enum "C" not found in ('NONE', 'CUDA', 'OPENCL')

>>> for compute_device_type in ('CUDA'):
...     print(compute_device_type)
...     
C
U
D
A

>>>

My take is that you are looping through all the characters in ‘CUDA’ and getting a TypeError exception each time, on which you pass. So nothing happens except the unraveling begins.

Perhaps try:

>>> for compute_device_type in ['CUDA']:
...     print(compute_device_type)
...     
CUDA

instead. In other words, list semantics instead of tuple.
[edit…]
That is, through the parentheses, you are inadvertently converting the string ‘CUDA’ into the four element tuple (‘C’, ‘U’, ‘D’, ‘A’) and iterating over that. See Built-in types: tuple at docs.python.org.

I’m a bit bemused by this first for loop. Why doesn’t a simple assignment turn the trick for you?
[/edit]
Good luck!

3

Well, your diagnosis is AFAIK correct, but the explaination is not. ('CUDA') isn’t a tuple, it’s just a value in parantheses.
>>> type( (“CUDA”) )
<class ‘str’>
But str is an iterable, iterating over it yields one character at a time.

("CUDA",)
on the other hand (note the trailing comma) is a tuple and would work just like a list.

1 Like
4

I want to thank both of you @grosgood and @ws01 for pinpointing the issue. I was indeed using a tuple initially until I removed all the value except for CUDA (which also explains why there is a loop). I feel really dumb for missing this. For anyone googling and stumbling on this post, I cleaned up the code and now use the following script:

import bpy
scene = bpy.context.scene
scene.cycles.device = 'GPU'
print("---> scene device: {}".format(scene.cycles.device))
prefs = bpy.context.preferences
cprefs = prefs.addons['cycles'].preferences
cuda, opencl = cprefs.get_devices()
print("---> cprefs: {}".format(cprefs))
print("---> cuda devices: {}".format(cuda))

cprefs.compute_device_type = 'CUDA'
print('---> compute device type set to', cprefs.compute_device_type)

for device in opencl:
    print('---> Deactivating', device)
    device.use = False
    print("---> device {} usage status: {}".format(device, device.use))

for device in cuda:
    print('---> Activating', device)
    device.use = (device.type != 'CPU')
    print("---> device {} usage status: {}".format(device, device.use))
5

Bingo! I stand corrected. And welcome to the board!

6

Thanks! :slight_smile: