Blender V2.82 vs V3.0.1 GPU is semi detected | AWS Server Nvidia A10G gpu

Hi,

I’m having trouble with gpu rendering a blend file with python script. I have one aws g5 server with Nvidia Tesla A10g GPU. I installed Cuda 11.6 with driver 510 and later on downgraded to Cuda 11.4 with driver 470.

    print("devices: ",bpy.context.preferences.addons['cycles'].preferences.devices)
    print("get_devices(): ",bpy.context.preferences.addons['cycles'].preferences.get_devices())
    print("Compute device type: ",bpy.context.preferences.addons['cycles'].preferences.get_devices_for_type('CUDA'))
    print("devices: ",bpy.context.preferences.addons['cycles'].preferences.devices)
    print("get_devices(): ",bpy.context.preferences.addons['cycles'].preferences.get_devices())
    print("devices: ",bpy.context.preferences.addons['cycles'].preferences.devices)
    print("get_devices(): ",bpy.context.preferences.addons['cycles'].preferences.get_devices())

    print("DEVICE USAGE: ")
    for device in bpy.context.preferences.addons['cycles'].preferences.devices:
    print("  - " + device.name + " : " + str(device.use))
       print("  - " + device.name + " : " + str(device.use))

I tried to run my python script using blender 2.8.2 (sudo snap install blender --channel=2.82/stable --classic), it perfectly worked fine and printed this:

Blender 2.82 (sub 7) (hash 375c7dc4caf4 built 2020-03-12 05:30:40)
/run/user/1000/gvfs/ non-existent directory
found bundled python: /snap/blender/37/2.82/python
devices:  <bpy_collection[0], CyclesPreferences.devices>
get_devices():  ([<bpy_struct, CyclesDeviceSettings("NVIDIA A10G")>, <bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>], [<bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>])
Compute device type:  [<bpy_struct, CyclesDeviceSettings("NVIDIA A10G")>, <bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>]
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  ([<bpy_struct, CyclesDeviceSettings("NVIDIA A10G")>, <bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>], [<bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>])
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  ([<bpy_struct, CyclesDeviceSettings("NVIDIA A10G")>, <bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>], [<bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32")>])
DEVICE USAGE: 
  - NVIDIA A10G : True
  - AMD EPYC 7R32 : False
  - NVIDIA A10G : True

Blender quit

I tried to run my python script using blender 3.0.1 (sudo snap install blender --channel=3.0/stable --classic), it couldn’t detect the gpus in get_devices(). Notice that when I call devices field, at the beginning it gives bpy_collection[0] , no device at all. After I run get_devices_for_type('CUDA'), devices field gives bpy_collection[3].

Blender 3.0.1 (hash dc2d18018171 built 2022-01-26 01:01:59)
/run/user/1000/gvfs/ non-existent directory
devices:  <bpy_collection[0], CyclesPreferences.devices>
get_devices():  None
Compute device type:  [<bpy_struct, CyclesDeviceSettings("NVIDIA A10G") at 0x7f22249b6508>, <bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32") at 0x7f22249b6590>]
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  None
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  None
DEVICE USAGE: 
  - NVIDIA A10G : True
  - AMD EPYC 7R32 : False
  - NVIDIA A10G : True

Blender quit

I also tried lots of different versions like 3.1 beta and 3.2 with different cuda configurations. Also, I built two versions of blender from scratch with optix and cycles enabled (tried optix 7.3 and optix 7.4) and docker builts for blender 3.0.1. Couldn’t make it work.

Can you help me fix this? I don’t know why blender 2.82 works fine and versions greater than 3 don’t use the GPU. I have a blend file which I’m rendering for testing. However, it needs at least version 3.0.1. When I use blender 3.0.1, I check it with nvidia-smi. It doesn’t use any gpu.

My Configurations:

Machine: AWS G5.xlarge instance
OS: ubuntu 20.04
Blender Version: 3.0.1, 3.1, 3.2 (those versions don’t use GPU)
GPU: NVIDIA A10G
CUDA: Tried 11.4 (nvidia driver 470) and 11.6 (nvidia driver 510)

I also tried it in google colab with version 3.0.0. It won’t detect the GPU. Look at get_devices() → None

get_devices() is deprecated in 3.0 and will not return anything, I suggest to instead do:

cycles_prefs = bpy.context.preferences.addons['cycles'].preferences
cycles_prefs.refresh_devices()
for device in cycles_prefs.devices:
  ...

You can also try running Blender with --debug-cycles for more detailed output…

2 Likes

@brecht Thank you for the explanation

My code:

cycles_prefs = bpy.context.preferences.addons['cycles'].preferences
cycles_prefs.refresh_devices()
print("devices: ",bpy.context.preferences.addons['cycles'].preferences.devices)
print("get_devices(): ",bpy.context.preferences.addons['cycles'].preferences.get_devices())
print("Compute device type: ",bpy.context.preferences.addons['cycles'].preferences.get_devices_for_type('CUDA'))
print("devices: ",bpy.context.preferences.addons['cycles'].preferences.devices)
print("get_devices(): ",bpy.context.preferences.addons['cycles'].preferences.get_devices())
print("devices: ",bpy.context.preferences.addons['cycles'].preferences.devices)
print("get_devices(): ",bpy.context.preferences.addons['cycles'].preferences.get_devices())


print("DEVICE USAGE: ")
for device in bpy.context.preferences.addons['cycles'].preferences.devices:
   print("  - " + device.name + " : " + str(device.use))
   

blender 3.2 alpha says this:

blender 3.0.1 says this:

Blender 3.0.1 (hash dc2d18018171 built 2022-01-26 00:40:57)
I0209 11:50:38.020174  2422 device.cpp:45] CUEW initialization succeeded
I0209 11:50:38.020217  2422 device.cpp:47] Found precompiled kernels
I0209 11:50:38.046281  2422 device.cpp:194] Device has compute preemption or is not used for display.
I0209 11:50:38.046308  2422 device.cpp:197] Added device "NVIDIA A10G" with id "CUDA_NVIDIA A10G_0000:00:1e".
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  None
Compute device type:  [<bpy_struct, CyclesDeviceSettings("NVIDIA A10G") at 0x7fc237dfd288>, <bpy_struct, CyclesDeviceSettings("AMD EPYC 7R32") at 0x7fc237dfd310>]
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  None
devices:  <bpy_collection[3], CyclesPreferences.devices>
get_devices():  None
DEVICE USAGE: 
  - NVIDIA A10G : True
  - AMD EPYC 7R32 : False
  - NVIDIA A10G : True
Read blend: /home/ubuntu/RenderingData/a.blend
I0209 11:50:39.186801  2422 device_impl.cpp:71] Using AVX2 CPU kernels.
I0209 11:50:39.187258  2422 device_impl.cpp:71] Using AVX2 CPU kernels.

My driver:

Wed Feb  9 11:46:44 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.42.01    Driver Version: 470.42.01    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A10G         Off  | 00000000:00:1E.0 Off |                    0 |
|  0%   23C    P0    60W / 300W |      2MiB / 22731MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I tried with nvidia driver 510. It won’t work somehow. It is not related to this GPU I think since I tried it with tesla T4 and again I couldn’t make it work.

Do you set the device type to CUDA in the preferences, and to GPU compute in the scene?

@brecht I found the problem. After setting GPU settings, I open a blender file and it resets the settings or read the settings from blender file. So, my settings were cleared. Now I read the blender file first and then apply my settings. I works, what a silly mistake by me. Thank you so much for helping me.

Wed Feb 9 12:30:28 2022
±----------------------------------------------------------------------------+
| NVIDIA-SMI 470.42.01 Driver Version: 470.42.01 CUDA Version: 11.4 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A10G Off | 00000000:00:1E.0 Off | 0 |
| 0% 27C P0 99W / 300W | 2119MiB / 22731MiB | 99% Default |
| | | N/A |
±------------------------------±---------------------±---------------------+

±----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 9699 C /snap/blender/1653/blender 2117MiB |
±----------------------------------------------------------------------------+

@brecht do you know how we close the issue?

There’s nothing to close really, this forum is not an issue tracker.

1 Like