2025-05-26 Design Session: XR/VR

[dfelinto]

Attendees:

• Dalai Felinto
• Pablo Vazquez

Meeting to go over the existing XR experience in Blender for the future 5.0 release. Followed by mapping out mixed reality projects we would like to see implemented in Blender.

UX Improvements of Existing Tool

This is a proposal on how to best integrate Scene Inspection with the rest of Blender:

• XR implemented as a core feature (not “add-on”).
• XR is “on” if there is an OpenXR Runtime available.
  • It doesn’t rely on add-ons.
  • When it is “on”, it shows the XR tab on the viewport.
• Footer icon:
  • If OpenXR runtime available, show (outlined) icon on footer.
  • If OpenXR + HMD detected, show filled icon on footer.
  • If OpenXR runtime not available, don’t show the icon.
  • If click on icon open Preferences > Input > XR.
• Some XR Settings are moved to the preferences.

Preferences

• Action Maps
  • A new checkbox in the header replaces the existing “Use Controller Actions” option.

Viewport

The remaining existing options stay on the “XR” sidebar panel:

• Start XR Session (operator)
• Tracking (new sub-panel)
  • Position
  • Absolute
• View
• Landmarks
• Viewport Display (formely Viewport Feedback)

Viewport Display

The existing warning should only show when the offending feature is on. At the moment it is likely only the “Mirror VR session” option.

Extended Reality

When launching an XR session, users have access to different XR session types:

• Scene inspection (existing feature, 100% VR)
• Hologram (Mixed Reality)
• Reality Bubble (Mixed Reality)

The session specific settings are only visible when that option is selected. The Mixed Reality modes require a compatible HMD.

When working with those modes, the artist will still be using Blender keyboard/mouse as they would normally. No editing happens while immersed in VR.

Hologram

Allow artists to preview the 3D Scene on their desk. It renders the entire viewport transparent.

The grid (and axis) is displayed in a circle containing the entire bounding box of the scene.

Settings:

• Scene origin (in MR).
  • Operator to set the hologram origin:
    • At controller.
    • At hand.
• Scale (1:1, 1:10, …, Custom).
• Rotation.

Reality Bubble

The artist define a “bubble” within which, the real world is shown. Outside this the artist gets to see the scene being modelled.

This feature leverages the existing Landmarks feature, to save/load different vantage points.

Settings:

• Use existing Landmarks feature.
• Reality radius.
• Scale (1:1, 1:10, …, Custom).
• Rotation.
• XYZ (copy from 3D Cursor).

Location Scouting

The other project that needs to be designed is Location Scouting. The meeting didn’t go over it though.

[dfelinto]

Some notes after talking with Julian Eisel about this:

• To check if an OpenXR runtime is available may be costly. To be investigated whether this would impact the startup time.
• To check if the HMD is available may not be possible, but overall this is not super important for the proposal here (it would help to gray out the Launch XR session though).
• The Viewport Display (formerly Viewport Feedback) only impacts performance because of how it was implemented. Maybe things changed with recent GPU refactors, but to properly address this the viewport buffer should be preserved while the overlay buffer is refreshed and composed together.

[Jeroen-Bakker]

Great to see some ideas about it.

HMD availability is a continuously process. You can disconnect and reconnect and even switch systemids, there can be multiple devices attached (not likely, but annoying when not supported). Runtime is available when the software/drivers are installed.

What I can imagine is a delayed detection of the runtime (should be there or not, but can load drivers/resources etc) Not something you want to do blocking on startup, but having popped-up a few seconds later should be fine.

As an experiment I moved the OpenXR instance and systemid creation time to startup. But that resulted in not able to switch HDM. #139152 - WIP: OpenXR/Vulkan: Share GPU resources between UI and OpenXR - blender - Blender Projects I will not continue with this experiment as it requires a redesign of OpenXR integration like mentioned in the design session. Just leaving it here for visibility.

Perhaps the icon could be not active and when clicked a preferred device could be selected. Next click will start preferred device. Context menu activation could be used to change preferred device. of course somehow stored in a preference for next time Blender starts.

[dfelinto]

More specific proposal by Julian on how to tackle OpenXR detection:

1. The best way to tell if there’s an OpenXR runtime available is by calling xrCreateInstance(), which might do stuff like dynamically linking the runtime

2. But it seems like we can abuse xrEnumerateApiLayerProperties() to query if there’s an active runtime. It should return XR_ERROR_RUNTIME_UNAVAILABLE if not

3. We could still delay actual instance creation until we start the XR session that way, and avoid any other VR/XR overhead until actually using it.

---

And reply from Hogan:

“Yeah I think abusing xrEnumerateApiLayerProperties / xrEnumerateInstanceExtensionProperties might be the cheapest, especially if you’re polling often for runtime availability”