VR Scene Inspection Feedback

Other Topics User Feedback
#1

The 2.83 release will be the first Blender version to include core VR support. This is just the first milestone named VR Scene Inspection, which aims to lay foundations for further, more feature rich use-cases.
Purpose of this thread is to gather user feedback on the VR Scene Inspection milestone.

See Also

Release Notes: https://wiki.blender.org/wiki/Reference/Release_Notes/2.83/Virtual_Reality
User Manual: https://docs.blender.org/manual/en/dev/getting_started/configuration/hardware.html#head-mounted-displays-virtual-reality
Milestone design task: https://developer.blender.org/T71347

Known Issues

Here are some known issues for the first milestone (order does not represent priority):

  • Limited usefulness of VR Landmarks (VR Scene Inspection Add-on)
    Due to unfortunate timing, it was not possible to get the VR landmarks polished to make them work well in all supposed use-cases. There are some issues with changing the active landmark remaining, and the creation of new ones needs more features (e.g. creating custom landmarks from current viewer position).
    Status: New Add-on features are in development. Fixes can be backported to 2.83 but general improvements will have to go into 2.90.
  • Base poses should probably be sent to OpenXR
    Currently, Blender takes the position and rotation deltas of the eyes and applies them to its base pose (i.e. the active landmark). There appears to be a way to let OpenXR runtimes know about that so the base pose can be applied there. We weren’t able to get that to work. (See next point)
  • Height positioning of base pose needs to be better
    Ideally, when opening a VR view, the eye distance to the physical floor would match the virtual one. It doesn’t appear like OpenXR deals with this sufficiently. If the previous point is addressed this might be solved, but it is a big unknown.
    Status: Help needed. Someone with experience with the OpenXR specification or other VR platforms is needed to find a solution.
  • VR Camera feedback is performance hungry
    The VR Camera feedback option allows drawing a camera-shaped gizmo in regular 3D views, indicating the current position and rotation of the VR viewer. That currently requires continuously redrawing the 3D Views it is in, because gizmos are drawn together with the viewport. A patch to address this was written, but rejected because of design concerns and conflicting design changes in master.
    Status: This could be addressed soon as part of regular development. It’s not a high priority but doable without much work.
  • VR mirror is performance hungry
    The first milestone requires a fully interactive mirror of the VR view in a regular 3D View. The only way to do this without bigger changes is by re-rendering the entire viewport, effectively as a 3rd eye. It’s tricky to improve this, but there are some ideas.
    Status: No concrete plans to address this. Add-on UI shows a warning now.
  • VR mirror could share more settings with VR view
    If a regular 3D View serves as VR mirror, it currently only shares shading settings with the VR view. It could share more settings, like the active View Layer or clipping distances.
    Status: Can be improved as regular developemnt
  • DirectX compatibility layer not working on some AMD cards.
    The compatibility layer simply fails on some cards/drivers, which is indicated with a pink screen.
    Status: Needs investigation - cause is unclear and no developer is available to investigate. Help welcome!
3 Likes
#2

The only issue that I had with this mode was not have a custom “walk mode” that I can use in VR like a debug camera mode to move free by the 3D world. You can use the classic walk mode, but the movement is with the position of the camera in the3d viewport, not in the VR scene.

#3

@sebastian_k , I’m continuing here from https://developer.blender.org/T71347. I tried your version of the addon:

  • Custom pose: very good! That is something I was also thinking about. It seems to affect the base point, not the VR camera, just as with landmarks as discussed above. A good thing to have would be a switch: “affect Camera / affect base point”.
  • In some cases it is useful to have the camera update live while you change the coordinates, in other cases you might want to input exact coordinates and then press Enter, and only then the camera would jump there. So we’d have two modes: “live update” like you have now, and “update when clicking the apply button”.
  • Displaying Landmarks in the viewport: yes this is also helpful, I’d probably constantly leave it on. Maybe also display the name of the landmark in the viewport?
#4

After using it a little longer, the problems I’ve encountered

  • If you use eevee it has a reduced color depth
  • Positioning yourself on the scene is very complicated, moving through it even more. For example, if I put a camera at 1.60 meters high and run the VR, with RiftS, the height I can have may be 1 meter more. I think the system is quite opaque in how it works.
#5

I think I found something about the AMD cards issue.
Looking for the word “pink” @ developer blender: D6190 (can’t add link) made me look for a interop extension, that seems to be WGL_NV_DX_interop or WGL_NV_DX_interop2.

This could be related to either:


or
https://community.amd.com/thread/166698
I’m new here, so I hope this helps in any way.

#6

SteamVR finally rolled out their first implementation of OpenXR support in Beta, so this should open up a lot more users, though their implementation is limited to DirectX for the renderer.

There’s a conversation with a developer at Valve asking about enabling OpenGL support for their implementation.

Does the D3D wrapper get used for both Oculus and Windows MR when using Scene Inspection, or is it only used for Windows MR? The pink hmd issue (which, unfortunately as an AMD user I am getting) might not be as much of a problem if it can go through OpenGL natively (assuming Valve implements it on their end). And then I think the SteamVR->WindowsMR bridge MIGHT handle converting it over for Windows MR users (I don’t have a Windows MR headset at home so I can’t test with the one opengl vr game I have)

#7

Does it allready work with steam vr? When I press the “Start VR Session” button with a HTC Vive I get a message stating:

“Failed to query OpenXR runtime information. Do you have an active runtime set up?”

What does that mean?

#8

Means it couldn’t find the openXR support driver, you’d have to opt in to the steamvr beta to get it for the vive.

1 Like
#9

Thank you. It is working now.
Since I am working on a VR project at the moment this is actually super useful. I was wasting so much time with exporting models and testing how they look in VR and now I can just change the geometry and see them hassle free.
Thanks to @julianeisel and everyone else involved. :smiley:

1 Like
#10

There was a Steam Beta update today (1.13.5) and now, when pressing the “start VR Session” button in Blender you get the following error mesage:

Error: No format matching OpenXR runtime supported swapchain formats found.

#11

Yes, unfortunately the latest updates have a number of issues. See https://steamcommunity.com/app/250820/discussions/8/2448217320134601817/#c2523653167122752247 & https://developer.blender.org/T78043.

I still have to check details, maybe we can do a workaround, although ideally SteamVR would handle this better.

1 Like
#12

Does anybody know how to revert back to an old beta version in Steam?

#13

First run feedback (using 2.92 release and a HP Reverb G2 running OpenXR):

  • Blender docs for WMR may be outdated. I don’t think I had to enable OpenXR?
  • Blender docs for Landmarks seem bare bones. I didn’t understand it at least.
  • There is no visual feedback for Landmarks in the 3D view… what are they? Cameras or points?
  • Ideally, I’d want to place a Landmark as an empty on the ground, and in VR, select them from a list, or simply point to them and teleport there.
  • Oh, is a “Custom Pose” an empty? But you can only input values manually? Not actually hook up an empty? And why use the wording “Pose”? That has very little to do with a “landmark”, and quite a lot to do with other things in Blender.
  • Huh, so a “Custom Camera” does give you a list to choose a camera from, so why not do that for empties as a Landmark as well? This was clearly made for someone to check shots for a movie, and not for someone checking architectural models, which is what I’m trying to do.
  • The Landmarks list ultimately “crashed” and refused to display anything for me:
    bild
    bild999Ă—972 101 KB
  • This “crash” was apparently so severe that it got saved with the file and the menu is not possible to open at all now, even after restarting Blender!
  • HP Reverb G2 controllers don’t show up at all in VR, but maybe there isn’t any controller support yet?
  • I appeared near the origin (I think) and after the above “crash”, under the ground… so I never really got to inspect my scene.

Has the development on this stalled? Seems like there was lots of noise about it a while back, but there’s been silence on it for a while now?