VR Scene Inspection Feedback

Other Development Topics User Feedback
#1

The 2.83 release will be the first Blender version to include core VR support. This is just the first milestone named VR Scene Inspection, which aims to lay foundations for further, more feature rich use-cases.
Purpose of this thread is to gather user feedback on the VR Scene Inspection milestone.

See Also

Release Notes: https://wiki.blender.org/wiki/Reference/Release_Notes/2.83/Virtual_Reality
User Manual: https://docs.blender.org/manual/en/dev/getting_started/configuration/hardware.html#head-mounted-displays-virtual-reality
Milestone design task: https://developer.blender.org/T71347

Known Issues

Here are some known issues for the first milestone (order does not represent priority):

  • Limited usefulness of VR Landmarks (VR Scene Inspection Add-on)
    Due to unfortunate timing, it was not possible to get the VR landmarks polished to make them work well in all supposed use-cases. There are some issues with changing the active landmark remaining, and the creation of new ones needs more features (e.g. creating custom landmarks from current viewer position).
    Status: New Add-on features are in development. Fixes can be backported to 2.83 but general improvements will have to go into 2.90.
  • Base poses should probably be sent to OpenXR
    Currently, Blender takes the position and rotation deltas of the eyes and applies them to its base pose (i.e. the active landmark). There appears to be a way to let OpenXR runtimes know about that so the base pose can be applied there. We werenā€™t able to get that to work. (See next point)
  • Height positioning of base pose needs to be better
    Ideally, when opening a VR view, the eye distance to the physical floor would match the virtual one. It doesnā€™t appear like OpenXR deals with this sufficiently. If the previous point is addressed this might be solved, but it is a big unknown.
    Status: Help needed. Someone with experience with the OpenXR specification or other VR platforms is needed to find a solution.
  • VR Camera feedback is performance hungry
    The VR Camera feedback option allows drawing a camera-shaped gizmo in regular 3D views, indicating the current position and rotation of the VR viewer. That currently requires continuously redrawing the 3D Views it is in, because gizmos are drawn together with the viewport. A patch to address this was written, but rejected because of design concerns and conflicting design changes in master.
    Status: This could be addressed soon as part of regular development. Itā€™s not a high priority but doable without much work.
  • VR mirror is performance hungry
    The first milestone requires a fully interactive mirror of the VR view in a regular 3D View. The only way to do this without bigger changes is by re-rendering the entire viewport, effectively as a 3rd eye. Itā€™s tricky to improve this, but there are some ideas.
    Status: No concrete plans to address this. Add-on UI shows a warning now.
  • VR mirror could share more settings with VR view
    If a regular 3D View serves as VR mirror, it currently only shares shading settings with the VR view. It could share more settings, like the active View Layer or clipping distances.
    Status: Can be improved as regular developemnt
  • DirectX compatibility layer not working on some AMD cards.
    The compatibility layer simply fails on some cards/drivers, which is indicated with a pink screen.
    Status: Needs investigation - cause is unclear and no developer is available to investigate. Help welcome!
3 Likes
#2

The only issue that I had with this mode was not have a custom ā€œwalk modeā€ that I can use in VR like a debug camera mode to move free by the 3D world. You can use the classic walk mode, but the movement is with the position of the camera in the3d viewport, not in the VR scene.

#3

@sebastian_k , Iā€™m continuing here from https://developer.blender.org/T71347. I tried your version of the addon:

  • Custom pose: very good! That is something I was also thinking about. It seems to affect the base point, not the VR camera, just as with landmarks as discussed above. A good thing to have would be a switch: ā€œaffect Camera / affect base pointā€.
  • In some cases it is useful to have the camera update live while you change the coordinates, in other cases you might want to input exact coordinates and then press Enter, and only then the camera would jump there. So weā€™d have two modes: ā€œlive updateā€ like you have now, and ā€œupdate when clicking the apply buttonā€.
  • Displaying Landmarks in the viewport: yes this is also helpful, Iā€™d probably constantly leave it on. Maybe also display the name of the landmark in the viewport?
#4

After using it a little longer, the problems Iā€™ve encountered

  • If you use eevee it has a reduced color depth
  • Positioning yourself on the scene is very complicated, moving through it even more. For example, if I put a camera at 1.60 meters high and run the VR, with RiftS, the height I can have may be 1 meter more. I think the system is quite opaque in how it works.
1 Like
#5

I think I found something about the AMD cards issue.
Looking for the word ā€œpinkā€ @ developer blender: D6190 (canā€™t add link) made me look for a interop extension, that seems to be WGL_NV_DX_interop or WGL_NV_DX_interop2.

This could be related to either:


or
https://community.amd.com/thread/166698
Iā€™m new here, so I hope this helps in any way.

#6

SteamVR finally rolled out their first implementation of OpenXR support in Beta, so this should open up a lot more users, though their implementation is limited to DirectX for the renderer.

Thereā€™s a conversation with a developer at Valve asking about enabling OpenGL support for their implementation.

Does the D3D wrapper get used for both Oculus and Windows MR when using Scene Inspection, or is it only used for Windows MR? The pink hmd issue (which, unfortunately as an AMD user I am getting) might not be as much of a problem if it can go through OpenGL natively (assuming Valve implements it on their end). And then I think the SteamVR->WindowsMR bridge MIGHT handle converting it over for Windows MR users (I donā€™t have a Windows MR headset at home so I canā€™t test with the one opengl vr game I have)

#7

Does it allready work with steam vr? When I press the ā€œStart VR Sessionā€ button with a HTC Vive I get a message stating:

ā€œFailed to query OpenXR runtime information. Do you have an active runtime set up?ā€

What does that mean?

#8

Means it couldnā€™t find the openXR support driver, youā€™d have to opt in to the steamvr beta to get it for the vive.

1 Like
#9

Thank you. It is working now.
Since I am working on a VR project at the moment this is actually super useful. I was wasting so much time with exporting models and testing how they look in VR and now I can just change the geometry and see them hassle free.
Thanks to @julianeisel and everyone else involved. :smiley:

1 Like
#10

There was a Steam Beta update today (1.13.5) and now, when pressing the ā€œstart VR Sessionā€ button in Blender you get the following error mesage:

Error: No format matching OpenXR runtime supported swapchain formats found.

#11

Yes, unfortunately the latest updates have a number of issues. See https://steamcommunity.com/app/250820/discussions/8/2448217320134601817/#c2523653167122752247 & https://developer.blender.org/T78043.

I still have to check details, maybe we can do a workaround, although ideally SteamVR would handle this better.

1 Like
#12

Does anybody know how to revert back to an old beta version in Steam?

#13

First run feedback (using 2.92 release and a HP Reverb G2 running OpenXR):

  • Blender docs for WMR may be outdated. I donā€™t think I had to enable OpenXR?
  • Blender docs for Landmarks seem bare bones. I didnā€™t understand it at least.
  • There is no visual feedback for Landmarks in the 3D viewā€¦ what are they? Cameras or points?
  • Ideally, Iā€™d want to place a Landmark as an empty on the ground, and in VR, select them from a list, or simply point to them and teleport there.
  • Oh, is a ā€œCustom Poseā€ an empty? But you can only input values manually? Not actually hook up an empty? And why use the wording ā€œPoseā€? That has very little to do with a ā€œlandmarkā€, and quite a lot to do with other things in Blender.
  • Huh, so a ā€œCustom Cameraā€ does give you a list to choose a camera from, so why not do that for empties as a Landmark as well? This was clearly made for someone to check shots for a movie, and not for someone checking architectural models, which is what Iā€™m trying to do.
  • The Landmarks list ultimately ā€œcrashedā€ and refused to display anything for me:
    bild
    bild999Ɨ972 101 KB
  • This ā€œcrashā€ was apparently so severe that it got saved with the file and the menu is not possible to open at all now, even after restarting Blender!
  • HP Reverb G2 controllers donā€™t show up at all in VR, but maybe there isnā€™t any controller support yet?
  • I appeared near the origin (I think) and after the above ā€œcrashā€, under the groundā€¦ so I never really got to inspect my scene.

Has the development on this stalled? Seems like there was lots of noise about it a while back, but thereā€™s been silence on it for a while now?

#14

Hi All,
It seems that this thread has not been active for some time, but I will give it a try anyway.

For some time now, I have been experimenting and hacking on a VR setup for Linux, and as I have played around with Blender in my spare time for a few years, I was delighted to discover the VR Scene Inspector. Thanks to Julian Eisel and all others involved.

Given that VR headsets with a reasonably useful resolution have now come down to a manageable price, I decided to invest some effort in implementing a basic Linux driver for the HP Reverb headset. It has now been accepted as part of the ā€˜Monadoā€™ project, -an open source implementation of OpenXR. Various Monado developers have since contributed additional HP Reverb specific code to the driver, such as improved rotational tracking and distortion correction.

The HP Reverb is the first WMR based HMD in Monado, but since different WMR headsets have a lot in common, I also hope that support for more models will come. At the moment the original and the G2 Reverb models are supported (-so far only rotational tracking, thought).

So now is the time for me to get involved with the VR aspects relating to blender. I have been looking a bit around for (active) Blender/VR forums for developers (- i.e, not targeted artists or other users). I havenā€™t found much of the sort yet.

I would like to get involved in this aspect of Blender development to the degree that my spare time allows me, and therefore it would be great if there was a forum where I, from time to time, could discuss Blender/VR issues. So I hope that someone here perhaps can point me in the right direction.

Again, thanks for the good work.
Regards,
Nis Madsen

3 Likes
#15

Hi Nis, there is a more active disciussion thread on the #XR channel of Blender.chat: blender.chat.
Cheers, Zsolt

1 Like
#16

Thanks, Zsolt. Please keep them comming if you come across more places.
Regards,
Nis

#17

Iā€™m having an issue with drawing modes within the VR runtime. I call it drawing modes, but I havenā€™t seen it documented anywhere, Iā€™ve noticed at least two modes using VR inspector with Eevee:

  1. Unlit scenes: render a full field of view at a lower bit depth, but a medium to high frame rate. This is suitable for viewing the model and showing others.

  2. Lit scenes: renders a smaller field of view, at a lower frame rate, with the benefit of a higher bit depth and a full lighting. This is ā€œokā€ as long as you hold your head still, which is not great in VR, Iā€™ve only used it a couple times.

These two modes seem reasonable, given the task of connecting my Quest 2. And Mode 1 is really all I need.

My Issue: I wanted to make some non-VR shots that were well lit, but I noticed that rendering performance wasnā€™t great with lighting on, so I went into render settings and began tweaking things (unrelated to VR, I thought) and improved both the appearance and performance of the lighting, I cut down the number of lights in the rooms as well to improve performance, but after I had done that I went back to the unlit scene in VR, and found the field of view low and frame rate sluggish, like in mode 2, even though the color bit depth was low like in mode 1.

I donā€™t know if this is a bug, I donā€™t know what settings affected this, I loaded a previous file and found that it played in what Iā€™m calling mode 1 just fine, so I donā€™t think my network connection changed. Itā€™s odd that changes I made to increase performance in one non-vr situation would reduce performance in VR, so I really donā€™t know.

#18

Does anybody know how the VR Scene Inspection runs in Cycles? I had tried but the scene result looked seem as rendered in Eevee.