GSoC 2019: Core Support of Virtual Reality Headsets through OpenXR

Contributing to Blender Summer of Code
15

Phase 1 Evaluation

I’ve just completed my part of the phase 1 GSoC evaluation (mentor evaluation and project feedback). This seems like a good point to do a public evaluation of the first 3.5 weeks from my POV.

General project management:

  • So far, I’ve been mostly working on my own. I know my way around in the code and I should be experienced enough to make reasonable decisions. Also, the first phase was mostly about preparing the internals, so there’s not much to feedback from a user’s view yet.
    Whenever I felt a need for feedback from Dalai or other devs, I got in touch with them. And I write weekly reports as requested.
    I’d be more than happy to hear from mentors if they think this was fine given the state of things.
  • For the couple of times I wanted feeback from other devs, they were there to help. That includes Dalai who has also been very responsive whenever needed.
  • University exams and assignment deadlines kept and keep me busy. So I haven’t been able to deliver with full steam yet. Friday in a week, the busiest phase is over, things should become more relaxed then.

Ongoing technical project challenges:

  • Biggest issue for my project is that I have to work in a very limited development environment. Only two OpenXR runtimes (platforms implementing the new & provisional OpenXR specification) are available: Windows Mixed Reality OpenXR Developer Preview and Collabora’s Monado.

    • Monado - although I strongly support it as a FOSS enthusiast - has limited features and worse, it doesn’t even want to run for me. Even after having gone through required hoops like updating my Linux distros to a testing version. This is for sure solvable, but takes time away from the project.
      I really hope this will change before too long.
    • The Windows platform of course requires me to use Windows and the Windows development environments. At least the OpenXR runtime is easy to set up and works.
    • So I have little choice but to use a Windows dev environment. One that I’m not used to and that turns out to have quite some quirks (see next point).

  • For the most time, I couldn’t use a C/C++ debugger at all. The Windows debugger keeps hitting whenever OpenXR, or the Windows MR runtime comes into play. I can continue execution to some point but not to where I need it now. Not sure where the issue lies, in the debugger, our code, the OpenXR SDK code or the runtime’s code. The same happens with the OpenXR SDK’s example applications though, suggesting it’s not a fault on our side.
    Either way this is a big problem.

  • Graphics, VR and OS dependent code is in general difficult to debug. Execution passes through level’s not in our control. You often have to pass around opaque handlers (pointers to memory with foreign data structures we can’t read). You rely on provided error detecting mechanisms which tend to be vague and at times broken even.

    For example, right now I’m trying to use the DirectX compatibility layer I added to pass viewport rendering to the Windows MR runtime (through OpenXR). For some reason the extension I use for DirectX compatibility (NV_DX_interop) fails. Even though I use it almost the same way I used it previously. I can’t use the Windows debugger (see above), the error happens in foreign code and the error message is not helpful and possibly broken (same error as described here).

It was expected that the bleeding edge nature of the project would present quite some challenges. It’s definitely not been clear sailing but I’ve overcome all challenges so far and I’m in schedule. Still frustrating…

Few words on the OpenXR specification:

  • I have no doubt the specification is going to be the standard for VR/AR/MR/…
  • The specification is documented quite fine given the early state. The OpenXR SDK contains a good reference implementation I use as a guide permanently.
  • More OpenXR runtimes are expected to appear soonish. I wouldn’t bet on that happening during the GSoC period though.
  • I think it was a good decision to encapsulate all OpenXR calls behind an abstraction (GHOST level by now). Not just because there’s low level OS and graphics library related code. There’s quite some complexity in OpenXR too, that can be simplified a lot through a layer of abstraction. I guess it’s comparable to Vulkan in that regard.

And finally on user testing:

  • Many people already offered help on testing. Gotta love this community!
  • The unfortunate thing is that users will also rely on the availability of OpenXR runtimes. I guess only testing using Windows MR runtime and therefore WMR headsets is usable for now.
  • I’m planning to document the requirements and (few) set up steps once the project is ready for user testing. I might also do a call for testing then. I expect that to happen in 2-3 weeks.
11 Likes