2025-04-09 Grease Pencil VR

Meetings
1

Attendees:

  • Dalai Felinto
  • Julian Eisel
  • Lazaro Palafox
  • Samuel Palafox
  • Sergey Sharybin

Notes from last year’s meeting :birthday:.

Lazaro and Samuel resumed working on the experimental Grease Pencil XR feature this year.

This meeting intends to go over the existing planning and discuss the best to make sure the development can be integrated with Blender in the future.

Grease Pencil XR team plan

  • Grease Pencil 3.0 (gpv3 from now on) to paint strokes in the 3D viewport using VR and XR controllers (Oculus Quest 2.0 as for now. Also testing on a Logitech XR brush).
  • Modify gpv3 brush size, strength and sensibility for both settings via XR controllers.
  • Keep the scale unified in the gpv3 strokes no matter the user’s interaction with world scale inside VR.
  • Implement gpv3’s onion skin visualization inside the VR 3D view.
  • Provide imm graphic UI components for gpv3 brush visualization inside VR.
  • Provide a debugging tool called “ui-mirror” that copies the screen’s frame buffer into a plane attached to the scene (or one hand’s controller), so the user can interact with Blender’s missing UI while working inside the VR scene (up to a point).
  • Implement basic core functionality to the XR controllers such as the ability to use KEY_MODIFIERS (CTRL, ALT, SHIFT) and perform action combos.

Current strategy

  • Expand the existing mouse struct with the depth, discarded when not needed.
  • Inside the VR mode simplify coordinations transformation.

Proposal: get one brush/operator reviewed before going too far into the code.

Code dive:

  • grease_pencil_paint.cc::PaintOperationExecutor
    • Julian suggests to be careful with if (is_xr) ... else ... branching, we can’t expect other developers to keep the XR branches working (see below).
  • struct InputSample expanded with is_xr and controller_position.
    • Better to have abstractions outside the main loop (leave them on the high level).
    • It comes with a price of duplicating functionality.
    • Suggestion: Create a PR so Julian and Sergey can review the flow of one operator and give feedback that applies to all.
    • Keep in mind that most people testing the code in the future won’t have a VR. Need to make sure the developer documentation covers emulators. done already

User interface

A brief discussion about the possible solutions for the UI.

Julian suggested individual regions as individual VR elements. This could be a middle-ground between the current (hacky) entire-UI in VR and a complete VR custom interface.

This would need to be implemented on the core drawing code, to be able to target different separate buffers. Those would need to be special offscreen regions.

Next steps

  • Planned release for the existing plan (Q2).
    • Prototype will be used by the Cat Walk project.
  • Submit initial PRs.
    • Grease Pencil paint.
      • The PR can be focused on one operator/brush first.
      • Even if the code is the complete code, the patch description can direct the review scope.
    • XR Tools
      • Modifiers keys.
    • UI Hack
      • The most experimental part, will be left to the end.
  • Add-on
    • It would simplify testing if all the changes are on top of the existing add-on.
  • Feedback / builds
    • After the prototype is released …
    • Create a PR from the branch (ideally re-based from main).
    • Create an official @blender-bot package build.
    • Create a devtalk thread.

Roadmap

The Grease Pencil XR seems the most promising first target for XR this year.

The other potential targets will be assessed and planned later, once this is more robust or landed.

14 Likes
2

This would be awesome! Especially in an educational setting, where first-time students have a hard time memorizing Blender’s UI. Putting them directly into Grease Pencil in VR mode, would be a great “shortcut” for them!

2 Likes
3

This is so exciting!