Developer Discussion: New Grease Pencil Data Structure Proposal

Introduction

When grease pencil was first introduced to Blender back in 2008(?) by Joshua Leung it was designed around drawing annotations into the viewport. Over a decade later, the community has built a feature-rich 2D animation tool out of the initial draft of grease pencil. Today, there are some challenges and limitations that we face with how grease pencil is structured. I’d like to summarize the issues and propose a way forward.

Current Code Related Problems

  • Inefficient data structures
    • Layers, Frames and Strokes are stored in linked-lists making them very slow to search and iterate through.
    • Large tree-like hierarchy: GPdata → Layers → Frames → Strokes → Point
      • E.g. to access a point, one needs to search through all the levels above to get to it.
  • Grease Pencil is separated from the rest of Blender. Integration with other modules is hard.
    • Some examples are:
      • Grease Pencil modifiers
      • Grease Pencil materials (will change with upcoming EEVEE rewrite)
      • Grease Pencil dopesheet
    • More internal examples:
      • Grease Pencil modes
      • Grease Pencil brush settings (might be necessary)
  • A large portion of the grease pencil code base is very old and also very verbose (not an issue exclusive to grease pencil in Blender of course).
    • It gets harder and harder to make bigger changes
    • New developers have a hard time contributing

Proposal

For the last couple of weeks I have been working together with the Grease Pencil module (Antonio, Daniel, Matias) as well as Hans (Geometry Nodes developer) and Amélie (Developer at Les Fées Spéciales) to work on a new proposal for grease pencil.

The main goals of this proposal are to solve the issues mentioned above and also open the door for features that are currently very hard to implement.

Here is a quick summary:

  • Move to C++ and a RAII idiom.
  • Use two separate arrays to store layers and frames.
  • Use CurvesGeometry to store the strokes in a frame.
  • Make use of attributes to store radius, opacity, vertex color, etc.

(For more details, take a look at the demo/test implementation.)

Some of the new possibilities with this proposal:

  • Overall improved performance (e.g. copy-on-write).
  • Grease Pencil fills with holes.
    • Idea would be to use an attribute on strokes to group them and then pass the whole group to the triangulation algorithm.
    • This could enable: boolean operation on fills, boolean eraser, etc.
  • Layer groups.
  • Stroke types (default: poly, but could also use e.g. bézier type).
  • Easier integration into Geometry Nodes (because the core structure would be based around the same as e.g. Curves).
  • More readable and concise code.

Test Implementation

To better understand how the new grease pencil structures could look like, how we would work with them, and how they could perform I created a branch gpencil-new-data-proposal that implements a test file with the new proposed structures. They are:

  • blender/source/blender/blenkernel/intern/gpencil_new_proposal_test.cc and
  • blender/source/blender/blenkernel/intern/gpencil_new_proposal.hh

Feedback on the code would be greatly appreciated!
Here are some first performance numbers that came out of this test:

Duplicate GP data-block
Layers: 10, Frames: 5K, Strokes: 500K, Points: 50M
New structure:  205ms
Old structure: 1410ms

Note: Since the new structure is still missing parts of the old structure, take these numbers with a grain of salt.

Discussion

This proposal would be a major rewrite of course. There is a good chance it will break python compatibility, etc. We would like to get opinions/suggestions/concerns from other developers on this.

Note: This thread is not meant for user feedback. Only for development related discussions on the proposal!

36 Likes

Generally speaking, using CurvesGeometry with attributes makes a lot of sense to me.

One thing that’s not an ideal fit is that the list of attributes would be duplicated per frame, and there would need to be some way to keep that in sync between frames. Maybe across layers too, depending if attributes are defined per layer or not.

Do you have an idea for how this would integrate with geometry nodes exactly? Would geometry nodes treat grease pencil data as its own geometry type that works directly with existing fields and curves nodes (applying the changes on all frames and layers)? Or would there be some conversion to and from a curves geometry to be able to use the curves nodes?

2 Likes

I am not sure if I understand exactly what you mean. If you mean that the CustomData layers would have to be allocated per frame, then yes that is true. But since every grease pencil frame can be entirely different from all other frames (they are just their own container of strokes after all) I don’t think that is a big issue. Maybe I am missing something.
EDIT: To clarify, with how grease pencil is currently used, it’s rare to modify multiple frames at the same time. The exception of course is multi-frame edit.

So from what I gathered so far, grease pencil would implement it’s own GeometryComponent. I don’t think we can fully reuse the curves geometry component code, since - as you mentioned - grease pencil works on frames and you would have to have some piece of code that extracts the actual CurvesGeometry that is then modified. I don’t know how this would work at the moment. Maybe you always use the data from the current scene frame, maybe there is a node that allows you to access a specific frame, or maybe you modify all frames by default with an optional selection of some sort. But once you get that curves geometry, you could make use of it’s attributes without much hassle I believe.

If you have a user interface with a list of attributes for the user to edit, then I think that would be expected to affect all frames. Adding, removing or renaming a color attribute again for every frame is inconvenient.

Ah yes, I suppose a panel in the object data properties with a list of attributes would affect all frames when modified.

I like the goal to make gpencil more integrated with the rest of blender and allow for geometry nodes and more stroke types.

However a lot of what you want to achieve can be done with less drastic measures. We can move away from linked lists to array lists and improve the current data structs to make them more compatible with the rest of blender. E.g. use float[3] and float3 for C++ instead of float x, y, z and improve the names of things like pressure to curve compatible names like radius. And when that is done, we can start to move it over to the new structs with less code changes.

This should already give us half of what you want before breaking changes come up (performance, cleaner code) and should not be controversial. Keep the patches small and concise and make the compatibility breaking change as delayed and lightweight as possible. We can assess each step individually, one at a time.

1 Like

There were some discussions on how we could move to this proposed new structure and I fully agree with you here:

I didn’t want to include a possible way of moving to this new structure precisely because this hasn’t been decided on and there are some open questions.
Switching over to arrays might be a way forward, though this change alone would have to touch pretty much all grease pencil related source files. The versioning part of the code would be straightforward so that’s good.
Overall I think I would personally prefer a “gradual” change (as much as possible), like you suggest. But I think this needs careful planning first.

5 Likes

my 5 cents from addon developer, that using gpencil from py. breaking changes are a bit tough thing, but if we get in exchange a chance for normal booleans (pixel-perfect eraser?) and geometry nodes support and some unification with other blender types (materials stuff, etc) - this is certainly worth rewriting the py code :slight_smile: hope to see this proposal go live some day!

5 Likes

From an animator’s standpoint, getting a possible future where grease pencil frames can live and be transformed in the same dope sheet is a huge win.

3 Likes

Look at this: ⚙ D15003 Add GP layers in main dopesheet (WIP)

1 Like