Project-Relative URIs in Blender

Contributing to Blender
1

TL;DR: I’m working with some game and film studios to establish a URI spec for hierarchical project-relative paths that are resolved using simple configuration files. The idea is to push concepts like work spaces, projects, and libraries down into the file paths themselves, such that users can reap the benefits of project-oriented infrastructure without needing to deal with environment variables, asset servers, DCC launchers, or application-specific project tools.

The SAP URI Scheme in a Nutshell

     ┌─ Anchor                                   
    ┌┴─┐                           
sap:proj::PreViz.Reference//scale_ref/human/tall.usd
└┬┘       └──────┬───────┘  └──────────┬───────────┘
 └─ Scheme       └─ Namespace          └─ Path
  1. The Scheme segment is what makes it a URI.
    • SAP stands for Semantically Anchored Path.
  2. The Anchor segment indicates the kind of directory to ascend to.
    • A directory is anchored by putting a .dirconf.[anchor].json file inside it. The anchor is included in the file name so that simple paths (ones without namespaces) can be resolved without opening and parsing the file.
    • There are seven anchors in the spec (see table), specifically to support both shot-, scene-, and sequence-relative paths inside projects and flexibly arranged shared resources and dependencies outside projects.
  3. The Namespace segment is an optional segment that’s resolved by reading the namespace dictionary inside the configuration file.
  4. The Path segment is resolved as a normal path.
Anchor Name
org Organization
dept Department
work Workspace
proj Project
lib Library
kit Kit
pkg Package

The information above isn’t enough to count as a proper, formal, specification. I’m in the process of cutting down the wish list that came together from early feedback sessions with game and film studios into a concise feature set appropriate for a 1.0 spec, however, so the baseline functionality shouldn’t be significantly more complicated than what’s described here.

Background

The need for a general-purpose project-relative URI spec originally came up in conversations about USD Asset Resolver plugins. USD’s layer-based workflow, where you can easily have 5-6 layers in a base asset, works best when you can minimize manual file path updates. It’s a great workflow when you have the infrastructure for it, but it’s hard for small teams and solo artists as there isn’t an easy way to incrementally integrate that infrastructure; the bulk of the complexity is front-loaded in a way that isn’t artist-friendly.

In talking to studios that have implemented their own project-relative asset resolvers, a consistent theme that came up was that the utility of project-relative paths increased in proportion to their lifespan — the longer they were able to go without reifying those project-relative paths into absolute paths, the more they liked them. From their perspective, using project-relative paths inside USD assets was nice, but the ability to use project-relative paths throughout their pipeline was what ultimately justified the investment. And in order to achieve those full-pipeline lifespans, many of them relied on in-tree configuration files that let their smaller tools resolve paths without having to call out to their asset server.

The SAP URI spec is an attempt to extract that last critical part — easy project-relative path resolution for what’s already on disk without pinging an asset server — in a form that’s easy to implement both inside USD as an asset resolver plugin and outside USD as a library for DCCs and in-house tools.

The Blender Side of Things

Just to be clear, the Blender developers haven’t committed to anything and I’m not asking them to. My intention is to keep Blender’s stakeholders in the loop, as Blender is a useful test environment for these kinds of tools and there’s a meaningful conversation to be had about Blender’s role in bridging the gap between studio and solo-artist workflows, but landing a SAP resolver in Blender isn’t my primary focus.

My primary focus is on solidifying the specification into something I can put a version number on and my secondary focus is on shipping a permissively licensed reference implementation. I’ll be pulling together Blender test builds as I go, to test the spec and test the reference implementation, but I won’t be treating any of that as an open pull request for the Blender developers to respond to.

From my perspective, this is a Blender-adjacent project rather than a Blender project. I’d be happy to pull together a SAP resolver implementation for Blender, if the developers are interested, but the main reason I’m writing this post is to avoid stepping on any toes when using Blender as a test bed.

What’s Up Next

While I’d love to be doing this full time, my schedule is at the mercy of the companies I contract with. I’m not keen on planning more than a month or two out at this point, as I still have to juggle private feedback rounds and set up the formal infrastructure for public feedback and specification development, but here’s the rough order of my current roadmap:

  1. Sort out glob pattern support in the base specification (next few weeks)
  2. Sort out good-enough-for-now config file structure in base specification (next few weeks)
  3. Confirm spec tweaks via the python implementation (February)
    • This will feed into a round of blender testing via an add-on, focused on configuration management using sap’s
      • The add-on is extremely bare-bones at the moment, I wrote it in one sitting, it isn’t ready for testing yet.
  4. Solidify dependencies for C++ implementation (Late February)
    • This is mostly a glaze vs yyjson debate that’s 90% settled
  5. Confirm spec tweaks via C++ implementation (Mid March)
    • USD-focused testing round

It might seem strange to see so much emphasis being placed on testing when the thing being tested is a file path specification, but we’ve all seen what happens when under-baked and over-baked specifications become load-bearing parts of production pipelines (looking at you, ini and yaml).

Where I Need Feedback Right Now

Globbing. There’s a lot of nuance to consider when it comes to supporting glob patterns (and equivalents) outside of the Path segment in SAPs, and I need people to talk me down from trying to capture all of it. An expression that captures ranges of anchors is probably reasonable (for things like cascading configurations), expressions that cross segment boundaries probably aren’t, and anything involving disjunctions should probably be off the table.

I’ll stick my head into the relevant meetings to ask questions and to show off progress where relevant. There are some ancillary conversations (e.g. distributing USD plugins and schemas via Blender’s Extensions platform) that I’m interested in having, as any help I can provide on those fronts will also make my own work easier.

In any case, I’ve already written too much so I’ll end it here. I’ll have updates soon.

2 Likes