Thoughts on making Cycles into a spectral renderer

Not sure what you mean with re-emitted.You have a emitter source that radiate the material then material change the emission color whatever you programmed your shader.

Fluorescence is re-emission:

  • A goes in
  • A gets absorbed by the material
  • electrons inside get bumped to some higher level
  • electrons fall to some lower level
  • B goes out

This is also why it takes time. But the time for those changes can be extremely short, making it effectively like regular reflection but with a complete change to the spectrum after reflection.

And the issue is, that Hero Wavelengths assume:

  • A goes in
  • A goes out

or

  • B goes in
  • B goes out

but can’t handle

  • A goes in
  • B goes out

The wavelength must stay constant!
A shader that demands the wavelengths change simply would not be supported at all by the usual Hero Wavelength implementation.

Why,i mean all what you have described are basic shader propertys as absorption.?

Listen. If your belief is that the shader is all that matters, then explain to me why getting real caustics has been such a difficult task even though regular reflection or refraction shaders would already very much imply the existence of caustics.

Shaders are not everything. The sampling techniques must actually explore the paths implied by those shaders efficiently. And regular Hero Wavelengths do not sample paths where the wavelength changes at all, let alone efficiently. It simply is not included as an option to sample in the technique.

This sounds odd.What happens with all the other shader which reflecting, absorbing ,scattering, emitting etc.Or as example all the propertys of the principled shader?

Precisely zero of those require changing wavelength across a single path of bouncing light?

Red photon goes in, red photon bounces several times, red photon hits the camera. At no point at all does it magically turn green. It just gets redirected and the intensity might get reduced (for physical materials)

What happens with light that gets absorped and scattered in a volume?You can programm a simple phaseshift to your shader.If i understand you right,then you mean to change the emitted source wavelength?I think this does not happen even in regular Cycles.
If that is the case i dont know.

Volume bounces are no different at all from surface bounces, except that they can happen anywhere within a volume, rather than definitely happening on a fixed surface.
Wavelengths generally stay constant across a path.
Unless you are looking at a Fluorescent path.
Note that a phase shift doesn’t change individual ray color. It may result in interference effects but those are completely unrelated to to Fluorescence. Those happen between different rays, rather than within a single ray. And such phase effects happen with surface bounces too. In particular, they are a big reason why metals look shiny

1 Like

You maybe have seen this papers,might be helpful.

This basically just says what I said above: To sample, you sample individual wavelength paths, not Hero Wavelengths, thereby running into the color noise issues Hero Wavelengths tried to fix.

I suspect there might be a “proper” (efficient) solution by using Continuous/Stochastic Multiple Importance Sampling (CMIS/SMIS) on the full render equation including Fluorescence. My guess is that it’d end up working out something like this:

  • instead of fixed-delta Hero wavelengths, sample the way SMIC tells us to, namely just uniformly randomly picking the set (this is a really easy change that improves on Hero Wavelength’s efficiency)
  • Sample whether a Fluorescence event occurs (this is done in the above paper)
  • if not, just do regular bounces
  • if there is a Fluorescence event, do the much more involved math of:
    • for every pair of wavelengths (λ, λ’) in your current random set (drawn with replacement)
      • evaluate how large the emission in λ’ is upon absorbing λ (weighing for energy conservation)
      • accumulate the new outgoing intensities

This would be much more expensive (quadratic rather than linear) in the number of wavelengths in a sampling set. But with any luck, it might just so turn out that this ends up having the right statistics, and it might avoid the color noise issues.

Whether this is actually true remains to be evaluated, but it sounds to me like something that at least could work. – For there to be any hope that this actually works, we definitely need SMC, because with vanilla Hero Wavelengths, you’d end up with a clear bias towards constant wavelength deltas.

If that actually works, an alternative slightly noisier but completely linear approach might be as simple as, rather than doing it for all pairs, just pick a single random pair. Again, remains to be proven.

These are 100% guesses. I suspect, if this is wrong, it may not be wrong by very much at all, and just requires some clever weighing to correct any biases introduced this way.

2 Likes

News from anyone on this project?

3 Likes

I don’t think @weizhen got to it yet (really hope we’ll get an update but I think she’s got other priorities)
@smilebags ran out of time and would feel more comfortable as a consultant on new efforts than take on the daunting task of polishing his old code base from pre Cycles X to work again. As I understand it, there were a bunch of fundamental design issues too, meaning it’s not so much a “make it work in new version” as it is a “start from scratch with lessons learned”.
I’m not entirely sure how the work balance was there but @pembem22 also significantly contributed to that branch and, possibly, more than smilebags by the end?

As for myself, I’ve only been testing the features. Haven’t contributed any code to Blender to date.
I’m considering figuring out how to do that at some point in principle but I doubt it’s a great idea to start with my own take on a spectral branch.

Ultimately, it comes down to priorities, available time, and financial sustainability (i.e. this was an unpaid community passion project)

4 Likes

I update the cycles-spectral-rendering branch once in a while and make sure the tests pass. I think there are a few changes I can contribute back into main to reduce the differences between the branches.

7 Likes

oh I had no idea, that’s great news!
Anything that reduces the friction towards a renewed effort for a spectral branch is a plus

1 Like

As far as I remember, there are a few roadblocks that need to be solved first:

  1. The white point issue.
  2. Better spectral reconstruction.
  3. Doubling the amount of render kernels to be able to switch between RGB/spectral is not an option.

There’s also the recently merged thin film iridescence feature, which I completely disabled in the branch, as I don’t see a trivial way to make it spectral yet.

I thought the main issue was the kernel doubling.

The whitepoint issue basically amounts to separating out RGB emissive materials from others, making emissive ones follow the scene color space conventions (whichever that might be) and otherwise going for E whitepoint, right?
Handling image textures would be a bit of an issue but in principle, whitepoint adapting is basically just one matrix multiplication. It’s mostly a question of getting the right matrix, ideally for any sort of colorspace.
I’m guessing there already is some sort of flag to separate out emissive from other kinds of shaders, right? If not, that’d certainly be an effort to add…

And honestly I feel like the spectral reconstruction we had was fine to start with? Sure, it’s not perfect, but to get us started, it’d work.
I suppose the big issue with it is for when you go out of gamut. At the very least there ought to be some sort of feedback in such situations. But really, that out of gamut issue is already a thing now.

I also recall, and this may not be a showstopper, but one of the standing issues was that the render passes didn’t get the correct colors.

Ironically the iridescence feature is gonna be much much simpler in spectra. Whatever it does now to approximate a spectral effect, all it will have to do then is push rays through the wavelength dependent thin film iridescence formula and that’s it.
I can’t say I know how to actually do that though.

Thanx for the update :slightly_smiling_face:

Any link to download it for testing?

None so far, you can find the branch here if you want to build it.

1 Like

I realize nothing is actually in development, but I played around with alternative spectrum → RGB variations (using an old build, did not attempt to build this new one, but should be the same)

The corners here are four different spectra that attempt to match the color in the center. (I added the central color in post, matching the RGB input values the spectra attempt to mimic)

reds_swatches
yellows_swatches
greens_swatch
cyans_swatch
blues_swatch
magentas_swatches

You might need to download and zoom in to tell the difference for some of these. Cyan in particular seems to be a great match regardless of what you do. The biggest errors are definitely along the line of purple.

The three new possible spectra for pure R/G/B are summing to 1 exactly (up to 64bit float rounding errors) by construction. The current set doesn’t quite do that I think.
The old variants also produced slightly negative values of some primaries in sRGB, going ever so slightly out of gamut. – This is less of a concern if you’re going to use AgX anyways, but might be a pain if you are expecting to already be in sRGB. It also means the corresponding corners might not be rendered quite right here as these negative channels would simply be clipped.
All these new approaches are completely positive within sRGB. The downside is that they all are less saturated.

They each just optimize the same target in slightly different ways.

Top Left is always the current method, Center is the expected color.

What do y’all think: which of these is best and why?