Thoughts on making Cycles into a spectral renderer

[brecht]

@smilebags

I understand the reasoning behind this view, but I do think it limits the usefulness of having spectral rendering in the first place. Isn’t the ‘incoming’ vector, and the ability to determine different reflectances for red, green and blue already defining a material based on view direction and wavelength?

None of the effects you describe would be impossible, but they would have to be implemented in BSDF, not as part of shader node evaluation.

The way OSL and other modern shading APIs are designed is that you can evaluate the shader nodes once for a shading point, and can then evaluation the resulting BSDFs multiple times for different view/light directions and wavelengths. This is important to support modern rendering algorithms and good importance sampling. It is something we are very unlikely to deviate from.

@smilebags

The reason I bring up the preprocessing step for colours is that the current best known spectral synthesis function (Meng) is iterative and quite expensive. Getting from a single wavelength to XYZ or RGB is very easy, but synthesizing a spectrum from an XYZ (or RGB) colour is the challenging part. Running an iterative function every time a colour is encountered is likely to get slow. I guess we’ll find out when we try it out.

One way or another, RGB to wavelength will have to be fast. In a production scene basically all surfaces will be textured.

@smilebags

I can understand why this might be low priority for some users, but there are many valid reasons to have it, and I feel that if possible, it shouldn’t intentionally be ‘designed out’ of the system.

We don’t have to leave it out by design, but if we can e.g. do a significant optimization by assuming no phosphorescence support, I would happily do it. It’s not an important feature at all, and can be easily emulated with existing shaders.

[smilebags]

@brecht

None of the effects you describe would be impossible, but they would have to be implemented in BSDF, not as part of shader node evaluation.

Yes. I thought due to the huge amount of things people tend to make with materials, it might be more effective to allow it to be done in nodes. Things like flourescent light spectral profile would have to come from somewhere, either the user or some internal collection of spectra.

@brecht

The way OSL and other modern shading APIs are designed is that you can evaluate the shader nodes once for a shading point, and can then evaluation the resulting BSDFs multiple times for different view/light directions and wavelengths.

That makes sense, though I don’t see how being able to utilise the wavelength in the node tree necessitates deviating from this model.

@brecht

One way or another, RGB to wavelength will have to be fast. In a production scene basically all surfaces will be textured.

There are reasonably quick methods (the one I implemented is simply a sum of 3 ‘primaries’) and some intermediate ones. Finding the best balance will be key here.

@brecht

We don’t have to leave it out by design, but if we can e.g. do a significant optimization by assuming no phosphorescence support, I would happily do it.

I agree. If it turns out supporting phosphorescence kills some large optimisations, I’m all for having it gone. I just want to keep it on the table, as to not accidentally forget that something like that would be nice to have.

[troy_s]

@smilebags

but synthesizing a spectrum from an XYZ (or RGB) colour is the challenging part

@brecht

One way or another, RGB to wavelength will have to be fast. In a production scene basically all surfaces will be textured

Burns has lookups for paint-like response that work fast enough to paint real-time. Given the small bit of research I have done on this, Manuka matches are damn close to the Burns curves. See here for some solid resources and, specifically regarding the Burns equivalency in Manuka, this document.

Folks with actual spectrally constructed inputs, the data is already there.

[smilebags]

@troy_s

specifically regarding the Burns equivalency in Manuka, this document .

This document is perfect. It highlights some of the issues we are going to run into, and how to fix them. It gives real examples of how a spectral renderer is ‘better’ for accurate light transport for those who don’t care about making rainbows, and seems to suggest giving the user access to wavelength isn’t mutually exclusive to multiple importance sampling. Wonderful read.

[smilebags]

This article and video highlights some of the real world influences of having access to define spectral properties of lights and materials.
https://www.fxguide.com/featured/physlight-innovation-at-weta-digital/

Again, while this might not be of the greatest concern to some users, anyone wanting to work with Cycles seriously in film or accurate rendering will love having these features.

[brecht]

@smilebags

seems to suggest giving the user access to wavelength isn’t mutually exclusive to multiple importance sampling

Where exactly? As far as I can tell with Hero wavelength sampling the user would have to specify the intensity of 4 or 8 wavelengths, or we’d have to evaluate the shader nodes 4 or 8 times automatically. This is just not practical.

Letting users define custom spectra with e.g. a color ramp is fine, but dealing with specific wavelengths in shader nodes seems either inefficient or exposing details of the sampling algorithm.

[smilebags]

Spectral rendering: Monte Carlo hero wavelength
hero determines importance sampling
trace a stratified set of wavelengths along with the path
combine using MIS

On page 28 seems to indicate that’s how they did it in manuka, but obviously I don’t know how exactly they implemented it. The following pages compare hero wavelength sampling with different numbers of wavelengths per hero.

Spectral importance sampling

• chromatic extinction (skin)
• wavelength dependent Bsdf (interference, iridescence, Rayleigh scattering)
• dispersion

This is also on page 49 which seems to suggest (along with page 54) that spectral importance sampling is critical and has helped Manuka a lot.

[brecht]

Ok, that just describes what Hero wavelength sampling is, not how spectral rendering is exposed to the user.

[smilebags]

I see.
Manuka uses Renderman compatible materials but I’m not sure how spectra are exposed.

The difference between exposing a ‘wavelength ramp’ and a wavelength value seems insignificant. I think I need to understand more about the evaluation of materials in order to contribute intelligently to the conversation, but having a wavelength ramp already gives you all the access that the ordinary wavelength value would if it was plugged into a RGB curves node.

Conceptually, dealing with wavelengths in shader nodes is identical to dealing with colours. At this (channel or wavelength), this is how light interacts with the surface. There’s nothing different that I can see that differs between existing materials and spectral ones. Plug a wavelength driven value into the diffuse colour slot and you have a spectral defined diffuse shader, that’s all there is to it.

[smilebags]

@brecht

with Hero wavelength sampling the user would have to specify the intensity of 4 or 8 wavelengths,

The user speficies the intensity over the whole domain regardless of the sampling technique. Rather than defining red, green and blue intensities, it is specified continuously over the domain. This is no different to the result of RGB upsampling.

[Scott_Burns]

@troy_s

Burns has lookups for paint-like response

Just to set the record straight, I don’t use lookups to generate spectral curves from RGB. They are algorithmically generated to produce the one unique curve that has some optimal property, such as minimum slope squared. I do compare them to existing “look-ups” of various paints on my website, but only to assess how realistic the generated ones are.

[troy_s]

I am pretty sure a pure primaries approach would be more idealized for the sake of upsampling.

I suspect that is what Manuka is doing. Centre based on dominant wavelength, arrive at some useful width around it, then sample via piecewise cubic or Bezier etc. The results would be more idealized and accurate.

Once three primaries are deduced, the other two compliments can be added to obtain a precise CIE xyY position.

The only downside is that spectral gamuts such as REC.2020 can’t be obtained.

[smilebags]

Primaries seems like the way to go, but there might be some improvements that can be made, I’ll see if I can have a look into it.

Generating the primaries is the hard part, considering these requirements:

• Values bounded by 0 and 1
• sum of RGB spectra = 1*
• spectra’s XYZ matches source spaces primary
• spectrum must be as smooth as possible based on some smoothness metric

(*) this is where I think some cleverness can be done. You can remove this limitation by mixing between the primaries and a separate white spectrum as values approach white. In this way, only the sum of any 2 primaries must be below 1.

[Scott_Burns]

Done.
http://scottburns.us/fast-rgb-to-spectrum-conversion-for-reflectances/
Published just today.

[smilebags]

Well, that didn’t take long.

Are the primaries you created based on sRGB? If so, those spectra look perfect for this application, and doesn’t even need to be blended with a white in order to be bounded by 1

[Scott_Burns]

Yes, based on sRGB. But when combining the primaries, you should use linear rgb (un-companded sRGB) to preserve the linearity (and additivity) of the underlying CMFs (color matching functions).

[smilebags]

@troy_s

The only downside is that spectral gamuts such as REC.2020 can’t be obtained.

This is true. This is why I feel having the ability to define materials spectrally is critical to the whole system. Manuka seems to agree. If the colour they want to represent lays outside of REC.709, they should be able to create the spectrum themselves. There’s just the one natural restriction of saturation vs luminosity in reflectances, but that’s a natural consequence of the system, not something that has to be enforced.

[smilebags]

Yep, that’s what I’m doing already, so I am going to try swapping out the spectra and giving yours a try. Could I perhaps get the spectra in a more accessible format such as comma separated list?

Conveniently your spectra start and end at the same values I used for my implementation, but could this be extended to work with the far reds? There isn’t a whole lot of contribution there but having it undefined in that region leads to some difficulties.

[Scott_Burns]

@smilebags

Could I perhaps get the spectra in a more accessible format such as comma separated list?

Ask and ye shall receive…

rho_R
0.021592459, 0.020293111, 0.021807906, 0.023803297, 0.025208132, 0.025414957, 0.024621282, 0.020973705, 0.015752802, 0.01116804, 0.008578277, 0.006581877, 0.005171723, 0.004545205, 0.00414512, 0.004343112, 0.005238155, 0.007251939, 0.012543656, 0.028067132, 0.091342277, 0.484081092, 0.870378324, 0.939513128, 0.960926994, 0.968623763, 0.971263883, 0.972285819, 0.971898742, 0.972691859, 0.971734812, 0.97234454, 0.97150339, 0.970857997, 0.970553866, 0.969671404

rho_G
0.010542406, 0.010878976, 0.011063512, 0.010736566, 0.011681813, 0.012434719, 0.014986907, 0.020100392, 0.030356263, 0.063388962, 0.173423837, 0.568321142, 0.827791998, 0.916560468, 0.952002841, 0.964096452, 0.970590861, 0.972502542, 0.969148203, 0.955344651, 0.892637233, 0.5003641, 0.116236717, 0.047951391, 0.027873526, 0.020057963, 0.017382174, 0.015429109, 0.01543808, 0.014546826, 0.015197773, 0.014285896, 0.015069123, 0.015506263, 0.015545797, 0.016302839

rho_B
0.967865135, 0.968827912, 0.967128582, 0.965460137, 0.963110055, 0.962150324, 0.960391811, 0.958925903, 0.953890935, 0.925442998, 0.817997886, 0.42509696, 0.167036273, 0.078894327, 0.043852038, 0.031560435, 0.024170984, 0.020245519, 0.01830814, 0.016588218, 0.01602049, 0.015554808, 0.013384959, 0.012535491, 0.011199484, 0.011318274, 0.011353953, 0.012285073, 0.012663188, 0.012761325, 0.013067426, 0.013369566, 0.013427487, 0.01363574, 0.013893597, 0.014025757

The extend the primaries into the far red, just repeat the last value in each set as many times as needed.

Pardon the excessive number of digits. The last half or so of each is just noise, but I didn’t feel like editing each one by hand.

[smilebags]

Thanks for that! How many digits do you think are useful? 0.xxxx? I’ll process them with a regex and post it back here