Thoughts on making Cycles into a spectral renderer

Something just came to mind - I believe I updated the CMFs to a newer data set but we might not have replaced the sRGB to spectrum lookups. If there’s a mismatch there it might mean a slight colour shift. I’ll have to take a proper look once I have some time again.

That’s a really helpful file, thanks

The good old MacBeth color chart. Although fine for not too pixel-perfect use in photography, it is not spectrally standardized at all. If you want to have science-quality physical reference material, you’ll need certified calibration tiles. Way too expensive for this project, I expect. Your best option is to have some MacBeth chart, and designate that as your reference material and keep it safely stored and locked-up.

But, what you really need is a digital reference that you, me, everybody agrees upon and uses for their digital tests. Again, any reference dataset works fine, as long as you pick one and share it with your peers (which MacBeth unfortunately didn’t like to do). Only when you want to compare your renderings to the physical world, you’ll need to go through the pain of producing a physical sample, measure it, and link it to your digital reference.

So name it, tag it, brand it, and stick to it.

Yeah based on the short amount of research I did to find that spectral data, I concluded the same. Thought I might as well though.
If nothing else, I now have a neat library of 24 spectrally plausible colors

I mean any number suffices to approximate this, though is always just an approximation.
The spectral branch doesn’t work with more color channels at all. It statistically approximates the result on a continuum which is guaranteed to converge to the right result.

That said, we don’t have dispersive BSDFs as yet. No wavelength-dependent IOR for now. That’s gonna come later.

We’re also not sampling invisible light right now. Nor do we have a way to get materials to take in a spectrum and transform it into another spectrum of equal energy (fluorescence/phosphorescence)

Extending the spectrum into UV and IR is interesting an I’d love to try making a flourescent BSDF but I’m not sure whether it breaks some Cycles rules, since it is direction-dependent.

@kram1032 said it better than I could regarding how to create dispersion. The plan is to not split the ray but to evaluate all wavelengths while using the path of the first ray. This will mean noisy dispersion if 0 roughness is used, but if there’s overlap in the paths of each wavelength then you’ll progressively get more and more benefit from the multiple-wavelength sampling.

Instead of being split, think of it as just taking one of those possible paths per sample. As you get more samples you get closer to the same result. You still end up with wavelength-dependent paths. As you say, even currently caustics are incredibly noisy so adding yet another dimension to the rendering equation is only going to make this worse.

The benefit of the approach I’ve described is that if a path has a non-zero probability for more than one wavelength, that ray will be less noisy than it would be if it was just 1 of 8 split rays, saving on intersection calculations.

Yes, each ray will take a different path to one another. The only difference is whether or not a single ray becomes 8 when it hits a wavelength-dependent material or not. We haven’t implemented this yet so it isn’t final, but it doesn’t seem practical to split the rays when hitting a wavelength-dependent interface.

I tried rendering the MacBeth Chart again with RGB and spectrally defined colors side by side. Three problems with this:

  1. I’m pretty sure these are not the same charts. Apparently the RGB data is from a 2005 chart whereas the spectral data is from at the latest 1997.
  2. in sRGB, the Cyan swatch (right-most column, third row from top) is actually out of gamut so the R component is clipped to 0 (I think this is kinda visible somehow, in that the hues of that swatch between spectral and RGB don’t quite match up, but maybe that’s just me convincing myself that that’s the case)
  3. most importantly, very obviously the luminosities don’t match up at all (the hues are pretty darn similar though) - I think what’s happening is, that the table I found is using gamma corrected sRGB whereas I actually put in linear colors? Not sure. I’m a bit surprised that the hues are that close then. I thought the gamma correction would also mess with that.

Here is the outcome: (top left spectral / bottom right RGB-defined)

and for convenience, here is the modified file. It’s almost identical to the one above, except every swatch is split in two

2 Likes

Interesting how similar they are, hey. I think we just need to define what we’re going to use spectrally and stick with that. We can always manually integrate the spectra to get the exact XYZ value of each patch in a spectrally-defined chart, then convert them to whatever space we need.

Indeed:

… and sRGB is in fact well-defined if you know what you’re doing. From the neutral colors in @kram1032’s comparison, I’d also guess that it is a simple gamma issue.

Would it be useful if I go and calculate the correct sRGB from the spectral data @kram1032 uncovered? In theory, we expect spectral-Cycles should to give no systematic deviation from RGB-Cycles, right?

that would be useful to me (though note that the cyan patch, I just tested, is indeed out of gamut. The red channel there goes to about -0.018 negative, which you can’t tell from the finalized png)
I have now added a perfectly white diffuse material to callibrate the scene with what Cycles would consider white as best I could:

exr result: https://www.dropbox.com/s/yh8sfn9nixvneyq/Spectral%20and%20RGB%20Colors%20white%20stripes.exr?dl=0

Note the light source isn’t E but 6500K so presumably that should come out at exactly white, right?

Blend File: https://www.dropbox.com/s/ob5q04t9bb2kuz2/split%20MacBeth%20Chart%20alternating%20background.blend?dl=0

I remember having seen discussions about that a while ago in this thread. It all depends on the colormanagement display-device configuration. I think it is best to export the renders to EXR floating-point format with the box for “Save as render” checked to bypass the display-device entirely (please verify this).

In our case, If we calculate sRGB values from the spectra using D65 (mandatory for sRGB) and put linear sRGB emitters side-by-side to the spectral reflective patches and light the reflective patches with D65 and render everything using Spectral-Cycles, then the emitters and reflective patches should yield the same RGB values in the EXR file (you might need to tune the light intensity). If we replace the spectral reflective patches by RGB-reflective patches (using the same linear sRGB as the respective emitter) and render everything using Spectral-Cycles again, then the same resulting image is expected. If we then render everything using RGB-Cycles, again the same resulting image is expected.

The above method avoids the uncertainty we have with Blender’s colormanagement.

(EXR also saves negative RGB values, which is nice)

that’s precisely why I linked the exr above

Finally, I got some time to work on this overview-scene. I incorporated some of @kram1032’s ideas too:

There is a lot to see in this image. Every block has special meanings and there is lots of room to extend this. The complete rendering took over 4 hours, but I exaggerated a bit on the sample count and resolution, I think :stuck_out_tongue: . The blend and exr-render from regular Blender 2.91.0 are here: https://we.tl/t-mJfZM2IYqv. Here are some explanations:

A) Refractive sphere with point light at its center in a holdout see-through box. The sphere has a glossy BSDF of type GGX with a roughness gradient from 0.0 through 1.0 in counter-clock-wise direction. I wonder what the dark band is through…
B) Refractive sphere with point light at its center in a holdout see-through box. The sphere has a glossy BSDF of type Beckmann with a roughness gradient from 0.0 through 1.0 in counter-clock-wise direction. Same dark band…
C) Refractive sphere with point light at its center in a holdout see-through box. The sphere has a translucent BSDF.
D) A thin 12-sided cylinder (dodecagon) with each side a different color (Red, G, B, C, M, Y, and gray scales) one-directional transparent BSDFs which are lit from the inside by a 2500K black-body point light. The floor is diffuse white and the roof is holdout see-through. The outer wall is (non-see-through) holdout. Thus, the colors you see are the single-scatter spectrum of the point-light in the center, surrounded by the spectra of each of the transparent transmission colors.
E) Same as D, but 6500K.
F) Mirror-surface tubes with white emitter at the north-end and holdout at the south-end. A see-through slit allows us to see its internals, but light cannot escape. The volume is a scattering-only white of density 5, 10, 20, 40, and 80 respectively. The mirror-surface extends each tube its sides to infinite west-east width and top-bottom height, such that we effectively render an infinite wide/high slab of finite thickness (north-south) where we observe the amount of light at each depth-step.
G) Same as F, but with constant density 10 and anisotropies -1, -0.5, 0.0, +0.5, and +1.0.
H) Same tubes as F, but no volumetric BSDF and just simple red (0.5, 0.0, 0.0), green, and blue diffuse surface instead of the mirror, hence not infinite slabs. Towards the right, these should merely show not only darker- but also higher saturated colors in Spectral-Cycles.
I) skipped for readability.
J) Integrating spheres with small see-through hold at the top and an equally sized diffuse white patch at the bottom. Inside, a white point-light is included behind a baffle. Rest of the surface is diffuse red, green, or blue like at H. (The holes should show homogeneous color, but don’t. There is something wrong here! The spheres just may be too small since those at O through U are fine)
K) Same tubes H, but with opposite colors in RGB-space - dunno, seemed logical to include.
L) Same spheres J, but with opposite colors in RGB-space - dunno, seemed logical to include.
M) Same spheres as with J, but its surface materials are achromatic diffuse BSDF stepping from 1/8 brightness at the north-end to 7/8 brightness at the south-end. The hole and white patch are a bit wider. The color from the central opening should be analytically calculable from the surface brightnesses. You can just see the very bright white dots at the edge of the openings which are the point-light sources (it is of no concern to the calculations, it is just annoying and the openings should be a little smaller to avoid this).
N) Same spheres as with M, but surface materials are achromatic glossy Beckmann BSDFs with 0.8 brightness and roughness from 0.0 at the north-end and 1.0 at the south-end. The color form the central opening should be the same for all due to the energy-conservation requirement on the surface BSDF.
O) From north-to-south: teardrop and integrating sphere having the same anisotropic Beckmann BSDF with roughness 0.3, then an integrating sphere and teardrop with same BSDF but isotropic instead. The teardrops are simple bulbous convex (?) shapes.
P) Same as O, but with GGX BSDF.
Q) Same as O, but with Ashikhmin-Shirley BSDF.
R) Same as O, but with MultiScatter-GGX BSDF.
S) Same as O, but with Glossy Toon BSDF.
T) Same as O, but with Diffuse Toon BSDF.
U) Same as O, but with Diffuse BSDF.

@smilebags: is this a useful progression?

6 Likes

Looks good, I think this is a great scene to test regressions, thanks!

Oh these are great! I love how comprehensive this is!
It’s neat how clearly visible the relative energy loss of the various glossy materials is. Beckman seems to come pretty close to correct, but without using a color picker to confirm that that isn’t just my eyes playing tricks to me, Multiscatter GCX comes much closer to the diffuse material’s performance still.

I’m not quite sure I understand how to interpret A and B yet, but if these are refractive spheres, it might be you see the light’s reflection distorted through the sphere and that’s what causes the outer ring? If so, this would change if you gave it different IORs.

Indeed it does. I will try to understand this one further and maybe improve the local context to improve others’ understanding before I finalize it. I will also add more parts this week like SSS which I totally forgot :open_mouth:. Then I will add proper descriptions inside the file and some annotations. Thereafter, it should be stable.

try reducing the bounce depth to literally 0 or 1 (whatever the minimum is to have these show up) - like, does that ring appear at the same time as the bottom or is it actually a bounce deeper?

Good idea, I will try.

I mentioned earlier that I expected the small spheres to be too small, but I found that the point-lights inside them were actually too large. I have reduced their size to zero which fixed that problem.