I think you are spot-on there. But I wonder; isn’t there some special file-format for sharing this kind of data, much like the IES file-format for lamps (examples)?
I recall older versions of LuxRender having a texture type for loading spectra from text files. I’m not sure it was a standardized format, but I still have a few sitting on my hard drive. One appears to be copy of this: http://genius.ucsd.edu/Public/WaterRefModels/pope97.dat
The other is one I hand-made from a wikipedia page, that has a spectrum for white LEDs:
Emission spectra of a typical "white" (GaN + Ce:YAG) LED. Reconstructed from http://en.wikipedia.org/wiki/File:White_LED.png
Intensity is relative, it is effectively a direct color L/Le value.
wavelength: nm
lambda intensity
(nm) (relative gain)
300 0.0
350 0.0
400 0.0
425 0.4
460 3.7
500 1.0
550 1.9
575 1.8
600 1.6
650 0.6
700 0.35
725 0.2
750 0.0
800 0.0I mean there’s a number of ways you could do it. The simplest would be a table in something like csv format which could be automatically read in as needed.
For a smaller footprint, some non-human-readable binary format could be picked instead.
And Blender could also support several formats if need be. It’d be hard to do that with something as convenient as drag-and-drop, but using a python script importer, pretty much any format would be viable.
That being said, different approaches ask for different representations. For instance, complex ior data is gonna have three columns (wavelength, real part of the ior, complex part of the ior) and if flourescence ever happens, it’s gonna be a pretty large matrix.
1 Like
Yep I’ve also created a file format which seems to work well enough, I’m sure anyone who has dealt with spectral data has probably come up with their own. It would be really nice if the loader could use a plugin architecture such that new file formats could be added easily.
As for complex data with multiple different spectra, I would probably suggest just splitting the columns out into their own files. Keep in mind these files would be authored once and used many times so the usability of them seems like a much higher priority than their ease of creation.
I’ve had thoughts of developing a standard for remote resource discovery and consumption, so that things like ‘asset servers’ could be added to Blenders resource manager once it exists, I think this would be a really powerful tool to have, and might allow us to make better use of all of the generosity people have put in to making textures, environments, models, and hopefully in the future, spectra.
4 Likes
I don’t think splitting out columns like that is a good idea. Keep it all together. It wouldn’t be much harder for a file importer to expect values in triplets rather than pairs. But accidentally pulling two unrelated files would quickly lead to nonsense.
Also, these files already exist. For instance, here, in csv or tab-separated txt:
https://refractiveindex.info/?shelf=main&book=Au
Having to split those into two when they already exist as one seems really silly to me.
It’s just a different kind of data.
3 Likes
I guess it depends on how inter-related the spectra are. It’s a trade-off for modularity.
That being said, for sure there are use-cases for things being bundled together, it would be great if we had the ability to load in named spectra from files (which wouldn’t be all that difficult to write the spec for, not sure about the Blender implementation) and somehow expose those names in the UI like in a dropdown or with multiple output colours from the node.
2 Likes
As far as I know there are, for now, only two relevant formats: Wavelength + spectral value at wavelength (this will work for transmission, absorption, reflection, emission, and scattering spectra), and Wavelength + IOR real part + IOR imaginary part (for complex IOR)
In the (far) future, if either fluorescence or, somehow, polarization becomes relevant, you’ll also want an entire matrix (for spectra with fluorescence - imagine having that handled one column a file, one file for every input wavelength. That would be ridiculous! But to be fair, yes, far future) or something like anisotropic IOR for birefringence and such which probably will also require three columns? Maybe four? Or if it’s complex IOR it might be as many as seven columns? Like, it’d be wavelength, and then, for each optical axis, a set of complex IOR values.
For now, though, two and three column support will suffice
EDIT: I just saw that website I linked above actually offers two different versions: Either the k (the imaginary IOR values) are available below the n (the real IOR values), or you can get it all in a table plus extra information on top.
I guess that would be one way to make it just two columns, but even then you’d have to support the switch from n to k if that’s how you prefer it.
Imo the three column version is nicer simply because it’s less redundant: Don’t have to specify the same wavelengths twice.
1 Like
I got an error while compiling @troy_s 's branch:
/intern/cycles/kernel/osl/../../util/util_types_spectral_color.h:31:39: error: ‘make_SPECTRAL_COLOR_DATA_TYPE’ was not declared in this scope
It’s caused by a problem with the macro CONCAT.
I fixed it by nesting the actual concatenation inside another macro like so:
#define CONCAT_HELPER(a, ...) a##__VA_ARGS__
#define CONCAT(a, ...) CONCAT_HELPER(a, __VA_ARGS__)
This ensures that the parameters passed to CONCAT are expanded before they get concatenated, so (for example) CONCAT(make_, SPECTRAL_COLOR_DATA_TYPE(f)) will expand to make_float8(f) instead of make_SPECTRAL_COLOR_DATA_TYPE(f)
I made a pull request on github
4 Likes
Awesome, thank you! I’ll take a look.
Awesome! This was the Linux compilation issue I was not able to fix, since my knowledge on C++ directives is very limited. Thanky you very much @alisealy.
3 Likes
Oh so does that mean automated Linux builds are a go now?
Asking here since it’s related to the topic: i would like to capture spectra data to make tests later on this build, what kind of instrument / spectrometer do you suggest to buy for that?
Maybe @troy_s knows better.
Getting into the rabbit hole of measuring spectra can be quite a jump. Plenty of things can go sideways. Given that very few folks already are integrating proper management into their plugins and such for Blender, I’d at least suggest some caution. It’s wiser to get folks onto proper management in the RGB tooling first, as Blender is a broken mess. Equally broken plugins and UI aren’t making spectral even remotely viable.
In terms of hardware, it depends on the budget and contexts. The most inexpensive available is the 11nm X-Rite i1 Studio at 440$ USD. The next class up is the ASeq Instruments LR1 at 750$ USD, bare bone model.
4 Likes
seems a good one but, it has a spectral range that goes from 400 to 700nm at 10nm steps, which is not ideal for the spectral implementation that considers wavelengths from 360 to 830nm.
The X-Rite i1 has a 380 nm to 730 nm x 10nm spec, so it’s a little better (with the downside of paying $440 instead of $300). Those extreme wavelengths <380 and >730 have very little impact on human color perception. To work in a 360-830 environment, I’d recommend a simple extrapolation treatment for the extremes. Or spend a LOT more money!
I actually have the i1, it has never crossed my mind to use it for recording spectra though! I guess I will just need software capable of using it for something other than screen calibration.
fwiw with the spectra I got so far, where not all of them got to cover one of the extremes or the other, I already did some guesstimating interpolation, and it worked pretty well.
10nm resolution also seems pretty fine for most materials. Although some have very sharp, steep falloffs very suddenly, and if those fall in between the 10nm gap, and moreover, they happen in the middle of the visible range (so more around 550nm rather than 380 and 730), I found that that can affect hue quite a lot actually.
The spectra I already converted from absorption to transmission can be found here:
By now they include melanin (two of the three types we have), and chloropyhll, as well as one experimental take on mixing water with chlorophyll to obtain something more like “natural” water. - I only eyeballed stuff there though. It’s not based on any real measurements, and it might be that the chlorophyll content is completely off.
Still, it gave me the more turquoise look I was hoping for compared to the more blue and, deep down, even purple look that extremely pure water apparently has (according to the spectral data)
1 Like
I vaguely recall reading that CIE recommends 5 nm resolution in most applications. I can’t seem to put my finger on that document, though.