Cycles OSL Camera - Feedback

Feature & Design Feedback
1

Hello everyone,

Over the last few weeks I’ve been working on Cycles support for writing custom cameras using OSL.
This can be used for e.g.:

  • Lens distortion
  • Realistic lens simulation
  • Panorama mappings
  • Custom camera projections
  • many more things, OSL is a powerful programming language after all

It should be pointed out that this is independent from OSL shading (and the associated performance hit), you can use the custom camera scripts without enabling that.

While there still are some limits/TODOs (see below), it’s already usable for initial testing, so I’d be happy to receive feedback on what can be improved to make this as useful and powerful as possible.

Current limits/TODOs

  • Camera scripts are only supported on CPU and OptiX currently
  • Ray differentials are broken, so e.g. the wireframe node will not work properly
  • Inverse mapping is broken, so e.g. the Vector pass will not work properly
  • Many of the builtin camera controls (e.g. sensor size, aperture, focus distance) are not yet exposed to the script, so you need to add your own parameters for those at the moment

How to test

  • Download build from Blender Builds - blender.org
  • Use CPU or OptiX
  • Set Camera to Panorama->Script mode
  • Select either an external .osl file or an internal Text datablock (like you would for the Script node)
  • Set parameters (if your script uses any)
  • Render

Here is a quick example of how to implement a perspective camera to get you started:

shader basic_perspective(float fov = 90.0, output vector pos = 0.0,
                         output vector dir = 0.0, output float T = 1.0) {
  float w = tan(0.5 * fov * (M_PI / 180.0));
  float h = w / (16.0 / 9.0);

  dir = normalize(vector(2.0 * (P.x - 0.5) * w, 2.0 * (P.y - 0.5) * h, 1.0));
}

Essentially, the script’s job is to turn a sensor position into an outgoing ray.
The sensor position (from 0 to 1) is available in P.x and P.y. If you need random numbers (e.g. for aperture sampling), you can find two uniform 0-1 variables in N.x and N.y.
The shader is expected to output a position (named pos) and direction (named dir) for the outgoing ray, as well as a transmission intensity (named T). If you set T to zero, the ray is skipped.
The script can have arbitrary input parameters, which will be displayed in the camera properties. Note that OSL parameter metadata is supported to set UI name, description, slider limits etc. for them.

39 Likes
2

Pretty cool! Is it possible to sample camera position/direction itself? or this is purely for sensor mapping? With camera data it would be ideal way to perform some screen-space segmentation into different parts of scene without tons of tedious setups

3

Awesome work you’re doing!

I have been testing the blender-4.5.0-alpha+main-PR129495.e5b8a11b3f70-windows.amd64-version and I have some notes:

When you rendering with lenses it is natural that some pixels become occluded. This is an issue with some AOV’s:

It seems like some AOV’s are sampled by the first sample, like Depth and Position. The Depth pass for example; if the first sample happens to be occluded (T = 0.0), the depth value will be registered as infinity, making the whole pass quite useless for compositing.

Passes that takes the average sample value is also affected by the discarded rays, like Normal, Mist and the Alpha channel. The Mist pass for example will be darkened towards the circle of illumination making it quite unusable for creating mist when the optical vignetting is strong.

Is it possible for the AOV’s to ignore the discarded rays and only account for the rays that hits the scene? The occluding is of course a natural effect when tracing the lens, but for artistic purposes having usable AOV’s like the Depth, Position, Mist etc is quite important.

Also, just to have it written somewhere, I said in the meeting we had that it would be nice if the Script Parameters did not trigger the shader to be recompiled when values are edited.

You have also mentioned that there will be plans to support an aperture texture, how will textures be implemented? Can we have more than one texture, like the “type texture (string filename, float s, float t, …params…)” described in the OSL documentation?

Some technical issues:

What is the correct way to force the Update Camera Script button in python?
Right now I have to have a window with the camera data panel opened, find the window and then executed the script:

context = bpy.context
    
for window in context.window_manager.windows:
    screen = window.screen
    for area in screen.areas:
        if area.type == 'PROPERTIES':
            with context.temp_override(window=window, area=area):
                bpy.ops.cycles.camera_script_update()
                break

It might just be me who doesn’t understand how to execute this properly.

Trying to set the internal camera script will work but it gives an error, same error for both lines of code:

bpy.data.objects["Camera"].data.cycles.script_mode = "INTERNAL"
bpy.data.objects["Camera"].data.cycles.script = bpy.data.texts["osl_camera_script"]

Error:

AttributeError: 'Context' object has no attribute 'camera'
File "E:\blender-4.5.0-alpha+main-PR129495.e5b8a11b3f70-windows.amd64-release\4.5\scripts\addons_core\cycles\properties.py", line 375, in update_camera_script
Traceback (most recent call last):
  File "E:\blender-4.5.0-alpha+main-PR129495.e5b8a11b3f70-windows.amd64-release\4.5\scripts\addons_core\cycles\properties.py", line 381, in update_camera_script
    osl.update_camera_script(context.camera, report)
                             ^^^^^^^^^^^^^^
AttributeError: 'Context' object has no attribute 'camera'
File "E:\blender-4.5.0-alpha+main-PR129495.e5b8a11b3f70-windows.amd64-release\4.5\scripts\addons_core\cycles\properties.py", line 375, in update_camera_script