Camera scene intersection: from pixel to ray

Archive Python API
1

Hi,

I would like to get the intersected objects and depth from specific pixel.
Currently, I can get only the first intersection by using depth map but I want also the occluded objects (behind the first object), i.e list for specific pixel with all the intersections and depth.

For that, I thought to use ray_cast for each object and get the intersection but I need to get ray from pixel first.

I have 2 questions:

  1. How can I get ray direction from specific pixel?
  2. The ray_cast function is for specific object, is there function that finds intersection it for all the objects in the scene efficiently?
2

If you are referring to the ray_cast python function it works based on the direction you shoot it, it’s not object based…
Here i used it for the Auto Focus feature in my add-on:

1 Like
3

Nice, thanks.
It worked:

ray = context.scene.ray_cast(context.scene.view_layers[0], obj.location, obj.matrix_world.to_quaternion() @ Vector((0.0, 0.0, -1.0)))

So you answered me for question (2).

Now remains question (1),
I want to do the intersection for each pixel in camera plane, so I have to get the direction and origin for each pixel.
How can I get the ray direction and origin from specific scene?

1 Like
4

Yeah not sure here… probably playing around with that Vector matrix should do the job pretty easily… based on the render resolution width and height the user sets maybe…
Just try at this point

5

How you get the vector from a position on the film is by creating two vectors in world space which follow the camera x and y axes, and then adding scaled components of each to the view vector, and normalising at the end if that’s necessary.

2 Likes
6

@smilebags
Thanks for the reply.

The question is more basic,
How do I get the position on the film?

If I’ll have it (or at least the corners and then I can add the scaled components as you said), I will be able to calculate it.

7

OK, I think I figure it out.

For perspective camera, I can use the Field of View and then to calculate the point on the film according to:
theta = field of view

corners:
(tan(theta) , tan(theta * aspect_ratio) , -1)
(-tan(theta) , tan(theta * aspect_ratio) , -1)
(tan(theta) , -tan(theta * aspect_ratio) , -1)
(-tan(theta) , -tan(theta * aspect_ratio) , -1)

For each pixel, progress it according to image resolution as @smilebags suggested and normalize it.

then, transform it to world coordinate:
cam.matrix_world @ vector

For Orthographic Camera:
The ray is always (0,0,-1) and the original is changes between
(-orthographic scale/2 , orthographic scale/2,0)

Again, should transform it to world coordinate:
cam.matrix_world @ vector

If someone has another insights, so please let me know.

Thanks for the answers.