Just recently it became possible to receive textures, and for OSX I have a prototype working. The current implementation grabs the byte buffer from the received texture and passes it on via the gpu.types.Image pixels method to an Image. This is very slow. More elegant would be to pass on the received texture directy without going via the GPU. (When sending from blender this is how it is done)
A superficial check on the blender code indicates it is possible to extend the python wrapper with a write method, though I am not sure if that would actually help.
By using the current implementation, it is possible to us the Image inside a node shader and thus using the regular render pipline.
What I don’t know: if the shader would use the new texture if the Image’s GPUTexture is changed this way
It is clear, that this approach can never work for the Cycles renderer, but it might work for EEVEE.
Question is: is it feasible or is it a pipe dream?
From what I understand, you have a Metal texture handle, and you want to use the contents of that as an image texture in EEVEE as efficiently as possible?
EEVEE also has an underlying Metal texture, wrapped in a GPUTexture. So I guess you’d want either:
Access to EEVEE’s Metal texture handle so you can do the copy yourself with the Metal API.
Ability to wrap your Metal texture handle in a GPUTexture, and some API function to copy data between two GPUTexture instances.
From what I understand, you have a Metal texture handle, and you want to use the contents of that as an image texture in EEVEE as efficiently as possible?
Thats correct.
Though I would like a solution that works for both OSX (Metal and maybe OpenGL) and Windows (OpenGL - I think). On Windows the tool is called Spout and it should be possible to do the same thing. In my addon the sending part is already implemented for both platforms.
In Metal/Vulkan handles are raw memory addresses and should be treated with care as their are no bounds checks on those platforms as they don’t do any runtime validation at all. To get this working on those platforms would also need the source textures to be created with the correct usage flags.
Lets say that those flags are setup correctly by the libraries and tools you are using. A direct copy of pixels isn’t sufficient as the texture format can be different, than requested just based on its usage (void GPU_texture_copy(GPUTexture *dst, GPUTexture *src); isn’t sufficient). This can be solved by exposing blit functionality. Currently we don’t have an API for this and each backend (OpenGL/Vulkan/Metal) does its own thing.
Wrapping a handle sounds easy, but might need more information how it was created, otherwise internal state of the GPUTexture might not work. For Metal specific blitting process can be done via a compute shader, which would select the correct shader based on some internal state. If this internal state is incorrect it may fail.
Next to that mipmaps needs to be updated, otherwise the rendering will be incorrect when looked at from different distances in EEVEE/Workbench. This exists in C/CPP, but isn’t exposed to the python module.