Is there interest in supporting a Metal version of Cycles for MacOS?


#1

I would be interested in helping to port Cycles to MacOS using Metal (since I’ve heard OpenCL problems seem to have caused MacOS GPU support to be dropped). I’m an experienced graphics programmer, and have been implementing a lot of compute kernels on MacOS with Metal recently (though mainly for cryptography and AI). I did a lot of CUDA a number of years back. I haven’t really looked at the cycles code so far so I would need a bit of time to get up to speed (and/or some assistance).

I’m starting to get some experience with Vulkan compute as well, though that hasn’t been a priority yet. But it should pay off if a port to Vulkan is desired as well.

FYI, up until it was disabled Cycles GPU rendering on my Mac was working fine as long as I disabled the AMD GPU (the Intel one worked fine). Sorry to see it go.


#2

Hi,

there has been a similar discussion about Metal for Cycles on the mailing list. Here’s a link: https://lists.blender.org/pipermail/bf-committers/2018-December/049720.html.

In summary, this is what Ton said:

[…]
But if you want to help us with Metal:

Cycles has been designed from the ground up to support multiple graphics
APIs.
It has good Cuda and OpenCL support for that reason. Here it’s possible to
add Metal as well. Cycles code base is much smaller and much better
designed (and newer :slight_smile: than Blender code.

Why not volunteer first to help with Metal for Cycles? If that project
looks like it works well, we (and you!) also get more insight in the
complexity of having this kind of work done for Blender. Adding Metal in
Cycles will take 3-6 months of work already.
[…]


#3

Some info on the GPU compute stuff in Cycles:

Most of the kernel code is in header files that are shared between the CPU, CUDA and OpenCL versions. These headers are then included into architecture-specific files which implement the entry points and basically act as stubs around the general functions.

For example, kernel.cu defines the kernel_cuda_path_trace kernel, which just figures out which pixel to process and then calls kernel_path_trace, which is defined in kernel_path.h for all three platforms.

To be able to write platform-independent code, we use a bunch of generic preprocessor defines that then are defined to the platform-specific keywords in the kernel_compat_*.h files (e.g. ccl_device is static inline on CPU, __device__ __inline__ on CUDA and nothing on OpenCL).

Then, each platform has a host-side implementation in the form of a device_*.cpp file which implements the Device API and handles the platform-specific calls.

Therefore, for getting Metal support into Cycles, you’d need to first implement a kernel_compat_metal.h and kernel.metal, then get the Cycles kernels building (you’ll have to e.g. tweak some of the util headers I guess) and then implement a device_metal.cpp.


#4

We are interested, with the caveat that we need a developer to maintain this for the long term. The reason we dropped macOS OpenCL is that no one was maintaining it, fixing bugs or helping getting new features or graphics cards to work with it.

For me personally it is not a priority given the smaller percentage of users that would benefit from this, and Metal not being an open standard. But if other developers can do it in a way that doesn’t add too much burden to the rest of Cycles development, it would be great.

Vulkan compute as far as I know does not yet have good support for a programming language with pointer support. This makes it difficult to port the Cycles kernel in a way where we can still share code with other platforms.


#5

Ok, starting to read code. Metal has some ugliness because there isn’t a pure C binding (there’s Objective-C and Swift, which might require wrapping up and build logistics).

Great info @lukasstockner97! Are there any docs on Cycles project organization, build setup, integration into Blender, etc…? Do I need to look at anything outside of internal/cycles?

Also, is there a better place to be having these conversations?


#6

This is the right place for developer discussion.

The Cycles code docs are here. Probably you would only need to make changes in intern/cycles.
https://wiki.blender.org/wiki/Source/Render/Cycles

For build setup, you just build Blender as a whole.
https://wiki.blender.org/wiki/Building_Blender


#7

Thanks for the wiki link.

I have the build working. I guess I meant documentation on the build setup since I would probably have to get Cmake to build Metal Shading Language libraries, etc… I’ll have to read up on Cmake. Somehow I’ve managed to never learn cmake after I went from C/C++ with imake and autoconf to languages/environments that didn’t require cmake.

So, for example, where would I find how cmake knows to run nvcc (I’ve seen where it is discovered in a .cmake file, I think).


#8

There is no documentation for anything that specific. You would search the CMakeLists.txt files for “NVCC” and see how it is used.


#9

For me personally it is not a priority given the smaller percentage of users that would benefit from this, and Metal not being an open standard.

As a Mac user it’s tough reading this. Apple Metal may not be open source and Mac users may be a smaller percentage of users but Blender is meant to be cross platform. It it’s not a priority for you, is it possible Blender could be Windows only at some point?


#10

No. This is only about a feature to accelerate rendering, not about working at all, which makes the choice easier.


#11

Thanks for your reply @brecht and for the clarification.


#12

And thanks @ChuckOcheret for looking into this.

Hope you find good homes for your expertise in solutions for Blender.


#13

Believe it or not, I’m making some slow progress. Since I work mostly on a Mac (no CUDA/Nvidia or OpenCL for Blender), I have to move to my Windows box to try and understand what’s going on in CUDA code. I can build on Windows from the command line but I can’t get a Visual Studio version to build with CUDA and can’t get Visual Studio to successfully debug Blender. Next I will try to get my crummy Linux laptop up and running to see if I can understand the OpenCL version instead. Or maybe I will try to reenable OpenCL on a local Mac build since ti was working perfectly on my Intel GPU (my Metal experiments have my build-in Intel GPU about 5 times faster than the default AMD GPU for similar compute problems - e.g. a modified Hilbert R-tree as a dynamic BVH for another project). So I’m just reading code at this point (both Blender and Cycles). But I am learning.

It would help so much to be in a room with someone knowledgeable in the codebase.

Happy Holidays!


#14

I’d be happy to help on windows, but you gotta do better than ‘I can’t get it to work’ What is the problem?


#15

Lol. I wasn’t asking for help tonight. But…

There are two things I seem to be having problems with.

1 - When I build on the command line, I am able to ‘make release’ to get a version of Blender that recognizes both of my Geforce GTX 1080Ti’s. Any other target doesn’t cause the GPU’s to be recognized. So that’s a fine start. But when I build on Visual Studio (I’m a complete novice with that - well after 10 years of not using it) I can run a version of Blender that doesn’t recognize my GPUs.

2 - When I attempted to debug using Visual Studio, I was getting some other error and it wouldn’t run at all. Unfortunately, I blew away the setup to start from scratch so I need to do it over again for VS before I can tell you what the error was.

And I need to go be with family now. I’ll be back later after I get the error message again.

Merry Xmas.


#16

ayyyyyy, you’re making it hard to help… we went from ‘i can’t get it to work’ to ‘it wouldn’t run at all’ even an error message or screenshot would have helped here.

1 - Only release compiles the cuda kernels, so this is expected behavior.

2 - I’m guessing here, but it sounds like you did a debug build not build the ‘INSTALL’ project

image