Cycles for Gaffer

Hi all,

I’m not sure if this is the best place but I’d like to start a dialogue about my adventures in porting Cycles to Gaffer!

Consider this as a WIP that I haven’t tried compiling, it started life a few months back before the refactors like how motion steps are applied:

I’ll be updating this as I go, currently just blocking out how to convert meshes and things as well as dealing with sessions and scenes in the Renderer.cpp which I think isn’t there just yet. I’m aiming for CPU and OSL support for the first go, and look into the GPU side afterwards.

Cheers all!

8 Likes

Hi,

Awesome, always great to see more integrations being worked on!

We’re happy to answer any questions you might have about how to use the API.

2 Likes

Hey Brecht,

Thank you, so far it has been pretty straightforward to map and I am liking that just about everything stems from ccl::Node with sockets which simplifies a lot of the integration, and everything else which are purely just param structs, I am treating as global strings to match Gaffer’s convention eg. “ccl:session:shadingsystem” will map accordingly to the right struct/object.

I guess the one thing that is a little difficult to grasp is how to do instancing. I was going to make an instancing cache much like the other backends of Gaffer which will store a collection of unique ccl::Objects but their ccl::Mesh pointer will point to the mesh that is meant to be instanced, with a check to see if it is being micropolygon-diced and making sure it will be unique and not instanced in this scenario. Is that the correct convention, or do I use the particle system somehow?

The blender sync and stand-alone code is extremely helpful though which has guided me for the most part. I’ll ask some more, probably when I get to how to implement AOVs or “passes” and the shading system properly beyond OSL, which Gaffer can manage for the most part itself and just offload the resulting .oso to Cycles (I’m imagining here).

Cheers!

That’s indeed the instancing convention. Micropoly is not quite production ready yet and we’d like to make the instancing work for that in some way, but at the moment you do indeed have to create multiple meshes.

Hey all,

I have a progress update, it does work and I am getting pixels in Gaffer out of Cycles… :slight_smile:


There are still a ways to go and I have the issues page on github for a rough outline of what’s left:

I do have a question for you Brecht, I am a little confused about the curves and particle system in Cycles. I’ve implemented right now a straight internal conversion of curves from Gaffer (IECore::CurvesPrimitive) to what curves.h has to attach to an object (untested though, just some basic code), but I am wondering is that more for things like grease pencil, and I should instead make a particle system via IECore::PointsPrimitive and then somehow bake data to that?

And IBL/world/dome rendering, I think I will implement this as supporting multiple skydome lights and just make a shader which just adds a bunch of textures together so that there are transforms to manipulate the wrapping, not sure yet. Or maybe just implement this exactly how Blender does it as a global.

Cheers!

8 Likes

Hey Alex, SUPER stoked about this! Gaffer’s handling of look-dev and lighting is amazing. Really cool to see the first steps of Cycles for Gaffer!

Great to know you are working in a gaffer implementation!

Will Cycles inside Gaffer help to render massive scenes? (I mean in comparison with rendering them inside Blender for example)

Like different scene structure or memory handling or something similar to be able to render billions of polygons even in a 64Gb system.

Cheers!

1 Like

I guess the advantage might be that it doesn’t need to load the scene in Gaffer itself which Blender might need to do and Cycles just needs to load the data. Speed-wise, the conversion step also should be multi-threaded and I’m trying to make already converted geo to be persistent which I think is still WIP for Blender 2.80.

2 Likes

The Cycles Curves primitive is for rendering hair, so I think what you’ve done is correct. There is no points primitive in Cycles currently. If you need a points primitive it would be done by making a sphere mesh and instancing it. Grease pencil is not rendered by Cycles (or EEVEE), it has its own rendering code.

I think combining it all into one Cycles background shader is fine.

Pretty interesting, do you have a github or something similar to discuss about this? I don´t want to mess up this thread with things not related to development, but I would like to ask you some other things :slight_smile:

Cheers!

2 Likes

Thanks for the reply Brecht. I think the confusion I have here is that there’s also a particles system which I was going to use to convert the cortex points primitive into, but I thought it also works as an instancer for hair somehow because I read in there there’s a CurvesParticleData struct, but I guess that’s just to ease the Blender addon to ultimately convert that to standard attributes?

Btw are you able to update the standalone repo again for me? :slight_smile: I’m curious to know the git technique you use to update it from the Blender repo.

JuanGea - yep: http://github.com/boberfly/GafferCycles you can use the issues page if you want. It also gives a good overview of what to expect if you dare choose to compile it right now… :slight_smile:

2 Likes

ParticleCurveData indeed should actually be part of the Blender exporter, not sure why it is in the Cycles core code.

I’ve updated the standalone repo now. We use a script for it:
https://developer.blender.org/diffusion/BDT/browse/master/utils/cycles_commits_sync.py

2 Likes

Hey Brecht,

Cheers for the info I managed to get some time to try out that script here, but I’m not sure if it is working as intended, I’m getting way too many false positives and a patch bomb of like 743 patches…

I changed the script so that:

CYCLES_START_COMMIT = b"39acdd14"
BLENDER_START_COMMIT = b"3866161d"

Which is a commit from 12/2/2019 so the script can work it out from there. Not too sure what went wrong, perhaps the blender2.8 merge into master did something funky?

Cheers!

1 Like

Yes, we haven’t merged yet from 2.8 in master, only from the blender2.7 branch. This will require some manual work.

2 Likes

Oh right I’m getting better results with matching against blender2.7 thanks! One snag with one of the patches though which removes the OpenCL mega kernel I’m looking into. Cheers!

1 Like

Cheers Brecht, I have it here if it helps you (the rename + modify of that one patch made it go haywire, but I was able to hack the patch file to not rename, apply it, then rename & add, and git am --continue from there).

1 Like

Thanks, that should help.

1 Like

Hi Brecht,

I was just implementing subdivision support with good success so far for adaptive mode. I’m just wondering though as it isn’t too obvious here, but how do you determine to use uniform subdivision instead? Is the tickbox here: https://docs.blender.org/manual/en/latest/render/cycles/settings/objects/adaptive_subsurf.html more or less letting the built-in blender uniform subdivider be responsible for that when adaptive is disabled, meaning there’s no built-in uniform subdivision in Cycles itself?

Cheers Brecht!

1 Like

That’s correct, there is no uniform subdivision in Cycles itself currently.

It wouldn’t be very hard to add, just using a fixed number of subdivision for every edge rather than determining it adaptively.

1 Like

Hey Brecht,

Yeah I was thinking it shouldn’t be too hard to implement, I’ll look into it and send a patch across when I get the chance if Blender wants to use it as an option, or just for Cycles standalone. I noticed the current adaptive code isn’t multi-threaded also, I’ll see if I can figure that one out also.

Cheers for the info!

2 Likes