Taking a look at OptiX 7.3 temporal denoising for Cycles

As a side note, if you need help with building Blender, you’re probably more likely to get a quick response out of someone over in the “Building Blender” chat channel over here (you sign into it using the same account as the one for this site): Blender Chat

However, you can still post any questions here if you want.

In theory setting up a build environment for Blender is relatively straight forward if you follow the guide. The thing I had the largest issue with while doing my first compile last year was with CUDA and OptiX.

Part of setting up CUDA and OptiX requires you modifying the cmake file for that Blender build. I initially tried to do this by opening the cmake file in a text editor but couldn’t figure out how to get it to work. Turns out there’s a easier way to do it by using a tool known as “cmake gui”. It was available on my Linux distrobution, but it may be different for you.

This issue may be Linux specific:
The other issue I had was to do with CUDA complaining about the wrong GCC version installed. There are two ways around this, install the gcc version CUDA expects or modify CUDA to stop complaining. At the moment CUDA 11.X is designed to be best used with gcc 9 and 10 (not sure about the latest CUDA 11.4). Blender is designed to be compiled with GCC 9.3.X. I personally went down the route of installing GCC 9.3.X and using that. However, if you want to go down the route of modifying CUDA to not complain about a higher gcc version, then you can do so by modifying a file located here: /usr/local/cuda/include/crt/host_config.h (if you can’t find it, try searching for it in other directories)

Search for the region containing this (searching for the term gcc finds it quickly):

 #error -- unsupported GNU version! gcc versions later than 10 are not supported! The nvcc flag '-allow-unsupported-compiler' can be used to override this version check; however, using an unsupported host compiler may cause compilation failure or incorrect run time execution. Use at your own risk.

#endif /* __GNUC__ > 10 */

and modify the end bit so it says something like:

#endif /* __GNUC__ > 20 */

(just change that last value to a value higher than your GCC version)

I hope everything goes well.