Hey, After watching some of the Bcon 2018 videos it seems AI is now being accepted as something Blender should support.
Would it be possible for whoever builds Blenders libs in the SVN to create some pre compiled libs for tensorflow so we can start looking at adding direct tensorflow support in Blender, Python,C,and C++ support would be the best support but even just the C or C++ api support would do to start with.
Cheers J
This generally works the other way around, devs develop with their own libs locally, and when the patch is close to landing I (or any of the other platform devs) script the libs for the SVN repository (embree got added after it made it to master even)
It’s not worth investing time in this until we know which framework would be used and which parts would be need to be bundled with Blender. If you’re a developer working with Tensorflow, you’ll want to build your own version optimized for your CPU or GPU anyway, since fast training performance is important.
I only use blender on windows right now, So will use my own version for now, just thought that maybe to help others who want to start adding AI related work would find it easier to download pre compiled libs from the svn ready to go.
e.g tensorflow1.1 cpu no avx cuda 9.1
tensorflow1.1 cpu no avx cuda 9.2
As multiple libs would be needed to support the exact gpu cuda version or cpu instruction set, that if we were to add these pre compiled libs many version would be needed.
Yep understood, I think a disscusion on what platform IS to be used could atleast give a basis for what to use to be able to add our AI’s in future to blender. What platform do you think is best here?
I already compile my own version of tensorflow, the good thing about compiling to libs in mordern tensorflow is it creates a few.lib and .dll for the whole release of that tensorflow version (not just final infernece ability,but the whole dev environmenty) of course a few headers etc are also included to use within the C++ framework for example like im using for my tests, but dev and final optimized gpu/cpu infernece is also included.
What im going for is adding the dev environment itself to blender, being able to develope protoypes directly in blender as well as running final models make more sense. AI’s like to eat data and when considering ai working with media blender could be used to create virtualized data sets directly to train new models. Like nvidias driverless car AI, alot of training material is virtualized using unreal engine to build a car simulator, being able to run on massive networks means you can have hundreds of years of virtual data generated by the game engine to train the AI, Like the AI that won the DOTA2 competition, that ai was playing the equivalant of 80 years of dota2 every day to train on.
Done right blender could be used to create these virtual datasets and run them all in the same platform.
Yes it sure be easy for you, not doing it would be easier for me though, I just lack the time to script every single library a dev may want to play with. I already have to build+test 4 different flavors of all libraries on windows(x86/x64/debug/release), once vs2019 drops, that will be 8 again, building and testing libs is REALLY time intensive. (Also there’s a slight chicken+egg problem, without any code that uses the library it’s hard for me to test if it’ll work/build/link properly with blender)
Also there are already complaints about the size of the libraries on svn, so adding hundreds of megabytes of libs that only benefit a few developers is not a direction i’d like to go in.
Don’t get me wrong, once a patch makes it through review, I’ll happily make sure the required libs are available to all users, but until that time it is unlikely to happen.
OK cool, Yep it took me 4 days of trying to get tensorflow to compile to libs like this, which was exactly why I thought IF you were willing people would love you for it so they dont have to go through the nightmare process.
I fully understand why you wouldnt do this unless you really had too. 
I couldn’t agree more 
At first though, I don’t think it is necessary to have a complete machine learning library. For inference, there are usually ways to strip everything off that is not needed.
I’ve been playing with Pytorch, and it’s a 500 megabyte library. I don’t remember how big Tensorflow is but I’d guess it’s about the same. These libraries are also constantly changing and breaking compatibility with the previous ones.
If you want to have AI in Blender, it’s should be quite easy. Just use Anaconda, make an environment, download Python version that Blender uses and download all AI stuff, then just use that Python inside Blender.
Anyhow… The main point here is that this AI stuff takes a lot of HD space. Multiple gigabytes for simple stuff. It’s all about big data, and to store that big data you need a lot of space. So I think it’s really infeasible to include in the base package.
Also if you want to write an addon that includes some of your trained models, it should be pretty easy to use pybind11 to make a Python module.
Something like style transfer are really cool though, and useful. Imagine making a matte painting, but doing parts in Blender that will stick out like a sore thumb. You can use style transfer in compositor to hide the fact that parts of it are actually a 3D model.
It shouldn’t be even that difficult to write if you want to make such an addon. Here’s a good topic about it: https://pytorch.org/tutorials/advanced/neural_style_tutorial.html
Wow, I would love for you to write up a more detailed tutorial on how to set up Blender with Tensorflow. Just imagine ex. Posenet or Deep Mimic: https://github.com/xbpeng/DeepMimic working within Blender.
All you really have to do is replace any image loading and saving functionality with Blender’s own, if you want to make it as addon. I haven’t looked into exporting models as a separate software so it would still require gigabytes for the functionality, and a complicated setup. Going through the Pytorch setup and tutorials should be enough if you know a little bit about making Blender addons.
I’m also sure Tensorflow has similiar guides and tutorials for setting it up and running projects with it.
Many deep learning projects on Github save the secret sauce and pretrained models for themselves and only show the best results they got from running 100 most powerful NVIDIA GPUs for a week, so be prepared for weeks of training.