Currently, roboticists and AI researchers are still struggling with lots of basic challenges to do simple manipulation tasks and enable a robot/manipulator to grab objects properly. However, they are also be looking for tools that provide realistic renderings and physics simulations and enable the robot in simulation to transfer its capabilities to the physical world in a seamless manner.
Current popular frameworks such as MuJoCo that many roboticists use for implementing their AI models for manipulation and planning tasks lack the notion of ‘material’ the way Blender has. That’s one of the main reasons why these frameworks are unable to provide realistic renderings in addition to having a good rendering pipeline (EEVEE in Blender 2.8+). Even if those frameworks did have a good system for making BPR materials, they would probably be still unable to provide realistic renderings in REAL TIME.
A framework like MuJoCo is able to give real-time physics simulation which is unrealistic as the parameters in the physics engine have been highly optimized for these purposes; it provides very simple notion of materials and also unrealistic renderings in addition to a good interactive mode.
I personally think Blender 2.8 could potentially be an ideal framework for roboticists to model their robots and enable them to interact with the world in a more realistic way that is not possible through other frameworks. Currently, Blender 2.8 lacks an interactive mode, its high-quality physics simulations are pretty slow but it provides real-time, realistic renderings. Although Blender is not designed for these purposes but with this post, I just wanted to bring this opportunity to Blender developers’ attention, and hopefully to Ton as well, that incorporating the capabilities that I mentioned above into Blender will potentially make it an ideal place for roboticists to use Blender as their main framework for their research.
I know the new part-time hire will start working on the interactive mode for Blender 2.8 soon and that’s why I decided to write this before development of the interactive mode begins. In addition, I have a strong feeling that Blender 2.8 could potentially become the ideal framework for roboticists as it already has a real-time rendering system and will soon have an interactive mode. So I hope the Blender Institute will invest more in developing the interactive mode to make it usable for roboticists as well. Of course, the ideal interactive mode for roboticists should not have the limitations of a framework like MuJoCo which I think Blender 2.8 has the potential to fill in the current gaps; mainly it should have more realistic and real-time physics simulations. I am also aware that real-time and realistic physics simulation requires a lot of computation but I hope there will be new solutions for that as well (or at least physical simulations can be improved in some way compared to MuJoCo).
In my opinion, in the long-run, this investment can also open doors to many new opportunities for Blender organization. It wouldn’t be hard to imagine that once Blender has a good, usable interactive mode for roboticists many companies will be willing to invest in developing tools based on Blender and many new developers will start working on Blender’s code and improve it’s capabilities and make it more suitable for their robotics or AI research.
there are many things that can improve Blender but like the devs always have stated is up to people to either contribute to existing Blender code base, make their own addons or make their own forks.
Bottom line the secret of success is coding. Open source projects are always in far bigger need of coders than they are of users, especially ones that are free to download and use.
On the physics front I know that they are in the Blender 2.8 roadmap but 2.8 make take years till it reaches 2.9. So yeah physics will definitely improve but I would not be holding my breath if I were you. Unless you can hold it for a few years
Blender has a massive potential because its such a powerful application, a titan of 2 million lines of code. So maintaining and extending it is no walk to the park and is always in need of more developers. Hence why we had the Code Quest, because without the funding it provided 2.8 would have been a very distant dream.
So unless you are willing to help (not emotionally but with coding) , convince coders you know to help or help raise funding to hire developers , feature requests , especially ones of such scale have very little chance of becoming a reality.
Open source is given open because its meant to be taken and carried forward by others, or else it defeat its whole purpose.
Currently Blender developers are super busy with 2.8 which is far from stable release and they have a ton on their plate.
when blender will acquire precise measurement capabilities, advanced cad modeling, and a “game logic” of the kind for robotic simulation … it will be the best tool to change forever the world of makers who already experiment with arduino, fpga, robotics, 3d printers ect …
^ ___ ^
@Ton in the future it would be interesting if an “open-robot” campaign will be launched to evolve blender also in this direction, with all the tutorials that would teach how to use blender in this sense and how to assemble a robot, from modeling to hardware programming to 3d printing to assembly
I’ve been exchanging emails with Benoit Bolsee and had some nice conversations about future of the interactive mode. Of course, he is planning everything but I tried to be helpful by providing him information on new physics solvers. Here’s what he told me in one of his emails about his current plan:
" … I have indeed started a project to bring interactive mode in blender 2.8 to a point where it can be used for game prototyping, interactive simulations, robotics, etc. It’s a one year project but I’m only working 1.5 day a week because I have other projects on the side. The project plan is as follow:
1- bring 'continued physics' mode back : physics simulation runs even if scene frame is not incrementing - done already for rigid body
2- bring local time to animations so that animations can run independently of scene frame
3- create an internal event system to exchange events and messages between scene entities
4- create events from physics simulation and interaction
5- create a logic node tree system to allow game logic (Points 3-5 will be done in collaboration with Jacques Luke who has been hired to create a real-time nodal logic framework in blender 2.8)
There is no guarantee that this interactive framework will be real time but it should be close to it. The graphic rendering will be Eevee. Everything will run from the 3D view, which can be customized to be just a window frame (or full screen) with no menus and buttons so that it looks like a game window. ..."
Based on the emails we exchanged later, Benoit has also decided to use one of the latest versions of Bullet in Blender for better performance and more stability.
You can see some more updates on this in a parallel thread here where I discuss the choices for soft body physics solver that the Interactive Mode developers are considering.
