I think the Code Quality Days are going to help a bit in that respect. Cleaning up & clarifying Blender’s code will benefit all developers, and that of course includes TDs. Personally I really welcome this, I’m sure I’ll have fun cleaning things up.
Learning the language C++ can be pretty hard, but I don’t think it’s up to the Blender Foundation or Institute to provide training for this. Go to university, get a degree in Computer Science.
a TD cannot go directly into the C++ code and modify improve things easily
I think this is the case for any program of the size, age, and complexity of Blender.
Yes, that’s frustrating. That’s also why there now is the Tracker Curfew to get such contributions either into Blender or rejected for clear reasons. At least there should be an answer, and not hang in limbo like that.
I agree with @LazyDodo though, in that it really helps to poke developers on Blender Chat and make sure that things stay active in their minds. With as many open patches as there are now in the tracker, it’s easy to overlook some.
Yes I do. But have you ever seen a collaboration of dozens of people working together, and then as an outsider tried to influence them? It always helps to know how those people are organised, and which paths are the smoothest to walk.
Just know that things hanging in limbo isn’t because the devs want it to be that way. It’s a side-effect of smart people having to do very difficult things that require all of their brains, just to do their development jobs. On top of that comes planning for future features & redesigns, decoding & fixing bug reports, and doing patch reviews. One thing we’re also working on is improving the code quality and way of working, so that less bugs are introduced, freeing up time to do more of the rest of the work.
It’s not up to the blender foundation to teach people Python but you made a course on it on the blender cloud, if the difficulty is the problem, then maybe the animation course should not be possible either, better go to the university and get someone teach you body mechanics, anatomy and many many other things.
I would have never expected that answer, but hey, don’t worry, no need for that course, I’ll tell many programmers I know that they should stop working and go to university, like the RedShift addon programmer or many others, because C++ is hard to learn and we should go to university, because is hard to learn C++
Please don’t take me the wrong way. You know I’m friendlier than that
What I meant to say is that there is, at least in the way that I look at it, a big difference between teaching people the basics of Blender scripting with Python and teaching someone all of C++ to a high enough level to help with Blender development. With just very limited knowledge of Python and copy-pasting some operators from Blender’s UI you can already create a script that makes your life easier. Taking this up a notch to understanding things a bit better and getting a bit more complex, that’s what I’ve done with Scripting for Artists. My point is that teaching people C++ from the same starting level as Scripting for Artists, or even experienced developers in non-C/C++ languages, is beyond what I can do next to my usual development work. Instead of basing themselves on whatever I can hack together in what little time I have available, I’d rather have people go to get proper education in the field from an appropriate educational institute (like a university). That was my point.
Of course there is a huge difference between that and an introduction into the structure of Blender for people who are already familiar with C and C++. We may do a training video series for that, maybe not, I don’t know. I think that better written developer documentation (and making sure it stays up to date) will take less time and be more useful for a wider audience.
But I’m not selling to anyone that I do an Open Source project, that I systematically refuse to make an API in C++ so that people can implement their ideas freely because I want it to be in the form of patches, and then I ignore the patches.
It’s quite insulting that comment about " smart people", I don’t know if you think that because we are artists we are not smart or we are not able to understand how a development works. I don’t know, do I have to show you my results on an intelligence test? Since you seem to need it, I studied an engineering degree.
@Alberto I think you missed the part where I agreed with you about how patch review should be. I also said I don’t like the current situation.
Any insult you may read in my text is in your head, not mine. Saying that developers are smart does NOT imply anything about anybody else.
About a C++ plugin framework, this has been discussed in plenty of other places already, so I won’t reiterate that here.
On a final note, everything I write and have written are my own opinion. I’m not a spokesperson for the Blender Foundation, nor so I speak for any other developer. I only tried to explain why I think things are as they are. There is plenty of stuff I’m also not happy with, and I’m doing what I can to improve this.
In case the case against the two libraries has been agreed upon, the title makes it an open ended question: good coding practices. Although T70016: Document performance profiling Blender and an equivalent in wiki about profiling python catch attention now.
If you want to flip around things then we need better possibilities to be able to modify Blender to extend it’s capabilities, so or we have better accessibility to code, or we have better tools and performance out of python, but we need a production ready tool set that allows to generate tools that will full-fill the needed areas, like a proper scatter tool that works properly with a LARGE amount of objects, something that right now it’s half possible because of several limitations, and a viewport visbility mode that will ease the work with such scenes.
And TBH I totally disagree with you about going to university to learn C++, and you have there a co-worker that is also one of the most pro-efficient developers right now that says that university was kind of useless for him, C++ is not so complex at all, I know Objective-C and it was easy to learn thanks to my previous knowledge of different script languages, after that I learned Java and C#, later Python, and it’s just a matter of studying it, C++ is more of the same thing, and even @Alberto also says this, and he went to the university for such a degree, and it’s not the only one.
Also there are people that cannot go to university for many reasons, not enough money, not enough time, family matters, personal matters, but even with all that they want to learn, they want to contribute, they have awesome ideas, they have the will, that “go to university” thing is just discouraging them from improving themselves and being more proactive, to be honest even is disheartening for me because as I said, I was not expecting that, the problem are not the words, but the thoughts behind them, only “university” guys can teach/learn C++.
The only problem right now with C++ in Blender is that there is a big mess in the amount of things you have to modify to affect just one little thing, and that code is not properly documented, so you have variable names that you have to “guess” what does that mean, or have a meeting with the original dev so they can explain what those are doing, or dig during 3 days in the code until you find where that came from, and that’s not related to the language but to the code organisation itself. I can understand that it’s as it is, no argument here, but a better explanation, walk through and/or documentation could be very welcome, because sometimes only the original creator of a module is able to modify the module, just because there is no proper documentation on the code, and I received that answer when I spoke with a module owner to analyse the options we had to hire an experienced developer to make an implementation, and the answer was honest, it’s better to not do it, right now only the owner of the module can really touch it without trouble, and I won’t mention who he was or what was this about, just that this was the situation, and I’ll always be grateful to the dev for being clear on the topic.
I don’t care about having a C++ API or not, if there is not going to be a C++ API, good, I don’t have a problem, but every time we say Python performance is BAD (because it’s BAD when you try to do anything half-big, like the importers) I don’t want to receive the answer about “modify Blender code”, because hey, I’m all about it, but then give the resources to understand Blender code, because even an experienced C++ programmer is not able to properly follow and understand Blender code to do small fixes, and I’ve read Blender code TONS of times.
Also we need to do tools for production in the same way anyone can do those tools with proper performance for Max/Maya/C4D, why? because we are competing with them, we live in the same market, we fight against them for projects, we support Blender and the Blenedr Foundation, Blender Institute and Blender Cloud so Blender can support us, exactly the same as the Blender Animation Studio team I suppose, some tools will be available as addons, with or without Blender, but we need to solve problems, and for that Python, if it’s the only option to generate such tools, has to have the same performance as C++ or at the very least similar performance, otherwise the thing is that Studios are purposely being limited on what they can do without modifying Blender code, which BTW is hard to modify because all what I said.
I did not liked that you went out with the “go to university” thing, and it felt personally bad, may make sense or not to you, but to me it was disheartening as I said, we are supporting open source, free software, freedom of knwoledge, and for that we have to support free knowledge, as I do for 3D in my channel, even if we can do also paid courses, because everyone has a rent to pay.
Personally you don’t have to do the course if you don’t want to of course, I understand that and support you if you say “I don’t want to create that course” for the reasons you said or for no reason at all, but if you want to make more people contribute, more developers being excited about helping with Blender, with bug fixing, then IMHO the “university” answer is plainly wrong.
Sorry to be so hard at you, but it really reached my emotions.
I’ve been following this discussion a bit, but the above kind of sums up the Python vs C++ debate for me. Experienced programmers will (of course) be able to learn C++. But while someone who has never coded has a good change of being able to pick up Python, C++ is simply a step too far as a first language. Your prior programming experience matters at lot. The current Python API is fairly easy to get into, once you get used to all the quirks, which makes it more accessible to a lot of people. And yes, don’t go to university to learn coding
Python performance is only bad when using pure Python to do all kinds of computations or data processing. The reason the importers are slow is that they do all parsing of text input in Python itself, instead of calling into a C/C++ library (like assimp) to do the heavy lifting. That’s why things like numpy exist, Python merely becomes the glue for specifying the high-level operations, the actual number crunching work is done outside of Python.
Having a C++ API would only make it somewhat easier to couple C/C++ code to Blender. The current Python API already allows that, cf. the comments on Cython earlier. Plus, if you know what you’re doing (i.e. experienced C++ programmers) you can use pybind11 or Boost.Python to create a Python module for a piece of C++ that is fast but can also be called from a Blender Python script.
But I think there’s currently a few things suboptimal in the Python API when it comes the above:
Building a Python extension module that works with Blender is not as easy as it can be (e.g. the missing headers noted in Including Python headers in distribution? by myself, challenges in building the module, how to properly install new Python modules in Blender’s environment, etc.). Improving this might help in attracting C++ developers that can provide optimized modules for certain tasks without having to know the Blender C++ code itself. Merely having a good API (the current Python API) is enough. Case in point: my PLY importer for Blender, written in C using a C library, but available as a Python library. But turning this into a true add-on is then not that straightforward again, as it requires the user to compile the source (although I haven’t looked if there’s general instructions for distributing Blender add-ons containing system-specific binaries, like a Python module).
what about an external merging software, were the users can merge their own .zip file (of C code) to the main build, then automatically compiling to create their own custom builds of various implementation? it would look like a C++ Plugins from the user point of view.
all this within a sexy GUI for users?
we can chose a clear name for this kind of addition to make a clear distinction between Python addons and “External Merge Parts”
a bit like what they are doing with the nexus mod community.
it can be done, no ? the software just need to merge lines of codes and keep track of them.
I guess you see extra benefit for this approach because it doubles as a distribution point for addons? Because I would always want to locally build the addon during development anyway, I wouldn’t use the online environment for that (it slows iteration during development).
I think one of the targets is to avoid the general idea of “package manager” as max turned out, and I agree with that view, because it’s dangerous, and the idea is to try to merge as much as possible in Blender and make addons a side-tool, not part of the main workflow with Blender, that’s the major problem with max for example, max without plugins is nothing.
But at the same time this presents some other problems, so the thing is that a middle ground could be great, that’s why I think Python is ok, but it needs improvements to make performance visible.
@PaulMelis if that’s the case (which I don’t doubt at all) why the official OBJ importer or the official FBX importers are so slow and the Gsoc project was suspended becasue it was using Boost too much? (not sure if that’s the reason exactly, is what I remember from the reason the student said he was given, also don’t know who was the mentor).
And the other thing is that it’s very hard to keep pace of state of the art, and for example Cycles is kind of stagnated, having custom AOV’s just now, Ligh Groups still waiting, and no other performance improvements for years it’s a problem, because the only improvements awaiting review right now are:
Adaptive Sampling: that helps in specific cases, not always, and it’s not super dramatic difference if it’s not one of those cases
Scramble distance, that is there since 2016
Dithered Sobol, that is also there since 2016
So if the last two get implemented Cycles will see kind of a speed up that we’ve been using, but it’s a speed up with old technology, something that could have been there since 2016, and there are no plans to improve this, not earlier nor faster, and that’s now with the current funds, that I could understand that the funds are limited, and are still limited, so it’s a must to be wise in choosing fights, but then… we need to be able to have alternatives that fill the gaps left out, and that’s the middle ground, that’s why Python has to be improved, because if we need a proper alternative to Cycles, we need it working at full speed for viewport rendering or for final rendering, and there are some limits created by Python/Blender right now, no matter how experienced is the actual dev of such engine.
And I’m thinking of LuxCore, they are doing an amazing job with the addon, right now it’s as fast as Cycles to export millions of instances, but there are some operations that could not be done faster because of Python limitations.
We’ve been forced to switch to LuxCore for our ArchViz work because we could not get good interior renders in a reasonable amount of time with state of the art hardware, we could not afford to not have caustics nowadays, the clients require that, and we have nothing in cycles about that, and there is nothing planned, and so on, and what pisses me off more is when I receive the answer “we don’t need that in the studio, so we may not do it”… when there is donation to the. development fund I hope the development direction is not handled towards the interest of the Animation Studio, but towards the general interest and to keep Blender as an amazing state of the art tool.
Don’t take me wrong, I love cycles, I just know it’s limitations right now for some type of work, I would love to not have those limitations, but it’s normal, but if it’s normal then we need to be a bit more open minded towards other options that could be used together with Blender, and I’m not talking about closed source software, just to be clear.
So I think the key is to be in a middle ground, Python is IMHO the best way to go so anyone can improve Blender, C++ would be amazing (without API, don’t care, I mean contributing to master source), but it needs more knowledge of course, but if we want anyone contributing we have to encourage that in both ways, Python and C++, not just one.
Now the @BD3D merge thing… I dont’ understand that too much, but I think that we should avoid the “maxification” of the Blender environment
Also @BD3D that there is GPL, so I think that will help avoiding that “maxification”, and that’s why I like GPL, that’s why there is no reason to have fear to addons, and that’s why I defend that middle ground, addons yes, but we need performance, in-code improvements h*ll yes, middle ground.
I want to clarify that in this case I refer only to the devs from the Institute paid by the Dev Fund, not to the devs paid by the Blender Cloud, which I assume will look for the best interests of Cloud customers and pipeline work for the studio as it’s stated as main goal (always for the best interest of Blender in general, but different priorities).
If the core idea of blender is that it’s a software build by its users, then we should give more power to its users. (duh)
Right now people who have other visions / visions that aren’t compatible with others, are simply forgotten In the dark, end users may never experience their ideas.
If a dev want to share his (forgotten) vision for blender he should have a platform somewhere to share it,to show it to everyone. GraphicAll and the blender market is a good start. But it need to be something much bigger than that imo.
If such platform doesn’t exist, « visions » will compete with each other’s and users will need to choose sides, which is not a good thing (See E-Cycles builds vs other vs main ect…). That’s why transforming builds into « pieces of puzzles » that can stack-up is a good solution for everyone imo. Even For the main devs. As they can see which vision is appreciated by the public and merge them.
I have to say that I think that I took @sybren a bit by the wrong path, and he didn’t meant to say what I understood, so I want to publicly say sorry to him, I’ve been a bit too hard on him, he is a nice guy, that cares, and maybe the “university” thing was not what I understood andhey, if he don’t have time to do the course, he should not do it at all!!! there are other people hre that can create such course if they want… step ahead people!! hehehe
I just want to say that I’m sure that everyone can learn C++ with a bit of effort and a pair of books/courses, and that everyone, specially the ones with a programming background in other languages can improve themselves and get to a point where they can contribute to Blender, as anything it will take effort and time, but I want to encourage all of you that wants to learn and to contribute, please don’t hesitate and contribute, I’m sure review process and implementations will improve (right @dfelinto ) , and you can count on me and with our build to test out things for the general public
Can we get back on topic? Python hasn’t been mentioned in quite a while. We all know there’s not going to be a C/C++ API, it’s listed as an anti-feature. This thread isn’t about that. I’m interested in discussions on how to improve the Python API that we already have.
I’m pretty sure C/C++ is not the issue, like any language it can be learned with a bit of effort and good online resources. What is hard to learn is the specific architecture of a big project like Blender, and it would be true even if the soft was made in another language. Lot of modules, interdependant parts, legacy code, etc. Its true for Blender but also any software of that size. And it’s also not stable in time, big parts of the architecture can be completely changed in the future, so any course on that would become obsolete at some point (not to say it is useless to create one, but it has to be maintained at least).
On the opposite, an API is generally stable for some years. And you can extend the software without understanding of the whole architecture. When I started digging in Blender few month ago, I agreed that since the soft is open source, a C/C++ API is not required. But now I think having to change the core code base to implement efficiently some functionalities that could be in an addon is a big overhead, in terms of time spent understanding the existing code, time implementing the feature, and time spent by reviewers checking and integrating the changes. It does not scale very well.
People mentionned that is is possible to implement things efficiently in python, but it generally requires implementing python modules in C/C++, and if you want to manipulate Blender’s data you have to do some unclean things anyway. I see that as a workaround.
So I must admit I don’t understand the choice of not having a C API. Since there is a Python API that basically wrap C functions calls, why not just doing the exact same in C ?
And I wonder if someone outside of the BF implements it, would it be accepted in the main repository (assuming the author accepts to maintain it) ? Because if not, for political/philosophical reason, I see that as a danger for Blender. If at some point many animation studios end up using Blender, and have a real need for a C API, they might choose to support a fork that provide one instead of the original. That’s not the case right now but IMHO it is something to think about.
Now when the goal is to do some quick hacks/fixes in Blender’s code base, indeed knowing C/C++ is required (but not at a high level, the code remains quite easy to read in my opinion), and I find a lot of values in all the differentials https://developer.blender.org/differential/. Just reading some of them gives a lot of clues about how to do things. I’m pretty sure it would be interesting to have tutorials that just explain how to implement some specific features with the associated differential as support material.
Sorry @Josephbburg for continuing the discussion on that ^^’ I wanted to give my opinion and there is definitely some interesting things to discuss in this thread.