@ideasman42 Is it ok if I work directly on Blender 2.8 code ?
I see you frequently merge master to 2.8 so getting the latest commits should not be an issue. Is there a reason to prefer the 2.79 [master] branch instead of the [blender 2.8] branch ?
[master] is delightfully stable compared to [blender 2.8], sure it doesn’t have all the shiny new features, but on the upside: it doesn’t have the instability of all those shiny new features either…
1 Like
@LazyDodo I am not afraid of the crashes, its part of the game anyway, but you make a valid point if I may not be able to use img_buf. I will give 2.8 a try and if all fails I will drop down to master. But if its a few crashes , I will stick with 2.8 cause I like to take advantage of the new code and it’s new features and I want to understand the new code base for my own fork which is why I decided to have a bite on this API as a warm up.
Ok I hope I ask this in the right place, after a day with battling with reducing my build times from 30 minutes to a minute, I managed to reduce it down to 10 seconds. Which makes coding bearable.
So first question I see the methods are formated like this example
static PyObject *M_imbuf_new(PyObject *UNUSED(self), PyObject *args, PyObject *kw)
{
int size[2];
--> static const char *_keywords[] = {"size", NULL};
static _PyArg_Parser _parser = {"(ii)|i:new", _keywords, 0};
if (!_PyArg_ParseTupleAndKeywordsFast(
args, kw, &_parser,
&size[0], &size[1]))
{
return NULL;
}
/* TODO, make options */
uchar planes = 4;
uint flags = IB_rect;
ImBuf *ibuf = IMB_allocImBuf(UNPACK2(size), planes, flags);
if (ibuf == NULL) {
PyErr_Format(PyExc_ValueError, "new: Unable to create image (%d, %d)", UNPACK2(size));
return NULL;
}
return Py_ImBuf_CreatePyObject(ibuf);
}
4th line, why the pointer variable is named _keywords[] instead of _args[] ?
Please note I am new to this Python C API dance, so I reading documentation while trying to understand Blender code using VS debugger. Things are starting to make sense but still I am in an infant mode.
Care to elaborate on this? Why did a build take 30 minutes for you? sure initial build will be a bit, but beyond that it should be both incremental compiles and links.
Probably I was doing something wrong and was rebuilding the whole thing.
In any case I am now using the lite release and 10 seconds are good enough for now and see in the future if I could reduce it down to 1. But right now my priority is to learn the Blender code and be useful. So that 1 second build time will have to wait.
I have developed a small library for C++ for live coding, which essentially build a super minimal executable and keep everything in DLLs so it keeps compile times in under a second and it does rebuids in the background while code is saved, it also keeps memory intact which is also handled by the DLLs. It was not my idea, I was inspired by another developer that publishes the handmade hero tutorials on youtube. His goal was to use C++ as a scripting language instead of python, like Blender does with its addons. Unreal uses also a similar technique for reloading/rebuilding C++ code, real time, while in its editor.
So build times is a temporary problem for now.
Of course Blender is far more monolithic so I will have to adjust that library, but as I said thats not what my priority is right now and for now I only care to understand the basics of the code , hence my question.
Allright them back to your question, it’s called keywords cause
- it contains the keywords
- it goes into the keywords field of the _PyArg_Parser structure.
but in all fairness, it’s just a variable name, could have been named LazyDodoLikesKittens and it would have still worked, however nicely chosen names do make the code more readable.
I’m not sure what the arguments could be for calling it _args.
1 Like
Well my logic was that since it refers to python arguments and not python keywords , I would expect it to be named _args but now I see as I suspected the rabbit hole goes deeper.
Yeah I am rather strict how I name things, but my motivation for this question was to understand the intention. To better understand how Blende deals with such functionality. I thought there was more it, glad to see the explanation is simple enough.
CPython’s convention is to use args for a tuple of ordered argumentrs, keywords or kw for a dict of named arguments.
@ideasman42 yes I know but that is what confused me
but what I did not know is that keyword arguments can be used as regular arguments as well in python taking their position into account so both version are the same
>>> i1 = imbuf.new([1024,768])
>>> i2 = imbuf.new(size=[1024,300])
>>> i1.size
(1024, 768)
>>> i2.size
(1024, 300)
I did not know python keywords had such behavior. So now the name makes sense and it is as it should be and good thing I asked cause as I suspected I was missing something and indeed I was
As you may suspect I tested it first using it as regular argument which is what confused me(see i1 in example).
Thanks mate.
Hooray!!!
My first patch, added support for image buffer duplication , get it now while it hot … just be careful so you wont be burned
More info here
Adding my creation here, mostly to keep eyes on it.
https://developer.blender.org/D3496
I would like to implement pixel access also. The python buffer api docs give me some idea, but do have in mind some existing implementation to get “inspiration”?
Icons can be an “inspiration” they are loaded in memory not from files but from inside blender source code. I would imagine the waveform preview most likely uses the imBuf struct.
if it does it should give you access to the pixel data via the rect member of the imBuf struct.
A typical usage is when using as a opengl texture like so
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, b32buf->x, b32buf->y, 0, GL_RGBA, GL_UNSIGNED_BYTE, b32buf->rect);
If you mean in terms of Python implementation I am in favor of Python objects that follow closely Blender source structs like imBuf in this case. So I would just add a rect python list that would contain the same data. That may also be good for performance, if you want direct access to pixels that may means that most likely you want access to a large collection of pixels and it would not be a good idea to introduce any conversion in that stage that would severely slow down the Python execution.
I can add rect myself to the implementation if @ideasman42 has no objections.
Yeah I was thinking python implementation
https://docs.python.org/3/c-api/buffer.html
I just quickly read through this, and realized that this probably isn’t as easy as exposing rect by getter and setter.
you can do this buffer if you want.
I need to finish proxies, then we can have python effect sequence.
Video producers not using blender will hate us 
No guarantees because I am about to leave for vacation but I completely agree bytearray buffer is a more appropriate solution.
A getter and setter most likely will need to be implemented anyway if I remember the Python C API correctly.
I don’t think it’s a great idea to do video processing via Python though, I don’t think Python would cut in this case. So far we exposed the c side so there is no big penalty for Python but if you start relying too much on Python for pixel access and manipulation your probably will get very slow results especially with hd videos.
So I think if video editing is your goal, it will be much better to implement what you want in C and then expose those functions to Python.
I am not in hurry…
Python for pixel access and manipulation your probably will get very slow results especially with hd videos.
That’s why I am working on proxies (indexed video buffers).
Sure C is preferable, but API with pixel access will enable anyone to do whatever processing they want. Also various generators and stuff like that.
1 Like
I run some performance tests for image processing in python. For now I was just drawing solid rectangle over whole image. As expected performance of pure python was terrible, but with numpy I was able to “redraw” whole image(each one frame) ~60x maintaining performance of 60fps.
here some lab notes:
Question is, if numpy is usefull for working with images. I have no experience with it…
But it seems that pillow is also using buffer interface, so it can be hooked up.
So as image generator this seems to be perfect.
I will try to do some processing / distortions to see how much we can do in python.
When the VM , and that does not apply just to CPython but most VMs, executes a function in a DLL (which is what a pyd library is) like numpy , it basically calls its highly optimized machine code. Which means the VM has to wait for the machine code to do what it wants to do and as such cannot affect directly the performance of the library although it can introduce overheads. Obviously that comes with the downside that it cannot do its own stuff either like GC, dynamic types and what makes Python so powerful.
Hence why Numpy is so fast or most libraries that are written in C , which applies to most CPython libraries as well. Of course C alone cannot offer top performance if you don’t know what you are doing so in the end there is no way avoiding C completely.
Numpy is a more general library but indeed has been made for top performance and is written in C. There are a ton of libraries you can use because nowadays every C/C++ libraries offer at least CPython wrappers. So its completely up to you.
I didn’t have time to read all of the posts but I thought T54314 could be relevant. Blender misses this functionality which is very bothering. Some people may not need to store all of the rendering images on disk immediately.
@kilon, I know, I just wanted to demonstrate, that this interface is useful and provide some “quantitative” info about how useful.
Anyway I have this buffer code “ready”, I just have to add selection of rect/rectf. But I am working on other stuff and don’t want to switch branches.
@AmirS
This module deals with image processing.
Your issue seems to be node viewer API stuff