How to check for an operator property

Trying to modify this to show up in the keymap instead of in toolsettings
https://developer.blender.org/D6322

Creator says it was done as a tool setting so it can know when to draw facedots in xray. Don’t really care about that either way. Can live without it, but I am pretty sure if I get an understanding of my problem it would allow this functionality.

What I’m interested in is getting this in the keymap, so it can be mapped to a modifier similar to the different selection modes.

I figured out how to get another checkbox in keymap, and where I want it to be, but I don’t know how to make a script check if that property is true/false, the same way it is currently checking if a tool setting is true/false.

Here is my new property ‘mesh_select_through’ in wm_operator_props.c

void WM_operator_properties_border(wmOperatorType *ot)
{
  PropertyRNA *prop;

  prop = RNA_def_int(ot->srna, "xmin", 0, INT_MIN, INT_MAX, "X Min", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_int(ot->srna, "xmax", 0, INT_MIN, INT_MAX, "X Max", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_int(ot->srna, "ymin", 0, INT_MIN, INT_MAX, "Y Min", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_int(ot->srna, "ymax", 0, INT_MIN, INT_MAX, "Y Max", "", INT_MIN, INT_MAX);
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);

  prop = RNA_def_boolean(ot->srna, "wait_for_input", true, "Wait for Input", "");
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
  prop = RNA_def_boolean(ot->srna, "mesh_select_through", true, "Select Through", "");
  RNA_def_property_flag(prop, PROP_HIDDEN | PROP_SKIP_SAVE);
}

It shows up in the keymap under ‘Wait for Input’ which is what I was hoping for. Now I just need the script to check for that property instead of its tool setting equivalent. This is done in two scripts from what I can tell, and getting it to work in one should be the same process as in the other.

I think I probably have to add an Include to something in that script, and then set a *prop variable to it? That’s the way it is checking for the tool setting, so it should work the same way, right?

Here’s what it does to check the ‘mesh_select_through’ tool setting and then do something about facedots in view3d_select.c

static bool do_mesh_box_select(ViewContext *vc,
                               wmGenericUserData *wm_userdata,
                               const rcti *rect,
                               const eSelectOp sel_op)
{
#ifdef DEBUG_TIME
  double t1 = PIL_check_seconds_timer();
#endif
  BoxSelectUserData data;
  ToolSettings *ts = vc->scene->toolsettings;

  view3d_userdata_boxselect_init(&data, vc, rect, sel_op);

  if (SEL_OP_USE_PRE_DESELECT(sel_op)) {
    if (vc->em->bm->totvertsel) {
      EDBM_flag_disable_all(vc->em, BM_ELEM_SELECT);
      data.is_changed = true;
    }
  }

  /* for non zbuf projections, don't change the GL state */
  ED_view3d_init_mats_rv3d(vc->obedit, vc->rv3d);


 GPU_matrix_set(vc->rv3d->viewmat);

  const bool use_zbuf = !(ts->mesh_select_through || XRAY_FLAG_ENABLED(vc->v3d));
  const bool show_face_dots = (vc->v3d->overlay.edit_flag & V3D_OVERLAY_EDIT_FACE_DOT) != 0;

It does a bunch of other stuff in view3d_select.c to make it all work, and something else in another script to draw facedots or not.

So I’m thinking all I have to do is set this up to check my operator property instead of the tool setting. But I don’t know how, and looking around has gotten me nothing, sorry. I wish I could just say

  PropertyRNA *prop = vc->scene->property->mesh_select_through;

and

 const bool use_zbuf = !(prop->mesh_select_through || XRAY_FLAG_ENABLED(vc->v3d));

and

#include "wm_operator_props.c"

But that doesn’t work and probably looks ridiculous to anybody who knows better.

Thanks for helping. There’s probably reasons to not do this, and other ways around it, but if anybody could please point me in a direction so I can become competent with dealing with this type of situation I would be grateful.

I wonder if I could just do something else entirely like getting a Tool Setting inside of the keymap?

So it occured to me to check wm_operators.c to see how it makes use of properties, so I did this:

  BoxSelectUserData data;
  ToolSettings *ts = vc->scene->toolsettings;
  wmOperator *op;
  PointerRNA *ptr;
  PropertyRNA *mesh_select_through = RNA_struct_find_property(op->ptr,
                                                                "mesh_select_through");

and

  const bool use_zbuf = !(mesh_select_through || XRAY_FLAG_ENABLED(vc->v3d));

and

if (ts->selectmode & SCE_SELECT_EDGE && !mesh_select_through) {
      mesh_foreachScreenEdge_clip_bb_segment(
          vc, do_mesh_box_select__doSelectEdge_pass0, &cb_data, clip_flag);
}

and

  if (ts->selectmode & SCE_SELECT_FACE) {
    if (mesh_select_through) {     
        mesh_foreachScreenFaceVerts(
            vc, do_mesh_box_select__doSelectFace, &data, V3D_PROJ_TEST_CLIP_NEAR | V3D_PROJ_TEST_CLIP_BB);
    }
    else {
       if (use_zbuf) {
        data.is_changed |= edbm_backbuf_check_and_select_faces(
            esel, vc->depsgraph, vc->obedit, vc->em, sel_op);
      }
      /* Xray Mode with face center selection */
      else {
        mesh_foreachScreenFaceCenter(
            vc, do_mesh_box_select__doSelectFaceCenter, &data, V3D_PROJ_TEST_CLIP_DEFAULT);
      }
    }
  }

And at least it builds, but it will crash when trying to box select.

The worst part is I don’t know why, other than something done here.

And apparently no crash log on windows, hilarious.

https://docs.blender.org/manual/en/latest/troubleshooting/crash.html

Did you try running this with a debugger attached? You should get a good callstack that gives indication of where it went wrong, and likely why.

1 Like

Thanks a lot, that’s exactly what I needed. Never done any debugging before, the stuff I’ve done is simple enough that it worked without much trouble if it was able to build.

Followed this
https://wiki.blender.org/wiki/Tools/Debugging/Python_Visual_Studio

Got this error -
Run-Time Check Failure #3 - The variable ‘op’ is being used without being initialized.

Seems so obvious now, I need to make wmOperator *op equal to something the same way that tool seetings is done
ToolSettings *ts = vc->scene->toolsettings;

Hopefully with a bit of looking around I’ll figure that out, thanks again

edit:
Well it finally works, such a simple thing that is really stupid looking now, but I guess when you’re just trying to figure out how to get something to build and not crash it’s easy to overlook.

First, to not crash, need to initialize ‘wmOperator *op’ not by copying how toolsettings was done because it does all kinds of crazy stuff, at least in my experience. Just put it in here:

static bool do_mesh_box_select(ViewContext *vc,
                               wmGenericUserData *wm_userdata,
                               const rcti *rect,
                               const eSelectOp sel_op,
                               wmOperator *op)

Then it won’t crash because it’s been initialized.

Second, the way I did this before (because that’s the way I found it done in wm_operators.c) it only checked if the property EXISTS, not if it is true or false.

 PropertyRNA *mesh_select_through = RNA_struct_find_property(op->ptr,
                                                                "mesh_select_through");

So if there’s a property called ‘mesh_select_through’ it will just do stuff, regardless of it being true or false.

Instead of ‘RNA_struct_find_property’ just say ‘RNA_boolean_get’ and it works as intended. If it’s true, it does one thing. If it’s false, it does another thing. In my case, this is a bool in the keymap for the intended things like box select, and it controls whether it selects through mesh, without needing to turn xray on/off.

  PropertyRNA *mesh_select_through = RNA_boolean_get(op->ptr,
                                                       "mesh_select_through");

I just do a alt-drag to select through, with shift-alt and ctrl-alt to extend and subtract. Before Blender I was used to selecting through being on all the time, but I kinda like select visible by default, with select through when needed just by holding alt. But who knows, I might flip around once I get going again actually using blender instead of futzing around with source code.

Point is it can be done without moving over to a checkbox under tool settings now, it can be mapped to whatever user wants.

Thanks again @jesterKing wasn’t a whole lot, but it was enough to get me thinking and acting a little different in my approach, without just telling me exactly what to do where I don’t learn anything.