Decoupling x-ray and limit selection to visible

[David-Cros]

I really like what you have done with this tool.
I use auto x-ray mode and it does what it says. The only thing I miss with this mode is the ability to set different behavior per tool. What I mean:
I would like to use the auto switch for xray only with box and lasso selection and use circle selection as a paint selection tool that does not select through the object. Currently this is not possible.
It would be nice to have an option in each tool’s keymap to override the xray parameter.

Other than that, very good job.
I also like the separate menu for xray.

[lcas]

I’ve done this with the header button mode for myself, for the same reason that I don’t like to select through with circle. Should be the same for auto xray, I’ll see about putting this back in.

Update / personal note about select fully enclosed faces:
A more straightforward way to do it, as opposed to running some EDBM stuff, is to change select modes (basically pressing 2 and then 3 with default keymap):

wmOperatorType *ot = WM_operatortype_find("MESH_OT_select_mode", true);
PointerRNA op_ptr;
WM_operator_properties_create_ptr(&op_ptr, ot);
RNA_enum_set_identifier(C, &op_ptr, "type", "EDGE");
WM_operator_name_call_ptr(C, ot, WM_OP_INVOKE_DEFAULT, &op_ptr);
RNA_enum_set_identifier(C, &op_ptr, "type", "FACE");
WM_operator_name_call_ptr(C, ot, WM_OP_INVOKE_DEFAULT, &op_ptr);
WM_operator_properties_free(&op_ptr);

It’s necessary to change select modes because if you run the fully enclosed edge select while in face mode, it will give you this weird selection of edges, but it wont be in edge mode and they are unresponsive unless you go into edge and back into face. You also need to do a ts->selectmode = SCE_SELECT_EDGE; a redo the esel stuff that happens before it checks what select mode you’re in. Otherwise it only works in xray / select through, and solid shading gives you a random selection, or nothing at all.

This is all pretty hacky, I am pretty sure the devs would just run a different selection script that directly selects fully enclosed faces instead of running a macro like I’m doing, but I don’t know how to do that without a lot of trial and error, potentially getting nowhere.

[slowburn]

Auto Xray is a nice addition. But I agree with the others about facedots in Xray mode. In the last build it worked really well with separate facedot controls and an option to always select by face area. Maybe people misunderstood how the options were called in the poll, but regardless of the poll, selecting faces in Xray mode doesn’t work properly without facedots.

[David-Cros]

I’ve spent more time on the latest build and I think that drag select options should not belong to the xray menu.
Here’s a how I see it:

The separate xray menu could stay with face center option:

[APEC]

Do not even need these checkboxes then
as this options would be in each selection type

@David-Cros

The separate xray menu could stay with face center option:

as about this option, in my opinion, it’s no needed.
better to leave it as is in Shading menu + option “X-Ray face centers” or similar

[David-Cros]

@APEC

Do not even need these checkboxes then
as this options would be in each selection type

You’re right

@APEC

as about this option, in my opinion, it’s no needed.
better to leave it as is in Shading menu + option “X-Ray face centers” or similar

Not sure for this one. I find the shading menu a bit crowded and the xray options doesn’t even appear in material and render preview.

[APEC]

@David-Cros

Not sure for this one. I find the shading menu a bit crowded and the xray options doesn’t even appear in material and render preview.

Agree with you. But developers very picky to changes in UI, even this little arrow on “Toggle X-Ray” with 2 options…

[Dragosh]

The blender UI is crowded as is so i dont see the need to shove everything under the Shading dropdown, it makes sense to add a small dropdown button to the Xray icon, it’s constructive, clear and intuitive to be there instead to be burried down under the Shading menu

[lcas]

[BUILD] Preview Select Through 31 Beta Weird Enclosed Face Subtract

[DIFF] Weird Preview sthru31

Made some changes, added some things. Made a new preview build, included a readme for building it yourself if interested. I thought I was nearly done with the initial debugging, but as I was doing a quick rundown of stuff I noticed some odd behaviour with my macro to do a select enclosed faces, when subtracting from selection. If you drag to subtract from selection, it will sometimes also deselect a random face, even if you don’t drag over enough of a face to qualify it for deselection:

It only happens in my hacky macro attempt at making select enclosed faces, so I’m going to see if there’s something I can figure out. I doubt I will get anything done in that department but you never know. I expect this feature will be missing for now.

Doesn’t matter if xray or not, box or lasso, load factory defaults, deleting this cube and make another. If you select some stuff, it will sometimes randomly do this sort of thing. If you select this group of faces as I have in the video, with a cube that is simply subdivided to 8x8 per side, it will probably do this, because I tried it just to see and it “worked”

What’s different:

1. Most of these things are now Toolsettings because it makes sense
2. Split facedots, xray and solid
3. An operator to toggle both xray and solid facedots. This operator also acts as a label, I don’t really like the way it sticks out, but that’s all I got for now.
4. Individual control over auto xray and select through header button. This is available with the option to link any of the three selection tools together so that you don’t need to keep toggling them on/off or keep in mind which mode one of them is in. If you want to keep box and lasso synchronized, which is what I plan on doing, you can do that. Or all 3. Or none of them.
5. The select all edges/faces and select xray facedots are now in the toolsettings menu, it changes dynamically based on which tool is selected, whether the tool itself or the fallback menu of nonselection tools. Select all edges and enclosed faces isn’t available for circle, so they will hide. Everything will hide for tweak tool, and probably anything else.
6. Changed the xray prop checkbox back to the operator, also made the magnitude slider say which type it was for clarity. I like the operator because it does exactly the same as if the property checkbox was there (it only changes the current mode) but removes any unnecessary or less useful keyboard shortcuts being made. Like me for example, I had both the header button xray and the property checkbox thats in the shading dropdown as shortcuts. The property one is less useful because it will only work in the shading mode it represents and not the other (wireframe vs shaded)

I know it’s not all together in one spot, but like some of you I didn’t really like what are essentially toolsettings in a non toolsetting area. I imagine I could also move the rest of the xray popover into toolsettings but I think that part works well enough to not go through figuring out how to make an enum for toolsettings.

I also figured out how to set new defaults, it was a bit more convoluted than I imagined but at least that should be good now as far as facedots being there in xray by default, and selecting by them as well. So this should act just like normal blender, unless you go and have a look at what’s going on and play around with it.

That’s the intention anyways, I’d appreciate if anybody would put this through it’s paces to see if anything else is weird. Last time proved that was definitely needed, and even led to a few interesting new features.

Some pics:

[David-Cros]

Hi,
There are a few things that could make this iteration more convenient for me but also a few bugs that make it unusable:
-When I use the auto Xray option, when I switch back to object mode, the dragged selection permanently shows objects in Xray shading
-The all edges option doesn’t seem to work
-When using auto x-ray you can still toggle the default x-ray with alt+z but have no visual feedback in the UI as the xray icon has been replaced by autoXray’s one. That’s why I think this option should belong to tools options panel.

Thanks again for your hard work

[lcas]

Thanks, I fixed auto xray, it will check for edit mode now and not mess up object mode. Another one of those things that should be obvious, but got through. Even if I was being really thorough checking everything, I’d still wonder if I missed something anyways, so it is nice to have some testers.

Select all edges is working for me with box and lasso in both solid and xray. Make video or tell me more so I can get an idea what is happening.

With auto xray button in the header replacing the usual xray button in that mode, it’s a compromise. I would be interested to see if there are any ideas though. I don’t want to add another button if I can avoid it, but I also want to have the most useful information available to user for whatever select through mode they are in. I will not be turning off access to manual xray regardless of mode, not that I think anybody would ask for that. I can also have autox available all the time, and then a choice between sthru button or keymap. I don’t see the point, but maybe it’s just me not using autox at all.

My reasoning for the button being the way it is currently, is that it’s important to see autoxray on/off status, and having 1 button rather than 2 is more important than seeing regular xray status. It’s usually pretty easy to know if xray is on just by looking at the mesh itself, but the button exists for a reason, so maybe there’s more to it than that.

More detail, probably not necessary:

The only other way to know if auto xray was on, would be to start a selection and see if it engages or not. Not knowing before selection could lead to a cancel out of selection, or an undo afterwards, because it wasn’t the way you are expecting it to be. The button being in the header has the potential for some subconscious / muscle memory type of thing where the button being lit in that area of the screen would let you know beforehand if autox was on, without needing to actively look there. Replacing / hiding the normal xray button when in this mode, instead of having a second one up there, is because this seems more valuable. Xray being on or not is usually known by looking at your mesh and noting whether it is semi-transparent or not.

Same logic goes for select through header button, and the keymap version of select through is an active choice based on whatever you assign it to in your keymap, so not necessary to show in that mode. Sthru keymap is an active choice for each selection, as opposed to a toggle situation with autox and sthru header button. Checking for some key being pressed or not, and lighting up a button, would be way more trouble than it’s worth I think, and would probably be annoying to user anyways.

Some options, maybe poll later:

1. A second button shows up for autox and sthru button modes when they’re enabled, because people might want to see the status of normal xray without opening the dropdown.
2. Always have 2 buttons up there, with auto xray always available, and then a 3rd button would show up if select through header button was chosen instead of select through keymap
3. Stay as is
4. Some other idea(s)

[Hologram]

Perhaps one alternative option could be to show Xray mode in the shading button? Like dashing the icon for when it is in Xray mode and have a 50% opacity fill as opposed to pure white.

Or, better yet, Auto Xray could be shown like Xray, but whereas Xray gets a fully blue fill of the square button icon in the header (default theme), Auto Xray would only have a blue outline of the square (so the button is only partially filled).

[lcas]

Some good ideas, got me thinking about condensing the header. Probably for my own stuff but I’ll share it here for anybody interested. With the icons changing appearance I’m going to just do what is currently available until I get something I like. Then afterwards maybe get some different icons, or figure out how they do the different draw styles for them to get some new icon effects.

[lcas]

I have select enclosed faces halfway working, xray / select through only. Getting it to work in xray is really easy for me, need to change like 3 lines in do_mesh_box_select__doSelectFace
but getting anything to play nice with bitmap / zbuffer stuff for near selection is too… mysterious to get it done anytime soon. I tried to look at how edges work with both xray and near select, but can’t get index to pass anything but junk and either crash or select nothing.

I’ll leave it out of the diff I will submit, but will include it with the 3.1 build I make sometime in the next week. Going to be busy for a few days, finally got a new gpu supposed to be here tomorrow

[Hologram]

Please also post the link to the patch once submitted so we can drop you some juicy tokens

Hopefully, once the devs get around to reviewing the patch they will actually help with the issues you are still experiencing.

[lcas]

I just remembered this post, that patch you linked has the solution to my problem I’ve been trying to figure out for like 2 weeks now.

I’ve been trying to do a select enclosed faces and not getting anywhere with the near select version of it. Was about to post a thread asking for help, showing my dysfunctional code modification:

static bool edbm_backbuf_check_and_select_faces(ViewContext *vc,
                                                struct EditSelectBuf_Cache *esel,
                                                Depsgraph *depsgraph,
                                                Object *ob,
                                                BMEditMesh *em,
                                                const eSelectOp sel_op)
{
  
  BMIter iter, viter;
  BMFace *efa;
  BMVert *eve;
  bool changed = false;
  ToolSettings *ts = vc->scene->toolsettings;

  const BLI_bitmap *select_bitmap = esel->select_bitmap;
  uint index = DRW_select_buffer_context_offset_for_object_elem(depsgraph, ob, SCE_SELECT_FACE);
  if (index == 0) {
    return false;
  }

  index -= 1;
  BM_ITER_MESH (efa, &iter, em->bm, BM_FACES_OF_MESH) {
    if (!BM_elem_flag_test(efa, BM_ELEM_HIDDEN)) {
      const bool is_select = BM_elem_flag_test(efa, BM_ELEM_SELECT);
      const bool is_inside = BLI_BITMAP_TEST_BOOL(select_bitmap, index);

      bool is_vinside = true;
      if (is_inside && ts->fully_enclosed_faces) {
        BM_ITER_ELEM (eve, &viter, efa, BM_VERTS_OF_FACE) {
          int i = BM_elem_index_get(eve);
          if (!BLI_BITMAP_TEST_BOOL(select_bitmap, i)) {
            printf("OUTSIDE = vert %i of face %i\n", i, index);
            is_vinside = false;
            break;
          }
          else {
            printf("........vert number %i of face number %i is INSIDE\n", i, index);
          }
        }
      }

      const int sel_op_result = ED_select_op_action_deselected(
          sel_op, is_select, is_inside & is_vinside);
      if (sel_op_result != -1) {
        BM_face_select_set(em->bm, efa, sel_op_result);
        changed = true;
      }
    }
    index++;
  }
  return changed;
}

And a video of it not working quite right. I remember looking at that patch when you posted it before and bookmarking it for later because I had seen some useful stuff in there. Still didn’t stop me from forgetting about it and messing around with this a dozen different ways that all don’t work in the same way.

Anyways, sorry about the delay I know I said I’d do the build a week or two ago. Shouldn’t be long now, and fully enclosed faces should be functional thanks to that patch made by @Harleya who is actually the reason I got into this whole thing in the first place. A couple years ago he pointed me in the right direction as far as building blender, when I was looking for some information on another forum, trying to figure out how to modify the cursor.

[lcas]

[BUILD] Blender 312 Select Through
[DIFF] Sthru312
[OPTIONAL] Alternate Header

Ok here’s a release build for 3.12, should be good as far as bugs (edit - Intersect selection doesn’t work with faces in xray/select through unless it’s by facedot, see posts below), optix should be working. I don’t think the HIP sdk is out yet, so you’ll need to compile the DIFF later if you need that. I couldn’t check it anyways since the latest AMD I have is a 7870.

Current behaviour of Blender is intact and should be unchanged if you reset to default. List of what it does, some pictures where applicable:

Facedots
Facedots are split into xray and solid shading. Just means you can toggle them independently. Facedots can be turned off in xray.

This is accessed from the viewport overlay popover. If anybody has a keymap for the facedot toggle in vanilla blender it will still work as usual (it toggles facedots in solid shading only). But if you assign the operator I put in its place it will toggle facedots for whatever mode you are currently in, solid or xray. Similar to the default xray button, it will toggle xray for either wireframe or shaded depending if you are in wireframe or not.

X-ray and Select Through
You can select occluded elements without turning on xray manually. This is available in 3 flavors, auto xray, select through header button, and select through keymap. Xray has it’s own popover and the settings for it are in there instead of Viewport Shading. These alternate xray modes are also available whenever you have box, lasso, or circle as your fallback tool for things like the transform tools.

Auto-Xray will turn on xray for you while drag selecting in box, lasso, or circle. When selection is finished, or cancelled out of, it will turn xray back off for you. Can be turned on and off, header button will reflect this.

The on/off status for auto xray and select through header button can be synchronized amongst any combination of the 3 selection tools.

Select through header button lets you select as if xray was on without changing anything visually. Toggles on or off like auto xray.

Select through keymap is the same as the header button version except instead of a toggle, you hold down whatever you assign it to in the keymap during each selection. You’ll need to do this keymap assignment for yourself, tooltip will show you how. It’s in the keymap for each of the 3 select tools, you will find a checkbox called ‘Mesh Select Through’

Drag Select Options
You can select edges and faces differently. Accessed from toolsettings from the two usual spots. Also available whenever you have box, lasso, or circle as your fallback tool for things like the transform tools. You can force select all edges instead of needing to fully enclose them. Circle select does not have fully enclosed edges or faces options, only the ability to disable xray facedot selection.

Faces are slightly more complicated. In short, xray facedots has priority when you are in xray and facedots are visible. This is the default behaviour. If you turn off ‘xray facedots’ it will select by face area aka intersecting faces. If you have enclosed faces on it will only select fully enclosed faces, but again, only if ‘xray facedots’ is off if you’re in xray with facedots visible.

Alternate Header
This is just something I’ve done for myself, and since it’s all python, you don’t need to compile to use it if you’re interested. It saves a little space by moving the shading buttons into the popover. To make more use of the button, it is actually just the xray button with a different icon depending on what mode you’re in. This is so you can have the usual xray state visible in button form when in either auto xray or select through header button, where you’d otherwise have to open the popover to see a button that lights up to show when xray is on.

Let me know if somethings wrong, thanks for helping everybody. I’ll wait a bit to be sure nothing’s messed up, and then go about submitting something. Based on what I read, they don’t want something that combines multiple features in one diff. So it might be split into 3 different diffs (facedot, select through, drag select options)

[Hologram]

@lcas FWIW it works really nice with auto X-ray. I’ve been testing a couple of combinations and sofar have not encountered any bugs. It’s simply great! Thank you so much for your work on this!

Small feature suggestion
What might still be a nice addition for Auto-Xray is if it were to work in object mode too, so you can see your selections. This was possible in the X-ray selection tools addon.

Installing the build?
Now, if I want this build to install this build as if it were the .exe downloaded from Blender.org, I need to extract it to the same folder location as where I would redirect the installer, right? Is there anything else to it? Also asking for others who may be looking to install this build.

[insideitall]

Awesome work @lcas thanks, going to check out your build asap.
I’ve got a question tho, i’ve been sussing out the various tickets in blender development about this issue and having a hard time following a lot of the disconnected pieces, is this intended to be merged with main and available with full release (after reviews ect)
Or is this an independent diff?

[Hologram]

I actually did find a bug, the intersecting selection method does not work with Auto-Xray enabled.
The add, subtract and new selection modes work fine in my keymap, though the intersect method does not (Ctrl+Shift+mouse drag). If I enable this option from the selection tools, it adds to the selection.

With the default keymap, the intersect mode also does not work, neither in X-ray, nor automatic X-ray mode:

In regular non-x-ray modes it works as expected.