How to implement smart selection in operators?

So with left and right being so split, addons that implement selection have to detect which selection mode the user defined.

how can I do it inside the modal() funcion of a operator?

no results for the search of ACTIONMOUSE or SELECTMOUSE in the api.

Personally I find it best best to leave it to the user to decide by giving them them option in addon preferences. Only the default keymap allows using the ‘Left’ / ‘Right’ setting, so polling this setting isn’t useful.

Not everyone will use the default keymap. It’s likely also unreliable to look for any mouse select keymap unless you implement an exhaustive check during addon registration, and even then, the user might have a different preference.

By allowing them set the select it by preferences, you only need to push it to the operator properties during invoke, then have a generic event collector i.e _event() that translates it when you call that instead of event.type .

An example script. Needs to be loaded in Blender so you can see and change the setting in addon preferences.

import bpy

bl_info = {
    "name":         "Select Modal",
    "blender":      (2, 80, 0),
    "location":     "3D View > Object Mode, Operator Search > Select Modal",
    "category":     "3D View",}

def get_prefs():
    addon = bpy.context.user_preferences.addons[__name__]
    return True if addon.preferences.select_button == 'LEFT' else False

class SelectModal(bpy.types.Operator):
    bl_idname = "object.select_modal"
    bl_label = "Select Modal"
    select = True

    # event translator
    def _event(self, context, event):
            if event.type == 'LEFTMOUSE':
                return 'LEFTMOUSE'
            elif event.type == 'RIGHTMOUSE':
                return 'RIGHTMOUSE'
            return ''
        else: # switch to right click if False
            if event.type == 'LEFTMOUSE':
                return 'RIGHTMOUSE'
            if event.type == 'RIGHTMOUSE':
                return 'LEFTMOUSE'
            return ''

    # modal assumes that left click select is default
    def modal(self, context, event):
        args = [self, event]

        if self._event(*args) == 'LEFTMOUSE':
      {'INFO'}, message='Select')
                return {'RUNNING_MODAL'}

        if self._event(*args) == 'RIGHTMOUSE':
      {'INFO'}, message='Cancel')
                return {'CANCELLED'}

        return {'RUNNING_MODAL'}

    def invoke(self, context, event):
        select_left = get_prefs() = select_left

        return {'RUNNING_MODAL'}

class SelectModalPrefs(bpy.types.AddonPreferences):
    bl_idname = __name__
    select_items = [
        ('LEFT', 'Left Button', '', 1),
        ('RIGHT', 'Right Button', '', 2)]

    select_button: bpy.props.EnumProperty(
        items=select_items, default='LEFT')
    def draw(self, context):
        layout = self.layout
        layout.prop(self, "select_button")

classes = [SelectModal, SelectModalPrefs]
register, unregister = bpy.utils.register_classes_factory(classes)

if __name__ == '__main__':