I can’t see the issue. The “click and drag anywhere” also exists here, and like you said, it works only when something is selected, so it’s all good, since this tweak mode is only actve when nothing is selected. And hiding the gizmo is really a minor thing.
This method has many advantages, as it allows to not just the transform tools to perform tweaks but basically most modeling tools as well, and this is huge…
Here’s a quick example of this tweak mode method working with the transform tools and also some modeling tools. The gizmo is hidden.
(nothing selected, mouse down = selects, drag = action, release = deselects)
This is the most flexible tweak mode method imo, as it allows you to tweak your mesh with several other tools, without the usual “click to select once and then click again to drag”.
You should give it a try, it’s really an amazing and fast workflow.