Disable Sketch when other tools are in use

Very often when I’m using a tool or moving an object, if I miss the little drag handle, it automatically enters sketching mode and draws a line I did not intend for.
Can we please disable this?

And can we make aiming for the handles more forgiving, give them a little more cushion by making their live clickable area larger?