Right mouse click on the selected patch.
To deselect several patches at once, right-mouse click and drag an de-selection box (thin blue rectangle).