Is deleting a saved payment method the same as cancelling a payment?

No. Removing a saved payment method does not cancel a payment you have already submitted.

It also may not cancel an Auto-Payment schedule that is already set up. To stop Auto-Payment, cancel it separately.