If you want to remove all taxonomy terms related to a post, wp_remove_object_terms() seems like the way to go. That function however expects the IDs of the terms to remove, there is no way to tell it to remove all terms at once without knowing the IDs beforehand.
Passing an empty $terms array or false or null to wp_set_object_terms() results in removing all terms associated to the post.