Ask your agent to give you a form called Termination of Buyer Agency. It will cancel oral or written agency agreements.
If you are an agent who wants to cancel the agency agreement, you may want to soften the blow and suggest that your client would be better off working with another agent who could more readily meet the client’s needs. Then refer that client to an agent in exchange for a referral fee.