All you can do is to explain exactly what you explained in your question. Appeal to his logic. Why would you eliminate all but one? Some may say get rid of him since he doesn’t trust you. If you want it to work, let him have his feelings as long as he’s not abusive. If he really cares about you, he’ll come around.