"It's almost as if you wanted him to leave."