"Either he goes or I go."