"We lose him, then we lose that Southern voting bloc."