"You can't train him like a colored fighter."