"It's not the going with the ladies I care about..."