"I quit bringing it up because every time I did, you'd say it's not the right time."