"Olivia wasn't able to give him a baby,"