"But for years, he supported both of us"