"Sometime earlier, Madison's wife had thrown him out"