"He would just come and go, literally."