"But while the Confederates are there, we can't get across."