"We have given Bulger miles and miles of leeway,"