"They're just speculating"