"No, Lord Grantham, I can't. Am I to stand here after 40 years of loyal service"