The recent AI agent security debacle must be reverberating quite a lot within the walls of the major proponents of AI agentic technology because they announced a brand new “movement” apparently with zero details available yet. If that isn’t a sign of flat-footedness then I don’t know what is.
Did OpenAI hack Hugging Face or didn’t they? and post raises the question of law breaking here which they suggest may be a case of “responsibility laundering” by blurring intention to do harm where the LLM in between OpenAI and the harmful actions done is to blame — and being an intangible, impossible to take to court, as well as opaque piece of software running model of a scale that defies analysis, the buck uselessly stops there. In my personal opinion (IANAL) that such damage occurred shows clearly this was due to insufficient caution wielding a powerful tool granted (intentionally or not) powerful privileges. Had an unintentional bug in complex classic software caused the same harm would/should the reaction be any different?
As Adam Shostack writes, “The team at OpenAI either can’t or won’t slow down to look at the output of these systems. Now, maybe, that’s the right call?” It looks like if you can avoid responsibility for collateral damage then it’s the right call (same as all the “move fast and break things” crowd does) …
My two cents: Sandboxing, perhaps doubled up, would be a good practice going forward - compared to all the model inference the software overhead must be miniscule.