Essays, link dumps, and opinion pieces about current events in the software landscape, offering what I hope is new perspective.
“The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man.” – George Bernard Shaw (Man and Superman)
You can reach out to me with constructive criticism or insights at infosec.exchange@lmk.
AI continues to dominate the news, largely involving security, and not in a good way.
From around the web:
[Read More]Prompt Injection as Role Confusion is my new favorite paper about a very obvious threat in hindsight that’s hard for us humans to see because we anthropomorphize LLMs so naturally. When Obi-Won Kenobi tells the stormtroopers that “These are not the droids you are looking for” to pass the checkpoint that’s role confusion: the guards foolishly think his words are their own thoughts. The very readable blog-style writeup explains the details, but I want to focus on the threat model perspective which is my bread and butter.
[Read More]This HuggingFace security incident disclosure has people talking about AI agent security. Today I saw such absolute positive spin that I found myself thinking “this must be a parody”: looking at the context I’m pretty sure that it isn’t … though some parodies stay in character all the way through.
[Read More]The GrapheneOS data wiping function is in the news due to a customer now facing serious legal jeopardy for using it at a border crossing. I don’t know the details of the case not exactly how the duress feature works, but from public knowledge we certainly can talk about learning from this.
[Read More]The recent AI agent security debacle must be reverberating quite a lot within the walls of the major proponents of AI agentic technology because they announced a brand new “movement” apparently with zero details available yet. If that isn’t a sign of flat-footedness then I don’t know what is.
[Read More]OpenAI writes: “Last week, Hugging Face disclosed a new kind of security incident(opens in a new window) after they detected and contained an AI agent that compromised their infrastructure, something we expect to become more commonplace with the proliferation of increasingly cyber-capable models. After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark(opens in a new window) of cyber capabilities.”
[Read More]Opinion
Back in 1999 had STRIDE anticipated LLMs at all, much less their particular threats, it would be remembered as an unbelievable feat of prognostication rather than just categories of threats. However, even if it had predicted present day machine learning capabilities I think it plays a different role than the subject of the latest Shostack + Associates White Paper #6 titled PHANTOM-B: A STRIDE Analog for LLMs (PHANTOM-B is a tool to structure how you answer the question “What can go wrong with the LLM parts of the system?)” clearly notes: “PHANTOM-B is intended to be used as prompts, rather than categories.” By “prompts” I believe that means suggestions for (non-digital) people — not inputs to LLMs.
[Read More]“People deserve to know whether or not the videos, photos, and content they see and read online is real or original,” said Senator Schatz. “Our bill is simple – if any digital content is made by artificial intelligence, it should be labeled so that people are aware and aren’t fooled or scammed.”
[Read More]
This month the link dump continues to evolve: breaking out a few article length posts and then various links and quick thoughts to share. As always this is all quick takes so please read with a grain of salt and I’m happy to get criticism where needed.
[Read More]According to the recent article, ‘Very blunt approach’:
eSafety Commissioner Julie Inman Grant has expressed reservations about the social media ban for under-16s pushed by Minister for Communications Anika Wells.
“What you’re effectively asking us to do with this is fence the ocean,” she said. “We might be able to create some friction and some degree of safety, but it’s a futile exercise if you think you’re totally stemming the ocean.”