Designing Secure Software

Opinion

Back in 1999 had STRIDE anticipated LLMs at all, much less their particular threats, it would be remembered as an unbelievable feat of prognostication rather than just categories of threats. However, even if it had predicted present day machine learning capabilities I think it plays a different role than the subject of the latest Shostack + Associates White Paper #6 titled PHANTOM-B: A STRIDE Analog for LLMs (PHANTOM-B is a tool to structure how you answer the question “What can go wrong with the LLM parts of the system?)” clearly notes: “PHANTOM-B is intended to be used as prompts, rather than categories.” By “prompts” I believe that means suggestions for (non-digital) people — not inputs to LLMs.

“People deserve to know whether or not the videos, photos, and content they see and read online is real or original,” said Senator Schatz. “Our bill is simple – if any digital content is made by artificial intelligence, it should be labeled so that people are aware and aren’t fooled or scammed.”

[Read More]

This month the link dump continues to evolve: breaking out a few article length posts and then various links and quick thoughts to share. As always this is all quick takes so please read with a grain of salt and I’m happy to get criticism where needed.

According to the recent article, ‘Very blunt approach’:

eSafety Commissioner Julie Inman Grant has expressed reservations about the social media ban for under-16s pushed by Minister for Communications Anika Wells.
“What you’re effectively asking us to do with this is fence the ocean,” she said. “We might be able to create some friction and some degree of safety, but it’s a futile exercise if you think you’re totally stemming the ocean.”

The NY Times teases a paywalled article, “A.I. won’t take all our jobs because it can’t reason like a human, Zeynep Tufekci writes.” linking to the article behind a paywall. Simply searching for [it can’t reason like a human, Zeynep Tufekci] provides a nice summary of the article, not only penetrating the paywall but also saving time and skipping the ads.

Neil Postman’s book, Amusing Ourselves to Death: Public Discourse in the Age of Show Business, is shockingly relevant today. He strips away the facade of mainstream media revealing its dark side and four decades later it’s all very recognizable as applicable today. Not only is it prescient, but making his points about the quaint legacy of now old school media makes everything very easy to follow, as well as demonstrating that today these same effects have kicked in orders of magnitude more so. Here are a few highlights but there is so much more in there.

In yet another breach of sensitive data (passports), Bruce Schneier makes a good point that using such powerful credentials as proof required for an unimportant purpose is the design flaw behind this operational snafu. This is the equivalent to the superintendent of a large apartment building giving the master key to someone who only needs access to a storage closet for a day.

TIL the origins of the concept of software waterfall development: it was first used to advise against the practice. There are a number of analogous examples of terminology for criticism being adopted by fans, either unaware or unswayed by the critic’s words.

The point of this fascinating story: now that we can vibe code there are all kinds of creative uses; and because these applications do not need the rigor that commercial software requires, prototyping “alpha” quality is no problem.

• The real AI; Is AI profitable?; Very Important Words the Tech Industry Ruined
• System modeling
• Guardrails?
• Anthropic sandboxing
• What’s a Virus?
• Threat model or triage guideline?
• Big Tech priorities

The real AI Seth Godin, based on Woz’s definition of “AI” (spoiler: Actual Intelligence).