This page is a collection of references and links for various topics mentioned in Designing Secure Software: a guide for developers.
It was surprising how quickly links changed during the writing of this book – I checked these while editing new drafts, the technical reviewer then found several that had changed, and then in production several more needed changing. Reports of errors, omissions, or broken links are welcome: please contact me via Linked In with details.
Updated: August 24, 2023
“For me, what makes life enjoyable is having a shared culture and shared references.” — Michael Sheen
Quick links: Intro 1 2 3 4 5 6 8 9 10 11 12 13 Afterword Exercises (none for Chapter 7)
Introduction
Read the original Microsoft technical article on threat modeling, “The Threats to Our Products” by Praerit Garg and this book’s author (April 1, 1999) at https://shostack.org/resources/early-threat-modeling
Learn all about the generally very useful skill of using thinking hats from the book Six Thinking Hats by Edward de Bono (Back Bay Books, 1999): https://www.debonogroup.com/services/core-programs/six-thinking-hats/
Explore the ancient computing history of the IBM System/360 and coding in Basic Assembly Language: https://www.righto.com/2019/04/iconic-consoles-of-ibm-system360.html?m=1 and http://bitsavers.trailing-edge.com/pdf/ibm/360/asm/SC20-1646-6_int360asm_Aug70.pdf
For my 1978 MIT thesis, see Chapter 5 references.
Learn about the “Princeton Word Macro Virus Loophole” in “The History of Internet Explorer” by Scott Schnoll: https://courses.cs.duke.edu/spring01/cps049s/class/html/iehistory.htm
Revisit the bad old days of rampant browser bugs in “Internet Explorer Security Issues (1996–2002)”: https://web.archive.org/web/20110714214614/http://www.nwnetworks.com/96-02iesecurity.htm
Chapter 1: Foundations
Read the newspaper report of the automobile test drive that went bad, “Car Thief to Pay for Damages,” by Caleb Loehrer (The Garden Island, Sunday, April 7, 2019): https://www.thegardenisland.com/2019/04/07/hawaii-news/car-thief-to-pay-for-damages/
Learn about IMDb deanonymization: https://www.cs.utexas.edu/~shmat/shmat_oak08netflix.pdf
Read about the Twitter security incident involving the logging of raw passwords: https://blog.x.com/en_us/topics/company/2018/keeping-your-account-secure
Chapter 2: Threats
Read the original Microsoft technical article on threat modeling, “The Threats to Our Products” by Praerit Garg and this book’s author (April 1, 1999) at https://shostack.org/resources/early-threat-modeling
As an example of hardware as an asset to be attacked, it is widely believed that the US created a cyber weapon that came to be known as Stuxnet that was designed to infect industrial centrifuge equipment and control it in such a way as to irreparably damage it: https://www.wired.com/2014/11/countdown-to-zero-day-stuxnet/
Learn about MEDJACK, an example of medical device exploits: https://www.cs.tufts.edu/comp/116/archive/fall2018/smeggitt.pdf
Get a quick overview of some prominent threat modeling methodologies: https://en.wikipedia.org/wiki/Threat_model#Generally_accepted_IT_threat_modeling_processes
Learn the details of the Facebook Beacon engagement ring fiasco: https://money.cnn.com/galleries/2010/technology/1012/gallery.5_data_breaches/3.html
Check out pre-2000 browser share statistics: https://en.wikipedia.org/wiki/Usage_share_of_web_browsers#Older_reports
Learn about Microsoft threat modeling tools:
- https://docs.microsoft.com/en-us/azure/security/azure-security-threat-modeling-tool
- https://github.com/Microsoft/threat-modeling-templates
- https://en.wikipedia.org/wiki/Data_flow_diagram
Take a look at some STRIDE examples:
-
BGP hijacking: https://www.cloudflare.com/learning/security/glossary/bgp-hijacking/
-
Superfish ad injection: https://www.cisa.gov/news-events/alerts/2015/02/20/lenovo-superfish-adware-vulnerable-https-spoofing
-
Spectre/Meltdown: https://meltdownattack.com/
-
MemCrashed: https://blog.cloudflare.com/memcrashed-major-amplification-attacks-from-port-11211/
-
SQL injection: https://xkcd.com/327/
Read why data flow diagrams (DFDs) may be insufficiently detailed for threat modeling: https://lirias.kuleuven.be/retrieve/571819
Chapter 3: Mitigations
Read about Code Access Security (CAS), now deprecated by Microsoft: https://docs.microsoft.com/en-us/dotnet/framework/misc/code-access-security
Chapter 4: Patterns
The term “patterns” is loosely based on the classic architecture text A Pattern Language: Towns, Buildings, Construction by Christopher Alexander, Sara Ishikawa, and Murray Silverstein (Oxford University Press, 1977).
Read about how a kid went right around Apple’s enforcement of screen time limits: https://web.archive.org/web/20221006220145/https://www.reddit.com/r/ios/comments/g8upnn/i_saw_that_my_8yearold_sister_was_on_her_iphone_6/
Learn about Web Assembly technology: https://webassembly.org/
Read about secure execution of Web Assembly: https://webassembly.org/docs/security/
Read the text of State of California Senate Bill No. 327 (2018): https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180SB327
The allowlist and blocklist example based on COVID-19 restrictions is from Hawaii’s Eighth Supplementary COVID-19 Proclamation: https://dod.hawaii.gov/hiema/eighth-supplementary-proclamation-covid-19/
Chapter 5: Cryptography
The US National Institute of Standards and Technology (NIST) publishes detailed recommendations for cryptography algorithms and guidelines: https://csrc.nist.gov/Projects/Cryptographic-Standards-and-Guidelines/
Read about Cloudflare’s lava lamp collection that serves as a random number generator: https://blog.cloudflare.com/randomness-101-lavarand-in-production/
Learn about the Navajo Code Talkers of WWII: https://americanindian.si.edu/nk360/code-talkers/
Read the original RSA paper, “A Method for Obtaining Digital Signatures and Public-Key Cryptosystems,” by Ron L. Rivest, Adi Shamir, and Leonard Adleman: https://web.archive.org/web/20230117055133/https://people.csail.mit.edu/rivest/Rsapaper.pdf
My published contribution to the RSA algorithm predates the ACM online digital library, but I was able to obtain a scan posted here. On the Signature Reblocking Problem in Public-Key Cryptosystems by Loren Kohnfelder, Communications of the ACM 21 (Feb. 1978), page 179.
Towards a Practical Public-Key Cryptosystem, is so old that I cannot recommend reading it except as a historical source: https://dspace.mit.edu/handle/1721.1/15993
Read a review of popular crypto libraries that evaluates usability, documentation, and the security of how they are used by test subjects: https://www.cs.umd.edu/class/fall2017/cmsc396H/papers/comparing-crypto-apis.pdf
Read about how security cameras unknowingly breach the privacy of their customers. Images from motion detecting cameras are encrypted, but by observing the presence or absence of traffic on the wire, attackers can infer whether the house is occupied or not because people inside trigger the cameras: https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/65154/Tyson Your Privilege Gives 2020 Accepted.pdf
Chapter 6: Secure Design
Read the design philosophy of the *nix operating system, to appreciate how well it has stood the test of time. http://www.catb.org/~esr/writings/taoup/html/philosophychapter.html
The company that I refer to mixing endianness in “Making Design Assumptions Explicit” was Elxsi (electronics × silicon): https://en.wikipedia.org/wiki/Elxsi
Read more about “known knowns,” including the original source of the term: https://en.wikipedia.org/wiki/There_are_known_knowns
Read about the Equifax breach: https://www.equifaxsecurity2017.com/
A study by Charles Perrow concluded that the failure at Three Mile Island was a consequence of the system’s immense complexity: https://www.nature.com/articles/477404a
Chapter 8: Secure Programming
Read Mitre’s top security bug rankings by category: https://cwe.mitre.org/top25/
See an excellent reference for securely using floating-point math: floating-point-gui.de
Get an overview of best practices for handling people’s names in software: https://www.w3.org/International/questions/qa-personal-names/
In the 2020 Pwn2Own event the @SSLab_Gatech team used six unique vulnerabilities, starting with a JIT bug and ending in a TOCTOU (race condition): https://x.com/thezdi/status/1240360442310074370
Learn all about Spectre and Meltdown attacks: https://meltdownattack.com/
Learn more about regular expression backtracking, which can incur heavy computation costs: https://www.regular-expressions.info/catastrophic.html
The footgun mentioned as an example (=
for ==
) was used by an attacker
to insert malicious code into the Linux kernel in 2003. See “The Linux
Backdoor Attempt of 2003,” by Ed Felten:
https://freedom-to-tinker.com/2013/10/09/the-linux-backdoor-attempt-of-2003/
More examples and discussion of footguns in the context of how to write underhanded code can be found in “Initial Analysis of Underhanded Source Code,” by David A. Wheeler: https://www.ida.org/-/media/feature/publications/i/in/initial-analysis-of-underhanded-source-code/d-13166.ashx
Here are some links if you want to learn more about the complete details of GotoFail (you can find much more by searching for its unique moniker, or check out https://gotofail.com/):
-
NIST vulnerability details: https://nvd.nist.gov/vuln/detail/CVE-2014-1266
-
Apple support posting: https://support.apple.com/en-us/HT202934
-
Speculation on this bug: https://www.cs.columbia.edu/~smb/blog/2014-02/2014-02-23.html
-
Source code (the excerpt in the book is based on the SSLVerifySignedServerKeyExchange function): https://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c
Chapter 9: Low-Level Coding Flaws
Some C compilers provide nonstandard assistance for handling arithmetic overflow. For example, see GNU Compiler Collection (GCC) built-in helper functions documentation: https://gcc.gnu.org/onlinedocs/gcc/Integer-Overflow-Builtins.html
The JavaScript underflow example code is online: https://jsfiddle.net/j0mrLwp7/
Here are some resources for learning more about Heartbleed:
-
The Heartbleed bug no longer has its own website (heartbleed.com). Instead, see the Wikipedia article.
-
Read how the “Heartbleed bug affects ‘almost everyone’”: https://www.digitaltrends.com/computing/internet-security-cloud-experts-weigh-heartbleed-bug/
-
The TLS heartbeat protocol is defined in RFC 6520: https://datatracker.ietf.org/doc/html/rfc6520
-
Here’s the vulnerable source code file: https://opensource.apple.com/source/Security/Security-55471/libsecurity_ssl/lib/sslKeyExchange.c
-
Here’s the fix for Heartbleed: Add heartbeat extension bounds check.
Chapter 10: Untrusted Input
Read the full story of how Dmitry Agarkov modified “small print” of his credit card agreement into exceptionally lucaritve terms: https://www.nasdaq.com/articles/updated-russian-man-turns-tables-on-bank-changes-fine-print-in-credit-card-agreement-then
Unicode Technical Report 36, “Unicode Security Considerations,” is a great detailed reference covering the issues mentioned briefly in this chapter: https://www.unicode.org/reports/tr36/
The source of the story about the intramural softball team named “No Game Scheduled” is Reddit, but factual or not, it’s a brilliant example of an injection attack (please only use this special power for good): https://web.archive.org/web/20221006201220/https://www.reddit.com/r/WhitePeopleTwitter/comments/lbnln3/name_of_our_intramural_softball_team_is_no_game/
The aphorism that opens the Character String Vulnerabilities section is from The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky (2003) https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/
Chapter 11: Web Security
Learn about Content Security Policy (CSP): https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP
Learn about Cross-Origin Resource Sharing (CORS): https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS
Learn about Let’s Encrypt, a free and open certificate authority operated by the nonprofit Internet Security Research Group (ISRG) that provides automated certificate issuance and renewal for web servers: https://letsencrypt.org/
Check out statistics on the rapid growth of HTTPS: https://letsencrypt.org/stats/
Read details on how domain names are treated as independent authorities: https://publicsuffix.org/
The Browser Security Handbook is a terrific compendium of web browser security technical details, but sadly it’s no longer actively maintained and many years out of date: https://code.google.com/archive/p/browsersec/wikis/Main.wiki
The Tangled Web: A Guide to Securing Modern Web Applications, by Michal Zalewski (No Starch Press, 2011), is based on the Browser Security Handbook (also written by Zalewski, also years out of date): https://nostarch.com/tangledweb.htm
Web technology is hypercompetitive because pushing through a new feature that gets into the world’s browsers provides incredible leverage. As a result, there are many competing standards bodies. Here are some of the best-known standards bodies, and the technologies they’re responsible for:
-
World Wide Web Consortium (W3C): HTTP, HTML, XHTML, CSS, DOM, PNG, SVG, and more
-
Web Hypertext Application Technology Working Group (WHATWG): “Living Standard” (a.k.a. HTML5) and more
-
Ecma International: JavaScript (also known as Ecma Script)
-
International Organization for Standardization (ISO): JPEG
-
Unicode Consortium: Unicode Standard and Unicode Technical Reports (UTRs)
-
Internet Engineering Task Force (IETF): Request for Comments (RFC) documents
-
Internet Assigned Numbers Authority (IANA): Name and number registries
Chapter 12: Security Testing
Learn about the MinUnit test framework used in C examples: https://github.com/siu/minunit/
Read about the iOS jailbreak vulnerability regresesion mentioned in the Security Regression Tests section: https://arstechnica.com/information-technology/2019/08/ios-vulnerability-that-let-you-jailbreak-your-iphone-is-once-again-dead/
The security test case example in “Testing for XSS Vulnerabilities” uses the excellent BeautifulSoup library for HTML parsing (and it’s a great XML parser, too): https://www.crummy.com/software/BeautifulSoup/bs4/doc/
The 32-bit counter overflow example in “Threshold Testing” refers to BaseCamp’s events table incident of 2018: https://m.signalvnoise.com/postmortem-on-the-read-only-outage-of-basecamp-on-november-9th-2018/
The leak detection example in “Leveraging Integration Testing” refers to the 2018 Twitter security alert: https://blog.x.com/en_us/topics/company/2018/keeping-your-account-secure.html
Chapter 13: Secure Development Best Practices
Read the ARIANE 5 Flight 501 Failure Report: https://web.archive.org/web/20230716034019/http://sunnyday.mit.edu/nasa-class/Ariane5-report.html
This is why you must check carefully when you copy-paste code from the web: the most frequently copied code snippet on GitHub turned out to have a bug. This wasn’t a security bug, but it could have been: https://programming.guide/worlds-most-copied-so-snippet.html
Read about how “dependency confusion” allowed a security researcher to inject code into dozens of organizations’ own code: https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
Here is just one example of how a component intended to bolster security can weaken it severely: https://nvd.nist.gov/vuln/detail/CVE-2019-3597
Read “The Most Dangerous Code in the World: Validating SSL Certificates in Non-Browser Software,” by Martin Georgiev et al.: https://www.cs.utexas.edu/~shmat/shmat_ccs12.pdf
Read about malicious look-alike libraries in PyPi: https://www.nbu.gov.sk/skcsirt-sa-20170909-pypi/
Afterword
Read “Reflections on Trusting Trust” by Ken Thompson: https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf
Also, see the original code that actually carries out the attack described. https://research.swtch.com/nih
Learn more about the by-design flaws in the automotive systems network CAN bus protocol design: https://medium.com/@lorenkohnfelder/unpatchable-security-flaws-8941c0cce790
See an example of unhelpful security fix documentation quoted in the book (but by no means is this practice specific to any particular software vendor): https://support.apple.com/en-us/HT210919
Read the full report on effective smartphone data security, “Data
Security on Mobile Devices: Current State of the Art, Open Problems, and
Proposed Solutions,” from Johns Hopkins University:
https://arxiv.org/abs/2105.12613
(Ironically, and very sadly, the website securephones.io
has an expired TLS certificate as of January 2023.)
I have written about why the demands of law enforcement to provide backdoor access only for them is missing the point: https://medium.com/@lorenkohnfelder/lets-move-beyond-the-ominous-going-dark-f2f20a932ca9
For my thoughts on even more speculative future directions of security, see the Rethink Security interview “A Conversation with Loren Kohnfelder”: https://rethinksecurity.io/posts/a-conversation-with-the-father-of-pki/
Appendix C: Exercises
Two links printed in the book’s text (both on page 271) have broken since publication.
-
To learn about Chromium security issues, see https://www.chromium.org/Home/chromium-security/
-
For underhanded crypto, see the Web Archive https://web.archive.org/web/20221205164132/https://underhandedcrypto.com/