Secure Systems

The following is a list of possible topics for the seminar in the summer term of 2019


you need a working knowledge of IT-Security basics for this course! You also need a software development background and possibly some systems knowledge.You will learn to build damage reducing systems and we will also investigate current attacks to check our understanding, but again: this is no IT-Security course!

Theoretical Background:

How Complex Systems Fail Building A "Simple" Distributed System - Formal Verification — Jack Vanlightly Books: Nassim Taleb: Black Swans, Antifragile, Skin in the Game. On bad statistics and bad predictions. Design Patterns for Distributed Control Applications Designing Distributed Control Systems: A Pattern Language Approach Veli-Pekka Eloranta, Johannes Koskinen, Marko Leppänen, Ville Reijonen Feedback Control for Computer Systems Introducing Control Theory to Enterprise Programmers By Philipp Janert LA+/pluscal modelling, invariants based designT Designing Distributed Systems with TLA+, Hillel Wayne Applied Performance Theory, Kavya Joshi The Systems Thinker – A Lifetime of Systems Thinking Samuel Arbesman: Complexity Science + Venture Capital Exploit Programming, From Buffer Overflows to “Weird Machines” and Theory of Computation, Sergey Bratus The Good, the Bad, and the Weird, Let’s automatically identify weird machines in software. MAMADROID : Detecting Android Malware by Building Markov Chains of Behavioral Models Enrico Mariconti † ,

Systems and Resilience:

Shuffle Sharding: Massive and Magical Fault Isolation by Colm MacCarthaigh GitHub - awslabs_route53-infima: Library for managing service-level fault isolation using Amazon Route 53 Open-sourcing homomorphic hashing to secure update propagation, Kevin Lewi, Wonho Kim Using Machine Learning to Ensure the Capacity Safety of Individual Microservices | Uber Engineering Blog Safe Client Behavior, Ariel Goh (SRECon Australia, video) How to Serve and Protect (with Client Isolation), Frances Johnson, This is another excellent talk from SRECon Asia Australia about protecting a service like Google Maps (with a plethora of internal and external clients) Isolation Without Containers, Tyler McMullen on WebAssembly DSHR's Blog: Economic Models Of Long-Term Storage Serverless Security And The Weakest Link (Or How Not to Get Nuked by App-DoS).html How should I organize my AWS accounts? | #NoDrama DevOps How many AWS accounts do I need? | #NoDrama DevOps Home | Least Authority simple secure storage When AWS Autoscale Doesn’t · Segment Blog also: hacker news Errata Security: Notes on Build Hardening build_safety_of_software_in_28_popular_home_routers fast18_slides_gunawi_0 : fail slow at scale.. Building Reliability in an Unreliable World, Greg Murphy describes how GameSparks have designed their platform to be tolerant of many things: unreliable and slow internet connectivity, cloud resources that can fail without warning or suffer performance degradation, poorly-performing or resource-heavy customer code in a multi-tenant environment. Mid-Air Plane Repair: Debugging in Production Maxim Fedorov Performance and Scalability Engineer @ WhatsApp

. A good starting point to the Boing disaster...

Anatomy of a Crime: Secure DevOps or Darknet Early Breach Detection, Dr. Sarah Lewis Cortes, Salesforce Securing a Security Company, Patrick Cable, Threat Stack, Inc. Laura Nolan: Black Swans - what breaks our systems Keynote: High Reliability Infrastructure Migrations - Julia Evans, Software Engineer, Stripe - YouTube Close Loops & Opening Minds: How to Take Control of Systems, Big & Small, Colm MacCarthaigh (slides and video) Is it Possible to Test Programmable Infrastructure? Matt Long at QCon London Made the Case for "Yes" Canary Analysis Service, Automated canarying quickens development, improves production safety, and helps prevent outages. Štěpán Davidovič with Betsy Beyer

Static software analysis:

clusterfuzz, testing, Autonomous Testing and the Future of Software Development, Will Wilson (AI-based testing) The Hurricane’s Butterfly: Debugging pathologically performing systems, Bryan Cantrill Ghidra: NSA stellt quelloffenes Software-Analyse-Tool vor | heise online


Google Researchers Say Spectre Will Haunt Us for Years. In this context see the paper on time as a missing abstraction in OS design Kalaschnikow: Preisgünstige Kamikaze-Drohne für kleine Armeen Spectre is here to stay, An analysis of side-channels and speculative execution, Ross Mcilroy, google Semiconductor Engineering .:. Chasing Reliability In Automotive Electronics electromechanical switching in the Bell System,..Hacker News Ginseng: keeping secrets in registers when you distrust the operating system April 5, 2019


Nancy G. Leveson, Engineering a Safer World, Systems Thinking Applied to Safety safety culture movement (see blog.mi... Alexander Wallrabenstein) Normalization of Deviance | Art is Art and Water is Water Semiconductor Engineering .:. ISO 26262-Functional safety Highly Automated Vehicle Safety Validation Why Silicon Valley’s “growth at any cost” is the new “unsafe at any speed” | Ars Technica Designing for Failure: How to Manage Thousands of Hosts Through Automation Monday, October 29, 2018 - 2:00 pm–2:30 pm, Brandon Bercovich, Uber


No! February 2019, Geoff Huston, What part of “No!” doesn’t the DNS understand?

Languages - Memory Safety, Type Safety and Thread Safety:

Microsoft: 70 percent of all security bugs are memory safety issues | ZDNet Simon Sapin on Twitter: "“[…] still had a buffer overrun discovered in 2016 (in code added in the 2001 and 2002) Diane Hosfelt, The Most Secure Program Is One That Doesn’t Exist (Rust) Fresh Async With Kotlin, Roman Elizarov Kotlin Native Concurrency Model, Nikolay Igotti What do you mean “thread-safe”?, Geoffrey Romer The Dos and Donts of Error Handling, Joe Armstrong Using Rust for Game Development, Catherine West Understanding Real-World Concurrency Bugs in Go, Tengfei Tu NETSCOUT Threat Intelligence Report, DAWN OF THE TERRORBIT ERA, Findings from Second Half 2018


If you want, I can store the encrypted password." A Password-Storage Field Study with Freelance Developers, Naikashina" Triton is the world’s most murderous malware, and it’s spreading - MIT Technology Review.html Alphabet’s Security Start-Up Wants to Offer History Lessons - The New York Times state-of-the-internet-security-retail-attacks-and-api-traffic-report-2019 Keep Calm and Authenticate: Why Adaptive is the Next Best Thing IAM and Account Modelling in AWS A First Look at the Crypto-Mining Malware Ecosystem: A Decade of Unrestricted Wealth, Sergio Pastrana Small World with High Risks: A Study of Security Threats in the npm Ecosystem, Markus Zimmermann Here's How the 2.09 Million EOS "Hack" Really Happened Once hailed as unhackable, blockchains are now getting hacked - MIT Technology Review Winning Systems & Security Practitioners 7. Attack Surface Reduction · Privacy, Power, & Protection In The Cyber Century Remotely compromise devices by using bugs in Marvell Avastar Wi-Fi: from zero knowledge to zero-click RCE – Embedi 3 ways state actors target businesses in cyber warfare, and how to protect yourself - TechRepublic SD-WAN_-_35C3_-_publishHow to hack software defined network and keep your sanity? Why Software Remains Insecure The benefits of quickly building bad software have so far outweighed the downsides, By Daniel Miessler in Information Security Modern Web Security, Lazy But Mindful Like a Fox, by Albert Yu Managing Secrets at Scale, by Mark Paluch The Untold Story of NotPetya, the Most Devastating Cyberattack in History | WIRED Crpytography that can't be hacked.


On Infrastructure at Scale: A Cascading Failure of Distributed Systems What Bugs Live in the Cloud?A Study of 3000+ Issues in Cloud Systems, Gunawi Metadata: Paper review. An Empirical Study on Crash Recovery Bugs in Large-Scale Distributed Systems Alpha Dominche Shuts Down: Is Commercial Coffee Tech Dead? | The Spoon, STAMPing on event-stream • Hillel Wayne Horrible story on javascript, npm and third party libraries. The Biggest IT Failures of 2018 - IEEE Spectrum Reviewing-Oppenheimer_-_Why-do-internet-services-fail Gray Failure: The Achilles’ Heel of Cloud-Scale Systems Peng Huang Microsoft Researc Software won't fix Boing's faulty airframe What We Learned from the Recent Mandrill Outage

Critical Infrastructure

Alle Jahre wieder: Frankreich am großen Blackout vorbeigeschrammt | Telepolis Die technisch hochgerüstete Gesellschaft ist verletzlicher denn je | NZZ PG&E insolvent: Waldbrände führen zu Klagen gegen US-Stromfirma Cyberattack on Venezuela?


Gartner: A Look at Emerging Types of Machine Learning for Fraud Detection It’s time for Practical AI, "We put brains in your hardware" SAFETY SECURITY MEDICAL Knstliche Intelligenz: Überall in Europa entscheiden schon Algorithmen | heise online Aadversarial WiFi Sensing using a Single Smartphone, Yanzi Zhu The crux of voice (in)security: a brain study of speaker legitimacy detection April 1, 2019


The Future of War, and How It Affects YOU (Multi-Domain Operations) - Smarter Every Day 211 - YouTube Missing Link: Überwacht die Überwacher, oder: Klagen gegen den Präventionsstaat | heise online Europäische Standards-Organisation warnt USA vor TLS 1.3 | heise online_files Geopolitics For Fun & Profit Money Machines: behind the financial industryAn Interview with an Anonymous Algorithmic Trader Missing Link: Predictive Policing - Verbrechensvorhersage zwischen Hype und Realität | heise online Die Geräte der Hacker im Überblick, WLANs stören, Tastatureingaben auslesen, heimlich Screenshots anfertigen – mit speziellen Hacker-Geräten kein Problem. Wer die Tools kennt, kann sich schützen.

Online Voting: | heise online, DARPA Secure Hardware Software Architecture

HCI-AI Interaction: 737 Max,

Embedded Control

New approaches to secure embedded systems? Microsoft solution with cloud and Linux OS and special hardware? How does it look?

Here is the list from summer 2018. You can find the results on

  1. Will we soon use fingerprints etc. in browsers to authenticate against services? A new standard evolves: WebAuthN

  2. Hardware Security: covert channels, race conditions, boot and system management and other weak points. Methods to find problems e.g. in hardware transactional memory. Retpoline from Google. Silent Corruptions, KELEMEN Péter, CERN IT. Hardware Architectures for Software Security, Joshua N. Edmison, Diss. Virginia Polytec. Institute.

  3. Trusted root of systems. Taking Teslas solution for cars we could look at the general principles of building a trusted root and try to come up with something for IoT. There is a larger report on the Tesla solution available.

  4. Adversarial Neuronal Networks. I would like to continue this topic from last term as it has the potential to affect NN use almost everywhere. How can we test NNs? How do we calculate reliability? Also: The Malicious Use of Artificial Intelligence, Forecasting, Prevention, and Mitigation, a very interesting study by lots of researchers.AAAI trip report, lots of adversarial stuff . Take a look at Jonas Miederers presentation in blog.mi.

  5. Resilience, system stability and change, robustness: what does the theory on cybernetics, complex adaptive systems and system theory say about damage resistence? A look at Nassim Taleb's Antifragility book might help too. How complex systems fail.

  6. Cloud security and the BeyondCorp approach of Google: Intranet is dead!

  7. Secure Architectures for Critical Infrastructures. How can we secure CI given all the security problems of hardware and software? Renn, Ortwin (Hrsg.): Das Energiesystem resilient gestalten: Szenarien – Handlungsspielräume – Zielkonflikte (Schriftenreihe Energiesysteme der Zukunft), München 2017.

  8. Formal Methods for large scale architectecures: TLA+ by Leslie Lamport. How to Build Static Checking Systems Using Orders of Magnitude Less Code, Fraser Brown, Andres Noetzli, Dawson Engler, Stanford Univ.. also Snarky, a high level language for verifiable computation

  9. Continuous Deployment and its security problems.DevOOPs: Attacks And Defenses For DevOps Toolchains Insomni'hack 24 March 2017

  10. Crash consistend applications (not all file-systems are created equal). Self-controlling software? Self-healing software? This topic will become more important with autonomous devices. Reducing Crash Recoverability to Reachability, Eric Koskinen Junfeng Yang Yale University Columbia University, Redundancy Does Not Imply Fault Tolerance: Analysis of Distributed Storage Reactions to Single Errors and Corruptions Aishwarya Ganesan, Ramnatthan Alagappan, Andrea C. Arpaci-Dusseau, and FAST 18 papers on fault-tolerant storage and what happens When during filesystem restore operation the power fails again?

  11. RustBelt: Securing the foundations of the Rust programming language Jung et al., POPL, 2018. Language security and formal verification. We should take a look at the ownership concept and compare it to an object capability. I have a tummy feeling that we are talking about the same thing here. And creating an object capability with exclusive but transferable ownership is just a capability desing pattern. And not to forget: how will Java fix the "unsafe" feature? (Rust has it also). Another thing: Checked C is an extension for the C language that is supposed to avoid buffer overruns etc. Worth looking at it as most system software is still written in C/C++.

  12. Serverless computing (FAAS): what are the implications for security? Serverless Security

  13. web assembly based on object capability principles? for caps: What are Capabilities?

  14. High-Assurance Cyber Military Systems (HACMS): Baking Hack Resistance Directly into Hardware. how does it work?

  15. A list of post mortems and what we can learn from them. Danluu, github

  16. The Importance of Features for Statistical Anomaly Detection David Goldberg, Yinan Shan, EBAY

  17. The fight for better software and systems, history: A design methodology for reliable software systems, by B. H. LISKOV, The MITRE Corporation, Bedford, Massachusetts (1972). Bernard Meyer's Design by Contract?

  18. Security Industrial Complex: EUs framework program on security. Military companies, boarder security etc. A view on global developments in military and surveillance industry.

  19. Investigate some attacks and vulnerabilities: beA (mail to lawywers), meltdown, ...Traffic hacksJavascript side channel attacks and Server side javascript injection

  20. Grey failures are failures which are very hard to find as they get frequently masked by fault-tolerant features of systems. They reduce theoretical availability and can lead to an unexpected system crash. A paper from azure engineers explains some of the problems found. Gray Failure: The Achilles’ Heel of Cloud-Scale Systems

  21. Smart Meter: how do we secure devices for IoT and Smart Home use? 9 vendors want certification from the BSI. Technological, economical and ecological considerations.

  22. How can Byzantine Protocols prevent attacks from participating nodes? What are the costs? What are the failure assumptions (DOS, wrong protocol etc.). Are those protocols alternatives for critical infrastructures. An example given by Liskov and Castro which was discussed on morningpaper. A comparison of blockchain consensus and byzantine consensus in Murat's paper summary of Bitcoin-NG

  23. The financial subsystem has shown a tendency for disaster several times in the past. Can we measure the risk inside this subsystem? There is an interesting EU project (part of FP7) that tries to identify and measure the risk. Systemic Risk Tomography Signals, Measurements, Transmission Channels and Policy Interventions . And while we are at it: let's take a look at current EU programs in security. Dan Geer just wrote a very interesting paper on global risks and interdependencies dubbed A Rubicon

  24. Bruce Schneier says that AI mightt tip the balance more towards defense against attacks because it does not suffer from the human weaknesses (speed, errors, scale). How will the black hats counter this development? And how can black hats use AI? General ML Security

  25. The train system of a country is certainly a critical infrastructure. We might be able to take a closer look at the german bundesbahn thanks to a contact I got recently.

  26. Evoting has been a topic for security a while ago. Modern societies will have to use the internet to increase participation. Hack-profing elections will be key to this.

  27. Is Quantum Computing a threat to security and which algorithms are affected?

  28. Smart Contracts are still a hot topic. Can we verify those automatically? Another look at smart contracts in Ethereum S.C.. What kind of improvements for security are possible with a not-turing-complete language?

  29. The autonomous war: robots, drones and other technology against human beings. How far are we? Killer Robots stoppen. And not to forget: there is a tight connection to the Financial Industry

  30. Hard words: Linus Torvalds, Public Mar 13, 2018, It looks like the IT security world has hit a new low. If you work in security, and think you have some morals, I think you might want to add the tag-line "No, really, I'm not a whore. Pinky promise" to your business card. Because I thought the whole industry was corrupt before, but it's getting ridiculous. At what point will security people admit they have an attention-whoring problem? We can take a look at the current state of the security-industrial complex.

  31. Human Performance and Software Errors. Its time to study software problems in detail because they are very much linked to human behavior and cause major loss of money or lives nowadays. Software Troubles

  32. Security is never far from legal problems. John Kingston wrote a paper on AI and Legal Liability which covers current opinions on this difficult subject.

  33. The world economic forum on: Environment and Natural Resource Security

  34. Two nice publications for secure system research. First, Future Now Institute published its report on 225 trends and a list of post mortems and lessons learned appeared on Github.

  35. Preparedness is a big part of a resilient system. Netflix uses its Simian Army/Chaos Monkeys to cause serious errors in production, just to test their resilience.

If you want to see what we did in past terms, take a look at my blog entry on 2016 .