Advanced Network Security And Malware
Instructor: Professor Sven Dietrich
Advanced network security and malware analysis is an advanced course for
individuals interested in the theory and practice of network security.
This course will study approaches, mechanisms, and tools used to make
networks and software systems more secure against malware, based on a
survey of recent and seminal research papers. We will motivate the study
by discussing common software security dangers (e.g., buffer overflow
attacks, control-hijacking), common network security dangers (malware,
botnets, APTs), and malware analysis tools. The majority of the course
will be divided into three main modules: architectural approaches to
building secure software (e.g., confinement, virtual machines, trusted
computing); software analysis (e.g., static analysis and testing, model
checking, dynamic analysis) for finding software flaws as well as
analyzing malicious code; and network detection techniques for
Students will be evaluated based on small projects, two papers, and
class participation. The projects will provide students with practical
experience with the tools and mechanisms studied in class. Students will
work on the projects in groups of two or three, and the projects will be
evenly spaced over the course of the semester.
The students are expected to have basic computer and network security
knowledge, as well as familiarity with Unix-based and Windows systems.
• Understand the state of the art in control-hijacking and associated
defenses in software systems.
• Understand the specific security architectures for system isolation
• Understand the strengths and limitations of various methods of
software analysis, and their application of
vulnerability discovery and
verification of security properties, as also applied to malware analysis.
• Understand the state of the art in botnet detection techniques.
Reference books and web resources
• Jelena Mirkovic, Sven Dietrich, David Dittrich, and Peter Reiher.
Internet Denial of Service: Attack and Defense Mechanisms, Prentice Hall
• Applied Crypto http://www.cacr.math.uwaterloo.ca/hac/
• Additional online up-to-date materials will be provided as time
progresses, mostly from research conferences in systems-oriented security.
Tentative topics (Part 1: 2.5-3 weeks; Part 2: 4 weeks; Part 3: 4.5-5 weeks)
• Part 1
◦ Ethics in Cybersecurity
▪ Menlo Report: guidelines for ethical cybersecurity research
◦ Software basics
▪ Review of the basics of code generation and trust.
▪ Buffer overflows (stack, heap, etc.)
▪ Separation, memory protection
▪ Virtual machines, sandboxing
▪ Isolation and confinement.
◦ Control-flow integrity
◦ Enforcing security properties at run-time
• Part 2
◦ Cryptography overview
◦ Trusted Computing
▪ Software security architectures
▪ Static analysis of C programs
▪ Dynamic analysis
▪ Software model checking
▪ Building verifiable systems
• Part 3
◦ Malicious code analysis
▪ Sandboxed analysis
▪ Virtual machine introspection
▪ Malicious code classification
◦ Botnet analysis & detection
▪ Activity-based (DDoS, click fraud)
▪ Command-and-Control (C&C) based
▪ Side-channel detection
▪ Correlation analysis
◦ Advanced Persistent Threats
▪ Data exfiltration
◦ Moving Target Defense
▪ Adaptive defenses
There will be weekly in-class discussion
• 25%x3 projects
• 5% class participation
• 20% for writing 2 research papers
All assignments must be your own work and submitted on time. Late
submission without prior permission will not be graded.