Adversarial Machine Learning and Model Robustness

Part of the Applied AI Security and Assurance curriculum, authored by Cameron Hopkin, CISSP, CEH, CHFI. Published as a reference, not an enrollable course.

Description

A rigorous treatment of the machine learning attack surface and the honest limits of defenses. This is where the curriculum stops being a machine learning program and becomes a security one.

Outcomes

• Build a full threat model for a machine learning system.
• Execute evasion, poisoning, extraction, and inference attacks.
• Evaluate robustness and articulate what an evaluation does and does not prove.

Modules

1. Threat models for machine learning and the end-to-end attack surface.
2. Evasion, adversarial examples, and transferability.
3. Data poisoning and backdoors.
4. Model extraction and stealing.
5. Membership inference and privacy attacks.
6. Defenses and their limits under adaptive attackers.

Signature lab

Poison a training set and then detect the poison. Separately, extract a deployed model through its API. Deliver both attack and detection as reusable tooling.

Reading anchors

• The adversarial examples line of work.
• Poisoning and backdoor research.
• Membership inference literature.
• MITRE ATLAS as the threat taxonomy, which extends MITRE ATT&CK to adversarial tactics against AI systems [6].

Research thread

A defense that survives an adaptive attacker, not just a static benchmark.