Language Models, Retrieval, and Agents

Part of the Applied AI Security and Assurance curriculum, authored by Cameron Hopkin, CISSP, CEH, CHFI. Published as a reference, not an enrollable course.

Description

Foundations through operations for reliable LLM applications, taught so the reliability question stays live throughout rather than being deferred to the end.

Outcomes

• Design and build retrieval-augmented generation systems and tool-using agents.
• Instrument them for retrieval quality and answer quality.
• Identify their silent failure modes.

Modules

1. LLM foundations, prompting, and context engineering.
2. Retrieval-augmented generation: indexing, chunking, and retrieval quality.
3. Tool use and function calling.
4. Agent architectures and memory.
5. Evaluation and reliability of generated output.
6. Failure analysis.

Signature lab

Build a retrieval-augmented agent, then instrument it to measure retrieval quality and answer quality and expose where it quietly breaks.

Research thread

Where agents fail without signaling it, and how an operator would know.