Breaking GenAI: Offensive Security for Modern AI System
Training Overview
Breaking GenAI: Offensive Security for Modern AI Systems is an advanced, hands-on training at CSA XCON 2026, Dehradun, focused on red teaming modern AI and GenAI systems used in real-world, high-risk environments.
As AI systems rapidly move from experimentation to production, red teaming AI is no longer optional. Global regulations, enterprise governance requirements, and real-world incidents now mandate systematic testing of foundational models and AI-powered applications before deployment. This training equips security professionals with the skills to proactively identify security, safety, and responsible AI failures across the AI lifecycle.
Participants will learn how to assess AI systems holistically by combining traditional security testing, adversarial machine learning, and Responsible AI red teaming techniques.
About the Training at CSA XCON 2026
This training is built on real-world red team methodologies used to assess production-grade AI systems, including large language models and multimodal AI applications.
Key highlights include:
- Red teaming AI systems beyond prompt injection and jailbreaks
- Understanding AI architecture, deployment patterns, and real-world risks
- Hands-on attacks against custom-built GenAI applications
- Mapping AI failures using industry frameworks and taxonomies
- Practical guidance on strengthening AI systems before release
The course strongly aligns with CSA XCON’s focus on future-ready cybersecurity, addressing one of the most critical and fast-evolving threat landscapes
What You Will Learn
Participants will gain hands-on experience with:
- AI and GenAI system architecture and threat surfaces
- Red teaming Large Language Models (LLMs) and multimodal models
- Prompting strategies for offensive testing: Zero-shot and few-shot prompting & Retrieval Augmented Generation (RAG) abuse
- Adversarial AI attack techniques including: Prompt manipulation and perturbation, Data poisoning & Model inversion and leakage
- Testing multimodal systems such as text, speech, and image models
- Using open-source tooling for AI red teaming: Hugging Face, LangChain, Semantic Kernel, Counterfit & NeMo Guardrails
- Mapping failures using: MITRE ATLAS & OWASP Top 10 for LLMs
- Identifying both intended and unintended failure modes
Training Experience & Expectations
- Fully hands-on and lab-driven training
- Interactive red teaming exercises against real AI applications
- Focus on methodology and failure analysis, not just tools
- Learn how attackers think about exploiting AI systems
- Practical mitigation strategies discussed after each attack phase
Participants will leave with a structured approach to breaking and securing AI systems in enterprise environments.
Regulatory & Mitigation Focus
In the advanced modules, participants will:
- Explore mitigation strategies for advanced AI threats
- Understand current and emerging AI regulatory expectations
- Learn how to conduct structured AI red teaming aligned with compliance needs
- Use a practical checklist of AI harms across: Base models, AI applications & Additional system features layered on top
This ensures learnings are immediately applicable to real-world governance, risk, and compliance requirements.
Who Should Attend
This training is ideal for:
- Information Security professionals
- Offensive security and red team engineers
- AI and ML practitioners building or deploying GenAI systems
- Security architects responsible for AI risk
- Professionals involved in AI governance and assurance
Skill Level
Intermediate (No prior hardware hacking experience required)
Participant Requirements
Participants should have:
- Basic knowledge of Python
- Ability to write simple scripts
- A laptop with internet access
All major operating systems are supported.
What Participants Will Receive
Each participant will receive:
- Access to AI applications and models for hands-on labs
- API access required for exercises (no personal subscriptions needed)
- Structured lab scenarios and red team playbooks
- Practical mitigation frameworks and checklists
- Exposure to real-world AI red teaming methodologies
Trainer
This training will be delivered by senior members of a globally recognized AI Red Team, with extensive experience assessing high-risk AI systems, foundational models, and AI-powered copilots in production environments. Their work has directly influenced modern AI red teaming practices across security, safety, and responsible AI domains.