ISC2 & IIA Joint Cyber/AI Training Day 2025 - LLMsec: A Practical Guide to Attacks and Mitigations - 10th Talk

About the ISC2 & IIA Joint Training Day

The November 17, 2025 event at Maggiano’s Little Italy in Bridgewater, NJ marked a historic collaboration between the ISC2 New Jersey Chapter and Institute of Internal Auditors (IIA) NJ Central Jersey chapter. This free, full-day training brought together cybersecurity professionals and internal auditors to address securing artificial intelligence systems.

As AI systems become embedded in business-critical functions, security professionals need to understand audit frameworks while auditors need technical security depth. This training day bridged that gap, addressing the critical need: organizations are deploying LLMs faster than they can properly audit or secure them.

Talk: LLMsec 2025: A Practical Guide to Attacks and Mitigations (Auditor-Focused Edition)

Tailoring for an Audit and Compliance Audience

This talk covered LLM security topics adapted for auditors and compliance professionals. The framing shifted to address audit-specific concerns: what controls should exist, how to test them, regulatory implications, risk quantification, and governance structures.

LLMs power business-critical applications (chatbots, developer copilots, security analysis, automated decision-making), bringing new attack surfaces traditional security models don’t address. For auditors: how do you audit systems that are probabilistic and whose behavior can’t always be predicted?

This talk covered prompt injection, jailbreaks, safety evasion, model extraction, and insecure tool integrations—contextualized with control objectives, audit procedures, evidence collection, and risk rating criteria.

Key Topics for Auditors and Compliance Professionals

Structured around auditable control domains:

• Input Validation: Pattern-based injection detection, content filtering; evidence includes validation logs and filter configurations

• System Prompt Security: Multi-layer defenses, delimiter separation; evidence includes system prompt documentation

• Output Filtering: Post-generation validation, sensitive data redaction; evidence includes DLP integration and moderation results

• Tool and Plugin Security: Whitelist-based function calling, parameter validation; evidence includes authorization matrices and execution logs

• Model Access Controls: API key management, rate limiting; evidence includes access control lists and authentication logs

• Monitoring and Logging: Real-time monitoring, anomaly detection, SIEM integration; evidence includes log retention policies and alert configurations

• Compliance and Privacy: Data minimization, consent management; evidence includes privacy impact assessments and data flow diagrams

Live Demonstrations with Audit Context

Live demos illustrated attacks and explained what auditors should look for during control tests, "good" vs. "bad" implementations, documentation approaches, and compensating controls. Each exploit paired with defensive strategies framed as auditable controls.

Audit Reporting and Risk Communication

Addressed communicating findings to stakeholders: risk rating frameworks, control maturity models, audit report structures, and board-level reporting translating technical vulnerabilities into business risks.

Practical Takeaways for Auditors

Attendees left with audit program templates, control checklists, risk assessment frameworks, testing methodologies, regulatory guidance (NIST, ISO 27001, SOC 2), and sample audit findings—bridging technical LLM security and audit practice.

Slides can be found here: View Slides