AI Security & Secure AI Developmentin Edinburgh

Module 1: Introduction to AI Security

• Understand the foundations of AI security and secure AI engineering.
  • Overview of AI, machine learning, and generative AI security.
  • Differences between traditional cybersecurity and AI security.
  • Security challenges across the AI lifecycle.

Module 2: AI Threat Landscape and Risk Assessment

• Identify security threats affecting AI systems and applications.
  • Common AI attack surfaces and vulnerabilities.
  • AI threat modelling and risk identification.
  • Security risk prioritisation and mitigation planning.

Module 3: Secure AI Development Lifecycle

• Integrate security practices into AI application development.
  • Secure requirements gathering and architecture design.
  • Security reviews during development and testing.
  • Secure deployment, maintenance, and lifecycle management.

Module 4: Large Language Model Security

• Understand vulnerabilities associated with large language models (LLMs).
  • LLM security risks and common attack patterns.
  • Model behaviour, limitations, and misuse scenarios.
  • Security considerations for LLM-powered applications.

Module 5: Prompt Injection and Input Security

• Examine attacks that manipulate AI application behaviour.
  • Direct and indirect prompt injection concepts.
  • Malicious instructions and untrusted input handling.
  • Input validation, instruction separation, and defensive design.

Module 6: Data Privacy and Information Protection

• Protect sensitive information throughout AI workflows.
  • Data leakage risks in prompts, outputs, and connected systems.
  • Data minimisation, masking, and secure data handling.
  • Privacy controls for AI training and inference workflows.

Module 7: Adversarial Machine Learning

• Explore attacks that exploit machine learning model behaviour.
  • Adversarial examples and evasion attacks.
  • Data poisoning and training data integrity.
  • Model robustness evaluation and defensive techniques.

Module 8: Model and API Security

• Secure AI models, endpoints, and application integrations.
  • Authentication and authorisation for AI APIs.
  • Rate limiting, input restrictions, and output validation.
  • Model access controls and secure endpoint configuration.

Module 9: AI Supply Chain and Dependency Security

• Understand security risks introduced by external AI components.
  • Third-party models, datasets, packages, and libraries.
  • Dependency vulnerability assessment and version management.
  • Model provenance and software supply chain considerations.

Module 10: Secure AI Architecture and Infrastructure

• Design AI environments with appropriate security controls.
  • Secure architecture for AI applications and model services.
  • Cloud security, network isolation, and secrets management.
  • Least-privilege access and environment separation.

Module 11: AI Output Security and Content Validation

• Evaluate risks associated with AI-generated outputs.
  • Insecure or misleading output handling.
  • Output validation, filtering, and content safety controls.
  • Secure integration of AI outputs with downstream applications.

Module 12: AI Security Testing and Evaluation

• Assess AI systems for vulnerabilities before deployment.
  • Security testing strategies and adversarial testing.
  • Red teaming and abuse-case analysis.
  • Vulnerability documentation and remediation verification.

Module 13: AI Security Monitoring and Incident Response

• Monitor AI systems and respond to security incidents.
  • Logging, anomaly detection, and suspicious activity monitoring.
  • AI security incident investigation and escalation.
  • Remediation, recovery, and lessons learned.

Module 14: Governance, Compliance, and Responsible AI Security

• Apply governance practices to AI security management.
  • Security policies, documentation, and accountability.
  • Overview of relevant AI security standards and frameworks.
  • Privacy, risk management, and responsible AI considerations.

Module 15: Capstone Project – Secure an AI-Powered Application

• Apply course concepts to a practical AI security scenario.
  • Define the application architecture and identify attack surfaces.
  • Prepare a threat model and AI security risk register.
  • Test selected vulnerabilities in an authorised lab environment.
  • Recommend security controls and document remediation steps.