AI Security & Secure AI Development in Heard Island and McDonald Islands

Problem Statement: Securing AI Applications in Banking

A hypothetical banking institution introduces AI-powered applications to improve customer support, assist fraud investigation, analyse financial documents, and provide employees with faster access to banking information. These applications integrate large language models (LLMs), internal knowledge repositories, customer service APIs, and financial data systems.

During a security review, the bank identifies several potential vulnerabilities. An AI chatbot may be manipulated through prompt injection, confidential information could be exposed through inappropriate data access, and insufficient API controls could allow unauthorised requests. The bank is also concerned about malicious documents entering its retrieval system, insecure third-party dependencies, and AI-generated responses containing misleading financial information.

The institution needs a secure AI development approach that protects sensitive banking information, reduces attack surfaces, establishes access controls, and supports ongoing security monitoring without compromising legitimate banking operations.

How the Challenges Were Overcome

The bank adopts a structured AI security approach that incorporates threat modelling, secure application design, vulnerability testing, and continuous monitoring.

• AI Threat Modelling: Identified potential attack surfaces across the chatbot, LLM, retrieval system, APIs, document repositories, and supporting infrastructure.
• Prompt Injection Protection: Introduced input-handling safeguards, separation of trusted instructions from retrieved content, and security testing for direct and indirect prompt injection attempts.
• Data Access Protection: Applied role-based access controls and permission checks so that AI responses could only use information authorised for the requesting user.
• Sensitive Data Safeguards: Reviewed data handling practices, implemented appropriate masking and redaction controls, and restricted the exposure of confidential financial and personal information.
• API and Infrastructure Security: Strengthened authentication, authorisation, rate limiting, secrets management, and secure configuration across connected services.
• AI Output Validation: Introduced response checks to identify unsupported financial claims, sensitive information disclosure, and unsafe instructions before responses reach users.
• Dependency and Model Reviews: Established review procedures for third-party models, libraries, datasets, and external AI services used within the application.
• Security Monitoring: Implemented logging, suspicious activity detection, incident escalation, and periodic security reviews to support ongoing risk management.

These measures help the bank establish multiple layers of protection while recognising that no single safeguard can eliminate every AI security risk.

Overall Implementation: Secure AI Development in Banking

Define the AI Application and Security Requirements

The team documents the intended use of the banking assistant, its users, connected systems, data sources, and security requirements. Sensitive operations are distinguished from general information requests, and system boundaries are clearly defined.

Conduct AI Threat Modelling

Potential threats are mapped across user inputs, document retrieval, model processing, API integrations, and generated outputs. The team evaluates risks such as prompt injection, data leakage, unauthorised access, malicious documents, and model misuse.

Implement Secure Architecture and Access Controls

The application is designed using least-privilege access, authenticated APIs, permission-aware retrieval, secure secrets management, and appropriate separation between development, testing, and production environments.

Secure Data and Retrieval Workflows

Banking documents are classified according to sensitivity, and retrieval permissions are enforced before relevant content is supplied to the model. Untrusted documents are treated as data rather than instructions, and sensitive information is protected through appropriate handling controls.

Perform Security Testing and Validation

The team conducts authorised security tests using representative attack scenarios. Testing covers prompt injection, access-control failures, information leakage, insecure API usage, malicious document content, and unsafe model outputs. Findings are documented, prioritised, and retested after remediation.

Establish Monitoring and Incident Response

Security events are recorded and reviewed to identify suspicious requests, repeated attack attempts, and possible data exposure. Incident response procedures define escalation responsibilities, investigation steps, containment measures, and post-incident improvements.

Review and Improve Security Controls

The organisation periodically reassesses the application as models, dependencies, data sources, and business requirements change. Security findings and test results inform updates to architecture, controls, documentation, and training.

Learner Outcomes and Evidence

Through this banking security scenario, learners can demonstrate practical understanding by preparing the following deliverables:

• AI Threat Model: Document the banking application's assets, trust boundaries, attack surfaces, threats, and potential consequences.
• AI Security Risk Register: Record identified vulnerabilities, likelihood and impact assessments, mitigation priorities, and responsible owners.
• Prompt Injection Test Plan: Develop authorised test cases for direct and indirect prompt injection and document the observed behaviour.
• Data Protection Checklist: Assess sensitive information handling, access permissions, data minimisation, and information disclosure risks.
• Secure Architecture Diagram: Illustrate application components, authentication boundaries, API connections, retrieval services, and security controls.
• API Security Assessment: Document authentication, authorisation, rate limiting, secrets management, and secure configuration requirements.
• AI Output Validation Plan: Define checks for unsupported financial claims, confidential information exposure, and unsafe responses.
• Incident Response Procedure: Outline how AI security events should be detected, reported, investigated, contained, and reviewed.
• Capstone Security Report: Present the threat model, testing results, prioritised recommendations, and proposed ongoing monitoring strategy.

Evidence of Successful Learning

Learner competence can be evaluated through a portfolio containing the threat model, risk register, security architecture diagram, test cases, remediation recommendations, and incident response plan. A practical demonstration can assess the learner's ability to identify AI-specific threats, explain the purpose of security controls, and recommend suitable safeguards for a banking application.

Evaluation criteria may include threat identification accuracy, quality of risk prioritisation, relevance of proposed controls, clarity of technical documentation, and understanding of secure AI development practices. These deliverables demonstrate applied learning rather than proof of production deployment or independently verified security improvements.

Case study note: This banking scenario is illustrative and intended for educational purposes. It does not represent a verified banking deployment or claim measured reductions in security incidents, fraud, or data breaches.