Real-world AI penetration testing: attack paths, findings and engineering outcomes
AI SECURITY CASE STUDY
Anonymised · October 2026
Engagements
3 anonymised case studies
Industry
B2B services · Data insights · Identity verification
Services
AI Penetration Testing
Findings
5 findings, Critical to Low
Validation status
All five verified closed
AI systems become security-critical when they can reach customer data, execute internal tools or make decisions that affect a business. Our testing examines what happens when attackers exploit those connections.
AI AGENT MANIPULATION
Through ordinary email, without credentials, our tester exposed internal architecture and triggered connected tools using attacker-controlled instructions.
Demonstrated: unauthorised tool execution and information disclosure.
AI-TO-DATABASE EXPLOITATION
Our tester enumerated 48 application tables, exposing the structure of sensitive account and user data.
Demonstrated: schema disclosure, not extraction of sensitive data values.
AI DOCUMENT MANIPULATION
Our tester embedded near-invisible content into a PDF. The AI classified it as containing multiple document types without flagging the hidden content.
Demonstrated: incorrect document classification, not a confirmed identity-verification bypass.
| # | Finding | Severity | OWASP LLM Top 10 | Status |
|---|---|---|---|---|
| 1 | Prompt injection exposed internal architecture and triggered AI tool actions | Critical | LLM01, LLM02, LLM06, LLM07 | Verified closed |
| 2 | Internal architecture disclosed through user-directed error escalation | Medium | LLM02, LLM07 | Verified closed |
| 3 | Placeholder prompt injection disclosed tool and model information | Medium | LLM01, LLM02, LLM07 | Verified closed |
| 4 | Prompt injection into an SQL tool enumerated the schema | High | LLM01, LLM02, LLM06 | Verified closed |
| 5 | Hidden PDF text poisoned AI document classification | Low | LLM01 | Verified closed |
Validation status: all five findings were fixed by the client and retested by CTB, and are verified closed.
Anonymised attack sequence from CTB's AI agent engagement.
WHAT MAKES THE APPROACH DIFFERENT
Testing is manual and human-reviewed. We use the OWASP Top 10 for LLM Applications (2025) as the baseline and apply MITRE ATLAS where a client needs deeper adversarial-threat mapping. A checklist ensures coverage, but attacks are chosen from your architecture, privilege boundaries and business impact.
BOUNDARY 1
Field and file injection, authentication, rate limits, cost abuse
BOUNDARY 2
Tenant isolation and propagation of the caller's permissions
BOUNDARY 3
Instruction override and what reaches an external provider
BOUNDARY 4
Actions or queries beyond the user's authority
BOUNDARY 5
Poisoned content and exposure of private content
BOUNDARY 6
Errors, verbose output, streamed events, output handling
A B2B services provider operated an AI agent that prepared customer quotes. It was reachable by email, built on a commercial frontier LLM and connected to more than 16 internal tools through the Model Context Protocol. Our tester had no credentials.
DEMONSTRATED
An unauthenticated sender, using email alone, obtained internal architecture details and made the agent run privileged tools on command. The evidence establishes information disclosure and tool execution. It does not establish compromise of the underlying infrastructure.
FIXED
The client remediated the reported behaviours by adding input validation and guardrails around the agent's tool use and disclosures.
VALIDATED
CTB retested the fixes. Verified closed.
A request for service availability at a non-existent address, combined with repeated requests for the full raw error data, led the agent to disclose its tool names, parameter keys, raw JSON error payloads, execution order and runtime indicators. The errors also showed which lookups reached external services and which reached the internal product database. The client fixed this by returning sanitised errors and keeping raw errors in server-side logs.
Fixed by client. Retested by CTB: verified closed.
Double curly brace placeholders in an address field made the model treat user data as instructions, and it disclosed the underlying model and version and an internal tool name. The client fixed this with input validation on every field and by separating instructions from data.
Fixed by client. Retested by CTB: verified closed.
A data insights platform exposed an AI assistant through an authenticated chat endpoint that accepted a free-text message. The model could call an SQL execution tool against the platform's PostgreSQL database, and the tool call was visible in the streamed response.
SEVERITY
High. An authenticated user could make the model run attacker-chosen queries against system catalogues, exposing the structure of sensitive tables. No data values were extracted.
DEMONSTRATED
Schema disclosure through arbitrary model-run SQL: four schemas, 48 application tables, and the column names and data types of the account and user tables, including credentials, multi-factor secrets, billing identifiers, contact details and permissions.
NOT DEMONSTRATED
Extraction of data values. The assistant declined to return values from sensitive columns. That refusal was the only control, and a model refusal can be bypassed through prompt manipulation.
FIXED AND VALIDATED
Verified closed. The client restricted the tool's database access and added input validation and guardrails on the SQL tool. Retested by CTB.
| # | Caller action | Model and tool behaviour | Result |
|---|---|---|---|
| 1 | Authenticated user sends a message asking about the database structure | Model calls the SQL execution tool; the call is visible in the response stream | Catalogue query runs |
| 2 | User asks for the tables in the application schema | Model issues further catalogue queries with no schema restriction | 48 tables listed |
| 3 | User asks for the columns of the account and user tables | Model returns column names and data types | Sensitive fields identified |
| 4 | User asks for row values from the sensitive columns | Model declines | Blocked by refusal only. No data extracted |
RECOMMENDED FIX
An identity verification platform used AI to classify uploaded documents. Our tester uploaded a legitimate identity document that also carried hidden text, and the classifier trusted the hidden text.
SEVERITY
Low. The demonstrated effect was misclassification of an uploaded document. No data was disclosed and no action beyond classification was triggered.
DEMONSTRATED
The platform detected explicit injection phrases such as instructions to ignore previous prompts. It did not detect fabricated content structured to look like real documents. Detection checked for known attack wording, not the authenticity of the content.
WHAT A HUMAN REVIEWER SEES
One legitimate identity document. Normal document text.
WHAT THE CLASSIFIER READ
Near-white text (#FCFCFC) at about 2 px, invisible on the page. Classified as multiple document types, with no anomaly flagged.
SEVERITY RATIONALE
RECOMMENDED FIX
FIXED AND VALIDATED
The client added input validation and guardrails on document content before classification. Retested by CTB.
Verified closed
Each finding in a CTB report carries the same fields, so a developer can reproduce the issue, understand the impact and apply a fix without a series of meetings. This is the lead finding from case study 1, anonymised.
TITLE
Unauthenticated prompt injection exposed internal architecture and triggered privileged AI tool actions
SEVERITY
CVSS:4.0/AV:N/AC:L/AT:N/PR:N/UI:N/VC:H/VI:H/VA:N/SC:N/SI:N/SA:N
VULNERABILITY DETAILS
The AI quote agent (a commercial frontier LLM with more than 16 internal tools), reachable through an inbound email address, trusted claims made inside the emails it received. An unauthenticated tester combined emotional manipulation, a fabricated authority claim (a forwarded email from a non-existent senior security manager) and progressive trust exploitation, using each disclosure as leverage for the next request.
The agent disclosed system prompt excerpts, its tool inventory with parameter schemas, and its deployment architecture. It also ran its sales escalation and diagram tools on command, repeatedly, with attacker-specified content.
VALIDATION STEPS
IMPACT
An unauthenticated outsider could map the internal environment in detail and trigger real business actions, such as repeated internal escalations and attacker-controlled content in generated output.
RECOMMENDATIONS
ADDITIONAL INFORMATION
We do not assume a customer's architecture. We review it with your team during scoping, then build the test plan around it. Depending on the access you give us, testing can be black-box or architecture-informed grey-box.
BLACK-BOX TESTING
Starts from the API alone and shows what an outsider or an ordinary customer can reach.
ARCHITECTURE-INFORMED GREY-BOX
Adds design information, such as the model setup, tools, data sources and permission model. Testers can then go straight to the boundaries that matter and cover attack paths that are not visible from outside.
| Area | Black-box testing | Architecture-informed grey-box testing |
|---|---|---|
| RAG and retrieval | Inject content through upload and input paths and observe the answers | Test cross-tenant retrieval and the influence of poisoned sources directly, using knowledge of the index and its access filters |
| Agentic workflows | Probe from outside which actions the agent will perform | Test each tool for actions beyond the user's authority and for chained tool misuse, using the tool inventory and approval flow |
| External model providers | Observe responses and error behaviour | Test what customer content leaves your environment and how provider errors surface, using the data-flow design |
| Database access | Attempt queries through the interface | Test whether the model can build queries beyond its scope, using the database role and approved views |
| Permissions | Test with one or two accounts | Test privilege boundaries between roles, and whether the AI enforces or bypasses them, using the role model |
| Multi-tenant systems | Test with accounts from two tenants where available | Test isolation across prompts, retrieval, tools and conversation history, using the tenancy design |
| Assurance area | The question we answer | Evidence from our work |
|---|---|---|
| Customer data isolation | Can one tenant use AI responses, retrieval or tools to reach another tenant's information? | Proposed approach |
| Agent permissions | Can a malicious input make the agent act beyond the user's authority? | Demonstrated Case study 1 |
| RAG and retrieval | Can poisoned documents or manipulated retrieval influence answers or expose private information? | Partly demonstrated Hidden-content poisoning in case study 3. Retrieval-specific testing is part of our proposed approach |
| AI-to-database security | Can the model build unauthorised queries or reach data beyond its approved scope? | Demonstrated at schema level Case study 2 |
| API abuse | Can a customer bypass controls, trigger excessive model consumption or expose internal information? | Partly demonstrated Internal information exposure in case studies 1 and 2. Consumption testing is part of our proposed approach |
| Safe deployment | Can testing run without disrupting live customers and dependent services? | Delivery approach Described below |
01 · ORGANISATION
CTB, as a company, is a CREST certified penetration testing provider and a founding signatory of the CREST AI Charter on responsible AI in security testing.
02 · HUMAN VALIDATION
Every AI finding is manually tested, reviewed by a person and re-run for reproducibility, with no automated scoring or LLM as judge.
03 · RECOGNISED METHODOLOGY
Our AI testing is aligned to the OWASP Top 10 for LLM Applications (2025), the OWASP AI Exchange and NIST AI RMF 1.0, with MITRE ATLAS applied where needed.
04 · INDIVIDUAL
These are held by our consultants personally, separate from CTB's CREST status. Our team also includes researchers recognised in the Facebook and Google Halls of Fame.
OSCP
Offensive Security Certified Professional
CRTP
Certified Red Team Professional
CRTO
Certified Red Team Operator
MCRTA
Multi Cloud Red Team Analyst
CREST CPSA
Practitioner Security Analyst
eWPTXv2
Web Application Penetration Tester eXtreme
Testing boundaries, evidence handling and access requirements are agreed before work begins.
TECHNICAL ACCOUNTABILITY
Each engagement is overseen by our Associate Director of Penetration Testing Services, with findings reviewed by experienced security consultants before reporting.
RULES OF ENGAGEMENT
Scope, environments, test accounts, testing windows and emergency contacts are agreed and recorded before testing starts.
CONTAINMENT
We test in staging with dummy data where possible. Volume and rate are agreed so that live customers and dependent services are not disrupted.
LIVE DATA PROTECTION
In production we access only the data assigned to our own test accounts. Any customer data we encounter is deleted immediately, and only a redacted copy goes into the report.
CRITICAL FINDINGS
Issues are reported as soon as they are identified, so your fixes can run in parallel with testing.
REPORTING
A report with severity-rated findings in the format shown in case study 1: title, vulnerability details, validation steps, impact, recommendations and additional information, with severity scored using CVSS 4.0.
RETESTING
Retests are typically completed within 24 to 48 hours of being initiated. This is our standard service target. The closure date depends on when your team fixes the findings.
WORKING ALONGSIDE YOUR ENGINEERING TEAM
We agree the scope and access requirements upfront, flag critical issues as they are discovered, and provide clear, reproducible findings your developers can act on. Once fixes are ready, we retest them so your team has confidence that the issues have been addressed.
CREST-certified penetration testing for evolving technology platforms, with context-aware assessments, tailored reporting and direct collaboration with skilled pentesters.
Read the story →Security & complianceTesting started within 24 hours of sign-off, with the first finding in the same window and evidence delivered on schedule.
Read the story →HR technologyExpert-led PTaaS surfaced a critical vulnerability within 24 hours, with a live dashboard and Slack and Jira workflows.
Read the story →Operational intelligenceFindings delivered during the test and prompt retesting of fixes, across two applications.
Read the story →- Next Step
Share an overview of your AI architecture: the models, APIs, tools and data sources the AI can reach, and your permission and tenancy model. We will use it to scope an architecture-informed engagement.
Talk to a Security Expert