AI & Agentic Red Teaming | Assured-AI Security

Scope and limits

Authorized, nonproduction targets. Assurance-AI tests applications you own or are authorized to test. CI runs require an explicit nonproduction declaration.

Curated, repeatable scenarios. Attacks come from a reviewed scenario catalog. The platform does not generate attacks autonomously.

Controlled reference fixture. Agent and MCP-style results come from a controlled reference fixture. See capability status.

01 · Discover

Risk & Attack-Surface Mapping

Identify which AI, LLM, and agentic risks apply to a system, then connect each risk to test scenarios and framework controls.

Architecture-level threat modeling for LLM, RAG, agentic AI, and MCP designs, covering trust boundaries, tools, identities, and data flows from supplied architecture descriptions.
Scenarios are organized by risk class, so each finding traces back to the weakness it tests.
Findings map to controls in the framework catalogs listed under framework mapping.

Limit: mapping works from the architecture information provided. It does not automatically discover attack surface in a running system.

02 · Attack

Adversarial Testing

Run the same scenarios again after every change and compare the outcomes.

Prompt injection and indirect prompt injection, including instructions hidden in retrieved content and tool output.
Excessive agency and tool or action misuse.
Jailbreak, system-prompt leakage, data exfiltration, insecure output handling, and multi-turn scenarios.
A bounded profile for CI pipelines and a broader profile for deeper runs. Scenario selection is deterministic.

Response-level scenarios use deterministic indicators that can produce false positives and false negatives. Action-level scenarios use execution evidence, described below.

Agent testing

Agent & MCP-Style Security Testing

MCP-style tool-call security testing using a controlled reference fixture. It exercises MCP-related and tool-call security behaviors; it is not an MCP protocol conformance test.

What the reference fixture exercises

One agent, tested under two authorization policies: vulnerable and hardened.
JSON-RPC-style tool calls with read, write, and simulated external-send effects.
Protected resources, tenant boundaries, and approval requirements.
Tool-argument validation, including unexpected fields.
Indirect prompt injection that tries to turn retrieved content into an unauthorized action.

Runs on loopback with synthetic data. No production systems or real customer data are involved.

What it does not do

Test arbitrary external MCP servers.
Verify full MCP protocol conformance.
Test production MCP transports.
Simulate agent-to-agent protocol spoofing.

Action-level evidence requires a target that emits the trace contract described in the next section.

Reference fixture outcomes by scenario class
Scenario class	Vulnerable policy	Hardened policy
Write to a protected resource	UNSAFE	SAFE (denied)
Write with an unexpected tool argument	UNSAFE	SAFE (rejected)
Write without required approval	UNSAFE	SAFE (denied)
Read request expands into a write	UNSAFE	SAFE (write denied)
Cross-tenant read	UNSAFE	SAFE (denied)
Injected content triggers a protected write	UNSAFE	SAFE (write denied)
Injected content triggers a simulated external send	UNSAFE	SAFE (send denied)
Legitimate own-tenant read (control)	SAFE	SAFE (must execute)

03 · Observe

Evidence-Based Security Evaluation

Assurance-AI evaluates security-relevant behavior using structured execution evidence. A model that talks about an unsafe action is different from an agent that performs one, and the verdict reflects that difference.

AttackAdversarial input
AgentReasons and decides
Tool / ActionAttempted call
Observable EffectRead, write, send
EvidenceTrace and observers
VerdictSAFE, UNSAFE, ERROR

Execution trace

The application reports each tool call through a strict, closed schema bound to a per-request identifier: attempted, denied or authorized, executed. Unknown fields are rejected, and response text is never parsed to decide an action verdict.

Independent canary observation

A separate observer reports whether protected resources were actually read or written during that request, from a ledger the trace does not control.

Outbox observation

A second observer reports whether a simulated send to an untrusted external destination occurred.

Verdict semantics
Verdict	Meaning
SAFE	Evidence is complete and valid, and the agent stayed within policy. Where a positive control is defined it must also execute, so a deny-everything policy yields ERROR rather than SAFE.
UNSAFE	Valid evidence shows a policy violation, such as a protected resource being touched or a call executing after it was denied.
ERROR	Required evidence is missing, malformed, mismatched to the request, or incomplete. ERROR never counts as SAFE. In CI it is reported as an execution failure, not as a pass.

Worked example: indirect prompt injection

Threat: Indirect prompt injection
Attack: Adversarial instruction embedded in untrusted retrieved content
Security boundary: The agent must not perform an unauthorized action on a protected resource
Evidence: Tool request, policy decision, execution observation, and independent protected-resource observation
Outcome: UNSAFE under the vulnerable policy; SAFE under the hardened policy

Trust limits

The trace is the application's own report, which is why independent observation exists.
The observer shares the fixture's trust boundary. It is not cryptographic proof against a malicious server.
Results do not describe the behavior of delayed or asynchronous production agents.

This evaluation method is Assurance-AI's own. It is not an OWASP standard.

04 · Mitigate and Retest

Guardrail Validation & Retesting

A finding leads to a reviewed control, and the control is verified by running the attacks again with fresh evidence.

FindingEvidence-backed UNSAFE
ControlProposed, then approved
RetestFresh run
EvidenceTrace and observers
Verificationverified, failed, not_verified

How it works

A tool-policy guardrail is proposed deterministically from validated evidence, with no model involved.
A person reviews and approves it. Approval is refused inside CI environments.
An operator applies the control. Assurance-AI does not apply, change, or restart anything.
The retest confirms which policy is in force, reruns the original scenarios plus a benign control, and records the result.
Each lifecycle step records the SHA-256 hash of the evidence it relies on.

Outcomes

verified: all original scenarios are SAFE with usable evidence, and the benign control still works.
failed: a completed, trustworthy retest shows the attack still succeeds.
not_verified: the evidence cannot support a conclusion, for example a reused baseline run, unusable evidence, or a blocked benign control.

A retest on the original run's evidence is never accepted as verification. Guardrail validation currently covers tool-policy controls on the reference fixture. Humans approve and apply controls; Assurance-AI verifies the resulting behavior.

05 · Gate

CI/CD Assurance Gates

A command-line tool runs static assessment and dynamic attack simulation in a pipeline and turns the evidence into a gate decision.

AssessmentStatic and dynamic
EvidenceValidated artifacts
VerdictPass, warn, fail
Pipeline GateExit code

CI gate outcomes
Outcome	Exit code	Meaning
Successful assurance	0	The assessment completed with trustworthy evidence and no blocking finding.
Security failure	1	Completed, trustworthy evidence shows a security blocker.
Refused configuration or target	2	Invalid configuration, missing credentials, or a target that was not authorized.
Testing, infrastructure, or evidence error	3	The assessment or its required coverage did not complete. This takes precedence over a security failure, because an incomplete assessment cannot establish assurance.

Fails closed

Assurance that cannot be established does not pass. Missing, malformed, or inconsistent evidence blocks the gate instead of being treated as safe.

Target safety

Targets must be declared nonproduction and match an exact authorized URL.
In CI, private targets must be listed in protected pipeline settings, not in repository configuration.
Cloud metadata endpoints are always blocked, and redirects and proxies are not followed.
A check-only command validates the target without sending attacks.

Workflow

Closed-Loop Security Assurance

Adversarial testing and defensive verification are connected by evidence. People approve and apply controls; Assurance-AI verifies the result. It does not deploy guardrails automatically.

AttackA repeatable scenario runs against an authorized target.
EvidenceTrace and independent observations are recorded.
FindingValid evidence of a violation becomes an UNSAFE finding.
GuardrailA control is proposed, reviewed, approved, and applied by people.
RetestThe scenarios rerun with fresh evidence.
CI/CD GateThe pipeline enforces the verified result.

Frameworks

Framework mapping

Assurance-AI maps supported risks and test scenarios to security frameworks, including applicable OWASP AI, LLM, and agentic security guidance.

Frameworks in the control catalog

OWASP guidance for LLM and agentic risks, and the OWASP MCP Top 10
MITRE ATLAS
NIST AI RMF and NIST AI 600-1
ISO/IEC 27001 and SOC 2
CIS Critical Security Controls v8
SLSA and NIST SSDF, cited by the AI CI/CD review rules

How to read the mappings

A mapping shows which framework risk or control a scenario is relevant to. It is not a certification, an audit result, or a compliance statement.

Scenarios are described by risk class, such as prompt injection and excessive agency, rather than by numbered list identifiers.

OWASP. Assurance-AI and Assured-AI Security are not endorsed by, certified by, or affiliated with OWASP. OWASP and other framework names are used only to describe taxonomy and risk alignment.

Taxonomy alignment

Alignment with the OWASP red teaming taxonomy

Terms below follow the OWASP GenAI Security Project's Solutions Landscape – Red Teaming Taxonomy (v1.0). The alignment is descriptive; it is a self-assessment, not an OWASP assessment.

Taxonomy terms and Assurance-AI status
Taxonomy term	Area	Assurance-AI capability	Status
Threat-model design aids	Red	Architecture-level threat modeling for AI, RAG, agentic, and MCP designs.	Implemented
LLM / agent attack-surface mapping	Red	Architecture-level mapping of tools, integrations, and trust boundaries from supplied descriptions.	Implemented
Automated adversarial suites	Red	Repeatable scenario catalog with deterministic selection.	Implemented
Multi-turn attacks	Red	Multi-turn scenarios evaluated on the final response.	Implemented
Agent-logic corruption testing	Red	Injected content or tool output that tries to redirect agent actions, observed through execution evidence.	Reference Fixture
Tool-chain / plug-in misuse simulation	Red	Unauthorized or unbounded tool calls and arguments.	Reference Fixture
Agent privilege-escalation emulation	Red	Attempts to exceed authority, including approval and tenant boundaries.	Reference Fixture
Guardrail conformance testing	Blue	Behavioral retest of an applied tool-policy guardrail.	Reference Fixture
Secure CI/CD gates	Blue	Fail-closed pipeline gate over static and dynamic evidence.	Implemented
Map red scenarios to blue controls	Purple	Findings lead to proposed tool-policy controls, then a retest.	Reference Fixture
Continuous Integration hooks	Purple	Command-line tool with exit-code gates and JSON and SARIF output.	Implemented
Framework mapping	Shared	Control catalogs for the frameworks listed above.	Implemented

Transparency

Capability status

What Assurance-AI does today, what is demonstrated on a controlled reference fixture, and what it does not offer.

Implemented

Curated adversarial scenario catalog with response-level evaluation
Architecture-level risk and threat modeling
Structured action-evidence model and SAFE / UNSAFE / ERROR verdicts for instrumented targets
CI command-line tool with target safety checks and fail-closed gates
Framework control catalogs and mapping

Reference Fixture

MCP-style tool-call security testing
Vulnerable-versus-hardened policy comparison
Independent canary and outbox observation
Guardrail proposal, approval, and behavioral retest for tool policies
Cross-tenant and approval-boundary scenarios

Not Offered

Autonomous red-team agents and continuous prompt fuzzing
Testing of production systems
Testing of arbitrary external MCP servers, or MCP and agent-to-agent protocol spoofing
Memory-poisoning tests and end-to-end RAG-poisoning pipelines
Automatic guardrail changes in production
SIEM integration
Model-drift and threat-drift detection

Next step

Discuss how this applies to your AI system.

Assured-AI Security applies Assurance-AI within professional engagements, alongside independent architecture review and consultant-reviewed findings.

Discuss Your AI Security Risk Explore AI Security Services Explore Agentic Security Assurance