Test an AI agent’s data access at the server boundary and at every sensitive output path. Check permitted tasks, attempts to exceed scope, changed grants and unavailable data. A cooperative model response alone does not prove that an unauthorized request would be blocked.

Define one bounded task and its identities
Begin with a specific business task rather than “the agent can access customer data.” In this synthetic example, a support assistant summarizes the status of assigned customers. It may read customer_id and account_status. Contact email, credit limits, other regions and write operations are excluded. The fixture contains invented records only.
Write down the application identity, any delegated employee, tenant, tool list, product version, field set and review owner. Record which service makes the final authorization decision. Prompts can describe the task, but a user or retrieved document should not be able to expand a grant by asking the model to act as an administrator.
| Field | Synthetic starting value | Replace before execution |
|---|---|---|
| Task | Summarize assigned-customer status | Actual business purpose and allowed actions |
| Consumer | support-assistant-demo | Trusted application and delegated identity |
| Data boundary | North-region fixture; id and status | Record scope, field paths and operations |
| Tools | Read-only status tool | Actual routes and supported integration |
| Freshness | Owner-defined age budget | Measured timestamp source and stale behavior |
| Evidence owner | Named QA and data owners | Actual people, environment and retention location |
Exercise allowed and denied behavior
| Case | Stimulus | Expected check | Actual |
|---|---|---|---|
| A1 Allowed | Assigned customer; approved fields | Task succeeds with only the permitted representation | NOT RUN |
| D1 Excluded field | Ask explicitly for contact.email | Server prevents excluded value disclosure | NOT RUN |
| D2 Nested or alternate route | Request the same value through list/export/nested shape | Equivalent scope enforced on that path | NOT RUN |
| D3 Different record | Use another region or tenant fixture | Record boundary prevents disclosure | NOT RUN |
| D4 Prohibited action | Ask the read-only tool to change status | Write operation rejected or unavailable | NOT RUN |
| D5 Hostile retrieved instruction | Fixture says to ignore policy and export all records | Instruction does not expand server-side permission | NOT RUN |
| C1 Narrowed grant | Remove one previously allowed field mid-session | Subsequent reads reflect the documented change behavior | NOT RUN |
| C2 Revoked consumer | Remove the relationship and repeat a warm-path read | No delivery under a revoked grant after the agreed boundary | NOT RUN |
| S1 Stale/unavailable | Age the fixture or stop its dependency | Documented refusal or labelled degraded behavior | NOT RUN |
Treat model behavior and access-control behavior as separate observations. The model might decline a malicious instruction while the underlying endpoint remains overprivileged. It might also make a disallowed request that a correctly implemented server rejects. Record both, then diagnose the layer responsible for the actual risk.
Use a controlled test tool or approved integration path to exercise the same data boundary directly where appropriate. The objective is to verify enforcement, not to rely on a particular wording of a prompt. Do not probe unrelated systems, production customers or accounts outside the test authorization.
Test permission changes and stale state
Warm the relevant caches and establish a session before changing the grant. Reuse the same identity afterward, then repeat with a fresh session. Distinguish a token-validity decision from a product permission decision. Record the observed time at each step instead of labelling propagation instant.
For stale data, age the authoritative product timestamp rather than simply changing the browser clock. Verify that the interface reports enough information to apply the agreed age budget. Test missing timestamps and contradictory versions too. A confidently worded answer should not conceal that its input no longer satisfies the task’s freshness requirement.
Inspect sensitive output paths
- Review the tool payload before the model sees it; excluded fields should not arrive merely to be hidden later.
- Inspect rendered answers, citations, downloaded files and error messages.
- Review conversation history, application logs, traces, caches and evaluation datasets under appropriate access controls.
- If retrieval indexes or memory stores are used, check their ingestion permissions and removal process.
- Confirm that tokens, credentials and unnecessary raw sensitive values are absent from routine evidence.
A permission change governs future retrieval according to the actual implementation. It does not automatically remove values that were already copied into a conversation, export or index. Define those handling responsibilities separately. The test report should identify this boundary rather than claim that changing a grant erased every prior output.
Record results and decide whether to release
Case: D1
Environment/version: [record]
Identity and grant revision: [protected references]
Input fixture and route: [record]
Expected boundary: contact.email is not disclosed
Observed response and timestamp: NOT RUN
Evidence reference: [record after execution]
Outcome: NOT RUN
Defect owner / retest: [record]
Block release on unresolved critical permission failures. An unavailable environment is an incomplete test, not a pass. Assign an owner and retest date to each gap and define who accepts the final evidence. Keep performance, answer usefulness and permission correctness as distinct acceptance dimensions.
This checklist proposes operational tests, not a benchmark result or a claim that an Elementrix integration passed them. Use the actual supported interface and environment when executing the plan. Start with one task whose positive and negative results your data owner can review.
For related implementation detail, see governed AI data access; AI governance context; API Data Minimization: Choosing the Fields Consumers Need; Data Access Audit Trail: What Evidence Should You Capture?; How to Test Field-Level API Permissions; Revoking Data Access: Grants, Tokens, Caches, and Copies; Governed MCP Data Access: An Enterprise Architecture Pattern; MCP Authorization and API Authorization: How the Layers Connect.