AI Assisted Triage
Status: ready
Intent
AI finds risks, links related checks and summarizes evidence. It never claims it verified the app.
Problem
An AI review sounds confident even when the input has no task, no platform state and no VoiceOver output.
Solution
Verdict: give AI bounded input (task, profile, evidence) and require a status on every item: hypothesis, confirmed, false positive, accepted risk. Only people move an item from hypothesis to confirmed, after a device check.
Why: AI is fast at widening the search and comparing structures. It doesn't run the app. The status keeps "might be a problem" separate from "is a problem".
Implementation
A bounded prompt:
Task: recover from a declined card
Profile: VoiceOver, iOS 27
Inputs: screenshot, VoiceOver transcript, SwiftUI code for the error state
Find likely blockers, related checks and missing evidence.
Don't mark anything as confirmed.
Output a table: finding, risk, evidence needed, suggested owner.
Transcript comparison, where AI is most reliable:
Expected order:
1. Error
2. Card number field
3. Try again
Actual transcript:
Pay, button
Card number, text field
Payment failed
Compare the two. List focus recovery risks.
Synthetic data only:
1. Synthetic names instead of real ones.
2. No emails, addresses, card numbers, health details.
3. No tokens, internal URLs or production logs.
4. Keep the shape of the task.
Output with a status per item:
| Finding | Status | Evidence needed | Owner |
|---|---|---|---|
| Error may be silent | hypothesis | VoiceOver transcript of the declined card flow | checkout QA |
| Two buttons named "Edit" | confirmed | screenshot + transcript | orders team |
| Low contrast on a label | false positive | the token maps to a 4.6:1 color | design system |
Check
Profile: QA lead
Task: review the AI triage output
Expected: every item has a status
hypotheses name the check that confirms them
confirmed items link to evidence
no personal data in the prompt
Pitfalls
- "Is this accessible?" without a task and a profile.
- AI output counted as a pass.
- AI rewriting exact evidence into vague prose.
- Uncertainty dropped in the release summary.
- Skipping the device check after an AI review.
WCAG mapping
AI can suggest a mapping. Keep it secondary to the task result.
4.1.2 Name, Role, Value: labels and traits in code.2.4.3 Focus Order: transcript comparison.3.3.1 Error Identification: error paths.1.4.4 Resize Text: screenshot matrix review.