pattern Status: ready

Evidence Based Release

Status: ready

Intent

The release decision is made on tested user tasks, not on a feeling, a scanner score or a marketing claim.

Problem

The team has open accessibility bugs, incomplete checks and App Store labels to fill in. Nobody can say what's a blocker, what's an accepted risk and what was actually tested.

Solution

Verdict: a matrix of critical tasks × profiles, each cell with a result and evidence. Blocker rules are written before the release meeting. Every accepted risk has an owner and a fix version.

Why: the matrix turns the release meeting into reading a table instead of arguing. Rules written in advance can't be bent under deadline pressure.

For legal scope (ADA, EAA, WAD, procurement, vendors), pair this with Regulatory Evidence Map.

Implementation

Release matrix:

Task Profile Result Evidence Decision
Checkout, declined card VoiceOver fail transcript blocker
Checkout Larger Text AX5 pass screenshots ship
Pickup point choice Switch Control pass recording ship
Brewing guide video Sound off pass captions screenshot ship

Blocker rule, written before the meeting:

Block the release when:
1. a payment, account recovery or support task can't be completed with a required profile
2. a claimed App Store label is contradicted by common task evidence
3. a known issue causes data loss or an unsafe action
4. a critical error has no recovery path

Severity scale: Chapter 16, Decision 3. Decision per severity: Chapter 18, Decision 2.

A failing record:

Task:          checkout after a declined card
Profile:       VoiceOver
Result:        fail
Evidence:      transcript + screenshot
Impact:        user can't recover the payment
Severity:      blocker
Label impact:  VoiceOver claim affected
Decision:      don't release
Owner:         checkout team

An accepted risk is a written decision, not silence:

Issue:      no captions on an optional promo video
Impact:     doesn't block onboarding or purchase
Decision:   accepted for this release
Owner:      media team
Fix in:     4.13

After release, feedback comes back into the same loop:

Feedback:  "VoiceOver can't change the payment method"
1. reproduce the task
2. create an evidence record
3. set severity
4. update the release checklist
5. re-check the VoiceOver label

Check

Profile:   release owner
Task:      decide whether to ship
Expected:  every critical task has evidence from this build
           blockers are named
           accepted risks have an owner and a fix version
           label impact is reviewed

Pitfalls

  1. A scanner pass called a user pass.
  2. Labels filled in from intent.
  3. A known blocker in payment, account recovery or support shipped without a written decision.
  4. An accepted risk without an owner.
  5. Failed evidence hidden because the release is late.
  6. An AI summary instead of evidence.

WCAG mapping

WCAG classifies the findings. The decision stays task-based.

  1. 2.4.3 Focus Order: focus blockers in critical flows.
  2. 3.3.1 Error Identification: error recovery blockers.
  3. 1.4.4 Resize Text: large text blockers.
  4. 4.1.2 Name, Role, Value: semantic blockers.
  1. Silent Error
  2. Focus Black Hole
  3. Swipe Trap
  4. App Store Label Drift
  5. Scanner Theater
  6. Timeout Pressure
  7. Compliance Theater
  8. Vendor Blind Spot
  1. Chapter 13
  2. Chapter 16
  3. Chapter 18
  1. Legal context without legal theater

Sources

  1. App Store Connect Help: Overview of Accessibility Nutrition Labels
  2. Apple Developer: Performing accessibility audits for your app
  3. ADA.gov: Title II web and mobile app rule
  4. European Commission: Web Accessibility Directive standards
  5. Your Europe: Accessibility requirements for services and products
  6. W3C WCAG 2.2