---
title: "First-Contact UX Walk Process"
audience: [developer]
summary: "A repeatable method for evaluating the console the way a stranger meets it: missions, walkers, scoring, and synthesis."
---

# First-contact UX walk process (v2)

A repeatable method for evaluating the console the way a stranger meets it.
Written 2026-08-22 for the M10 10g re-walk; versioned because the first run
will teach us what to change. If a step earns its keep, keep it; if it
produces noise, cut it in v2.

## The problem this solves

Maintainers cannot judge first-contact UX. Whoever built a screen knows why
it exists, what every label means, and what happens after every click, so
nothing on it can confuse them. The two instruments this process uses to get
around that are **manufactured ignorance** (evaluators who genuinely lack the
project context, and are forbidden from acquiring it) and **a written rubric**
(so every screen is scored against the same questions and findings are
comparable instead of anecdotal).

## Roles

| Role | Who | Context they get | Context they must NOT get |
|------|-----|------------------|---------------------------|
| Orchestrator | Full-context agent (maintainer's session) | Everything | — |
| Naive walker | Fresh agent, no project memory | README, a login URL, credentials, a persona mission | The codebase, internal docs, design docs, this process doc |
| Heuristic reviewer | Fresh agent | The rubric + one screen at a time | The codebase |
| Honesty auditor | Fresh agent | One screenshot + the matching model card from `docs/models/` | — |
| Outside model | Non-Claude model (e.g. via opencode) | Same as naive walker or reviewer, per lane | Same exclusions |

The walker exclusion rule is the load-bearing part: confusion is the data,
and a walker who peeks at the code stops producing it.

## Phases

### 1. Build the yardstick

Research current practice (admin-console layout and prioritization,
onboarding and empty states, progressive disclosure, form and microcopy
design, heuristic-evaluation method itself) and distill it into a rubric:
a short file of checkable questions applied per screen. The rubric must
include:

- an **information-architecture lens** ("is this control on the right page
  at all?"), because misplaced controls masquerade as copy problems;
- an explicit **anti-overwhelm score** (choices visible per screen, what is
  above the fold, what is progressively disclosed), so "add more explanation
  everywhere" cannot become the default remedy;
- an **honesty lens** ("does the page imply anything the system does not
  actually do, or hide something it does?").

Output: the rubric file under `docs/`.

### 2. Naive walkthroughs

Fresh agents walk the real running app in a browser, one persona mission
each. Minimum personas: (a) an admin standing the console up for the first
time, (b) a new member trying to accomplish the product's core promise,
(c) an evaluator reading the README deciding whether to adopt. Each persona
walks **both** an empty instance (true first boot; empty states are pure
first-contact surface) and a populated one.

Walk protocol:

- The walker narrates the expectation **before** each click ("I expect this
  button to…"), so mismatches are caught, not just dead ends.
- Every question the walker asks aloud becomes a test: can the UI answer it
  within one click? If not, that is a finding.
- The walker keeps a friction log: where they hesitated, what they guessed
  wrong, what they wanted to know and could not learn.

Output: one friction log per walk, under `status/`.

### 3. Multi-model second opinions

Different models carry different priors; a screen that reads clearly to one
family may confuse another. Outside models (via opencode or similar) run the
text lane cold: read the README and setup docs as a stranger, critique page
copy extracted as text, and, when the pipe supports images, score screenshots
against the rubric. Claude agents keep the live-browser lane.

A model that cannot read images runs the transcript lane: an
accessibility-tree snapshot of each screen state, read as text. Models on that
lane reached the same root causes as the reviewers who read the screenshots,
so a pipe without images costs the run little.

### 4. Honesty audit

For each major screen: one reviewer, one screenshot, the matching model card,
one question — does this page tell the truth about what the system does?
This phase exists because the model cards are the written ground truth for
what each subsystem actually does, which makes the comparison mechanical
instead of impressionistic. A drift the audit reports is one of three kinds,
and each kind goes somewhere different: the card lags a design the maintainer
already took, so the card is edited; the screen is wrong, so the drift becomes
a finding for triage; or the card and the screen name one thing two ways,
which the maintainer decides.

### 5. Code verification

A full-context agent reads the templates, the handlers, and the recorded
decisions behind every claim the evidence lanes produced, before the synthesis
sees any of it. A claim the code contradicts is dropped or downgraded, and a
claim the seed or the test stack produced is filed as an environment artifact
rather than a product defect: six of run 1's nineteen findings were
environment artifacts this pass caught.

### 6. Synthesize, triage, fix, re-walk

The orchestrator dedups and clusters all findings into a report with five
buckets: copy fix, layout fix, missing explanation, structural IA question,
works-as-designed. The maintainer triages the report; approved fixes become
an OpenSpec change. The acceptance test is a literal **re-walk**: a fresh
naive agent runs the same mission, and the previously logged friction points
must not recur.

## Method rules (added after the v1 Phase 1 research pass)

The 2026-08 research pass (four reports; distilled into
[`first-contact-rubric.md`](first-contact-rubric.md)) hardened the method in
six ways:

- **Force the novice.** AI walkers left to their defaults complete tasks
  *better* than real first-time humans and under-report learnability
  failures (CHI 2026, "Synthetic Cognitive Walkthrough"). The walker prompt
  must explicitly instruct confused-first-timer role-play; this is the
  single most load-bearing prompt line in the process.
- **Walkers never see the rubric.** The rubric is for heuristic reviewers
  and the orchestrator; a rubric-primed walker produces checklist answers,
  not genuine confusion.
- **Evaluators are independent until synthesis.** Pooling findings early
  anchors everyone on the first list and collapses the coverage benefit of
  multiple reviewers.
- **Two passes per screen** in heuristic review: free-form impressions
  first, then question-by-question; the second pass catches what the first
  glosses over.
- **Severity discipline.** Nielsen 0–4; record the observation first,
  assign severity in a separate pass; dedup by root cause, not reporter;
  triage must use all three buckets (fix-before-retest / track /
  won't-fix) — an empty won't-fix bucket means triage didn't happen.
- **Stop condition is behavioral, not a headcount.** Walk the planned
  persona × state matrix as the floor, then extend until two consecutive
  missions surface nothing new. When the extensions keep surfacing new
  findings and that rule is never met, the orchestrator stops the walks on
  judgment and the synthesis records the non-convergence as a result of the
  run. The report must state the known blind spots of AI walkers (listed in
  the rubric) and the model versions used.

Every frequency and criticality figure in operator UX work is an estimate. The
deployment collects no usage telemetry and has no pool of operators to
interview, so how often an action runs and how much damage it can do are
derived from the route surface and each handler's cascade rather than
measured.

## Standing guardrails

- **Coverage is enumerated, not sampled.** Pull the full route list from the
  router before the heuristic pass; the report states anything skipped.
- **Serialize browser sessions.** One debug browser instance means one
  walker at a time; walks are short, so this costs little.
- **Prove the browser tool's click path before the walks start.** Click a
  plain, unboosted link with the tool the walkers are told to use. A click
  tool that silently does nothing turns every "clicking X did nothing" report
  into a claim the synthesis has to filter out, and the walker cannot tell the
  two apart.
- **Reset state between walks.** A walker inherits the previous walker's
  leftovers otherwise, and the "first boot" premise breaks. A reset is
  three things, not one (v1 lesson): the database state, the session
  store (`valkey-cli FLUSHALL`; console sessions outlive a database
  reset), and the debug browser's profile (removed and relaunched, so no
  cookies or history carry over). Log each reset per mission.
- **Pre-annotate seed infidelities** (v1 lesson) in a `seed-notes.md`
  next to the run before any evaluator sees evidence: pinned clocks,
  seeded rows that lack what real rows have, integrations not composed,
  designed behaviors that look like bugs. Reviewers, auditors, and the
  synthesis read it; walkers never do.
- **Findings cite the rubric question they fail**, so triage argues about
  severity, not about whether something is a problem.

## Definition of done for a UI change

A change that touches templates or `app.css` is not complete until:

1. `make lint` is green (the page-anatomy rules, `design-system.md` §6, with
   the allowlist no larger than before the change).
2. `make screens` has run against the stack and the report names the
   screens it marked changed or new (`test/screens/out/changes.txt`).
3. The maintainer has reviewed the contact sheets (`test/screens/out/
   operator.html`, `member.html`) against the neighbouring pages with the
   Sameness checklist below, at both widths, and accepted the baseline
   (`make screens-accept`).
4. A state the capture utility cannot reach without an interaction it does
   not script (an open editor, a staged batch, a tray) has been captured by
   hand in the browser at both widths into a review sheet beside the change,
   the report names each such state and points at the sheet, the maintainer
   reviews it by the same checklist, and the scripted-state gap is logged in
   `status/issues.md` naming those states.
5. The notebook behind the change, if any, has been harvested (the `harvest`
   skill; `status/MAINTAINING.md`, "Landing a change"), so no tracked file
   cites a path under `status/explorations/`; the `notebook-citation` lint
   rule checks it.

Only then is the change archived.

### H. Sameness (every screen, against its neighbours)

- H1. Same title size and header layout: `h1.h2`, lead below, count or one
  action in the right-hand slot, location trail above where the page sits below a rail entry.
- H2. Same section headers (`h2.h5`, count or one action) and the same
  disclosure pattern for secondary sections.
- H3. Same list scaffold on every list of records: search, facet pills,
  paging, the searched-to-nothing state.
- H4. Same badge meaning: one label and one tone per state, title case,
  badges for states only.
- H5. Same empty state: the shared part, no boxes, no bare "No X found."
- H6. Same table density and the same use of `<code>` for identifiers.
- H7. Nothing on the mobile sheet is cut off, overflows, or hides navigation.

Findings cite the H question they fail, like the rubric's A–G. The 2026-08
rubric ([`first-contact-rubric.md`](first-contact-rubric.md)) is v1 and is
not edited; this section extends it.

## Run history

| Run | Date | Scope | Outcome |
|-----|------|-------|---------|
| v2, second run (10k.4 acceptance) | 2026-08-31 → 09-01 (M10 10k.4) | Whole console after the anatomy sweep and htmx 4, operator + member, plus section H | Nine missions (five floor + M7/M8/M9 extensions; the behavioral stop rule was not reached — four consecutive extensions each surfaced something new — and the walks were stopped by judgment, recorded as a result), three independent heuristic reviews (96 raw findings), eight honesty audits, three outside-model cold reads + two transcript reads + one vision read, a 22-item code verification pass, first-boot and demo capture sets, 34 transcripts. 39 deduplicated findings ACC-1 … ACC-39, none at severity 4; the four v1 sev-4s resolved, five of seven v1 sev-3s recurred (two after their designed fixes). Method: resets were three things (database, Valkey, browser profile), seed infidelities were pre-annotated; two tooling incidents recorded (an inert MCP click tool; `opencode run` blocking on stdin under detached execution). Triage pending. |
| v1, first run | 2026-08 (M10 10g) | Whole console, operator + member | Evidence phases complete 2026-08-22: 8 missions, 48-screen dual-state sweep, 8 honesty audits, 3 heuristic reviews, 2 outside models, code verification. 19 deduplicated findings; 6 environment artifacts caught by the verification pass (the lesson: v2 must flush Valkey + browser sessions on reset and pre-annotate seed infidelities). Triage pending. |