cgalo5758
408fa6f5a6
- Add shared ui_*.html parts (pageHeader, sectionHeader, statusBadge, emptyState) parsed into every template set - Add anatomy lint rules with a shrinking allowlist and screen-coverage check - Add make screens capture harness with contact sheets and baseline diff - Compose member and FedWiki regions server-side so pages arrive complete - Rebuild Domains and Integrations on the parts as pilots
179 lines
8.9 KiB
Markdown
179 lines
8.9 KiB
Markdown
---
|
||
title: "First-Contact UX Walk Process"
|
||
audience: [developer]
|
||
summary: "A repeatable method for evaluating the console the way a stranger meets it: missions, walkers, scoring, and synthesis."
|
||
---
|
||
|
||
# First-contact UX walk process (v1)
|
||
|
||
A repeatable method for evaluating the console the way a stranger meets it.
|
||
Written 2026-08-22 for the M10 10g re-walk; versioned because the first run
|
||
will teach us what to change. If a step earns its keep, keep it; if it
|
||
produces noise, cut it in v2.
|
||
|
||
## The problem this solves
|
||
|
||
Maintainers cannot judge first-contact UX. Whoever built a screen knows why
|
||
it exists, what every label means, and what happens after every click, so
|
||
nothing on it can confuse them. The two instruments this process uses to get
|
||
around that are **manufactured ignorance** (evaluators who genuinely lack the
|
||
project context, and are forbidden from acquiring it) and **a written rubric**
|
||
(so every screen is scored against the same questions and findings are
|
||
comparable instead of anecdotal).
|
||
|
||
## Roles
|
||
|
||
| Role | Who | Context they get | Context they must NOT get |
|
||
|------|-----|------------------|---------------------------|
|
||
| Orchestrator | Full-context agent (maintainer's session) | Everything | — |
|
||
| Naive walker | Fresh agent, no project memory | README, a login URL, credentials, a persona mission | The codebase, internal docs, design docs, this process doc |
|
||
| Heuristic reviewer | Fresh agent | The rubric + one screen at a time | The codebase |
|
||
| Honesty auditor | Fresh agent | One screenshot + the matching model card from `docs/models/` | — |
|
||
| Outside model | Non-Claude model (e.g. via opencode) | Same as naive walker or reviewer, per lane | Same exclusions |
|
||
|
||
The walker exclusion rule is the load-bearing part: confusion is the data,
|
||
and a walker who peeks at the code stops producing it.
|
||
|
||
## Phases
|
||
|
||
### 1. Build the yardstick
|
||
|
||
Research current practice (admin-console layout and prioritization,
|
||
onboarding and empty states, progressive disclosure, form and microcopy
|
||
design, heuristic-evaluation method itself) and distill it into a rubric:
|
||
a short file of checkable questions applied per screen. The rubric must
|
||
include:
|
||
|
||
- an **information-architecture lens** ("is this control on the right page
|
||
at all?"), because misplaced controls masquerade as copy problems;
|
||
- an explicit **anti-overwhelm score** (choices visible per screen, what is
|
||
above the fold, what is progressively disclosed), so "add more explanation
|
||
everywhere" cannot become the default remedy;
|
||
- an **honesty lens** ("does the page imply anything the system does not
|
||
actually do, or hide something it does?").
|
||
|
||
Output: the rubric file under `status/`.
|
||
|
||
### 2. Naive walkthroughs
|
||
|
||
Fresh agents walk the real running app in a browser, one persona mission
|
||
each. Minimum personas: (a) an admin standing the console up for the first
|
||
time, (b) a new member trying to accomplish the product's core promise,
|
||
(c) an evaluator reading the README deciding whether to adopt. Each persona
|
||
walks **both** an empty instance (true first boot; empty states are pure
|
||
first-contact surface) and a populated one.
|
||
|
||
Walk protocol:
|
||
|
||
- The walker narrates the expectation **before** each click ("I expect this
|
||
button to…"), so mismatches are caught, not just dead ends.
|
||
- Every question the walker asks aloud becomes a test: can the UI answer it
|
||
within one click? If not, that is a finding.
|
||
- The walker keeps a friction log: where they hesitated, what they guessed
|
||
wrong, what they wanted to know and could not learn.
|
||
|
||
Output: one friction log per walk, under `status/`.
|
||
|
||
### 3. Multi-model second opinions
|
||
|
||
Different models carry different priors; a screen that reads clearly to one
|
||
family may confuse another. Outside models (via opencode or similar) run the
|
||
text lane cold: read the README and setup docs as a stranger, critique page
|
||
copy extracted as text, and, when the pipe supports images, score screenshots
|
||
against the rubric. Claude agents keep the live-browser lane.
|
||
|
||
### 4. Honesty audit
|
||
|
||
For each major screen: one reviewer, one screenshot, the matching model card,
|
||
one question — does this page tell the truth about what the system does?
|
||
This phase exists because the model cards are the written ground truth for
|
||
what each subsystem actually does, which makes the comparison mechanical
|
||
instead of impressionistic.
|
||
|
||
### 5. Synthesize, triage, fix, re-walk
|
||
|
||
The orchestrator dedups and clusters all findings into a report with five
|
||
buckets: copy fix, layout fix, missing explanation, structural IA question,
|
||
works-as-designed. The maintainer triages the report; approved fixes become
|
||
an OpenSpec change. The acceptance test is a literal **re-walk**: a fresh
|
||
naive agent runs the same mission, and the previously logged friction points
|
||
must not recur.
|
||
|
||
## Method rules (added after the v1 Phase 1 research pass)
|
||
|
||
The 2026-08 research pass (four reports; distilled into
|
||
`ux-rubric-2026-08.md`) hardened the method in six ways:
|
||
|
||
- **Force the novice.** AI walkers left to their defaults complete tasks
|
||
*better* than real first-time humans and under-report learnability
|
||
failures (CHI 2026, "Synthetic Cognitive Walkthrough"). The walker prompt
|
||
must explicitly instruct confused-first-timer role-play; this is the
|
||
single most load-bearing prompt line in the process.
|
||
- **Walkers never see the rubric.** The rubric is for heuristic reviewers
|
||
and the orchestrator; a rubric-primed walker produces checklist answers,
|
||
not genuine confusion.
|
||
- **Evaluators are independent until synthesis.** Pooling findings early
|
||
anchors everyone on the first list and collapses the coverage benefit of
|
||
multiple reviewers.
|
||
- **Two passes per screen** in heuristic review: free-form impressions
|
||
first, then question-by-question; the second pass catches what the first
|
||
glosses over.
|
||
- **Severity discipline.** Nielsen 0–4; record the observation first,
|
||
assign severity in a separate pass; dedup by root cause, not reporter;
|
||
triage must use all three buckets (fix-before-retest / track /
|
||
won't-fix) — an empty won't-fix bucket means triage didn't happen.
|
||
- **Stop condition is behavioral, not a headcount.** Walk the planned
|
||
persona × state matrix as the floor, then extend until two consecutive
|
||
missions surface nothing new. The report must state the known blind
|
||
spots of AI walkers (listed in the rubric) and the model versions used.
|
||
|
||
## Standing guardrails
|
||
|
||
- **Coverage is enumerated, not sampled.** Pull the full route list from the
|
||
router before the heuristic pass; the report states anything skipped.
|
||
- **Serialize browser sessions.** One debug browser instance means one
|
||
walker at a time; walks are short, so this costs little.
|
||
- **Reset state between walks.** A walker inherits the previous walker's
|
||
leftovers otherwise, and the "first boot" premise breaks.
|
||
- **Findings cite the rubric question they fail**, so triage argues about
|
||
severity, not about whether something is a problem.
|
||
|
||
## Definition of done for a UI change
|
||
|
||
A change that touches templates or `app.css` is not complete until:
|
||
|
||
1. `make lint` is green (the page-anatomy rules, `design-system.md` §6, with
|
||
the allowlist no larger than before the change).
|
||
2. `make screens` has run against the stack and the report names the
|
||
screens it marked changed or new (`test/screens/out/changes.txt`).
|
||
3. The maintainer has reviewed the contact sheets (`test/screens/out/
|
||
operator.html`, `member.html`) against the neighbouring pages with the
|
||
Sameness checklist below, at both widths, and accepted the baseline
|
||
(`make screens-accept`).
|
||
|
||
Only then is the change archived.
|
||
|
||
### H. Sameness (every screen, against its neighbours)
|
||
|
||
- H1. Same title size and header layout: `h1.h2`, lead below, count or one
|
||
action in the right-hand slot, back link above where there is a parent.
|
||
- H2. Same section headers (`h2.h5`, count or one action) and the same
|
||
disclosure pattern for secondary sections.
|
||
- H3. Same list scaffold on every list of records: search, facet pills,
|
||
paging, the searched-to-nothing state.
|
||
- H4. Same badge meaning: one label and one tone per state, title case,
|
||
badges for states only.
|
||
- H5. Same empty state: the shared part, no boxes, no bare "No X found."
|
||
- H6. Same table density and the same use of `<code>` for identifiers.
|
||
- H7. Nothing on the phone sheet is cut off, overflows, or hides navigation.
|
||
|
||
Findings cite the H question they fail, like the rubric's A–G. The 2026-08
|
||
rubric (`status/explorations/ux-walks-2026-08/ux-rubric-2026-08.md`) is v1
|
||
and is not edited; this section extends it.
|
||
|
||
## Run history
|
||
|
||
| Run | Date | Scope | Outcome |
|
||
|-----|------|-------|---------|
|
||
| v1, first run | 2026-08 (M10 10g) | Whole console, operator + member | Evidence phases complete 2026-08-22: 8 missions, 48-screen dual-state sweep, 8 honesty audits, 3 heuristic reviews, 2 outside models, code verification. 19 deduplicated findings in `ux-walks-2026-08/findings-2026-08.md`; 6 environment artifacts caught by the verification pass (see that file's process record — v2 must flush Valkey + browser sessions on reset and pre-annotate seed infidelities). Triage pending. |
|