cgalo5758
5829091881
The contained runner that drove the 2026-09 security audit, the README review and four rounds of design ideation lived only inside the ignored notebook. It moves to scripts/agent-runner/: the prepare, tools, audit, ideation and teardown scripts, the compose and container files, the allowlist proxy and the prompt templates, with the paths that assumed the notebook fixed and findings written to a caller-named directory. Run outputs, transcripts and the round-specific sheet scripts stay behind. docs/agent-runner.md states the method: the disposable git archive copy and the fail-closed proxy, how a task is shaped, union rather than intersection of findings across models, separate adjudication of every finding against the source, and the evidence a finding must carry.
106 lines
5.0 KiB
Markdown
106 lines
5.0 KiB
Markdown
# Agent runner
|
|
|
|
Runs a model from another family (DeepSeek, Gemini, GLM/Z.AI, Kimi) over a
|
|
disposable copy of this repository, inside a container whose only route off-box
|
|
is an allowlist proxy. Two lanes share that container: a **security lane**
|
|
(`run-audit.sh`), which audits a named slice of the code, and an **ideation
|
|
lane** (`run-ux.sh`), which answers a design brief with a proposal and a static
|
|
mockup. Each lane runs a fresh container per model, so no run inherits another
|
|
run's context.
|
|
|
|
The method these lanes serve, and the rules a finding has to clear before it
|
|
counts, are in [`docs/agent-runner.md`](../../docs/agent-runner.md).
|
|
|
|
## What it touches, and what it does not
|
|
|
|
- **Working tree:** never mounted. The model sees `git archive HEAD` unpacked
|
|
into `.code/` beside these scripts, which `teardown.sh` throws away. Edits it
|
|
makes go nowhere.
|
|
- **Network:** the model container sits on a Docker network marked `internal`,
|
|
which has no route off-box. Its only exit is a tinyproxy sidecar that denies
|
|
every host except the patterns in `allowlist.txt`. This is fail-closed: a
|
|
misconfigured proxy costs the run its network rather than leaking the code.
|
|
In practice the proxy refuses semgrep's telemetry and opencode's registry
|
|
fetches while allowing the model API. Read the denials with
|
|
`docker compose exec egress-proxy cat /tmp/tinyproxy.log`.
|
|
- **Credentials:** one secret enters a container, the opencode auth file at
|
|
`~/.local/share/opencode/auth.json`, mounted read-only. It holds the
|
|
model-provider keys. SSH keys, forge tokens, `test/secrets/` and the host
|
|
opencode config are never mounted. Point `AUDIT_AUTH_JSON` elsewhere if your
|
|
auth file lives elsewhere.
|
|
- **Live stack:** not involved. There is no database, no identity provider, no
|
|
webhook surface. Both lanes read code.
|
|
- **Findings:** land in the directory you name with `OUT`. Keep it outside this
|
|
one: this directory is tracked and a run's output is not.
|
|
|
|
## Security lane
|
|
|
|
```
|
|
./prepare.sh # once: export HEAD, build images, start the proxy
|
|
OUT=<dir> ./run-tools.sh # offline scanner pass -> $OUT/tools/
|
|
OUT=<dir> ./run-audit.sh # every model x every scope task -> $OUT/<timestamp>/
|
|
./teardown.sh # remove the containers, networks, images and the copy
|
|
```
|
|
|
|
The scanner pass runs first and offline: gosec, govulncheck, semgrep,
|
|
staticcheck and gitleaks write machine-readable output that task `08` gives the
|
|
model to triage. `run-audit.sh` runs it itself if `$OUT/tools/` is missing.
|
|
|
|
Inputs are the three files the prompt is built from:
|
|
|
|
| File | What it is |
|
|
|---|---|
|
|
| `AUDITOR.md` | How the model works, the evidence every finding must carry, the output shape |
|
|
| `CONTEXT.md` | The auth model, request pipeline and trust boundaries, supplied as fact |
|
|
| `scope/NN-*.md` | One slice each: the files, a one-sentence threat model, falsifiable questions |
|
|
|
|
Subsets and knobs:
|
|
|
|
```
|
|
OUT=<dir> MODELS="deepseek/deepseek-v4-pro" ./run-audit.sh 02-operator-boundary
|
|
OUT=<dir> TIMEOUT=3600 VARIANT=high ./run-audit.sh
|
|
```
|
|
|
|
`VARIANT` is reasoning effort and defaults to `max`; set it empty to omit the
|
|
flag. Model ids drift, so confirm them with `opencode models` and pass
|
|
`MODELS=` when the defaults in `run-audit.sh` are stale. To add a provider,
|
|
authenticate it on the host with `opencode auth login`, add its API host to
|
|
`allowlist.txt`, and name it in `MODELS`.
|
|
|
|
## Ideation lane
|
|
|
|
```
|
|
./prepare.sh # once, same as above
|
|
./run-ux.sh <brief-dir> # one lane per model -> <brief-dir>/out/<timestamp>/
|
|
```
|
|
|
|
The brief is yours; the runner carries no design content. `<brief-dir>` holds
|
|
four files, each overridable by the variable of the same name: `DESIGNER.md`
|
|
(how the model works and what it must produce), `CONTEXT.md` (the surface, its
|
|
constraints, what survives any redesign), `SCREENS.md` (what exists today) and
|
|
`brief.md` (the deliverable). `EXTRA=<file>` appends a file to the prompt;
|
|
`INCLUDE="<file> ..."` copies files into the disposable tree and names them for
|
|
the model to read. `OUT=<dir>` moves the output elsewhere.
|
|
|
|
Each lane writes `proposal.md`, `proposal.err` and `mockup.html`; `_static/`
|
|
holds the stylesheets the mockups link, so a mockup opens in a browser.
|
|
|
|
## Teardown
|
|
|
|
`./teardown.sh` removes the containers, both networks, the two built images and
|
|
the disposable copy, then prints a residue check. `--deep` also drops the base
|
|
images and prunes the build cache; `--verify-only` removes nothing and just
|
|
reports. Findings are untouched, because they never live here.
|
|
|
|
Every model container runs with `--rm`, so no opencode session or snapshot data
|
|
survives a run, and `auth.json` was read-only, so nothing was written back to
|
|
it.
|
|
|
|
## First build
|
|
|
|
`prepare.sh` is slow the first time. Everything the run needs from the network
|
|
is fetched at build time, where the Docker daemon still has normal egress: the
|
|
Go toolchain, opencode, the five scanners, the semgrep rule corpus, the Go
|
|
vulnerability database, and this project's module cache. At run time the
|
|
scanners are fully offline.
|