--- title: "Building an Integration" audience: [developer] summary: "How to build an integration: the internal/integrations/ tree, the mandatory provider manifest, optional capability hooks (routes, workflows, config, UI), the database-authorship contract, and the capability-not-transport dispatch rule — FedWiki is the fully-conformant worked example, Stripe is partially conformant pending a follow-up." --- # Building an integration — authoring guide How to add an *integration* — member-console's term for a compiled-in package that composes against one external service — to the console. FedWiki is the fully-conformant worked example throughout this guide; Stripe is partially conformant (§9) pending the `payments-provider-seam` follow-up. Landed by OpenSpec change `integration-extraction` (M9 9d), which builds on the resource-key/manifest contract from the earlier `provider-extension-contract` change (M9 9a + 9c). ## Terminology Two terms, two meanings — used consistently across code, docs, and the database: - **Provider** — the *domain entity*: a row in the `core.providers` registry, with a kind, a manifest of lifecycle operations and states, and owned resource keys. Providers are what entitlements, resource keys, and pool provisions reference; they exist in the database at runtime (§3). - **Integration** — the *code unit*: the compiled-in tree under `internal/integrations//` that implements a provider — migrations, workflows, HTTP handlers, config declaration, UI assets — and registers it. `Integration.Provider()` is the bridge: an integration ships exactly one provider. "Extension" is a retired term: earlier M9 documents said "extension" or "provider extension" for what this guide calls an integration; it named no third concept. (The upstream design corpus draws the same line — provider as the genus, integration as the plumbing.) ## Distribution model Integrations are **first-party and in-tree**. Everything an integration imports lives under `internal/`, which the Go toolchain restricts to this module — an out-of-tree repository cannot import the contract surface, and there is no dynamic-plugin loading; composition is compile-time, at `registry.go`. To build an integration: branch this repository (external contributors: fork), add your `internal/integrations//` tree plus the one registry line, and open a PR. An out-of-tree model — separate integration modules compiled in by a builder tool, Caddy/xcaddy-style — would require publishing a semver-stable public contract API and is an explicit non-goal for now; see `status/issues.md` § Integration architecture. ## 1. What an integration is An integration is **one tree plus one registry line**: - One directory, `internal/integrations//`, owns everything the integration ships: its database module, its Temporal workflows, its HTTP handlers, its templates and static assets, and its provider-registry manifest. - One element in `internal/integrations/registry.go`'s `All()` installs it: ```go func All() []Integration { return []Integration{ fedwiki.New(), stripe.New(), } } ``` Adding an integration means adding a tree plus one slice element — nothing else in this file changes, and no `init()`-time self-registration is involved (blank-import self-registration was considered and rejected: it hides the composition root and complicates testing subsets). **Import-direction rule** (mirrors the FK rule for schemas): integration trees MAY import core packages (`internal/db`, `internal/integration`, `internal/server`, `internal/workflows`, `internal/config`, ...). Core packages MUST NOT import integration trees. Exactly two composition roots consume the registry: `cmd/start.go` (routes, workflows, config, UI wiring) and `internal/migrate/sources.go` (migration sources). `internal/server` and `internal/workflows` never import `internal/integrations` — they each declare the capability interface their own domain cares about (§3), and an integration satisfies it structurally, without either side importing the registry package. ## 2. Tree layout — FedWiki as the worked example ``` internal/integrations/fedwiki/ fedwiki.go # Adapter: Key, Provider, MigrationSource, plus the # optional capability hooks it implements (routes, # workflows, config, UI assets) store/ # sqlc module: migrations/, queries/, generated code migrations/00001_init.sql queries/*.sql *.sql.go, models.go, querier.go, provider.go, sqlc.yaml workflows/ # Temporal workflows and activities web/ # HTTP handlers: member API, HTMX partials, operator page templates/ # key-prefixed html/template files static/ # key-prefixed static assets (JS, ...) ``` `store` and `workflows` are separate Go packages from the top-level adapter package (`fedwiki`), not files folded into it: `web` imports `store` and `workflows` directly (its handler configs are typed against `store.Querier` and it calls workflow-starting functions from HTTP handlers), and the adapter package must import `web` to construct those handlers. Folding `store`'s or `workflows`' code into the adapter package would close an import cycle (adapter → web → store/workflows → adapter). Keeping them as sibling subpackages — imported by both the adapter and `web`, never the reverse — satisfies "one tree owns everything the integration ships" at the directory level without a cycle. `store`'s package name deliberately stays `fedwiki` (matching its pre-extraction name) rather than `store`: `sqlc.yaml`'s `gen.go.package` field pins it, so regenerated code stays byte-identical across the move. Only the import path changed; callers alias it `fwmod` by convention. `workflows` has no such pin (hand-written Go, not sqlc-generated). The adapter itself does not import `internal/integrations`: its methods return or accept the concrete types the capability interfaces declare (`db.MigrationSource`, `integration.ProviderSource`, `server.Deps`, `config.ConfigKey`, `io/fs.FS`), so it satisfies those interfaces structurally. `registry.go` — in package `integrations` — is the only place a concrete adapter is named as an `integrations.Integration`. Stripe's tree (`internal/integrations/stripe/`) follows the identical shape (`store/`, `workflows/`, `web/`) for the parts it has extracted; see §9 for what still lives outside it. ## 3. Mandatory surface Every registered integration implements the mandatory `Integration` interface: ```go type Integration interface { Key() string Provider() integration.ProviderSource MigrationSource() db.MigrationSource } ``` - **`Key()`** — the integration's short identifier (e.g. `"fedwiki"`, `"stripe"`); must match the `Key` in the manifest `Provider()` returns. - **`MigrationSource()`** — the integration's `db.MigrationSource` (own embedded FS, own goose ledger). See §4. - **`Provider()`** — the provider-registry manifest, detailed below. ### Concepts: provider, capability rows vs. operational rows **Provider** is the genus for every external service the console integrates, recorded in the `core.providers` registry. Each provider declares a `provider_kind`: | kind | examples | appears in operator "Integrations" | |------|----------|-----------------------------------| | `provisioning` | FedWiki, (future) NextCloud, Discourse | **yes** | | `payment` | Stripe | no | | `notification`, `tax` | (future) | no | The operator Integrations surface is exactly the `provider_kind = 'provisioning'` subset. A payment provider like Stripe is a registry row for genus completeness but never shows there. **Capability rows vs operational rows.** A provider's *capabilities* — the lifecycle operations it supports and the resource keys it owns — are registered from code at boot (the manifest below). Its *operational* state — `status`, plan-ladder bindings — is DB-canonical and operator-editable. A boot re-registration refreshes capabilities and never clobbers operational state. ### Resource keys: namespace-by-provider-key The `core.resource_keys` table is a single flat, globally-unique namespace (everything FKs into it, so a typo is a constraint violation, not a silent mismatch). Ownership is a column, not a separate namespace: - **Platform** owns the *unprefixed* namespace: `seats`, `members`, a pooled `storage_bytes`. `provider IS NULL`. - **You** own `_*`: `fedwiki_sites`, `nextcloud_storage_bytes`. `provider = ''`. Three separate roles — never conflate them: | role | example | use | |------|---------|-----| | bare key (machine id / FK target) | `fedwiki_sites` | the stored, queried key | | `provider` column (grouping/filter) | `fedwiki` | "all FedWiki resources"; `NULL` = platform | | `display_name` (UI label) | `Wiki Sites` | the only string shown to users | There is **no dotted `provider.resource` form** — it collides with `schema.table` notation and is redundant for prefixed keys. Treat resource keys as a bare id plus a `provider` attribute; never parse a dotted string. **Shape (`kind`)** — every key declares `kind ∈ {'boolean','numeric'}` at INSERT time. `numeric` keys are conferred by `limit` rules and consumed via `numeric_entitlements`; `boolean` keys are conferred by `boolean` rules and consumed via `boolean_entitlements` (Discourse's `discourse_posting` is the boolean template). The operator rule-authoring form derives the rule type from `kind`, so a misdeclared kind locks the key to the wrong rule shape — declare it explicitly rather than riding the column default (`'numeric'`). **Provider-key hygiene** (enforced at registration): a provider key matches `^[a-z0-9]+$` (no underscores) and may not be the leading `_`-token of any platform key (and vice versa) — the provider-key space and the resource-key space share one string space. **Metering is independent of `provider`.** A `provider IS NULL` key may be metered (pooled storage, per-seat billing); do not couple "is metered" to "has a provider". ### Lifecycle operations The contract's verb set: `create`, `set_status`, `delete`, plus a read class `list` / `describe`. `create` provisions a new instance and `delete` permanently (irreversibly) removes one; **`set_status(target)`** moves an existing instance between the lifecycle **states** the provider declares (e.g. `active` / `readonly` / `archived`), and is the reversible, both-directions path. `active` is the implicit baseline; a provider declaring `set_status` declares its non-`active` states in its manifest (persisted to `provider_states`). Declare **only the verbs you actually implement**. The console owns *intent* (counts, limits, requested transitions) and may keep an **observed projection** of per-instance state refreshed from the read class; the provider remains canonical. Dispatch — how a mutating verb is actually carried out — is per-integration, not mandated by this manifest; see §5 for the decision rule. > FedWiki declares `create`/`set_status`/`delete`/`list`/`describe` and the > states `active`/`readonly`/`archived`, backed by FarmManager v0.4.1's > reversible PATCH-status transitions. ### The manifest ```go func (providerSource) ProviderManifest() integration.Manifest { return integration.Manifest{ Key: "fedwiki", Kind: integration.KindProvisioning, DisplayName: "FedWiki", OperatorSurfacePath: "/operator/integrations/fedwiki", // "" if none; keep new surfaces under /operator/integrations/ Operations: []integration.Operation{integration.OpCreate, integration.OpSetStatus, integration.OpDelete, integration.OpList, integration.OpDescribe}, States: []integration.State{integration.StateActive, integration.StateReadonly, integration.StateArchived}, // required iff OpSetStatus ResourceKeys: []string{"fedwiki_sites"}, // each MUST be "_…" } } ``` `internal/integration.RegisterProviders` validates the manifest, upserts the registry rows, reconciles the operation and state sets, and stamps `resource_keys.provider` for your owned keys — all in one transaction, idempotent. Validation covers: the provider-key regex and provider-key/platform-key nesting; kind must be a known `ProviderKind`; every declared `Operation` must be known; declaring `OpSetStatus` requires `active` plus at least one non-`active` state, and declaring states without `OpSetStatus` is rejected; every `ResourceKeys` entry must be `"_"`-prefixed; duplicate provider keys across providers are rejected. **Registering it at boot** requires no hand-maintained list: `cmd/start.go` collects `integ.Provider()` from every entry in `integrations.All()` and calls `integration.RegisterProviders` once, after migrations have run (so the resource keys each manifest names already exist for stamping — a manifest cannot invent a new resource key purely by declaring it; the key row must already exist from your own migration). **Owning your resource keys** — seed/own your `_*` keys in `core.resource_keys` from your own migration stream (FK'ing into `core`), declaring `kind` explicitly in the INSERT (see "Shape (`kind`)" above). `provider` is left `NULL` by migrations; boot registration stamps it. Your Down migration `DELETE`s the seeded key rows (they are your stream's writes; "Down mirrors Up" §7 applies to cross-schema seeds too — see the Discourse `00001`'s Down). Historical exception: FedWiki's `fedwiki_sites` row predates this rule and lives in core's seed stream (`00002_seed_resource_keys.sql`); copy Discourse, not FedWiki, here. **Operator surface** — set `OperatorSurfacePath` and implement the routes capability (§4) to serve it; derive the route you register from the same manifest value your adapter returns, so the two never drift apart (see FedWiki's `RegisterRoutes`, which reads `fwmod.ProviderSource().ProviderManifest().OperatorSurfacePath` rather than hardcoding the path a second time). Keep the path under `/operator/integrations/` — that namespace is where operators expect integration surfaces, unmatched paths under it 404 in-shell, and your settings page already lives at `/operator/integrations//settings`. The operator sidebar's Integration group enumerates provisioning providers from the registry generically — **no template edit is needed** to add your nav entry. **Operator surface anatomy** — the page's *content* is yours, but follow the shared skeleton so operators aren't relearning a layout per integration (the 2026-07-22 fresh-eyes audit found exactly that jarring — FedWiki and Discourse had converged on different idioms): 1. **Identity**: an `

` naming the integration, a one-sentence intro that defines any term of art the page uses (what a "sweep"/"sync" is), a "Part of [Integrations]" link, and a Settings button to `/operator/integrations//settings`. Declare `IAPosition: "integration:integrations:"` and `ActiveCapability: "integrations"` so the sidebar highlights both the parent section and your entry. 2. **Delivery health**: if your integration runs a reconciliation schedule, lead with a "Sync health" card showing the latest run's status and time via `workflows.ScheduleRunStatus(ctx, temporal, .SyncScheduleID)`. Resolve runs through the *schedule*, never by describing your workflow ID directly — schedule-spawned executions get timestamp-suffixed IDs, so a bare-ID describe reports "no sync yet" forever while sweeps run happily. 3. **Delivered state**: the integration-specific meat — FedWiki's site inventory, Discourse's group mappings and links. 4. **Actions**: mutations last, using the shared confirm-action modal for destructive ones and the HTMX + CSRF-header pattern throughout. Both existing pages follow this order; copy either. **Guardrails** — `member-console lint` flags retired/bare resource-key literals (`ResourceKey: "..."` and raw `resource_key = '...'`); keep your keys namespaced. Add a registration test (see `internal/integration/registration_db_test.go`) asserting your rows + stamp. ## 4. Optional capability hooks An integration lacking a capability simply doesn't implement the corresponding interface — the composition root discovers each hook by type assertion against the concrete adapter returned from `integrations.All()` (e.g. `if rp, ok := integ.(server.RouteProvider); ok { ... }`), so there is no stub method to write for a capability you don't need. ### Routes — `server.RouteProvider` ```go type RouteProvider interface { RegisterRoutes(mux *http.ServeMux, deps Deps) error CSRFExemptPaths() []string } ``` `RegisterRoutes` constructs your handlers from `Deps` (the DB connection, core module queriers, the Temporal client, the auth config, the logger — no core code ever names an integration handler type) and registers them on the shared mux. A handler that composes the operator page shell (`operator.html`) must also parse the shell's own partial dependencies — today `partials/operator_lookup_result.html` — into its template set: html/template's escape analysis requires every referenced template to exist even on branches your page never executes (parse only the shell and every render 500s; see `internal/server/operator.go:66` and the Discourse operator handler for the worked pattern). `CSRFExemptPaths` declares any request paths that authenticate a different way — e.g. a webhook verified by the provider's own signature scheme. Declared paths bypass **both** CSRF protection and session auth: the composition root collects every mount's exempt paths into the CSRF middleware's ignore list and into the auth middleware's public-path set (without the latter, provider deliveries bounce off the `/login` redirect — caught live against a real Discourse; no handler-level test can see it because the middleware stack isn't in the test). Your integration declares the exemption itself instead of a literal path string being hardcoded into core. Declare an exemption only for a route you are actually registering, and only for routes that genuinely verify every request themselves (Stripe's webhook adapter returns no exempt path when its webhook secret is unconfigured and the route itself isn't mounted). ### Workflows — `workflows.WorkflowProvider` ```go type WorkflowProvider interface { RegisterWorkflows(w worker.Worker, database *sql.DB, logger *slog.Logger) Startup(ctx context.Context, c client.Client, taskQueue string, database *sql.DB, logger *slog.Logger) error } ``` `RegisterWorkflows` runs at worker construction and registers your Temporal workflows/activities against the shared worker. `Startup` runs once per boot, after the Temporal client is connected and the worker has started: use it for one-off workflow starts (Stripe hand-starts its webhook-processor and outbox-poller workflows here) or schedule creation (FedWiki sets up its sync schedule here). By convention, `Startup` implementations log their own failures and return `nil` — a failing integration should degrade that integration's background functionality, not halt boot — though the composition root also logs a non-nil error defensively. Integration-specific configuration (a farm API URL, a webhook secret, ...) is read by the integration itself (typically from viper), not threaded through either method's generic parameters. ### Config — `config.ConfigProvider` ```go type ConfigProvider interface { ConfigSpec() []ConfigKey } ``` See §6. ### UI assets — `integrations.UIProvider` ```go type UIProvider interface { Templates() fs.FS Static() fs.FS } ``` Both filesystems are key-namespaced: **every template file, every defined template name, and every static asset path an integration contributes must be prefixed with its key** (FedWiki's `fedwiki_*.html` templates and `fedwiki-create-form.js` conform today). The composition root enforces this at startup: `Templates` is parsed into the shared core template set, and any `*.html` file whose name lacks the key prefix — or any template parse error — **panics** rather than returning an error, since this is a static contract-conformance check on the integration's own asset naming (a startup-time bug, never a runtime condition), mirroring the `template.Must` convention core's own template set already uses. `Static` is mounted under the existing `/static/` route at the per-key subpath `/static//` — same origin, no new route surface, no CSP change. A mount with a nil or empty `Templates`/`Static` is a no-op (an integration may implement `UIProvider` for only one of the two). ### Member dashboard cards — `server.DashboardCardProvider` ```go type DashboardCardProvider interface { DashboardCards() []server.DashboardCard } ``` An integration with a member-facing surface declares dashboard cards; core's `index.html` renders one **generic card shell** per declaration (header from `Title`, body self-loading via HTMX from `PartialPath`, re-loading on the optional `RefreshEvent` fired on ``) in registry order, and names no integration anywhere. A `DashboardCard` is a shell, not markup: - **The body is integration-owned and HTMX-delivered.** Core never renders integration templates inline into the dashboard page — no template-set coupling, no html/template escape-analysis surprises. Serve the body from your own partial route (session-authenticated inside your handler, like the rest of your member partials). - **Modal markup travels inside the partial**, not at page level. Give each modal `hx-preserve` (with a stable `id`) so an open modal survives the refresh-event re-render — htmx carries the same-id DOM node across the swap; without it Bootstrap orphans the backdrop of a modal that is open when the card refreshes (FedWiki's create flow is exactly this case). - **`Scripts` are page-level**: each entry renders as `