member-console/docs/models/provider-integration.md

title, audience, summary

title	audience	summary
Providers and Integrations	developer	How external services plug in: the provider registry, boot-time manifests, resource-key ownership, the shared webhook and outbox substrate, and runtime-managed configuration.

Providers and Integrations

Purpose

The console is a hub for external services, and this model is the socket they plug into. Two words carry the whole design and must not be swapped: a provider is the domain entity — a row in the registry recording an external service's kind, its declared lifecycle capabilities, and the resource keys it owns; an integration is the code unit — the tree under internal/integrations/<key>/ that implements and registers exactly one provider. ("Extension" is a retired synonym.) Three providers exist today: FedWiki (wiki hosting, the full per-instance lifecycle), Discourse (forums, delivering converged group membership — a managed forum group whose member set the console recomputes and pushes), and Stripe (payments — its webhook and outbox workers already live in its integration tree, while checkout, fulfillment, and catalog sync remain core-side pending the tracked payments-provider seam). One wording rule for this card: "provider kind" always means the registry's four-way category, and "key kind" always means a resource key's boolean/numeric shape — the bare word "kind" is never used alone.

The registry splits every provider's rows into two halves with different owners. Capability rows — the kind, the declared lifecycle verbs and states, the owned resource keys, the operator-surface path — are stamped from a typed manifest in code at every boot, idempotently. Operational rows — the provider's status, anything an operator edits — are database-canonical, and a boot re-registration never touches them. Everything registry-driven follows: the operator sidebar lists providers from the registry, the settings pages render from declared config specs (with operator overrides stored in one table and secrets never stored at all), and adding an integration is one code tree plus one line in a compile-time list — no core schema change.

Two shared substrate tables serve all providers: core.webhook_events stores verified inbound events, and core.outbox carries outbound actions that must commit atomically with a domain write. Dispatch transport is a capability choice, not a mandate: Stripe enqueues through the outbox because its actions must ride the caller's transaction; FedWiki and Discourse dispatch directly through Temporal because their workflows own the database writes end-to-end.

Vocabulary reused from the Entitlements card: resource key, key kind (boolean/numeric), boolean and numeric entitlements, usage counters. From Payments and Billing: webhook event store, outbox, dead-letter. Temporal is the workflow engine the console runs background jobs on; goose is the SQL migration tool, and each migration stream keeps its own goose version-ledger table with its own 00001-up numbering.

Where this lives

Registry and substrate tables sit in the core schema (internal/db/migrations/00001_init.sql); each integration owns its own schema and migration stream.

Piece	What it is	Where
core.providers	The registry: provider (the key, and the primary key; named slug until 00014 renamed it to upstream's ratified name. Providers are a vocabulary, so the primary key is the key and no second key column is added; see docs/identifiers.md), provider kind, display name, the operator-surface path (the URL under which the provider's operator pages mount), and the operator-editable status with its suspended_at/retired_at timestamps	00001_init.sql; 00010_schema_hardening.sql; 00014_provider_column_roles_uniqueness.sql
core.provider_operations	One row per declared lifecycle verb per provider; the verb set is closed to five: create, set_status, delete, list, describe	00001_init.sql
core.provider_states	One row per declared lifecycle state per provider — deliberately per-provider vocabulary, format-checked but not a closed list	00001_init.sql
core.resource_keys (provider, kind)	Ownership and shape on the flat key namespace: provider names the owning provider or is empty for platform keys; the key kind is boolean or numeric	00001_init.sql; 00008_resource_key_kinds.sql
Discourse convergence	The set-shaped delivery in action: a sweep recomputes each managed forum group's desired member set from boolean entitlements and pushes adds/removes, refreshing an observed projection	internal/integrations/discourse/workflows/; discourse.observed_group_members
core.webhook_events, core.outbox	The shared inbound and outbound substrate (described in the Payments and Billing card); both deliberately carry no foreign keys, staying provider-generic	00001_init.sql
core.integration_config_overrides	Operator-set values for non-secret config keys, overlaid at boot over environment and defaults	00009_integration_config_overrides.sql
The provider contract	The manifest types every integration fills in: kind, operations, states, resource keys, operator-surface path	internal/integration/contract.go
Boot registration	Validates every manifest, upserts registry rows, reconciles verb and state sets, and stamps key ownership — one idempotent transaction	internal/integration/registration.go
integration.Enqueue	The only supported write into core.outbox; takes the caller's transaction so the domain write and the queued action commit together	internal/integration/outbox.go
The integration interface and registry	The mandatory capability (key, manifest, migration source) and the compile-time list of integrations	internal/integrations/integration.go; internal/integrations/registry.go
Optional capability hooks	Routes, Temporal workflows, config specs, UI assets, and member-dashboard cards — each an optional interface the composition root discovers by type assertion	cmd/start.go; docs/building-an-integration.md
Config overlay	Applies stored overrides over environment over defaults at boot, validates enum keys, refuses secrets, and snapshots the effective values so the settings page can mark saved-but-not-yet-applied changes as pending a restart	internal/config/overlay.go
Operator settings surface	One generic handler renders /operator/integrations/<key>/settings for every integration from its declared config spec	internal/server/operator_integration_settings.go
Registry-driven operator surfaces	The sidebar's Integrations group lists provisioning-kind providers from the registry; the Integrations landing page deliberately lists every provider kind, so Stripe gets a row and a reachable settings link too	internal/server/operator_pages.go
Status presentation (ux-honest-surfaces)	The Integrations list and the landing surface's System region each show two distinct, honestly-labeled facts per provider — the operator-managed registry status (labeled so it cannot read as a health check; never changed by any connectivity probe) and a configuration-readiness signal (configured/missing-key names) derived from the same required-key resolution the settings page performs. Neither surface claims "connected" or "read live". The System region's outbox report also names, in its caption, which integrations' queued work it actually covers (today: Stripe's only) — an empty or draining outbox is never evidence that an uncovered integration is healthy	internal/config.RequiredKeysUnresolved; internal/server.configurationReadiness (shared by operator_pages.go and operator_overview.go)
The three manifests	FedWiki (provisioning, all five verbs, states, owns fedwiki_sites), Discourse (provisioning, read verbs only, owns discourse_posting), Stripe (payment, no verbs, no keys)	internal/integrations/*/store/provider.go
Per-integration schemas and migrations	Each integration's own schema, roles, and goose ledger with native numbering	internal/integrations/*/store/migrations/
Migration ordering	Core first, then core-module streams (domains), then integrations in interchangeable order	internal/migrate/sources.go
Authoring guide	The normative how-to for building an integration, including the dispatch decision rule and conformance notes	docs/building-an-integration.md
Conformance record	The clause-level check of Discourse against the upstream design's provider-contract document (numbered doc-39 in design/documents/)	status/explorations/doc39-conformance-2026-08/doc39-conformance-2026-08-21.md

Invariants

An invariant is a rule the model guarantees everywhere; code that would break one is wrong even if it works locally. Tags: [db] — a schema constraint rejects violations; [db-code] — database-side code upholds it; [app] — only Go code upholds it.

1. [db] providers.provider is a key in the ui-vocabulary sense: manifests name provider keys literally and the integration settings path carries one, so the identifier policy's consumer test admits the column. Registry hygiene is CHECK-enforced (chk_providers_provider_hygiene): a provider key is lowercase alphanumeric with no underscores, a provider kind is one of payment / provisioning / notification / tax, a declared operation is one of the five verbs, and the operator-editable status is one of active / suspended / retired (with suspended_at/retired_at recording when a provider entered those states). Declared states are format-checked but deliberately not a closed list.
2. [app] A provider declaring set_status must list the active state plus at least one other in its manifest; a provider without set_status lists no states. Boot validation rejects violations.
3. [app] Every provider-owned resource key is prefixed with the provider's key plus underscore, and the key row must already exist from a migration — a manifest stamps ownership onto existing keys, it cannot invent them. The rule's home for the row is the integration's own stream; fedwiki_sites is the one grandfathered exception, seeded core-side, and the authoring guide says to copy Discourse.
4. [db] A key's provider must name a registered provider or be empty — a foreign key enforces it. Boot validation additionally rejects (app-side) a provider key that would nest inside a platform key's first underscore-separated word: a provider named external would be refused because the platform key external_domain_claims starts with that word.
5. [app] Boot re-registration refreshes capability rows in one transaction and never clobbers the operational status column — operators own it.
6. [app] The operator sidebar shows only provisioning-kind providers; the Integrations landing page deliberately lists every provider kind, and any integration declaring config keys gets a reachable settings page regardless of kind — Stripe included.
7. [app] integration.Enqueue is the only path that inserts into core.outbox, and it rides the caller's transaction — a domain write and its queued action commit or roll back together. (The drainer and the operator retry surface update existing rows' statuses; they never insert.)
8. [app] Inbound webhook dedup is handler-level: Postgres requires a partitioned table's unique constraints to include the partition column, so the uniqueness necessarily includes the received-at timestamp and cannot reject a redelivery on its own — every receiver therefore inserts with an existence check on (provider, event id). Adding the "obvious" two-column unique index is exactly the fix the partitioning forbids.
9. [db] The substrate tables carry no foreign keys into any module table; entity references are text or payload fields, keeping both tables provider-generic.
10. [app] Secret config keys are never stored: the settings page rejects writes to them and the overlay refuses to apply them; secrets live in the environment only.
11. [app] Config resolves override → environment → declared default, once, at single-threaded boot. An unrecognized stored override is skipped; an invalid one aborts startup; changes take effect on restart only.
12. [app] Schema references point from integration schemas into core, never the reverse and never sideways between integrations — which is exactly why integration migration order is interchangeable. Nothing mechanical blocks a violation; review must.
13. [app] Integration UI is key-namespaced — templates, template names, and static paths all carry the integration key, and the composition root panics at startup on an unprefixed template — and member-dashboard cards are generic shells: core renders a header and a loading body per declaration, naming no integration anywhere; the card body is served by the integration's own routes.
14. [app] An integration's Key() method and its manifest's Key must be the same string, and boot asserts it (integration.AssertKeyMatch): a mismatch stops startup with an error naming the integration and both values, instead of registering the provider under one key while mounting its UI and settings under the other. Key here is Go contract vocabulary, not a column: the value it carries is written to providers.provider. The contract spelled these identifiers "slug" until entity-keys task 8.2 renamed them; the grammar ^[a-z0-9]+$ and the no-separator rule are unchanged, because a provider key prefixes resource keys.
15. [app] Every operator surface presenting a provider's core.providers.status must label it as a registry/lifecycle value, never as connectivity or health — the console performs no provider health checks, so any wording implying one (e.g. "connected") would be a claim the system cannot back. A provider with any RequiredGroup-tagged config key unresolved must be visibly marked not-configured wherever its status renders, derived from the one shared resolution (config.RequiredKeysUnresolved) — never a second, independently-drifting readiness check.

Dimensions

• Provider vs integration — the registry row vs the code tree. One-to-one, and the words are not interchangeable.
• Provider kind — payment / provisioning / notification / tax; the sidebar shows only the provisioning subset, while the Integrations landing page and settings surface span all kinds (invariant 6).
• Capability rows vs operational rows — boot-stamped from code and reconciled every boot, vs database-canonical and operator-owned. The line runs through the middle of core.providers.
• Mandatory surface vs optional hooks — every integration has a key, a manifest, and migrations; routes, workflows, config, UI, and dashboard cards are opt-in interfaces discovered at composition time.
• Dispatch transport — outbox handoff when dispatch must be atomic with a domain commit; direct Temporal when the workflow owns its own writes. A capability decision per integration, not a doctrine.
• Inbound vs outbound substrate — webhook_events (verified events in, status-polled) vs outbox (actions out, retried to dead-letter).
• Key ownership × shape × metering — ownership (the provider column) and metering (whether usage of the key is measured, via the usage counters in the entitlements model) are independent: a platform-owned key may be metered. Shape is not fully independent — as built, only numeric keys have usage machinery, so a boolean key cannot be metered.
• The three roles of a key — the bare stored id (fedwiki_sites), the grouping attribute (the provider column), and the display name shown to humans. A dotted provider.key form does not exist in storage and must never be parsed out of a key string.
• Config key classes — secret vs overridable; boot-required vs runtime-managed; enum-closed vs free. Orthogonal, all declared in the spec.
• Per-instance lifecycle vs converged-set delivery — FedWiki's resource is instance-shaped (create / set_status / delete per site); Discourse's is set-shaped (a member-set converged by reconciliation, manifest declaring only the read verbs). The verb set models the first natively and merely permits the second — the registry cannot yet say "this provider reconciles by design".
• Registry status vs configuration readiness — the operator-managed core.providers.status (a stored lifecycle value, changed only by an operator) vs whether the declared required config actually resolves (a read-time computation, config.RequiredKeysUnresolved). Independent facts, both shown, neither one a health check (invariant 15).