Files
member-console/design/documents/reference-modular-monolith-schema-decomposition.md
T
cgalo5758 bfe9cee0fe Consolidate design docs into documents directory
- Remove per-module projection files (README, architecture, companion,
  interfaces, model) under design/<module>/
- Add design/documents/ with numbered design docs, references, policies,
  and manifest
- Update design/README.md to describe the directory as a mirror of
  membcons-db's normative surfaces
- Record Decisions 140-141 in companion and glossary; update
  data-model.md schema organization
2026-08-21 00:55:42 -05:00

26 KiB
Raw Blame History

Decomposing a PostgreSQL monolith into modular schemas

PostgreSQL's native schema mechanism is the ideal tool for partitioning 3080 tables across 36 domain modules within a single database, giving you enforceable boundaries, zero performance overhead, full ACID transactions across modules, and a clean migration path to microservices if ever needed. The approach — one schema per bounded context, no cross-schema foreign keys, roles enforcing access, and events for cross-module communication — is well-validated by Shopify, Kamil Grzybek's reference implementation, and practitioners across the DDD community. What follows is a deep synthesis of the patterns, tradeoffs, PostgreSQL-specific techniques, and pitfalls that matter for getting this right.


The PostgreSQL Wiki's own Database Schema Recommendations states it plainly: "The recommendation is to create a single database with multiple named schemas. This is different than a common (and older) practice of creating multiple databases." For an application with 36 domain areas, this hits a sweet spot between three levels of data isolation:

No isolation (single public schema) works for tiny apps but becomes unmanageable past ~20 tables. You cannot determine which tables belong to which module, and boundaries exist only as conventions that erode under deadline pressure. Schema-per-module (logical isolation) makes ownership visible at a glance — ordering.orders, billing.invoices, shipping.shipments — and enables enforcement via PostgreSQL's privilege system. Separate databases provide the strongest isolation but sacrifice cross-database JOINs entirely (PostgreSQL does not support cross-database queries natively), require separate connection pools, complicate backups, and add operational overhead disproportionate to the benefit at this scale.

The critical insight is that PostgreSQL schemas are purely a logical namespace. The query planner treats schema1.table1 JOIN schema2.table2 identically to a same-schema join. They share the same buffer pool, statistics system, connection pool, and transaction context. For 36 schemas containing 3080 total tables, there is no measurable performance cost.

A practical schema layout for this scenario:

-- Domain module schemas
CREATE SCHEMA identity;      -- users, authentication, roles
CREATE SCHEMA ordering;      -- orders, carts, order_items
CREATE SCHEMA catalog;       -- products, categories, pricing
CREATE SCHEMA billing;       -- payments, invoices, ledger
CREATE SCHEMA shipping;      -- shipments, tracking, addresses

-- Infrastructure schemas
CREATE SCHEMA shared;        -- cross-cutting reference data (currencies, countries)
CREATE SCHEMA extensions;    -- PostgreSQL extensions (pgcrypto, uuid-ossp)
CREATE SCHEMA reporting;     -- cross-module materialized views, read models

This separates cleanly from the third-party integration schemas the team already planned, and replaces the monolithic "core" schema with domain-aligned boundaries.


Enforcing boundaries with roles, privileges, and search_path

Schema separation without enforcement is decoration. PostgreSQL's privilege system turns schema boundaries into hard walls. The pattern is to create a non-login owner role per schema and a separate application role with minimum necessary privileges:

-- Owner role (cannot log in — security best practice)
CREATE ROLE ordering_owner NOLOGIN;
CREATE SCHEMA ordering AUTHORIZATION ordering_owner;

-- Application role with restricted access
CREATE ROLE ordering_app LOGIN PASSWORD '...';
GRANT USAGE ON SCHEMA ordering TO ordering_app;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA ordering TO ordering_app;
ALTER DEFAULT PRIVILEGES FOR ROLE ordering_owner IN SCHEMA ordering
    GRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO ordering_app;
GRANT USAGE ON ALL SEQUENCES IN SCHEMA ordering TO ordering_app;

-- Critical: set search_path so unqualified names resolve to this module's schema
ALTER ROLE ordering_app SET search_path = ordering, extensions, shared;

The ordering_app role physically cannot read or write billing schema tables. If a developer writes code that accidentally queries across module boundaries using this connection, PostgreSQL returns a permission error. Milan Jovanović recommends going further: use separate connection strings per module even within the same database, so each module's ORM context connects with its own credentials.

Three important search_path practices deserve attention. First, always use fully-qualified table names (ordering.orders, not just orders) in application code and migrations to eliminate ambiguity. Second, include the extensions schema in each role's search_path so extension functions like gen_random_uuid() resolve without schema-qualifying every call. Third, remove public from the search_path if you aren't using the public schema — this prevents accidental object creation there and closes a subtle security vector where any user with CREATE privilege on a schema in your search_path could create trojan-horse functions.

For granting other modules read access to specific data, create read-only roles per schema:

CREATE ROLE ordering_reader NOLOGIN;
GRANT USAGE ON SCHEMA ordering TO ordering_reader;
GRANT SELECT ON ALL TABLES IN SCHEMA ordering TO ordering_reader;
ALTER DEFAULT PRIVILEGES FOR ROLE ordering_owner IN SCHEMA ordering
    GRANT SELECT ON TABLES TO ordering_reader;

-- Grant to the reporting role
GRANT ordering_reader TO reporting_app;

One frequently missed detail: GRANT ... ON ALL TABLES only affects existing tables. Without ALTER DEFAULT PRIVILEGES, any table created after the grant is invisible to the grantee. This is the single most common source of "why can't my app see the new table" bugs in multi-schema setups.


The cross-schema foreign key question

PostgreSQL fully supports foreign keys between schemas — CREATE TABLE shipping.parcels (order_id INT REFERENCES ordering.orders(id)) works perfectly. The question is whether you should use them, and here the architectural answer diverges from the database-level answer.

Within a module, use foreign keys freely. They enforce data integrity, document relationships, and enable CASCADE operations. The tables belong to the same bounded context, will evolve together, and will be extracted together.

Between modules, avoid foreign keys. Store the referenced entity's ID as a plain column (customer_id UUID with no FK constraint) and validate through the owning module's API. This is the consensus across DDD practitioners, Kamil Grzybek's reference implementation, Shopify's Packwerk guidance ("hold foreign IDs referencing models across package boundaries, but do NOT hold strictly enforced foreign database keys"), and Sam Newman's decomposition patterns.

The reasoning is straightforward: cross-schema FKs create physical coupling between modules. Module A's migration that drops or renames a column in its table will break Module B's FK constraint. Extraction to a separate database later requires removing every cross-module FK — and discovering them all under production pressure is unpleasant. More subtly, cross-schema FKs encourage developers to think of cross-module data as "just another table" rather than as an external dependency requiring an explicit contract.

The tradeoff is real: without database-level referential integrity, you can end up with orphaned references (an order.customer_id pointing to a deleted customer). The mitigation is application-level validation through module APIs, combined with periodic consistency checks for critical references. For a modular monolith where modules share a database and communicate in-process, this risk is manageable. The orphaned-reference problem is identical to what microservices face, and the industry has well-established patterns for handling it.

Three alternatives to cross-schema FKs worth considering:

  • Cross-schema views as contracts: A module publishes a read-only view exposing only the data other modules need, without exposing raw tables. The reporting schema is a natural home for these.
  • Materialized views as module interfaces: For data that's queried frequently but doesn't need real-time freshness, materialized views in a reporting schema can aggregate cross-module data. Use REFRESH MATERIALIZED VIEW CONCURRENTLY for zero-downtime refreshes.
  • Event-driven local copies: Module B subscribes to Module A's CustomerUpdated events and maintains a minimal local copy (shipping.customer_cache with just id, name, address). Most decoupled approach; directly prepares for microservice extraction.

Naming conventions that prevent confusion

When both domain modules and third-party integrations use PostgreSQL schemas, confusion is inevitable without clear conventions. Three naming strategies work in practice:

Flat domain names (recommended for 36 modules): Use simple, descriptive names — ordering, billing, identity, catalog. Third-party schemas get a prefix: ext_stripe, ext_sendgrid, plugin_analytics. Infrastructure schemas use their function: extensions, shared, reporting. This is the simplest scheme and scales to ~10 schemas before needing more structure.

Prefix-based grouping (for larger systems): core_ordering, core_billing, ext_stripe, infra_extensions. This groups related schemas alphabetically in \dn output and makes ownership immediately obvious.

Functional grouping (for closely related subdomains): sales_orders, sales_payments, warehouse_inventory, warehouse_shipping. Useful when a single domain area has enough tables to warrant sub-schemas.

Regardless of scheme, follow PostgreSQL conventions: lowercase with underscores for all identifiers (PostgreSQL folds unquoted identifiers to lowercase), avoid the pg_ prefix (reserved for system schemas), and keep names under 63 characters (the NAMEDATALEN limit). Within schemas, the Bytebase PostgreSQL Style Guide and community consensus recommend plural nouns for tables (orders, order_items), v_ prefix for views, mv_ prefix for materialized views, and descriptive function names (calculate_discount, get_user_by_id).

A powerful PostgreSQL-native technique for making ownership visible: create a role synonymous with each schema name as its owner. Running \dn+ in psql then shows each schema's owner immediately, making the mental model of "who owns what" concrete.


How modules talk to each other inside a monolith

Once data is partitioned into schemas, modules need explicit communication channels rather than reaching into each other's tables. Kamil Grzybek classifies four integration styles with clear tradeoffs, and the practitioner consensus is to combine synchronous calls for reads with asynchronous events for state changes.

Direct method calls (synchronous) are the simplest pattern. Each module exposes a public interface — a façade or service interface — defined in a shared contracts library. Implementations are internal. Dependency injection provides the implementation at runtime. Module A calls userClient.ofId(userId) to get user data from the Identity module. This is fast (in-memory, no serialization), simple to implement, and easy to understand. The downside is temporal coupling: if Module B is slow, Module A blocks. When extracting to microservices, every direct call becomes a network call requiring circuit breakers, retries, and timeout handling.

Domain events (asynchronous) decouple modules temporally. When an order is placed, the Ordering module publishes an OrderPlaced event. The Billing module subscribes and creates an invoice. The Shipping module subscribes and creates a shipment. Within a monolith, this can be an in-process event bus (Spring's ApplicationEventPublisher, MediatR in .NET, or a simple observer pattern). Events should be integration events — small, stable contracts containing only the data consumers need, defined in a shared contracts library, not the publishing module's internals. Grzybek emphasizes: "Integration Events should be as small as possible, as they are part of the contract made available by the given module."

The outbox pattern bridges these approaches for reliability. Instead of publishing events directly, the module writes domain events to an outbox table in the same transaction as the domain change. A background process reads the outbox and dispatches events. This guarantees at-least-once delivery (consumers must be idempotent) and translates directly to message-broker-based microservice communication when you extract. Grzybek's reference implementation uses this exact pattern. The PostgreSQL-specific advantage is that the outbox write participates in the same ACID transaction as the domain write, giving you atomic "save + publish" semantics that are impossible to achieve with an external message broker alone.

The recommended hybrid for a greenfield modular monolith:

  • Queries/reads across modules: Direct method calls via interfaces. Module A calls Module B's public API to fetch data. Simple, fast, strongly consistent.
  • State change notifications: Asynchronous events via outbox pattern. Module A commits its change + event to its outbox; Module B reacts asynchronously.
  • Shared reference data: Read-only access to a shared schema, or local cached copies synchronized via events.

DDD bounded contexts as the organizing principle

Eric Evans' definition maps directly to schema boundaries: "Explicitly define the context within which a model applies. Explicitly set boundaries in terms of team organization, usage within specific parts of the application, and physical manifestations such as code bases and database schemas." Each bounded context becomes a PostgreSQL schema containing 520 tables representing that context's aggregates.

The shared kernel pattern deserves careful attention because it's both useful and dangerous. Evans defines it as "some subset of the domain model that the two teams agree to share... Keep this kernel small." In database terms, the shared kernel is the shared schema containing data that genuinely belongs to no single module: user identity reference data, currencies, countries, and similar lookup tables. The kernel must be minimal. If substantial domain logic or many tables accumulate in shared, it signals that bounded context boundaries are wrong and should be redrawn.

For the user table specifically — the hardest shared-data problem in any modular system — the strongest pattern is to designate an identity module as the single owner. Other modules store only user_id as a plain column and retrieve user details through the Identity module's API. When other modules need user attributes for display or search, they maintain local projections synchronized via UserUpdated events containing only the fields they need. The Shipping module might maintain shipping.recipients(id, user_id, display_name, default_address) — a tiny, purpose-built copy of the data it actually uses.

DDD context mapping patterns inform how schemas interact. The most relevant for database design are:

  • Customer-Supplier: The upstream module owns its tables; the downstream module references by ID and calls the upstream API. This is the default relationship between most modules.
  • Anti-Corruption Layer: The downstream module defines its own model and translates from the upstream module's representation. In database terms, Module B has its own tables with its own structure, populated via adapter code that translates events or API responses from Module A.
  • Open Host Service: A module publishes a stable, well-documented API (interfaces + event contracts) as its "open host." Its database schema is private; consumers interact only through the API.
  • Separate Ways: Two modules have no data relationship. Completely independent schemas, no events, no shared data.

The key aggregate design rule that informs schema decomposition: aggregates reference other aggregates only by ID. Within a schema, tables forming one aggregate can have FK relationships. But a table in the Ordering schema should never FK to a table in the Catalog schema — it stores product_id as a value and retrieves product details through the Catalog module's API when needed.


Lessons from Shopify, Basecamp, and the modular monolith movement

Shopify's componentization is the most documented large-scale modular monolith journey. With 2.8 million lines of Ruby, 1,000+ developers, and 32 million requests per minute during Black Friday, they chose a modular monolith over microservices after concluding: "All the things we liked about our monolith were a result of the code living in and being deployed to one place, and all the issues we were experiencing were a direct result of a lack of boundaries." They reorganized from technical layers (controllers/, models/) to business domains (orders/, checkout/, billing/), structured each as a Rails Engine, and built Packwerk — an open-source static analysis tool that enforces boundary violations at CI time. Packwerk checks two types of violations: dependency violations (referencing undeclared dependencies) and privacy violations (accessing a module's private internals). Their explicit database guidance: "hold foreign IDs referencing models across package boundaries, but do NOT hold strictly enforced foreign database keys."

DHH's Majestic Monolith philosophy takes a more pragmatic stance. Basecamp serves millions of users with ~12 programmers from a single Rails codebase. His evolution — the Citadel pattern — keeps the monolith at center but supports it with "Outposts," small extracted services for specific divergent behavior. Chat (Campfire) was extracted because its polling architecture diverged from the monolith's patterns. The lesson: extract only when there's a concrete technical or organizational reason, not preemptively.

Kamil Grzybek's reference implementation (modular-monolith-with-ddd, 13,000+ GitHub stars) provides the most complete technical blueprint. Each module has four sub-assemblies (Application, Domain, Infrastructure, IntegrationEvents), its own IoC container, and its own database schema. Only the IntegrationEvents assembly is shared between modules. Cross-module communication is exclusively asynchronous via an events bus with the outbox + inbox pattern. Architecture tests using NetArchTest verify that modules don't access each other's internals.

The industry trend is unmistakable. Sam Newman at QCon London: "Most companies would do better with the highly underrated option of a modular monolith." Amazon Prime Video achieved a 90% infrastructure cost reduction moving from microservices back to a monolith. Twilio Segment collapsed 140+ microservices into a single monolith after discovering that 3 full-time engineers spent most of their time firefighting operational issues. Google's 2023 research paper "Towards Modern Development of Cloud Applications" endorses starting monolithic with modular structure.


Preserving the migration path to microservices

Sam Newman describes the schema-per-module modular monolith as a "hedging architecture" — it preserves all your options. The patterns that make later extraction easy are exactly the patterns described above: schema isolation, no cross-module FKs, module APIs as contracts, and event-based integration. Newman notes that many systems adopting this architecture never need to go further, citing a revenue service that remained a monolith with multiple associated database schemas for years.

What makes extraction easy:

  • Modules with low coupling, clear data ownership, and event-based communication extract cleanly. You replace in-process method calls with HTTP/gRPC calls, swap the in-memory event bus for a message broker, and point the extracted service at its own database seeded from its schema's data.
  • The outbox pattern translates directly: swap the background process that reads the outbox and dispatches in-process events with one that publishes to Kafka/RabbitMQ.
  • If module APIs are already defined as interfaces with DTOs (not domain entities), the interface becomes the service's API contract with minimal changes.

What makes extraction hard:

  • Cross-module JOINs in business logic (not just reporting) create dependencies that require replacing database joins with API calls + application-level joining — a significant refactor.
  • Cross-module transactions. Within PostgreSQL, a single BEGIN...COMMIT can span ordering.orders and billing.invoices. Once extracted to separate databases, you need sagas with compensating transactions. Design so cross-module transactions are rare; if every operation requires one, your boundaries are wrong.
  • Shared reference data. The shared.users table works fine in a single database but becomes a distributed data problem when extracting.

The recommended extraction sequence (when the time comes): split code first, then database. Newman recommends this because it reveals what data the extracted service actually needs. Use the Strangler Fig pattern — new functionality goes to the new service, existing functionality migrates incrementally. The Branch by Abstraction pattern lets you swap implementations behind a feature flag, running both old and new paths simultaneously before cutting over. Change Data Capture (Debezium with PostgreSQL logical replication) enables synchronizing data between the old schema and the new service's database during transition.


What teams typically miss

Reporting is the silent killer of clean decomposition. Teams design beautiful module boundaries for writes but break every analytics query. The mitigation is straightforward for a schema-per-module PostgreSQL setup: cross-schema JOINs work natively with zero overhead, so reporting continues to work as-is. Create a reporting schema with materialized views for complex analytics. Only introduce full CQRS with event-driven read models when you actually extract modules to separate databases. Martin Fowler warns explicitly: "For most systems CQRS adds risky complexity... I've certainly seen cases where it's made a significant drag on productivity."

Schema migration tooling needs per-module configuration. With Flyway, configure separate migration script directories per schema; each schema gets its own flyway_schema_history table. With Liquibase, use separate SpringLiquibase beans per schema. An initialization script should create all schemas before module-specific migrations run: CREATE SCHEMA IF NOT EXISTS ordering; CREATE SCHEMA IF NOT EXISTS billing; Always make migrations backward-compatible — never drop a column and change code in the same deployment. Use the expand/contract pattern: add new column, deploy code writing to both, backfill, deploy code reading from new, drop old.

Testing across boundaries requires a deliberate strategy. Unit tests stay within modules. Module integration tests use real databases (Testcontainers is ideal) to verify the module's public API contract. System integration tests boot the full application and exercise cross-module workflows. Architecture tests (ArchUnit, Spring Modulith, NetArchTest, Packwerk) run in CI to verify no dependency cycles, no access to other modules' internals, and respect for the allowed dependency graph. These architecture tests are non-negotiable — without automated enforcement, boundaries degrade within weeks under delivery pressure.

"We'll share the DB for now" becomes permanent. This is the most common way teams undermine their own architecture. The decision to share a table between modules — "just temporarily, we'll fix it later" — creates coupling that accumulates and never gets resolved. Establish the rule from day one: one writer per dataset, no cross-schema direct access, always go through module APIs.

PostgreSQL extensions need a dedicated schema. Extensions are database-wide objects, but their functions, types, and operators live in schemas. By default they install into the first schema in search_path (usually public). Create a dedicated extensions schema: CREATE EXTENSION pgcrypto WITH SCHEMA extensions; Note that some extensions (notably PostGIS) fix their schema in their control file and cannot be relocated.

Don't over-engineer for hypothetical microservices. Newman: "Microservices are not a good choice for most startups." The modular monolith delivers approximately 80% of microservices' organizational benefits (clear ownership, independent development, visible boundaries) at a fraction of the operational cost. Extract only when concrete evidence — scaling needs, team autonomy requirements, compliance isolation — justifies the complexity. A well-structured modular monolith with schema-per-module in PostgreSQL may well be the final architecture, and that's a perfectly good outcome.


Conclusion

The architecture that emerges from this research is neither novel nor surprising — it's the convergence of DDD bounded contexts, PostgreSQL's native schema mechanism, and hard-won lessons from teams like Shopify who've walked this path at scale. Create one schema per domain module, enforce boundaries with PostgreSQL roles, communicate through module APIs and domain events, avoid cross-schema foreign keys, and use the outbox pattern for reliable event delivery. Keep a minimal shared schema for truly cross-cutting reference data. Use cross-schema JOINs freely for reporting — they cost nothing in PostgreSQL — and defer CQRS complexity until you actually need separate databases.

The most important insight is strategic, not technical: the schema-per-module modular monolith is not a compromise or a stepping stone. It is a legitimate, production-proven architecture that preserves full optionality. If extraction to microservices is later justified, every pattern described here — isolated schemas, ID-only cross-module references, event contracts, outbox tables — translates directly to a distributed architecture. If extraction is never needed, you have a clean, maintainable, well-bounded system running on a single PostgreSQL database with full ACID transactions and simple operations. Either outcome is a win.