pgevolve
Postgres-specific declarative schema management.
pgevolve treats a directory of CREATE-style SQL files as the
source of truth for one or more Postgres schemas, introspects a live
database to derive its current state, and computes ordered,
dependency-aware migration plans that bring the database to the
desired state. It refuses to lose data unless explicitly authorized
in a per-plan intent file.
Current release: v0.3.9 (Postgres 14–18). See the Changelog for per-release detail.
What to read
- New? Start with Install and Quick start.
- Day-to-day user? Commands, Configuration, Cookbook, and Troubleshooting.
- Asking "does pgevolve support X?" — the Reference section is the living capability catalogue. Every claimed feature has a fixture; status legend at the top of each spec file.
- Curious how it works? Architecture explains the parser → IR → diff → planner → executor pipeline.
- Considering contributing? Read the Constitution, the v1.0 charter, and any recent Specs + Plans to see how features get designed and shipped.
Project links
- Source: github.com/saosebastiao/pgevolve
- Library API: docs.rs/pgevolve-core
- Issues: GitHub Issues
License
Dual-licensed under MIT or Apache-2.0.
Installation
v0.3.x is pre-release; the only supported install is from source.
Prerequisites
- Rust 1.95 or newer. The repo pins
1.95inrust-toolchain.toml, sorustupwill pick it up automatically. - Docker (optional). Required only for
pgevolve validate --shadow, for the tier-3/4/5 test suites, and for any local property-test runs. - Postgres 14–18. pgevolve introspects through
pg_catalog; major versions outside this range are not tested.
Build the binary
git clone https://github.com/saosebastiao/pgevolve.git
cd pgevolve
cargo build --release -p pgevolve
The release binary lands at target/release/pgevolve. Add it to your
PATH (or copy it to ~/.local/bin/, /usr/local/bin/, etc.):
install -m 0755 target/release/pgevolve ~/.local/bin/pgevolve
pgevolve --version
Verify the install
A no-Postgres smoke check:
pgevolve --help
pgevolve init --dir /tmp/pgevolve-smoke
ls /tmp/pgevolve-smoke
# → pgevolve.toml schema/ plans/ .gitignore
If you have Docker available and want to verify the shadow path works end-to-end:
cd /tmp/pgevolve-smoke
echo '[shadow]
backend = "testcontainers"
postgres_version = "16"' >> pgevolve.toml
mkdir -p schema/app
cat > schema/app/0001-init.sql <<'SQL'
-- @pgevolve schema=app
CREATE SCHEMA app;
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
CONSTRAINT users_pkey PRIMARY KEY (id)
);
SQL
pgevolve validate --shadow
# → pgevolve validate --shadow: round-trip matched (1 object(s))
Upgrading
Until a stable release ships, upgrade by pulling and rebuilding:
cd pgevolve
git pull
cargo build --release -p pgevolve
install -m 0755 target/release/pgevolve ~/.local/bin/pgevolve
The pgevolve metadata schema is upgraded idempotently by every command
that touches the DB, so there's no separate migration step on upgrade.
Pre-built binaries / package managers
cargo install pgevolve (the canonical install path described above)
is published to crates.io as of
v0.3.3. The following additional distribution channels are not yet
shipped:
- GitHub-release prebuilt binaries for Linux x86_64 / macOS arm64 / Windows x86_64 — planned for v1.0.
- Homebrew formula — under consideration; no commitment.
- Docker image — under consideration; no commitment.
If you depend on one of these channels, open an issue saying so — demand drives prioritization.
Getting started
A walkthrough of pgevolve's full loop on a new project: scaffold → author SQL → plan → review → apply → status. Expect 5-10 minutes.
1. Initialize a project
mkdir myapp && cd myapp
pgevolve init
This creates:
myapp/
├── .gitignore
├── pgevolve.toml ← project configuration
├── plans/ ← future plan directories live here
└── schema/ ← your SQL goes here
Open pgevolve.toml and edit at least [environments.dev].url to a
DSN you can connect to. For the rest of the walkthrough we'll assume:
[project]
name = "myapp"
schema_dir = "schema"
plan_dir = "plans"
layout_profile = "schema-mirror"
[managed]
schemas = ["app"]
[planner]
strategy = "online"
[environments.dev]
url = "postgres://postgres@localhost:5432/myapp_dev"
Heads-up.
pgevolve applywrites to the database. Use a throwaway database for this walkthrough — or run a local one via Docker:docker run --rm -d -p 5432:5432 -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=myapp_dev postgres:16.
2. Author the first version of the schema
The schema-mirror layout profile wants
schema/<schema>/<kind>/<name>.sql. For a users table in schema app:
mkdir -p schema/app/tables
mkdir -p schema/app/_schema # the `_schema.sql` lives at schema/app/
Create schema/app/_schema.sql:
-- @pgevolve schema=app
CREATE SCHEMA app;
Create schema/app/tables/users.sql:
-- @pgevolve schema=app
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT users_pkey PRIMARY KEY (id)
);
The -- @pgevolve schema=app directive lets pgevolve associate any
unqualified objects in the file with the app schema (the CREATE TABLE here is already qualified, so the directive is mainly future-
proofing).
3. Lint and (optionally) shadow-validate
Quick check that your source parses and obeys the layout profile:
pgevolve lint
# pgevolve lint: 0 findings
If you have Docker available, you can round-trip the IR through an ephemeral Postgres to catch normalization surprises before they hit your real database:
# Add a [shadow] block to pgevolve.toml first:
echo '
[shadow]
backend = "testcontainers"
postgres_version = "16"' >> pgevolve.toml
pgevolve validate --shadow
# pgevolve validate --shadow: round-trip matched (1 object(s))
4. Plan the first migration
pgevolve plan --db dev
# Wrote plan abc1234567890123 to plans/2026-05-11-abc1234567890123 (1 group(s), 3 step(s), 0 intent(s))
Inspect what got written:
ls plans/2026-05-11-abc1234567890123/
# intent.toml manifest.toml plan.sql
cat plans/2026-05-11-abc1234567890123/plan.sql
You'll see the same DDL you authored, wrapped in -- @pgevolve directive
comments that pgevolve's executor reads. For details on the directive
format and the three-file layout, see plan-format.md.
5. Apply
pgevolve apply plans/2026-05-11-abc1234567890123 --db dev
# applied (apply_id=<uuid>)
The app.users table now exists in myapp_dev:
psql myapp_dev -c '\d app.users'
6. Make a change
Add a display_name column to schema/app/tables/users.sql:
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
display_name text,
created_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT users_pkey PRIMARY KEY (id)
);
Plan and apply the change:
pgevolve diff --db dev
# 1 change(s):
# - AlterTable
# alter table app.users (1 op(s))
pgevolve plan --db dev
# Wrote plan xyz9876543210xyz to plans/2026-05-11-xyz9876543210xyz (1 group(s), 1 step(s), 0 intent(s))
cat plans/2026-05-11-xyz9876543210xyz/plan.sql
# … contains ALTER TABLE app.users ADD COLUMN display_name text;
pgevolve apply plans/2026-05-11-xyz9876543210xyz --db dev
# applied (apply_id=<uuid>)
7. See history
pgevolve status --db dev
# 2 recent apply/applies:
# <uuid> plan=abc1234567890123 status=succeeded started=… finished=…
# <uuid> plan=xyz9876543210xyz status=succeeded started=… finished=…
pgevolve status --db dev --apply-id <uuid>
# apply <uuid> plan=xyz9876543210xyz status=succeeded
# started_at=… finished_at=…
# pgevolve=0.1.0 source_rev=- target=<hash>
# steps (1):
# [ 1] g1 add_column status=succeeded
What's next
- A destructive change. Drop the
display_namecolumn you just added — pgevolve will write anintent.tomlwithapproved = falseand refuse to apply until you flip it. See troubleshooting.md. - The cookbook (cookbook.md) covers patterns for
adding FKs / setting
NOT NULL/ dropping columns safely, declaring GRANTs and row-level-security policies, tuning storage parameters, and how the online-rewrite policies change the plan shape. - Configuration (configuration.md) has the
full
pgevolve.tomlreference, including per-environment strategy overrides for production-grade deployments. - Cluster surface. Roles live above the per-database layer. If your
deployment owns its own Postgres cluster, scaffold a parallel cluster
project with
pgevolve cluster init(separate from the per-DB project you just created) to manageCREATE ROLEdeclaratively. See the cluster spec.
Cookbook
Concrete migration patterns, with the plan shape pgevolve produces and why. Every recipe assumes you've gone through Getting started and have a working project.
Add a nullable column
The simplest case. The planner emits one ALTER TABLE ADD COLUMN step
in a single transactional group.
-- Before
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
CONSTRAINT users_pkey PRIMARY KEY (id)
);
-- After (add `display_name`)
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
display_name text,
CONSTRAINT users_pkey PRIMARY KEY (id)
);
pgevolve plan --db dev →
-- @pgevolve group id=1 transactional=true
BEGIN;
-- @pgevolve step=1 kind=add_column destructive=false targets=app.users
ALTER TABLE app.users ADD COLUMN display_name text;
COMMIT;
No intent required.
Add a NOT NULL column to a populated table
This is where pgevolve earns its keep. Adding a NOT NULL column with
a default to a large table is normally an ACCESS EXCLUSIVE scan;
pgevolve does it in two steps:
-- After
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
display_name text NOT NULL,
CONSTRAINT users_pkey PRIMARY KEY (id)
);
If display_name already existed as nullable, the planner emits the
four-step CHECK pattern:
BEGIN;
-- @pgevolve step=1 kind=add_check_for_not_null
ALTER TABLE app.users
ADD CONSTRAINT __pgevolve_chk_display_name CHECK (display_name IS NOT NULL) NOT VALID;
COMMIT;
BEGIN;
-- @pgevolve step=2 kind=validate_constraint
ALTER TABLE app.users VALIDATE CONSTRAINT __pgevolve_chk_display_name;
COMMIT;
BEGIN;
-- @pgevolve step=3 kind=set_column_nullable
ALTER TABLE app.users ALTER COLUMN display_name SET NOT NULL;
-- @pgevolve step=4 kind=drop_constraint
ALTER TABLE app.users DROP CONSTRAINT __pgevolve_chk_display_name;
COMMIT;
SET NOT NULL is cheap once the validated CHECK proves no NULL rows
exist; Postgres skips the table scan.
If you want the old-style single-step SET NOT NULL (e.g., on a
guaranteed-empty table), set
[planner.online_rewrites].not_null_via_check_pattern = false for that
environment.
Add a foreign key without locking
Adding an FK to an existing table normally locks during validation.
pgevolve emits the NOT VALID + VALIDATE pattern across two
transaction groups:
-- After
CREATE TABLE app.invoices (
id bigint NOT NULL,
customer_id bigint NOT NULL,
CONSTRAINT invoices_pkey PRIMARY KEY (id),
CONSTRAINT invoices_customer_fk FOREIGN KEY (customer_id) REFERENCES app.customers (id)
);
If app.invoices is new in this plan, the FK rides inline with the
CREATE TABLE. If app.invoices already exists:
-- @pgevolve group id=1 transactional=true
BEGIN;
-- @pgevolve step=1 kind=add_constraint_not_valid destructive=false targets=app.invoices
ALTER TABLE app.invoices ADD CONSTRAINT invoices_customer_fk
FOREIGN KEY (customer_id) REFERENCES app.customers (id) NOT VALID;
COMMIT;
-- @pgevolve group id=2 transactional=true
BEGIN;
-- @pgevolve step=2 kind=validate_constraint destructive=false targets=app.invoices
ALTER TABLE app.invoices VALIDATE CONSTRAINT invoices_customer_fk;
COMMIT;
The two groups are committed independently. If step 2 fails, step 1
stays committed and you can pgevolve plan again to retry only the
validation.
Add a non-unique index concurrently
CREATE INDEX users_email_idx ON app.users (email);
If app.users already exists in the live database, the planner
rewrites this to CREATE INDEX CONCURRENTLY in its own
non-transactional group:
-- @pgevolve group id=1 transactional=false
-- @pgevolve step=1 kind=create_index_concurrent destructive=false targets=app.users_email_idx,app.users
CREATE INDEX CONCURRENTLY users_email_idx ON app.users USING btree (email);
If the index is UNIQUE, pgevolve uses the locking variant
(CREATE UNIQUE INDEX) — see the
indexes.md rationale.
Drop a column
CREATE TABLE app.users (
id bigint NOT NULL,
email text NOT NULL,
- legacy_email text,
CONSTRAINT users_pkey PRIMARY KEY (id)
);
pgevolve plan produces a destructive step and writes an intent.toml
with approved = false:
plan_id = "..."
[[intent]]
id = 1
step = 1
kind = "drop_column"
target = "app.users.legacy_email"
reason = "drops column legacy_email"
approved = false
pgevolve apply refuses to run while approved = false. Edit the
file, commit the change, and the apply succeeds.
Drop a table
Same approval flow as drop-column, with extra data_loss_warning
flag set in the destructiveness record.
Forward-reference FK cycle (chicken-and-egg)
Sometimes two tables FK each other:
CREATE TABLE app.posts (
id bigint NOT NULL,
author bigint NOT NULL,
CONSTRAINT posts_pkey PRIMARY KEY (id),
CONSTRAINT posts_author_fk FOREIGN KEY (author) REFERENCES app.users (id)
);
CREATE TABLE app.users (
id bigint NOT NULL,
latest_post bigint,
CONSTRAINT users_pkey PRIMARY KEY (id),
CONSTRAINT users_latest_post_fk FOREIGN KEY (latest_post) REFERENCES app.posts (id)
);
The planner detects the cycle and extracts one of the FKs into a
post-pass ALTER TABLE ADD CONSTRAINT step. Both tables are created
without that FK first; the FK is added (with NOT VALID rewrite if the
target is large) afterward.
Rename a column or table
pgevolve does not detect renames today — they diff as drop+add, which is destructive for columns (data loss). If you need to rename:
- Add the new column / table, and a backfill in a data migration (a step pgevolve does not handle).
- Cut over reads / writes to the new name.
- Drop the old in a separate, intent-approved plan.
A future version may detect renames via a developer-supplied hint
(e.g., a -- @pgevolve rename directive). For now the safety-first
posture stands.
Re-apply after a partial failure
If step 4 of a 5-step plan fails:
- For a transactional group: the group rolls back. Earlier steps in
the group are also rolled back. Step 4's
error_messageis inpgevolve.plan_steps. Re-plan from the current live state and the re-apply skips the steps that already committed in earlier groups. - For an autocommit group (e.g.,
CONCURRENTLYstep): earlier steps stay committed. Re-planning produces a smaller plan that picks up from where you stopped.
You don't manually fix anything in the plan directory. You re-run
pgevolve plan --db <env> and apply the new plan.
Managing views
Create a simple view
-- schema/app/views/active_users.sql
CREATE VIEW app.active_users AS
SELECT id, email FROM app.users WHERE deleted_at IS NULL;
pgevolve plan emits one create_view step in a transactional group.
Add a column to an existing view (compatible change)
If the new column is appended at the end of the SELECT list, the body
change is compatible: Postgres can apply it without dropping the view.
pgevolve emits CREATE OR REPLACE VIEW:
-- After: add `created_at`
CREATE VIEW app.active_users AS
SELECT id, email, created_at FROM app.users WHERE deleted_at IS NULL;
-- @pgevolve step=1 kind=create_view destructive=false targets=app.active_users
CREATE OR REPLACE VIEW app.active_users AS
SELECT id, email, created_at FROM app.users WHERE deleted_at IS NULL;
Reorder columns (incompatible change → DROP + CREATE)
Reordering columns or changing a column type is incompatible with
CREATE OR REPLACE VIEW. pgevolve emits an explicit drop_view followed
by create_view:
-- After: move `email` before `id`
CREATE VIEW app.active_users AS
SELECT email, id FROM app.users WHERE deleted_at IS NULL;
-- @pgevolve step=1 kind=drop_view destructive=true intent_id=1 targets=app.active_users
DROP VIEW app.active_users;
-- @pgevolve step=2 kind=create_view destructive=false targets=app.active_users
CREATE VIEW app.active_users AS SELECT email, id FROM app.users WHERE deleted_at IS NULL;
The drop_view step is destructive — you must flip approved = true in
intent.toml before applying.
Dependent-view cascade
If view B selects from view A, modifying A's body incompatibly
automatically cascades to B. pgevolve walks the body_dependencies
graph and emits explicit DROP + CREATE steps for every affected view.
The plan is fully auditable: no hidden CASCADE drops.
To opt out of automatic cascade and instead get an error listing the affected views, set:
[planner.online_rewrites]
view_drop_create_dependents = false
Managing user-defined types
Define an enum
-- schema/app/types/order_status.sql
CREATE TYPE app.order_status AS ENUM ('pending', 'processing', 'shipped', 'delivered');
pgevolve plan emits one create_type step.
Add a value to an existing enum
Append a new label at the end (or position it with BEFORE/AFTER in source):
-- After: add 'cancelled'
CREATE TYPE app.order_status AS ENUM ('pending', 'processing', 'shipped', 'delivered', 'cancelled');
-- @pgevolve step=1 kind=alter_type_add_value destructive=false targets=app.order_status
ALTER TYPE app.order_status ADD VALUE 'cancelled' AFTER 'delivered';
No intent required. The step is transactional (Postgres 12+).
Rename an enum value
-- After: rename 'processing' to 'in_progress'
CREATE TYPE app.order_status AS ENUM ('pending', 'in_progress', 'shipped', 'delivered', 'cancelled');
-- @pgevolve step=1 kind=alter_type_rename_value destructive=false targets=app.order_status
ALTER TYPE app.order_status RENAME VALUE 'processing' TO 'in_progress';
Drop an enum value (ReplaceWithCascade)
Postgres does not support ALTER TYPE … DROP VALUE. Removing a value
triggers a ReplaceWithCascade: DROP TYPE CASCADE + CREATE TYPE.
All columns and views referencing the type are recreated in the same
transactional group.
-- @pgevolve step=1 kind=drop_type destructive=true intent_id=1 targets=app.order_status
DROP TYPE app.order_status CASCADE;
-- @pgevolve step=2 kind=create_type destructive=false targets=app.order_status
CREATE TYPE app.order_status AS ENUM ('pending', 'shipped', 'delivered');
Flip approved = true in intent.toml before applying.
Create a domain with a CHECK constraint
-- schema/app/types/positive_int.sql
CREATE DOMAIN app.positive_int AS integer
NOT NULL
CHECK (VALUE > 0);
pgevolve plan emits one create_type step (domain uses the same step kind as enum/composite).
Add a CHECK constraint to an existing domain
-- After: also reject values above one million
CREATE DOMAIN app.positive_int AS integer
NOT NULL
CONSTRAINT positive_int_lower CHECK (VALUE > 0)
CONSTRAINT positive_int_upper CHECK (VALUE <= 1000000);
-- @pgevolve step=1 kind=alter_domain_add_constraint destructive=false targets=app.positive_int
ALTER DOMAIN app.positive_int ADD CONSTRAINT positive_int_upper CHECK (VALUE <= 1000000);
Drop an attribute from a composite type (ReplaceWithCascade)
Postgres supports ALTER TYPE … DROP ATTRIBUTE only when no column
or function depends on the composite. pgevolve always uses
ReplaceWithCascade for composite attribute drops to handle the
general case safely:
-- Before
CREATE TYPE app.address AS (street text, city text, zip text);
-- After: drop 'zip'
CREATE TYPE app.address AS (street text, city text);
-- @pgevolve step=1 kind=drop_type destructive=true intent_id=1 targets=app.address
DROP TYPE app.address CASCADE;
-- @pgevolve step=2 kind=create_type destructive=false targets=app.address
CREATE TYPE app.address AS (street text, city text);
Managing functions and procedures
Define a simple SQL function
-- schema/app/functions/add_one.sql
CREATE FUNCTION app.add_one(x integer)
RETURNS integer
LANGUAGE sql
IMMUTABLE STRICT
AS $$
SELECT x + 1
$$;
pgevolve plan emits one create_or_replace_function step.
Define a PL/pgSQL function with a static body
-- schema/app/functions/get_active_user.sql
CREATE FUNCTION app.get_active_user(p_id bigint)
RETURNS app.users
LANGUAGE plpgsql
STABLE
AS $$
DECLARE
r app.users;
BEGIN
SELECT * INTO r FROM app.users WHERE id = p_id AND deleted_at IS NULL;
RETURN r;
END
$$;
pgevolve extracts the app.users dep edge from the static SELECT statement at parse time.
Replace a function body (in-place)
Edit the function's SQL or PL/pgSQL body. If the language and return type are unchanged, pgevolve plan emits a single create_or_replace_function step — no DROP needed.
-- After: tighten to active users only
CREATE FUNCTION app.get_active_user(p_id bigint)
RETURNS app.users
LANGUAGE plpgsql
STABLE
AS $$
DECLARE
r app.users;
BEGIN
SELECT * INTO r
FROM app.users
WHERE id = p_id AND deleted_at IS NULL AND suspended = false;
RETURN r;
END
$$;
-- @pgevolve step=1 kind=create_or_replace_function destructive=false targets=app.get_active_user
CREATE OR REPLACE FUNCTION app.get_active_user(p_id bigint) ...;
No intent required. If the return type or language changes, pgevolve falls back to DROP FUNCTION CASCADE + CREATE OR REPLACE FUNCTION (destructive — requires intent approval).
Add an overload (same name, different arg types)
PL/pgSQL functions support overloading on arg types. Just add the second definition:
-- schema/app/functions/format_name.sql (integer overload)
CREATE FUNCTION app.format_name(user_id integer)
RETURNS text
LANGUAGE sql STABLE
AS $$
SELECT first_name || ' ' || last_name FROM app.users WHERE id = user_id
$$;
-- schema/app/functions/format_name_text.sql (text overload)
CREATE FUNCTION app.format_name(raw_name text)
RETURNS text
LANGUAGE sql IMMUTABLE STRICT
AS $$
SELECT initcap(raw_name)
$$;
pgevolve tracks each overload independently; the identity is qname + arg_types_normalized.
Define a procedure with COMMIT in the body
-- schema/app/procedures/process_batch.sql
CREATE PROCEDURE app.process_batch(batch_size integer)
LANGUAGE plpgsql
AS $$
DECLARE
r record;
BEGIN
FOR r IN SELECT id FROM app.jobs WHERE status = 'pending' LIMIT batch_size LOOP
UPDATE app.jobs SET status = 'done' WHERE id = r.id;
COMMIT;
END LOOP;
END
$$;
pgevolve detects COMMIT in the body and emits the step with transactional=false (outside a transaction block). The procedure-contains-commit lint warning fires as a reminder that the procedure cannot participate in a larger transaction.
Use -- @pgevolve dep: for dynamic SQL
If a function uses EXECUTE (dynamic SQL), pgevolve cannot extract deps statically. Declare them with a directive:
-- schema/app/functions/refresh_summary.sql
CREATE FUNCTION app.refresh_summary()
RETURNS void
LANGUAGE plpgsql
AS $$
BEGIN
-- @pgevolve dep: app.summary
EXECUTE 'REFRESH MATERIALIZED VIEW app.summary';
END
$$;
Without the directive, the plpgsql-dynamic-sql lint rule fires as an Error. The directive tells pgevolve that app.summary is a dependency, so the planner can order the refresh after any changes to that MV.
Tune storage parameters (reloptions)
Storage parameters (fillfactor, autovacuum_*, parallel_workers,
GIN fastupdate, BRIN pages_per_range, …) are declared inline on
the object. Each typed key has a None default that means "unmanaged"
— pgevolve will neither set nor reset it.
Declare on a new or existing table
-- schema/app/tables/orders.sql
CREATE TABLE app.orders (
id bigint PRIMARY KEY,
customer_id bigint NOT NULL,
placed_at timestamptz NOT NULL
) WITH (
fillfactor = 80,
autovacuum_enabled = true,
autovacuum_vacuum_scale_factor = 0.05,
parallel_workers = 4
);
If app.orders already exists in the catalog without these settings,
pgevolve plan emits a single batched ALTER TABLE:
ALTER TABLE app.orders SET (fillfactor = 80, autovacuum_enabled = true,
autovacuum_vacuum_scale_factor = 0.05, parallel_workers = 4);
Both the inline WITH (…) form on CREATE TABLE and a separate
ALTER TABLE app.orders SET (…); are accepted in source.
Per-AM index reloptions
Indexes accept access-method-specific options, validated at parse time so PG-invalid combinations fail fast:
CREATE INDEX orders_customer_id_idx ON app.orders (customer_id)
WITH (fillfactor = 80); -- B-tree: 50..=100
CREATE INDEX orders_tags_idx ON app.orders USING gin (tags)
WITH (fastupdate = false, gin_pending_list_limit = 4096);
CREATE INDEX orders_placed_at_idx ON app.orders USING brin (placed_at)
WITH (pages_per_range = 32, autosummarize = true);
fillfactor ranges differ per AM: B-tree 50..=100, GiST 10..=100,
SP-GiST 90..=100. BRIN and GIN reject fillfactor outright.
Removing a managed reloption
Removing a value from source does not issue a RESET. This is
the same lenient pattern used by owner, grants, and RLS policies —
pgevolve never destructively undoes state on the catalog side just
because source went quiet.
To clear a reloption you previously managed:
- Apply
ALTER TABLE app.orders RESET (fillfactor);out-of-band (psql, your DBA tooling, a one-off migration). - Remove the
fillfactor = 80declaration from source.
On the next pgevolve plan run, both source and catalog read None
and the diff is empty.
The unmanaged-reloption lint
If the catalog has a reloption that source doesn't declare, the
unmanaged-reloption lint fires as a warning. This includes both
typed keys (e.g., the DBA set fillfactor = 70 directly) and
extension keys (e.g., pg_partman.retention_keep_table = 'true').
Waive via [[lint_waiver]] in intent.toml if the drift is
intentional.
First-apply caveat.
CREATE TABLE … WITH (…)against a brand-new (not-yet-in-catalog) object currently emits theCREATEstep without the inlineWITH (…)clause. The reloptions land on the secondplan+applycycle as anALTER … SET. This is a known v0.3.x limitation (seedocs/spec/reloptions.md); convergent in two iterations.
Grant a role read-only access to a table
pgevolve models per-object owner and grants as of v0.3.1. Both
follow the lenient drift policy: declaring owner = None (the
default in source) means "unmanaged" — the differ will neither set nor
reset the owner. The same applies to grants: declared grants are
added/kept; catalog grants you haven't declared surface as the
unmanaged-grant lint warning but are never silently revoked.
-- schema/app/tables/orders.sql
CREATE TABLE app.orders (
id bigint PRIMARY KEY,
customer_id bigint NOT NULL,
placed_at timestamptz NOT NULL
);
-- @pgevolve owner: app_owner
GRANT SELECT ON app.orders TO reporting;
GRANT SELECT (id, placed_at) ON app.orders TO analytics_readonly;
pgevolve plan emits:
ALTER TABLE app.orders OWNER TO app_owner;
GRANT SELECT ON TABLE app.orders TO reporting;
GRANT SELECT (id, placed_at) ON TABLE app.orders TO analytics_readonly;
To revoke an existing grant, simply remove the GRANT line from source.
The differ emits an explicit REVOKE because the grant was previously
managed (it appeared in your Vec<Grant> and is now gone). This is
distinct from grants the catalog already had but source never claimed:
those are unmanaged, never auto-revoked, and surface via the
unmanaged-grant lint.
For cross-cutting defaults (e.g., "every new table in app is owned by
app_owner and grants SELECT to reporting"), use
ALTER DEFAULT PRIVILEGES:
ALTER DEFAULT PRIVILEGES IN SCHEMA app
GRANT SELECT ON TABLES TO reporting;
pgevolve models these as first-class IR; see
docs/spec/grants.md for the full surface.
Enable row-level security on a table
pgevolve models per-table rls_enabled, rls_forced, and an embedded
policies: Vec<Policy> as of v0.3.2.
-- schema/app/tables/documents.sql
CREATE TABLE app.documents (
id bigint PRIMARY KEY,
owner text NOT NULL,
body text NOT NULL
);
ALTER TABLE app.documents ENABLE ROW LEVEL SECURITY;
CREATE POLICY owner_can_read ON app.documents
FOR SELECT
USING (owner = current_user);
CREATE POLICY owner_can_write ON app.documents
FOR INSERT
WITH CHECK (owner = current_user);
pgevolve plan emits one step per change (CREATE/ALTER/DROP POLICY,
ENABLE/DISABLE/FORCE/NOFORCE ROW LEVEL SECURITY). Any change to a
policy's command (e.g., FOR SELECT → FOR UPDATE) goes through
DROP + CREATE because Postgres has no ALTER POLICY … CHANGE COMMAND.
Two policy attributes use NormalizedExpr for diff (same canon as
CHECK constraints): USING (…) and WITH CHECK (…). Whitespace and
keyword-case differences between source and pg_policies therefore
don't trigger spurious recreates.
The lenient-drift rule applies: a policy in the catalog that source
doesn't declare surfaces as unmanaged-policy (warning) instead of an
auto-DROP. Remove a managed policy from source to drop it explicitly.
See docs/spec/policies.md for the full
attribute matrix.
Manage cluster roles
The role surface is cluster-level, not per-database. pgevolve manages
it via a separate project type (pgevolve-cluster.toml + a roles/
tree) and a parallel command family: pgevolve cluster init / diff / plan / apply / status.
mkdir myapp-cluster && cd myapp-cluster
pgevolve cluster init
Author roles as CREATE ROLE SQL:
-- roles/app_owner.sql
CREATE ROLE app_owner WITH NOLOGIN;
-- roles/reporting.sql
CREATE ROLE reporting WITH LOGIN NOINHERIT;
GRANT app_owner TO reporting;
The full role-attribute matrix is supported (LOGIN/NOLOGIN,
SUPERUSER/NOSUPERUSER, CREATEDB, CREATEROLE, REPLICATION,
BYPASSRLS, CONNECTION LIMIT, VALID UNTIL). Passwords are
intentionally not modeled — set them out-of-band so they never
appear in source-controlled SQL.
pgevolve cluster plan
pgevolve cluster apply plans/2026-05-23-<id>
The per-database commands (pgevolve plan, pgevolve apply, etc.)
treat the role names mentioned in GRANT / OWNER TO clauses as
references — they don't create the roles. Use pgevolve cluster …
to manage role lifecycle once at the cluster level, then reference
those role names across all the per-DB projects that share the cluster.
See docs/spec/cluster.md for the full project
layout and the role-attribute surface.
Set up logical replication
Postgres logical replication is declared in two places: a publication on the source database and a subscription on the target database. pgevolve manages both as first-class objects since v0.3.4 (publications) and v0.3.5 (subscriptions).
Step 1 — Declare the publication (source database project)
-- schema/publications/pub_orders.sql
CREATE PUBLICATION pub_orders
FOR TABLE app.orders, app.order_items
WITH (publish = 'insert, update, delete');
pgevolve plan against the source DB emits:
-- @pgevolve step=1 kind=create_publication destructive=false targets=pub_orders
CREATE PUBLICATION pub_orders
FOR TABLE app.orders, app.order_items
WITH (publish = 'insert, update, delete');
Step 2 — Declare the subscription (target database project)
Keep credentials out of source SQL by using the ${VAR} interpolation syntax:
-- schema/subscriptions/sub_orders.sql
CREATE SUBSCRIPTION sub_orders
CONNECTION 'host=primary.example.com dbname=app user=repl_user password=${REPL_PWD}'
PUBLICATION pub_orders
WITH (binary = true, streaming = on);
The literal ${REPL_PWD} is stored verbatim in plan.sql. It is never
resolved at plan time — only at apply time. This means:
- The plan file is safe to commit and code-review.
- The credential is never written to disk.
pgevolve plan against the target DB emits:
-- @pgevolve step=1 kind=create_subscription destructive=false targets=sub_orders
CREATE SUBSCRIPTION sub_orders
CONNECTION 'host=primary.example.com dbname=app user=repl_user password=${REPL_PWD}'
PUBLICATION pub_orders
WITH (binary = true, streaming = on);
Step 3 — Apply with the credential in the environment
# In CI or your deploy script — never in source
export REPL_PWD="$(vault kv get -field=password secret/repl_user)"
pgevolve apply plans/2026-05-26-<plan-id>
pgevolve scans the plan for ${...} references before opening any connection.
If REPL_PWD is not set, it prints a clear error and exits before touching
the database:
error: unresolved env-var reference ${REPL_PWD} in step 1 (create_subscription)
Changing the connection string
Edit the CONNECTION value in source and re-plan:
-- Updated host after a primary failover
CREATE SUBSCRIPTION sub_orders
CONNECTION 'host=newprimary.example.com dbname=app user=repl_user password=${REPL_PWD}'
PUBLICATION pub_orders
WITH (binary = true, streaming = on);
pgevolve plan detects the connection-string change and emits:
-- @pgevolve step=1 kind=alter_subscription_connection destructive=false targets=sub_orders
ALTER SUBSCRIPTION sub_orders
CONNECTION 'host=newprimary.example.com dbname=app user=repl_user password=${REPL_PWD}';
Adding a publication to an existing subscription
-- Extend sub_orders to also consume pub_users
CREATE SUBSCRIPTION sub_orders
CONNECTION 'host=primary.example.com dbname=app user=repl_user password=${REPL_PWD}'
PUBLICATION pub_orders, pub_users
WITH (binary = true, streaming = on);
-- @pgevolve step=1 kind=alter_subscription_add_publication destructive=false targets=sub_orders
ALTER SUBSCRIPTION sub_orders ADD PUBLICATION pub_users;
Lint: plaintext password caught at plan time
pgevolve's subscription-password-in-source lint fires at parse time if
the source SQL contains a literal password:
-- This triggers a hard lint error — never commit plaintext credentials
CREATE SUBSCRIPTION bad_sub
CONNECTION 'host=primary.example.com dbname=app user=repl password=hunter2'
PUBLICATION pub_orders;
error[subscription-password-in-source]: CONNECTION string contains a literal
password= value. Use ${VAR} env-var interpolation instead.
--> schema/subscriptions/bad_sub.sql:2
The lint is severity Error and not waivable.
See docs/spec/subscriptions.md for the full
option matrix, lint rules, and operational verb rejection.
Run the same plan against multiple environments
A plan is bound to a specific target_identity. If you generate
against dev but want to apply against staging:
pgevolve apply plans/2026-05-11-abc1234567890123 --db staging \
--allow-different-target
This is intentionally explicit. The much safer pattern is to plan twice (once per environment) and review both plans — drift between environments will show up as a plan difference.
Multi-column statistics for correlated columns
When two or more columns are strongly correlated (e.g., status and region
always appear together), Postgres tends to drastically underestimate the
selectivity of combined predicates. CREATE STATISTICS teaches the planner
about these correlations.
Declare the statistic
-- schema/app/tables.sql
CREATE TABLE app.orders (
id bigint NOT NULL,
status text NOT NULL,
region text NOT NULL,
amount numeric NOT NULL,
CONSTRAINT orders_pkey PRIMARY KEY (id)
);
-- schema/app/statistics.sql
CREATE STATISTICS app.orders_status_region
ON (status, region)
FROM app.orders;
pgevolve plan --db dev →
-- @pgevolve group id=1 transactional=true
BEGIN;
-- @pgevolve step=1 kind=create_statistic destructive=false targets=app.orders_status_region
CREATE STATISTICS app.orders_status_region ON (status, region) FROM app.orders;
-- @pgevolve step=2 kind=create_statistic destructive=false targets=app.orders_status_region
ANALYZE app.orders;
COMMIT;
No intent required.
Limit to specific kinds
If you only want functional dependency tracking (the cheapest kind):
CREATE STATISTICS app.orders_dep
(dependencies)
ON (status, region)
FROM app.orders;
The kinds clause accepts any non-empty subset of ndistinct, dependencies,
mcv. Omitting the clause enables all three (Postgres default).
Raise the analyze target for fine-grained estimates
The statistics_target controls how many rows the analyzer samples when
building the statistic. The Postgres default is -1 (inherit the column
setting, usually 100). Raising it to 500 gives much more accurate estimates
for skewed distributions:
-- Before (no target override)
CREATE STATISTICS app.orders_status_region
ON (status, region)
FROM app.orders;
-- After (raise target)
CREATE STATISTICS app.orders_status_region
ON (status, region)
FROM app.orders;
ALTER STATISTICS app.orders_status_region SET STATISTICS 500;
pgevolve plan --db dev →
-- @pgevolve group id=1 transactional=true
BEGIN;
-- @pgevolve step=1 kind=alter_statistic_set_target destructive=false targets=app.orders_status_region
ALTER STATISTICS app.orders_status_region SET STATISTICS 500;
COMMIT;
This uses the cheap AlterStatisticSetTarget path — no DROP + CREATE needed.
What triggers a destructive ReplaceStatistic
Changing the column list or the kinds requires a DROP STATISTICS + CREATE STATISTICS because Postgres has no in-place ALTER for those fields:
-- Before
CREATE STATISTICS app.orders_status_region
ON (status, region) FROM app.orders;
-- After — add amount column
CREATE STATISTICS app.orders_status_region
ON (status, region, amount) FROM app.orders;
pgevolve plan --db dev →
-- @pgevolve group id=1 transactional=false
-- @pgevolve step=1 kind=replace_statistic destructive=true targets=app.orders_status_region intent=required
DROP STATISTICS app.orders_status_region;
CREATE STATISTICS app.orders_status_region ON (status, region, amount) FROM app.orders;
intent=required means you must acknowledge the destructive step in
pgevolve.toml or pass --intent on the CLI before the plan can be applied.
Lint: unmanaged statistics
If a statistic exists in the live database but is not declared in source,
pgevolve emits an unmanaged-statistic warning (severity Warning, waivable):
WARN unmanaged-statistic: statistics app.orders_status_region exists in the
catalog but is not declared in source. Add it to source or waive this
lint in pgevolve.toml.
To waive it:
# pgevolve.toml
[[lint.waive]]
rule = "unmanaged-statistic"
target = "app.orders_status_region"
reason = "Legacy statistic, managed out-of-band."
See docs/spec/statistics.md for the full surface,
step-kind matrix, and catalog reader notes.
Troubleshooting
Common errors and what to do about them. Each entry shows the actual error string pgevolve emits, the cause, and the fix.
Pre-flight failures (exit code 2)
Target identity mismatch
target identity mismatch: plan=abc12345abcd1234 live=ff00112233445566
Cause. The plan was built against a different database than the one
you're applying to. target_identity is a hash of
(current_database, host, port, cluster_name, system_identifier);
applying a dev plan to prod (or vice versa) hits this.
Fix.
- If the difference is intentional: re-run
pgevolve applywith--allow-different-target. - Otherwise: re-plan against the correct environment with
pgevolve plan --db <env>.
Unapproved intent
unapproved destructive intents: 1
Cause. The plan declares one or more destructive steps; the
corresponding [[intent]] rows in intent.toml still have
approved = false.
Fix. Open intent.toml, review each [[intent]] row, and change
approved = false to approved = true for the ones you authorize.
Commit the change in your code-review tool of choice before applying.
Drift detected
drift detected since planning: 3 change(s)
Cause. The live database changed between when you ran pgevolve plan and when you ran pgevolve apply — typically because someone
(or another tool) ran DDL out of band.
Fix.
- Inspect what changed:
pgevolve diff --db <env>. Compare tomanifest.toml'starget_snapshot_json. - If the drift is intentional: re-plan with
pgevolve plan --db <env>and apply the new plan. - If the drift is unintentional: investigate the source. Don't
paper over it with
--allow-drift.
--allow-drift exists as a documented escape hatch for "I know the
drift is harmless"; it should be a thinking step, not a reflex.
Apply failures (exit code 3)
Advisory lock held
pgevolve advisory lock is held by another session
Cause. Another pgevolve apply is running, or one crashed without
releasing the lock cleanly.
Fix.
-
Wait for the other apply to finish.
-
If you're sure no one else is applying:
SELECT held_by, held_since, pgevolve_version FROM pgevolve.lock;shows who claims to hold the lock. The session-scoped advisory lock releases automatically when its session disconnects, so a stale
pgevolve.lockrow often clears itself the moment the next acquirer takes the lock. Stuck rows from a crash are clearable by:SELECT pg_advisory_unlock_all();in the session that holds it, or by terminating that session via
pg_terminate_backend.
Step failed
step 4 (group 2) failed: [42P07] relation "app.users" already exists
Cause. Postgres rejected the SQL. The bracketed code (42P07 here)
is the SQLSTATE;
the rest is the server message.
Fix. Inspect the audit log to see the exact step and SQL:
pgevolve status --db <env>
pgevolve status --db <env> --apply-id <uuid>
Or directly:
SELECT step_no, kind, status, error_message, sql_text
FROM pgevolve.plan_steps
WHERE apply_id = '<uuid>'
ORDER BY step_no;
Common subtypes:
42P07 relation already exists→ you're trying to create something that's already there. Usually means the live state drifted; re-plan.23505 duplicate key value violates unique constraint→ an FK validation or unique-index build failed because the existing data doesn't satisfy the constraint. Fix the data first (out of band), then re-plan.42501 permission denied→ the connection's role lacks the privilege. Connect as a sufficiently-privileged role (typically the schema owner) or grant the missing privileges out of band.25006 cannot run inside a transaction block(CREATE INDEX CONCURRENTLY) → almost certainly indicates a plan-format bug; file an issue.
Lint / validation failures (exit code 1)
Parse error
error: parse error: SyntaxError(...): ERROR: syntax error at or near "..." at /path/to/file.sql:42:1
Cause. pg_query couldn't parse one of your SQL statements. The
file path and line are in the error message.
Fix. Run the offending SQL against a real Postgres to see the same error. Most often it's a typo or an unsupported feature.
Unsupported object kind
error: unsupported object kind: <kind> at /path/to/file.sql:1:1
Cause. You wrote a statement type that's not in pgevolve's whitelist for the current release (e.g., a Postgres feature pgevolve doesn't yet model).
Fix. See docs/spec/objects.md for the
current coverage and roadmap. Views, materialized views, user-defined
types (enum/domain/composite), functions, procedures, triggers, and
extensions are supported as of v0.2.
Layout-profile violation
error: [schema_mirror_path] table should be at `app/tables/users.sql`; found at `schema/oops/users.sql` (schema/oops/users.sql:1:1)
Cause. Your file is in a path that the configured layout profile doesn't permit.
Fix. Either move the file, or switch [project].layout_profile
to one whose rules match your existing layout. free-form enforces no
path rules.
managed_schemas_match
error: [managed_schemas_match] schema `audit` is declared in source but not listed in `[managed].schemas`
Cause. Your source declares a schema that's not in your
[managed].schemas list — meaning pgevolve would ignore everything in
it.
Fix. Add the schema name to [managed].schemas, or remove it from
the source.
Config errors (exit code 4)
Missing config
config error: i/o reading pgevolve.toml: No such file or directory (os error 2)
Cause. pgevolve.toml doesn't exist at the path pgevolve is looking
at (default ./pgevolve.toml).
Fix. Run pgevolve init if this is a new project, or pass
--config <path> if your config lives elsewhere.
Unknown environment
unknown environment: `prod`
Cause. --db prod referenced an environment that's not in
pgevolve.toml.
Fix. Add an [environments.prod] block, or use the correct env
name.
Invalid strategy
parse error: ... unknown variant `bogus`, expected `atomic` or `online`
Cause. [planner].strategy (or [environments.<env>].strategy)
is not "atomic" or "online".
Fix. Use one of the two valid values.
Shadow validation
Docker not available
--shadow requires Docker. Install Docker or run without --shadow.
Cause. pgevolve validate --shadow couldn't run docker info.
Fix. Either install Docker (and ensure your user can run it without
sudo), or drop the --shadow flag — non-shadow validate doesn't
require Docker.
Shadow round-trip mismatch
pgevolve validate --shadow: 1 mismatch(es):
- tables.app.users.columns.email.collation: `None` vs `Some(...)`
Cause. Your source IR doesn't match what pgevolve gets back after applying it to a fresh Postgres of the configured version. Usually indicates a Postgres normalization that the IR doesn't account for, or a bug in pgevolve's introspection.
Fix. Report this as an issue with the source file + the error
output. Until it's fixed, you can pin a different [shadow] postgres_version to see if the mismatch is version-specific.
When all else fails
Open an issue at https://github.com/saosebastiao/pgevolve/issues with:
- The exact command you ran.
- The full output (stderr + stdout).
- The relevant slice of
pgevolve.toml. - The Postgres version (
SELECT version();). - Your
pgevolve --version.
Sensitive output? Redact the DSN — pgevolve never prints passwords, but your environment variables might.
Command reference
Per-command details with realistic invocations. The capability spec lists every flag with its implementation status; this file shows them in context.
Global flags
These apply to every subcommand.
| Flag | Effect |
|---|---|
--config <path> | Read config from <path> instead of ./pgevolve.toml. |
| `--format human | json |
-v, -vv | Increase log verbosity (info → debug → trace). Logs go to stderr. |
--quiet | Errors only. |
-h, --help | Per-command help. |
--version | Print the binary version. |
pgevolve init
Scaffolds a new project.
USAGE: pgevolve init [--dir <path>] [--force]
| Flag | Default | Effect |
|---|---|---|
--dir <path> | . | Directory to initialize. |
--force | off | Overwrite an existing pgevolve.toml. |
Creates: pgevolve.toml, schema/, plans/, and adds a .gitignore
section if one doesn't exist yet. Refuses to overwrite an existing
pgevolve.toml unless --force.
pgevolve lint
USAGE: pgevolve lint [--format human|json]
Parses the source tree, runs the universal rules and the configured
layout-profile rules, and prints any findings. Exit 0 on no errors;
exit 1 on any error-severity finding.
The --format flag is the top-level pgevolve --format ... flag and
must precede the subcommand. --format sql is rejected for lint
(only meaningful for diff).
Human format (default)
Layout violations look like:
error: [schema_mirror_path] table should be at `app/tables/users.sql`; found at `schema/oops/users.sql` (schema/oops/users.sql:1:1)
pgevolve lint: 1 finding(s), 1 error(s)
JSON format
pgevolve --format json lint emits a stable structured document:
{
"findings": [
{
"severity": "error",
"rule": "schema_mirror_path",
"message": "table should be at `app/tables/users.sql`; found at `schema/oops/users.sql`",
"location": { "file": "schema/oops/users.sql", "line": 1, "column": 1 }
}
],
"total": 1,
"errors": 1
}
Severity values are stringified ("error", "warning", "lint-at-plan").
Findings without a known source location omit the location field.
Universal rules (e.g., closed_world_references, managed_schemas_match)
are listed in the spec.
pgevolve validate
USAGE: pgevolve validate [--shadow] [--shadow-validate] [--shadow-strict]
Parses the source tree (subsumes lint parse).
| Flag | Effect |
|---|---|
--shadow | Round-trip the IR through an ephemeral Postgres of the version named in [shadow].postgres_version. Requires Docker. |
--shadow-validate | Cross-check the source dep graph against pg_depend in a shadow Postgres. See Shadow validation. |
--shadow-strict | Promote shadow-validation warnings to errors. Requires --shadow-validate. |
Without --shadow the command reports parse success and 0 lint
findings. With --shadow it additionally:
- Starts a
postgres:<major>-alpinecontainer. - Builds a plan from
(empty, source)and applies it. - Introspects the shadow DB into a
Catalog. - Diffs the source IR against the introspected IR.
- Reports any divergences as
Findings on stderr.
Exit 0 on match; 1 on any divergence.
pgevolve diff
USAGE: pgevolve diff --db <env> [--url <dsn>] [--shadow-validate] [--shadow-strict]
Prints the change set between the source IR and a live database.
Always exits 0; this is informational.
| Flag | Effect |
|---|---|
--db <env> | Environment name from [environments.<env>]. |
--url <dsn> | Override the resolved DSN. |
--shadow-validate | Cross-check the source dep graph against pg_depend in a shadow Postgres. See Shadow validation. |
--shadow-strict | Promote shadow-validation warnings to errors. Requires --shadow-validate. |
Output formats (selected with the global --format flag):
human(default) — one-line summary per change, indented details.json— the sameChangeSetserialized.sql— naive ALTER SQL with no online rewrites. For review only; runplanfor the applyable form.
pgevolve diff --db dev
# 1 change(s):
# - AlterTable
# alter table app.users (1 op(s))
pgevolve plan
USAGE: pgevolve plan --db <env> [--url <dsn>] [-o <dir>] [--shadow-validate] [--shadow-strict]
The full pipeline: parse → diff → order → rewrite → group →
Plan::from_grouped → write_plan_dir.
| Flag | Default | Effect |
|---|---|---|
--db <env> | — (required) | Environment to plan against. |
--url <dsn> | — | Override the resolved DSN. |
-o <dir> | <plan_dir>/<YYYY-MM-DD>-<short-id> | Output directory. |
--shadow-validate | off | Cross-check the source dep graph against pg_depend in a shadow Postgres. See Shadow validation. |
--shadow-strict | off | Promote shadow-validation warnings to errors. Requires --shadow-validate. |
plan also enforces LintAtPlan findings: if any unwaived
LintAtPlan-severity finding is present (e.g., column-position drift),
the command exits 2 and writes no plan directory. Acknowledge findings
with [[lint_waiver]] rows in intent.toml — see
configuration.
pgevolve plan --db dev
# Wrote plan abc1234567890123 to plans/2026-05-11-abc1234567890123 (1 group(s), 1 step(s), 0 intent(s))
If diff is empty, plan still writes a directory with zero groups —
useful for asserting "no changes" in CI.
pgevolve apply
USAGE: pgevolve apply <plan-dir> --db <env> [--url <dsn>]
[--allow-different-target] [--allow-drift]
Reads a plan directory and applies it.
| Argument / flag | Effect |
|---|---|
<plan-dir> | Path to a directory previously written by pgevolve plan. |
--db <env> | Environment to apply against. |
--url <dsn> | Override the resolved DSN. |
--allow-different-target | Skip the target-identity match check. Use only when you're intentionally re-targeting (e.g., applying a staging plan to dev for local testing). |
--allow-drift | Skip the drift recheck. Use only when re-applying after intentional out-of-band changes. |
Exit codes (spec §13):
| Code | Cause |
|---|---|
0 | Success |
2 | Pre-flight mismatch (target-identity / drift / unapproved intent) |
3 | Apply error (lock held / step failed) |
1 | Anything else |
Approval flow for destructive plans
When plan produces destructive intents, they're written to
intent.toml with approved = false. apply reads them and refuses
to run until they're flipped. See
plan-format.md.
pgevolve status
USAGE: pgevolve status --db <env> [--url <dsn>] [--apply-id <uuid>] [--limit <n>]
| Flag | Default | Effect |
|---|---|---|
--db <env> | — (required) | |
--url <dsn> | — | |
--apply-id <uuid> | — | Print per-step detail for one specific apply. |
--limit <n> | 10 | Cap on the recent-applies list. |
pgevolve status --db dev
# 3 recent apply/applies:
# <uuid-1> plan=abc1234567890123 status=succeeded started=2026-05-11T18:00:00Z finished=2026-05-11T18:00:03Z
# …
With --format json, emits a serializable shape for automation.
pgevolve bootstrap
USAGE: pgevolve bootstrap --db <env> [--url <dsn>]
Installs or upgrades the pgevolve metadata schema (the
pgevolve.bootstrap_version, apply_log, plan_steps, and lock
tables). Other commands auto-bootstrap, so this is mostly useful for
pre-bootstrapping a fresh DB before the first apply.
pgevolve dump
USAGE: pgevolve dump --db <env> -o <dir>
Introspect a live database and write source-format SQL to <dir>/schema.sql.
| Flag | Default | Effect |
|---|---|---|
--db <env> | — (required) | Environment name from [environments.<env>]. |
--url <dsn> | — | Override the resolved DSN. |
-o, --output <dir> | — (required) | Output directory. Created if it doesn't exist. |
The command:
- Connects to the database using the resolved DSN.
- Reads the catalog for all managed schemas (from
[managed].schemas). - Renders every object as a
CREATEstatement in dependency order: schemas → tables (inline PK/UK/CHECK) → FKALTER TABLE ADD CONSTRAINT→ standalone indexes → sequences. - Writes the result to
<dir>/schema.sql.
pgevolve dump --db dev -o /tmp/schema-snapshot
# wrote 4096 bytes to /tmp/schema-snapshot/schema.sql
# note: output does not include pgevolve directives; add them before running `pgevolve lint`
Scope notes (v0.3.x):
- The entire catalog is written to a single
schema.sqlfile. Multi-file layout followinglayout_profileis not yet implemented. - The output does not include pgevolve source directives
(
-- pgevolve: intent = ...etc.), so it cannot be fed directly topgevolve lintor used withparse_directorywithout first adding those directives. Afterdump, add directives manually or use a futurepgevolve annotatehelper. - Coverage: schemas, tables (with inline PK / UK / CHECK constraints and
FK ALTERs), standalone indexes, sequences, views, and materialized
views are emitted. Functions, procedures, triggers, user-defined
types, extensions, and role-level state (owners, grants, RLS policies,
reloptions) are not yet emitted by
dump.
The primary use case is adoption: pointing dump at an existing production
database to produce a starting schema/ tree for a new pgevolve project.
pgevolve graph
USAGE: pgevolve graph [--graph-format dot|mermaid] [-o <path>] [--plan <dir>]
Render the source dependency graph. Read-only; no database connection required.
| Flag | Default | Effect |
|---|---|---|
--graph-format dot|mermaid | dot | Output format. dot is Graphviz DOT; mermaid is Mermaid flowchart syntax. Note: named --graph-format, not --format, to avoid a clap collision with the global --format flag. |
-o, --out <path> | stdout | Write output to a file instead of stdout. |
--plan <dir> | — | Render the dep graph captured inside an existing plan directory. Not yet implemented — errors with "not yet implemented"; reserved for a future sub-spec. |
# DOT output to stdout (pipe to `dot -Tpng -o deps.png` for a diagram)
pgevolve graph
# Mermaid output to a file
pgevolve graph --graph-format mermaid -o schema/deps.md
Used by the conformance suite's L8 dep-graph golden layer: fixtures assert byte-stable DOT output for a given source tree.
pgevolve doctor
USAGE: pgevolve doctor --db <env> [--url <dsn>]
Project health check. Read-only; does not modify the database or write any files.
| Flag | Effect |
|---|---|
--db <env> | Environment name from [environments.<env>] (required). |
--url <dsn> | Override the resolved DSN. |
Reports:
- Bootstrap status (whether the
pgevolveschema is installed). If not installed, the report tells you to runpgevolve bootstrap. - NOT VALID constraints in managed schemas (candidates for a follow-up
VALIDATE CONSTRAINT). - INVALID indexes in managed schemas (candidates for a follow-up
REINDEX CONCURRENTLY). - Source object count vs. catalog object count (quick sanity check for unexpected drift) — schemas, tables, indexes, sequences.
- Recent failed applies from
pgevolve.apply_log(only when bootstrapped).
pgevolve doctor --db dev
# pgevolve doctor — env dev
# bootstrap: ok
# drift: none
# source: 1 schemas, 4 tables, 3 indexes, 1 sequences
# catalog: 1 schemas, 4 tables, 3 indexes, 1 sequences
# recent applies: no failures
Exit codes:
0— every check passes (bootstrap installed, no drift, no recent apply failures).1— any of: bootstrap missing, NOT VALID constraint, INVALID index, or a failed apply in the recent log. The specific issue is printed in the report; exit code lets the command be scripted into deploy pre-flights.
A pgevolve.apply_log query error is not counted as an issue (the
table may not exist on very old bootstrap versions); the message is
printed but the command still returns 0 for that signal alone.
pgevolve rewrite-table (CLI skeleton)
USAGE: pgevolve rewrite-table <qname> --db <env> --confirm-rewrite
Destructive table rewrite. Not yet implemented — the CLI surface is stable but the command currently errors with a not-yet-implemented message. The implementation lands with the column-reorder sub-spec.
| Argument / flag | Effect |
|---|---|
<qname> | Qualified table name to rewrite (e.g., app.users). |
--db <env> | Environment to operate against (required). |
--confirm-rewrite | Explicit confirmation flag — required to guard against accidental invocation (required). |
The intended use case is column-position reorder: when pgevolve plan
detects column-position drift and you have an approved
[[lint_waiver]] for the relevant column-position-drift finding,
this command performs the shadow-copy table rewrite to materialise the
new column order.
pgevolve cluster (v0.3.0+)
Cluster-level commands manage state shared across an entire Postgres
cluster (today: roles; future: tablespaces, GUCs, foreign servers).
They use a parallel project type — a directory containing
pgevolve-cluster.toml and a roles/ tree — that is separate from a
per-DB pgevolve project. See docs/spec/cluster.md
for the surface and the project shape.
USAGE: pgevolve cluster [--config <path>] <subcommand>
Subcommands:
init [<path>] Scaffold a new cluster project.
diff Show the diff between source roles and the live cluster.
plan Write a cluster plan directory under `cluster-plans/<id>/`.
apply [<id>] Apply a cluster plan. Defaults to the most recent.
status List cluster plans under `cluster-plans/`.
| Flag | Effect |
|---|---|
--config <path> | Path to pgevolve-cluster.toml. Defaults to ./pgevolve-cluster.toml. |
Notes:
- Cluster commands read
pgevolve-cluster.toml, notpgevolve.toml. - The connection DSN comes from
[connection].dsninpgevolve-cluster.toml. The role used must be able to readpg_authid(typically superuser). - The
[bootstrap].roleslist names roles pgevolve treats as PG-owned and never diffs in or out (default["postgres"]; cloud Postgres typically needs additional entries, e.g.["postgres", "cloudsqlsuperuser"]). - Passwords are not stored in source; set them out-of-band.
- v0.3.0 limitations: cluster apply does not yet read
intent.tomlto gate destructive role drops, does not take an advisory lock, and does not write to a per-DB-style apply log. Reviewcluster-plans/<id>/plan.sqlbefore applying. Seedocs/spec/cluster.md.
Shadow validation
--shadow-validate is an optional opt-in cross-check available on
validate, diff, and plan. When set, pgevolve boots a shadow
Postgres (using the [shadow] block in pgevolve.toml) and verifies
the source dependency graph against the pg_depend catalog view.
Discrepancies are reported as warnings; --shadow-strict promotes
them to errors (and requires --shadow-validate).
# Check dep graph consistency; warn on discrepancies
pgevolve validate --shadow-validate
# Same but fail on any discrepancy
pgevolve validate --shadow-validate --shadow-strict
# Shadow-validate during plan; fail if dep graph diverges
pgevolve plan --db dev --shadow-validate --shadow-strict
Shadow backend selection follows [shadow].backend in pgevolve.toml
(auto | testcontainers | dsn). See
configuration.
Configuration reference
Everything in pgevolve.toml. The
capability spec lists what's
implemented vs. planned; this file walks through how to use what's
implemented today.
pgevolve init creates a minimal config; this guide explains each
section.
File location
By default pgevolve loads ./pgevolve.toml. Override with --config <path> on any command.
[project]
[project]
name = "myapp" # informational
schema_dir = "schema" # relative to the config file
plan_dir = "plans" # relative to the config file
layout_profile = "schema-mirror" # one of the built-ins, or a path
| Key | Default | Notes |
|---|---|---|
name | — (required) | Shown in --help and logs. Purely informational. |
schema_dir | "schema" | Where source *.sql lives. Resolved relative to the config file. |
plan_dir | "plans" | Where pgevolve plan writes new plan directories. |
layout_profile | "schema-mirror" | One of schema-mirror, kind-grouped, feature-grouped, free-form, or a path to a *.toml file declaring a custom profile. See the spec. |
[managed]
[managed]
schemas = ["app", "billing"]
ignore_objects = ["app.legacy_etl_*", "billing.audit_*"]
| Key | Default | Notes |
|---|---|---|
schemas | [] | List of schema names pgevolve is responsible for. Anything outside this list is ignored by diff, plan, and apply. An empty list means "no schemas are managed" (lint will not enforce schema match). |
ignore_objects | [] | Qname or glob patterns to exclude even within managed schemas. Useful for legacy tables that aren't yet pgevolve-controlled. |
min_pg_version | 14 | Minimum PG major version the project targets. Accepted values: 14, 15, 16, 17, 18. Gates PG-version-specific source features (e.g., publication row filters need PG 15+). |
The
[managed]filter is the safety net. Even if your source tree declares only one table, an unfiltered apply would emit drops for every catalog object outside your control. The filter prevents that.
[planner]
[planner]
strategy = "online" # "atomic" | "online"
| Key | Default | Notes |
|---|---|---|
strategy | "online" | "atomic" puts everything in one transaction and disables every online rewrite; "online" enables the configured rewrites (next section). |
[planner.online_rewrites]
[planner.online_rewrites]
create_index_concurrent = true
fk_not_valid_then_validate = true
check_not_valid_then_validate = true
not_null_via_check_pattern = true
refresh_mv_concurrently = true
view_drop_create_dependents = true
Each switch defaults to true. Setting any to false disables that
specific rewrite without dropping to atomic mode. Useful for
environments where you want online behavior in general but need to opt
out of one pattern (e.g., a managed-service Postgres that disallows
CONCURRENTLY).
| Switch | Rewrite | Spec |
|---|---|---|
create_index_concurrent | Non-unique CreateIndex on an existing table → CREATE INDEX CONCURRENTLY. Same for DropIndex. | indexes.md |
fk_not_valid_then_validate | ADD FOREIGN KEY on an existing table → ADD ... NOT VALID + VALIDATE CONSTRAINT (two transaction groups). | pipeline.md |
check_not_valid_then_validate | Same shape, for CHECK constraints. | pipeline.md |
not_null_via_check_pattern | SET NOT NULL on a populated column → four-step ADD CHECK NOT VALID / VALIDATE / SET NOT NULL / DROP CHECK. | pipeline.md |
refresh_mv_concurrently | Upgrade REFRESH MATERIALIZED VIEW to REFRESH … CONCURRENTLY when the MV has at least one unique index. Has no effect under strategy = "atomic". | cli.md |
view_drop_create_dependents | Walk the body_dependencies graph and emit explicit DROP + CREATE steps for every view transitively affected by an upstream change. When false, the planner errors instead of cascading dependent-view recreations. | cli.md |
[environments.<name>]
[environments.dev]
url = "postgres://localhost/myapp_dev"
[environments.prod]
url_env = "DATABASE_URL_PROD" # read DSN from env var (recommended)
strategy = "online" # overrides [planner].strategy for --db=prod
[environments.test]
url = "postgres://localhost/myapp_test"
strategy = "atomic" # opt-out for fast / hermetic test DB
| Key | Notes |
|---|---|
url | Explicit DSN. Mutually exclusive with url_env. |
url_env | Name of an environment variable holding the DSN. Read at command time. Mutually exclusive with url. |
strategy | Optional per-environment override of [planner].strategy. |
Omit both url and url_env to fall through to PGEVOLVE_DATABASE_URL
and libpq env vars (PGHOST, PGUSER, etc.).
Connection precedence
For pgevolve <cmd> --db <env>:
--url <dsn>(CLI argument)[environments.<env>].url[environments.<env>].url_envPGEVOLVE_DATABASE_URL- libpq env vars
~/.pgpass
[shadow]
[shadow]
backend = "auto" # auto | testcontainers | dsn
url = "postgres://localhost/myapp_shadow" # for backend = "dsn"
url_env = "PGEVOLVE_SHADOW_URL" # alternative to url
reset = "drop_schema_cascade" # drop_schema_cascade | none
extensions = ["pgcrypto", "uuid-ossp"]
postgres_version = "17" # major version; used to select container or validate DSN
| Key | Default | Notes |
|---|---|---|
backend | "auto" | How to obtain a shadow Postgres. See below. |
url | — | DSN for an existing Postgres to use as shadow. Requires backend = "dsn" or backend = "auto" with a URL set. Mutually exclusive with url_env. |
url_env | — | Name of an environment variable holding the shadow DSN. Alternative to url. |
reset | "drop_schema_cascade" | How to clean the shadow DB between runs. "drop_schema_cascade" drops all schemas under [managed].schemas; "none" leaves the DB as-is (useful for DSN backends where you manage teardown yourself). |
extensions | [] | Extensions to install in the shadow DB before any apply. Names must match [a-zA-Z_][a-zA-Z0-9_-]*. |
postgres_version | "16" | Major version: "14", "15", "16", or "17". Pick the version that matches production. Used to select the container image or to validate a provided DSN. |
backend values
"auto"(default): usesurl/url_envif set; otherwise tries testcontainers if Docker is available; otherwise errors with a helpful message."testcontainers": always uses Docker. Hermetic; requires Docker to be running. Pullspostgres:<major>-alpine."dsn": connects to a user-supplied Postgres. No Docker required. Useful for developers without Docker or for projects with pre-installed extensions (TimescaleDB, PostGIS, etc.).
pgevolve validate --shadow and the --shadow-validate flag on
plan / diff / validate all read this block. Without it those
flags error out with a helpful message.
[[lint_waiver]]
[[lint_waiver]]
rule = "column-position-drift"
target = "app.users"
reason = "applied via separate rewrite-table operation; see PR #234"
[[lint_waiver]] rows acknowledge LintAtPlan-severity findings so
that pgevolve plan doesn't refuse with exit 2. Live in the plan's
intent.toml, not in pgevolve.toml.
| Key | Notes |
|---|---|
rule | Exact rule name of the finding to waive (e.g., "column-position-drift"). Must be non-empty. |
target | Substring matched against the finding's message (typically the qualified object name). Must be non-empty. |
reason | Free-form justification. Shown in --format human output. |
Match semantics: a waiver applies when rule equals the finding's rule
name and target is a substring of the finding's message. A waiver
that matches zero findings is reported as a warning ("unused waiver").
Preflight at apply time validates structural well-formedness: both
rule and target must be non-empty strings. A malformed waiver row
causes preflight to exit 2.
Multiple [[lint_waiver]] rows are supported — use one per finding.
[cluster]
[cluster]
project = "../my-cluster"
Optional. Links this per-DB project to a sibling cluster project (managed
via pgevolve cluster … against a pgevolve-cluster.toml). When set,
cluster-aware lints (e.g. grant-references-unknown-role) cross-check
grantee role names against the linked cluster project's declared roles.
| Key | Notes |
|---|---|
project | Path to the cluster project directory (containing pgevolve-cluster.toml). Relative paths resolve against pgevolve.toml's directory. |
See docs/spec/cluster.md for the cluster surface
and docs/spec/grants.md for the cross-check rules.
Worked example: production-grade config
[project]
name = "ledger"
schema_dir = "schema"
plan_dir = "plans"
layout_profile = "schema-mirror"
[managed]
schemas = ["app", "billing", "audit"]
ignore_objects = ["audit.legacy_*"]
[planner]
strategy = "online"
[planner.online_rewrites]
# Production DB doesn't allow CONCURRENTLY (RDS-style).
create_index_concurrent = false
fk_not_valid_then_validate = true
check_not_valid_then_validate = true
not_null_via_check_pattern = true
[environments.dev]
url = "postgres://localhost/ledger_dev"
strategy = "atomic" # fast local iteration
[environments.staging]
url_env = "DATABASE_URL_STAGING"
[environments.prod]
url_env = "DATABASE_URL_PROD"
[shadow]
backend = "testcontainers"
postgres_version = "16"
CI runs pgevolve validate --shadow on every PR. The dev developer
runs pgevolve plan --db dev, then pgevolve plan --db staging once
the diff stabilizes; the same plan directory is applied to staging and
production after intent-file approval.
Plan format
A plan directory is the unit of code review. This guide explains what each file means and how the executor consumes it.
plans/2026-05-11-abc1234567890123/
├── plan.sql ← canonical artifact (commit this)
├── intent.toml ← destructive approvals (commit this; flip approvals when ready)
└── manifest.toml ← plan id, version metadata, embedded pre-image (commit this)
All three files are plain text. Commit them to the same repo as your
schema/ tree — the plan directory is part of the migration history.
plan.sql
The applyable artifact. Reads cleanly with psql -f plan.sql for
people who want to bypass the executor — pgevolve only relies on the
structured -- @pgevolve directive comments to drive transactions and
audit logging.
Header directives
-- @pgevolve plan id=abc1234567890123 version=0.3.3 ruleset=1 created=2026-05-11T18:42:11Z
-- @pgevolve source_rev=git:c0ffeeabc
-- @pgevolve target=tid-xyz
-- @pgevolve intents_required=2
| Directive | Meaning |
|---|---|
plan id=<16-hex> | Short plan id. Must match intent.toml and manifest.toml. |
version=<x.y.z> | The pgevolve version that produced the plan. |
ruleset=<n> | Planner ruleset version. Bumps mean the rewrites changed. |
created=<rfc3339> | UTC timestamp. |
source_rev=<rev> | Optional source-tree revision (git rev-parse HEAD if you're in a git repo). |
target=<id> | Stable identifier of the database (hash of host/port/dbname/cluster). |
intents_required=<n> | How many destructive intents this plan declares. |
Group and step directives
Each transactional group is wrapped in BEGIN; ... COMMIT;. Each step
has its own directive line:
-- @pgevolve group id=1 transactional=true
BEGIN;
-- @pgevolve step=1 kind=create_table destructive=false targets=app.invoices
CREATE TABLE app.invoices ( ... );
-- @pgevolve step=2 kind=add_constraint_not_valid destructive=false targets=app.invoices.invoices_customer_fk
ALTER TABLE app.invoices ADD CONSTRAINT invoices_customer_fk
FOREIGN KEY (customer_id) REFERENCES app.customers(id) NOT VALID;
COMMIT;
-- @pgevolve group id=2 transactional=false
-- @pgevolve step=3 kind=create_index_concurrent destructive=false targets=app.invoices.invoices_customer_idx
CREATE INDEX CONCURRENTLY invoices_customer_idx ON app.invoices (customer_id);
-- @pgevolve group id=3 transactional=true
BEGIN;
-- @pgevolve step=4 kind=validate_constraint destructive=false targets=app.invoices.invoices_customer_fk
ALTER TABLE app.invoices VALIDATE CONSTRAINT invoices_customer_fk;
-- @pgevolve step=5 kind=drop_column destructive=true intent_id=1 targets=app.users.legacy_email
ALTER TABLE app.users DROP COLUMN legacy_email;
COMMIT;
| Directive field | Meaning |
|---|---|
group id=<n> | 1-indexed group number. |
| `transactional=true | false` |
step=<n> | 1-indexed step number, contiguous across all groups. |
kind=<step_kind> | Same vocabulary as plan::raw_step::StepKind. |
| `destructive=true | false` |
intent_id=<n> | Present only on destructive steps; references the same id in intent.toml. |
targets=<qname1>,<qname2>,... | Comma-separated list of affected qualified names. |
Step kinds — v0.2 additions (views and materialized views)
The following step kinds were added in v0.2 alongside view and materialized view support:
| Kind | SQL emitted | Transactional | Notes |
|---|---|---|---|
create_view | CREATE VIEW <qname> AS <body> | yes | Used for new views and for incompatible body replacements (recreate). |
drop_view | DROP VIEW <qname> | yes | Destructive — requires intent approval. |
create_materialized_view | CREATE MATERIALIZED VIEW <qname> AS <body> | yes | |
drop_materialized_view | DROP MATERIALIZED VIEW <qname> | yes | Destructive — requires intent approval. |
refresh_materialized_view | REFRESH MATERIALIZED VIEW [CONCURRENTLY] <qname> | no (CONCURRENTLY); yes (without) | Upgraded to CONCURRENTLY under online strategy when a unique index is present. |
alter_view_set_reloption | ALTER VIEW <qname> SET (security_barrier = …) | yes | Also handles security_invoker. |
comment_on_view | COMMENT ON VIEW <qname> IS '…' | yes | Used for both regular views and materialized views. |
Step kinds — v0.2 additions (user-defined types)
The following step kinds were added in v0.2 alongside enum, domain, and composite type support:
| Kind | SQL emitted | Transactional | Notes |
|---|---|---|---|
create_type | CREATE TYPE <qname> AS ENUM (…) / CREATE DOMAIN … / CREATE TYPE <qname> AS (…) | yes | One step kind covers all three type families. |
drop_type | DROP TYPE <qname> | yes | Destructive — requires intent approval. |
alter_type_add_value | ALTER TYPE <qname> ADD VALUE '<label>' [BEFORE|AFTER '<ref>'] | yes | |
alter_type_rename_value | ALTER TYPE <qname> RENAME VALUE '<from>' TO '<to>' | yes | |
alter_domain_add_constraint | ALTER DOMAIN <qname> ADD CONSTRAINT <name> CHECK (…) | yes | |
alter_domain_drop_constraint | ALTER DOMAIN <qname> DROP CONSTRAINT <name> | yes | Destructive — requires intent approval. |
alter_domain_set_default | ALTER DOMAIN <qname> SET DEFAULT <expr> / DROP DEFAULT | yes | None default clears the existing default. |
alter_domain_set_not_null | ALTER DOMAIN <qname> SET NOT NULL / DROP NOT NULL | yes | |
alter_type_add_attribute | ALTER TYPE <qname> ADD ATTRIBUTE <name> <type> | yes | |
alter_type_drop_attribute | ALTER TYPE <qname> DROP ATTRIBUTE <name> | yes | Destructive — requires intent approval. |
alter_type_alter_attribute_type | ALTER TYPE <qname> ALTER ATTRIBUTE <name> TYPE <new_type> | yes | |
comment_on_type | COMMENT ON TYPE|DOMAIN <qname> IS '…' | yes | Uses COMMENT ON DOMAIN for domain types; COMMENT ON TYPE for enums and composites. |
ReplaceWithCascade. When a type change cannot be applied in place (e.g., dropping or reordering enum values, reordering composite attributes, or changing a domain's base type), the planner emitsDROP TYPE <qname> CASCADEfollowed byCREATE TYPE. Both steps appear in the plan asdrop_type(destructive — requires intent approval) andcreate_type.
Step kinds — v0.2 additions (functions and procedures)
The following step kinds were added in v0.2 alongside function and procedure support:
| Kind | Description |
|---|---|
create_or_replace_function | Idempotent CREATE OR REPLACE FUNCTION. Covers both initial create and in-place body/attribute changes when the language and return type are compatible. |
drop_function | DROP FUNCTION <qname>(<arg_signature>). Destructive — requires intent approval. |
comment_on_function | COMMENT ON FUNCTION <qname>(<args>) IS '…'. |
create_or_replace_procedure | CREATE OR REPLACE PROCEDURE. If the body contains COMMIT or ROLLBACK, the step runs outside a transaction. |
drop_procedure | DROP PROCEDURE <qname>. Destructive — requires intent approval. |
comment_on_procedure | COMMENT ON PROCEDURE <qname> IS '…'. |
ReplaceWithCascadefor functions. When a function's language or return type changes,CREATE OR REPLACE FUNCTIONis not safe. The planner emitsDROP FUNCTION <qname> CASCADE(destructive — requires intent approval) followed bycreate_or_replace_function. TheCASCADEpropagates to all dependent views and functions automatically.
Step kinds — v0.2 additions (extensions, triggers, partitions)
| Kind | Description |
|---|---|
create_extension | CREATE EXTENSION [IF NOT EXISTS] <name> [WITH SCHEMA <s>] [VERSION '<v>']. |
drop_extension | DROP EXTENSION <name> CASCADE. Destructive — requires intent approval. |
alter_extension_update | ALTER EXTENSION <name> UPDATE TO '<v>'. |
comment_on_extension | COMMENT ON EXTENSION <name> IS '…'. |
create_trigger | CREATE [CONSTRAINT] TRIGGER <name> … ON <table> …. |
drop_trigger | DROP TRIGGER <name> ON <table>. Destructive — requires intent approval. |
comment_on_trigger | COMMENT ON TRIGGER <name> ON <table> IS '…'. |
attach_partition | ALTER TABLE <parent> ATTACH PARTITION <child> FOR VALUES …. |
detach_partition | ALTER TABLE <parent> DETACH PARTITION <child>. |
Step kinds — v0.3 additions (cluster roles)
These step kinds appear only in cluster plans (pgevolve cluster plan) — not in per-DB plans.
| Kind | Description |
|---|---|
create_role | CREATE ROLE <name> WITH <options>. |
drop_role | DROP ROLE <name>. Destructive — requires intent approval (gating not yet enforced; see v0.3.0 limits in docs/spec/cluster.md). |
alter_role | ALTER ROLE <name> WITH <options>. |
grant_role_membership | GRANT <role> TO <member>. |
revoke_role_membership | REVOKE <role> FROM <member>. |
comment_on_role | COMMENT ON ROLE <name> IS '…'. |
Step kinds — v0.3.1 additions (ownership and grants)
| Kind | Description |
|---|---|
alter_object_owner | ALTER <kind> <qname> OWNER TO <new_owner>. |
grant_object_privilege | GRANT <priv> ON <kind> <qname> TO <grantee> [WITH GRANT OPTION]. |
revoke_object_privilege | REVOKE <priv> ON <kind> <qname> FROM <grantee>. |
grant_column_privilege | GRANT <priv> (<col>, …) ON TABLE <qname> TO <grantee> [WITH GRANT OPTION]. |
revoke_column_privilege | REVOKE <priv> (<col>, …) ON TABLE <qname> FROM <grantee>. |
alter_default_privileges | ALTER DEFAULT PRIVILEGES FOR ROLE <x> [IN SCHEMA <y>] GRANT/REVOKE <priv> ON … TO/FROM <z>. |
Step kinds — v0.3.2 additions (row-level security)
| Kind | Description |
|---|---|
create_policy | CREATE POLICY <name> ON <table> …. |
drop_policy | DROP POLICY <name> ON <table>. |
alter_policy | ALTER POLICY <name> ON <table> TO … USING (…) WITH CHECK (…). |
set_table_row_security | ALTER TABLE <qname> { ENABLE | DISABLE } ROW LEVEL SECURITY. |
set_table_force_row_security | ALTER TABLE <qname> { FORCE | NO FORCE } ROW LEVEL SECURITY. |
Step kinds — v0.3.3 additions (storage reloptions)
| Kind | Description |
|---|---|
set_table_storage | ALTER TABLE <qname> SET (fillfactor = …, autovacuum_* = …, …). Batched per object. |
set_index_storage | ALTER INDEX <qname> SET (fillfactor = …, …). |
set_materialized_view_storage | ALTER MATERIALIZED VIEW <qname> SET (fillfactor = …, …). |
intent.toml
plan_id = "abc1234567890123"
[[intent]]
id = 1
step = 5
kind = "drop_column"
target = "app.users.legacy_email"
reason = "drops column legacy_email"
approved = false
Every destructive step gets one [[intent]] row. intent.toml also supports [[step_override]] rows (see below). The executor:
- Reads this file at apply time.
- Refuses to run while any row has
approved = false. - Records the approval state in the audit log.
Approving destructive intents
Open intent.toml, change approved = false to approved = true for
each row you want to allow, and commit the change. The plan id field
must stay intact; pgevolve cross-checks it against plan.sql and
manifest.toml and rejects mismatches.
Approval is intentional friction. A reviewer should be the one flipping
approved = true, not the same person who authored the change. Treat the diff inintent.tomlas the "are you really sure?" gate.
[[step_override]] — suppress or skip steps
intent.toml also accepts [[step_override]] rows. Step overrides let you suppress specific planner steps (e.g., skip a refresh_materialized_view during a maintenance window without regenerating the plan):
[[step_override]]
kind = "refresh_materialized_view"
target = "app.daily_summary"
suppress = true
| Field | Required | Notes |
|---|---|---|
kind | yes | Step kind to match (e.g., refresh_materialized_view, create_view). |
target | yes | Qualified object name the override applies to. |
suppress | no | Default false. When true, the matching step is silently omitted from execution. The plan is otherwise applied normally. |
When to use. Step overrides are appropriate for one-off operational situations (e.g., deferring an expensive
REFRESHto off-peak hours). They are not a substitute for intent approval — destructive steps still requireapproved = trueeven when a[[step_override]]is present.
manifest.toml
plan_id = "abc1234567890123"
plan_hash = "abc1234567890123…" # full 64-char hex
pgevolve_version = "0.3.3"
planner_ruleset_version = 1
source_rev = "git:c0ffeeabc"
target_identity = "tid-xyz"
created_at = "2026-05-11T18:42:11Z"
target_snapshot_json = "..." # embedded pre-image catalog as pretty-printed JSON
| Field | Meaning |
|---|---|
plan_id | Short plan id; matches plan.sql and intent.toml. |
plan_hash | Full 64-char BLAKE3 hex. Recomputable from (source, target, version, ruleset). |
pgevolve_version | The pgevolve build that wrote the plan. |
planner_ruleset_version | Planner ruleset version. |
source_rev | Optional source-tree revision. |
target_identity | Target-DB identity hash. |
created_at | UTC timestamp. |
target_snapshot_json | Embedded pre-image Catalog as JSON. Used at apply time for drift detection — pgevolve compares this against the live state to make sure no out-of-band changes happened since planning. |
Round-trip property
write_plan_dir(&p, dir); read_plan_dir(dir) == p — modulo the
grafted destructive_reason (which lives in intent.toml, not in
plan.sql). The round-trip is property-tested over random catalogs.
What pgevolve does and does NOT touch
| File | pgevolve writes | pgevolve reads at apply |
|---|---|---|
plan.sql | yes (planner) | yes (executor) |
intent.toml | yes (planner) with approved=false | yes (executor reads the flipped values) |
manifest.toml | yes (planner) | yes (executor cross-checks plan id + reads pre-image for drift) |
pgevolve never modifies a plan directory after it's written. Edits to
intent.toml's approval flags are made by users (typically in a code
review). Anything else is suspicious.
Object kinds
Every top-level Postgres object kind pgevolve does, will, or won't
manage. See ../README.md for the status legend.
Tables and schemas — core surface
| Object | Status | Notes |
|---|---|---|
SCHEMA | ✅ Implemented | CREATE / DROP / COMMENT ON. Schemas are listed in [managed].schemas; everything outside the list is ignored by the differ and lint.Tests: tier-1: crates/pgevolve-core/src/ir/schema.rs::tests, parse/builder/create_schema_stmt.rs::tests, diff/schemas.rs::tests; tier-C: failure/parse/duplicate-schema |
TABLE | ✅ Implemented | CREATE / DROP / ALTER for every v0.1 column / constraint operation. See column-types.md and constraints.md for nested capability. Column reorder is detected but not yet applied.Tests: tier-1: crates/pgevolve-core/src/ir/table.rs::tests, parse/builder/create_stmt.rs::tests, diff/tables.rs::tests; tier-C: objects/tables/create-simple, drop-simple, add-column-nullable, comment-on-table |
CREATE TABLE … (LIKE source [INCLUDING …]) | ✅ Implemented | LIKE clauses are expanded into concrete IR during a deferred pass that runs after all tables are assembled. The following INCLUDING options are supported: DEFAULTS, IDENTITY, GENERATED, STORAGE, COMPRESSION, COMMENTS (table and column), CONSTRAINTS (CHECK only; PK/UNIQUE belong to INCLUDING INDEXES), INDEXES (PK/UNIQUE constraints + plain CREATE INDEX indexes with Postgres-faithful ChooseIndexName naming), STATISTICS (extended statistics), and INCLUDING ALL. Chained or interleaved LIKE clauses work in any declaration order; a LIKE cycle or self-reference is a parse error. Using a view or MV as the source is rejected with a clear diagnostic. FOREIGN KEY constraints are never copied (Postgres behaviour). EXCLUDE constraints are not modeled in pgevolve's IR and cannot appear on a LIKE source. CHECK constraint names are copied verbatim, exactly as Postgres does on LIKE — and because unnamed checks are named the Postgres-faithful way ({table}_{col}_check for a column check, {table}_check for a table-level check; see constraints), the clone's check names match the live catalog. Generated index / constraint / statistic names are verified byte-for-byte against live Postgres 14–18.Tests: tier-1: crates/pgevolve-core/src/parse/builder/table_like.rs::tests, parse/builder/choose_name.rs::tests; tier-3: crates/pgevolve-core/tests/table_like_round_trip.rs (live-PG name fidelity); tier-C: objects/tables/create-like-bare, create-like-including-all, create-like-multiple, create-like-interleaved |
INDEX | ✅ Implemented | Six access methods; partial, expression, INCLUDE, NULLS NOT DISTINCT, opclass, collation, tablespace. See indexes.md.Tests: tier-1: crates/pgevolve-core/src/ir/index.rs::tests, parse/builder/index_stmt.rs::tests, diff/indexes.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs |
SEQUENCE | ✅ Implemented | CREATE / DROP / ALTER. OWNED BY modeled. Identity-backing sequences derived from SERIAL / GENERATED AS IDENTITY columns.Tests: tier-1: crates/pgevolve-core/src/ir/sequence.rs::tests, parse/builder/create_seq_stmt.rs::tests, diff/sequences.rs::tests, diff/sequence_op.rs::tests; tier-2: parser/equivalent_pairs/0002-serial-desugar |
COMMENT | ✅ Implemented | On schemas, tables, columns, indexes, sequences, constraints. Tests: tier-1: crates/pgevolve-core/src/parse/builder/comment_stmt.rs::tests; tier-C: objects/tables/comment-on-table, comment-on-column |
Inheritance (INHERITS) | ⛔ Not planned | Declarative partitioning supersedes inheritance for v0.1's target use cases. |
Partitioning
| Feature | Status | Notes |
|---|---|---|
Declarative partitioned table (PARTITION BY) | ✅ Implemented | Range, list, hash partition strategies. partition_by: Option<PartitionBy> on Table. Source Forms 1, 2, and 3 all unified into the same IR.Tests: tier-1: crates/pgevolve-core/src/ir/partition.rs::tests; tier-C: objects/partitions/create-range-parent-and-two-partitions, create-list-parent, create-hash-parent-and-partitions, create-default-partition |
Partition attach / detach (ATTACH PARTITION / DETACH PARTITION) | ✅ Implemented | TableChange::AttachPartition / DetachPartition. Bounds rebound = detach + reattach. DetachPartition is destructive; intent required.Tests: tier-1: crates/pgevolve-core/src/parse/builder/alter_table_attach_partition.rs::tests, plan/rewrite/partitions.rs::tests; tier-C: objects/partitions/attach-existing-standalone, attach-form-vs-declarative-form-equivalent, detach-to-standalone, replace-bounds, add-partition, drop-partition |
| Sub-partitioning | ✅ Implemented | A table may have both partition_by (is a partitioned parent) and partition_of (is a partition child).Tests: tier-C: objects/partitions/subpartitioned |
DETACH PARTITION CONCURRENTLY | ⛔ Not planned | The non-concurrent form is used for now; concurrent detach adds apply-time complexity for minimal benefit. |
| Partition pruning at plan time | 🔮 Future | Plan can skip unaffected partitions when a change touches only the parent. |
Views
| Object | Status | Notes |
|---|---|---|
VIEW | ✅ Implemented | Stored SQL view. NormalizedBody::from_sql canonicalizes the SELECT body on both the source side (T3/T4 parse pass) and the catalog side (T5 catalog reader), so cosmetically-different views diff equal. security_barrier and security_invoker reloptions are modeled.Tests: tier-1: crates/pgevolve-core/src/ir/view.rs::tests, parse/builder/create_view_stmt.rs::tests, parse/normalize_body.rs::tests, diff/views.rs::tests; tier-C: objects/views/create-simple, create-with-aliases, drop, replace-body-compatible, replace-body-incompatible, comment-on-view |
MATERIALIZED VIEW | ✅ Implemented | Physically-stored view. WITH NO DATA initial state honored. REFRESH MATERIALIZED VIEW step kind lands with the planner; upgraded to REFRESH MATERIALIZED VIEW CONCURRENTLY under online strategy when the MV has a unique index (refresh_mv_concurrently = true).Tests: tier-1: crates/pgevolve-core/src/parse/builder/create_materialized_view_stmt.rs::tests, plan/rewrite/refresh_mv_concurrently.rs::tests; tier-C: objects/materialized_views/create-simple, index-on-mv, refresh-concurrently, replace-body, with-no-data-override |
security_barrier reloption | ✅ Implemented | Modeled as View::security_barrier: Option<bool>. Emitted as ALTER VIEW … SET (security_barrier = …) via the alter_view_set_reloption step kind.Tests: tier-C: objects/views/security-barrier-toggle |
security_invoker reloption | ✅ Implemented | Modeled as View::security_invoker: Option<bool>. Same step kind as security_barrier.Tests: tier-C: objects/views/security-invoker-toggle |
CREATE VIEW ... WITH CHECK OPTION | ✅ Supported | Per-view check_option: Option<CheckOption> (Local / Cascaded). Both source forms parsed (SQL clause + WITH-options). Diff emits CREATE OR REPLACE VIEW. change_kinds: [alter_view_set_check_option] |
Recursive views (WITH RECURSIVE) | 📋 Planned, v0.5.3 | Requires cycle-aware dep-graph handling. See roadmap.md. |
Functions, procedures, triggers
| Object | Status | Notes |
|---|---|---|
FUNCTION (SQL language body) | ✅ Implemented | SQL bodies canonicalized via NormalizedBody. CREATE OR REPLACE FUNCTION for in-place changes; signature changes are Drop + Create. Full attribute matrix (volatility, strict, security, parallel, leakproof, cost, rows).Tests: tier-1: crates/pgevolve-core/src/ir/function.rs::tests, parse/builder/create_function_stmt.rs::tests, diff/routines.rs::tests; tier-2: crates/pgevolve-core/tests/functions_round_trip.rs; tier-C: objects/functions/create-sql-simple, replace-body, replace-volatility, replace-return-type-cascade, create-with-overload-pair, create-with-table-return, comment-on-function |
FUNCTION (PL/pgSQL body) | ✅ Implemented | PL/pgSQL bodies parsed via pg_query::parse_plpgsql; static SQL deps extracted; dynamic SQL closed by -- @pgevolve dep: directives.Tests: tier-1: crates/pgevolve-core/src/parse/builder/plpgsql.rs::tests; tier-C: objects/functions/create-plpgsql-simple, function-with-dynamic-sql-directive, create-trigger-function |
FUNCTION (other PL languages — PL/Python, PL/Perl, etc.) | 📋 Planned, v0.4.2 | Requires support for CREATE EXTENSION for the language first. See roadmap.md. |
PROCEDURE | ✅ Implemented | Same as functions, qname-only identity. COMMIT/ROLLBACK in body auto-detected; step runs with transactional=OutsideTransaction. Tests: tier-1: crates/pgevolve-core/src/ir/procedure.rs::tests; tier-C: objects/procedures/create-simple, create-with-commit, replace-body, drop-procedure, comment-on-procedure |
TRIGGER | ✅ Implemented | BEFORE/AFTER/INSTEAD OF; FOR EACH ROW/STATEMENT; WHEN clause; UPDATE OF columns; REFERENCING transition tables; CONSTRAINT TRIGGER with DEFERRABLE/INITIALLY DEFERRED. Any structural diff → Drop + Create. Tests: tier-1: crates/pgevolve-core/src/ir/trigger.rs::tests, parse/builder/create_trigger_stmt.rs::tests, diff/triggers.rs::tests, plan/rewrite/triggers.rs::tests, plan/rewrite/emit/trigger.rs::tests; tier-C: objects/triggers/create-row-trigger-simple, create-statement-trigger, create-instead-of-on-view, create-with-transition-tables, create-constraint-trigger, replace-event-list, replace-function, replace-when-clause, drop-simple, comment-on |
EVENT TRIGGER | ✅ Supported | Database-global object (bare name, no schema). CREATE EVENT TRIGGER … ON <event> with optional WHEN TAG IN (…) command-tag filter; ALTER … ENABLE/DISABLE/ENABLE REPLICA/ENABLE ALWAYS, ALTER … OWNER TO, DROP, and COMMENT ON. Lenient owner + lenient drop, mirroring publications/subscriptions: unmanaged event triggers surface via the unmanaged-event-trigger lint and are never auto-dropped. Extension-owned event triggers are excluded from introspection. change_kinds: [create_event_trigger, drop_event_trigger, alter_event_trigger_enable, alter_event_trigger_owner, comment_on_event_trigger] |
AGGREGATE | ✅ Supported | User-defined aggregates: CREATE AGGREGATE (ordinary form — SFUNC + STYPE + optional FINALFUNC/INITCOND), ALTER … OWNER TO, DROP, and COMMENT ON. State and final functions must be managed SQL/plpgsql functions; source rejects references to unmanaged or built-in functions via the aggregate-references-unmanaged-function lint. The reader skips ordered-set aggregates, moving aggregates, and aggregates whose state function is in an unreadable language. Rename is drop + create; identity is (schema, name, arg_types). change_kinds: [create_aggregate, drop_aggregate, alter_aggregate_owner, comment_on_aggregate] |
Custom types
| Object | Status | Notes |
|---|---|---|
ENUM (CREATE TYPE ... AS ENUM) | ✅ Implemented | ALTER TYPE … ADD VALUE [BEFORE|AFTER], RENAME VALUE. Dropping or reordering values triggers ReplaceWithCascade (DROP TYPE CASCADE + CREATE TYPE).Tests: tier-1: crates/pgevolve-core/src/ir/user_type.rs::tests, parse/builder/create_enum_stmt.rs::tests, diff/types.rs::tests, ir/canon/renumber_enum_sort_orders.rs::tests; tier-2: crates/pgevolve-core/tests/types_round_trip.rs; tier-C: objects/enums/create-simple, add-value-at-end, add-value-before-existing, rename-value, drop-value-cascade-recreate, comment-on-enum |
DOMAIN (CREATE DOMAIN) | ✅ Implemented | NOT NULL, CHECK, default. ALTER DOMAIN ADD/DROP CONSTRAINT, SET/DROP DEFAULT, SET/DROP NOT NULL. Base-type change triggers ReplaceWithCascade.Tests: tier-1: crates/pgevolve-core/src/parse/builder/create_domain_stmt.rs::tests; tier-C: objects/domains/create-simple, create-with-check-and-default, add-check-constraint, set-default, toggle-not-null, comment-on-domain |
COMPOSITE TYPE (CREATE TYPE ... AS (...)) | ✅ Implemented | ADD ATTRIBUTE, DROP ATTRIBUTE, ALTER ATTRIBUTE TYPE. Attribute reordering triggers ReplaceWithCascade.Tests: tier-1: crates/pgevolve-core/src/parse/builder/create_composite_type_stmt.rs::tests; tier-C: objects/composites/create-simple, add-attribute, alter-attribute-type, comment-on-composite |
RANGE TYPE (CREATE TYPE ... AS RANGE) | ✅ Implemented (v0.3.8) | UserTypeKind::Range variant: subtype, subtype_opclass, collation, canonical, subtype_diff, multirange_type_name. Structural changes go through ReplaceWithCascade (PG has no in-place ALTER for these fields). Auto-generated multirange types filtered from pg_type via typtype != 'm'. Dep edges: Range → subtype Type, canonical Function, subtype_diff Function.Tests: tier-1: crates/pgevolve-core/src/ir/user_type.rs::tests, parse/builder/create_stmt.rs::tests, diff/types.rs::tests, plan/rewrite/emit/user_type.rs::tests, plan/edges.rs::tests; tier-C: objects/ranges/create-simple-int4range, create-with-opclass, create-with-subtype-diff-fn, column-with-range-type, drop |
BASE TYPE (CREATE TYPE ... ( INPUT = ..., OUTPUT = ... )) | ⛔ Not planned | Requires C-language functions; out of scope. |
Extensions
| Object | Status | Notes |
|---|---|---|
EXTENSION | ✅ Implemented | Source: CREATE EXTENSION [IF NOT EXISTS] name [WITH SCHEMA s] [VERSION 'v'] in .sql files. Catalog: pg_extension joined with pg_namespace. Differ: Create, Drop (CASCADE; intent required), AlterUpdate, ReplaceWithCascade for schema changes (intent required), CommentOn. Objects installed by extensions (pg_depend.deptype='e') are excluded from every other catalog query.Tests: tier-1: crates/pgevolve-core/src/ir/extension.rs::tests, parse/builder/create_extension_stmt.rs::tests, diff/extensions.rs::tests, plan/rewrite/extensions.rs::tests; tier-C: objects/extensions/create-simple, create-with-schema, create-with-version, drop-simple, replace-schema, comment-on, scenarios/extension-owned-objects-ignored |
Extension version upgrade (ALTER EXTENSION ... UPDATE) | ✅ Implemented | Non-destructive. Emits ALTER EXTENSION foo UPDATE TO 'v'; when source pins a version different from the installed one.Tests: tier-C: objects/extensions/version-pin-noop, version-unpinned-noop, lint-unpinned-warning |
Triggers
Tests (whole Triggers section): tier-1: crates/pgevolve-core/src/ir/trigger.rs::tests, parse/builder/create_trigger_stmt.rs::tests, diff/triggers.rs::tests, plan/rewrite/triggers.rs::tests, plan/rewrite/emit/trigger.rs::tests, lint/rules/trigger_references_unmanaged_function.rs, lint/rules/trigger_references_unmanaged_table.rs; tier-C: every fixture under crates/pgevolve-conformance/tests/cases/objects/triggers/ (12 fixtures) plus objects/triggers/lint-unmanaged-function, lint-unmanaged-table.
IR shape
Trigger is a flat struct in pgevolve-core::ir::trigger:
| Field | Type | Notes |
|---|---|---|
qname | QualifiedName | schema.trigger_name — pgevolve uses the schema of the table, not a separate trigger namespace |
table_name | QualifiedName | Target relation (table, view, or MV) |
function_name | QualifiedName | Trigger function (must return TRIGGER) |
timing | TriggerTiming | Before | After | InsteadOf |
events | Vec<TriggerEvent> | One or more of Insert | Update | Delete | Truncate |
for_each | ForEach | Row | Statement |
when_clause | Option<NormalizedExpr> | WHEN predicate, normalized for canonical comparison |
update_columns | Vec<Identifier> | Column list for UPDATE OF col, …; empty means all columns |
referencing | Option<TransitionTables> | OLD TABLE AS old_tbl / NEW TABLE AS new_tbl names |
constraint | bool | true for CREATE CONSTRAINT TRIGGER |
deferrable | bool | Constraint trigger deferred-ability flag |
initially_deferred | bool | true for INITIALLY DEFERRED; false for INITIALLY IMMEDIATE |
comment | Option<String> | COMMENT ON TRIGGER value |
Catalog::triggers: Vec<Trigger> — flat collection, sorted by (table_name, qname) after canonicalize().
Parser support
CREATE [CONSTRAINT] TRIGGER name timing event [OR event …] ON table [REFERENCING …] [FOR [EACH] {ROW|STATEMENT}] [WHEN (expr)] EXECUTE {FUNCTION|PROCEDURE} fn()— all documented Postgres syntax variants accepted.COMMENT ON TRIGGER name ON table IS '…'— accepted alongsideCOMMENT ON FUNCTIONandCOMMENT ON EXTENSION.ALTER TRIGGERin source files — rejected at statement classification with a structural error. The onlyALTER TRIGGERPostgres exposes is a rename; pgevolve does not support trigger renames.
Catalog reader
Queries pg_trigger joined with pg_class (for the relation name), pg_namespace, and pg_description. Two filters apply:
NOT tgisinternal— excludes system-generated internal triggers (e.g., deferrable constraint enforcement triggers).NOT EXISTS (SELECT 1 FROM pg_depend WHERE objid = tg.oid AND deptype = 'e')— excludes triggers installed by extensions; consistent with thedeptype='e'filter applied to every other catalog query.
Differ
| Scenario | Change variant |
|---|---|
| Trigger present in source, absent in catalog | TriggerChange::Create |
| Trigger absent in source, present in catalog | TriggerChange::Drop (destructive — intent required) |
| Comment-only diff | TriggerChange::CommentOn |
| Any structural diff (timing, events, for-each, function, WHEN clause, UPDATE OF columns, REFERENCING, constraint/deferrable flags) | TriggerChange::Drop + TriggerChange::Create |
There is no ALTER TRIGGER for body-level changes in Postgres; the only path is drop + recreate. CommentOn is always emitted separately when only the comment differs.
Planner steps
| Step kind | Description |
|---|---|
CreateTrigger | CREATE [CONSTRAINT] TRIGGER … |
DropTrigger | DROP TRIGGER name ON table — destructive; gated on intent approval |
CommentOnTrigger | COMMENT ON TRIGGER name ON table IS '…' |
DropTrigger is placed in the same destructive ordering bucket as DropTable and DropFunction. CreateTrigger is placed after the target relation and trigger function are both created/updated.
Dependency edges
| Edge | Meaning |
|---|---|
Trigger → Table / View / MV | Target relation must exist before the trigger is created |
Trigger → Function | Trigger function must exist (and be up-to-date) before the trigger is created |
Both edges are DepSource::Structural. The function edge also ensures that a function change that triggers ReplaceWithCascade (drop + recreate the function) will cascade a drop + recreate of any trigger that references it, in the correct order.
Lint rules
| Rule | Severity | Condition |
|---|---|---|
trigger-references-unmanaged-table | Warning | The trigger's table_name schema is not in [managed].schemas |
trigger-references-unmanaged-function | Warning | The trigger's function_name schema is not in [managed].schemas |
Out of scope / notable gaps
ALTER TRIGGER … RENAME TO— not supported. Rename is treated as Drop + Create (old name disappears, new name appears).- Event triggers (
CREATE EVENT TRIGGER) — a separate object kind, now ✅ Supported. See theEVENT TRIGGERrow in the table above. WHENclause dependency extraction — the WHEN predicate is stored as aNormalizedExprfor canonical diffing but its column references are not added as explicit dep edges. Renames of referenced columns will surface as a structural diff, prompting a Drop + Create.
Partitioning (detail)
Tests (whole Partitioning section): tier-1: crates/pgevolve-core/src/ir/partition.rs::tests, parse/builder/alter_table_attach_partition.rs::tests, plan/rewrite/partitions.rs::tests, lint/rules/partition_references_unmanaged_parent.rs; tier-2: crates/pgevolve-core/src/catalog/queries/partitions.rs, partitioned_tables.rs; tier-C: every fixture under objects/partitions/ (12 fixtures) plus failure/partitions/reject-partition-to-nonpartitioned, reject-rekey.
IR shape
Partitioning is modeled as two optional fields on Table in pgevolve-core::ir::table, backed by types in pgevolve-core::ir::partition:
partition_by: Option<PartitionBy> — present on partitioned-parent tables.
| Field | Type | Notes |
|---|---|---|
strategy | PartitionStrategy | Range | List | Hash |
columns | Vec<PartitionColumn> | Ordered partition key elements |
Each PartitionColumn carries kind: PartitionColumnKind (Column(Identifier) or Expr(NormalizedExpr)), an optional collation: Option<QualifiedName>, and an optional opclass: Option<QualifiedName>.
partition_of: Option<PartitionOf> — present on partition-child tables.
| Field | Type | Notes |
|---|---|---|
parent | QualifiedName | Schema-qualified parent table name |
bounds | PartitionBounds | The FOR VALUES … clause |
PartitionBounds variants:
| Variant | Syntax | Fields |
|---|---|---|
Range { from, to } | FOR VALUES FROM (…) TO (…) | from: Vec<BoundDatum>, to: Vec<BoundDatum> |
List { values } | FOR VALUES IN (…) | values: Vec<BoundDatum> |
Hash { modulus, remainder } | FOR VALUES WITH (MODULUS m, REMAINDER r) | modulus: u32, remainder: u32 |
Default | DEFAULT | — |
BoundDatum — Literal(NormalizedExpr) | MinValue | MaxValue.
A table may have both partition_by and partition_of set simultaneously (sub-partitioning).
Source surface — three syntactic forms
All three forms parse into the same Table IR:
| Form | Source syntax | Notes |
|---|---|---|
| Form 1 (inline) | CREATE TABLE child PARTITION OF parent FOR VALUES … | Parent and child in the same file or directory; child inherits columns from parent. |
| Form 2 (standalone) | CREATE TABLE child PARTITION OF parent FOR VALUES … in a separate file | Identical parse result to Form 1. |
| Form 3 (attach) | Plain CREATE TABLE child (…) + separate ALTER TABLE parent ATTACH PARTITION child FOR VALUES … | The parser merges the ATTACH PARTITION statement into the child's partition_of, producing the same IR as Form 2. A conformance fixture verifies Form 2 and Form 3 generate identical plans. |
Catalog reader
Two catalog queries:
SELECT_PARTITIONED_TABLES—pg_class.relkind = 'p'+pg_get_partkeydef(c.oid). Reads the partition key definition for each partitioned-parent table and re-parses it intoPartitionBy. Filters: NOT extension-owned.SELECT_PARTITIONS—pg_class.relispartition = true+pg_get_expr(c.relpartbound, c.oid). Reads the partition bounds text and re-parses it intoPartitionOf. Joinspg_inheritsto get the parent name. Filters: NOT extension-owned; scoped to managed schemas.
Both queries apply the NOT EXISTS (pg_depend deptype='e') filter consistent with every other catalog query.
Differ
| Scenario | Change variant |
|---|---|
partition_of present in source, absent in catalog | TableChange::AttachPartition { parent, child, bounds } |
partition_of absent in source, present in catalog | TableChange::DetachPartition { parent, child } |
partition_of present on both sides, bounds differ | TableChange::DetachPartition + TableChange::AttachPartition (rebound) |
partition_by present on both sides, strategy or key differs | UnsupportedDiff — no safe in-place rekey path in Postgres |
Either side is a partition (partition_of.is_some()) | Column and constraint diff suppressed — partition children inherit columns from the parent |
Planner steps
| Step kind | Description |
|---|---|
AttachPartition | ALTER TABLE parent ATTACH PARTITION child FOR VALUES … — non-destructive |
DetachPartition | ALTER TABLE parent DETACH PARTITION child — destructive; gated on intent approval |
AttachPartition is ordered in the same post-create bucket as CreateIndex (after the parent and child tables both exist). DetachPartition is ordered in the same destructive bucket as DropTable.
For a CreateTable on a partition child (Form 1 / Form 2 source), the planner emits the CREATE TABLE … PARTITION OF parent FOR VALUES … SQL directly; no separate AttachPartition step is needed. AttachPartition is emitted only when an existing standalone table is being attached to a parent, or when bounds are rebounding.
Dependency edges
| Edge | Meaning |
|---|---|
Table (child partition) → Table (parent) | Parent table must exist before the child partition is created or attached |
The edge is DepSource::Structural. It ensures that when both a parent and a child partition are new, the parent's CreateTable is ordered before the child's.
Lint rules
| Rule | Severity | Condition |
|---|---|---|
partition-references-unmanaged-parent | Error | partition_of.parent schema is not in [managed].schemas |
Out of scope / notable gaps
DETACH PARTITION CONCURRENTLY— not emitted. The non-concurrentDETACH PARTITIONis used, which takes anAccessExclusiveLock. Concurrent detach is listed as ⛔ not planned for now.FOREIGN TABLE PARTITION OF— foreign-table partitions are not modeled. Foreign tables are 🔮 Future.- Per-partition
TABLESPACEand storage parameters — partition bounds + reloptions are modeled (partitions areTablein IR, so they inherit table reloptions automatically). Per-partitionTABLESPACEoverrides are ✅ Supported (see theTABLESPACErow under Storage and physical layout). - Partition pruning at plan time — pgevolve does not skip unaffected partitions when only the parent changes. All managed partitions are included in every diff. Pruning is 🔮 Future.
- Pre-flight partition-overlap detection — pgevolve does not validate that declared bounds are non-overlapping before applying. Postgres enforces this at DDL time; a failed
ATTACH PARTITIONwill surface as an apply error.
Security and roles
| Object | Status | Notes |
|---|---|---|
ROLE (CREATE ROLE / USER) | ✅ Supported | Cluster-level surface (pgevolve cluster …). Full attribute matrix + role membership. Passwords intentionally not modeled — set out-of-band. See cluster.md.Tests: tier-1: crates/pgevolve-core/src/ir/cluster/role.rs::tests, parse/cluster/create_role.rs, parse/cluster/alter_role.rs, diff/cluster.rs::tests; tier-2: crates/pgevolve-core/tests/cluster_parse.rs, cluster_catalog.rs; tier-C: cluster/roles/ (8 fixtures) |
GRANT / REVOKE (object permissions) | ✅ Supported | Per-object grants: Vec<Grant> on all 8 grantable IR types (Schema, Sequence, Table, View, MV, Function, Procedure, UserType). Column-level grants on tables/views/MVs. Lenient drift policy: catalog grants to roles outside source surface as grants-to-unmanaged-role warning, never silently revoked. See grants.md.Tests: tier-1: crates/pgevolve-core/src/ir/grant.rs::tests, diff/grants.rs::tests, diff/default_privileges.rs::tests, diff/owner_op.rs::tests; tier-2: crates/pgevolve-core/tests/catalog_grants.rs; tier-C: objects/grants/ (7 fixtures) |
Row-level security policies (POLICY) | ✅ Supported | Per-table rls_enabled + rls_forced flags + embedded policies: Vec<Policy>. USING / WITH CHECK use NormalizedExpr canon (shared with check constraints). Command-kind changes go through DROP + CREATE. See policies.md.Tests: tier-1: crates/pgevolve-core/src/ir/policy.rs::tests, diff/policies.rs::tests, plan/rewrite/policies.rs::tests; tier-2: crates/pgevolve-core/tests/catalog_policies.rs; tier-C: objects/policies/ (11 fixtures) |
| Security barriers / leakproof flags | 🔮 Future | Less commonly used; lands alongside fine-grained policy review. |
SECURITY LABEL | ⛔ Not planned | Used primarily by SE-Linux integration; out of scope. |
Replication and federation
| Object | Status | Notes |
|---|---|---|
PUBLICATION | ✅ Supported | Logical-replication source-side metadata. All 5 forms (explicit FOR TABLE, FOR ALL TABLES, FOR TABLES IN SCHEMA PG15+, row filters PG15+, column lists PG15+). publish bitset + publish_via_partition_root. Lenient drift via unmanaged-publication. change_kinds: [create, drop, replace, alter_add_table, alter_drop_table, alter_set_table, alter_add_schema, alter_drop_schema, alter_set_publish, alter_set_via_root, comment_on] Tests: tier-1: crates/pgevolve-core/src/ir/publication.rs::tests, parse/builder/publication_stmt.rs, diff/publications.rs; tier-C: objects/publications/ (12 fixtures) |
SUBSCRIPTION | ✅ Supported | Logical-replication subscriber-side metadata. Per-field lenient WITH options (enabled, slot_name, binary, streaming, two_phase, disable_on_error PG15+, password_required PG16+, run_as_owner PG16+, origin PG16+, failover PG17+). CONNECTION supports ${VAR} env-var interpolation resolved at apply preflight; plan.sql stores unresolved placeholders. Lenient drift via unmanaged-subscription; hard-error on plaintext password in source. change_kinds: [create, drop, alter_connection, alter_add_publication, alter_drop_publication, alter_set_options, comment_on]Tests: tier-1: crates/pgevolve-core/src/ir/subscription.rs::tests, parse/builder/subscription_stmt.rs, diff/subscriptions.rs; tier-C: objects/subscriptions/ (12 fixtures) |
FOREIGN DATA WRAPPER (FDW) | 📋 Planned, v0.5.0 | First-class FDW lifecycle (CREATE SERVER, USER MAPPING, IMPORT FOREIGN SCHEMA). See roadmap.md. |
FOREIGN TABLE | 📋 Planned, v0.5.0 | Lands with FDWs. See roadmap.md. |
Storage and physical layout
| Object | Status | Notes |
|---|---|---|
TABLESPACE | ✅ Supported | Cluster-level object (bare name, no schema), managed via the pgevolve cluster … surface. CREATE TABLESPACE (with OWNER, LOCATION, WITH (options)), ALTER … OWNER TO, ALTER … SET (options), DROP (intent-gated), and COMMENT ON. Lenient owner + lenient options. LOCATION is immutable, so a location drift surfaces via the tablespace-location-drift advisory rather than a destructive recreate. Filesystem-layout management (directory creation, mount points) stays out of scope. The IR also carries the tablespace attribute on tables and indexes; per-partition overrides are ✅ Supported (see row below). Tablespaces are declared in a tablespaces/ cluster-source directory. change_kinds: [create_tablespace, drop_tablespace, alter_tablespace_owner, set_tablespace_options, comment_on_tablespace] |
TABLE … TABLESPACE / per-partition TABLESPACE override | ✅ Supported | CREATE TABLE … TABLESPACE <ts> and CREATE TABLE … PARTITION OF … TABLESPACE <ts> on regular tables, partitioned parents, and partition children. ALTER TABLE … SET TABLESPACE is RequiresApproval on a leaf table (full rewrite + ACCESS EXCLUSIVE lock) and Safe on a partitioned parent (metadata-only, no rewrite). pg_default is normalized to the implicit default — declaring TABLESPACE pg_default is a no-op and causes no spurious diff. Per-partition overrides are tracked on the Table.tablespace field and compared independently of the parent. change_kinds: [set_tablespace] |
TABLE ... USING <access method> | ✅ Supported | Per-table access method on CREATE TABLE … USING <am>. Parsed and rendered from the access_method attribute on tables, read from pg_class.relam. The built-in heap is the implicit default, so it is canonicalized to "none" on both source and catalog sides — declaring USING heap is a no-op. Changing the access method on an existing table is not auto-rewritten (a full table rewrite is out of scope and ALTER TABLE … SET ACCESS METHOD is PG 15+); instead the change surfaces via the table-access-method-change advisory. |
WITH (storage_parameter = ...) (table reloptions) | ✅ Supported | Typed fields for fillfactor + parallel_workers + toast_tuple_target + user_catalog_table + vacuum_truncate; autovacuum_* keys (and any unknown/extension keys) live in the untyped extra: BTreeMap<String, String> bag (Postgres validates autovacuum values on apply). Lenient drift policy. See reloptions.md.Tests: tier-1: crates/pgevolve-core/src/parse/builder/reloptions.rs::tests, diff/reloptions.rs::tests; tier-2: crates/pgevolve-core/tests/catalog_reloptions.rs; tier-C: objects/reloptions/table-fillfactor, table-autovacuum-disabled, table-multi-set, alter-table-set-after-create, partition-inherits-reloptions |
| Index reloptions | ✅ Supported | Per-AM validation: B-tree 50..=100 fillfactor, GiST 10..=100, SP-GiST 90..=100, BRIN/GIN no fillfactor; fastupdate (GIN), gin_pending_list_limit (GIN), buffering (GiST), deduplicate_items (B-tree), pages_per_range + autosummarize (BRIN). Tests: tier-C: objects/reloptions/index-fillfactor, index-brin-pages-per-range, index-gin-fastupdate |
| Materialized view reloptions | ✅ Supported | Same key set as tables (autovacuum_*, fillfactor, etc.). Tests: tier-C: objects/reloptions/mv-fillfactor |
Toast options (STORAGE EXTERNAL / EXTENDED / PLAIN / MAIN) | ✅ Supported | Per-column TOAST storage; canon strips type-default. Tests: tier-1: crates/pgevolve-core/src/lint/rules/storage_downgrade_not_retroactive.rs::tests; tier-C: objects/columns/set-storage-external, set-storage-plain-warning, set-storage-type-default-noop, create-table-with-storage |
TOAST compression (COMPRESSION pglz / lz4) | ✅ Supported | Per-column codec; canon preserves None (cluster default_toast_compression GUC).Tests: tier-1: crates/pgevolve-core/src/lint/rules/compression_change_not_retroactive.rs::tests; tier-C: objects/columns/set-compression-lz4 |
Operators, casts, collations, text search
| Object | Status | Notes |
|---|---|---|
OPERATOR / OPERATOR CLASS / OPERATOR FAMILY | 📋 Planned, v0.5.1 | Heavy admin objects; lower priority than user-facing surface. See roadmap.md. |
CAST | ✅ Supported | Managed; WITH FUNCTION / WITHOUT FUNCTION / WITH INOUT; EXPLICIT / ASSIGNMENT / IMPLICIT contexts. CREATE CAST, DROP, and COMMENT ON. WITH FUNCTION is constrained to managed SQL/plpgsql functions — source rejects references to unmanaged or built-in functions via the cast-references-unmanaged-function lint. System casts (function oid < 16384) and extension-owned casts are excluded from introspection. No ALTER CAST in Postgres, so any structural change is drop + create; identity is (source_type, target_type). change_kinds: [create_cast, drop_cast, comment_on_cast] |
COLLATION | ✅ Implemented (v0.3.8) | Per-column collation supported since v0.1; CREATE COLLATION lands as a first-class IR object in v0.3.8: libc / ICU / PG 17+ builtin providers, deterministic toggle, COMMENT, RENAME. Source uses locale = 'X' shorthand or explicit lc_collate + lc_ctype; IR always stores the latter. version field is read-only (ALTER COLLATION … REFRESH VERSION deferred to v0.3.9). Structural changes go through ReplaceCollation (destructive). See collations.md.Tests: tier-1: crates/pgevolve-core/src/ir/collation.rs::tests, parse/builder/create_collation_stmt.rs::tests, diff/collations.rs::tests; tier-C: objects/collations/ (6 fixtures), scenarios/column-references-managed-collation. change_kinds: [create_collation, drop_collation, rename_collation, replace_collation, comment_on_collation, alter] |
TEXT SEARCH DICTIONARY | ✅ Supported | Managed. CREATE TEXT SEARCH DICTIONARY (TEMPLATE reference + OPTIONS list), ALTER … (options), ALTER … OWNER TO (lenient), DROP, and COMMENT ON. A TEMPLATE change reads as drop + create (no in-place ALTER). TEMPLATE is an unmanaged environment reference (C-language function; never auto-created or dropped by pgevolve); unqualified template names resolve to pg_catalog. change_kinds: [create_ts_dictionary, drop_ts_dictionary, alter_ts_dictionary, alter_ts_dictionary_owner, comment_on_ts_dictionary] |
TEXT SEARCH CONFIGURATION | ✅ Supported | Managed. CREATE TEXT SEARCH CONFIGURATION (PARSER reference + token→dictionary MAPPING list), ALTER … ADD MAPPING FOR, ALTER … ALTER MAPPING FOR, ALTER … DROP MAPPING FOR, ALTER … OWNER TO (lenient), DROP, and COMMENT ON. A PARSER change reads as drop + create (no in-place ALTER). PARSER is an unmanaged environment reference (C-language function; never auto-created or dropped by pgevolve); unqualified parser names resolve to pg_catalog. COPY= on CREATE CONFIGURATION is out of scope. Known limitation: a functional index or generated column whose expression calls to_tsvector('schema.config', …) carries an implicit dependency on that text-search configuration that the dep-graph does NOT track (no expression-level TS-config dep edges); such an index may be ordered before its configuration at apply time. The TS objects themselves round-trip correctly; this is a planner gap to address in a future release. change_kinds: [create_ts_configuration, drop_ts_configuration, add_ts_config_mapping, alter_ts_config_mapping, drop_ts_config_mapping, alter_ts_configuration_owner, comment_on_ts_configuration] |
TEXT SEARCH PARSER | ⛔ Not planned | Requires C-language functions; unmanaged environment reference only. |
TEXT SEARCH TEMPLATE | ⛔ Not planned | Requires C-language functions; unmanaged environment reference only. |
Statistics, rules, and other helpers
| Object | Status | Notes |
|---|---|---|
STATISTICS (CREATE STATISTICS) | ✅ Supported | Multi-column statistics objects (ndistinct, dependencies, mcv) + PG14+ expression statistics. Explicit names required (no anonymous form). Granular differ — ALTER SET STATISTICS for target, ReplaceStatistic for any other change. unmanaged-statistic lint. change_kinds: [create_statistic, drop_statistic, replace_statistic, alter_statistic_set_target, comment_on_statistic] |
RULE | ⛔ Not planned | Largely superseded by triggers; pg_query already discourages new rules. |
SERVER (FDW server) | 📋 Planned, v0.5.0 | Lands with FDWs. See roadmap.md. |
USER MAPPING | 📋 Planned, v0.5.0 | Lands with FDWs. See roadmap.md. |
What pgevolve deliberately does not manage
| Object | Status | Reason |
|---|---|---|
DATABASE itself | ⛔ Not planned | Database creation is a cluster-admin step; pgevolve assumes the DB exists. |
TABLESPACE directories | ⛔ Not planned | Filesystem-level setup. |
Cluster-wide settings (postgresql.conf) | ⛔ Not planned | Different lifecycle and audit story. |
| Backups, restores, and physical replication | ⛔ Not planned | Outside the schema-management remit. |
| Data itself (row contents) | ⛔ Not planned | pgevolve plans never INSERT / UPDATE / DELETE. Data migrations are users' responsibility. |
PG 18-only features
These features ship only on Postgres 18+. They are not part of the v0.3.6 PG 18 catalog-support work; each gets its own roadmap entry.
| Feature | Status | Notes |
|---|---|---|
Virtual generated columns (GENERATED ALWAYS AS (...) VIRTUAL) | 📋 Planned, v0.4.1 | New GeneratedKind::Virtual variant alongside the existing stored generated columns. Requires [managed].min_pg_version >= 18. |
NOT NULL NOT VALID constraint variant | 🔮 Future | Allows declaring a NOT NULL constraint without validating existing rows. Useful for large-table migrations. |
Column types
Every Postgres type family with pgevolve's support status. The IR's
ColumnType enum is the source of truth; entries below mirror its
variants and document the gaps.
See ../README.md for the status legend.
Numeric
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests (variant parse + display); tier-2: crates/pgevolve-core/tests/fixtures/parser/equivalent_pairs/0001-int-aliases, 0004-timestamp-tz; tier-C: objects/columns/alter-column-type-widening, alter-column-type-narrowing.
| Type | Status | Notes |
|---|---|---|
boolean | ✅ Implemented | change_kinds: [add, change_type] |
smallint (int2) | ✅ Implemented | change_kinds: [add, change_type] |
integer (int, int4) | ✅ Implemented | change_kinds: [add, change_type] |
bigint (int8) | ✅ Implemented | change_kinds: [add, change_type] |
real (float4) | ✅ Implemented | change_kinds: [add, change_type] |
double precision (float8) | ✅ Implemented | change_kinds: [add, change_type] |
numeric (decimal) | ✅ Implemented | Including precision and scale (numeric(p), numeric(p, s)). Unbounded numeric round-trips. change_kinds: [add, change_type] |
smallserial / serial / bigserial | ✅ Implemented | Desugared at parse time into the underlying integer column + owned sequence; round-trips through introspection by detecting the nextval(...) default.Tests: tier-1: crates/pgevolve-core/src/parse/builder/desugar_serial.rs::tests; tier-2: fixture parser/equivalent_pairs/0002-serial-desugar |
money | ⛔ Not planned | Locale-dependent representation; discouraged by Postgres docs. Use numeric instead. |
Character
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests; tier-2: crates/pgevolve-core/tests/fixtures/parser/equivalent_pairs/0003-varchar-aliases.
| Type | Status | Notes |
|---|---|---|
text | ✅ Implemented | change_kinds: [add, change_type] |
varchar(n) / character varying(n) | ✅ Implemented | change_kinds: [add, change_type] |
varchar (unbounded) | ✅ Implemented | change_kinds: [add, change_type] |
char(n) / character(n) | ✅ Implemented | change_kinds: [add, change_type] |
char (unbounded; single-char) | ✅ Implemented | change_kinds: [add, change_type] |
name | ⛔ Not planned | Internal PG type; you should not be using this in application schemas. |
"char" (single-byte; quoted) | ⛔ Not planned | Internal PG type. |
Binary
Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs.
| Type | Status | Notes |
|---|---|---|
bytea | ✅ Implemented | change_kinds: [add, change_type] |
Date / time
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests; tier-2: crates/pgevolve-core/tests/fixtures/parser/equivalent_pairs/0004-timestamp-tz.
| Type | Status | Notes |
|---|---|---|
date | ✅ Implemented | change_kinds: [add, change_type] |
time | ✅ Implemented | Sub-second precision (time(p)); no time zone. change_kinds: [add, change_type] |
time with time zone (timetz) | ✅ Implemented | Including time(p) with time zone. change_kinds: [add, change_type] |
timestamp | ✅ Implemented | With sub-second precision. change_kinds: [add, change_type] |
timestamp with time zone (timestamptz) | ✅ Implemented | The recommended type for most workflows. change_kinds: [add, change_type] |
interval | ✅ Implemented | Including field constraints (interval year, interval day to hour, etc.) and sub-second precision (interval(N)). pg_query encodes the SQL field qualifier as an INTERVAL_MASK bitmask in the first typmod argument; pgevolve decodes it to the canonical form so the source and catalog paths converge (no spurious ALTER COLUMN TYPE). change_kinds: [add, change_type] |
Networking
Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs.
| Type | Status | Notes |
|---|---|---|
inet | ✅ Implemented | change_kinds: [add, change_type] |
cidr | ✅ Implemented | change_kinds: [add, change_type] |
macaddr | ✅ Implemented | change_kinds: [add, change_type] |
macaddr8 | ✅ Implemented | change_kinds: [add, change_type] |
UUID, JSON, XML
Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs.
| Type | Status | Notes |
|---|---|---|
uuid | ✅ Implemented | change_kinds: [add, change_type] |
json | ✅ Implemented | Lacks a default btree opclass, so the IR generator deliberately avoids indexing json columns. change_kinds: [add, change_type] |
jsonb | ✅ Implemented | change_kinds: [add, change_type] |
jsonpath | 🔮 Future | Rarely used as a column type; mostly an expression-level value. |
xml | 🔮 Future | Requires --with-libxml Postgres build; less common in modern stacks. |
Bit strings
Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests.
| Type | Status | Notes |
|---|---|---|
bit(n) | ✅ Implemented | Fixed-length. change_kinds: [add, change_type] |
bit varying(n) (varbit) | ✅ Implemented | Variable-length. change_kinds: [add, change_type] |
Arrays
Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests.
| Type | Status | Notes |
|---|---|---|
<element>[] (single-dimension) | ✅ Implemented | Element type and dimension count modeled in IR. change_kinds: [add, change_type] |
Multi-dimensional arrays (int[][]) | 🟡 Partial | The IR carries dims: u8 so multi-dimensional declarations parse, but the differ treats all dimensions ≥ 1 identically (Postgres itself does not enforce dimension counts). change_kinds: [add, change_type] |
| Array element constraints | 🔮 Future | E.g., CHECK (array_length(col, 1) = 3). Modeled as a CHECK constraint, not as an array attribute. |
Range and multirange types
| Type | Status | Notes |
|---|---|---|
Built-in range types (int4range, int8range, numrange, tsrange, tstzrange, daterange) | 🔮 Future | Lands with user-defined range types. |
Built-in multirange types (int4multirange, etc.) | 🔮 Future | Lands with range types. |
| User-defined range types | 🔮 Future | Depends on CREATE TYPE ... AS RANGE (see objects.md). |
Geometric
| Type | Status | Notes |
|---|---|---|
point, line, lseg, box, path, polygon, circle | 🔮 Future | Niche; lands when there is concrete user demand. |
Text search
| Type | Status | Notes |
|---|---|---|
tsvector | 🔮 Future | Useful as a STORED generated column; lands with the broader text-search story. |
tsquery | 🔮 Future | Mostly an expression-level type. |
Object identifier types
| Type | Status | Notes |
|---|---|---|
oid, regclass, regtype, regnamespace, etc. | ⛔ Not planned | These are catalog-internal types; user schemas should not depend on them. Catalogs returning regclass are converted to qualified-name strings at introspection time. |
User-defined types
Tests: tier-1: crates/pgevolve-core/src/ir/user_type.rs::tests; tier-C: objects/enums/, objects/domains/, objects/composites/ (see objects.md for fixture list).
| Type | Status | Notes |
|---|---|---|
Enum (CREATE TYPE ... AS ENUM) | 📋 Planned, v0.2 | A column typed as an enum is modeled as ColumnType::UserDefined(qname) today; v0.2 adds first-class enum diff (including ALTER TYPE ... ADD VALUE). |
Composite (CREATE TYPE ... AS (...)) | 📋 Planned, v0.2 | Same as enum: typed columns work today as UserDefined; v0.2 adds first-class composite diff. |
Domain (CREATE DOMAIN) | 📋 Planned, v0.2 | First-class diff including NOT NULL, CHECK, default. |
Range (CREATE TYPE ... AS RANGE) | 🔮 Future | Lands with range columns. |
Base type (CREATE TYPE ... ( INPUT = c_func, OUTPUT = c_func )) | ⛔ Not planned | Requires C-language functions. |
Catch-all fallback
| Variant | Status | Notes |
|---|---|---|
ColumnType::Other { raw: String } | ✅ Implemented | Any type pgevolve doesn't recognize is preserved as a raw string. Two Other types compare equal iff their strings match — pgevolve makes no claim about semantic equivalence. Lets the system parse unknown types without aborting, which is essential for adopting an existing DB. PostGIS: parameterized geometry(Point,4326) / geography(...) are preserved with their typmod, the subtype is normalized to lowercase so the source and format_type catalog forms compare equal, and the schema-qualified form (public.geometry(Point,4326)) is handled too.Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests |
ColumnType::UserDefined(QualifiedName) | ✅ Implemented | Schema-qualified reference to a user-defined type. The IR doesn't introspect the type's structure in v0.1 — that lands with first-class custom types in v0.2. Tests: tier-1: crates/pgevolve-core/src/ir/column_type.rs::tests, user_type.rs::tests |
Type-level attributes
| Attribute | Status | Notes |
|---|---|---|
NOT NULL | ✅ Implemented | Column-level; not a constraint. The SET NOT NULL via CHECK pattern rewrite avoids long locks (see pipeline.md).Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/tests::set_not_null_on_existing_column_emits_four_steps, crates/pgevolve-core/src/diff/columns.rs::tests; tier-2: parser/equivalent_pairs/0007-not-null-via-pk |
DEFAULT <literal> | ✅ Implemented | Booleans, integers, floats, text, bytea, NULL. Tests: tier-1: crates/pgevolve-core/src/ir/default_expr.rs::tests; tier-C: objects/columns/set-default, drop-default |
DEFAULT <sequence> | ✅ Implemented | nextval('seq') recognized; canonicalized at parse + introspect time.Tests: tier-1: crates/pgevolve-core/src/ir/default_expr.rs::tests; tier-C: objects/columns/add-column-with-default |
DEFAULT <expression> | ✅ Implemented | Any other expression preserved as canonical text (lowercased keywords, sorted commutative operands, paren-folded). Tests: tier-1: crates/pgevolve-core/src/parse/normalize_expr.rs::tests; tier-2: parser/equivalent_pairs/0005-default-cast-strip |
COLLATE <collation> | ✅ Implemented | Per-column collation. pg_catalog.default is treated as "no collation" so it doesn't appear as drift.Tests: tier-1: crates/pgevolve-core/src/ir/canon/filter_pg_defaults.rs::tests |
GENERATED ALWAYS AS IDENTITY / GENERATED BY DEFAULT AS IDENTITY | ✅ Implemented | Including sequence option overrides (START, INCREMENT, MINVALUE, MAXVALUE, CACHE, CYCLE).Tests: tier-1: crates/pgevolve-core/src/ir/sequence.rs::tests, parse/builder/create_stmt.rs::tests |
GENERATED ALWAYS AS (expr) STORED (computed columns) | ✅ Implemented | Stored generated columns. Tests: tier-1: crates/pgevolve-core/src/ir/column.rs::tests; tier-C: objects/columns/add-generated-column |
GENERATED ALWAYS AS (expr) VIRTUAL | ⛔ Not planned | Postgres only supports STORED through at least PG 17; this row will move to ✅ Implemented if/when Postgres adds it. |
| Per-column comments | ✅ Implemented | Tests: tier-1: crates/pgevolve-core/src/parse/builder/comment_stmt.rs::tests; tier-C: objects/tables/comment-on-column |
Constraints
Constraint kinds pgevolve models, plus their attributes.
See ../README.md for the status legend.
Kinds
| Kind | Status | Notes |
|---|---|---|
PRIMARY KEY | ✅ Implemented | Single- and multi-column. INCLUDE (covering) columns supported.Tests: tier-1: crates/pgevolve-core/src/ir/constraint.rs::tests; tier-2: parser/equivalent_pairs/0006-pk-inline-vs-table-constraint; tier-C: objects/tables/create-simple |
UNIQUE | ✅ Implemented | Single- and multi-column. INCLUDE and NULLS NOT DISTINCT (PG 15+) supported.Tests: tier-1: crates/pgevolve-core/src/ir/constraint.rs::tests; tier-C: objects/tables/add-constraint-unique |
FOREIGN KEY | ✅ Implemented | Full attribute matrix below. The forward-reference cycle case is broken by the planner's FK-extraction post-pass. Tests: tier-1: crates/pgevolve-core/src/ir/constraint.rs::tests, plan/rewrite/fk_not_valid_validate.rs; tier-C: objects/tables/add-constraint-foreign-key, failure/ast-resolution/fk-to-missing-table |
CHECK | ✅ Implemented | Expression preserved as canonical text; redundant string-literal casts that Postgres's deparser adds ('x'::text) are normalized away so the source matches the introspected catalog. NO INHERIT flag supported.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/check_not_valid_validate.rs; tier-C: objects/tables/add-constraint-check, drop-constraint-check, check-string-literal |
NOT NULL (column-level) | ✅ Implemented | Modeled as Column::nullable rather than as a Constraint. The SET NOT NULL via CHECK pattern rewrite avoids long locks (see pipeline.md).Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/tests::set_not_null_on_existing_column_emits_four_steps |
EXCLUSION (EXCLUDE USING gist (...)) | 🔮 Future | Used primarily with range types; lands alongside range-type column support. |
FOREIGN KEY attributes
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/constraint.rs::tests, parse/builder/create_stmt.rs::tests; tier-C: objects/tables/add-constraint-foreign-key.
| Attribute | Status | Notes |
|---|---|---|
| Local columns + referenced columns | ✅ Implemented | Order significant. change_kinds: [add] |
| `ON UPDATE { NO ACTION | RESTRICT | CASCADE |
| `ON DELETE { NO ACTION | RESTRICT | CASCADE |
SET NULL (col, …) / SET DEFAULT (col, …) (column-restricted action; PG 15+) | ✅ Implemented | change_kinds: [add] |
MATCH SIMPLE (default) | ✅ Implemented | change_kinds: [add] |
MATCH FULL | ✅ Implemented | change_kinds: [add] |
MATCH PARTIAL | ⛔ Not planned | Never implemented by Postgres itself. |
DEFERRABLE / NOT DEFERRABLE, INITIALLY DEFERRED / INITIALLY IMMEDIATE | ✅ Implemented | change_kinds: [add, set_deferrable] |
NOT VALID + VALIDATE CONSTRAINT rewrite for adds on existing tables | ✅ Implemented | See pipeline.md.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/fk_not_valid_validate.rs |
NOT VALID constraints persisted as-is | ⛔ Not planned | The IR represents only fully-validated constraints; the NOT VALID state is an intermediate planner artifact. |
| NOT VALID drift detection and auto-resolution | ✅ Implemented | The catalog reader detects pg_constraint.convalidated = false (from a partial-apply) and the differ emits Change::ValidateConstraint. The planner emits ALTER TABLE ... VALIDATE CONSTRAINT. No user action required. See pipeline.md.Tests: tier-2: crates/pgevolve-core/tests/catalog_drift.rs |
CHECK attributes
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/constraint.rs::tests, plan/rewrite/check_not_valid_validate.rs; tier-C: objects/tables/add-constraint-check, drop-constraint-check.
| Attribute | Status | Notes |
|---|---|---|
| Boolean predicate | ✅ Implemented | Preserved as canonical text. change_kinds: [add] |
NO INHERIT | ✅ Implemented | change_kinds: [add] |
NOT VALID + VALIDATE CONSTRAINT rewrite for adds on existing tables | ✅ Implemented | change_kinds: [validate] |
| Table-level vs. column-level placement | ✅ Implemented | Treated identically at IR level. change_kinds: [add] |
PRIMARY KEY / UNIQUE attributes
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/constraint.rs::tests; tier-2: parser/equivalent_pairs/0006-pk-inline-vs-table-constraint.
| Attribute | Status | Notes |
|---|---|---|
INCLUDE (col, …) covering columns | ✅ Implemented | change_kinds: [add] |
NULLS NOT DISTINCT (UNIQUE only, PG 15+) | ✅ Implemented | change_kinds: [add] |
WITH (storage_parameter = ...) (constraint reloptions) | 🔮 Future | The IR doesn't yet model constraint storage parameters. |
USING INDEX <name> (attach existing index) | ⛔ Not planned | pgevolve always creates the underlying index implicitly; reusing pre-existing indexes is an adoption-path corner case. |
USING INDEX TABLESPACE <name> | 🔮 Future | Lands with broader tablespace modeling. |
Constraint-level features
| Feature | Status | Notes |
|---|---|---|
| Constraint name preserved across diff | ✅ Implemented | Constraints are paired by qname; renaming a constraint registers as drop+add. Tests: tier-1: crates/pgevolve-core/src/diff/constraints.rs::tests |
| Unnamed-constraint auto-naming | ✅ Implemented | An unnamed constraint is auto-named exactly as Postgres's ChooseIndexName/ChooseConstraintName would, so it pairs with the introspected catalog instead of showing a spurious drop+add: {table}_pkey, {table}_{col}_key (UNIQUE), {table}_{col}_check (column CHECK) / {table}_check (table CHECK), with makeObjectName length truncation and collision counters. LIKE copies constraint names verbatim, as Postgres does. Verified byte-for-byte against live PG 14–18.Tests: tier-1: crates/pgevolve-core/src/parse/builder/choose_name.rs::tests, parse/builder/create_stmt.rs::tests; tier-3: crates/pgevolve-core/tests/table_like_round_trip.rs |
| Constraint comments | ✅ Implemented | Tests: tier-1: crates/pgevolve-core/src/parse/builder/comment_stmt.rs::tests |
ALTER TABLE ... VALIDATE CONSTRAINT as an explicit step | ✅ Implemented | Used by the FK / CHECK NOT VALID rewrites. Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/fk_not_valid_validate.rs, check_not_valid_validate.rs; tier-2: crates/pgevolve-core/tests/catalog_drift.rs |
ALTER TABLE ... RENAME CONSTRAINT | 🔮 Future | Today a rename diffs as drop+add (semantically equivalent but a larger lock). |
| Multi-column constraint reordering | ⛔ Not planned | The column order inside a constraint is semantically meaningful; changing it is a drop+add. |
Indexes
Index access methods and per-index options.
See ../README.md for the status legend.
Access methods
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/index.rs::tests, parse/builder/index_stmt.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs (per-method introspection).
| Method | Status | Notes |
|---|---|---|
btree | ✅ Implemented | The default; works on every type with a default opclass. change_kinds: [create, drop] |
hash | ✅ Implemented | change_kinds: [create, drop] |
gin | ✅ Implemented | Useful for jsonb, full-text search, array containment. change_kinds: [create, drop] |
gist | ✅ Implemented | The R-tree-style method; used by geometric types, range types, exclusion constraints. change_kinds: [create, drop] |
brin | ✅ Implemented | Block-range; large append-only tables. change_kinds: [create, drop] |
spgist | ✅ Implemented | Space-partitioned GiST. change_kinds: [create, drop] |
| Custom access methods (e.g., from extensions) | 🔮 Future | Once CREATE EXTENSION lands, the IR can store opaque method names. |
Per-index options
Tests (whole section): tier-1: crates/pgevolve-core/src/ir/index.rs::tests, parse/builder/index_stmt.rs::tests, render/index.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs.
| Option | Status | Notes |
|---|---|---|
| Indexed column or expression | ✅ Implemented | Both column references and (expr) indexes. change_kinds: [create] |
Column sort order (ASC / DESC) | ✅ Implemented | change_kinds: [create] |
NULLS FIRST / NULLS LAST | ✅ Implemented | change_kinds: [create] |
| Per-column collation | ✅ Implemented | change_kinds: [create] |
Per-column operator class (<col> <opclass>) | ✅ Implemented | change_kinds: [create] |
UNIQUE | ✅ Implemented | change_kinds: [create] |
NULLS NOT DISTINCT (PG 15+) | ✅ Implemented | UNIQUE indexes only. change_kinds: [create] |
INCLUDE (col, …) covering columns | ✅ Implemented | change_kinds: [create] |
WHERE <predicate> (partial index) | ✅ Implemented | Predicate preserved as canonical text. change_kinds: [create] |
WITH (storage_parameter = ...) (index reloptions) | 🟡 Partial | The IR doesn't yet model index storage parameters (fillfactor, gin_pending_list_limit, etc.). Planned alongside table reloptions in v0.2.Tests: tier-1: crates/pgevolve-core/src/parse/builder/reloptions.rs::tests, diff/reloptions.rs::tests; tier-C: objects/reloptions/index-fillfactor, index-brin-pages-per-range, index-gin-fastupdate |
TABLESPACE <name> | ✅ Implemented | Stored on the IR; assumes the tablespace exists. change_kinds: [create] |
Index comment (COMMENT ON INDEX) | ✅ Implemented | Tests: tier-1: crates/pgevolve-core/src/parse/builder/comment_stmt.rs::tests |
Online-rewrite rules for indexes
| Rule | Status | Notes |
|---|---|---|
CREATE INDEX on an existing table → CREATE INDEX CONCURRENTLY (non-unique) | ✅ Implemented | Gated by [planner.online_rewrites].create_index_concurrent. Concurrent creates run as their own non-transactional group.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/tests::create_index_on_existing_table_rewrites_to_concurrent, unique_create_index_does_not_rewrite_to_concurrent, atomic_policy_disables_concurrent_index_rewrite |
DROP INDEX on an existing index → DROP INDEX CONCURRENTLY (non-unique) | ✅ Implemented | Same gating. Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/concurrent_index.rs, crates/pgevolve-core/src/plan/rewrite/tests |
CREATE UNIQUE INDEX CONCURRENTLY | ⛔ Not planned (v0.1) | A failed concurrent unique-index build leaves behind an INVALID index that must be cleaned up out-of-band. v0.1 plays it safe and uses the locking variant; an opt-in policy may land in v0.2.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/tests::unique_create_index_does_not_rewrite_to_concurrent |
| INVALID index drift detection and auto-resolution | ✅ Implemented | The catalog reader detects pg_index.indisvalid = false (from a failed CREATE INDEX CONCURRENTLY) and the differ emits Change::RecreateIndex. The planner emits DROP INDEX + CREATE INDEX. No user action required. See pipeline.md.Tests: tier-2: crates/pgevolve-core/tests/catalog_drift.rs |
REINDEX [CONCURRENTLY] | 🔮 Future | Useful as an ops command; not yet a planner step kind. |
Index naming
Tests: tier-1: crates/pgevolve-core/src/parse/builder/index_stmt.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs (constraint-backing index identification).
| Aspect | Status | Notes |
|---|---|---|
Explicit index name (CREATE INDEX <name> ON ...) | ✅ Implemented | The standard case; the IR requires a name. change_kinds: [create] |
| Anonymous index (PG auto-generated name) | ⛔ Not planned | All managed indexes must be named in source. Anonymous indexes from PRIMARY KEY / UNIQUE constraints are tied to those constraints, not standalone indexes. |
| Constraint-backing indexes (PK / UNIQUE auto-created) | ✅ Implemented | Tracked as part of the constraint, not as a separate Index. change_kinds: [create] |
Object grants + ownership + default privileges
pgevolve manages Postgres object permissions declaratively. Every grantable IR object (Schema, Sequence, Table, View, MaterializedView, Function, Procedure, UserType) carries:
owner: Option<Identifier>— opt-in object ownership.grants: Vec<Grant>— per-object ACL entries.
Plus the top-level Catalog.default_privileges: Vec<DefaultPrivilegeRule>
mirroring pg_default_acl.
Source surface
ALTER TABLE app.t OWNER TO app_owner;
GRANT SELECT ON app.t TO readers;
GRANT INSERT (name) ON app.t TO readers;
GRANT EXECUTE ON FUNCTION app.foo(int) TO readers;
ALTER DEFAULT PRIVILEGES FOR ROLE app_owner IN SCHEMA app GRANT SELECT ON TABLES TO readers;
REVOKE statements are rejected in source — revokes come from the diff against the catalog.
Tests: tier-1: crates/pgevolve-core/src/ir/grant.rs::tests, parse/builder/grants.rs, parse/builder/owner_stmt.rs::tests, parse/builder/default_privileges.rs; tier-2: crates/pgevolve-core/tests/catalog_grants.rs; tier-C: objects/grants/owner, objects/grants/table, objects/grants/schema, objects/grants/sequence, objects/grants/function, objects/grants/default-privs.
Drift policy
Catalog grants to roles not declared in source are tolerated and
surface as a grants-to-unmanaged-role warning, never silently
revoked. This protects out-of-band workflows (e.g., temp consultant
grants) while still surfacing the drift so operators can decide.
Tests: tier-1: crates/pgevolve-core/src/lint/rules/grants_to_unmanaged_role.rs::tests, crates/pgevolve-core/src/diff/grants.rs::tests; tier-C: objects/grants/lint.
Cluster-link option
Optional [cluster] block in pgevolve.toml:
[cluster]
project = "../my-cluster"
When set, the grant-references-unknown-role lint cross-checks
every grantee role name (in grants, owners, default-priv targets)
against the linked cluster project's roles/*.sql. Missing role →
Error severity, catching typos pre-apply.
When absent, per-DB independence preserved.
Tests: tier-1: crates/pgevolve-core/src/lint/rules/grant_references_unknown_role.rs::tests.
Ownership semantics
owner: None— unmanaged. The differ ignores ownership for this object; shadow-validate also ignores any catalog-side owner.owner: Some(role)— managed. Diff emitsALTER <KIND> ... OWNER TO rolewhen catalog disagrees.
Source authors opt in per-object by writing an ALTER ... OWNER TO
statement. Omitting it leaves the object unmanaged.
Tests: tier-1: crates/pgevolve-core/src/parse/builder/owner_stmt.rs::tests, crates/pgevolve-core/src/diff/owner_op.rs::tests; tier-C: objects/grants/owner.
Passwords
Passwords are not modeled. v0.3.0's cluster surface already says this for roles; this sub-spec inherits the same stance for object-level permissions.
Lint rules
grants-to-unmanaged-role(warning, waivable) — catalog has grants to roles not in source. Tests: tier-1:crates/pgevolve-core/src/lint/rules/grants_to_unmanaged_role.rs::tests; tier-C:objects/grants/lint.revoke-from-owner(error, non-waivable) — diff would REVOKE from the object's owner. PG silently rejects such REVOKEs; we pre-empt. Tests: tier-1:crates/pgevolve-core/src/lint/rules/revoke_from_owner.rs::tests.grant-references-unknown-role(error, when[cluster].projectis set) — grantee not declared in the linked cluster source. Tests: tier-1:crates/pgevolve-core/src/lint/rules/grant_references_unknown_role.rs::tests.
Out of scope
- DATABASE, TABLESPACE, LANGUAGE, FOREIGN TABLE grants — cluster-level or unmanaged.
- LARGE OBJECT grants — not declarative.
- Row-level security policies — v0.3.2.
Row-level security policies
pgevolve manages Postgres RLS declaratively. Tables carry:
rls_enabled: bool—ALTER TABLE t ENABLE/DISABLE ROW LEVEL SECURITY.rls_forced: bool—ALTER TABLE t FORCE/NO FORCE ROW LEVEL SECURITY(applies even to the table owner).policies: Vec<Policy>— embedded; policies can't exist orphan.
Source surface
CREATE POLICY author_only ON app.docs
AS PERMISSIVE -- (default)
FOR ALL -- (default)
TO public -- (default)
USING (author = current_user);
ALTER TABLE app.docs ENABLE ROW LEVEL SECURITY;
ALTER TABLE app.docs FORCE ROW LEVEL SECURITY;
ALTER POLICY and DROP POLICY are rejected in source — both
come from the diff against the catalog.
Tests: tier-1: crates/pgevolve-core/src/ir/policy.rs::tests, parse/builder/policy_stmt.rs::tests, diff/policies.rs::tests; tier-2: crates/pgevolve-core/tests/catalog_policies.rs; tier-C: objects/policies/simple-permissive-policy, policy-with-check, policy-with-roles, restrictive-policy, enable-rls, disable-rls, force-rls-toggle, drop-policy-on-source-removal.
Command-kind changes recreate
PG's ALTER POLICY can change roles, USING, and WITH CHECK but
NOT the command kind. If source changes a policy from FOR SELECT
to FOR INSERT, pgevolve emits DROP POLICY + CREATE POLICY as
two separate plan steps.
Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/policies.rs::tests; tier-C: objects/policies/alter-policy-command-recreates, alter-policy-roles.
Cross-cluster role validation
Policy TO clauses reference roles. The v0.3.1 cross-cluster lint
grant-references-unknown-role extends to policy roles when
[cluster].project is set in pgevolve.toml.
FORCE without policies = denial
PG's behavior: FORCE ROW LEVEL SECURITY with no policies defined
denies every row, including for the table owner. Almost always a
configuration mistake. The force-rls-without-policies lint warns
on this state.
Tests: tier-1: crates/pgevolve-core/src/lint/rules/force_rls_without_policies.rs::tests; tier-C: objects/policies/lint.
WITH CHECK validity
WITH CHECK is invalid on FOR SELECT and FOR DELETE policies
(PG rejects). The source parser pre-empts with a clear error.
Tests: tier-1: crates/pgevolve-core/src/parse/builder/policy_stmt.rs::tests.
Out of scope
ALTER POLICY ... RENAME TO ...in source — rejected. Operators can drop+create.SECURITY LABEL— not planned.leakproof/security_barrieron views — future.
Storage parameters / reloptions
pgevolve models PG WITH (storage_parameter = …) reloptions on tables,
indexes, and materialized views. Each relkind has a typed *StorageOptions
struct with named fields for the well-known options plus an extra: BTreeMap<String, String> for extension-registered or otherwise-unknown
keys.
Semantics — None always means "unmanaged"
Every typed field is Option<T>. The semantics follow v0.3.1's owner
pattern:
Tests (whole semantics table): tier-1: crates/pgevolve-core/src/diff/reloptions.rs::tests, crates/pgevolve-core/src/ir/reloptions.rs::tests, crates/pgevolve-core/src/ir/canon/reloptions.rs::tests.
| source | catalog | differ action |
|---|---|---|
None | None | no-op |
None | Some(x) | no-op — surface as unmanaged-reloption lint warning |
Some(x) | None | ALTER … SET (key = x); |
Some(x) | Some(x) | no-op |
Some(x) | Some(y) (x ≠ y) | ALTER … SET (key = x); |
Some(x) removed from source → None | Some(x) | no-op (lenient) |
Removing a reloption from source does NOT issue RESET. To clear a
managed reloption:
- Issue
ALTER TABLE t RESET (fillfactor)out-of-band. - On the next plan run, catalog reads
None, source is alsoNone, diff is empty.
This is the same trade-off as v0.3.1's owner: Option<Identifier> —
"unmanaged" must be safe to declare without triggering destructive resets.
Source surface
-- Inline at creation:
CREATE TABLE app.t (id bigint) WITH (fillfactor = 80, autovacuum_enabled = false);
-- ALTER post-creation:
ALTER TABLE app.t SET (parallel_workers = 4);
-- Indexes:
CREATE INDEX i ON app.t (id) WITH (fillfactor = 70);
-- Materialized views:
CREATE MATERIALIZED VIEW m WITH (fillfactor = 90) AS SELECT * FROM ...;
ALTER ... RESET (...) and ALTER ... RESET () are rejected in source.
Tests: tier-1: crates/pgevolve-core/src/parse/builder/reloptions.rs::tests; tier-2: crates/pgevolve-core/tests/catalog_reloptions.rs; tier-C: objects/reloptions/table-fillfactor, table-autovacuum-disabled, table-multi-set, mv-fillfactor, alter-table-set-after-create, index-fillfactor, index-brin-pages-per-range, index-gin-fastupdate, partition-inherits-reloptions.
Per-relkind validation
Tests: tier-1: crates/pgevolve-core/src/parse/builder/reloptions.rs::tests (per-relkind range checks).
Parser enforces PG's documented ranges at parse time:
- Tables / MVs
fillfactor: 10..=100 - B-tree index
fillfactor: 50..=100 - GiST / Hash index
fillfactor: 10..=100 - SP-GiST index
fillfactor: 90..=100 - BRIN / GIN index
fillfactor: not supported →ParseError parallel_workers: 0..=1024toast_tuple_target: 128..=8160pages_per_range(BRIN): 1..=131072- Numeric scale factors: NaN rejected
Supported keys
Tables / Materialized Views
fillfactor, parallel_workers, toast_tuple_target, user_catalog_table,
and vacuum_truncate are typed fields. The autovacuum_* keys (including
log_autovacuum_min_duration) are carried in the untyped extra: BTreeMap<String, String> bag alongside unknown/extension keys — they are not
typed or validated at parse time (Postgres validates autovacuum values on apply).
Indexes
fillfactor, fastupdate (GIN), gin_pending_list_limit (GIN),
buffering (GiST/SP-GiST, values: on/off/auto), deduplicate_items
(B-tree, PG 13+), pages_per_range + autosummarize (BRIN).
Lint
unmanaged-reloption(warning, waivable) — catalog has a typed reloption or extra-bag key not declared in source. Per the lenient drift policy, the differ doesn't RESET; the lint surfaces the drift so operators can decide. Tests: tier-1:crates/pgevolve-core/src/lint/rules/unmanaged_reloption.rs::tests; tier-C:objects/reloptions/lint.
Out of scope
toast.*prefixed options (apply to TOAST tables). Rare; deferred.- Active RESET via source. Operators clear out-of-band.
- Per-partition tablespace overrides. Per-partition reloptions are
supported (partitions are
Tablein IR).
Known limitation: new objects with inline reloptions
Currently CREATE TABLE … WITH (…) / CREATE INDEX … WITH (…) / CREATE MATERIALIZED VIEW … WITH (…) source statements that target a brand-new (not-yet-in-catalog) object emit only the CREATE step without the WITH (…) clause. The reloptions are picked up on the next plan run as unmanaged-reloption warnings or, if source still declares them, as an ALTER … SET (…) step.
Workaround: run pgevolve twice (or apply once, plan again). Convergent in 2 iterations.
This is the same general gap that affects owner/grants/policies/RLS on new objects in v0.3.x — the inline new-object rendering doesn't yet include cross-cutting state. Tracked for a future v0.3.x maintenance release that closes the gap uniformly.
Publications
pgevolve models CREATE PUBLICATION as a first-class declarative IR object.
A publication is a per-database global namespace object (not schema-qualified)
that controls which tables and schemas are replicated via Postgres logical
replication.
Source surface
All five Postgres syntactic forms are supported:
-- Form 1: FOR ALL TABLES — captures every current and future table
CREATE PUBLICATION pub_all FOR ALL TABLES;
-- Form 2: explicit FOR TABLE list (PG 14+)
CREATE PUBLICATION pub_tables
FOR TABLE app.orders, app.items
WITH (publish = 'insert, update, delete');
-- Form 3: FOR TABLES IN SCHEMA (PG 15+; requires min_pg_version = 15)
CREATE PUBLICATION pub_schema
FOR TABLES IN SCHEMA app, billing;
-- Form 4: row filter on a table (PG 15+; requires min_pg_version = 15)
CREATE PUBLICATION pub_filtered
FOR TABLE app.orders WHERE (status = 'active');
-- Form 5: explicit column list (PG 15+; requires min_pg_version = 15)
CREATE PUBLICATION pub_columns
FOR TABLE app.orders (id, status, created_at);
-- Mixed: schemas + tables in one publication (PG 15+)
CREATE PUBLICATION pub_mixed
FOR TABLE app.users, TABLES IN SCHEMA billing
WITH (publish = 'insert, update', publish_via_partition_root = true);
The WITH clause parameters are:
| Parameter | Values | Default |
|---|---|---|
publish | comma-separated list of insert, update, delete, truncate | all four |
publish_via_partition_root | true / false | false |
Tests: tier-1: crates/pgevolve-core/src/parse/builder/publication_stmt.rs; tier-C: objects/publications/.
Semantics — lenient at the publication grain
Unlike field-level leniency (as in reloptions), pgevolve applies leniency at the whole-publication level:
| source | catalog | differ action |
|---|---|---|
| Publication absent | Publication absent | no-op |
| Publication absent | Publication present | no-op — surface as unmanaged-publication lint warning |
| Publication present | Publication absent | CREATE PUBLICATION |
| Publication present, same | Publication present, same | no-op |
| Publication present, scope differs | Publication present, scope differs | granular ALTER steps |
| Publication present, publish/via-root differs | Publication present, differs | ALTER PUBLICATION … SET (publish = …) / SET (publish_via_partition_root = …) |
A publication in source is fully managed (all fields tracked). A publication absent from source is left alone and surfaces via lint. This mirrors the v0.3.1 owner/grants pattern.
Mode-swap: ReplacePublication
When the scope mode changes between AllTables and Selective, no safe
in-place ALTER PUBLICATION path exists in Postgres. pgevolve emits
DROP PUBLICATION + CREATE PUBLICATION (a ReplacePublication logical
operation), which is destructive and requires intent approval.
Tests: tier-C: objects/publications/replace-all-tables-to-selective.
PG-version gating via [managed].min_pg_version
Three source features require Postgres 15+:
FOR TABLES IN SCHEMA(schema-scope)- Row filters (
WHERE (...)on table entries) - Explicit column lists
Using any of these features when [managed].min_pg_version is below 15
(the default is 14) is a lint error (publication-feature-requires-pg-version,
Error severity, not waivable). This surfaces the version dependency at lint
time rather than at apply time with an opaque Postgres syntax error.
# pgevolve.toml
[managed]
min_pg_version = 15
Tests: tier-C: objects/publications/lint-pg-version-schema-scope,
lint-pg-version-row-filter, lint-pg-version-column-list.
Supported publish options
| Field | IR name | PG catalog column | Default |
|---|---|---|---|
insert | PublishKinds::insert | pg_publication.pubinsert | true |
update | PublishKinds::update | pg_publication.pubupdate | true |
delete | PublishKinds::delete | pg_publication.pubdelete | true |
truncate | PublishKinds::truncate | pg_publication.pubtruncate | true |
At least one kind must be enabled; an all-false PublishKinds is rejected at
canon time.
publish_via_partition_root
When true, INSERT/UPDATE/DELETE operations on partition children are
reported using the partition root's row identity (the parent table's replica
identity). This is PG 13+ behavior; pgevolve tracks it as a plain bool on
Publication. No version gate is applied (PG 13 is below our minimum of PG 14).
11 StepKind variants
| Step kind | SQL emitted |
|---|---|
CreatePublication | CREATE PUBLICATION … |
DropPublication | DROP PUBLICATION name (destructive; intent required) |
ReplacePublication | DROP PUBLICATION name + CREATE PUBLICATION … (mode-swap; destructive) |
AlterPublicationAddTable | ALTER PUBLICATION name ADD TABLE … |
AlterPublicationDropTable | ALTER PUBLICATION name DROP TABLE … |
AlterPublicationSetTable | ALTER PUBLICATION name SET TABLE … (full table-list replacement) |
AlterPublicationAddSchema | ALTER PUBLICATION name ADD TABLES IN SCHEMA … (PG 15+) |
AlterPublicationDropSchema | ALTER PUBLICATION name DROP TABLES IN SCHEMA … (PG 15+) |
AlterPublicationSetPublish | ALTER PUBLICATION name SET (publish = '…') |
AlterPublicationSetViaRoot | ALTER PUBLICATION name SET (publish_via_partition_root = …) |
CommentOnPublication | COMMENT ON PUBLICATION name IS '…' |
4 lint rules
| Rule | Severity | Condition | Waivable? |
|---|---|---|---|
unmanaged-publication | Warning | A publication is in the catalog but not in source | Yes |
publication-captures-unmanaged-table | Warning | A Selective publication references a table whose schema is not in [managed].schemas | Yes |
publication-row-filter-references-unmanaged-column | Warning | A row filter references a column not present in the IR for the table (because the table is unmanaged or the column was dropped) | Yes |
publication-feature-requires-pg-version | Error | A PG 15+ feature (schema-scope, row filter, column list) is used but min_pg_version < 15 | No |
Tests: tier-1: crates/pgevolve-core/src/lint/rules/unmanaged_publication.rs::tests,
publication_captures_unmanaged_table.rs::tests,
publication_row_filter_references_unmanaged_column.rs::tests,
publication_feature_requires_pg_version.rs::tests; tier-C: objects/publications/lint-*.
Out of scope
SUBSCRIPTION— consumer side; implemented as of v0.3.5. Seesubscriptions.mdfor the full surface.GRANTon publications — Postgres does not support object-level grants on publications; they have no ACL. Out of scope by PG design.ALTER PUBLICATION … RENAME TO— not supported. Rename is treated as Drop + Create (old name disappears, new name appears).- Replication slots and origins — cluster-level admin objects outside the schema-management remit.
Subscriptions
pgevolve models CREATE SUBSCRIPTION as a first-class declarative IR object.
A subscription is a per-database global namespace object (not schema-qualified)
that controls which publications a database subscribes to via Postgres logical
replication.
Source surface
Three representative CREATE forms are supported:
-- Form 1: minimal — subscribe to a single publication
CREATE SUBSCRIPTION sub_main
CONNECTION 'host=replica.example.com dbname=app user=repl password=${REPL_PWD}'
PUBLICATION pub_all;
-- Form 2: subscribe to multiple publications with WITH options
CREATE SUBSCRIPTION sub_filtered
CONNECTION 'host=replica.example.com dbname=app user=repl password=${REPL_PWD}'
PUBLICATION pub_orders, pub_users
WITH (binary = true, streaming = on, two_phase = false);
-- Form 3: all currently-supported options
CREATE SUBSCRIPTION sub_full
CONNECTION 'host=replica.example.com dbname=app user=repl password=${REPL_PWD}'
PUBLICATION pub_all
WITH (
enabled = true,
slot_name = 'myslot',
binary = false,
streaming = parallel, -- PG 16+
two_phase = false,
disable_on_error = true, -- PG 15+
origin = any -- PG 16+
);
Operational verb forms are rejected at parse time (see below).
${VAR} env-var interpolation in CONNECTION strings
The CONNECTION string almost always contains credentials that must not be
stored in source control. pgevolve supports ${VAR} placeholder syntax
anywhere inside the connection string. The literal ${VAR} tokens are:
- Stored verbatim in the source IR and in
plan.sql. - Never logged, persisted, or echoed during plan generation.
- Resolved at apply-time preflight: before pgevolve opens any database
connection, it scans every step's SQL for
${...}references, resolves each against the process environment (std::env::var), and fails with a clear error if any reference is unset.
Example workflow:
# Set the credential in the shell before applying — never commit it
export REPL_PWD="$(vault kv get -field=password secret/repl)"
pgevolve apply plans/2026-05-26-abc1234567890123
If REPL_PWD is not set, pgevolve refuses to start the apply and prints:
error: unresolved env-var reference ${REPL_PWD} in step 1 (create_subscription)
Do not use $$-quoting or single-quote escapes inside the password value —
the substitution is literal string replacement before the SQL is sent to
tokio-postgres. The connection string itself is a libpq DSN, not SQL.
Per-field lenient WITH options
Every field is Option<T>. None means "unmanaged" — pgevolve neither sets
nor resets the option. Some(value) means "managed" — the differ emits an
ALTER to converge the live subscription.
| Option | IR field | Postgres default | PG version | Notes |
|---|---|---|---|---|
enabled | enabled: Option<bool> | true | 14+ | Whether the subscription is running |
slot_name | slot_name: Option<Identifier> | subscription name | 14+ | Publisher-side slot name |
binary | binary: Option<bool> | false | 14+ | Binary copy / binary replication protocol |
streaming | streaming: Option<StreamingMode> | off | 14+ | off / on / parallel (parallel is PG 16+) |
two_phase | two_phase: Option<bool> | false | 14+ | Two-phase commit handling |
disable_on_error | disable_on_error: Option<bool> | false | 15+ | Disable subscription on apply error |
password_required | password_required: Option<bool> | true | 16+ | Subscription owner must supply a password |
run_as_owner | run_as_owner: Option<bool> | false | 16+ | Run apply worker as subscription owner |
origin | origin: Option<OriginMode> | any | 16+ | any / none — replicate only non-replicated sources |
failover | failover: Option<bool> | false | 17+ | Subscription survives failover |
CREATE-only fields (create_slot, copy_data): these are accepted in
source CREATE statements (so users can declare them) but the differ never
includes them in AlterSubscriptionSetOptions deltas — pg_subscription
does not store the CREATE-time decision, so there is nothing to diff against.
Diff-modulo-password behavior
The connection field on a Subscription stores the raw connection string
verbatim, including unresolved ${VAR} placeholders. The differ compares
connection strings as opaque strings: if the source string differs from the
catalog-read string, an alter_connection step is emitted.
Because pg_subscription.subconninfo stores the live connection string
(with the resolved password at subscription-create time), the catalog reader
replaces any password=<value> segment with password=${__PGEVOLVE_REDACTED}
before diff comparison. This prevents a spurious alter_connection step
every plan cycle due to the round-trip asymmetry between ${VAR} in source
and a real password in pg_subscription.
Supported lint rules
| Rule | Severity | Condition | Waivable? |
|---|---|---|---|
unmanaged-subscription | Warning | A subscription is in the catalog but not in source | Yes |
subscription-references-undeclared-publication | Warning | A subscription lists a publication not declared in source (may still work, but pgevolve cannot track it) | Yes |
subscription-feature-requires-pg-version | Error | A PG-version-gated option (disable_on_error PG15+, password_required/run_as_owner/origin PG16+, failover PG17+, streaming = parallel PG16+) is used but [managed].min_pg_version is below the minimum | No |
subscription-password-in-source | Error | The CONNECTION string contains a literal password= value (not a ${VAR} reference) | No |
The last two rules are not waivable — they catch security and compatibility problems that would surface at apply time or in a security audit.
Tests: tier-1: crates/pgevolve-core/src/lint/rules/unmanaged_subscription.rs::tests,
subscription_references_undeclared_publication.rs::tests,
subscription_feature_requires_pg_version.rs::tests,
subscription_password_in_source.rs::tests; tier-C: objects/subscriptions/lint-*.
Operational verb rejection
The following operational SQL forms are rejected at parse time with a clear error message:
ALTER SUBSCRIPTION s REFRESH PUBLICATION; -- rejected
ALTER SUBSCRIPTION s SKIP (lsn = '0/12345678'); -- rejected
ALTER SUBSCRIPTION s ENABLE; -- rejected (use WITH (enabled = true))
ALTER SUBSCRIPTION s DISABLE; -- rejected (use WITH (enabled = false))
These are point-in-time operations, not declarative state. pgevolve only accepts the source-state-expressing forms that can be diffed and re-applied safely:
ALTER SUBSCRIPTION s CONNECTION '...';
ALTER SUBSCRIPTION s ADD PUBLICATION pub_b;
ALTER SUBSCRIPTION s DROP PUBLICATION pub_a;
ALTER SUBSCRIPTION s SET PUBLICATION pub_b;
ALTER SUBSCRIPTION s SET (binary = true);
ALTER SUBSCRIPTION s OWNER TO new_owner;
pg_subscription superuser restriction
Postgres restricts pg_subscription catalog access to superusers.
pgevolve's catalog reader therefore requires superuser (or pg_monitor)
privileges to read existing subscriptions. If the plan user lacks these
privileges, subscriptions are treated as absent from the catalog (the differ
emits a create_subscription step). Apply with a superuser role or grant
pg_monitor to the plan user.
Conformance fixtures
12 fixtures under crates/pgevolve-conformance/tests/cases/objects/subscriptions/.
All fixtures carry [fixture] apply = false because subscriptions require
a publisher cluster at apply time and the conformance harness targets a
single ephemeral Postgres instance. The fixtures validate parse, diff, plan,
and lint without attempting to apply.
Tests: tier-C: objects/subscriptions/create-minimal,
create-with-options, create-multi-publication, drop, alter-connection,
alter-add-publication, alter-drop-publication, alter-set-options,
alter-enabled-disable, comment-on, lint-password-in-source,
lint-pg-version-gating.
8 StepKind variants
| Step kind | SQL emitted |
|---|---|
CreateSubscription | CREATE SUBSCRIPTION … |
DropSubscription | DROP SUBSCRIPTION name (destructive; intent required) |
AlterSubscriptionConnection | ALTER SUBSCRIPTION name CONNECTION '…' |
AlterSubscriptionAddPublication | ALTER SUBSCRIPTION name ADD PUBLICATION … |
AlterSubscriptionDropPublication | ALTER SUBSCRIPTION name DROP PUBLICATION … |
AlterSubscriptionSetPublication | ALTER SUBSCRIPTION name SET PUBLICATION … (full replacement) |
AlterSubscriptionSetOptions | ALTER SUBSCRIPTION name SET (key = value, …) |
CommentOnSubscription | COMMENT ON SUBSCRIPTION name IS '…' |
Out of scope
ALTER SUBSCRIPTION … RENAME TO— not supported. Rename is treated as Drop + Create (old name disappears, new name appears).ALTER SUBSCRIPTION … REFRESH PUBLICATION— operational verb; rejected in source. Run out-of-band when needed.ALTER SUBSCRIPTION … SKIP (lsn = …)— point-in-time skip of a replication conflict; not a declarative property. Run out-of-band.- Subscription statistics (
pg_stat_subscription, worker tables) — runtime telemetry, not schema management state. - Replication slots — cluster-level admin objects; see
docs/spec/cluster.mdfor the cluster surface.
Statistics
pgevolve models CREATE STATISTICS as a first-class declarative IR object.
A statistic is a schema-qualified object (pg_statistic_ext) that captures
cross-column correlations for the Postgres query planner.
Source surface
Four syntactic forms are supported:
-- Form 1: basic — all three kinds enabled (pg default)
CREATE STATISTICS app.orders_s ON (status, region) FROM app.orders;
-- Form 2: explicit kinds — only the requested kinds
CREATE STATISTICS app.orders_dep ON (status, region) FROM app.orders
(dependencies);
-- Form 3: expression statistic (PG 14+)
CREATE STATISTICS app.orders_expr ON (lower(region), status) FROM app.orders;
-- Form 4: mixed columns + expression, explicit kinds
CREATE STATISTICS app.orders_mixed ON (status, lower(region)) FROM app.orders
(ndistinct, mcv);
The kinds clause accepts any non-empty subset of:
ndistinct— multi-column distinct-value countsdependencies— functional dependencies between columnsmcv— most-common-value lists per column combination
Omitting the kinds clause enables all three (Postgres default).
Explicit names required. The anonymous form (CREATE STATISTICS ON (...) FROM t)
is rejected at parse time, mirroring the no-anonymous-indexes policy. Every
statistic managed by pgevolve must carry a schema-qualified name.
Semantics — lenient at the statistic grain
pgevolve applies leniency at the whole-statistic level:
| source | catalog | differ action |
|---|---|---|
| Statistic absent | Statistic absent | no-op |
| Statistic absent | Statistic present | no-op — surfaced as unmanaged-statistic lint warning |
| Statistic present | Statistic absent | CREATE STATISTICS |
| Statistic present, same | Statistic present, same | no-op |
Statistic present, statistics_target differs | Statistic present, differs | ALTER STATISTICS … SET STATISTICS n (granular cheap path) |
| Statistic present, any other field differs | Statistic present, differs | DROP STATISTICS + CREATE STATISTICS (ReplaceStatistic) |
A statistic present in source is fully managed. A statistic absent from source is left alone and surfaced via lint.
Granular differ — two paths
Postgres has no ALTER STATISTICS for column lists or kinds (it would require
rebuilding the statistics object anyway). pgevolve therefore emits:
AlterStatisticSetTarget(ALTER STATISTICS name SET STATISTICS n) when onlystatistics_targetchanged. This is cheap and non-destructive.ReplaceStatistic(DROP STATISTICS+CREATE STATISTICS) for any other structural change (columns, kinds, target table rename). This is destructive and requires intent approval.
5 StepKind variants
| Step kind | SQL emitted |
|---|---|
CreateStatistic | CREATE STATISTICS … |
DropStatistic | DROP STATISTICS name (destructive; intent required) |
ReplaceStatistic | DROP STATISTICS name + CREATE STATISTICS … (structural change; destructive) |
AlterStatisticSetTarget | ALTER STATISTICS name SET STATISTICS n |
CommentOnStatistic | COMMENT ON STATISTICS name IS '…' |
1 lint rule
| Rule | Severity | Condition | Waivable? |
|---|---|---|---|
unmanaged-statistic | Warning | A statistic is in the catalog but not in source | Yes |
Tests: tier-1: crates/pgevolve-core/src/lint/rules/unmanaged_statistic.rs::tests;
tier-C: objects/statistics/lint-unmanaged.
Out of scope
- Anonymous form (
CREATE STATISTICS ON (...) FROM t) — rejected at parse time. Explicit names are required so pgevolve can track identity across migrations. INCLUDEclause (PG 18+) — not yet modeled. Deferred to a future patch.ALTER STATISTICS … RENAME TO— not supported. Rename is treated as Drop + Create (old name disappears, new name appears).GRANTon statistics — Postgres does not support object-level grants onpg_statistic_extobjects. Out of scope by PG design.
Catalog reader
Statistics are read from pg_statistic_ext joined with pg_namespace:
stxname,stxnamespace— name + schema.stxrelid— target table OID (resolved toQualifiedNameviapg_class).stxkind— char[] with'd'(dependencies),'f'(ndistinct),'m'(mcv).stxkeys— int2vector of column attribute numbers (resolved toIdentifierviapg_attribute).- Expression statistics (PG 14+):
pg_get_statisticsobjdef_expressions(oid)returns a text[] of expression SQL; each entry is parsed + canonicalized viaNormalizedExpr. stxstattarget— thestatistics_targetoverride;-1maps toNone.pg_description—COMMENT ON STATISTICS.
Tests: tier-2: crates/pgevolve-core/tests/statistics_round_trip.rs;
tier-C: objects/statistics/.
Conformance fixtures
9 fixtures under crates/pgevolve-conformance/tests/cases/objects/statistics/:
| Fixture | Covers |
|---|---|
create-simple | Basic two-column, all-kinds statistic |
create-explicit-kinds | (ndistinct) only |
create-expression | PG 14+ expression column |
alter-set-target | AlterStatisticSetTarget cheap path |
replace-columns | Structural change → ReplaceStatistic |
replace-kinds | Kinds change → ReplaceStatistic |
drop-simple | DropStatistic |
comment-on | COMMENT ON STATISTICS |
lint-unmanaged | unmanaged-statistic warning |
Collations
pgevolve models CREATE COLLATION as a first-class declarative IR object.
A collation is a schema-qualified object (pg_collation) that names a
locale-data provider plus the lc_collate / lc_ctype strings used by
sort and ctype-aware comparisons. Per-column COLLATE clauses (always
supported) reference collations by qname; v0.3.8 adds the ability to
manage the collation objects themselves.
Source surface
Three syntactic forms are supported, mirroring Postgres:
-- Form 1: libc + locale shorthand (default provider).
CREATE COLLATION app.de_DE (locale = 'de_DE.utf8');
-- Form 2: explicit provider + locale shorthand.
CREATE COLLATION app.case_insensitive
(provider = icu, locale = 'und', deterministic = false);
-- Form 3: explicit lc_collate + lc_ctype.
CREATE COLLATION app.mixed
(provider = libc, lc_collate = 'C', lc_ctype = 'en_US.utf8');
The IR always stores lc_collate + lc_ctype separately. When the source
used locale = 'X', the parser normalizes to lc_collate = 'X' +
lc_ctype = 'X'. The renderer collapses back to the locale = '...'
shorthand when the two are equal, so the round-trip is lossless and
canonical.
Tests: tier-1: crates/pgevolve-core/src/parse/builder/create_collation_stmt.rs;
tier-C: objects/collations/.
IR shape
Collation is a flat struct in pgevolve-core::ir::collation:
| Field | Type | Notes |
|---|---|---|
qname | QualifiedName | schema.collation_name |
provider | CollationProvider | Libc | Icu | Builtin (PG 17+) |
lc_collate | String | From pg_collation.collcollate |
lc_ctype | String | From pg_collation.collctype |
deterministic | bool | Default true. PG 12+; ICU only when false |
version | Option<String> | Read-only pg_collation.collversion. Differ ignores; ALTER COLLATION … REFRESH VERSION deferred to v0.3.9 |
owner | Option<Identifier> | Lenient: None = unmanaged, Some(role) = differ emits ALTER COLLATION … OWNER TO |
comment | Option<String> | COMMENT ON COLLATION qname IS '…' |
Catalog::collations: Vec<Collation> — flat collection, sorted by
qname after canonicalize(). The canon pass also rejects the invalid
libc + nondeterministic combination (Postgres would reject at runtime;
pgevolve surfaces it at canon time with a clearer error).
BUILTIN_COLLATIONS: &[&str] — default, C, POSIX, und-x-icu,
unicode, ucs_basic. These shortnames bypass the
column-references-unmanaged-collation lint even when they have no
schema qualifier, because Postgres seeds them at initdb and they are
always available.
Semantics — lenient at the collation grain
pgevolve applies leniency at the whole-collation level (consistent with publications / subscriptions / statistics):
| source | catalog | differ action |
|---|---|---|
| Collation absent | Collation absent | no-op |
| Collation absent | Collation present | no-op — surfaced as unmanaged-collation lint warning |
| Collation present | Collation absent | CREATE COLLATION |
| Collation present, same | Collation present, same | no-op |
| Collation present, comment differs | Collation present, differs | COMMENT ON COLLATION |
| Collation present, owner differs | Collation present, differs | ALTER COLLATION … OWNER TO |
| Collation present, structural change (provider / lc_collate / lc_ctype / deterministic) | Collation present, differs | DROP COLLATION + CREATE COLLATION (ReplaceCollation; destructive) |
Postgres has no in-place ALTER COLLATION for provider / locale /
deterministic, so any structural change emits ReplaceCollation, which
is destructive and requires intent approval.
5 StepKind variants
| Step kind | SQL emitted |
|---|---|
CreateCollation | CREATE COLLATION qname (provider = …, locale = …, deterministic = …) |
DropCollation | DROP COLLATION qname (destructive; intent required) |
RenameCollation | ALTER COLLATION qname RENAME TO new_name |
ReplaceCollation | DROP COLLATION + CREATE COLLATION (structural change; destructive) |
CommentOnCollation | COMMENT ON COLLATION qname IS '…' |
ALTER COLLATION … OWNER TO is emitted via the standard
Change::AlterObjectOwner path with OwnedObjectId::Qualified, shared
with every other ownable IR kind.
5 lint rules
| Rule | Severity | Condition | Waivable? |
|---|---|---|---|
unmanaged-collation | Warning | A collation is in the catalog but not in source | Yes |
column-references-unmanaged-collation | Warning | A column's collation references a collation outside [managed].schemas and not in BUILTIN_COLLATIONS | Yes |
range-type-references-unmanaged-subtype | Warning | A Range user type's subtype references a user type outside [managed].schemas and not a pg_catalog built-in | Yes |
nondeterministic-collation-requires-pg-12 | Error | A Collation has deterministic = false but [managed].min_pg_version < 12 | No |
builtin-provider-requires-pg-17 | Error | A Collation uses CollationProvider::Builtin but [managed].min_pg_version < 17 | No |
Tests: tier-1:
crates/pgevolve-core/src/lint/rules/unmanaged_collation.rs::tests,
column_references_unmanaged_collation.rs::tests,
range_type_references_unmanaged_subtype.rs::tests,
nondeterministic_collation_requires_pg_12.rs::tests,
builtin_provider_requires_pg_17.rs::tests;
tier-C: objects/collations/.
Conformance fixture pointers
7 fixtures cover the collation surface (+1 cross-cutting scenario):
| Fixture | Covers |
|---|---|
objects/collations/create-libc | Basic libc + locale shorthand |
objects/collations/create-icu | ICU provider + locale shorthand |
objects/collations/create-nondeterministic | ICU + deterministic = false |
objects/collations/drop | DropCollation |
objects/collations/comment-on | CommentOnCollation |
objects/collations/replace-on-locale-change | Structural change → ReplaceCollation |
scenarios/column-references-managed-collation | Cross-cutting: column COLLATE references a managed Collation |
Dependency edges
| Edge | Meaning |
|---|---|
Table → Collation | A column with collation = Some(qname) adds an edge so the collation is created before the table that uses it |
The edge is DepSource::Structural. Range → Collation is also added
when a UserTypeKind::Range carries a collation: Some(qname).
Property tests (v0.3.8)
crates/pgevolve-testkit/src/ir_generator/collation.rs generates 0–2
libc collations per managed schema with a deterministic-only, safe-locale
pool. ICU + nondeterministic + builtin variants are deliberately excluded
from the proptest soak to avoid PG-version gating; the conformance
fixtures cover those paths instead.
Out of scope (deferred to v0.3.9+)
CREATE COLLATION FROM existing_collation— clone syntax. Round-trip identity is ambiguous; the source declares a clone but the catalog shows the resolved provider / locale. Deferred. See the design doc.ALTER COLLATION … REFRESH VERSION— read-onlyversionfield is modeled but the differ ignores it. Acollation-version-driftlint and explicit REFRESH step are planned for v0.3.9. See the design doc linked above.GRANTon collations — Postgres does not support object-level grants onpg_collationobjects. Out of scope by PG design.
Cluster-level surface
pgevolve manages cluster-level state — roles (v0.3.0), with tablespaces, cluster settings, foreign servers, and user mappings planned — through a parallel project type and command family separate from per-database projects.
Project shape
my-cluster/
pgevolve-cluster.toml
roles/
app.sql
ops.sql
Commands
pgevolve cluster init [path]— scaffold a new cluster projectpgevolve cluster diff— show diff between source and live clusterpgevolve cluster plan— write a cluster plan directorypgevolve cluster apply [<plan_id>]— apply a cluster planpgevolve cluster status— list applied/pending plans
Tests: tier-4: crates/pgevolve/tests/cluster_api.rs::build_cluster_plan_empty_roles_dir, build_cluster_plan_surfaces_role_loses_superuser_finding, build_cluster_plan_creates_new_role; tier-1: crates/pgevolve/src/commands/cluster/*::tests, crates/pgevolve/src/cluster_config.rs::tests.
Currently managed
| Object | Status | Tests |
|---|---|---|
| Roles (CREATE/ALTER/DROP ROLE, CREATE USER) | ✅ v0.3.0 | tier-1: crates/pgevolve-core/src/ir/cluster/role.rs::tests, parse/cluster/*::tests; tier-2: crates/pgevolve-core/tests/cluster_parse.rs, cluster_catalog.rs; tier-C: cluster/roles/create-simple-role, cluster/roles/create-login-user, cluster/roles/alter-role-attributes, cluster/roles/drop-role-intent-gated |
| Role membership (GRANT role TO target) | ✅ v0.3.0 | tier-C: cluster/roles/add-membership; tier-4: crates/pgevolve/tests/cluster_api.rs::build_cluster_plan_creates_new_role |
| Tablespaces | 🔮 Future | |
| Cluster GUCs (postgresql.conf) | 🔮 Future | |
| Foreign servers / user mappings | 🔮 Future | |
| Databases list | 🔮 Future |
Passwords
Passwords are not stored in source. The catalog reader skips
rolpassword; the source parser drops PASSWORD '…' clauses
silently. Set passwords out-of-band (psql, secret manager, etc.).
Tests: tier-1: crates/pgevolve-core/src/parse/cluster::tests (PASSWORD-clause silent drop); tier-2: crates/pgevolve-core/tests/cluster_parse.rs.
Bootstrap roles
The [bootstrap].roles list in pgevolve-cluster.toml names roles
that pgevolve treats as PG-owned and never diffs in or out. Defaults
to ["postgres"]. Cloud Postgres (RDS, Cloud SQL, etc.) typically
needs additional entries (e.g. ["postgres", "cloudsqlsuperuser"]).
Tests: tier-1: crates/pgevolve/src/cluster_config.rs::tests; tier-1: crates/pgevolve-core/src/diff/cluster.rs::tests (bootstrap-role filter behavior).
Linking from per-DB projects
Per-DB projects can lint-check grantee role names against the cluster
project by setting [cluster].project = "../my-cluster" in
pgevolve.toml. See docs/spec/grants.md for details.
Known limitations (v0.3.0)
- Cluster apply does not yet write to a per-DB-style
pgevolve.apply_log. Thepgevolve cluster statuscommand lists plan directories rather than reading applied state from the DB. Will be addressed when the cluster executor reaches feature-parity with the per-DB one. - DROP ROLE steps are marked
destructive: truein the emit pipeline, but cluster apply does not yet readintent.tomlto gate them — it executes whatever is inplan.sql. Operators should reviewcluster-plans/<id>/plan.sqlbefore runningpgevolve cluster apply. - No advisory lock is taken during cluster apply; concurrent applies against the same cluster are not protected.
- Object-level GRANT/REVOKE (e.g.
GRANT SELECT ON TABLE) is per-DB, not cluster. It ships in v0.3.1. - Row-level security policies ship in v0.3.2.
CLI
The pgevolve binary's command surface, flags, output formats, exit
codes, and configuration schema.
See ../README.md for the status legend.
Commands
| Command | Status | Notes |
|---|---|---|
pgevolve init [--dir <path>] [--force] | ✅ Implemented | Scaffolds pgevolve.toml, schema/, plans/, and a .gitignore block.Tests: tier-4: crates/pgevolve/tests/cli_e2e.rs::end_to_end_init_plan_apply_status |
pgevolve lint | ✅ Implemented | Runs universal lint rules + the configured layout-profile rules. Exits 1 on any error-severity finding. Tests: tier-4: crates/pgevolve/tests/lint_format.rs |
pgevolve validate | ✅ Implemented | Parses source IR and runs lint. Exits 1 on any error finding. Tests: tier-4: crates/pgevolve/tests/cli_e2e.rs |
pgevolve validate --shadow | ✅ Implemented | As above + round-trip through an ephemeral Postgres of the [shadow].postgres_version.Tests: tier-4: crates/pgevolve/tests/shadow_validate.rs, shadow_validate_flag.rs, shadow_validate_views.rs |
| `pgevolve diff --db | json | sql]` |
pgevolve plan --db <env> [--url <dsn>] [-o <dir>] | ✅ Implemented | Full pipeline; writes the plan directory. Output path defaults to <plan_dir>/<YYYY-MM-DD>-<short-id>.Tests: tier-4: crates/pgevolve/tests/api_build_plan.rs, cli_e2e.rs::end_to_end_init_plan_apply_status |
pgevolve apply <plan-dir> --db <env> [--url <dsn>] [--allow-different-target] [--allow-drift] | ✅ Implemented | Executes a plan directory. See "Exit codes" below. Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::apply_succeeds_end_to_end_and_persists_audit_rows, chaos_apply.rs |
| `pgevolve status --db | json]` | ✅ Implemented |
pgevolve bootstrap --db <env> [--url <dsn>] | ✅ Implemented | Explicit install/upgrade of the pgevolve metadata schema. (Other commands auto-bootstrap.)Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::bootstrap_is_idempotent |
pgevolve dump --db <env> -o <dir> | ✅ Implemented | Introspect a live DB and write <dir>/schema.sql containing CREATE statements for all managed schemas, tables, constraints, indexes, and sequences. Multi-file layout following layout_profile is deferred to v0.1.2+. Output does not include pgevolve source directives; add them manually before running pgevolve lint.Tests: tier-2: crates/pgevolve-core/tests/dump_round_trip.rs |
pgevolve graph [--graph-format dot|mermaid] [-o <path>] [--plan <dir>] | ✅ Implemented | Render the source dep graph. Read-only. --graph-format (not --format; collides with global flag) defaults to dot. --plan <dir> is deferred.Tests: tier-4: crates/pgevolve/tests/graph_command.rs |
pgevolve doctor --db <env> [--url <dsn>] | ✅ Implemented | Project health check: bootstrap status, NOT VALID constraints, INVALID indexes, source/catalog object counts, recent failed applies. Tests: tier-4: crates/pgevolve/tests/doctor_command.rs::doctor_help_includes_command |
pgevolve rewrite-table <qname> --db <env> --confirm-rewrite | 🟡 Partial | CLI surface stable; implementation lands with v0.2 partitioning / column-type-change sub-spec. Tests: tier-4: crates/pgevolve/tests/doctor_command.rs::rewrite_table_refuses_without_confirm_flag, rewrite_table_with_confirm_reports_not_yet_implemented |
pgevolve fmt | 🔮 Future | Rewrite source files into the configured layout. Lint identifies the violations; fmt would mechanically fix them. |
pgevolve check | 🔮 Future | Alias for lint && validate && plan --dry-run. Pure convenience. |
Tests (whole section): tier-1: crates/pgevolve/src/cli.rs::tests; tier-4: crates/pgevolve/tests/cli_e2e.rs::help_lists_all_nine_commands.
Global flags
| Flag | Status | Notes |
|---|---|---|
--config <path> | ✅ Implemented | Defaults to ./pgevolve.toml. |
| `--format human | json | sql` |
-v / -vv (verbosity) | ✅ Implemented | Bumps tracing filter to debug / trace. |
--quiet | ✅ Implemented | Filter set to error. |
-h / --help / --version (clap built-ins) | ✅ Implemented | Tests: tier-4: crates/pgevolve/tests/cli_e2e.rs::help_lists_all_nine_commands |
Shadow-validation flags (plan, diff, validate)
| Flag | Status | Notes |
|---|---|---|
--shadow-validate | ✅ Implemented (scaffold) | Opt-in cross-check against the shadow Postgres. v0.1 is a no-op for body-bearing objects (none exist yet); v0.2 sub-specs deepen coverage. Tests: tier-4: crates/pgevolve/tests/shadow_validate_flag.rs, shadow_validate_views.rs |
--shadow-strict | ✅ Implemented (scaffold) | Requires --shadow-validate. Treats shadow mismatches as errors rather than warnings.Tests: tier-4: crates/pgevolve/tests/shadow_validate.rs |
Output formats
| Format | Default for | Status | Notes |
|---|---|---|---|
human | every command except dump | ✅ Implemented | Hierarchical text, color-on-tty when stdout is a TTY (color polish 🔮 Future). Tests: tier-4: crates/pgevolve/tests/lint_format.rs::lint_default_format_is_human |
json | optional everywhere | ✅ Implemented | Stable schema; every top-level object carries a schema_version field (📋 to be added uniformly in v0.1.1).Tests: tier-4: crates/pgevolve/tests/lint_format.rs::lint_json_format_emits_structured_output |
sql | diff only | ✅ Implemented | Naive ALTER SQL with no online rewrites — for code review only; users run plan for the applyable form.Tests: tier-4: crates/pgevolve/tests/lint_format.rs::lint_sql_format_is_rejected |
Exit codes
Spec §13. Implemented in commands::apply::run; other commands follow
the same convention.
| Code | Meaning | Status | Tests |
|---|---|---|---|
0 | Success | ✅ Implemented | tier-4: crates/pgevolve/tests/cli_e2e.rs::end_to_end_init_plan_apply_status |
1 | Lint or validation error (or any unmapped error) | ✅ Implemented | tier-4: crates/pgevolve/tests/lint_waiver_e2e.rs::plan_refuses_unwaived_column_position_drift |
2 | Drift or pre-flight mismatch (target identity, drift, unapproved intents) | ✅ Implemented | tier-4: crates/pgevolve/tests/executor_smoke.rs::apply_rejects_target_identity_mismatch |
3 | Apply error (lock held, step failed) | ✅ Implemented | tier-4: crates/pgevolve/tests/executor_smoke.rs::apply_rolls_back_transactional_group_on_failure |
4 | Config or CLI input error | ✅ Implemented | tier-4: crates/pgevolve/tests/doctor_command.rs::rewrite_table_refuses_without_confirm_flag |
Connection precedence
Mirrors psql. First non-empty source wins.
Tests (whole section): tier-1: crates/pgevolve/src/connection.rs::tests.
| Order | Source | Status | Notes |
|---|---|---|---|
| 1 | --url <dsn> CLI argument | ✅ Implemented | |
| 2 | [environments.<env>].url | ✅ Implemented | |
| 3 | [environments.<env>].url_env (env var name) | ✅ Implemented | |
| 4 | PGEVOLVE_DATABASE_URL env var | ✅ Implemented | |
| 5 | libpq env vars (PGHOST, PGUSER, PGPASSWORD, etc.) | ✅ Implemented | Implicit via tokio_postgres::connect(""). |
| 6 | ~/.pgpass | ✅ Implemented (via libpq) | Same path libpq uses. |
pgevolve.toml schema
Tests (whole section): tier-1: crates/pgevolve/src/config.rs::tests.
| Section | Status | Notes |
|---|---|---|
[project] (name, schema_dir, plan_dir, layout_profile) | ✅ Implemented | Required. |
[managed] (schemas, ignore_objects) | ✅ Implemented | Empty schemas list means "lint doesn't enforce schema match"; the filter still applies. |
[planner] (strategy) | ✅ Implemented | atomic or online. |
[planner.online_rewrites] (per-rewrite switches) | ✅ Implemented | Six switches: create_index_concurrent, fk_not_valid_then_validate, check_not_valid_then_validate, not_null_via_check_pattern, refresh_mv_concurrently, view_drop_create_dependents.Tests: tier-1: crates/pgevolve-core/src/plan/policy.rs::tests |
[environments.<name>] (url, url_env, strategy) | ✅ Implemented | Per-env strategy override. |
[shadow] (backend, url, url_env, reset, extensions, postgres_version) | ✅ Implemented | Full schema: backend = "auto" (auto-select testcontainers or DSN); url / url_env for DSN override; reset = "drop_schema_cascade"; extensions = ["pgcrypto"]; postgres_version = "17".Tests: tier-4: crates/pgevolve/tests/shadow_backend.rs |
[extensions] (declared extensions and versions) | 📋 Planned, v0.2 | Lands with extension support. |
[grants] (high-level grant tables) | 📋 Planned, v0.3 | Lands with roles + grants. |
[planner.online_rewrites] — v0.2 view / MV keys
| Key | Default | Notes |
|---|---|---|
refresh_mv_concurrently | true | Upgrade REFRESH MATERIALIZED VIEW to REFRESH MATERIALIZED VIEW CONCURRENTLY when the MV has at least one unique index. Has no effect under strategy = "atomic".Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/refresh_mv_concurrently.rs::tests; tier-C: objects/materialized_views/refresh-concurrently |
view_drop_create_dependents | true | When true, the planner walks the body_dependencies graph and emits explicit DROP + CREATE steps for every view transitively affected by an upstream change. When false, the planner errors instead of cascading dependent-view recreations — useful if you want to review every affected view manually.Tests: tier-1: crates/pgevolve-core/src/plan/recreate_views.rs::tests; tier-C: scenarios/dependency-chains/view-on-view-column-drop |
[shadow] block (full example)
[shadow]
backend = "auto"
url = "..."
url_env = "..."
reset = "drop_schema_cascade"
extensions = ["pgcrypto"]
postgres_version = "17"
intent.toml schema
Beyond the [[intent]] rows written by the planner, intent.toml supports two user-authored table kinds: [[lint_waiver]] and [[step_override]].
Tests (whole section): tier-4: crates/pgevolve/tests/lint_waiver_e2e.rs::lint_waiver_survives_intent_toml_round_trip, plan_proceeds_with_matching_lint_waiver; tier-1: crates/pgevolve-core/src/plan/serialize.rs::tests, deserialize.rs::tests.
[[lint_waiver]]
[[lint_waiver]]
rule = "column-position-drift"
target = "app.users"
reason = "..."
| Field | Required | Notes |
|---|---|---|
rule | yes | Stable rule identifier (e.g., column-position-drift). Non-empty. |
target | yes | Qualified object name the waiver applies to. Non-empty. |
reason | no | Free-text explanation; encouraged for auditing. |
[[step_override]]
Step overrides allow suppressing or modifying individual planner steps. Useful when you want to skip, for example, a refresh_materialized_view step during a maintenance window.
[[step_override]]
kind = "refresh_materialized_view"
target = "app.daily_summary"
suppress = true
| Field | Required | Notes |
|---|---|---|
kind | yes | Step kind to match (e.g., refresh_materialized_view). Must be a valid StepKind name. |
target | yes | Qualified object name the override applies to. Non-empty. |
suppress | no | Default false. When true, the matching step is omitted from the plan entirely. |
Logging
| Aspect | Status | Notes |
|---|---|---|
tracing + tracing-subscriber | ✅ Implemented | |
RUST_LOG env var override | ✅ Implemented | |
Structured fields on every span (apply_id, step_no, qname) | 🟡 Partial | The executor sets apply_id; richer per-step fields land alongside structured CLI JSON. |
| stderr-only log output, stdout reserved for data | ✅ Implemented |
Lint and layout
Universal lint rules (always applied), built-in layout profiles (one of four selected per project), and the custom-profile mechanism.
See ../README.md for the status legend.
Universal rules
These apply regardless of layout profile and most are enforced at parse
time. The lint engine runs them defensively over a built SourceTree.
| Rule | Status | Enforced by | Tests |
|---|---|---|---|
Every statement parses cleanly under pg_query | ✅ Implemented | Parser. | tier-1: crates/pgevolve-core/src/parse/statement.rs::tests; tier-C: failure/parse/duplicate-schema |
Every CREATE is schema-qualified or has a file-level -- @pgevolve schema=... directive | ✅ Implemented | Parser. | tier-1: crates/pgevolve-core/src/parse/directives.rs::tests, parse/mod.rs::tests |
| No object qname appears twice across the tree | ✅ Implemented | Parser (raises DuplicateObject); lint double-checks. | tier-1: crates/pgevolve-core/src/lint/rules/no_duplicate_qnames.rs; tier-C: failure/parse/duplicate-schema |
| Only v0.1 MVP object kinds appear in source | ✅ Implemented | Parser (raises UnsupportedObjectKind). | tier-1: crates/pgevolve-core/src/parse/statement.rs::tests |
No ALTER statement outside the FK forward-reference whitelist | ✅ Implemented | Parser. | tier-1: crates/pgevolve-core/src/parse/builder/alter_table_stmt.rs::tests |
| Every FK target table exists in the source tree | ✅ Implemented | Lint engine (closed_world_references). | tier-1: crates/pgevolve-core/src/lint/rules/closed_world_references.rs; tier-C: failure/ast-resolution/fk-to-missing-table |
| Every indexed table exists in the source tree | ✅ Implemented | Lint engine. | tier-1: crates/pgevolve-core/src/lint/rules/closed_world_references.rs |
Every sequence's OWNED BY target exists in the source tree | ✅ Implemented | Lint engine. | tier-1: crates/pgevolve-core/src/lint/rules/closed_world_references.rs |
[managed].schemas matches the schemas declared in source (two-way) | ✅ Implemented | Lint engine (managed_schemas_match). Silent when managed.schemas is empty. | tier-1: crates/pgevolve-core/src/lint/rules/managed_schemas_match.rs |
| Every column referenced by a constraint exists in its parent table | 🔮 Future | Mostly caught by Postgres at apply time; could be brought forward to lint time. | |
| Every type referenced by a column exists (or is built-in) | 🔮 Future | Same: caught by Postgres today. | |
column-position-drift — table's column order in source disagrees with target catalog | ✅ Implemented | Severity LintAtPlan (see below). Source is canonical. Resolution: reorder source, add [[lint_waiver]] in intent.toml, or run pgevolve rewrite-table. | tier-1: crates/pgevolve-core/src/lint/rules/column_position_drift.rs; tier-2: crates/pgevolve-core/tests/lint_position_drift.rs; tier-C: failure/lint-at-plan/column-position-drift-no-waiver |
view-shadows-table — a VIEW or MATERIALIZED VIEW shares a qualified name with a managed table | ✅ Implemented | Severity Error. Views and tables occupy the same namespace in Postgres; pgevolve rejects the ambiguity at parse time. | tier-1: crates/pgevolve-core/src/lint/rules/view_shadows_table.rs |
mv-no-unique-index — a MATERIALIZED VIEW has no unique index and thus cannot use REFRESH CONCURRENTLY | ✅ Implemented | Severity Warning. Resolution: add a unique index on the MV, or set refresh_mv_concurrently = false in [planner.online_rewrites]. | tier-1: crates/pgevolve-core/src/lint/rules/mv_no_unique_index.rs; tier-C: objects/materialized_views/create-no-unique-index-online |
view-body-references-unmanaged-schema — a view body dependency edge points to a schema not in [managed].schemas | ✅ Implemented | Severity Warning. pgevolve cannot track schema changes for objects it does not manage; a cross-schema dependency is a portability risk. | tier-1: crates/pgevolve-core/src/lint/rules/view_body_references_unmanaged_schema.rs |
type-shadows-table — a user-defined type shares a qualified name with a managed table, view, or MV | ✅ Implemented | Severity Error. Postgres uses one namespace for relations and types; the conflict would be rejected at apply time. | tier-1: crates/pgevolve-core/src/lint/rules/type_shadows_table.rs |
enum-value-collision — an enum type declares duplicate value labels | ✅ Implemented | Severity Error. Defense-in-depth; the source parser also rejects duplicates. | tier-1: crates/pgevolve-core/src/lint/rules/enum_value_collision.rs |
composite-attribute-collision — a composite type declares duplicate attribute names | ✅ Implemented | Severity Error. Defense-in-depth; the source parser also rejects duplicates. | tier-1: crates/pgevolve-core/src/lint/rules/composite_attribute_collision.rs |
domain-check-references-unmanaged-type — a domain's CHECK expression references a schema not in [managed].schemas | ✅ Implemented | Severity Warning. pgevolve cannot track changes to objects it does not manage; the reference is a portability risk. Silent when [managed].schemas is empty. | tier-1: crates/pgevolve-core/src/lint/rules/domain_check_references_unmanaged_type.rs |
plpgsql-dynamic-sql — PL/pgSQL body uses EXECUTE without a -- @pgevolve dep: directive | ✅ Implemented | Severity Error. Resolved by adding -- @pgevolve dep: schema.name directives to declare the dynamic references explicitly. | tier-1: crates/pgevolve-core/src/lint/rules/pl_pgsql_dynamic_sql.rs; tier-C: objects/functions/function-with-dynamic-sql-directive, scenarios/function-with-dynamic-sql-directive-clears-lint |
procedure-contains-commit — procedure body contains COMMIT or ROLLBACK | ✅ Implemented | Severity Warning. pgevolve auto-detects transaction control statements and runs the step with transactional=OutsideTransaction. | tier-1: crates/pgevolve-core/src/lint/rules/procedure_contains_commit.rs; tier-C: objects/procedures/create-with-commit |
function-references-unmanaged-schema — routine body dep edge targets an unmanaged schema | ✅ Implemented | Severity Warning. pgevolve cannot track changes to objects it does not manage; the cross-schema dependency is a portability risk. Silent when [managed].schemas is empty. | tier-1: crates/pgevolve-core/src/lint/rules/function_references_unmanaged_schema.rs |
Severity tiers
Tests (whole section): tier-1: crates/pgevolve-core/src/lint/mod.rs (Severity enum), crates/pgevolve-core/src/lint/rules/mod.rs::tests; tier-4: crates/pgevolve/tests/lint_waiver_e2e.rs::plan_refuses_unwaived_column_position_drift, plan_proceeds_with_matching_lint_waiver.
| Tier | Status | Behaviour |
|---|---|---|
Error | ✅ Implemented | Fails lint (exit 1). |
Warning | ✅ Implemented | Reported but does not fail lint. |
LintAtPlan | ✅ Implemented | Drift / divergence detected at plan time that pgevolve declines to act on without explicit user instruction. pgevolve plan exits with code 2 unless the finding is waived via a matching [[lint_waiver]] row in intent.toml. |
Layout profiles
A profile expresses where an object should live on disk. Selected by
[project].layout_profile. All built-ins ship in
pgevolve_core::lint::profile.
schema-mirror (strictest)
Tests: tier-1: crates/pgevolve-core/src/lint/profile/schema_mirror.rs::tests.
| Convention | Status | Notes |
|---|---|---|
Tables, indexes, sequences live at <schema>/<kind_plural>/<name>.sql | ✅ Implemented | <kind_plural> is tables / indexes / sequences. |
Schemas live at <schema>/_schema.sql | ✅ Implemented | Where you put the CREATE SCHEMA for that schema. |
| One object per file (schemas excepted) | ✅ Implemented |
kind-grouped
Tests: tier-1: crates/pgevolve-core/src/lint/profile/kind_grouped.rs::tests.
| Convention | Status | Notes |
|---|---|---|
Tables / indexes / sequences live at <kind_plural>/<schema>.<name>.sql | ✅ Implemented | |
Schemas live at schemas/<name>.sql | ✅ Implemented | |
| One object per file | ✅ Implemented |
feature-grouped
Tests: tier-1: crates/pgevolve-core/src/lint/profile/feature_grouped.rs::tests.
| Convention | Status | Notes |
|---|---|---|
Every file lives under <schema_dir>/<some-feature-dir>/ (no direct children) | ✅ Implemented | |
| Multiple objects per file are allowed | ✅ Implemented | |
| Cross-feature overlap forbidden (no object spans two feature dirs) | 🔮 Future | Rigorously defining "overlap" was non-trivial; lighter spec-only check ships now, fuller version when there is clear demand. |
free-form
Tests: tier-1: crates/pgevolve-core/src/lint/profile/free_form.rs::tests.
| Convention | Status | Notes |
|---|---|---|
| No path constraints | ✅ Implemented | Only universal rules apply. |
custom
A user-defined profile loaded from a TOML path passed in
[project].layout_profile.
Tests (whole section): tier-1: crates/pgevolve-core/src/lint/profile/custom.rs::tests.
| Mechanism | Status | Notes |
|---|---|---|
[[patterns]] table with regex + assertions | ✅ Implemented | Regex applied to the path relative to schema_dir. First match wins. |
Assertion: schema_matches_capture | ✅ Implemented | Requires the regex's ?P<schema> capture to equal the object's qname.schema. |
Assertion: name_matches_capture | ✅ Implemented | Requires the regex's ?P<name> capture to equal the object's bare name. |
Assertion: kind_matches_capture with allowed_values = { capture_value = "kind", … } | ✅ Implemented | Maps the regex's ?P<kind> capture to one of schema / table / index / sequence. |
Assertion: one_object_per_file | ✅ Implemented | |
| Embedded scripting (Rhai / Lua / …) | ⛔ Not planned | Out of scope for v0.1; the regex+assertion mechanism is intentionally declarative. |
Lint output
| Aspect | Status | Notes |
|---|---|---|
Severity::Error / Severity::Warning | ✅ Implemented | Errors fail the lint (exit 1); warnings don't. Tests: tier-4: crates/pgevolve/tests/lint_format.rs::lint_default_format_is_human |
Stable rule identifiers (managed_schemas_match, schema_mirror_path, …) | ✅ Implemented | Used for filtering and --explain in the future.Tests: tier-1: crates/pgevolve-core/src/lint/rules/mod.rs::tests |
Source location (file:line:column) on every finding | ✅ Implemented | When available; some findings (e.g., aggregated profile rules) don't have a single location. Tests: tier-4: crates/pgevolve/tests/lint_format.rs::lint_json_format_emits_structured_output |
--explain <rule> to print the rule's rationale + example fix | 🔮 Future | Lands when there are enough rules to make explanations valuable. |
--deny <rule> / --allow <rule> overrides | 🔮 Future | Configurable per-rule severity. |
--format json lint output | ✅ Implemented | pgevolve lint --format json emits a stable structured document with findings[], total, and errors. Severity values are stringified ("error", "warning", "lint-at-plan"). --format sql is rejected for lint (sql output is meaningful only for diff).Tests: tier-4: crates/pgevolve/tests/lint_format.rs::lint_json_format_emits_structured_output, lint_sql_format_is_rejected |
Pipeline
The path from source SQL to applied DDL, phase by phase. Each phase has its own implementation crate / module, status, and the design doc / plan that drove it.
See ../README.md for the status legend.
Phase summary
flowchart TD
SQL["schema/*.sql"] -- parse --> SourceIR["Source IR"]
DB[("live Postgres")] -- introspect --> CatalogIR["Catalog IR"]
SourceIR -- canonicalize --> Source["Catalog (source)"]
CatalogIR --> Target["Catalog (target)"]
Source --> Diff{{diff}}
Target --> Diff
Diff --> CS["ChangeSet"]
CS -- order --> OCS["OrderedChangeSet"]
OCS -- rewrite --> Steps["Vec<RawStep>"]
Steps -- group_steps --> Groups["Vec<TransactionGroup>"]
Groups -- "Plan::from_grouped" --> Plan["Plan"]
Plan --> PlanSql["plan.sql"]
Plan --> Intent["intent.toml"]
Plan --> Manifest["manifest.toml"]
Plan -- "apply()" --> DB
Parsing (pgevolve_core::parse)
Tests (whole section): tier-1: crates/pgevolve-core/src/parse/mod.rs::tests, parse/statement.rs::tests, parse/directives.rs::tests; tier-2: crates/pgevolve-core/tests/parser_corpus.rs, parse_directory.rs.
| Aspect | Status | Notes |
|---|---|---|
pg_query-based statement classification | ✅ Implemented | Postgres's own parser, exposed via the pg_query crate. |
| Whitelist of source-side DDL kinds | ✅ Implemented | CREATE SCHEMA/TABLE/INDEX/SEQUENCE, ALTER TABLE (limited to FK whitelist), COMMENT ON. Everything else rejects with UnsupportedObjectKind. |
Per-file -- @pgevolve schema=<name> directive | ✅ Implemented | Allows unqualified objects in a file to default to a named schema. |
| Per-object source location tracking | ✅ Implemented | Powers lint findings and round-trip cross-checks. |
parse_directory and parse_directory_with_locations | ✅ Implemented | The first returns a Catalog; the second adds the qname → SourceLocation map for the linter. |
| Deterministic file order | ✅ Implemented | Walks paths in sort order so identical inputs produce identical output. Tests: tier-2: crates/pgevolve-core/tests/determinism.rs |
| Multi-file project layout enforced by the layout profile | ✅ Implemented | See lint-and-layout.md. |
AST canonicalization pass (pgevolve_core::parse::ast_canon)
Runs immediately after source parse, before the AST resolution pass. For each view and materialized view in the provisional catalog, the pass:
- Calls
NormalizedBody::from_sql(seeparse/normalize_body.rs) onraw_bodyto fillbody_canonical. - Walks the body AST to extract
DepEdgerecords withDepSource::AstExtractedprovenance, fillingbody_dependencies. - Resolves each referenced relation against the provisional catalog. Unresolved references surface as
AstCanonError::UnresolvedReference. - Fills
columnsfrom the SELECT target list when no explicit alias list was provided (using Postgres's column-naming algorithm: explicit alias → rightmostColumnRefname →"?column?"fallback).
The same NormalizedBody::from_sql call is used on the catalog side (T5 reader queries pg_get_viewdef), so source-side and catalog-side canonical texts are directly comparable by the differ. The v0.2 property test (view_canonicalization_closed_under_pg_rewrite) verifies this closure invariant.
| Aspect | Status | Notes |
|---|---|---|
NormalizedBody::from_sql canonicalization | ✅ Implemented | pg_query parse + deparse + redundant-qualifier strip (SELECT users.id FROM app.users → SELECT id FROM app.users for single-relation FROM clauses, so PG14's qualified pg_get_viewdef output matches PG17's unqualified form) + whitespace collapse. Source: parse/normalize_body.rs.Tests: tier-1: crates/pgevolve-core/src/parse/normalize_body.rs::tests; tier-2: crates/pgevolve-core/tests/normalize_body.rs, ast_canon.rs |
| Dep-edge extraction from view body AST | ✅ Implemented | DepEdge { from: NodeId::View, to: NodeId::Table/View/Mv, source: AstExtracted }.Tests: tier-2: crates/pgevolve-core/tests/ast_canon.rs, dep_edges.rs |
body_dependencies integration into planner ordering | ✅ Implemented | View → table and view → view edges are part of the dep-graph; creates and drops respect the topo order. Tests: tier-1: crates/pgevolve-core/src/plan/ordering.rs::tests, plan/graph.rs::tests; tier-C: scenarios/dependency-chains/linear-3-layer-create, scenarios/view-uses-function |
AST resolution pass (pgevolve_core::parse::resolve)
Runs after the AST canonicalization pass. Validates structural references before diff.
| Aspect | Status | Notes |
|---|---|---|
| FK targets validated against declared tables | ✅ Implemented | Surfaces unresolved references as ParseError::AstResolution with source location.Tests: tier-2: crates/pgevolve-core/tests/ast_resolution.rs; tier-C: failure/ast-resolution/fk-to-missing-table |
| Default-using sequences validated against declared sequences | ✅ Implemented | Same error path. Tests: tier-2: crates/pgevolve-core/tests/ast_resolution.rs |
| View body cross-references validated against declared objects | ✅ Implemented | The AST canon pass (ast_canon.rs) surfaces unresolved view body references as AstCanonError::UnresolvedReference.Tests: tier-2: crates/pgevolve-core/tests/ast_canon.rs |
Catalog reader (pgevolve_core::catalog)
Tests (whole section): tier-1: crates/pgevolve-core/src/catalog/mod.rs::tests, catalog/filter.rs::tests, catalog/version.rs::tests, catalog/rows.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs, functions_round_trip.rs, types_round_trip.rs, dump_round_trip.rs.
| Aspect | Status | Notes |
|---|---|---|
Version detection (pg_control_system, server_version_num) | ✅ Implemented | PG 14/15/16/17 tested per major. |
| Per-version SQL strings for each catalog query | ✅ Implemented | |
| Schemas, tables, columns, constraints, indexes, sequences | ✅ Implemented | Mirrors the v0.1 IR surface. |
Dependencies (sequence OWNED BY, default → sequence) | ✅ Implemented | |
pg_catalog.default collation normalized to "none" | ✅ Implemented | Avoids phantom drift on every text column. |
Sequence / function / collation PG defaults normalized to None | ✅ Implemented | PG stores explicit values for things the user didn't declare (sequence min/max, function procost=100/prorows=1000, implicit pg_catalog.default collation on text columns). The catalog reader returns raw values; ir::canon::filter_pg_defaults strips them on both the source-built and catalog-read Catalog. One place for "next time PG returns a surprising default."Tests: tier-1: crates/pgevolve-core/src/ir/canon/filter_pg_defaults.rs::tests |
| Views and materialized views | ✅ Implemented | read_views and read_materialized_views query pg_views / pg_matviews, call pg_get_viewdef for the body text, and feed it through NormalizedBody::from_sql so the catalog-side canonical text is directly comparable with the source-side canonical text. |
| Functions, procedures | ✅ Implemented | pg_proc joined with pg_language, pg_type, pg_namespace; body reconstructed via pg_get_functiondef. |
| User-defined types (enums, domains, composites) | ✅ Implemented | pg_type joined with pg_enum, pg_attribute, pg_constraint, pg_attrdef. |
| Extensions | ✅ Implemented | pg_extension joined with pg_namespace and pg_description. Extension-owned objects excluded via pg_depend deptype='e' filter. |
| Triggers | ✅ Implemented | pg_trigger joined with pg_class, pg_namespace, pg_description. Filtered: NOT tgisinternal; NOT extension-owned. |
| Partitioning (parents + children) | ✅ Implemented | pg_class.relkind='p' + pg_get_partkeydef for partitioned parents; relispartition=true + pg_get_expr(relpartbound) for partition children. Both queries: NOT extension-owned; scoped to managed schemas. |
Cluster surface: roles, role attributes, role membership (pg_authid, pg_auth_members) | ✅ Implemented | Returned in ClusterCatalog, queried via the pgevolve cluster … subcommand family. v0.3.0. |
Per-object owner + grants (object-level + column-level) + ALTER DEFAULT PRIVILEGES rules | ✅ Implemented | pg_class.relowner, pg_class.relacl, pg_attribute.attacl, pg_default_acl decoded into owner / grants / Catalog::default_privileges. v0.3.1. |
Row-level security: per-table rls_enabled / rls_forced + pg_policies | ✅ Implemented | pg_class.relrowsecurity / relforcerowsecurity + a join on pg_policies for embedded Vec<Policy>. v0.3.2. |
Storage parameters / reloptions (pg_class.reloptions::text[]) | ✅ Implemented | Decoded into typed TableStorageOptions / IndexStorageOptions with extra: BTreeMap for unknown keys. Materialized views share the table decoder. v0.3.3. |
CREATE STATISTICS, publications, subscriptions, FDWs | 📋 Planned / 🔮 Future | See docs/spec/objects.md. |
Catalog filtering by [managed] schemas + [managed].ignore_objects globs | ✅ Implemented | Unmanaged schemas don't appear in the IR at all. |
Catalog drift detection — returns (Catalog, DriftReport) | ✅ Implemented | See "Catalog drift detection" section below. Tests: tier-2: crates/pgevolve-core/tests/catalog_drift.rs |
Catalog drift detection
The catalog reader returns a DriftReport alongside the Catalog. The differ and planner consume it to automatically recover from partial-apply states.
| Drift kind | Detection | Diff emit | Planner emit | Status | Tests |
|---|---|---|---|---|---|
NOT VALID constraints (pg_constraint.convalidated = false) | Catalog reader | Change::ValidateConstraint | ALTER TABLE ... VALIDATE CONSTRAINT | ✅ Implemented | tier-2: crates/pgevolve-core/tests/catalog_drift.rs |
INVALID indexes (pg_index.indisvalid = false) | Catalog reader | Change::RecreateIndex | DROP INDEX + CREATE INDEX | ✅ Implemented | tier-2: crates/pgevolve-core/tests/catalog_drift.rs |
Both drift kinds are auto-recovery paths — the user doesn't author NOT VALID or INVALID states; pgevolve detects and resolves them. This covers recovery from crashed FK NOT VALID + VALIDATE rewrites and failed CREATE INDEX CONCURRENTLY.
Differ (pgevolve_core::diff)
Tests (whole section): tier-1: every crates/pgevolve-core/src/diff/*.rs::tests module (tables, columns, constraints, indexes, sequences, views, triggers, types, routines, policies, grants, reloptions, owner_op, schemas, extensions, cluster, default_privileges); tier-1: crates/pgevolve-core/src/diff/changeset.rs::tests, change.rs::tests, destructiveness.rs::tests.
| Aspect | Status | Notes |
|---|---|---|
Catalog::diff produces a structured Vec<Difference> for assertions | ✅ Implemented | Tests: tier-1: crates/pgevolve-core/src/diff/mod.rs::tests, ir/difference.rs |
diff(target, source) → ChangeSet produces the unordered planner input | ✅ Implemented | Each ChangeEntry carries a Destructiveness classification.Tests: tier-1: crates/pgevolve-core/src/diff/changeset.rs::tests |
| Pair-by-qname semantics | ✅ Implemented | Tables / indexes / sequences pair by qualified name; columns / constraints pair by bare name within their parent table. Tests: tier-1: crates/pgevolve-core/src/diff/tables.rs::tests, columns.rs::tests |
| Column reorder detection | 🟡 Partial | Detected as columns.<order> difference but the planner does not yet emit a reorder step (Postgres has no ALTER COLUMN ... POSITION, so this would require table rewrite).Tests: tier-1: crates/pgevolve-core/src/lint/rules/column_position_drift.rs::tests; tier-2: crates/pgevolve-core/tests/lint_position_drift.rs |
| Index option change detection | ✅ Implemented | Triggers ReplaceIndex (drop + create).Tests: tier-1: crates/pgevolve-core/src/diff/indexes.rs::tests |
| Constraint rename detection | ⛔ Not planned | Diffs as drop+add. |
| Destructiveness classification | ✅ Implemented | Three levels: Safe, RequiresApproval, RequiresApprovalAndDataLossWarning.Tests: tier-1: crates/pgevolve-core/src/diff/destructiveness.rs::tests |
Planner (pgevolve_core::plan)
Ordering
Tests (whole subsection): tier-1: crates/pgevolve-core/src/plan/ordering.rs::tests, plan/ordered.rs::tests, plan/graph.rs::tests, plan/edges.rs::tests; tier-2: crates/pgevolve-core/tests/dep_edges.rs, determinism.rs; tier-5: property plan_id_is_deterministic, create_graph_topo_sorts_or_only_fk_cycles (crates/pgevolve-core/tests/property_tests.rs).
| Aspect | Status | Notes |
|---|---|---|
| Three-phase ordering: creates → modifies → drops | ✅ Implemented | Each phase topologically sorted by the appropriate graph. |
| Source-side dependency graph for creates / modifies | ✅ Implemented | Schema ← table, table ← index, FK ← both endpoints, sequence ← owning table, table ← default-using sequence. |
| Target-side dependency graph for drops | ✅ Implemented | Same edges; drop order is the reverse topo sort. |
| FK forward-reference cycle handling | ✅ Implemented | Cycles are broken by extracting offending FKs into a post-pass DeferredFkAdd list.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/emit/deferred_fk.rs; tier-C: failure/cycle |
| Deterministic tie-break | ✅ Implemented | Topological sort uses a BTreeSet/min-heap by Ord; identical inputs produce byte-identical plans.Tests: tier-2: crates/pgevolve-core/tests/determinism.rs |
Rewrites
Tests (whole subsection): tier-1: crates/pgevolve-core/src/plan/rewrite/tests.rs (covers concurrent index, FK NOT VALID, CHECK NOT VALID, set-not-null pattern, atomic-vs-online gating).
| Rule | Status | Notes |
|---|---|---|
Concurrent index create (CREATE INDEX CONCURRENTLY) on existing tables | ✅ Implemented | Non-unique only. Excluded for new tables, unique indexes, and atomic policy. Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/tests::create_index_on_existing_table_rewrites_to_concurrent |
Concurrent index drop (DROP INDEX CONCURRENTLY) on existing non-unique indexes | ✅ Implemented | Same gating. Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/concurrent_index.rs |
FK NOT VALID + VALIDATE CONSTRAINT for adds on existing tables | ✅ Implemented | Splits across two transaction groups so step A (cheap) commits before step B (table scan). Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/fk_not_valid_validate.rs |
CHECK NOT VALID + VALIDATE CONSTRAINT for adds on existing tables | ✅ Implemented | Same pattern as FK. Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/check_not_valid_validate.rs |
SET NOT NULL on populated columns via the CHECK pattern (4 steps) | ✅ Implemented | ADD CHECK NOT VALID → VALIDATE → SET NOT NULL (cheap once validated) → DROP CONSTRAINT.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/set_not_null_check_pattern.rs, plan/rewrite/tests::set_not_null_on_existing_column_emits_four_steps |
Per-environment policy override ([environments.<env>].strategy = "atomic") | ✅ Implemented | Atomic mode disables every online rewrite. Tests: tier-1: crates/pgevolve-core/src/plan/policy.rs::tests, plan/rewrite/tests::atomic_policy_disables_concurrent_index_rewrite |
REFRESH MATERIALIZED VIEW CONCURRENTLY upgrade | ✅ Implemented | When refresh_mv_concurrently = true (default) and the MV has a unique index, the planner emits REFRESH MATERIALIZED VIEW CONCURRENTLY instead of the locking variant. Gated on strategy = online.Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/refresh_mv_concurrently.rs::tests; tier-C: objects/materialized_views/refresh-concurrently |
Dependent-view recreation cascade (recreate_views::extend_with_dependent_recreations) | ✅ Implemented | When a table drop, column change, or incompatible view-body replace is detected, the planner walks body_dependencies transitively and emits explicit DROP + CREATE steps for every affected view. Controlled by view_drop_create_dependents switch (default true).Tests: tier-1: crates/pgevolve-core/src/plan/recreate_views.rs::tests, plan/rewrite/views.rs::tests; tier-C: scenarios/dependency-chains/view-on-view-column-drop |
ALTER TYPE ... ADD VALUE (enum value add) online rewrite | 📋 Planned, v0.2 | Lands with enum support. Tests: tier-1: crates/pgevolve-core/src/plan/rewrite/types.rs::tests; tier-C: objects/enums/add-value-at-end, add-value-before-existing |
ALTER COLUMN ... TYPE online rewrite (e.g., int → bigint) | 🔮 Future | Currently emits a single ALTER COLUMN ... TYPE step, which can rewrite the entire table. The "USING expr + new column + rename" pattern is a candidate v0.3 rewrite.Tests: tier-C: objects/columns/alter-column-type-widening, alter-column-type-narrowing |
REINDEX CONCURRENTLY for bloated indexes | 🔮 Future | Not currently emitted by the planner; users invoke manually. |
Plan-time lint gate
| Aspect | Status | Notes |
|---|---|---|
run_drift_lints called after diff, before writing plan | ✅ Implemented | Any LintAtPlan finding without a matching [[lint_waiver]] in intent.toml causes pgevolve plan to exit with code 2.Tests: tier-4: crates/pgevolve/tests/lint_waiver_e2e.rs::plan_refuses_unwaived_column_position_drift, plan_proceeds_with_matching_lint_waiver; tier-C: failure/lint-at-plan/column-position-drift-no-waiver |
column-position-drift as a LintAtPlan finding | ✅ Implemented | See lint-and-layout.md.Tests: tier-2: crates/pgevolve-core/tests/lint_position_drift.rs |
Step grouping
Tests (whole subsection): tier-1: crates/pgevolve-core/src/plan/grouping.rs::tests.
| Aspect | Status | Notes |
|---|---|---|
Adjacent steps with the same TransactionConstraint coalesce into one group | ✅ Implemented | |
Transactional groups run inside one BEGIN; … COMMIT; | ✅ Implemented | |
| Non-transactional groups run as autocommit singletons | ✅ Implemented | Each CONCURRENTLY step is its own atomic unit. |
Plan format (pgevolve_core::plan::{serialize, deserialize})
Tests (whole section): tier-1: crates/pgevolve-core/src/plan/serialize.rs::tests, plan/deserialize.rs::tests, plan/plan.rs::tests.
| File | Status | Notes |
|---|---|---|
plan.sql | ✅ Implemented | Canonical artifact: directive header + per-group BEGIN/COMMIT + per-step -- @pgevolve step=… directive lines + SQL bodies. Runs cleanly under psql -f even without pgevolve's executor. |
intent.toml | ✅ Implemented | One [[intent]] row per destructive step; user must flip approved = true before applying. |
manifest.toml | ✅ Implemented | Plan id (full hex), version metadata, target identity, embedded pre-image catalog as JSON. |
Round-trip property: read_plan_dir(write_plan_dir(p)) == p | ✅ Implemented | Property-tested. Tests: tier-5: property in crates/pgevolve-core/tests/property_tests.rs |
| Cross-file plan-id mismatch detection | ✅ Implemented | All three files must agree on plan_id. |
Deterministic PlanId (BLAKE3 over bincode-encoded (source, target, version, ruleset)) | ✅ Implemented | Identical inputs always produce the same id. Tests: tier-5: plan_id_is_deterministic (crates/pgevolve-core/tests/property_tests.rs) |
Executor (pgevolve::executor)
Tests (whole section): tier-4: crates/pgevolve/tests/executor_smoke.rs, chaos_apply.rs; tier-5: drift_recovery_property in crates/pgevolve/tests/pg_property_tests.rs.
| Stage | Status | Notes |
|---|---|---|
Bootstrap pgevolve.bootstrap_version / apply_log / plan_steps / lock tables | ✅ Implemented | Idempotent; append-only migration list. Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::bootstrap_is_idempotent |
Singleton advisory lock (pg_try_advisory_lock) | ✅ Implemented | Lock key derived from ASCII PGEVOLVE. Session-scoped; released on disconnect or via release_lock.Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::advisory_lock_contention |
Target-identity computation (BLAKE3 of (db, host, port, cluster_name, system_identifier)) | ✅ Implemented | Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::target_identity_is_stable_across_reconnects, target_identity_differs_between_distinct_databases |
| Preflight: identity match | ✅ Implemented | Bypassed only with --allow-different-target.Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::apply_rejects_target_identity_mismatch |
| Preflight: drift recheck | 🟡 Partial | The plan slot exists but the executor's drift check is stubbed; the CLI's apply currently forces allow_drift = true. Phase-9 follow-up. |
| Preflight: intent approval enforcement | 🟡 Partial | The plan's intents field is loaded but the executor doesn't re-check approved = true from disk. Phase-9 follow-up. |
Preflight: [[lint_waiver]] structural validation | ✅ Implemented | Preflight validates that every [[lint_waiver]] row has non-empty rule and target. Documented limitation: does not re-run drift lints at apply time (source not available); the live-catalog recheck stub will land in a future task.Tests: tier-4: crates/pgevolve/tests/lint_waiver_e2e.rs::lint_waiver_survives_intent_toml_round_trip |
Audit row writes (open_apply_log, mark_step_*, close_apply_log) | ✅ Implemented | Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::apply_succeeds_end_to_end_and_persists_audit_rows, status_queries_return_recent_apply_with_steps |
Transactional group execution (single BEGIN…COMMIT) | ✅ Implemented | A step failure rolls back the group; every step in the group ends up failed (the offender) or rolled_back (the rest).Tests: tier-4: crates/pgevolve/tests/executor_smoke.rs::apply_rolls_back_transactional_group_on_failure |
| Autocommit group execution | ✅ Implemented | Stops on first failure; earlier steps stay succeeded.Tests: tier-4: crates/pgevolve/tests/chaos_apply.rs |
abort_after_step testkit hook (chaos harness) | ✅ Implemented | Cleanly aborts after a named step; the apply_log row goes to aborted.Tests: tier-4: crates/pgevolve/tests/chaos_apply.rs |
Real SIGKILL-mid-apply chaos | 🔮 Future | The clean-abort path covers recovery semantics; literal SIGKILL is more invasive and reserved for v0.2's chaos coverage. |
Shadow validation (pgevolve validate --shadow)
Tests (whole section): tier-4: crates/pgevolve/tests/shadow_validate.rs, shadow_validate_flag.rs, shadow_validate_views.rs, shadow_backend.rs.
| Aspect | Status | Notes |
|---|---|---|
| Ephemeral Postgres per configured major version | ✅ Implemented | testcontainers-backed; the IR is applied via the same planner + executor pipeline. Tests: tier-4: crates/pgevolve/tests/shadow_validate.rs::shadow_round_trip_succeeds_on_clean_source |
| Round-trip introspection + diff | ✅ Implemented | Mismatches are reported as line-by-line Findings on stderr.Tests: tier-4: crates/pgevolve/tests/shadow_validate_views.rs, shadow_validate_flag.rs |
--shadow without Docker | ✅ Implemented | Exits with a clear error rather than crashing inside testcontainers. Tests: tier-4: crates/pgevolve/tests/shadow_validate.rs::shadow_without_section_errors_cleanly |
Testing
pgevolve's test surface is structured in seven tiers. Each tier catches a different class of bug; together they form the gate for releases.
See ../README.md for the status legend.
Tier matrix
| Tier | What it catches | Where it lives | Status | Needs Docker |
|---|---|---|---|---|
| 1 | Unit-level invariants — IR equality, parser output for one statement, single function behavior | Inline #[cfg(test)] mod tests in every src/ file | ✅ Implemented | no |
| 2 | Fixture corpora — parsing a real *.sql snippet, comparing IR vs expected | crates/pgevolve-core/tests/parser_corpus.rs, crates/pgevolve-core/tests/parse_directory.rs | ✅ Implemented | no |
| 3 | Conformance — fixture-driven regression gate (L1–L9); see below | crates/pgevolve-conformance/ | ✅ Implemented | yes |
| 4 | Executor + CLI end-to-end — apply a plan against real PG, assert side effects + audit rows | crates/pgevolve/tests/{executor_smoke,cli_e2e,chaos_apply,shadow_validate}.rs | ✅ Implemented | yes |
| 5 | Property tests — random valid Catalogs exercised across the pipeline | crates/pgevolve-core/tests/property_tests.rs (pure) and crates/pgevolve/tests/pg_property_tests.rs (PG-bound) | ✅ Implemented | partial |
| 6 | Mutation tests — flip code, verify a test fails | not implemented | 🔮 Future | n/a |
| 7 | Soak — high-case property runs over multiple PG versions, weekly | .github/workflows/soak.yml | ✅ Implemented | yes |
Tier C (conformance) assertion layers
Tier C is the canonical regression gate. Each fixture drives the full pipeline; assertion layers are evaluated in order. Layers L1–L4 shipped with v0.1; L5–L9 landed in v0.2 readiness.
| Layer | Name | Status | Notes |
|---|---|---|---|
| L1 | parse | ✅ Implemented | Source parses cleanly. |
| L2 | lint | ✅ Implemented | No lint errors. |
| L3 | plan | ✅ Implemented | Plan produces expected steps. |
| L4 | apply | ✅ Implemented | Plan applies cleanly against real PG. Runs in-process via pgevolve::api::build_plan + pgevolve::executor::apply_plan — no subprocesses, no per-fixture binary rebuild. |
| L5 | minimality | ✅ Implemented | Re-plan after L4 apply asserts empty diff and empty plan groups. |
| L6 | no-collateral-damage | ✅ Implemented | Opt-in touches_only allow-list; asserts no unlisted objects were modified. |
| L7 | intent-shape | ✅ Implemented | Mandatory-on-destructive; matches [[expect.intent]] against the generated intent.toml. |
| L8 | dep-graph golden | ✅ Implemented | Byte-compares rendered DOT against expected/dep-graph.dot. |
| L9 | topological-order | ✅ Implemented | Declared partial orders respected by step sequence. |
Full details and authoring contract: crates/pgevolve-conformance/AUTHORING.md.
Fixture authoring subtrees
Each subtree under crates/pgevolve-conformance/fixtures/ has a specific contract:
| Subtree | Contract |
|---|---|
objects/ | One fixture per object kind / change kind combination. Exercises L1–L5 at minimum. |
scenarios/ | Multi-object, multi-step scenarios (e.g., rename dance, online rewrite sequences). |
intent/ | Destructive-change fixtures; must include [[expect.intent]] blocks (L7). |
failure/ | Fixtures that must fail at a specific phase (parse error, lint error, plan error). Uses [expect.failure]. |
regressions/ | Scaffolded by cargo xtask capture-regression; each fixture is linked to a GitHub issue. |
v0.2 view / MV fixture coverage (Tier C)
As of T11, there are 15 conformance fixtures covering views and materialized views:
| Subtree | Fixtures |
|---|---|
objects/views/ | 8 fixtures: create-simple, create-with-aliases, comment-on-view, drop, replace-body-compatible, replace-body-incompatible, security-barrier-toggle, security-invoker-toggle |
objects/materialized_views/ | 6 fixtures: create-simple, create-no-unique-index-online, index-on-mv, refresh-concurrently, replace-body, with-no-data-override |
intent/ | 1 fixture: drop-view-requires-intent |
scenarios/dependency-chains/ | 2 fixtures covering transitive view recreation |
Total: 15 fixtures across objects/views/, objects/materialized_views/, intent/, and scenarios/dependency-chains/.
fixture.toml schema additions
| Key | Status | Notes |
|---|---|---|
[budget].seconds | ✅ Implemented | Per-fixture time budget; exceeded fixtures fail in CI. |
[pg.expect].<major> | ✅ Implemented | Per-PG-major expected output overrides. |
[expect.plan.per_pg.pgN] | ✅ Implemented | Override plan expectations for a specific PG major. |
[[expect.intent]] | ✅ Implemented | L7 intent-shape matching rows. |
[expect.dep_graph] | ✅ Implemented | L8 dep-graph golden reference. |
[expect.failure] | ✅ Implemented | Phase and message for expected-failure fixtures. |
expect.plan.touches_only | ✅ Implemented | L6 collateral-damage allow-list. |
expect.plan.order | ✅ Implemented | L9 partial-order declarations. |
expect.plan.minimality | ✅ Implemented | L5 minimality assertion toggle (on by default). |
Full schema in crates/pgevolve-conformance/AUTHORING.md.
TestPgBackend pluggability
| Mechanism | Status | Notes |
|---|---|---|
PGEVOLVE_TEST_PG_MODE env var | ✅ Implemented | Selects backend: testcontainers (default), compose, or dsn. |
dev/docker-compose.pg.yml | ✅ Implemented | Ships for compose mode; pre-warms containers across test runs. |
xtask additions
| Task | Status | Notes |
|---|---|---|
cargo xtask coverage --check | --gaps | ✅ Implemented | (capability × change-kind × major) coverage matrix gate. --check fails if any cell is uncovered; --gaps prints the gap report. |
cargo xtask fixture-cost | ✅ Implemented | Per-fixture timing report; helps identify slow fixtures. |
cargo xtask capture-regression --seed <hex> --issue <n> | ✅ Implemented | Scaffold a regression fixture linked to a GitHub issue. |
cargo xtask verify-regression <fixture-dir> | ✅ Implemented | Confirm a regression fixture exercises a real bug (fails without the fix). |
cargo xtask property-status --max-age-days N | ✅ Implemented | Open-issue compliance gate; uses gh to check issue status. |
cargo xtask diagnose-pg-version <fixture-dir> --pg-major N | ✅ Implemented | Per-PG-major fixture diagnostic for version-specific failures. |
What each property test checks (Tier 5)
| Property | Status | Notes |
|---|---|---|
plan_id_is_deterministic | ✅ Implemented | Two PlanId::compute(source, target, ver, ruleset) invocations return the same bytes; different ruleset returns different bytes. Pure; no Docker.Tests: tier-5: crates/pgevolve-core/tests/property_tests.rs::plan_id_is_deterministic |
create_graph_topo_sorts_or_only_fk_cycles | ✅ Implemented | Either the create-graph topologically sorts cleanly, or every node in the cycle is an FK-bound Table / Constraint. Pure.Tests: tier-5: crates/pgevolve-core/tests/property_tests.rs |
view_canonicalization_closed_under_pg_rewrite | ✅ Implemented | v0.2. For a fixed set of representative view bodies: create the view in an ephemeral PG, query pg_get_viewdef, canonicalize both source and catalog bodies via NormalizedBody::from_sql, assert the canonical texts match. Docker-gated; #[ignore]'d. A divergence indicates a canonicalization bug.Tests: tier-5: crates/pgevolve/tests/pg_property_tests.rs |
round_trip_property | ✅ Implemented | Apply a random catalog to PG; re-introspect; assert structural equality. Needs Docker. Tests: tier-5: crates/pgevolve/tests/pg_property_tests.rs |
idempotency_property | ✅ Implemented | Diff of (applied catalog, applied catalog) is empty. Tests: tier-5: crates/pgevolve/tests/pg_property_tests.rs |
end_to_end_equivalence_property | ✅ Implemented | Apply initial; apply a random mutation; introspect; assert equal to mutated. Exercises IRMutator.Tests: tier-5: crates/pgevolve/tests/pg_property_tests.rs |
drift_recovery_property | ✅ Implemented | Apply with abort_after_step = N; re-plan from partial state; apply to completion; assert equal to target.Tests: tier-5: crates/pgevolve/tests/pg_property_tests.rs |
arb_view_dependency_graph | 🔮 Deferred | Spec §12 step 12.2. Requires a non-trivial proptest generator for arbitrary view dep graphs. Not load-bearing for the closure invariant; deferred post-v0.2. |
PGEVOLVE_PROPERTY_CASES controls the Docker-bound test case count
(default 3 locally; CI's pg-matrix uses 50; soak uses 5000).
What lives in pgevolve-testkit
| Module | Status | Notes |
|---|---|---|
EphemeralPostgres (testcontainers wrapper, PG 14-18) | ✅ Implemented | The binary has a separate ShadowPostgres for validate --shadow. |
PgCatalogQuerier (tokio_postgres-backed CatalogQuerier) | ✅ Implemented | Mirrored in the binary as pgevolve::pg_querier. |
catalog_snapshotter (canonical JSON renderer) | ✅ Implemented | Powers tier-3 goldens. |
MigrationFixture (tier-4 fixture loader) | ✅ Implemented | One seed fixture under crates/pgevolve-core/tests/fixtures/e2e/. |
arbitrary_catalog IR generator | ✅ Implemented | Schemas + tables (with bigint PK) + indexes + sequences + a curated set of column types. Richer coverage (FKs, CHECK, multi-column UNIQUE, generated columns) lands in v0.2. |
arbitrary_mutation IR mutator | ✅ Implemented | Nine mutation kinds with cascade semantics. |
assert_canonical_eq | ✅ Implemented | Wraps Catalog::diff; renders failures as an indented diff list. |
CI / soak
| Workflow | Status | Notes |
|---|---|---|
ci.yml — fmt + clippy + tier 1+2 (no Docker) on every push / PR | ✅ Implemented | |
ci.yml — pg-matrix job runs tier 3-5 on PG 14/15/16/17 with PROPTEST_CASES=50 | ✅ Implemented | Needs Docker on the runner. |
ci.yml — auto-capture on flaky property failure + property-status compliance gate | ✅ Implemented | Uses cargo xtask capture-regression and cargo xtask property-status. |
property-tests.yml — dedicated property-test workflow with coverage matrix gate | ✅ Implemented | Runs cargo xtask coverage --check on every push to main and on PRs. |
soak.yml — manual workflow_dispatch + weekly cron at PROPTEST_CASES=5000 per PG major | ✅ Implemented | |
| Mutation testing (cargo-mutants, Stryker-style) | 🔮 Future | Once the spec stabilizes; mutation testing flags rules that aren't actually tested. |
| Code coverage badges | 🔮 Future | cargo-llvm-cov is the obvious tool. |
Architecture
A guided tour of pgevolve's internals: the crates, the data flow, the key invariants, and the design decisions that shaped each.
TL;DR
pgevolve is built on a declarative IR. Source SQL and live-database
introspection both fold into the same Catalog type; the planner
computes the difference; the executor applies the difference under
strict transactional and audit guarantees.
flowchart TD
SQL["schema/*.sql"] -- parse --> AST["AST"]
AST -- "ast_resolution" --> Source["Catalog (source)"]
DB[("live Postgres")] -- introspect --> CatPair["Catalog (target) + DriftReport"]
Source --> Diff{{diff}}
CatPair --> Diff
Diff --> CS["ChangeSet"]
CS -- order --> OCS["OrderedChangeSet"]
OCS -- rewrite --> Steps["Vec<RawStep>"]
Steps -- group_steps --> Groups["Vec<TransactionGroup>"]
Groups -- "Plan::from_grouped" --> Plan["Plan"]
Plan --> PlanSql["plan.sql"]
Plan --> Intent["intent.toml"]
Plan --> Manifest["manifest.toml"]
Plan -- "apply()" --> DB
Every box and arrow is a module-level boundary. Sections below walk each.
Crate layout
crates/
├── pgevolve-core/ I/O-free library: IR, parser, diff, planner, plan format, lint
├── pgevolve-core-macros/ Proc-macro crate: `#[derive(DiffMacro)]` for IR `Diff` impls
├── pgevolve/ CLI binary + library API (the only crate that depends on tokio_postgres)
├── pgevolve-testkit/ Internal-only test infra (publish = false)
└── xtask/ `cargo xtask bless` for regenerating goldens
pgevolve-core — the brain
| Module | Responsibility |
|---|---|
identifier | Identifier (single SQL name) and QualifiedName (schema.name). Quoting / validation. |
ir/ | The data model. Catalog, Schema, Table, Column, Index, Sequence, Constraint, plus user types, functions, procedures, views, MVs, ColumnType (canonical type form), DefaultExpr, NormalizedExpr, NormalizedBody. Most IR structs derive their Diff impl via #[derive(DiffMacro)]. |
ir/canon/ | The single canonicalization pipeline. Four ordered passes (filter_pg_defaults, sentinel_view_columns, renumber_enum_sort_orders, sort_and_dedupe) that run on both source-built and catalog-read Catalogs. Catalog::canonicalize is a thin wrapper. |
parse/ | Source-side SQL → IR. Wraps pg_query. Includes ast_resolution (post-parse structural validation), ast_canon (view-body canonicalization + MV index parent promotion), and normalize_body (statement-scope canonicalizer with cross-PG-version qualifier stripping). |
catalog/ | Live-PG → IR. Defines CatalogQuerier (sync trait) and the per-version SQL strings; returns (Catalog, DriftReport) capturing NOT VALID constraints and INVALID indexes. Catalog reader produces raw IR values; PG-default elision lives in ir/canon/. The actual tokio_postgres adapter lives in the binary. |
diff/ | Catalog × Catalog → ChangeSet. Pair-by-qname semantics; destructiveness classification. Includes Change::ValidateConstraint and Change::RecreateIndex for drift recovery. |
plan/ | The planner: order → rewrite → group → write/read. Plan format and PlanId hashing. plan::edges holds DepEdge / DepSource for typed dep provenance. |
lint/ | Universal rules + four built-in layout profiles + custom-profile regex+assertion mechanism. Includes Severity::LintAtPlan tier and column-position-drift rule. |
Invariant: pgevolve-core does no I/O at the type level. The only
filesystem walk is parse::parse_directory, which is the explicit
entry point. Everything else is library-style data manipulation.
pgevolve — the binary and the runtime
| Module | Responsibility |
|---|---|
api/ | Library entry points for embedding pgevolve in other tools and tests. build_plan(schema_dir, client, opts) -> Plan runs the full parse→introspect→diff→order→rewrite→group pipeline without CLI ceremony. |
cli | clap subcommand definitions. |
commands/ | One file per subcommand. init, lint, validate, diff, plan, apply, status, bootstrap, dump (stub), graph, doctor, rewrite_table (skeleton). The plan command is a thin shim over api::build_plan; apply is a thin shim over executor::apply_plan. |
config | pgevolve.toml loader + validation. Includes [shadow] block with backend / url / extensions fields. |
connection | DSN resolution (CLI > env.url > env.url_env > PGEVOLVE_DATABASE_URL > libpq env). |
executor/ | The apply loop: bootstrap, lock, target-identity, preflight (checks [[lint_waiver]] well-formedness), audit, execute, status. Exposes apply(plan_dir, ...) and apply_plan(&Plan, ...) as library entry points. |
pg_querier | tokio_postgres-backed CatalogQuerier. Mirrors the testkit one to avoid pulling testcontainers into the binary. |
shadow/ | ShadowBackend trait + testcontainers and dsn impls. Backend selected via [shadow].backend in pgevolve.toml. Replaces the old shadow_pg.rs module. |
target_identity | BLAKE3 hash of (current_database, host, port, cluster_name, system_identifier). |
pgevolve-testkit — internal-only test infra
Holds EphemeralPostgres (testcontainers wrapper), PgCatalogQuerier
(the same adapter the binary uses, exposed for tier-3 tests), the
MigrationFixture loader, the IR generator + mutator, and the
assert_canonical_eq helper. Not published; publish = false in
Cargo.toml.
xtask — workspace-local tooling
A binary invoked via cargo xtask <subcommand>. Currently only
bless, which regenerates tier-3 catalog goldens by running the
fixtures against ephemeral containers and writing canonical JSON.
Data flow, in more detail
Parse → IR (source)
parse_directory(root, ignores):
- Walks
rootin sorted order, picking up*.sqlfiles. - Runs
pg_query::parseon each file. - Classifies every top-level statement against the v0.1 whitelist
(
CREATE SCHEMA / TABLE / INDEX / SEQUENCE, the FK-whitelistALTER TABLE,COMMENT ON). - Builds an IR object per statement.
- Tracks every object's
SourceLocationfor the linter. - AST resolution pass (
parse::ast_resolution): runs between parse and canonicalize; validates structural references (FKs, default sequences) and surfaces source-located errors. v0.1 uses structural edges only; v0.2 view/function sub-specs will addAstExtractedandAstDeclaredprovenance. - Calls
Catalog::canonicalizeat the end (sorts collections, rejects duplicate qnames).
Output: a Catalog. Optionally, with
parse_directory_with_locations, a (Catalog, HashMap<String, SourceLocation>) for the linter.
Introspect → IR (target)
pgevolve_core::catalog::read_catalog(querier, filter):
- Detects the server version (PG 14/15/16/17).
- For each
CatalogQuerykind (Schemas, Tables, Columns, etc.) picks the per-version SQL string and runs it via the querier. - Decodes rows into typed
Values, includingconvalidated(NOT VALID constraints) andindisvalid(INVALID indexes). - Assembles a
Catalogand canonicalizes. - Returns a
DriftReportalongside theCatalogcapturing NOT VALID constraints (Change::ValidateConstraint) and INVALID indexes (Change::RecreateIndex). These recover automatically from partial-apply states.
The CatalogQuerier is a synchronous trait — the binary's
PgCatalogQuerier bridges to async tokio_postgres via
block_in_place. This keeps pgevolve-core runtime-agnostic.
Diff
pgevolve_core::diff::diff(target, source) → ChangeSet:
- Tables, indexes, sequences pair by qualified name.
- Columns and constraints inside a table pair by bare name.
- Each
ChangeEntrycarries aDestructivenesstag:Safe,RequiresApproval, orRequiresApprovalAndDataLossWarning.
Planner: order
pgevolve_core::plan::order(target, source, changes) → OrderedChangeSet. Three buckets:
- Creates and additive ops — topo-sorted via the source-side dependency graph.
- Modify-in-place — same graph (column-type changes, constraint replacements).
- Drops — reverse-topo-sorted via the target-side graph.
The dependency graph has these edge sources (spec §6.4):
schema ← table ← column-default-using-sequencetable ← indexFK constraint ← both endpointssequence ← owning table (OWNED BY)
FK cycles (chicken-and-egg between two tables) are broken by
extracting offending FKs into a post-pass DeferredFkAdd list and
re-running the topo sort. The deferred FKs become ALTER TABLE ADD CONSTRAINT steps after both tables are created.
Planner: rewrite
pgevolve_core::plan::rewrite(ordered, target, policy) → Vec<RawStep>.
Each change becomes one or more RawSteps. Four documented online
rewrites (gated by PlannerPolicy):
- Concurrent index —
CREATE INDEX CONCURRENTLYfor non-unique indexes on existing tables. Runs in its own non-transactional group. - FK NOT VALID + VALIDATE — Adding an FK on an existing table splits into two steps in two transaction groups.
- CHECK NOT VALID + VALIDATE — Same shape for CHECK constraints.
- SET NOT NULL via CHECK pattern — Four-step pattern that avoids
the long
ACCESS EXCLUSIVEof a naiveSET NOT NULL.
Strategy::Atomic short-circuits every rewrite — one big transaction,
no online tricks. Useful for hermetic dev / test environments.
Planner: group
group_steps(steps) → Vec<TransactionGroup> coalesces adjacent steps
with the same TransactionConstraint. Each transactional group runs
inside one BEGIN; … COMMIT;. Non-transactional groups host
CONCURRENTLY operations (autocommit singletons).
Plan format
Plan::from_grouped assigns 1-indexed step numbers, allocates an
intent_id per destructive step, and computes the PlanId.
PlanId derivation (pgevolve_core::plan::plan::PlanId::compute):
BLAKE3(
"pgevolve-plan-id-v1\n"
|| pgevolve_version || 0x00
|| planner_ruleset_version (big-endian u32) || 0x00
|| bincode(source_catalog) || 0x00
|| bincode(target_catalog)
)
Bincode is used because its encoding is deterministic across runs and
machines. Identical inputs produce identical bytes; the hash is the
identity. serde_json was rejected here because float / map orderings
aren't byte-deterministic across versions.
Three-file on-disk format:
plan.sql— canonical artifact. Runs cleanly underpsql -f. Directive comments (-- @pgevolve ...) carry the structured data the executor needs.intent.toml— destructive intents,approved = falseby default.manifest.toml— plan id (full hex), version metadata, target identity, embedded pre-image catalog as JSON.
Executor
pgevolve::executor::apply(plan_dir, client, filter, overrides):
read_plan_dir— load the three files; cross-check the plan id.bootstrap_metadata— idempotent install ofpgevolve.*tables.try_acquire_lock—pg_try_advisory_lock(PGEVOLVE_LOCK_KEY).run_preflight— target-identity check, drift recheck, intent approval check.open_apply_log— insertapply_logrow (statusrunning), pre-populateplan_stepsaspending.execute_plan— per-group transactional or autocommit execution; audit each step's transition.close_apply_log— set statussucceeded/failed/aborted.release_lock— clear the lock row + advisory unlock.
v0.2 readiness additions
The following types and modules were added as foundation for v0.2 sub-specs (views, functions, types, etc.). They are live in the codebase but v0.1 does not yet produce the richer variants.
DepEdge / DepSource (pgevolve-core::plan::edges): dependency
edges in the planner graph are now first-class typed values. DepSource
carries provenance:
Structural— v0.1 edge sources (FK endpoints, sequence ownership, index-to-table, etc.).AstExtracted— will be emitted by the AST resolution pass once v0.2 view/function body parsing lands.AstDeclared— will be emitted when a-- @pgevolve dep:directive is present in source SQL.
NormalizedBody (pgevolve-core::parse::normalize_body): a
statement-scope canonicalizer paralleling NormalizedExpr. v0.2 body-
bearing objects (views, materialized views, functions, procedures) all
canonicalize through it. Includes a cross-PG-version pass that strips
redundant table qualifiers from column refs in single-relation
SELECT bodies — PG14's pg_get_viewdef keeps the qualifier while
PG17 strips it; this pass aligns the two.
DriftReport: returned alongside Catalog from read_catalog.
Contains NOT VALID constraints and INVALID indexes that may be present in
a database after an interrupted apply. pgevolve doctor surfaces these;
pgevolve plan emits Change::ValidateConstraint /
Change::RecreateIndex steps to resolve them.
Severity::LintAtPlan: a new lint severity tier between Warning
and Error. Plan-time gate: pgevolve plan exits 2 on any unwaived
LintAtPlan finding. The first rule at this severity is
column-position-drift. Waive via [[lint_waiver]] in intent.toml.
PlanError::BodyCycle / PlanError::AstResolution: new error
variants in the planner. v0.1 does not produce them; they exist as typed
seams so v0.2 body-bearing objects have a home to fail into.
CLI additions: graph (dep graph render), doctor (health check),
rewrite-table (skeleton; errors "not yet implemented"),
--shadow-validate / --shadow-strict flags on plan / diff /
validate.
v0.3: cross-cutting state
v0.3 ships the cross-cutting state series. Where v0.2 added new object kinds (views, types, functions, triggers, partitions), v0.3 adds new state dimensions that attach to existing IR objects:
| Release | Adds (per-object field) | Reaches |
|---|---|---|
| v0.3.0 | cluster surface: ClusterCatalog, Role, RoleAttributes | pgevolve cluster … subcommands |
| v0.3.1 | owner: Option<Identifier> + grants: Vec<Grant> | Schema, Table, View, MV, Sequence, Function, Procedure, UserType |
| v0.3.2 | rls_enabled: bool, rls_forced: bool, policies: Vec<Policy> | Table |
| v0.3.3 | storage: *StorageOptions (typed fields + extra: BTreeMap) | Table, Index, MaterializedView |
All four sub-specs share a common design pattern:
- Typed
Option<T>fields, not bare booleans or strings. Closed sets become enums; open sets get a typed field plus anextra: BTreeMap<String, String>escape hatch (reloptions). Numeric f64 fields wrap aNotNanF64newtype so they participate inEq/Hash/Ord. - Lenient drift. Source
Nonemeans "this attribute is unmanaged" and the differ produces noReset*/Revoke*/DropPolicychange. Removing a value from source therefore never causes a destructive ALTER on the catalog side. To clear a managed value, the operator issues theRESET/REVOKEout-of-band; the next plan run sees both sides asNoneand the diff is empty. - Per-feature
unmanaged-*lints. Because the differ never resets on its own, each sub-spec ships a lint that surfaces catalog values not declared in source:unmanaged-grant,unmanaged-policy,unmanaged-reloption. All warnings, waivable via[[lint_waiver]]inintent.toml. - One
Set*Changevariant per dimension. No pairedReset*variant — the lenient policy makes the reset path unreachable. The differ emits a sparse delta: only the fields where source isSome(_)AND target disagrees flow into the change.
Known limitation: new objects don't carry cross-cutting state inline
When the differ encounters an object in source that the target catalog
doesn't have, it emits a single Change::CreateTable / CreateIndex /
CreateMaterializedView step. The corresponding renderer (sql::create_table,
sql::create_index, etc.) emits the bare CREATE … (…) statement —
without the WITH (…) reloption clause, without ALTER … OWNER TO,
without inline GRANTs or CREATE POLICY companions, and without
ENABLE ROW LEVEL SECURITY.
The state lands on the second plan run: after apply, the catalog
reader sees the new object with PG-default attributes, the differ
notices source declares non-default values, and emits the appropriate
AlterObjectOwner / GrantObjectPrivilege / CreatePolicy /
SetTableStorage steps. Convergent in two plan iterations.
A uniform fix — emitting cross-cutting state inline at object creation
or in companion steps adjacent to the Create* — is tracked for a
future v0.3.x maintenance release. Until then, operators authoring
brand-new objects with cross-cutting state should expect to run
pgevolve plan and pgevolve apply twice to fully converge.
The canonicalization pipeline
Both the source parser and the catalog reader produce Catalog values
that may carry IR fields PG fills in implicitly (sequence min/max,
function cost/rows, pg_catalog.default collations, fractional enum
sort orders, redundant view-column type info). For source and catalog
to compare equal, both sides must run an identical canonicalization
pass.
Catalog::canonicalize is a thin wrapper around ir::canon::canonicalize
which runs four ordered passes, in this order:
filter_pg_defaults— values equal to PG's documented defaults becomeNone(sequence min/max, function cost/rows, columnpg_catalog.defaultcollation).sentinel_view_columns— view/MV column types collapse to a sharedview_columnsentinel. Body changes are captured bybody_canonical(an AST hash); per-output-column types are redundant and unresolvable from source statically.renumber_enum_sort_orders— each enum'ssort_ordervalues are re-indexed to1.0, 2.0, 3.0, …in current order.sort_and_dedupe— every IR collection is sorted by its canonical key and duplicates raiseIrError. Runs last so duplicate detection sees post-normalization values.
When PG returns a default we hadn't expected, the fix lands in one of those four passes. Catalog readers and source builders are kept "raw" — they never filter — so the rule is discoverable in one place.
Library API for embedding pgevolve
pgevolve exposes a small library surface for tools and tests that
need to run plan/apply in-process rather than spawning the binary:
pgevolve::api::build_plan(schema_dir, client, opts) -> Planconsumes atokio_postgres::Client, runs the full parse→introspect→diff→order→rewrite→group pipeline, and returns aPlanvalue. Noprintln!, no waiver-prompt UX, no--shadow-validate, no on-disk plan directory.pgevolve::executor::apply_plan(&Plan, &mut client, &filter, overrides)applies an in-memoryPlan. The disk-basedexecutor::apply(path, &mut client, ...)is a thin shim that callsread_plan_dirthen delegates toapply_plan.Plan::approve_all_intents()(onpgevolve_core::plan::Plan) marks every destructive intent as approved. For test harnesses that build plans programmatically; production apply still requiresintent.toml-based approval.
The conformance suite uses these entry points exclusively — it spawns
no subprocesses. The CLI commands (pgevolve plan, pgevolve apply)
are themselves thin wrappers over these library entry points plus
CLI-only UX (printing, exit codes).
Key invariants
These are testable, must-hold-or-the-build-breaks properties.
Catalog::diffis byte-deterministic. Identical IRs produce an empty diff. Two different IRs always produce the same diff.PlanId::computeis byte-deterministic. Same inputs ⇒ same id, on any machine.write_plan_dirthenread_plan_dirround-trips (modulo destructive_reason, which is grafted fromintent.toml).- Topological order is deterministic. Ties broken by the smallest
node per
Ord; the planner's output is byte-stable. - No I/O in
pgevolve-coreat the type level. The only fs walk is the explicitparse_directory. - The advisory lock is singleton.
try_acquire_locksucceeds for at most one session at a time. Property-tested. - No partial success. Apply either succeeds end-to-end or reports
the exact failed step in
pgevolve.plan_steps. - No silent data loss. Destructive steps require approved
intents; pre-flight refuses to run with
approved = false.
Design decisions worth knowing
Why an IR (and not just diff SQL text)?
Postgres has many ways to write the same thing: 'foo'::text vs
'foo', NUMERIC vs NUMERIC(38, 0), int4 vs integer. A
text-level diff would noise-trip on every cosmetic difference. The IR
canonicalizes — paren folding, keyword case, type aliases, etc. — so
that semantically-equal inputs produce equal Catalog values.
Why three files in a plan directory (vs. one)?
plan.sqlis the review artifact. Reviewers read SQL.intent.tomlis the approval artifact. The diff in a PR forintent.tomlis the exact destructive change being authorized.manifest.tomlis the integrity artifact. The embedded pre-image- full hex hash + plan-id cross-check means the executor can refuse to run a tampered plan.
Splitting these means the right people review the right surface.
Why three-phase ordering (vs. one topological sort)?
Drops have to run in reverse of creates. Modify ops can reference either pre- or post-image. Splitting into three buckets with two graphs (source for creates/modifies, target for drops) is the smallest model that handles every case correctly.
Why FK-cycle extraction (vs. deferred constraints or topological-sort failures)?
Inline FKs in CREATE TABLE create chicken-and-egg cycles when two
tables FK each other. Postgres supports DEFERRABLE constraints, but
that's a runtime semantics shift and not all FKs are deferrable.
Extracting the offending FKs into ALTER TABLE ADD CONSTRAINT after
both tables exist is the surgical fix.
Why bincode for PlanId?
The hash payload doesn't need to be human-readable. Bincode is binary, deterministic, and several times faster than the alternatives. Note: pinned to v2 because v3 dropped the serde feature.
Why does pgevolve-core not depend on tokio_postgres?
Keeps the library testable without a running runtime, and makes it
plausible to add other backends (file-based, raw libpq, etc.) without
restructuring. The CatalogQuerier trait is the integration point;
the binary's pg_querier is the only impl today.
Why are advisory locks session-scoped, not transaction-scoped?
Apply spans multiple transactions (e.g., one transactional group + one autocommit group). A transaction-scoped lock would release between groups; a session-scoped one stays held for the whole apply.
Why does validate --shadow re-implement parts of apply?
Because it has to apply the source IR to a fresh database from
scratch, with target_identity set to the live shadow's identity (not
whatever was in the source pgevolve.toml). It builds a plan
in-memory and writes to a tempdir, then calls the same executor::apply
the regular apply command uses.
Where each invariant is tested
| Invariant | Test |
|---|---|
| Diff determinism | Tier 1 unit tests in diff/ + tier 5 property test plan_id_is_deterministic (which transitively requires diff determinism). |
PlanId determinism | plan_id_is_deterministic property test. |
| Plan round-trip | read_plan_dir_round_trips_whole_plan (unit) + round_trip_property (PG-bound property test). |
| Topo-sort determinism | deterministic_under_insertion_order_changes + property test on ordered changes. |
pgevolve-core no-I/O | Compile-time: pgevolve-core has no tokio / tokio_postgres in its deps. |
| Lock singleton | advisory_lock_contention tier-4 test. |
| No partial success | apply_rolls_back_transactional_group_on_failure tier-4 test. |
| No silent data loss | Intent approval is checked at preflight (test pending; phase-9 follow-up). |
IR
Deeper dive on the data model that everything diffs against. Source for
this doc lives in crates/pgevolve-core/src/ir/.
Top-level shape
#![allow(unused)] fn main() { pub struct Catalog { pub schemas: Vec<Schema>, pub extensions: Vec<Extension>, pub tables: Vec<Table>, pub indexes: Vec<Index>, pub sequences: Vec<Sequence>, pub views: Vec<View>, pub materialized_views: Vec<MaterializedView>, pub types: Vec<UserType>, pub functions: Vec<Function>, pub procedures: Vec<Procedure>, pub triggers: Vec<Trigger>, pub default_privileges: Vec<DefaultPrivilegeRule>, } }
Flat collections, no nesting (e.g., Table::indexes) for a reason:
indexes live in their own namespace within a schema and reference their
table by qname. Hierarchical nesting would have to maintain referential
integrity at every mutation; flat lists + name-based references defer
that to canonicalization time.
Cluster-level state (Role, RoleAttributes) lives in a sibling
ClusterCatalog rather than on Catalog, since cluster surface is
managed via the separate pgevolve cluster … subcommand family. See
docs/spec/cluster.md.
The implicit name-based relationships look like this:
erDiagram
Catalog ||--o{ Schema : "schemas[]"
Catalog ||--o{ Table : "tables[]"
Catalog ||--o{ Index : "indexes[]"
Catalog ||--o{ Sequence : "sequences[]"
Catalog ||--o{ View : "views[]"
Catalog ||--o{ MaterializedView : "materialized_views[]"
Table ||--o{ Column : "columns[]"
Table ||--o{ Constraint : "constraints[]"
View ||--o{ ViewColumn : "columns[]"
MaterializedView ||--o{ ViewColumn : "columns[]"
Index }o--|| Table : "table (qname)"
Sequence }o--o| Table : "owned_by (optional)"
Schema ||--o{ Table : "qname.schema"
Schema ||--o{ Index : "qname.schema"
Schema ||--o{ Sequence : "qname.schema"
Schema ||--o{ View : "qname.schema"
Schema ||--o{ MaterializedView : "qname.schema"
Canonicalization
Catalog::canonicalize() delegates to ir::canon::canonicalize, which
runs four ordered passes:
filter_pg_defaults— IR field values that match PG's documented defaults becomeNone(sequence min/max, function cost/rows, columnpg_catalog.defaultcollation). Both source-built and catalog-readCatalogs pass through this, so a function declared withoutCOSTround-trips byte-equal with the catalog reading of the same function.sentinel_view_columns— view and materialized-view column types collapse to a sharedview_columnsentinel. Body changes are already captured bybody_canonical(an AST hash); per-column types are redundant info derived from the body.renumber_enum_sort_orders— each enum'ssort_ordervalues are re-indexed to1.0, 2.0, 3.0, …in current order. PG stores floats; source assigns sequential 1-indexed; this pass aligns the two.sort_and_dedupe— sort each collection by its canonical key (schema.name,qname, etc.); reject duplicates. Runs last so duplicate detection sees the post-normalization state.
The output is byte-stable: identical inputs always produce identical
serialized output. This is what makes PlanId deterministic.
When PG returns a default we hadn't expected, the fix lands in one of
the four passes (most commonly filter_pg_defaults). Catalog readers
and source builders are kept "raw" — they never filter — so the rule
is discoverable in one place.
Failure modes (only sort_and_dedupe is fallible):
IrError::InvalidIdentifier("duplicate schema: foo")— twoSchemas with the same name.- Same for tables / indexes / sequences / views / MVs / types / functions / procedures.
Diff derive
Most IR structs derive their Diff impl rather than hand-writing it:
#![allow(unused)] fn main() { #[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, DiffMacro)] pub struct Sequence { pub qname: QualifiedName, // default strategy (Display) #[diff(via_debug)] pub data_type: ColumnType, pub start: i64, // ... #[diff(via_debug)] pub min_value: Option<i64>, // ... } }
The derive (from the pgevolve-core-macros crate, re-exported as
pgevolve_core::ir::eq::DiffMacro) supports three field attributes:
- no attribute — emit
diff_field(name, &self.x, &other.x). RequiresPartialEq + Display. #[diff(skip)]— omit the field entirely.#[diff(via_debug)]— emitdiff_field(name, format!("{:?}", ...)). ForOption<T>,Vec<T>, enums withoutDisplay.#[diff(nested)]— emitprefix_diffs(name, self.x.diff(&other.x)). For fields whose type already implementsDiff.
Hand-written impls remain for: Catalog (orchestrates diff_keyed
over many Vec<T> collections), Function (custom qname(args) key
in path), Table / View / MaterializedView (pair columns by name
with order-drift reporting), UserType (intentional dump-all on
inequality), and the enum impls (ConstraintKind, DefaultExpr,
ColumnType).
Identifier and QualifiedName
Identifier is a single SQL identifier. Two constructors:
Identifier::from_unquoted(s)— accepts[a-z_][a-z0-9_$]*shapes; rejects anything that would need quoting. This is what the parser and the user-facing config consume.Identifier::from_quoted(s)— accepts the body of a double-quoted identifier (with""→"already unescaped).
render_sql() returns the canonical SQL form, quoting only when
necessary. Two Identifiers compare equal iff their as_str()
representations match.
QualifiedName { schema, name } — schema-qualified schema.name.
Used wherever a top-level object name appears.
ColumnType
The single most fact-laden type in the IR. ColumnType is the
canonical form of a Postgres column type:
#![allow(unused)] fn main() { pub enum ColumnType { Boolean, SmallInt, Integer, BigInt, Real, DoublePrecision, Numeric { precision: Option<u16>, scale: Option<i16> }, Text, Varchar { len: Option<u32> }, Char { len: Option<u32> }, Bytea, Date, Time { precision: Option<u8>, with_tz: bool }, Timestamp { precision: Option<u8>, with_tz: bool }, Interval { fields: Option<String>, precision: Option<u8> }, Bit { len: u32, varying: bool }, Uuid, Json, Jsonb, NetAddress(NetAddressKind), Array { element: Box<ColumnType>, dims: u8 }, UserDefined(QualifiedName), Other { raw: String }, } }
Why the canonical form matters
PG accepts many spellings for the same type:
| Source spelling | Catalog form | Canonical IR |
|---|---|---|
int | integer | ColumnType::Integer |
int4 | integer | ColumnType::Integer |
decimal | numeric | ColumnType::Numeric { .. } |
bool | boolean | ColumnType::Boolean |
timestamptz | timestamp with time zone | ColumnType::Timestamp { with_tz: true, .. } |
varchar (no length) | character varying (no length) | ColumnType::Varchar { len: None } |
ColumnType::parse_from_pg_type_string does the source-side and
catalog-side normalization to the same canonical form, so diff
operates on equality rather than on textual difference.
Other and UserDefined — the escape hatches
Other { raw: String }— types pgevolve doesn't recognize. Treated as opaque strings; twoOthers match iff their raw strings match byte-for-byte. This lets the parser handle types it doesn't understand instead of aborting.UserDefined(QualifiedName)— qualified references to user-defined types (enums, domains, composites). v0.1 doesn't introspect their structure; v0.2 will.
DefaultExpr
#![allow(unused)] fn main() { pub enum DefaultExpr { Literal(LiteralValue), Sequence(QualifiedName), Expr(NormalizedExpr), } }
Literal— a typed literal (bool, integer, float, text, bytea, NULL).Sequence— detected fromnextval('schema.seq'::regclass)or the barenextval('schema.seq'). Both forms normalize to the sameQualifiedName.Expr(NormalizedExpr)— any other expression, preserved as canonical text.
NormalizedExpr
#![allow(unused)] fn main() { pub struct NormalizedExpr { pub canonical_text: String, pub ast_hash: [u8; 32], } }
The canonical text is what you get after:
- Lowercasing keywords.
- Sorting operands of commutative operators (
a + b≡b + a). - Stripping redundant casts (
'foo'::text→'foo'if the column is alreadytext). - Folding redundant parens.
Two NormalizedExprs compare equal iff their canonical_texts match.
The ast_hash is a BLAKE3 hash of the canonical text, kept for fast
equality checks and as a stable identity for an expression.
NormalizedBody
#![allow(unused)] fn main() { pub struct NormalizedBody { pub canonical_text: String, pub canonical_hash: [u8; 32], } }
NormalizedBody is the statement-scope counterpart to NormalizedExpr.
Where NormalizedExpr canonicalizes a single expression (e.g., a column
default or a CHECK predicate), NormalizedBody canonicalizes the body
of a body-bearing object — a view's SELECT statement, a function's
body, a trigger's action. The same canonicalization rules apply (keyword
case, redundant parens, etc.), plus one cross-version normalization:
for any SelectStmt whose FROM names exactly one relation, the
canonicalizer rewrites ColumnRef [<from-alias>, <col>] → [<col>].
PG14's pg_get_viewdef keeps the redundant qualifier while PG17 strips
it; this pass makes both forms equal so the differ doesn't see a
phantom body change on PG14.
In v0.1 no objects carry a body; NormalizedBody is scaffolding for
v0.2 views and functions. It is kept in pgevolve-core::parse::normalize_body
so body-bearing objects added by v0.2 sub-specs can reuse the same
diffing semantics as NormalizedExpr.
Table
#![allow(unused)] fn main() { pub struct Table { pub qname: QualifiedName, pub columns: Vec<Column>, pub constraints: Vec<Constraint>, pub partition_by: Option<PartitionBy>, // partitioned parent pub partition_of: Option<PartitionOf>, // partition child pub comment: Option<String>, // --- v0.3 cross-cutting state --------------------------------- pub owner: Option<Identifier>, // v0.3.1: None = unmanaged pub grants: Vec<Grant>, // v0.3.1: empty = no grants pub rls_enabled: bool, // v0.3.2: PG default false pub rls_forced: bool, // v0.3.2: PG default false pub policies: Vec<Policy>, // v0.3.2: RLS policies pub storage: TableStorageOptions, // v0.3.3: WITH (…) } }
The first six fields are the v0.1/v0.2 surface (structure, columns,
constraints, partitioning, comment). The last six are
v0.3 cross-cutting state —
each one follows the lenient-drift convention: Option<T> / Vec<T> /
bool where the absent / empty / false state means "unmanaged."
Index and MaterializedView carry a subset of the same v0.3 fields:
Indexaddsstorage: IndexStorageOptions.MaterializedViewaddsowner,grants, andstorage(MaterializedViewStorageOptions = TableStorageOptions).
Grant, Policy, *StorageOptions
Defined in crates/pgevolve-core/src/ir/{grant,policy,reloptions}.rs.
Grant { grantee, privileges, with_grant_option, columns: Option<Vec<Identifier>> }.columns = Some(_)is a column-level grant on a table/view/MV;Noneis object-level. Canonicalized: stable sort by(grantee, privileges).Policy { name, command, permissive, roles, using, with_check }. USING / WITH CHECK reuseNormalizedExpr(same canon as CHECK constraints). Command-kind changes (e.g.,FOR SELECT→FOR UPDATE) diff as DROP + CREATE.TableStorageOptions/IndexStorageOptions— typedOption<T>fields for the well-known reloption keys (fillfactor, autovacuum_*, parallel_workers, fastupdate, buffering, pages_per_range, …) plusextra: BTreeMap<String, String>for extension or unknown keys. f64 fields wrapNotNanF64soEq/Hash/Ordwork. Per-AM fillfactor ranges are enforced at parse time (seedocs/spec/reloptions.md).
Column attributes
#![allow(unused)] fn main() { pub struct Column { pub name: Identifier, pub ty: ColumnType, pub nullable: bool, pub default: Option<DefaultExpr>, pub identity: Option<Identity>, pub generated: Option<Generated>, pub collation: Option<QualifiedName>, pub comment: Option<String>, } }
nullable = falsecorresponds toNOT NULL— modeled as a column-level boolean rather than as aConstraintbecause it's significantly cheaper to diff.identity—GENERATED ALWAYS / BY DEFAULT AS IDENTITYwith the backing sequence options.generated—GENERATED ALWAYS AS (expr) STORED. Postgres doesn't yet supportVIRTUAL.collation— only the explicit collation; the catalog reader normalizespg_catalog.defaulttoNoneso it doesn't appear as drift on every text column.
Constraint
#![allow(unused)] fn main() { pub struct Constraint { pub qname: QualifiedName, pub kind: ConstraintKind, pub deferrable: Deferrable, pub comment: Option<String>, } pub enum ConstraintKind { PrimaryKey { columns: Vec<Identifier>, include: Vec<Identifier> }, Unique { columns: Vec<Identifier>, include: Vec<Identifier>, nulls_distinct: bool }, ForeignKey(ForeignKey), Check { expression: NormalizedExpr, no_inherit: bool }, } }
Constraints are paired by qname.name (within a table) during diff.
Two constraints with the same name but different bodies diff as a
"replace" — pgevolve emits DROP CONSTRAINT + ADD CONSTRAINT.
Index
#![allow(unused)] fn main() { pub struct Index { pub qname: QualifiedName, pub table: QualifiedName, pub method: IndexMethod, pub columns: Vec<IndexColumn>, pub include: Vec<Identifier>, pub unique: bool, pub nulls_not_distinct: bool, pub predicate: Option<NormalizedExpr>, pub tablespace: Option<Identifier>, pub comment: Option<String>, } }
Indexes are first-class IR objects (paired by their own qname, not by their backing table). This makes "rename the index" or "change the opclass on column 2" a single-row diff entry.
View and MaterializedView
Added in v0.2. Source: crates/pgevolve-core/src/ir/view.rs.
#![allow(unused)] fn main() { pub struct View { pub qname: QualifiedName, pub columns: Vec<ViewColumn>, pub body_canonical: NormalizedBody, pub body_dependencies: Vec<DepEdge>, pub security_barrier: Option<bool>, pub security_invoker: Option<bool>, pub comment: Option<String>, // raw_body: parser-internal sentinel; not serialized. } pub struct MaterializedView { pub qname: QualifiedName, pub columns: Vec<ViewColumn>, pub body_canonical: NormalizedBody, pub body_dependencies: Vec<DepEdge>, pub comment: Option<String>, // raw_body: parser-internal sentinel; not serialized. } }
body_canonical: NormalizedBody
The canonicalized SELECT body. NormalizedBody::from_sql (in
parse/normalize_body.rs) feeds the raw SQL through pg_query's
parse + deparse cycle, strips redundant table-qualifier prefixes from
column refs whose FROM names a single relation (see NormalizedBody
above), and collapses whitespace. The same function is called on the
source side (T3/T4 parse pass) and the catalog side (T5 reader, which
calls pg_get_viewdef). Because both sides go through the same
normalization, the differ compares canonical texts directly without
knowing anything about SQL semantics.
canonical_hash (BLAKE3 of the text, domain-separated with
pgevolve-normalized-body-v1\n) is kept for fast equality checks and
stable identity.
body_dependencies: Vec<DepEdge>
Dependency edges extracted from the body AST by the T4 AST canonicalization
pass (parse/ast_canon.rs). Each DepEdge has:
#![allow(unused)] fn main() { pub struct DepEdge { pub from: NodeId, // NodeId::View or NodeId::Mv pub to: NodeId, // NodeId::Table, NodeId::View, or NodeId::Mv pub source: DepSource, // DepSource::AstExtracted } }
body_dependencies is what makes the planner's dependent-recreation walk
possible (see plan/recreate_views.rs). It is also what the
view-body-references-unmanaged-schema lint rule checks.
ViewColumn
#![allow(unused)] fn main() { pub struct ViewColumn { pub name: Identifier, pub column_type: ColumnType, pub comment: Option<String>, } }
A single named column in a view or materialized view. When constructed
from the source parser (T3), column_type is set to
ColumnType::Other { raw: "unresolved" } as a sentinel; the T4 AST
canonicalization pass fills in the resolved type. When built from the
live catalog (T5), column_type is parsed from
format_type(a.atttypid, a.atttypmod).
UserType
#![allow(unused)] fn main() { pub struct UserType { pub qname: QualifiedName, pub kind: UserTypeKind, pub comment: Option<String>, } pub enum UserTypeKind { Enum { values: Vec<EnumValue> }, Domain { base: ColumnType, nullable: bool, default: Option<NormalizedExpr>, check_constraints: Vec<DomainCheck>, collation: Option<QualifiedName> }, Composite { attributes: Vec<CompositeAttribute> }, } }
UserTypes live in Catalog::types: Vec<UserType>, sorted by qname after
canonicalize(). Source lives in crates/pgevolve-core/src/ir/user_type.rs.
EnumValue
#![allow(unused)] fn main() { pub struct EnumValue { pub name: String, pub sort_order: f32, // mirrors pg_enum.enumsortorder } }
sort_order is f32 (matching Postgres's real4) to enable byte-stable
round-trip. Eq and Hash are implemented using the IEEE 754 bit pattern.
DomainCheck
#![allow(unused)] fn main() { pub struct DomainCheck { pub name: Identifier, pub expression: NormalizedExpr, } }
Domain defaults and CHECK expressions use NormalizedExpr — the same
canonicalized-text representation as column defaults and inline CHECK
constraints. Two NormalizedExprs compare equal iff their canonical_texts
match, making domain diffs insensitive to whitespace and keyword case.
CompositeAttribute
#![allow(unused)] fn main() { pub struct CompositeAttribute { pub name: Identifier, pub ty: ColumnType, pub collation: Option<QualifiedName>, } }
Function
Added in v0.2. Source: crates/pgevolve-core/src/ir/function.rs.
#![allow(unused)] fn main() { pub struct Function { pub qname: QualifiedName, pub args: Vec<FunctionArg>, pub arg_types_normalized: NormalizedArgTypes, pub return_type: ReturnType, pub language: FunctionLanguage, pub body: NormalizedBody, pub body_dependencies: Vec<DepEdge>, pub volatility: Volatility, pub strict: bool, pub security: SecurityMode, pub parallel: ParallelSafety, pub leakproof: bool, pub cost: Option<f32>, pub rows: Option<f32>, pub comment: Option<String>, } }
Functions live in Catalog::functions: Vec<Function>, sorted by (qname, arg_types_normalized) after canonicalize().
Identity rule
Function identity is (qname, arg_types_normalized). arg_types_normalized covers the IN/INOUT/VARIADIC args only (mirrors Postgres's proargtypes), enabling overloads with the same name but different input signatures to coexist.
NormalizedArgTypes stores the canonical type list and a BLAKE3 hash of the comma-joined type strings for fast equality and ordering.
body_dependencies: Vec<DepEdge>
Dependency edges extracted from the body AST by the T4 body parser (parse/builder/plpgsql.rs):
- SQL bodies: extracted from
RangeVarnodes in the SQL AST (schema-qualified references only). - PL/pgSQL bodies: extracted from static embedded SQL statements (
PLpgSQL_stmt_execsql). Dynamic SQL (EXECUTE) edges must be declared explicitly via-- @pgevolve dep: schema.namedirectives; undeclaredEXECUTEsites fire theplpgsql-dynamic-sqllint rule.
DepEdge.source is DepSource::AstExtracted for parsed edges and DepSource::AstDeclared for directive edges.
Procedure
Added in v0.2. Source: crates/pgevolve-core/src/ir/procedure.rs.
#![allow(unused)] fn main() { pub struct Procedure { pub qname: QualifiedName, pub args: Vec<FunctionArg>, pub language: FunctionLanguage, pub body: NormalizedBody, pub body_dependencies: Vec<DepEdge>, pub security: SecurityMode, pub commits_in_body: bool, pub comment: Option<String>, } }
Procedures live in Catalog::procedures: Vec<Procedure>, sorted by qname after canonicalize().
Identity rule
Procedure identity is qname only (no arg-type disambiguation). pgevolve v0.2 deliberately restricts procedures to a single definition per qualified name. This simplifies the plan-format and intent model; overloading can be added in a future sub-spec.
commits_in_body
Set to true by the PL/pgSQL body parser when it detects PLpgSQL_stmt_commit or PLpgSQL_stmt_rollback nodes anywhere in the body AST (including inside IF, LOOP, etc.). The planner uses this flag to emit the step with TransactionConstraint::OutsideTransaction, since a procedure containing COMMIT/ROLLBACK cannot run inside an outer BEGIN … COMMIT block.
Sequence
#![allow(unused)] fn main() { pub struct Sequence { pub qname: QualifiedName, pub data_type: ColumnType, // SmallInt / Integer / BigInt pub start: i64, pub increment: i64, pub min_value: Option<i64>, // None == PG type-default pub max_value: Option<i64>, pub cache: i64, pub cycle: bool, pub owned_by: Option<SequenceOwner>, pub comment: Option<String>, } }
The catalog reader normalizes PG's per-type defaults for min_value /
max_value to None (the same reasoning as collation normalization).
What's deliberately not in the IR
NOT VALIDconstraints. The IR represents only fully-validated constraints;NOT VALIDis an intermediate planner artifact.- Auto-generated index names. All indexes must be named in
source. Constraint-backing indexes are tied to the constraint, not
modeled as separate
Indexes. - Row data. pgevolve never reads or writes table contents.
postgresql.confsettings. Roles are in scope as of v0.3.0 via the separateClusterCatalog; cluster-level GUCs and tablespaces remain out of scope.pg_catalog/information_schema— unmanaged schemas don't appear in the IR.
How the diff walks the IR
Catalog::diff(other) -> Vec<Difference>:
- Pair-by-key over each top-level collection (
schemas,tables,indexes,sequences). - For paired objects, recurse into nested collections (columns, constraints).
- For each unmatched-on-the-left key, emit
present → removed. - For each unmatched-on-the-right key, emit
missing → added. - For each matched pair, recurse into the per-field diff.
The output Vec<Difference> is flat: every leaf change has a
slash-or-dot path like tables.app.users.columns.email.nullable. The
linter uses this for findings; the differ converts it into a
ChangeSet of higher-level Change enum variants.
Planner
The planner turns a ChangeSet into a flat, applyable
Vec<TransactionGroup>. Four sub-phases: order → rewrite → group →
wrap. Source lives in crates/pgevolve-core/src/plan/.
flowchart LR
CS["ChangeSet"] --> Order["1. order"]
Order --> OCS["OrderedChangeSet"]
OCS --> Rewrite["2. rewrite"]
Rewrite --> Steps["Vec<RawStep>"]
Steps --> Group["3. group"]
Group --> TGs["Vec<TransactionGroup>"]
TGs --> Wrap["4. wrap"]
Wrap --> Plan["Plan"]
Sub-phase 1: order
order(target, source, changes) → OrderedChangeSet.
The output is three buckets:
#![allow(unused)] fn main() { pub struct OrderedChangeSet { pub creates_and_adds: Vec<ChangeEntry>, pub modifies: Vec<ChangeEntry>, pub drops: Vec<ChangeEntry>, pub deferred_fks: Vec<DeferredFkAdd>, } }
Each bucket is independently topologically sorted. The graph has the edge sources from spec §6.4:
schema ← table ← default-using-sequencetable ← indexFK constraint ← both endpoints (owning + referenced)sequence ← owning table (OWNED BY)
Creates and modifies use the source-side graph; drops use the target-side graph (reversed, so dependents drop before what they depend on).
Determinism
The graph itself uses BTreeMap / BTreeSet. Kahn's algorithm with a
min-heap by Ord breaks ties: when multiple nodes are simultaneously
eligible, the smallest one (by Ord on NodeId) wins. Result: the
same (target, source, changes) triple always produces the same
ordered output.
FK cycle extraction
Inline FKs in CREATE TABLE introduce a chicken-and-egg cycle when
table A FKs table B and B FKs A. The planner:
flowchart TD
Start([order(target, source, changes)]) --> Topo1["topological sort"]
Topo1 --> Cycle1{Cycle?}
Cycle1 -- No --> Done([emit OrderedChangeSet])
Cycle1 -- Yes --> Strip["strip FKs spanning cycle nodes<br/>(self-FKs exempt)"]
Strip --> Topo2["topological sort (reduced)"]
Topo2 --> Cycle2{Cycle?}
Cycle2 -- No --> Defer["emit DeferredFkAdd entries"]
Cycle2 -- Yes --> Fail([PlanError::UnbreakableCycle])
Defer --> Done
- Runs the topological sort.
- On
Err(Cycle { nodes }), walksreduced = sourceand removes every FK constraint whose owning + referenced tables are both in the cycle (self-referential FKs are exempt because they don't induce a graph cycle — the row insertion happens after the table exists). - Re-runs the topological sort on the reduced graph.
- The extracted FKs become
DeferredFkAddentries — emitted asALTER TABLE ... ADD CONSTRAINTsteps after both tables are created. - If the second topo sort also cycles, returns
PlanError::UnbreakableCyclewith the rendered node names.
To avoid emitting the same FK twice (inline and deferred),
strip_deferred_fks removes the matching constraint from every
relevant Change::CreateTable body.
Sub-phase 2: rewrite
rewrite(ordered, target, policy) → Vec<RawStep>.
Each top-level Change dispatches to an emitter that produces one or
more RawSteps. The bulk of this is straightforward SQL generation
(pgevolve_core::plan::rewrite::sql); the interesting bits are the
four online rewrites plus two drift-recovery variants.
Drift-recovery changes
read_catalog now returns a DriftReport alongside the Catalog.
The diff stage folds these into the ChangeSet as two additional
Change variants:
Change::ValidateConstraint — emitted when the catalog contains a
NOT VALID constraint (left behind by an interrupted FK / CHECK NOT VALID
apply). The rewrite emits:
ALTER TABLE <schema>.<table> VALIDATE CONSTRAINT <name>;
This step is InTransaction and safe; it's equivalent to resuming the
second half of the FK NOT VALID + VALIDATE online rewrite.
Change::RecreateIndex — emitted when the catalog contains an
INVALID index (left behind by a failed CREATE INDEX CONCURRENTLY). The
rewrite emits:
DROP INDEX CONCURRENTLY IF EXISTS <schema>.<name>;
CREATE INDEX CONCURRENTLY <name> ON <schema>.<table> (...);
Both steps are OutsideTransaction (autocommit), consistent with the
normal concurrent-index handling.
Online rewrite 1: CREATE INDEX CONCURRENTLY / DROP INDEX CONCURRENTLY
plan::rewrite::concurrent_index::should_rewrite_create:
#![allow(unused)] fn main() { policy.create_index_concurrent() && target.table_exists(&idx.table) && !idx.unique }
Three gating conditions:
- Policy enables the rewrite.
- The target table already exists (i.e., is not being created in this plan — concurrent create on a brand-new table would be wasted transaction-isolation work).
- The index is not
UNIQUE. A failedCREATE UNIQUE INDEX CONCURRENTLYleaves an INVALID index that has to be cleaned up out-of-band; v0.1 plays it safe.
When all three hold, the rewrite emits the step with
TransactionConstraint::OutsideTransaction. group_steps then puts
it in its own non-transactional group. DROP INDEX mirrors:
should_rewrite_drop reads the index's uniqueness from the target
catalog (since the index doesn't exist on the source side), and emits
DROP INDEX CONCURRENTLY under the same conditions.
Online rewrite 2 + 3: NOT VALID + VALIDATE (FK and CHECK)
plan::rewrite::fk_not_valid_validate::should_rewrite:
#![allow(unused)] fn main() { policy.fk_not_valid_then_validate() && matches!(c.kind, ConstraintKind::ForeignKey(_)) && target.table_exists(qname) }
Same three-condition shape. The rewrite emits two steps:
#![allow(unused)] fn main() { [ RawStep { kind: AddConstraintNotValid, transactional: InTransaction, … }, RawStep { kind: ValidateConstraint, transactional: InTransaction, … }, ] }
Both steps are InTransaction, but they live in separate
TransactionGroups at grouping time. That separation is the whole
point: step A (the cheap NOT VALID addition) commits independently
of step B (the expensive scan-the-table validation). If step B fails,
step A stays committed and the user re-plans only the validation.
How the separation happens: each RawStep is InTransaction, but
group_steps coalesces only adjacent steps that share their
TransactionConstraint. The grouping function doesn't currently split
adjacent InTransaction steps, so this is actually achieved by
emitting them with intervening steps that flip the constraint —
or by the executor's group boundary semantics treating each
ValidateConstraint as its own commit point.
Current implementation honesty. As of v0.1,
mark_step_succeededcommits the audit row inside the same transaction as the DDL. A later step's failure rolls back the entire group's DDL and the audit updates, then re-marks them outside the transaction. The NOT VALID / VALIDATE separation is structural in the plan (the directives saygroup id=1thengroup id=2) but the executor's per-group transaction boundaries are what actually commit step A before step B starts.
CHECK rewrite (check_not_valid_validate.rs) is the same shape, just
matching on ConstraintKind::Check { .. }.
Online rewrite 4: SET NOT NULL via CHECK pattern
plan::rewrite::set_not_null_check_pattern::should_rewrite:
#![allow(unused)] fn main() { policy.not_null_via_check_pattern() && target.column_exists(qname, col) }
When the column already exists in the target (i.e., this is "make an existing column NOT NULL", not "add a brand-new NOT-NULL column"), the rewrite emits four steps:
ADD CONSTRAINT __pgevolve_chk_<col> CHECK (<col> IS NOT NULL) NOT VALID; -- step 1
VALIDATE CONSTRAINT __pgevolve_chk_<col>; -- step 2
ALTER COLUMN <col> SET NOT NULL; -- step 3 (cheap once 2 validated)
DROP CONSTRAINT __pgevolve_chk_<col>; -- step 4
Steps 1, 2, and 4 are individually transactional but go in their own groups so the user can observe progress. Step 3 is cheap (Postgres uses the validated CHECK to skip the table scan), so it could ride with step 4, but pgevolve keeps them as separate steps for clarity in the audit log.
The synthesized __pgevolve_chk_<col> constraint name is reserved.
Users shouldn't create constraints with this prefix in their source.
Online rewrite 5: REFRESH MATERIALIZED VIEW CONCURRENTLY
plan::rewrite::refresh_mv_concurrent::should_rewrite:
#![allow(unused)] fn main() { policy.refresh_mv_concurrently() && target.mv_has_unique_index(&mv.qname) }
Two gating conditions:
- Policy enables the rewrite (
refresh_mv_concurrently = true; default enabled under online strategy). - The materialized view has at least one unique index in the target catalog (checked before the step is emitted, not at plan time — at plan time the MV may be brand new and the index is being created in the same plan).
When both hold, the step is emitted with TransactionConstraint::OutsideTransaction and kind RefreshMaterializedView. group_steps places it in its own non-transactional group. The mv-no-unique-index lint rule fires as a Warning when an MV exists with no unique index, alerting the user that concurrent refresh is unavailable.
View OR-REPLACE compatibility predicate
Source: crates/pgevolve-core/src/diff/views.rs — body_is_or_replace_compatible.
A view body change is OR-REPLACE compatible if and only if:
- The new SELECT list is a superset of the old one (columns can be appended but not removed or reordered).
- The types of existing columns are unchanged.
When the predicate returns true, the differ emits ViewChange::ReplaceBody { compatible: true } and the planner uses CREATE OR REPLACE VIEW. When false, it emits compatible: false and the planner uses DROP VIEW + CREATE VIEW (destructive — requires intent approval) plus the dependent-recreation cascade below.
Dependent-recreation walk
Source: crates/pgevolve-core/src/plan/recreate_views.rs — extend_with_dependent_recreations.
After the differ produces the initial ChangeSet, the planner calls extend_with_dependent_recreations to find every view transitively affected by the changes. Triggers that start the walk:
DropTable— any view with abody_dependenciesedge to the dropped table.AlterTablewithDropColumnorAlterColumnTypeops — views referencing the affected column.View(ReplaceBody { compatible: false })— views depending on the view being replaced.Mv(ReplaceBody { .. })— views depending on the MV being replaced.View(Drop)orMv(Drop)— views depending on the dropped object.
The walk is a BFS over the body_dependencies reverse index (object → views that reference it). The result set is topologically sorted (dependency-first) before being appended to changes as ReplaceBody { compatible: false } entries. The planner then processes those into DROP + CREATE steps.
Policy gate. When PlannerPolicy::view_drop_create_dependents() returns false, the walk still runs to detect affected views, but returns Err(affected) instead of modifying changes. The caller surfaces a human-readable error naming every affected view.
No double-emission. Views already present in changes (because the differ itself produced a ReplaceBody for them) are excluded from the walk's output — the filter prevents emitting the same step twice.
User-type compatibility predicates
Source: crates/pgevolve-core/src/diff/types.rs.
enum_can_alter_in_place
#![allow(unused)] fn main() { pub(crate) fn enum_can_alter_in_place( catalog_vals: &[EnumValue], source_vals: &[EnumValue], ) -> bool }
Returns true (in-place ALTER TYPE is safe) when:
- No labels are removed, or removed labels are all paired with added labels at the same list position (rename heuristic — same list length, position-paired exclusive names).
- Preserved labels (those present in both lists) appear in the same relative order as in the catalog.
Returns false (triggers ReplaceWithCascade) when:
- Any label is removed without a same-position replacement (true drop).
- Preserved labels would be reordered.
composite_can_alter_in_place
#![allow(unused)] fn main() { pub(crate) fn composite_can_alter_in_place( catalog_attrs: &[CompositeAttribute], source_attrs: &[CompositeAttribute], ) -> bool }
Returns true when the preserved attributes (those present in both lists) appear in the same relative order in both. New attributes may be appended anywhere. Returns false (triggers ReplaceWithCascade) when preserved attributes would be reordered.
ReplaceWithCascade fallback
When either predicate returns false, or when a domain's base type changes, the differ emits UserTypeChange::ReplaceWithCascade { source, catalog }. The planner converts this into two steps:
DROP TYPE <qname> CASCADE(drop_type, destructive — requires intent approval).CREATE TYPE <qname> …(create_type, safe).
Both steps go into the same transactional group. The CASCADE propagates automatically to all dependent columns, views, and functions, so no additional dependent-recreation walk is needed for types (unlike views, which require the explicit extend_with_dependent_recreations pass).
Function OR-REPLACE predicate
Source: crates/pgevolve-core/src/diff/routines.rs — function_can_or_replace.
#![allow(unused)] fn main() { pub(crate) fn function_can_or_replace(catalog: &Function, source: &Function) -> bool }
Returns true (safe to use CREATE OR REPLACE FUNCTION) when:
- The language is unchanged (
sql→sqlorplpgsql→plpgsql). - The return type is unchanged (same
ReturnTypevariant and inner type). Any return-type change (scalar → void, etc.) forces a DROP + CREATE path. - The OUT / INOUT parameter names and types are unchanged. These form part of the effective return type in Postgres and are rejected by
CREATE OR REPLACE FUNCTIONif they differ.
When the predicate returns false, the differ emits FunctionChange::ReplaceWithCascade { source, catalog }. The planner converts this into:
DROP FUNCTION <qname>(<arg_sig>) CASCADE(drop_function, destructive — requires intent approval).CREATE OR REPLACE FUNCTION …(create_or_replace_function, safe).
Both steps go into the same transactional group.
Routine-cascade extension to the dep walker
The dependent-recreation walk (recreate_views.rs) is extended for routines: if a DROP TABLE or a DROP VIEW would orphan a function or procedure body dependency, the cascade also includes the affected routines as ReplaceWithCascade entries. This mirrors the view-cascade behaviour — no hidden CASCADE drops; every affected object appears explicitly in the plan.
Procedures with commits_in_body = true are placed in their own non-transactional step regardless of the surrounding group (matching the OutsideTransaction constraint emitted by the planner for those procedures).
Atomic mode
Strategy::Atomic short-circuits every online rewrite — every
should_rewrite_* returns false. Output: one step per change, all
transactional, all in one group. The executor commits the whole thing
in a single BEGIN/COMMIT or fails atomically.
Useful for hermetic dev / test environments where you don't care about locking and just want the apply to be a single atomic event.
v0.3 step kinds
Each v0.3 cross-cutting state dimension adds one or more StepKind
variants. They are all transactional, all sparse (only fields where
source declares a value and target disagrees flow into the step):
| Release | StepKind variants |
|---|---|
| v0.3.0 | CreateRole, DropRole, AlterRole (cluster-side) |
| v0.3.1 | AlterObjectOwner, GrantObjectPrivilege, RevokeObjectPrivilege, GrantColumnPrivilege, RevokeColumnPrivilege, AlterDefaultPrivileges |
| v0.3.2 | CreatePolicy, DropPolicy, AlterPolicy, EnableRowSecurity, DisableRowSecurity, ForceRowSecurity, NoForceRowSecurity |
| v0.3.3 | SetTableStorage, SetIndexStorage, SetMaterializedViewStorage |
All four sub-specs follow the same convention: one Set* variant per
state dimension, no paired Reset* — the lenient drift policy makes
the reset path unreachable. See
architecture.md § v0.3: cross-cutting state
for the design rationale.
Sub-phase 3: group
group_steps(steps) → Vec<TransactionGroup>:
#![allow(unused)] fn main() { pub struct TransactionGroup { pub id: u32, // 1-indexed pub transactional: bool, pub steps: Vec<RawStep>, } }
Adjacent steps with the same TransactionConstraint coalesce into one
group. Each transition flips a new group. Empty input produces an
empty output.
Example: 3 in-tx steps, then 1 out-of-tx step (concurrent index), then 2 in-tx steps → 3 groups (3 steps, 1 step, 2 steps).
Sub-phase 4: wrap
Plan::from_grouped(groups, source, target, target_identity, source_rev, version, ruleset_version):
- Walks every step in every group, assigning a contiguous 1-indexed
step_noacross all groups. - For each step where
destructive == true, allocates the nextintent_idand creates aDestructiveIntentrow (id, step, kind, target, reason). The step gets itsintent_idfilled. - Computes
PlanIdover(source, target, version, ruleset_version). - Captures
created_atand the target snapshot.
Output: a Plan ready to write to disk.
PlanId derivation
PlanId::compute:
#![allow(unused)] fn main() { let mut h = blake3::Hasher::new(); h.update(b"pgevolve-plan-id-v1\n"); h.update(pgevolve_version.as_bytes()); h.update(&[0]); h.update(&planner_ruleset_version.to_be_bytes()); h.update(&[0]); h.update(&bincode::serde::encode_to_vec(source, cfg)?); h.update(&[0]); h.update(&bincode::serde::encode_to_vec(target, cfg)?); PlanId(*h.finalize().as_bytes()) }
Bincode is used because its encoding is byte-stable across runs and across machines. The domain-separator prefix prevents accidental cross-protocol collision with other BLAKE3 uses in the codebase.
What's tested
| Property | Test |
|---|---|
| Topological order is deterministic | graph::tests::deterministic_under_insertion_order_changes |
| Linear chain produces dependency-correct creates | ordering::tests::linear_schema_table_index_orders_in_dependency_order |
| FK between independent tables orders referenced first | ordering::tests::fk_between_independent_tables_orders_referenced_first |
| Two-table FK cycle extracts at least one FK | ordering::tests::two_table_fk_cycle_extracts_one_or_more_fks |
| Three-way FK cycle breaks at least one | ordering::tests::three_way_fk_cycle_breaks_at_least_one |
| Self-referential FK doesn't cycle | edges::tests::self_referential_fk_does_not_cycle |
| Concurrent index rewrite gates correctly | rewrite::tests::create_index_on_existing_table_rewrites_to_concurrent, unique_create_index_does_not_rewrite_to_concurrent, etc. |
| FK / CHECK NOT VALID rewrite gates | add_fk_on_existing_table_emits_two_steps, add_check_on_existing_table_emits_two_steps |
| SET NOT NULL pattern gates | set_not_null_on_existing_column_emits_four_steps, set_not_null_with_atomic_policy_stays_single_step |
group_steps partitions on transactional boundary | grouping::tests::transition_creates_new_group |
PlanId deterministic across runs | plan_id_is_deterministic_across_calls, plus tier-5 property test |
| Drop graph runs in reverse dependency order | ordering::tests::drops_run_in_reverse_dependency_order |
| Deferred FKs are stripped from inline CREATE TABLE | indirectly via two_table_fk_cycle_extracts_one_or_more_fks (asserts remaining + deferred FKs == 2) |
What's currently incomplete
- Column reorder. The diff detects column position drift
(
columns.<order>paths) but the planner doesn't emit a reorder step. Postgres has noALTER COLUMN ... POSITION, so this would require a table rewrite. As of v0.2 readiness, this is surfaced as aSeverity::LintAtPlanfinding (column-position-drift) rather than being silently ignored.pgevolve planexits2on an unwaived finding; waive with[[lint_waiver]]+ a separaterewrite-tableinvocation (implementation pending v0.2 sub-spec). - The
ALTER TYPE ... ADD VALUEenum rewrite. Lands with enum support (v0.2). Adding an enum value can't run in a transaction (in PG ≤ 11) or has restrictions (≥ 12); the rewrite will mirror theCONCURRENTLYgroup-by-group story. - Online column-type change (e.g.,
int → bigint). The planner currently emits a singleALTER COLUMN ... TYPEstep which can rewrite the entire table. A future rewrite will use the "shadow column + USING expression + rename" pattern.
Executor
The apply loop. Source lives in crates/pgevolve/src/executor/.
Entry point
pgevolve::executor::apply(plan_dir, client, filter, overrides):
#![allow(unused)] fn main() { pub async fn apply( plan_dir: &Path, client: &mut tokio_postgres::Client, filter: &CatalogFilter, overrides: ApplyOverrides, ) -> Result<ApplyOutcome, ApplyError>; }
The flow, mapped to spec §8:
flowchart TD
Start([apply(plan_dir, client, …)]) --> Read["1. read_plan_dir"]
Read --> Bootstrap["2. bootstrap_metadata"]
Bootstrap --> Lock["3. try_acquire_lock"]
Lock --> Preflight["4. run_preflight<br/>(identity · drift · intent)"]
Preflight --> Open["5. open_apply_log<br/>(apply_log + plan_steps)"]
Open --> Exec["6. execute_plan<br/>(per-group)"]
Exec --> Close["7. close_apply_log<br/>(succeeded/failed/aborted)"]
Close --> Unlock["8. release_lock"]
Unlock --> Done([ApplyOutcome])
read_plan_dir(plan_dir)— load the three files; cross-check plan id.bootstrap_metadata(client)— install or upgrade thepgevolve.*tables. Idempotent.try_acquire_lock(client, &actor)—pg_try_advisory_lockon the singleton key.run_preflight(client, &plan, filter, preflight_overrides)— target-identity match, drift recheck, intent approval.open_apply_log(client, &plan, &actor)—apply_logrow +plan_stepsrows.execute_plan(client, &plan, apply_id, abort_after_step)— execute each group in order.close_apply_log(client, apply_id, status, error_message).release_lock(client)— clear the lock row +pg_advisory_unlock.
On failure: the apply_log row gets status failed / aborted, the
lock is released, and the error bubbles up.
The pgevolve metadata schema
Installed by bootstrap_metadata (executor::bootstrap). All tables
live in the pgevolve schema.
| Table | Role |
|---|---|
pgevolve.bootstrap_version | Append-only list of applied schema migrations. Bootstrap is idempotent because it consults this table before running any DDL. |
pgevolve.apply_log | One row per apply attempt. Status, plan id + hash, version metadata, actor, started/finished timestamps, error message. |
pgevolve.plan_steps | One row per step within an apply. Status, SQL text, targets, error message. |
pgevolve.lock | Singleton row tracking who holds the advisory lock (informational; the actual lock is the session-scoped advisory lock). |
The metadata schema is append-only at the migration level.
bootstrap_metadata looks at bootstrap_version, runs migrations
whose version exceeds the current, and inserts a new
bootstrap_version row. Schema changes to the pgevolve schema itself
ship as new entries in BOOTSTRAP_MIGRATIONS.
Singleton advisory lock
The lock key is derived from the ASCII bytes b"PGEVOLVE":
#![allow(unused)] fn main() { pub const PGEVOLVE_LOCK_KEY: i64 = i64::from_be_bytes(*b"PGEVOLVE"); }
try_acquire_lock calls pg_try_advisory_lock(PGEVOLVE_LOCK_KEY) and
updates the pgevolve.lock row on success. The lock is **session-scoped
** — Postgres releases it automatically when the session disconnects,
which means a crashed apply releases the lock without any cleanup
action. The pgevolve.lock row is purely informational: the next
acquirer's UPDATE overwrites it.
release_lock clears the lock row and calls pg_advisory_unlock for
clean shutdowns.
Target-identity check
compute_target_identity(client):
#![allow(unused)] fn main() { let row = client.query_one( "SELECT current_database(), inet_server_addr()::text, inet_server_port(), current_setting('cluster_name', true), (SELECT system_identifier::text FROM pg_control_system())", &[], ).await?; }
The five fields are BLAKE3-hashed with NUL separators and a domain prefix; the first 16 hex characters are the target identity. Stable across reconnects to the same DB; different across different DBs.
The preflight check fails with ApplyError::TargetIdentityMismatch
when the live identity doesn't match plan.metadata.target_identity
unless overrides.allow_different_target is set.
Preflight
run_preflight(client, &plan, filter, overrides) runs three checks:
- Target-identity match. Always enforced unless
allow_different_target. - Drift recheck. Re-introspect the live catalog and diff against
plan.metadata.target_snapshot. Fails withApplyError::DriftDetected(n)if any drift is found, unlessallow_driftis set. - Intent enforcement. Iterate
plan.intents, refuse to run any destructive step whose intent isn't approved.
v0.1 status. The drift recheck is stubbed (
read_live_catalogreturnsCatalog::empty()). The CLI'sapplycommand forcesallow_drift = trueinternally to compensate. The intent recheck is also currently a TODO —Plan::read_from_dirreadsintent.toml's approval state but the executor doesn't consult it. Both lands in v0.1.x once the binary-side catalog reader is threaded into preflight.
Execution
execute_plan(client, plan, apply_id, abort_after_step):
#![allow(unused)] fn main() { for group in &plan.groups { if group.transactional { execute_transactional_group(client, apply_id, group, abort_after_step).await?; } else { execute_autocommit_group(client, apply_id, group, abort_after_step).await?; } } }
Transactional groups
#![allow(unused)] fn main() { let tx = client.transaction().await?; for step in &group.steps { mark_step_running(tx.client(), apply_id, step.step_no).await?; if let Err(e) = tx.batch_execute(&step.sql).await { let err_msg = render_pg_error(&e); tx.rollback().await?; // After rollback: write final audit rows on the bare client. mark_step_failed(client, apply_id, step.step_no, &err_msg).await?; mark_steps_rolled_back(client, apply_id, group.id).await?; return Err(ApplyError::StepFailed { … }); } mark_step_succeeded(tx.client(), apply_id, step.step_no).await?; if abort_after_step == Some(step.step_no) { // Break out of the loop AFTER the success mark. return Err(ApplyError::AbortedAfterStep { step_no: step.step_no }); } } tx.commit().await?; }
Key subtleties:
- Audit updates ride inside the transaction.
mark_step_runningandmark_step_succeededare run ontx.client(), so they're part of the same transaction as the DDL. If the transaction rolls back, the audit updates also revert. - After rollback, audit is re-marked on the bare client. The
failing step gets
mark_step_failed; every other step in the group getsmark_steps_rolled_back(which updatespendingandrunningandsucceededrows in the group — because the pre-rollbacksucceededstate was reverted by the rollback). render_pg_errorextracts SQLSTATE + server message from thetokio_postgres::Error. Without it, the error display is just "db error" — useless for debugging.
Autocommit groups
Each step runs on the bare client (no transaction). On failure: mark
the failing step failed, return StepFailed. Earlier steps stay
succeeded; later steps stay pending.
This is the right semantics for CONCURRENTLY groups: each
CONCURRENTLY operation is its own atomic Postgres operation, and a
failure mid-group doesn't roll back the predecessors.
The abort_after_step testkit hook
ApplyOverrides::abort_after_step: Option<u32> is the chaos hook.
When set to Some(n), the executor cleanly aborts after the step
whose step_no == n succeeds. The error is ApplyError::AbortedAfterStep,
and close_apply_log sets the row to aborted rather than failed.
Used by the testkit's chaos harness to validate recovery semantics without the full ceremony of SIGKILL. The recovery property the harness tests:
apply(target, source, abort_after_step=N)— runs through step N, then aborts.- Introspect the live database → partial state.
apply(partial, source)— re-plans from the partial state and runs to completion.- Live state == source.
ApplyError taxonomy
#![allow(unused)] fn main() { pub enum ApplyError { Postgres(tokio_postgres::Error), PlanIo(PlanIoError), Catalog(CatalogError), LockHeld, TargetIdentityMismatch { plan: String, live: String }, DriftDetected(usize), UnapprovedIntents { count: usize, details: Vec<(u32, String, String)> }, StepFailed { step_no: u32, group_no: u32, error: String }, AbortedAfterStep { step_no: u32 }, } }
The CLI's commands/apply.rs maps these to exit codes:
ApplyError variant | Exit code |
|---|---|
TargetIdentityMismatch / DriftDetected / UnapprovedIntents | 2 (preflight) |
LockHeld / StepFailed | 3 (apply error) |
| Anything else | 1 |
AbortedAfterStep | (testkit-only; should not reach the CLI) |
Audit rows in detail
apply_log lifecycle
INSERT INTO pgevolve.apply_log (..., status) VALUES (..., 'running');
-- … execution …
UPDATE pgevolve.apply_log SET status = 'succeeded', finished_at = now() WHERE …;
'failed',
'aborted',
stateDiagram-v2
[*] --> running: open_apply_log
running --> succeeded: close_apply_log (Ok)
running --> failed: close_apply_log (StepFailed)
running --> aborted: close_apply_log (AbortedAfterStep)
The CHECK constraint enforces the four valid statuses.
plan_steps lifecycle
open_apply_log pre-populates every step row as pending. Per-step
transitions:
stateDiagram-v2
[*] --> pending: open_apply_log
pending --> running: mark_step_running
running --> succeeded: mark_step_succeeded
running --> failed: mark_step_failed
pending --> rolled_back: mark_steps_rolled_back
running --> rolled_back: mark_steps_rolled_back
succeeded --> rolled_back: mark_steps_rolled_back
mark_steps_rolled_back is the cleanup after a transactional rollback:
it flips every step in the group whose status is pending /
running / succeeded to rolled_back. The "include pending" is the
subtle bit — when audit rows are part of the rolled-back transaction,
their succeeded update vanishes and they revert to pending.
Recovery from partial apply
When an apply fails or aborts mid-flight:
- The
apply_logrow stays around with statusfailed/abortedand anerror_message. pgevolve.plan_stepsrecords which steps succeeded, which failed, which rolled back.- The next
pgevolve planre-reads the live catalog, which now reflects whatever DDL committed. The new plan diffs from that state. - No special recovery command is needed. Re-planning produces a plan that picks up where the previous one stopped.
This is what drift_recovery_property tests: random catalog, abort
after random step, re-plan, re-apply, assert live state matches the
target.
Concurrency semantics
- Two
pgevolve applyinvocations on the same database serialize via the advisory lock. The second one fails withApplyError::LockHeld. - An apply and unrelated user DDL (e.g., someone running
psqlat the same time) do not serialize. pgevolve's lock is namespaced; it doesn't take a relation lock. Concurrent DDL from outside pgevolve may cause the apply to see drift, which the preflight check would catch (once it's wired up; see the v0.1 status note above). - Concurrent
pgevolve planruns against the same database are safe:planis read-only.
Why session-scoped advisory locks?
A transaction-scoped advisory lock would release at every
COMMIT. pgevolve's apply spans multiple transactional groups plus
non-transactional CONCURRENTLY groups; a transaction-scoped lock
would release between them and let a second apply in.
Session-scoped advisory locks hold across the entire session, which is exactly the right scope. The single client connection's session is the apply's lifespan; disconnecting (or finishing cleanly) releases.
pgevolve v1.0 charter
This document defines what pgevolve v1.0 means: the gate that triggers the 1.0 cut, the stability commitments that hold across v1.x, the quality bar that every release meets, the Postgres-version support window, the cadence, and the parking lot of items deferred past 1.0.
The charter is living-until-1.0. At the 1.0 cut its sections
either retire (the gate-tracking ones) or get partially merged into
CONSTITUTION.md (the stability commitments +
cadence rules).
Design rationale and the brainstorm transcript live at
superpowers/specs/2026-05-28-v1-charter-design.md.
§1. What v1.0 means
A release of pgevolve where the project commits, in writing, to:
- a defined surface that won't break in the v1.x line,
- a defined quality bar that every release meets,
- a defined Postgres-version support window that rolls with the upstream EOL schedule.
The 1.0 cut happens when both of the following are true:
- Feature checklist complete (§4) — every
📋 Plannedrow currently inspec/roadmap.mdplus the explicit v1.0-blockers listed in §4 below are shipped. - Quality gates green (§3) — the existing per-push CI gate plus a new nightly proptest soak have been running clean for at least 30 consecutive days.
The maintainer cuts the 1.0 release manually. No formal RC cycle (per the "ship-when-ready" cadence in §5); the 30-day clean-CI window functions as the de-facto RC.
§2. Stability commitments
Stable in v1.x — no breaking changes without a major bump
pgevolveCLI: command names, flags, exit codes, argument shapes.pgevolve.tomlconfig schema.- On-disk plan format:
plan.sqldirective headers,intent.tomlschema,manifest.tomlschema, the BLAKE3 plan-id derivation. - The set of supported PG majors must include the Postgres community-supported versions at the time of each minor release (see §6).
Explicitly unstable in v1.x — can break without a major bump
- The
pgevolve-corelibrary API. Library consumers depend on path / git deps or pin to exact patch versions. The crate's docs.rs page leads with this notice. - Lint rule IDs and severities — new rules may land, old rules may be removed, severities may change up or down.
StepKindnames inplan.sql— backward-compatible additions only; renames bump major.- Catalog snapshot JSON schema.
- Internal modules (anything
pub(crate)).
Deprecation policy for the stable surface
Any breaking change starts with a one-minor-cycle deprecation: the old form keeps working with a stderr warning in version N, becomes a hard error in N+1. CHANGELOG notes the deprecation in N and the removal in N+1.
§3. Quality gates (gate #2 for the 1.0 cut)
All five must hold on main for 30 consecutive days before the 1.0
tag:
| Gate | What it checks | Where it runs |
|---|---|---|
| Per-push CI | cargo fmt --check, clippy -D warnings, cargo doc -D warnings, cargo deny check, lib tests, full conformance suite on PG 14/15/16/17/18 | .github/workflows/ci.yml on every push |
| Property-test soak (nightly) | The full Tier-5 property-test set at PROPTEST_CASES=5000 per PG major | .github/workflows/soak.yml cron (already exists; will bump cases to 5000 and add a 1.0-readiness check that flips green/red based on the most recent 30 days of soak runs) |
| Cargo deny advisories | Zero open advisories for ≥ 7 days before the tag | per-push + nightly |
| GH Actions disk-space | The PG-matrix conformance job survives without no space left on device (the v0.3.8 flake) | per-push; runner-cleanup step OR larger runner |
| Cargo doc clean | No broken intra-doc links across the workspace | per-push (enforced since v0.3.8) |
Out of scope for the 1.0 gate: mutation testing (cargo-mutants), fuzzer (cargo-fuzz), code-coverage badges, perf benchmarks. Each gets its own follow-up brainstorm if/when desired.
Operational definition of "30 consecutive days clean"
- The most recent 30 days of pushes to
mainhad all required CI jobs end insuccesson first attempt (reruns count against the streak — the v0.3.8 disk-space flake would have reset it). - Soak cron ran every night during the window with
success. - The streak is tracked manually pre-1.0 (a one-liner check the maintainer runs before tagging). Post-1.0 we may automate.
§4. Feature completeness (gate #1 for the 1.0 cut)
Every 📋 Planned row in spec/roadmap.md must
be ✅ Implemented and shipped. The v1.0 blockers:
| Source | Items |
|---|---|
| v0.4.0 | EVENT TRIGGER, per-partition TABLESPACE, TABLE … USING access method |
| v0.4.1 | AGGREGATE, PG 18 virtual generated columns |
| v0.4.2 | TABLESPACE (cluster object), PL-language wiring → non-SQL FUNCTION bodies |
| v0.4.3 | TEXT SEARCH family |
| v0.5.0 | FDW family (FDW, SERVER, USER MAPPING, FOREIGN TABLE, IMPORT FOREIGN SCHEMA) |
| v0.5.1 | OPERATOR / OPERATOR CLASS / OPERATOR FAMILY |
| v0.5.2 | CAST |
| v0.5.3 | Recursive views (WITH RECURSIVE) — promoted from "🔮 Future" to a v1.0 blocker. Requires cycle-aware dep-graph handling. |
That's 12 sub-spec slots across 8 minor releases. Every row is independent (modulo declared dep edges in the roadmap). The order is fixed; the timing slips.
The roadmap document owns the slotting + dep-edge analysis. The charter just lists the items.
§5. Cadence + versioning
Post-1.0, the rhythm stays the same as today: ship-when-ready, no
time-boxed schedule. Each release is a minor (new feature) or patch
(bugfix). pgevolve commits to main only — no release branches — so
every release tag points at a commit on main that passed the
per-push CI gate.
Versioning rules
- Major bump (v1 → v2): only when a breaking change to the stable surface (§2) is unavoidable. Likely never needed in 2026; pgevolve will plan to ship v1.x for the indefinite future.
- Minor bump (v1.N → v1.N+1): new object kind, new lint rule, new CLI subcommand, new config key, new PG major support.
- Patch bump (v1.N.M → v1.N.M+1): bug fixes only. The v0.3.8 → v0.3.9 emergency-fix flow is the patch template.
Release ceremony
Codified in ../CLAUDE.md directive 11:
push main, sign + push tag, wait for full 5-PG-major CI green,
then cargo publish and cargo yank the prior version if it had a
shipped bug. This rule is binding for every release, v0.x and v1.x.
§6. Postgres-version support commitment
v1.0 supports every Postgres major version supported by upstream at the time of the v1.0 release. The current set is PG 14–18. The version window rolls automatically:
- When a PG major reaches upstream EOL, the next minor release of
pgevolve drops support and removes its conformance fixtures. Each
removal lands in its own commit tagged
chore(eol): drop PG X. The CHANGELOG explicitly calls out the drop. - When a PG major is released (e.g., PG 19 in late 2026), it's added in the next minor release of pgevolve after the corresponding catalog-query work lands. The v0.3.6 release ("PG 18 catalog support") is the precedent.
- Per-version code paths are marked
// PG X+ onlyat the call site so the EOL-drop is a mechanical grep + delete.
No LTS branch. Bug fixes target current main only. Anyone needing
a fix on an older minor cuts their own patch.
§7. Explicitly post-1.0 (parking lot)
These are tracked but won't gate v1.0:
| Item | Why post-1.0 |
|---|---|
Library API stability for pgevolve-core | Tied to a clear set of external consumers materializing first. |
pgevolve-core re-export polish / typed errors at the public boundary | Same. |
| Partition pruning at plan time | Optimization, not correctness. |
SECURITY LABEL integration | Used primarily by SE-Linux; low demand. |
| Security-barrier / leakproof per-function flag review | Lands alongside finer-grained policy review. |
RULE, BASE TYPE, INHERITS, DETACH PARTITION CONCURRENTLY | Already ⛔ Not planned — kept here as "stays out". |
| Mutation testing, fuzzer, perf benchmarks | Quality investments deferred per §3. |
Distribution surface beyond cargo install (Homebrew, Docker, GH Releases binaries) | Picked up once user-facing demand justifies. |
Explicit rename support (a -- @pgevolve rename old -> new directive for columns / renamable objects) | Renames are a bounded future feature; until then they are a documented limitation (§8). Deferred per the 2026-06 architecture review. |
| Full expression/body normalization (commutative-operand sorting, deep paren-folding) | Subtle and partly redundant with the pg_query deparser; the common-case qualifier-recursion fix ships for 1.0, the hard cases are a documented limitation (§8). Deferred per the 2026-06 architecture review. |
Full explicit type-dependent recreation (DROP each dependent + recreate views/functions, replacing the type DROP … CASCADE entirely) | The 1.0 work (Phase 4) makes the CASCADE auditable — the plan names every dependent it will destroy — but keeps the gated CASCADE teardown. The larger recreation engine (which still cannot save dependent column data on an incompatible type change) is deferred. |
The charter is amendable: items can move from post-1.0 into the active checklist (§4) with a CHANGELOG note. Items can also move out of the checklist if the maintainer judges them not v1.0-blocking, with a CHANGELOG note.
§8. Known limitations at 1.0
These are accepted behaviors at 1.0 — not bugs, and not gating items. Each is documented so users know the boundary; the corresponding future work (where any) lives in the §7 parking lot. All resolved in the 2026-06 architecture review.
| Limitation | Behavior | Why accepted |
|---|---|---|
| Column / object rename | A rename is diffed as DROP-old + ADD-new and flagged with a data-loss warning; the operator must intervene. | The prime directive (no silent data loss) is preserved by the warning. Explicit rename is a bounded future feature (§7), deliberately not built for 1.0 to keep the core simple. |
| Column reorder | Unsupported; column order changes are not planned. | Reordering requires a full table rewrite — prohibited for data-bearing tables (Constitution §11) — and column order rarely carries logical meaning. |
| Expression-equivalence edge cases | Boolean operands in a different order (a AND b vs b AND a), differing parenthesization, or an unnamed expression column in a view SELECT (SELECT 1 → pg_get_viewdef renders 1 AS "?column?") may produce a spurious (no-op) diff on some views/expressions. The common multi-table / subquery qualifier cases are normalized for 1.0; name view expression columns to avoid the ?column? case. | The failure direction is always safe — a spurious recreate, never data loss. Full normalization is deferred (§7) as subtle and partly redundant with the deparser. |
| Extension replacement CASCADE not enumerated | DROP EXTENSION … CASCADE (emitted when an extension must be dropped+recreated) silently destroys objects depending on what the extension provides; the destructive warning says so generically but cannot name them. | pgevolve manages the extension declaration, not its provided objects / membership (that's pg_depend data outside the managed model), so the dependents aren't enumerable. The operator must review manually. |
pgevolve Project Constitution
This document records the guiding principles that govern every decision made in this project — code, architecture, specifications, plans, and tooling. It binds all contributors, human and AI alike. To amend it, open a PR with a clear rationale and obtain at least one maintainer approval.
1. Purpose
pgevolve is a Postgres-specific declarative schema management tool. Its job is to be correct, safe, and complete — not clever. This constitution exists so that day-to-day decisions (what crate to reach for, how to model a domain concept, when to ship) are made consistently against a shared set of values rather than re-litigated case by case.
2. Licensing
Everything published under this project is dual-licensed as MIT OR Apache-2.0. This is stated in [workspace.package].license in Cargo.toml and must be the declared license for every crate in the workspace.
We do not accept dependencies with copyleft licenses (GPL, AGPL, LGPL, EUPL, or similar). We do not accept proprietary dependencies that would restrict redistribution or compel relicensing. This is enforced at PR time by cargo-deny via deny.toml (added in CLEAN-7). A failing cargo deny check licenses is a hard block on merge — no exceptions without an explicit maintainer decision recorded in the PR.
3. Dependencies Are a Liability
Every dependency we accept is a vector for bugs, supply-chain attacks, license violations, and unmaintained code. The convenience benefit must be weighed honestly against that risk.
Before adding a dependency, ask: Is it actively maintained? Does it have a meaningful user base? Has it been audited, or does it have a strong track record? Is its license compatible with MIT OR Apache-2.0? Could we reasonably implement the functionality ourselves in a bounded amount of effort? If a crate fails more than one of these tests, the default answer is to not add it. If no suitable crate exists, we write the code ourselves rather than accepting a poor dependency.
cargo-deny runs in CI against the deny.toml config. It checks licenses, detects known-vulnerable versions via the RustSec advisory database, and flags duplicate major versions of key crates. A clean cargo deny check is required for merge.
4. Make Illegal States Unrepresentable
Rust's type system is one of the most powerful tools we have for eliminating entire classes of bugs before a test is ever written. We use it aggressively.
Closed sets of values are always Rust enums, never strings or integers. Optional fields are always Option<T>, never sentinel values. Raw strings are not used where a validated type exists — Identifier and QualifiedName (in crates/pgevolve-core/src/identifier.rs) are the canonical examples: you cannot accidentally pass an unvalidated, un-case-folded string where a Postgres identifier is expected. Where an operation has sequential phases, typestate patterns make it impossible to invoke a later-phase function before an earlier-phase has completed. The goal is not to be exhaustive about which patterns to apply — it is to internalize the discipline of asking, for every type boundary: "can the caller construct an invalid value here, and if so, why?"
This approach reduces the testing footprint without sacrificing safety. A property that the type system enforces does not need a test. Fast compiler feedback replaces a slow test-run-and-fix loop.
5. Full Postgres Support
pgevolve's goal is to support the complete superset of Postgres features that any real application might use. "Most common features" is not an acceptable scope boundary. If Postgres supports it and an application can use it, pgevolve must eventually support it.
We use the official Postgres parser made available by the pg_query crate (pg_query = "6" in [workspace.dependencies]). Parsing is not reimplemented. The pgevolve-conformance crate provides the authoritative conformance suite — it tests the full planning and application pipeline against all supported Postgres versions. Sub-spec implementation work is scoped to what Postgres provides, not what is convenient to implement. Gaps in conformance are tracked as issues, not quietly tolerated.
6. Postgres Version Support
We support every Postgres version that the Postgres community actively maintains. The currently supported versions are 14, 15, 16, 17, and 18. The conformance suite runs against all five.
When a version reaches end of life per the Postgres versioning policy, we drop it from the support matrix and remove any code that existed solely for compatibility with that version. This is a feature, not a chore: EOL drops pay down maintenance debt. Postgres 14 reaches EOL in November 2026 and will be dropped at that time.
7. Rust Conventions
We follow Rust community best practices without exception.
The workspace [workspace.lints] section in Cargo.toml is the canonical lint configuration. clippy::all, clippy::pedantic, and clippy::nursery are enabled as warnings; unsafe_code and missing_docs at the workspace level enforce documentation and memory-safety discipline. Production code contains no unwrap() or expect() calls — use explicit error propagation with typed errors (thiserror for library errors, anyhow for binary entry points). Box<dyn Error> is not used as a return type. cargo fmt and cargo clippy must be clean before merge. cargo doc must build without warnings.
unwrap() and expect() are permitted in tests and xtask tooling where a panic is an acceptable test failure signal.
8. Readability, Correctness, Then Performance
The codebase should be easy to read by someone who is not its author. This means: names that describe what a thing is, not how it is implemented; small modules with one clear responsibility; explicit over implicit; no premature abstraction.
Correctness comes before performance. We do not optimize code that has not been measured to be a bottleneck. The domain model — Postgres schemas, the IR, the planner, the change-set representation — is the load-bearing abstraction of this project. Its clarity is not traded away for runtime efficiency. When performance work is warranted, it is done with profiler evidence and reviewed as carefully as any other change.
The simplest architecture capable of solving the full problem is the right architecture. Scope changes that would require materially more complex architecture are worth pushing back on — complexity is a cost that compounds.
9. CI/CD and Release Discipline
Every PR must pass the full CI suite before it can merge: cargo fmt --check, cargo clippy, cargo test, cargo doc, and cargo deny check. No exceptions. A green CI run is a necessary (not sufficient) condition for merge.
Releases follow Semantic Versioning. CHANGELOG.md at the repository root is the authoritative release-notes source; every release entry must be present before tagging. Release tags are signed. The [workspace.package].version field and the CHANGELOG.md entry must agree at the time of tagging.
10. Security
We write secure code by default. We do not defer security concerns for later; they are addressed when the code is written.
When a vulnerability is reported — whether by an external researcher, a user, or a contributor — the response is: acknowledge, investigate, fix, and disclose. The person who reported it is doing the project a favor and is treated as such. Blame is not part of the process. The expected response timeline is: acknowledge within 7 days, provide a fix or mitigation within 30 days for medium-severity issues, and faster for criticals at the maintainer's discretion. Security issues are disclosed publicly after a fix is available, with a CVE filed where applicable.
A SECURITY.md documenting the reporting process is a TODO for the repository root.
11. Schema Evolution Strategy: Preserve Data, Prefer In-Place Change
The prime directive of this project is that a declarative schema change must never lose data. Every architectural decision in the diff/plan/render pipeline serves that directive first and convenience second.
Data-bearing tables are never dropped and recreated to effect a change.
They are evolved in place with ALTER, using lock-light online-rewrite patterns
— ADD CONSTRAINT ... NOT VALID + VALIDATE, CREATE INDEX CONCURRENTLY,
REFRESH ... CONCURRENTLY, and the CHECK-then-SET NOT NULL decomposition —
to keep locks short and downtime minimal. A drop-and-recreate-with-copy strategy
for tables is prohibited: on any non-trivial table it is categorically more
dangerous than in-place ALTER (it breaks inbound foreign keys from other
tables, loses sequence ownership / identity / partition attachments / OID-based
extension dependencies, doubles disk, and turns an O(1) metadata change into an
O(rows) outage).
Drop/create ("recreate") is the bounded fallback only for objects Postgres
gives no in-place ALTER path — views and materialized views, enum / composite
/ range types, and collations. Even then it is performed as an explicit,
auditable, dependency-ordered teardown and rebuild, never DROP ... CASCADE:
the plan must name every dependent it drops and recreates, so the operator can
review exactly what will happen. Containing drop/create to these objects — and
keeping it explicit — is a binding architectural rule, not a stylistic
preference.
The breadth of ALTER support across object kinds (sequences, publications,
subscriptions, policies, text-search configurations, …) is catalog coverage in
service of the Full Postgres Support goal (§5), not incidental complexity. It
is not a candidate for "simplification" by switching to drop/create — for these
metadata-only objects drop/create is strictly worse (it churns or destroys state
such as a sequence's last_value or a subscription's replication slot).
Relationship to the v1.0 charter
This constitution states the always-binding principles for the project — they apply to every commit, v0.x and v1.x alike. The v1.0 charter is the parallel document that states what pgevolve v1.0 specifically commits to (stable surface, quality gate, PG-version policy, post-1.0 parking lot). At the 1.0 cut, the charter's stability + cadence sections will be merged into this constitution and the rest of the charter retired.
Releasing pgevolve
This runbook applies the Constitution §9
release discipline + the CLAUDE.md §11 "never cargo publish until CI is green" rule (added 2026-05-28 after the v0.3.8
disaster).
Canonical executable form
For a normal release, run:
scripts/release.sh X.Y.Z
The script walks every step below, gates on the pre-flight verify, waits for CI green on both the push and the tag, and prompts before each irreversible action (publish, push, tag). If you bail at any prompt, nothing irrecoverable has happened.
The prose runbook below documents what the script is doing, for when something goes wrong and you need to step through manually.
Pre-flight
Before starting the release commit, the following must be true:
- All open PRs targeting the release have merged.
-
cargo test --workspace --all-targetsis green locally. -
cargo clippy --workspace --all-targets -- -D warningsis green. -
cargo fmt --all -- --checkis clean. -
cargo doc --workspace --no-depsbuilds with zero warnings (run withRUSTDOCFLAGS=-D warnings). -
cargo deny checkpasses. Install if missing:cargo install cargo-deny. -
The
[Unreleased]section ofCHANGELOG.mdaccurately describes everything in the release. -
The previous release tag's signature verifies (
git verify-tag <prev-tag>) — proves your signing key is configured.
Release commit
-
Update versions — three locations must agree:
Cargo.toml→[workspace.package].versioncrates/pgevolve-core-macros/Cargo.toml→[package].version(not workspace-inherited because it's a proc-macro crate with its own version literal)- The next CHANGELOG section header
-
Date-stamp the CHANGELOG entry:
## [X.Y.Z] — YYYY-MM-DDThe CI
changelogjob (in.github/workflows/ci.yml) verifies that theCargo.tomlversion has a matching## [X.Y.Z] — DATEline. -
Rebuild so
Cargo.lockpicks up the new version:cargo build --workspace -
Commit:
git add Cargo.toml Cargo.lock CHANGELOG.md crates/pgevolve-core-macros/Cargo.toml git commit -m "release: vX.Y.Z"
Push main + WAIT FOR CI GREEN
Push the release commit:
git push origin main
Then wait for the per-push CI run to finish green across all 5 PG majors. Per CLAUDE.md §11, this is non-negotiable — today's v0.3.8 disaster happened because the maintainer published immediately after the tag push while CI was still running. CI then failed on PG 15 + 16 (broken ICU SQL); the buggy v0.3.8 was already on crates.io.
COMMIT=$(git rev-parse HEAD)
RUN_ID=$(gh run list --branch main --commit "$COMMIT" --limit 1 --json databaseId --jq '.[0].databaseId')
gh run watch "$RUN_ID" --exit-status
--exit-status makes the command return non-zero if any job fails;
do not proceed past this step until it returns 0.
Tag
Tags must be signed. Per Constitution §9, an unsigned release tag is not a valid release.
git tag -s vX.Y.Z -m "pgevolve vX.Y.Z
<short release summary, 2-3 lines>"
Verify locally:
git verify-tag vX.Y.Z
If git tag -s complains, your signing key isn't configured. See:
git config user.signingkey <KEY-ID>git config gpg.format ssh(for SSH signing — recommended in 2026)git config gpg.ssh.allowedSignersFile <path>(for verification)
The first time you sign, also enable signing-by-default for safety:
git config commit.gpgsign true
git config tag.gpgsign true
Push tag
git push origin vX.Y.Z
The tag push does not trigger CI by itself in the current workflow setup — CI already ran on the underlying commit when you pushed main above. If for some reason the tag is on a different commit than main's HEAD, wait for CI green on that commit too before publishing.
Publish to crates.io
When the release is ready for crates.io, publish in dependency order:
cargo publish -p pgevolve-core
# Wait ~30 seconds for the index to sync, then:
cargo publish -p pgevolve
pgevolve-core-macros is a proc-macro crate that's only published when
its own version actually changes (it has its own [package].version
literal, not workspace-inherited). In practice that's rare: the macros
crate has stayed at 0.2.1 since v0.2.x while the rest of the workspace
has bumped through v0.3.x. Only publish it when the literal version in
crates/pgevolve-core-macros/Cargo.toml changed in the release commit:
# Only if pgevolve-core-macros version was bumped this release:
cargo publish -p pgevolve-core-macros
# Then wait ~30s for the index, then publish pgevolve-core, then pgevolve.
scripts/release.sh deliberately omits the macros publish — running it
after a macros bump means manually publishing macros first, before
re-running the script's publish step. (If macros bumps become more
frequent, the script should grow a conditional check.)
pgevolve-conformance, pgevolve-testkit, and xtask are all
publish = false and stay local.
For pre-publish sanity:
cargo publish --dry-run -p pgevolve-core
cargo publish --dry-run -p pgevolve
# Add macros to the dry-run set if its version was bumped this release.
Yank a prior version (if shipping a fix)
If the version you just published replaces a buggy prior version (v0.3.9 replacing the broken v0.3.8 is the canonical example), yank the prior so cargo prefers the new version:
cargo yank --version <prior> pgevolve-core
cargo yank --version <prior> pgevolve
The CHANGELOG entry for the fix release should call out the yank
explicitly in a ### Yanked section. Yanking does not remove the
prior version from crates.io — cargo can still resolve it if pinned —
but it stops new installs from picking it up.
Post-release
- Push the tag (already done above; this is the reminder bullet).
- Verify the badge updates: README's
[![crates.io]and[![Soak]badges refresh within a few minutes. - Open a new
[Unreleased]section at the top ofCHANGELOG.mdso future commits have a place to land. Single line:## [Unreleased]. Stays empty until the next release lands work in it. - Optionally bump the workspace version to
X.(Y+1).0-devto make accidental crates.io uploads of a staleX.Y.Zversion impossible. (Cargo rejects publishes with-devpre-release tags by default without--allow-dirty-style overrides.) - Create a GitHub release from the new tag with the CHANGELOG section as the body. Surfaces the release in GitHub's release feed + RSS + API.
- If this release closes a v1.0-checklist row (per
v1.md§4), flip the row's status inspec/objects.mdand remove it fromspec/roadmap.md's Active matrix in a follow-up docs commit.
Historical notes
The v0.1.0 (commit adb0177) and v0.2.0 (commit 3087a5b) tags
predate the Constitution §9 "release tags are signed" mandate and
are annotated-but-unsigned. The 2026-05-21 constitution audit flagged
them; the maintainer decided NOT to re-sign retroactively because
rewriting historical tags would break consumers who reference them
(e.g., Cargo.lock git deps). Every tag from v0.2.1 onward is signed.
Future audits: for t in $(git tag); do git verify-tag "$t" 2>&1 | head -1; done should show every tag from v0.2.1 forward returning
Good "git" signature.
Changelog
All notable changes to pgevolve are documented in this file.
The format is loosely based on Keep a Changelog, and the project follows Semantic Versioning.
[0.4.6] — 2026-06-23
Fixed
- String-literal CHECK constraints diffed spuriously on PG 14–17 (#47). A
CHECKwhose expression contained a string literal — e.g.CHECK (email <> '')— produced a perpetual spurious diff against a live database, becausepg_get_constraintdefdeparses the literal with an explicit::textcast (email <> ''::text) on PG 14–17 (PG 18 dropped it) while the source parse keepsemail <> ''. The expression canonicalizer (normalize_expr) now recursively strips the always-redundant string-literal→text cast ('x'::text) throughout an expression tree, so both paths converge. The rule is precise — a string-literalA_Constcast to a bare text-family type with no typmod — so meaningful coercions like'5'::int,'2024-01-01'::date, and'x'::varchar(5)are preserved. Verified by a live-PG apply round-trip across PG 14–18. - Unnamed CHECK constraint naming (#48). An inline unnamed
CHECKwas named{table}_check, but Postgres names a single-column check{table}_{col}_check, forcing a spurious diff against the live catalog. Unnamed checks are now named the Postgres-faithful way via theChooseIndexNameport: a column-level check →{table}_{col}_check, a table-level check →{table}_check. This also corrects the v0.4.5LIKE INCLUDING CONSTRAINTSbehavior: Postgres copies CHECK constraint names verbatim onLIKE(it does not re-derive), so the LIKE path now copies the source name verbatim rather than re-deriving (reverting the #45 heuristic). Verified against live PG 14–18 with unnamed-CHECK round-trip tests. - Over-long generated index/constraint names truncated incorrectly (#49).
choose_nametruncated only the table-name component when a generated name exceededNAMEDATALEN(63 bytes); Postgres'smakeObjectNameshrinks the table-name and column-addition components alternately. For an index/constraint driven by long column names this produced a wrong (possibly over-length) name. The port now mirrors Postgres's alternating-shrink algorithm. The common case (long table, short columns) is unchanged; verified by a long-column live-PG round-trip across PG 14–18.
[0.4.5] — 2026-06-23
Added
CREATE TABLE … (LIKE source [INCLUDING …])support (#43).LIKEclauses are now expanded into concrete IR during a deferred pass (crates/pgevolve-core/src/parse/builder/table_like.rs) that runs after every table is assembled. The followingINCLUDINGoptions are supported:DEFAULTS,IDENTITY,GENERATED,STORAGE,COMPRESSION,COMMENTS(table and column),CONSTRAINTS(CHECK only — PK/UNIQUE belong toINCLUDING INDEXES),INDEXES(PK/UNIQUE constraints plus plainCREATE INDEXindexes),STATISTICS(extended statistics), andINCLUDING ALL. Chained or interleaved LIKE clauses work regardless of declaration order; a cycle or self-reference is detected and reported as a parse error. Using a view or materialized view as a LIKE source is rejected with a clear diagnostic. Auto-derived names for constraint-backing and plain indexes are generated via a Rust port of Postgres'sChooseIndexName(choose_name.rs), producing names that match live PG 14–18 output (verified by conformance fixtures). Known limitations: (1)EXCLUDEconstraints are not modeled in pgevolve's IR and therefore cannot appear on a LIKE source — nothing to copy. (2) Unnamed source CHECK constraints are re-derived to the clone's name ({clone}_check, matching pgevolve's inline convention; see #45) — full Postgres column-form fidelity ({table}_{col}_check) is tracked with the broader constraint-naming work in #44/#46. Conformance coverage:objects/tables/create-like-bare,create-like-including-all,create-like-multiple,create-like-interleaved.
Fixed
interval(N)/interval <fields>type modifiers misclassified toOther(#41). A column declaredinterval(6),interval hour, orinterval hour to minuteparsed without error but was silently stored asColumnType::Other, producing a perpetual spuriousALTER COLUMN TYPEdiff against a live database (the catalog'sformat_typeemits the canonical form, which does map toInterval).pg_queryinjects the PGINTERVAL_MASKfields bitmask as the first typmod argument (interval(6)→[32767, 6]); the renderer now decodes that bitmask and emits the canonical string, andparse_canonicalgained arms for theinterval <fields>andinterval <fields>(p)forms, so the source and catalog paths converge to the sameColumnType::Interval.- Schema-qualified PostGIS types dropped their typmod (#42).
public.geometry(Point,4326)took the user-defined-type branch and discarded the(Point,4326)modifier, so the sourceUserDefinedcould never match the catalog'sOther. Schema-qualifiedgeometry/geographydeclarations that carry a typmod are now routed through the canonical stringify path (preserving the schema), and the casing-normalization helper recognizes a schema-qualified head, so both sides converge. Non-geo user-defined types and modifier-free declarations are unchanged. - Inline unnamed
UNIQUEconstraint naming (#44). An inlineUNIQUE (a, b)was auto-named{table}_key, which never matches Postgres's{table}_{col}_key, forcing a spurious diff. Unnamed unique constraints are now named via the sameChooseIndexNameport used by the LIKE path, with per-table collision counters.PRIMARY KEY({table}_pkey) and explicitly-named constraints are unchanged. LIKE INCLUDING CONSTRAINTSre-derives unnamed CHECK names (#45). A copied unnamed CHECK previously kept the source table's name (base_check) on the clone; it is now re-derived to the clone's name (clone_check) so aLIKEclone matches an equivalent hand-written table.
Internal
- Extracted a shared
resolve_orderhelper for the LIKE dependency-ordering passes and lockedchoose_nametruncation behavior at theNAMEDATALENboundary (#46).
[0.4.4] — 2026-06-18
Added
- Recursive views (
WITH RECURSIVE). Confirmed support forCREATE RECURSIVE VIEW,CREATE VIEW … WITH RECURSIVE, recursive materialized views, andCREATE OR REPLACEof a recursive body, with conformance coverage across PG 14–18. No code change was required: a recursive CTE's self-reference is unqualified, and the view body-dependency walkers only emit edges for schema-qualified references, so no dependency self-edge is ever produced; the body round-trips via the canonical deparse. The earlier roadmap assumption that recursive views needed cycle-aware dep-graph work was incorrect.
Fixed
- Parameterized PostGIS types (
geometry/geography(<subtype>,<srid>)). A column declaredgeometry(Point,4326)failed to parse with "could not stringify type name" (#40), blocking any PostGIS-backed schema atpgevolve validate. Type modifiers are anexpr_list, so the subtype token (Point/MultiPolygon/…) is parsed as a bareword (ColumnRef), not a constant; the type-name stringifier now handles bareword typmod arguments in addition to literals. The geometry/geography subtype is also normalized to lowercase on both the source-parse and catalog paths so it does not produce a perpetual spuriousALTER COLUMN TYPEdiff (the source parser lowercases the subtype whilepg_catalog.format_typeemits canonical TitleCase, and the catch-all type compares by exact string). Baregeometry/geography, the array formgeometry(Point,4326)[], and composite/domain/function-argument declarations are covered. Two adjacent pre-existing limitations were filed as follow-ups:interval(N)typmods silently misclassify (#41) and schema-qualifiedpublic.geometry(…)drops its typmod (#42).
[0.4.3] — 2026-06-08
Added
- Table
TABLESPACEplacement.CREATE TABLE … TABLESPACE <ts>andCREATE TABLE … PARTITION OF … TABLESPACE <ts>on regular tables, partitioned parents, and partition children.ALTER TABLE … SET TABLESPACEis RequiresApproval on a leaf table (full rewrite +ACCESS EXCLUSIVElock) and Safe on a partitioned parent (metadata-only, no rewrite).pg_defaultis normalized to the implicit default — declaringTABLESPACE pg_defaultproduces no spurious diff. Per-partitionTABLESPACEoverrides are tracked independently of the parent on theTable.tablespacefield. TEXT SEARCH DICTIONARYandTEXT SEARCH CONFIGURATIONsupport. ManagedTEXT SEARCH DICTIONARY(TEMPLATE reference + OPTIONS list;CREATE,ALTER … (options),ALTER … OWNER TO,DROP,COMMENT ON) andTEXT SEARCH CONFIGURATION(PARSER reference + token→dictionary MAPPING list;CREATE,ALTER … ADD/ALTER/DROP MAPPING,ALTER … OWNER TO,DROP,COMMENT ON). Identity is(schema, name); a template/parser change reads as drop + create (Postgres has no in-placeALTERfor those), and owner is lenient. PARSER and TEMPLATE are unmanaged environment references (C-language functions); pgevolve never auto-creates or drops them, and unqualified template/parser/dictionary names resolve topg_catalog.COPY=onCREATE TEXT SEARCH CONFIGURATIONis out of scope. Known limitation: a functional index or generated column whose expression callsto_tsvector('schema.config', …)carries an implicit dependency on that text-search configuration that the dep-graph does not track (no expression-level TS-config dep edges); such an index may be ordered before its configuration at apply time. The TS objects themselves round-trip correctly; this is a planner gap to address in a future release.
[0.4.2] — 2026-06-08
Added
CASTsupport. Custom casts viaCREATE CAST(WITH FUNCTION/WITHOUT FUNCTION/WITH INOUT;EXPLICIT/ASSIGNMENT/IMPLICITcontexts),DROP, andCOMMENT ON. Managed: casts are auto-dropped when removed from source.WITH FUNCTIONis constrained to managed SQL/plpgsql functions — source rejects references to unmanaged or built-in functions via thecast-references-unmanaged-functionlint. Built-in/system casts (pg_cast.oid < 16384) and extension-owned casts are excluded from introspection. NoALTER CASTin Postgres, so any structural change is drop + create; identity is(source_type, target_type).
[0.4.1] — 2026-06-07
Added
AGGREGATEsupport. User-defined aggregates viaCREATE AGGREGATE(ordinary form: SFUNC + STYPE + optional FINALFUNC/INITCOND),ALTER … OWNER TO,DROP, andCOMMENT ON. State and final functions must be managed SQL/plpgsql functions; source rejects references to unmanaged or built-in functions via theaggregate-references-unmanaged-functionlint. The reader skips ordered-set aggregates, moving aggregates, and aggregates whose state function is in an unreadable language. Rename is drop + create; identity is(schema, name, arg_types).
0.4.0 — 2026-06-06
Added
EVENT TRIGGERsupport.CREATE EVENT TRIGGER(withON <event>andWHEN TAG IN (...)filter),ALTER … ENABLE/DISABLE/ENABLE REPLICA/ENABLE ALWAYS,ALTER … OWNER TO,DROP, andCOMMENT ON. Database-global with a lenient drop policy (unmanaged event triggers surface via theunmanaged-event-triggerlint, never auto-dropped), mirroring publications/subscriptions. Extension-owned event triggers are excluded from introspection.TABLESPACEsupport (cluster object). Cluster-levelTABLESPACEmanagement via thepgevolve cluster …surface:CREATE(withOWNER,LOCATION,WITH (options)),ALTER … OWNER TO,ALTER … SET (options),DROP(intent-gated), andCOMMENT ON. Lenient owner + lenient options.LOCATIONis immutable, so a location drift surfaces via thetablespace-location-driftadvisory rather than a destructive recreate. Filesystem-layout management (directory creation, mount points) stays out of scope. Tablespaces are declared in atablespaces/cluster-source directory.- Table access method support.
CREATE TABLE … USING <am>is parsed and rendered, with the access method read frompg_class.relam. The built-inheapis normalized to the implicit default (declaringUSING heapis a no-op). An access-method change on an existing table surfaces via thetable-access-method-changeadvisory — pgevolve does not auto-rewrite tables (ALTER TABLE … SET ACCESS METHODis PG 15+ and implies a full rewrite). SubscriptionOptions.connect— CREATE-only directive. New IR field that maps to PG'sCREATE SUBSCRIPTION ... WITH (connect = ...)option.connect = falsecreates the subscription without dialing the publisher (the only way to construct a subscription against a publisher that may not be reachable). The option is CREATE-only —pg_subscriptiondoesn't store it — so the catalog reader always returnsNoneand the diff never emits an ALTER for it. Closes #14.
Changed
- Cluster apply reaches per-DB parity.
pgevolve cluster applynow bootstrapspgevolvemetadata, acquires the singleton advisory lock, runs cluster preflight (identity match + intent approval), writes anapply_logrow, executes via the per-DB group executor, and closes the audit row.pgevolve cluster planwrites the canonical 3-file plan layout (structuredplan.sqlheaders +intent.toml+manifest.tomlwithtarget_identity). The clustertarget_identityformat iscluster:{system_identifier_hex16}. Closes #7.
Fixed
Diff + planner
- New tables, sequences, schemas, and indexes carry their full attribute set on CREATE. The diff previously emitted only the bare CREATE for newly-declared objects; owner, grants, policies, storage options, and the RLS-forced bit were silently dropped and required a subsequent ALTER cycle to land. Fresh applies and empty-DB bootstraps now produce live state matching the source IR on the first apply.
CREATE INDEXemits theWITH (...)storage clause. The clause was previously dropped at render time; index reloptions (fillfactor,fastupdate, etc.) needed a follow-upALTER INDEX SET (...)to land. Now emitted inline.- Column STORAGE is no longer inline-rendered in
CREATE TABLE/ADD COLUMN. Inline STORAGE is PG 16+ syntax. The renderer now emitsALTER COLUMN ... SET STORAGEas a follow-up step, restoring PG 14/15 compatibility. CREATE SUBSCRIPTIONruns outside transaction whencreate_slot != false. PG forbids tx-blockCREATE SUBSCRIPTIONwithcreate_slot = true(the default). Planner now setsTransactionConstraint::OutsideTransactionbased on subscription options. Closes #11.DROP SUBSCRIPTIONruns outside transaction. PG forbids tx-block DROP when the subscription has an attached slot. The IR can't tell at diff time whether a slot is attached; conservative path always uses out-of-transaction. Closes #26.streamingsubscription option emits boolean form on PG ≤15. PG ≤15 accepts onlytrue/falseforstreaming; theparallelkeyword is PG 16+. Renderer no longer emits string forms. Closes #24.- Subscription reader accepts boolean substream column values. Companion to the renderer fix above; PG's
pg_subscription.subsubstreamis returned via::textcast asf/t/false/true/parallel. Closes #28. - REVOKE statements emit before GRANT in the same diff step. PG's
REVOKE priv FROM roleremoves the privilege regardless ofgrant_option; ordering GRANT-before-REVOKE silently cancelled a just-emitted grant. Affectswith_grant_optionupgrades and downgrades on tables, sequences, schemas, views, materialized views, functions, procedures, and types. Closes #33. default_privilegesdiff emits REVOKE when grants shrink. Previously, removing a grant from an existing rule left the live database with the old grant intact. Closes #23.default_privilegesrules carry their grant set when newly declared. Was previously a no-op in the source-only diff branch. Closes #25.- Column-level REVOKE elided when the column is dropped in the same plan. PG cascade-revokes column ACLs during column drop; the explicit REVOKE then failed with
column does not exist. Closes #39. - Column-grant diff ignores columns being dropped. When a column carrying a multi-column grant is dropped (live
GRANT SELECT (id, price), source keepsGRANT SELECT (id)), the diff previously emittedREVOKE SELECT (id, price)— naming the droppedprice— which failed with[42703] column "price" of relation "X" does not exist. The table grant diff now strips dropped columns from the target grants before diffing, so the column drop alone converges the privilege and no spurious REVOKE/GRANT is emitted. (Generalises #39, which only covered grants whose entire column list was dropped.) DROP SCHEMAcascade-emitsDROP COLLATION. The diff previously left dependent collations incatalog.collations, tripping PG error2BP01on apply. The dependency graph now also carries a Collation → Schema edge so the drops sort correctly. Closes #38.change_node()returns the correctNodeIdvariant per object kind. Owner/grant changes on sequences/schemas/views/types/MVs/procedures/collations previously defaulted toNodeId::Table(qname), causing incorrect interleaving across object kinds when sorting plan steps. Closes #36 (one of two root causes).
Reader + canonicalization
- PG-implicit grants stripped from
default_privilegescanonical IR. PG silently injects(grantee = target_role, full self-priv)into everypg_default_acl.defaclacl, and additionally injects(Public, USAGE)on TYPES and(Public, EXECUTE)on FUNCTIONS. Both the catalog reader and the source parser now strip these implicit entries so source and live IRs match. Without this, the diff emitted spurious REVOKEs that deleted entirepg_default_aclrows, manifesting aspresent vs removeddivergences in round-trip tests. Closes #34 + #37. default_privilegesimplicit-PUBLIC strip now runs in canon too. The reader/parser strip above did not cover IR built by the testkit generator (which bypasses the SQL parser): an implicit(Public, USAGE)on TYPES — or(Public, EXECUTE)on FUNCTIONS — survived on the source side while the live reader stripped it, reappearing asdefault_privileges.*.TYPES/FUNCTIONS: present vs removed. The canon pass now applies the same strip, then drops any rule it empties out, so every source path (parser and generator) normalises identically to live.- Empty publications are skipped on read. PG permits an empty publication —
CREATE PUBLICATION p;, or a selective publication whose last table/schema was dropped (PG silently empties it rather than dropping it). pgevolve cannot model one (the parser and canon reject emptySelectivescopes, so it can never appear in source). The catalog reader previously failed the entire introspection withempty Selective scope (no tables, no schemas); it now skips the unmanaged empty publication, keeping the database readable. (Surfaced by the lenient drop policy: dropping a publication's last table leaves the publication empty because the publication itself is never auto-dropped.) Rangetypesubtype_opclassandcollationcanonicalize to None when matching PG defaults. PG resolves and stores the default opclass (e.g.timestamptz_ops) and thepg_catalog.defaultcollation even when the user omits them; canon now strips the resolved value back to None so source IR matches live IR after read-back. Closes #35.- Policy roles
[PUBLIC, X]canonicalize to[PUBLIC]. PG accepts the list at CREATE POLICY time but silently drops named roles when PUBLIC is present (PUBLIC includes all roles). Canon now aligns source IR with PG's stored form. Closes #31. - Owner self-grants stripped from source IR. The live catalog reader has long stripped grants where
grantee == ownerfrom tables/sequences/schemas/views/etc.; source IR now matches symmetrically. The asymmetry causeddiff(live, source)to be non-empty whenever the IR generator (or a user) happened to write the owner into the grants list. Closes #36 (second root cause). default_privilegesrules with empty grant sets stripped in canon. PG has no DDL form for a zero-grant default-privilege rule; the rule only materializes inpg_default_aclwhen at least one grant is in effect. Canon now removes empty-grant rules from source IR so they don't show aspresent vs removeddivergences.- RLS policy predicates respect PG's per-command matrix.
FOR INSERTpolicies use onlyWITH CHECK(PG rejectsUSINGfor INSERT with error42601);FOR SELECT/DELETEuse onlyUSING;FOR UPDATE/ALLuse both. Closes #22.
Test infrastructure (no user-visible change)
- Soak workflow now sets
TEST_PWDso subscription apply doesn't trip the env-var preflight. - Testkit IR generator tightened across many shapes: subscription options gated by PG version (
origin,failover,two_phase,disable_on_error); RLS policy predicates gated by command; default_privileges restricted to valid(grantee, object_kind)and(schema, object_type)matrices; index storage options gated by access method; STATISTICS columns restricted to types with a default B-tree opclass; publication scope guards empty Selective output and PG 15+FOR ALL TABLES IN SCHEMAform; mutation cascades clean dependent references ondrop_schema(default privileges, types, statistics, views, collations) anddrop_column(constraints, grants, statistics, indexes); grant generator never combinesWITH GRANT OPTIONwithPUBLIC. - Mutator drop cascades extended to match PG's automatic dependency handling, eliminating spurious
Statistic(Create)divergences and empty-publication read errors in the soak:drop_tablenow also drops extended statistics targeting the table and removes the table from publications (dropping any publication left empty);drop_columndrops a statistic entirely when one of its columns is dropped (PG removes the whole object, it does not shrink the column list — verified empirically) and removes the table from any publication whose column list names the dropped column;drop_schemaadditionally drops statistics whose target table lived in the schema and prunes publications referencing the schema or its tables. cargo xtask soak-streaktracks the consecutive-clean-soak-day counter feeding the v1.0 release gate (sub-project C).Plan::from_grouped_with_idconstructor lets callers supply a pre-computedPlanId(used by cluster apply, which hashesClusterCatalograther than per-DBCatalog).
Removed
pgevolve::executor::apply_cluster_steps(public API). Callers that previously built aVec<RawStep>and applied it directly should now build aPlanand useapply_cluster_planinstead.
[0.3.9] — 2026-05-28
Patch for the broken v0.3.8 — no new features.
Fixed
- Collation catalog reader on PG 15 and PG 16. v0.3.8 shipped with
SQL that used
pg_collation.colllocalefor "PG 16+" — butcolllocalewas introduced in PG 17, not PG 16. PG 15 addedcolliculocale(ICU-only) and PG 17 renamed it tocolllocale(generic, since the newbuiltinprovider also uses it). ICU rows on PG 15 and PG 16 leftcollcollateNULL, sopgevolvereturned emptylc_collatestrings and either crashed on decode or triggered a "column c.colllocale does not exist" SQL error. Three-way per-version SQL: PG 14 (legacycollcollate), PG 15/16 (colliculocale), PG 17/18 (colllocale). Commits09cd563a30d7f3.
- Tier-3 catalog round-trip snapshots re-blessed. Stage 2 of the
v0.3.8 plan added
Catalog::collations: Vec<Collation>but the JSON snapshot fixtures undercrates/pgevolve-core/tests/fixtures/catalog/pg{14,15,16,17}/were never re-blessed for the new field. Stage 11's pre-release verify ran the conformance suite (-p pgevolve-conformance) rather than the tier-3 round-trip inpgevolve-core, so the gap surfaced only in CI on the v0.3.8 push. All 28 snapshots re-blessed viacargo xtask bless. Commit09cd563.
Yanked
- v0.3.8 yanked from crates.io. ICU collation reads against PG 15 or PG 16 fail; users should upgrade to v0.3.9.
[0.3.8] — 2026-05-28
Added
CREATE COLLATIONas a first-class IR object. libc / ICU / PG 17+builtinproviders with thedeterministictoggle, RENAME, andCOMMENT ON COLLATIONall managed. Source may use thelocale = 'X'shorthand or explicitlc_collate+lc_ctype; the IR always stores the latter and the renderer collapses back to the shorthand when they match.pg_collation.collversionis read-only (differ ignores it);ALTER COLLATION … REFRESH VERSIONand the matchingcollation-version-driftlint are deferred to v0.3.9.CREATE TYPE … AS RANGE— additiveUserTypeKind::Rangevariant on the existing user-type machinery. Models subtype, subtype opclass, collation, canonical fn, subtype_diff fn, and an optional custom multirange type name. Any structural change goes through the existingReplaceWithCascadepath — Postgres has no in-place ALTER for range fields. Auto-generated multirange types filtered frompg_typeviatyptype != 'm'.- 5 new lint rules:
unmanaged-collation,column-references-unmanaged-collation,range-type-references-unmanaged-subtype,nondeterministic-collation-requires-pg-12,builtin-provider-requires-pg-17. - 5 new
StepKindvariants for collation lifecycle:CreateCollation,DropCollation,RenameCollation,ReplaceCollation,CommentOnCollation. Dep-graph gainsNodeId::Collationplus 4 edge types (Column → Collation, Domain → Collation, Range → Collation, CompositeAttribute → Collation). - 11 conformance fixtures: 6 under
objects/collations/(create-libc,create-icu,create-nondeterministic,drop,comment-on,replace-on-locale-change) and 5 underobjects/ranges/(create-simple-int4range,create-with-opclass,create-with-subtype-diff-fn,drop,column-with-range-type), plus thescenarios/column-references-managed-collationcross-kind fixture. The originally-plannedobjects/collations/renamewas substituted toreplace-on-locale-change(rename is exercised in unit + property tests; replace-on-structural-change is the higher-value end-to-end path). The originally-plannedobjects/ranges/create-with-canonical-fnwas substituted tocreate-with-subtype-diff-fn(canonical-fn requires authoring a matching C/PLpgSQL function, which is out of scope for v0.3.8 fixtures).
Fixed
- Range-type round-trip — the differ now treats source-side
Nonefor range-type optional fields (opclass, collation, canonical, subtype_diff, multirange_type_name) leniently, matching the established "sourceNonemeans unmanaged" pattern. Previously any catalog-side default would spuriously diff against an unpinned source. Surfaces a newresolve_user_defined_typescanon pass that resolvesColumnType::Other(qname)references againstCatalog::typesafter the source parse pass — applies to any user-defined type, not just ranges. (054364a) - ICU collation locale reader on PG 16+ —
pg_collation.colllocalereplacedcollcollate/collctypefor ICU rows in PG 16+. The reader now selects the right column per PG major; previously ICU collations dumped on PG 16+ surfaced as empty locales. (51fc476)
Out of scope (deferred to v0.3.9+)
CREATE COLLATION FROM existing_collationform.ALTER COLLATION … REFRESH VERSIONand the matchingcollation-version-driftlint.- Multirange-type customization beyond
multirange_type_name(no per-multirange opclass / canonical fn surface yet). - A first-class
MultirangeIR object distinct fromRange— multiranges are still modeled implicitly via the parent range.
[0.3.7] — 2026-05-27
Added
CREATE STATISTICS— multi-column statistics objects (ndistinct, dependencies, mcv) with PG 14+ expression statistics. Explicit names required (anonymous form rejected, mirroring index-naming policy). Granular differ:AlterStatisticSetTargetfor the cheapSET STATISTICS npath;ReplaceStatistic(DROP + CREATE) for any other change since PG has no in-place ALTER for column lists or kinds.CREATE VIEW … WITH CHECK OPTION— per-viewcheck_option: Option<CheckOption>(Local|Cascaded). Parser folds both SQL-clause and WITH-options forms; differ emitsCREATE OR REPLACE VIEW.- 5 new StepKind variants for STATISTICS + 1 for views:
CreateStatistic,DropStatistic,ReplaceStatistic,AlterStatisticSetTarget,CommentOnStatistic,AlterViewSetCheckOption. unmanaged-statisticlint (Warning, waivable) — standard v0.3.x lenient-drift surface.- 9 conformance fixtures (3 views + 6 statistics).
Closes
Third and fourth items from the post-v0.3.3 agreed roadmap
(STATISTICS was 📋 Planned in objects.md; CREATE VIEW … WITH CHECK OPTION was 🔮 Future).
[0.3.6] — 2026-05-27
Added
- Postgres 18 catalog support.
PgVersion::Pg18variant;catalog/queries/pg18.rsthin re-export of shared (PG 18 is fully backward-compatible with PG 17 catalog queries — no divergences found). Tier-2/3/C suites green under PG 18. CI matrix exercises PG 18 on every push. [managed].min_pg_versionnow accepts18.
Notes
- v0.3.6 is catalog-read + conformance only. New PG 18 IR features (virtual generated columns, etc.) are explicitly deferred to v0.4.1 per the roadmap.
- Constitution §6 now reads "14, 15, 16, 17, and 18" as actively-maintained PG majors.
[0.3.5] — 2026-05-26
Added
- SUBSCRIPTION as a first-class IR object. Per-field lenient
SubscriptionOptions(enabled, binary, streaming Off/On/Parallel, two_phase, disable_on_error PG15+, password_required + run_as_owner- origin PG16+, failover PG17+). Opaque CONNECTION string with
${VAR}env-var interpolation.
- origin PG16+, failover PG17+). Opaque CONNECTION string with
- Apply-time
${VAR}resolution. Source IR and plan.sql store unresolved${VAR}placeholders. Preflight scans every step's SQL, resolves against process env, fails before any DB connection if a reference is unset. Secrets never persist to disk. - 8 new StepKind variants for subscription operations.
- 4 lint rules:
unmanaged-subscription(Warning),subscription-references-undeclared-publication(Warning),subscription-feature-requires-pg-version(Error, not waivable),subscription-password-in-source(Error, not waivable) — catches plaintextpassword=at parse time. [fixture] applyflag in the conformance harness so fixtures with cross-cluster side-effects (subscriptions) can validate parse/diff/plan/lint without applying.- 12 conformance fixtures under
objects/subscriptions/.
Closes
Second item from the post-v0.3.3 agreed roadmap (next: CREATE VIEW WITH CHECK OPTION).
[0.3.4] — 2026-05-26
Added
- PUBLICATION as a first-class IR object. All 5 PG syntactic
forms (explicit FOR TABLE, FOR ALL TABLES, FOR TABLES IN SCHEMA
PG15+, row filters PG15+, column lists PG15+).
PublicationScopesum-type encodes PG's mutual exclusion of AllTables vs Selective. - Granular ALTER PUBLICATION semantics. 11 new StepKind variants (add/drop/set per table, add/drop per schema, set publish, etc.) — each plan step is independently auditable and rollback-safe.
[managed].min_pg_versionconfig key. Defaults to 14; raise to 15+ to use row filters, column lists, or schema-scope. PG-version-gated source features fail at lint time (publication-feature-requires-pg-version, Error) instead of at apply with a Postgres syntax error.- 4 lint rules:
unmanaged-publication(Warning),publication-captures-unmanaged-table(Warning),publication-row-filter-references-unmanaged-column(Warning),publication-feature-requires-pg-version(Error, not waivable). - 12 conformance fixtures under
objects/publications/.
Closes
Slipped from the v0.3 roadmap commitment (next: v0.3.5 SUBSCRIPTION).
[0.3.3] — 2026-05-23
Added
- Storage parameters / reloptions on tables, indexes, materialized views. Typed
Option<T>fields for the well-known keys (fillfactor, autovacuum_*, parallel_workers, fastupdate, buffering, pages_per_range, etc.) plusextra: BTreeMap<String, String>for extension-registered or unknown keys. Tables and MVs share the autovacuum substruct since PG documents identical key sets. - Per-AM fillfactor validation at parse time: B-tree 50..=100, GiST 10..=100, SP-GiST 90..=100, BRIN/GIN reject fillfactor.
- Lenient drift policy: source
Nonealways means "unmanaged" — never triggersRESET.unmanaged-reloptionlint surfaces catalog reloptions not in source. - 3 new StepKind variants:
SetTableStorage,SetIndexStorage,SetMaterializedViewStorage. One ALTER step per relkind per diff (batches multiple keys into one SET). unmanaged-reloptionlint (warning, waivable).- Source parser for
WITH (...)on CREATE TABLE/INDEX/MATERIALIZED VIEW andALTER ... SET (...).RESET (...)rejected in source. - Catalog reader decodes
pg_class.reloptions::text[]into typed structs. - 11 conformance fixtures.
Known limitations
CREATE TABLE/INDEX/MATERIALIZED VIEW … WITH (…)against a brand-new object emits the CREATE without the inlineWITH. Convergent on the next plan run viaALTER … SET. Same gap as owner/grants/policies on new objects in v0.3.x; will be closed uniformly in a follow-up.
Closes
Slipped v0.2 commitment from docs/spec/objects.md (table reloptions row marked 🟡 Partial). Per-partition storage parameters also satisfied (partitions inherit since they're Table in IR).
[0.3.2] — 2026-05-22
Added
- Row-level security policies —
Tablegainsrls_enabled,rls_forced, andpolicies: Vec<Policy>. Policies carrypermissive,command,roles,using,with_check. USING / WITH CHECK reuseNormalizedExprcanonicalization shared with check constraints. - Source parser:
CREATE POLICY+ fourALTER TABLE ... { ENABLE | DISABLE | FORCE | NO FORCE } ROW LEVEL SECURITYsubcommands.ALTER POLICYandDROP POLICYrejected in source (diff-driven). - Differ: 5 new Change variants (
CreatePolicy,DropPolicy,AlterPolicy,SetTableRowSecurity,SetTableForceRowSecurity). Command-kind changes recreate (DROP + CREATE) because PG doesn't allowALTER POLICYto change the command. - Catalog reader: new
pg_policiesquery +relrowsecurity/relforcerowsecurityon the tables query. - Two lint additions:
grant-references-unknown-role(existing) now also walks policyTOclauses.force-rls-without-policies(new, Warning) — fires when a table has FORCE RLS enabled but no policies defined (PG would deny all rows).
- Conformance: 11 new fixtures under
objects/policies/.
Closes
v0.3 security/permissions trilogy: roles (v0.3.0) → grants (v0.3.1) → policies (v0.3.2).
[0.3.1] — 2026-05-22
Added
- Object grants + ownership — all 8 grantable IR types (Schema, Sequence, Table, View, MaterializedView, Function, Procedure, UserType) gain
owner: Option<Identifier>+grants: Vec<Grant>. Column-level grants on tables/views/MVs. - Default privileges —
Catalog.default_privileges: Vec<DefaultPrivilegeRule>mirroringpg_default_acl. SupportsFOR ROLE x IN SCHEMA y GRANT/REVOKE ... ON {TABLES, SEQUENCES, FUNCTIONS, TYPES, SCHEMAS}. - Lenient drift policy — catalog grants to roles outside source surface as
grants-to-unmanaged-rolewarning, never silently revoked. - Optional cluster-link —
[cluster].projectin pgevolve.toml validates grantee role names against the linked cluster project'sroles/*.sqlvia thegrant-references-unknown-rolelint (Error severity). - Three new lint rules:
grant-references-unknown-role(Error, cluster-aware) — grantee not in linked cluster source.grants-to-unmanaged-role(Warning) — catalog grants to roles not declared in source.revoke-from-owner(Error) — REVOKE would target object's owner.
- Six new StepKind variants:
AlterObjectOwner,GrantObjectPrivilege,RevokeObjectPrivilege,GrantColumnPrivilege,RevokeColumnPrivilege,AlterDefaultPrivileges.
Catalog reader
- New
catalog::grantsmodule decodes PG aclitem text format. All 6 family queries gain<obj>owner+<obj>acl::text[]. Tables/views/MVs also decodepg_attribute.attaclfor column-level grants. Owner self-grants stripped (PG materializes them when any explicit grant exists; they're implicit by ownership).
Source parser
- Three new builder modules: object-level
GRANT,ALTER ... OWNER TO,ALTER DEFAULT PRIVILEGES.GRANT ALLexpands per object type. Column-level grants extracted fromAccessPriv.cols. REVOKE rejected in source.
Shadow validate
validate --shadownow respects "unmanaged owner" semantics: when source declaresowner = None(noOWNER TO), shadow-validate ignores any catalog-side owner. Similarly for grants — only managed grantees compared.
Conformance
- 13 new fixtures under
objects/grants/covering table/schema/function/sequence/owner/default-privs/lint sub-roots. Two cluster-link fixtures deferred (harness extension out of scope for v0.3.1).
[0.3.0] — 2026-05-22
Added
- Cluster-level surface — new project type (
pgevolve-cluster.toml + roles/), new command family (pgevolve cluster init/diff/plan/apply/status), new executor running against a superuser DSN. Per Decision 23 of the v0.2 architecture review. ROLE/CREATE USERfully managed —ClusterCatalog.roleswith full PG attribute matrix (superuser, createdb, createrole, inherit, login, replication, bypass_rls, connection_limit, valid_until), plus role membership via inlineIN ROLEorGRANT role TO target. Passwords intentionally not modeled.- Two new universal lint rules:
role-loses-superuser(warning) — fires onALTER ROLE … NOSUPERUSERwhen the role had superuser.role-membership-cycle(error) — detects cycles in the projected post-apply membership graph; pre-empts PG's apply-time rejection.
- Conformance harness — new
authoring = "cluster"mode + seven fixtures undercases/cluster/roles/. - Property tests —
arbitrary_role_attributes,arbitrary_cluster_cataloggenerators with cycle-free membership; diff round-trip invariant:diff_cluster(A, B)applied toAyieldsBmodulo canonicalization.
Catalog reader
- New
read_cluster_catalog(querier, bootstrap_roles)queryingpg_authid+pg_auth_members. Filterspg_*predefined roles and caller-supplied bootstrap roles.
Known v0.3.0 gaps (closing in 0.3.x)
- Cluster apply: reads
plan.sqland runs each statement in a transaction.intent.tomldestructive-step gates and apply-log tracking are deferred. - No advisory lock during cluster apply; concurrent applies are not protected.
- Object-level GRANT/REVOKE (per-DB) lands in v0.3.1.
- RLS policies (per-DB) land in v0.3.2.
[0.2.1] — 2026-05-21
Added
- Per-column TOAST storage —
STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }is now a managedColumnattribute. Source parser accepts both inline (col text STORAGE EXTERNAL, PG 16+ syntax) andALTER COLUMN SET STORAGEforms. Differ emits non-destructiveSET STORAGEsteps; canon strips type-default values so explicit and implicit defaults are equivalent. - Per-column TOAST compression —
COMPRESSION { pglz | lz4 }is now a managed attribute.Nonepreserves the clusterdefault_toast_compressionGUC; explicitpglzorlz4overrides it.SET COMPRESSION DEFAULTround-trips through the parser asNone. - Two new lint rules (surfaced as
Plan.advisory_findingsand printed bypgevolve planto stderr):storage-downgrade-not-retroactive— warns when a SET STORAGE change reduces toastability (e.g.EXTERNAL → MAIN), since existing TOASTed values aren't rewritten until UPDATE or VACUUM FULL.compression-change-not-retroactive— warns on any compression change for the same reason.
Catalog reader
COLUMNS_QUERYnow selectsattstorageandattcompressionfrompg_attribute. No per-version split; both columns are present in PG 14+ (the project MSRV).
Conformance
- Five new fixtures under
objects/columns/:set-storage-external,set-storage-plain-warning,set-compression-lz4,create-table-with-storage,set-storage-type-default-noop.
0.2.0 — 2026-05-21
Extends the v0.1 surface with views, materialized views, user-defined types, functions/procedures, extensions, triggers, and declarative partitioning as fully-managed objects. The differ, planner, linter, conformance suite, and property tests all cover the new object kinds. Ships alongside a project constitution (docs/CONSTITUTION.md), cargo-deny-enforced license + advisory policy (deny.toml), CLAUDE.md agent guidance, and shadow validation for view bodies (T13).
Added — internal architecture (2026-05-19)
pgevolve-core-macroscrate — internal proc-macro crate exposing#[derive(DiffMacro)]. Most IR structs (Schema,Sequence,Column,Constraint,ForeignKey,Procedure,Index) now derive theirDiffimpl with#[diff(skip)]/#[diff(via_debug)]/#[diff(nested)]field attributes. Hand-written impls retained where they have non-trivial logic (Catalog,Function,Table,View,MaterializedView,UserType, and the enum impls). Removes ~250 lines of mechanical boilerplate.ir::canonpipeline — every IR-value normalization rule moved into a single ordered pipeline. Four named passes:filter_pg_defaults(sequence min/max, function cost/rows,pg_catalog.defaultcollation →None);sentinel_view_columns(view/MV column types → shared sentinel);renumber_enum_sort_orders(enum sort_order →1.0, 2.0, 3.0, …);sort_and_dedupe(canonical-key sort + duplicate detection).Catalog::canonicalizeis now a thin wrapper. Catalog reader and source builders are kept "raw" — they no longer filter PG defaults. The rule for the next PG-default surprise lives in one place.pgevolve::api::build_plan— library entry point that runs the full parse→introspect→diff→order→rewrite→group→assemble pipeline and returns aPlanvalue. Noprintln!, no waiver-prompt UX, no--shadow-validate, no on-disk plan directory.pgevolve::executor::apply_plan(&Plan, …)— sibling toapply(plan_dir, …)that takes an in-memoryPlan. The disk-basedapplyis now a thin shim that callsread_plan_dirthen delegates. CLIplan/applycommands are thin wrappers overapi::build_plan/executor::apply_planplus CLI UX.Plan::approve_all_intents— helper onpgevolve_core::plan::Planfor test harnesses building plans programmatically. Production apply still requires explicitintent.tomlapproval.
Changed — conformance suite (2026-05-19)
- Conformance Layer 4 (apply roundtrip) now runs in-process via
pgevolve::api::build_plan+pgevolve::executor::apply_plan. The subprocess scaffolding (cargo_bin,run_pgevolve,plan_and_locate,patch_intent_toml_approve_all,write_project) is gone — ~150 fewer lines incrates/pgevolve-conformance/src/assertions/apply.rs. Faster (no per-fixture binary rebuild + spawn) and easier to debug (assertions can inspect thePlanvalue rather than its on-disk rendering).
Added — shadow validation for view bodies (T13)
--shadow-validatenow cross-checks view + materialized view bodies against an ephemeral Postgres.render_catalogemits views/MVs (after sequences);cross_checkqueriespg_get_viewdeffor each declared view/MV, re-parses through the source canonicalizer, and compares against the source IR's body canonical text + body_dependencies (the latter walked viapg_rewrite → pg_depend → pg_class). Defaults to warnings;--shadow-strictpromotes mismatches to errors.- New
crates/pgevolve-core/src/render/view.rs(render_view,render_materialized_view). - Docker-gated integration tests in
crates/pgevolve/tests/shadow_validate_views.rs. - Closes the deferred T13 plan from sub-spec #1 (views/MVs).
Added — IR (functions and procedures)
Function { qname, args, arg_types_normalized, return_type, language, body, body_dependencies, volatility, strict, security, parallel, leakproof, cost, rows, comment }flat IR type inpgevolve-core::ir::function.Procedure { qname, args, language, body, body_dependencies, security, commits_in_body, comment }flat IR type inpgevolve-core::ir::procedure.FunctionArg { name, mode: ArgMode, ty, default }— argument declaration with IN/OUT/INOUT/VARIADIC modes.NormalizedArgTypes { types, canonical_hash }— BLAKE3 hash over comma-joined IN/INOUT/VARIADIC type strings; the function identity disambiguator for overloads.ReturnType—Scalar,SetOf,Table { columns },Trigger,EventTrigger,Void.FunctionLanguage—Sql|PlPgSql.Catalog::functions: Vec<Function>andCatalog::procedures: Vec<Procedure>— flat collections, sorted by(qname, arg_types_normalized)/qnameaftercanonicalize().
Added — pipeline (functions and procedures)
- Source parser —
CREATE FUNCTIONandCREATE PROCEDUREparse into theFunction/ProcedureIR. Full attribute matrix (volatility, strict, security, parallel, leakproof, cost, rows). Dollar-quote body extraction for both SQL and PL/pgSQL languages. - PL/pgSQL body parser (
parse/builder/plpgsql.rs) — wraps the body in a syntheticCREATE FUNCTIONand callspg_query::parse_plpgsql. Extracts static embedded SQL dep edges (PLpgSQL_stmt_execsql), detectsCOMMIT/ROLLBACKnodes forcommits_in_body, and scans-- @pgevolve dep:directives for dynamic SQL. - AST resolution — validates routine body dep edges against the catalog; unresolved managed-schema references surface as warnings.
- Catalog reader — queries
pg_proc(withpg_language,pg_type,pg_namespace) to reconstructFunctionandProcedurefrom a live database. Handles multi-arg functions, OUT args, overloads, and all attribute columns. - Differ —
FunctionChangevariants:Create,Drop,OrReplace,ReplaceWithCascade,CommentOn.ProcedureChangevariants:Create,Drop,OrReplace,CommentOn. - OR-REPLACE compatibility predicate (
function_can_or_replace) — returnstruewhen language, return type, and OUT/INOUT parameters are all unchanged; falls back toReplaceWithCascadeotherwise. - Planner — 6 new step kinds:
CreateOrReplaceFunction,DropFunction,CommentOnFunction,CreateOrReplaceProcedure,DropProcedure,CommentOnProcedure. Procedures withcommits_in_body = trueare placed in non-transactional steps. NodeId::Function/NodeId::Procedure— added to the dep graph; body dep edges drive correct creation/drop ordering relative to their referenced tables, views, and types.
Added — lint rules (functions and procedures)
plpgsql-dynamic-sql(Error) — PL/pgSQL body usesEXECUTEwithout a-- @pgevolve dep:directive.procedure-contains-commit(Warning) — procedure body containsCOMMITorROLLBACK; runs withtransactional=OutsideTransaction.function-references-unmanaged-schema(Warning) — routine body dep edge targets an unmanaged schema.
Added — tests (functions and procedures)
- ~22 conformance fixtures (Tier C):
objects/functions/andobjects/procedures/covering SQL functions, PL/pgSQL functions, procedures, overloads, dep-edge extraction,ReplaceWithCascade, and all three lint rules. - Property test
plpgsql_canonicalization_is_idempotent(#[ignore], pure, no Docker) — for each body in the representativePLPGSQL_BODIEScorpus,parse_routine_body → canonical_text → re-parse → canonical_textproduces byte-identical output. Closes the round-trip invariant the differ relies on.
Added — IR (triggers)
Trigger { qname, table_name, function_name, timing: TriggerTiming, events: Vec<TriggerEvent>, for_each: ForEach, when_clause: Option<NormalizedExpr>, update_columns: Vec<Identifier>, referencing: Option<TransitionTables>, constraint: bool, deferrable: bool, initially_deferred: bool, comment: Option<String> }flat IR type inpgevolve-core::ir::trigger.TriggerTiming—Before|After|InsteadOf.TriggerEvent—Insert|Update|Delete|Truncate.ForEach—Row|Statement.TransitionTables { old_table: Option<Identifier>, new_table: Option<Identifier> }— REFERENCING clause for transition tables.Catalog::triggers: Vec<Trigger>flat collection, sorted by(table_name, qname)aftercanonicalize().
Added — pipeline (triggers)
- Source parser —
CREATE [CONSTRAINT] TRIGGERparses into theTriggerIR. BEFORE/AFTER/INSTEAD OF timing, INSERT/UPDATE/DELETE/TRUNCATE events, FOR EACH ROW/STATEMENT, WHEN clause (asNormalizedExpr), UPDATE OF column list, and REFERENCING transition tables all modeled.ALTER TRIGGERin source files rejected at statement classification. COMMENT ON TRIGGERparser arm — recognized alongsideCOMMENT ON FUNCTIONandCOMMENT ON EXTENSIONin the comment-statement path.- Catalog reader — queries
pg_triggerjoined withpg_class,pg_namespace, andpg_description. Filters:NOT tgisinternal(system-generated triggers excluded);NOT EXISTS (pg_depend deptype='e')(extension-owned triggers excluded). Reconstructs all modeled fields including constraint, deferrable, and initially-deferred flags. - Differ —
TriggerChangevariants:Create,Drop,CommentOn. Any structural difference (timing, events, for-each, when-clause, function, columns, transition tables, constraint flags) emitsDrop+Create— there is noALTER TRIGGERfor body-level changes.CommentOnis comment-only. - Planner — 3 new step kinds:
CreateTrigger,DropTrigger(destructive; intent required),CommentOnTrigger.DropTriggeris placed in the same destructive bucket asDropTableandDropFunction. NodeId::Trigger— added to the dep graph;Trigger → Table/View/MVedges ensure the target relation exists before the trigger is created;Trigger → Functionedges ensure the trigger function exists before the trigger fires.
Added — lint rules (triggers)
trigger-references-unmanaged-table(Warning) — trigger's target relation is not in any managed schema.trigger-references-unmanaged-function(Warning) — trigger function is not in any managed schema.
Added — tests (triggers)
- Conformance fixtures (Tier C):
objects/triggers/covering create/drop/comment, BEFORE/AFTER/INSTEAD OF variants, ROW vs STATEMENT, WHEN clause, UPDATE OF columns, REFERENCING transition tables, CONSTRAINT TRIGGER with DEFERRABLE, and both lint rules.
Added — IR (partitioning)
partition_by: Option<PartitionBy>andpartition_of: Option<PartitionOf>fields added toTableinpgevolve-core::ir::table.- New
ir/partition.rsmodule:PartitionBy { strategy: PartitionStrategy, columns: Vec<PartitionColumn> }— thePARTITION BYclause on a partitioned parent.PartitionStrategy—Range|List|Hash.PartitionColumn { kind: PartitionColumnKind, collation: Option<QualifiedName>, opclass: Option<QualifiedName> }— a single partition key element with optional collation and opclass overrides.PartitionColumnKind—Column(Identifier)|Expr(NormalizedExpr).PartitionOf { parent: QualifiedName, bounds: PartitionBounds }— thePARTITION OF parent FOR VALUES …clause on a partition child.PartitionBounds—Range { from, to }|List { values }|Hash { modulus, remainder }|Default.BoundDatum—Literal(NormalizedExpr)|MinValue|MaxValue.
Added — pipeline (partitioning)
- Source Form 1 —
CREATE TABLE child PARTITION OF parent FOR VALUES …parsed directly intoTable { partition_of: Some(…) }. The parent's key is inferred from itspartition_by. - Source Form 2 — standalone
CREATE TABLE child PARTITION OF parent FOR VALUES …in a separate file. Identical IR as Form 1. - Source Form 3 —
ALTER TABLE parent ATTACH PARTITION child FOR VALUES …combined with a standalone childCREATE TABLE(no inlinePARTITION OF). The parser merges the attach statement into the child'spartition_of, producing the same IR as Form 2. Equivalence of Form 2 and Form 3 is verified by a conformance fixture. - Sub-partitioning — a
Tablemay have bothpartition_by(it is itself a partitioned parent) andpartition_of(it is a partition of another parent). - Catalog reader — two new queries:
SELECT_PARTITIONED_TABLES(pg_class.relkind='p'+pg_get_partkeydef) reads partitioned-parent keys;SELECT_PARTITIONS(relispartition=true+pg_get_expr(relpartbound, oid)) reads partition children and re-parses the bounds text. Both filters applyNOT EXISTS (pg_depend deptype='e'). - Differ —
TableChange::AttachPartition { parent, child, bounds }andTableChange::DetachPartition { parent, child }variants. Bounds change on a stable parent → DetachPartition + AttachPartition. Parentpartition_byrekey →UnsupportedDiff(no safe in-place path in Postgres). Column and constraint diff is suppressed when either the source or target side is a partition (columns are inherited from the parent). - Planner — 2 new step kinds:
AttachPartition(non-destructive) andDetachPartition(destructive; intent required).AttachPartitionis placed in the same post-create ordering bucket asCreateIndex;DetachPartitionis placed in the same destructive bucket asDropTable. NodeIddep edge — child partition → parent table (DepSource::Structural). Ensures the parent exists (and has itspartition_byapplied) before the child is attached.
Added — lint rules (partitioning)
partition-references-unmanaged-parent(Error) — a partition child'spartition_of.parentschema is not in[managed].schemas. Prevents silent attach failures when the parent table is outside pgevolve's control.
Added — tests (partitioning)
- 14 conformance fixtures (Tier C):
objects/partitions/covering:create-range-parent-and-two-partitions— RANGE parent withFOR VALUES FROM … TO ….create-list-parent— LIST parent.create-hash-parent-and-partitions— HASH parent withFOR VALUES WITH (MODULUS m, REMAINDER r).create-default-partition—DEFAULTpartition.add-partition— attaching a new partition to an existing parent.drop-partition— detach + destructive intent path.replace-bounds— bounds change → detach + reattach.attach-existing-standalone— Form 3 attach of a pre-existing standalone table.attach-form-vs-declarative-form-equivalent— Form 2 vs Form 3 produce identical plans.detach-to-standalone— detach leaves the child as a standalone table.subpartitioned— sub-partitioned child (bothpartition_byandpartition_ofset).lint-unmanaged-parent—partition-references-unmanaged-parentfires when parent is in an unmanaged schema.- Reject path fixtures for invalid bounds expressions.
Changed — differ (partitioning)
diff_tablesnow skips column and constraint diffing when either the source or the target side of a table pair is a partition (partition_of.is_some()). Partition children inherit their column list from the parent; diffing inherited columns produces spurious changes. The partition bounds themselves are diffed viaAttachPartition/DetachPartitioninstead.
Added — IR (extensions)
Extension { name, schema: Option<Identifier>, version: Option<String>, comment: Option<String> }flat IR type inpgevolve-core::ir::extension.Catalog::extensions: Vec<Extension>flat collection.canon::sort_and_deduperejects duplicate extension names.
Added — pipeline (extensions)
- Source parser —
CREATE EXTENSION [IF NOT EXISTS] name [WITH SCHEMA s] [VERSION 'v']parses into theExtensionIR.CASCADE,FROM old_version, and unknown options rejected with structural errors.ALTER EXTENSIONin source files rejected at statement classification.COMMENT ON EXTENSIONparsing added. - Catalog reader — queries
pg_extensionjoined withpg_namespaceandpg_description. The reader for every other object kind (tables, indexes, sequences, functions, types, views, MVs) gains aNOT EXISTS (pg_depend deptype='e')filter so extension-owned objects never appear as drift. - Differ —
ExtensionChangevariants:Create,Drop,AlterUpdate,ReplaceWithCascade,CommentOn. Source-Nonefor schema, version, or comment means "any catalog value", so unpinned source declarations don't diff against any installed version. - Planner — 4 new step kinds:
CreateExtension,DropExtension(destructive),AlterExtensionUpdate,CommentOnExtension. Schema changes go throughDropExtension+CreateExtensionwith linked intent. NodeId::Extension— added to the dep graph;Extension → Schemaedges force the schema to exist before the extension is created.
Added — lint rules (extensions)
extension-version-unpinned(Warning) —CREATE EXTENSION foo;without aVERSIONclause.extension-references-unmanaged-schema(Error) —WITH SCHEMA gisbutgisisn't in the source catalog.
Added — tests (extensions)
- 11 conformance fixtures (Tier C):
objects/extensions/covering create/drop/replace/comment paths plus version-pin and version-unpinned no-op cases.scenarios/extension-owned-objects-ignoredexercises thepg_depend deptype='e'filter.scenarios/create-order-schema-firstverifies theExtension → Schemadep ordering.
Changed — conformance suite (extensions)
- Apply-layer post-check (
assertions::apply) switched from strictcanonical_eqto a differ-based convergence check. Source IR withNonefor schema/version/comment now correctly converges against any catalog reading that has concrete values for those fields.
Added — IR (user-defined types)
UserType { qname, kind: UserTypeKind, comment }flat IR type inpgevolve-core::ir::user_type.UserTypeKind::Enum { values: Vec<EnumValue> }— ordered label list withsort_order: f32mirroringpg_enum.enumsortorder.UserTypeKind::Domain { base, nullable, default, check_constraints, collation }— domain defaults and CHECK expressions useNormalizedExprfor canonical comparison.UserTypeKind::Composite { attributes: Vec<CompositeAttribute> }— each attribute carries name, type, and optional collation.Catalog::types: Vec<UserType>— flat collection, sorted byqnameaftercanonicalize().
Added — pipeline (user-defined types)
- Source parser —
CREATE TYPE … AS ENUM,CREATE DOMAIN,CREATE TYPE … AS (…)all parse into theUserTypeIR. Duplicate labels / attributes rejected at parse time. - AST resolution —
UserDefined(QualifiedName)column type references resolved againstCatalog::typesafter the source parse pass. - Catalog reader — queries
pg_type,pg_enum,pg_attribute(for composites), andpg_constraint/pg_attrdef(for domains) to reconstructUserTypefrom a live database. - Differ —
UserTypeChangevariants:Create,Drop,EnumAddValue,EnumRenameValue,DomainAddCheck,DomainDropCheck,DomainSetDefault,DomainSetNotNull,CompositeAddAttribute,CompositeDropAttribute,CompositeAlterAttributeType,CommentOn,ReplaceWithCascade. - Compatibility predicates —
enum_can_alter_in_place(preserved labels maintain relative order; renames position-paired) andcomposite_can_alter_in_place(preserved attributes maintain relative order). Both fall back toReplaceWithCascadewhen the predicate returnsfalse. - Planner — 12 new step kinds:
CreateType,DropType,AlterTypeAddValue,AlterTypeRenameValue,AlterDomainAddConstraint,AlterDomainDropConstraint,AlterDomainSetDefault,AlterDomainSetNotNull,AlterTypeAddAttribute,AlterTypeDropAttribute,AlterTypeAlterAttributeType,CommentOnType. NodeId::Type— added to the dep graph; edges from type → column (column'sColumnType::UserDefined) and type → type (domain base type) drive correct creation/drop ordering.
Added — lint rules (user-defined types)
type-shadows-table(Error) — a user-defined type shares a qualified name with a table, view, or MV.enum-value-collision(Error) — an enum type declares duplicate value labels.composite-attribute-collision(Error) — a composite type declares duplicate attribute names.domain-check-references-unmanaged-type(Warning) — a domain's CHECK expression references a schema outside[managed].schemas.
Added — tests (user-defined types)
- 20 conformance fixtures (Tier C):
objects/enums/(8),objects/domains/(6),objects/composites/(4),objects/user_type_lints/(2). - Property test
enum_add_value_preserves_existing_values(#[ignore], pure, no Docker) — for any random initial label list and a new distinct label,diff_user_typesemits exactly oneEnumAddValuechange.
Added — IR (views and materialized views)
ViewandMaterializedViewflat IR types inpgevolve-core::ir::view.ViewColumn— named column with resolved type and optional comment; used by both views and MVs.body_canonical: NormalizedBody— parsed-and-deparsed SELECT body in canonical form. Enables cosmetically-different but semantically-identical view bodies to diff equal.body_dependencies: Vec<DepEdge>— dependency edges extracted from the body AST withDepSource::AstExtractedprovenance. Powers the dependent-recreation walk and theview-body-references-unmanaged-schemalint.security_barrierandsecurity_invokerreloptions onView.
Added — pipeline (views and materialized views)
- AST canonicalization pass (
parse/ast_canon.rs) — runs after source parse; callsNormalizedBody::from_sqlon each view body, extractsDepEdges, resolves references against the provisional catalog, and fills in column types. - Catalog reader —
read_viewsandread_materialized_viewsquerypg_views/pg_matviews, callpg_get_viewdef, and feed the result throughNormalizedBody::from_sql. Source-side and catalog-side canonical texts are directly comparable. - Differ —
ViewChangeandMvChangevariants. OR-REPLACE compatibility predicate (body_is_or_replace_compatible) determines whether a body change emitsCREATE OR REPLACE VIEW(compatible) orDROP + CREATE(incompatible). - Planner — 7 new step kinds:
CreateView,DropView,CreateMaterializedView,DropMaterializedView,RefreshMaterializedView,AlterViewSetReloption,CommentOnView. - Online rewrites —
REFRESH MATERIALIZED VIEW CONCURRENTLYupgrade (when unique index present); dependent-view recreation cascade (recreate_views::extend_with_dependent_recreations).
Added — configuration
[planner.online_rewrites].refresh_mv_concurrently(defaulttrue) — upgradeREFRESHtoREFRESH CONCURRENTLYwhen the MV has a unique index.[planner.online_rewrites].view_drop_create_dependents(defaulttrue) — cascade dependent-view recreations; setfalseto error instead of auto-cascading.[[step_override]]rows inintent.toml— suppress individual plan steps by kind + target.
Added — lint rules (views and materialized views)
view-shadows-table(Error) — a view or MV shares a qualified name with a managed table.mv-no-unique-index(Warning) — an MV has no unique index;REFRESH CONCURRENTLYunavailable.view-body-references-unmanaged-schema(Warning) — a view body dependency edge points to an unmanaged schema.
Added — tests (views and materialized views)
- 15 conformance fixtures (Tier C):
objects/views/(8),objects/materialized_views/(6),intent/drop-view-requires-intent(1),scenarios/dependency-chains/(2). - Property test
view_canonicalization_closed_under_pg_rewrite(#[ignore], Docker-gated) — verifiesNormalizedBody::from_sqlclosure under the PG rewrite for a fixed set of representative view bodies. - Property test
arb_view_dependency_graph(#[ignore], Docker-free) — generates random view DAGs over a generated table corpus, mutates a leaf-table column, and asserts the resulting plan recreates exactly the transitively-dependent views in valid topological order. Closes the deferred test from sub-spec #1 §12.2. Newarbitrary_view_cataloggenerator inpgevolve-testkit.
0.1.0 — 2026-05-17
First tagged release. The v0.1 surface manages schemas, tables (with columns/constraints/comments), indexes, and sequences against Postgres 14, 15, 16, and 17.
Added — pipeline
- Parser (
pgevolve-core::parse) —*.sql→ IR viapg_query. Tracks source locations for every parsed object. Recognises-- @pgevolvedirectives (schema=…,dep:…). - AST resolution pass — runs between parse and canonicalize. Validates structural references (FKs against declared tables; default-using sequences against declared sequences). Surfaces unresolved references with source-located errors before any DB touch.
- Catalog reader (
pgevolve-core::catalog) — live PG → IR via per-PG-major SQL strings and a syncCatalogQueriertrait. Returns(Catalog, DriftReport). The drift report captures NOT VALID constraints and INVALID indexes for auto-recovery. - Differ (
pgevolve-core::diff) — pair-by-qname, structural;ChangeSetplus higher-levelChangeenum. Drift entries fold intoChange::ValidateConstraint/Change::RecreateIndex. - Planner (
pgevolve-core::plan) — order → rewrite → group → wrap. Deterministic topo sort (Kahn + min-heap tiebreak). FK cycle extraction viaDeferredFkAdd. Four online rewrites:CREATE INDEX CONCURRENTLY,FK NOT VALID + VALIDATE,CHECK NOT VALID + VALIDATE,SET NOT NULL via CHECK pattern. - Plan format — three-file directory (
plan.sql,intent.toml,manifest.toml); deterministicPlanId(BLAKE3 over bincoded canonical IRs);[[intent]]rows withapproved: bool;[[lint_waiver]]rows to acknowledgeLintAtPlanfindings;RecordedFindingrows inmanifest.tomlfor apply-time waiver recheck. - Executor (
pgevolve::executor) — bootstrap, advisory lock, per-step audit, preflight (target identity, drift recheck, intent approval, lint-waiver recheck). Per-group transactional or autocommit execution. - Linter (
pgevolve-core::lint) — universal rules + four built-in layout profiles (schema-mirror,kind-grouped,feature-grouped,free-form) plus a regex+assertion custom-profile mechanism. NewSeverity::LintAtPlantier (gates plan with exit code 2 unless waived) and a newcolumn-position-driftrule.
Added — IR
- Top-level types:
Catalog,Schema,Table,Column,Constraint,Index,Sequence, plusColumnType,DefaultExpr,NormalizedExpr,NormalizedBody(the statement-scope counterpart for v0.2 body-bearing objects). - Dep-graph types:
DepEdge { from, to, source: DepSource }withStructural(v0.1) +AstExtracted/AstDeclared(v0.2) provenance.
Added — CLI
pgevolve init— scaffold project files.pgevolve lint [--format human|json]— universal + layout-profile rules.pgevolve validate [--shadow] [--shadow-validate] [--shadow-strict]— source-tree validation.pgevolve diff --db <env> [--format human|json|sql] [--shadow-validate]— print the change set.pgevolve plan --db <env> [-o <dir>] [--shadow-validate]— write a plan directory. Refuses with exit 2 on unwaivedLintAtPlanfindings.pgevolve apply <plan-dir> --db <env>— execute a plan.pgevolve status --db <env>— recent applies and per-step state.pgevolve dump --db <env> -o <dir>— introspect a live DB and write a fully-populatedschema/tree via the new IR → SQL emitter (pgevolve-core::render).pgevolve bootstrap --db <env>— install/upgrade the metadata schema.pgevolve graph [--graph-format dot|mermaid] [-o <path>]— render the dep graph.pgevolve doctor --db <env>— project health check (drift, dangling intents, recent apply failures).pgevolve rewrite-table <qname> --db <env> --confirm-rewrite— skeleton; full implementation lands with a v0.2 sub-spec.
Added — config
pgevolve.tomlwith[project],[managed],[planner],[planner.online_rewrites],[environments.<env>], and a new[shadow]block (backend = auto | testcontainers | dsn; per-backendurl,url_env,reset,extensions,postgres_version).
Added — test infrastructure
pgevolve-testkit—EphemeralPostgres,PgCatalogQuerier,MigrationFixture, IR generator + mutator,TestPgBackendpluggable backend trait with testcontainers / compose / dsn impls (selected viaPGEVOLVE_TEST_PG_MODE).pgevolve-conformance— Tier C suite with five fixture authoring subtrees (objects/,scenarios/,intent/,failure/,regressions/) and nine assertion layers (L1 diff, L2 plan structural, L3 plan-SQL golden, L4 apply roundtrip, L5 minimality, L6 no-collateral-damage, L7 intent shape, L8 dep-graph golden, L9 topological order). Runtime budgets enforced per-fixture and suite-total.dev/docker-compose.pg.yml— PG 14/15/16/17 on stable ports for fast local test iteration in compose mode.cargo xtasksubcommands:bless --conformance,coverage --check|--gaps,fixture-cost,capture-regression,verify-regression,property-status,diagnose-pg-version.
Added — workflows
ci.yml— fmt, clippy, unit + tier-2 tests, conformance matrix across PG 14/15/16/17, property-status compliance gate.property-tests.yml— nightly property test runs with auto-capture of failures intoregressions/.soak.yml— weekly high-case property runs.
Known limitations of v0.1
pgevolve rewrite-tableis a CLI skeleton — invoking it errors with "not yet implemented." The implementation lands with a v0.2 sub-spec (partitioning / column-type-change).pgevolve dumpwrites a singleschema.sqlfile. Multi-file layout following[project].layout_profileis deferred to v0.1.x+.- Views, materialized views, functions, procedures, triggers, user-defined types, extensions, and declarative partitioning are not in v0.1; they land per v0.2 sub-spec series.
--shadow-validateis a scaffold cross-check. v0.1 has no body- bearing objects so the cross-check has nothing to do beyond a trivial structural-edge count; v0.2 sub-specs deepen it.
pgevolve roadmap
This document orders every remaining 🔮 Future / 📋 Planned object kind
in objects.md into target releases. The ordering
principle is Postgres dependency order × user impact: prerequisite
objects ship first; within a dep-respecting slot, the objects that
unblock the most real applications go earlier.
Version numbers may slip; the order does not. Each row links to a
plan stub under ../superpowers/plans/_skeleton/;
the stub is promoted to a dated plan when brainstorming begins.
See also ../v1.md — the v1.0 charter defines the
gate that triggers the 0.x → 1.0 cut, the stability commitments, and
the quality bar. The roadmap below is the slotted feature schedule;
the charter is the meaning of "done".
Shipped
| Released | Object / sub-feature | Plan |
|---|---|---|
| v0.3.4 | PUBLICATION | 2026-05-26-publications.md |
| v0.3.5 | SUBSCRIPTION | 2026-05-26-subscriptions.md |
| v0.3.6 | PG 18 catalog support | 2026-05-26-postgres-18-support.md |
| v0.3.7 | STATISTICS + VIEW ... WITH CHECK OPTION | 2026-05-27-statistics-and-check-option.md |
| v0.3.8 | CREATE COLLATION + RANGE TYPE | 2026-05-28-collation-and-range-type.md |
| v0.4.0 | EVENT TRIGGER | 2026-06-04-event-trigger.md |
| v0.4.0 | TABLESPACE (cluster object) | 2026-06-05-tablespace.md |
| v0.4.0 | TABLE ... USING <access method> | 2026-06-06-table-access-method.md |
| v0.4.1 | AGGREGATE (ordinary: SFUNC + STYPE) | 2026-06-06-aggregate.md |
| v0.4.2 | CAST | 2026-06-07-cast.md |
| v0.4.3 | Per-partition TABLESPACE | 2026-06-08-table-tablespace.md |
| v0.4.3 | TEXT SEARCH family (DICTIONARY + CONFIGURATION) | 2026-06-08-text-search.md |
| unreleased | Recursive views (WITH RECURSIVE) — already supported (verified PG 14–18); conformance coverage added, no code change needed | 2026-06-10-recursive-views-design.md |
Active matrix
The 1.0 cut happens when this matrix is empty. See
../v1.md §4 for the full v1.0 feature checklist (this
matrix is the source of truth; the charter restates it).
| Target | Object / sub-feature | Plan | Notes |
|---|---|---|---|
| blocked | PG 18 virtual generated columns | 2026-06-07-virtual-generated-columns-design.md | Design complete; blocked upstream. The pg_query crate (latest 6.1.1) wraps libpg_query 17 and rejects VIRTUAL syntax; libpg_query C has an 18.0.0 tag but no Rust release wraps it, and cargo publish forbids git deps. Unblock by bumping pg_query once a PG-18 crates.io release lands, then proceed to writing-plans. |
| v0.4.2 | PL-language wiring → non-SQL FUNCTION bodies | _skeleton/pl-language-wiring.md | Enables PL/Python, PL/Perl, etc. Depends on: CREATE EXTENSION (shipped v0.2.x) for the language extension. |
| v0.5.0 | FDW family | _skeleton/fdw-family.md | FDW, SERVER, USER MAPPING, FOREIGN TABLE, IMPORT FOREIGN SCHEMA; includes secrets handling. Internal slot order within v0.5.0: FDW → SERVER → USER MAPPING → FOREIGN TABLE → IMPORT FOREIGN SCHEMA. |
| v0.5.1 | OPERATOR / OPERATOR CLASS / OPERATOR FAMILY | _skeleton/operator-family.md | Heavy admin surface. Depends on: functions + custom types (both shipped v0.2.x). |
Future (no version commitment)
| Object / feature | Why deferred |
|---|---|
| Partition pruning at plan time | Optimization, not correctness |
SECURITY LABEL integration | Used primarily by SE-Linux; low demand |
| Security-barrier / leakproof per-function flag review | Lands alongside finer-grained policy review |
Explicitly out of scope
These remain ⛔ Not planned (rationale lives in objects.md):
RULE— superseded by triggersBASE TYPE— requires C-language functionsINHERITS— superseded by declarative partitioningDETACH PARTITION CONCURRENTLY— minimal benefit, high apply-time complexityDATABASEitself,TABLESPACEfilesystem layout, cluster-wide settings, backups, data
Ordering rationale
Two principles, applied in order:
- Postgres dependency order.
CREATE COLLATIONprecedesTEXT SEARCH. PL-language wiring precedes non-SQL/plpgsqlFUNCTIONbodies. FDWSERVER/USER MAPPINGprecedeFOREIGN TABLE. - User impact / demand. Within a dep-respecting slot, the objects
that unblock the most real applications go earlier.
STATISTICS,EVENT TRIGGER,RANGE TYPE,VIEW ... WITH CHECK OPTION, andCREATE COLLATIONrank high.OPERATOR FAMILYandCASTrank low.
How to use this document
- Adding a new object kind: insert a row in the active matrix at the
appropriate version, link to a
_skeleton/stub, and updateobjects.mdto flip the status from 🔮 to 📋. - Starting brainstorming on an object: promote the
_skeleton/<topic>.mdfile to<YYYY-MM-DD>-<topic>.mdat the top ofdocs/superpowers/plans/, flipstatus: skeleton→status: brainstorming, and update the roadmap row's plan link. - Slipping a version: edit only the
Targetcolumn; the order does not change.
Specs
Design documents — one per sub-spec / feature. Each spec captures the
brainstorming session that produced it, the decisions made, and the
final design before implementation. The matching plan in
../plans/ decomposes the design into tasks.
| Date | Spec |
|---|---|
| 2026-05-09 | pgevolve (initial design) |
| 2026-05-11 | Conformance test suite |
| 2026-05-11 | Views and materialized views |
| 2026-05-15 | Test strategy v2 |
| 2026-05-15 | v0.2 architecture review |
| 2026-05-18 | Functions / procedures |
| 2026-05-18 | User types |
| 2026-05-19 | Canon consolidation |
| 2026-05-19 | Diff-derive macro |
| 2026-05-19 | In-process apply runner |
| 2026-05-20 | Extensions |
| 2026-05-20 | Rewrite split |
| 2026-05-20 | Triggers |
| 2026-05-21 | Cluster roles |
| 2026-05-21 | Column physical attributes |
| 2026-05-21 | Partitioning |
| 2026-05-22 | Grants and ownership |
| 2026-05-22 | RLS policies |
| 2026-05-22 | Table reloptions |
| 2026-05-26 | PG 18 + object roadmap |
| 2026-05-26 | Publications |
| 2026-05-26 | Subscriptions |
| 2026-05-27 | Statistics + view check option |
| 2026-05-28 | Collation + range type |
| 2026-05-28 | v1.0 charter |
| 2026-05-28 | Roadmap revision |
| 2026-05-28 | Testing + infra maturation |
| 2026-05-28 | CI/CD maturation |
| 2026-05-28 | Docs website |
Plans
Implementation plans — one per spec. Each plan decomposes its matching spec into bite-sized tasks with verbatim code, exact commands, and TDD-shaped checkpoints. Most plans are executed via the subagent-driven-development skill: a fresh subagent per task with controller review between.
Skeleton stubs for upcoming roadmap items live under
_skeleton/; they get promoted to dated plans
when brainstorming begins.