Pipeline

The path from source SQL to applied DDL, phase by phase. Each phase has its own implementation crate / module, status, and the design doc / plan that drove it.

See ../README.md for the status legend.

Phase summary

flowchart TD
    SQL["schema/*.sql"] -- parse --> SourceIR["Source IR"]
    DB[("live Postgres")] -- introspect --> CatalogIR["Catalog IR"]
    SourceIR -- canonicalize --> Source["Catalog (source)"]
    CatalogIR --> Target["Catalog (target)"]
    Source --> Diff{{diff}}
    Target --> Diff
    Diff --> CS["ChangeSet"]
    CS -- order --> OCS["OrderedChangeSet"]
    OCS -- rewrite --> Steps["Vec<RawStep>"]
    Steps -- group_steps --> Groups["Vec<TransactionGroup>"]
    Groups -- "Plan::from_grouped" --> Plan["Plan"]
    Plan --> PlanSql["plan.sql"]
    Plan --> Intent["intent.toml"]
    Plan --> Manifest["manifest.toml"]
    Plan -- "apply()" --> DB

Parsing (pgevolve_core::parse)

Tests (whole section): tier-1: crates/pgevolve-core/src/parse/mod.rs::tests, parse/statement.rs::tests, parse/directives.rs::tests; tier-2: crates/pgevolve-core/tests/parser_corpus.rs, parse_directory.rs.

AST canonicalization pass (pgevolve_core::parse::ast_canon)

Runs immediately after source parse, before the AST resolution pass. For each view and materialized view in the provisional catalog, the pass:

1. Calls NormalizedBody::from_sql (see parse/normalize_body.rs) on raw_body to fill body_canonical.
2. Walks the body AST to extract DepEdge records with DepSource::AstExtracted provenance, filling body_dependencies.
3. Resolves each referenced relation against the provisional catalog. Unresolved references surface as AstCanonError::UnresolvedReference.
4. Fills columns from the SELECT target list when no explicit alias list was provided (using Postgres's column-naming algorithm: explicit alias → rightmost ColumnRef name → "?column?" fallback).

The same NormalizedBody::from_sql call is used on the catalog side (T5 reader queries pg_get_viewdef), so source-side and catalog-side canonical texts are directly comparable by the differ. The v0.2 property test (view_canonicalization_closed_under_pg_rewrite) verifies this closure invariant.

AST resolution pass (pgevolve_core::parse::resolve)

Runs after the AST canonicalization pass. Validates structural references before diff.

Catalog reader (pgevolve_core::catalog)

Tests (whole section): tier-1: crates/pgevolve-core/src/catalog/mod.rs::tests, catalog/filter.rs::tests, catalog/version.rs::tests, catalog/rows.rs::tests; tier-3: crates/pgevolve-core/tests/catalog_round_trip.rs, functions_round_trip.rs, types_round_trip.rs, dump_round_trip.rs.

Catalog drift detection

The catalog reader returns a DriftReport alongside the Catalog. The differ and planner consume it to automatically recover from partial-apply states.

Both drift kinds are auto-recovery paths — the user doesn't author NOT VALID or INVALID states; pgevolve detects and resolves them. This covers recovery from crashed FK NOT VALID + VALIDATE rewrites and failed CREATE INDEX CONCURRENTLY.

Differ (pgevolve_core::diff)

Tests (whole section): tier-1: every crates/pgevolve-core/src/diff/*.rs::tests module (tables, columns, constraints, indexes, sequences, views, triggers, types, routines, policies, grants, reloptions, owner_op, schemas, extensions, cluster, default_privileges); tier-1: crates/pgevolve-core/src/diff/changeset.rs::tests, change.rs::tests, destructiveness.rs::tests.

Planner (pgevolve_core::plan)

Ordering

Tests (whole subsection): tier-1: crates/pgevolve-core/src/plan/ordering.rs::tests, plan/ordered.rs::tests, plan/graph.rs::tests, plan/edges.rs::tests; tier-2: crates/pgevolve-core/tests/dep_edges.rs, determinism.rs; tier-5: property plan_id_is_deterministic, create_graph_topo_sorts_or_only_fk_cycles (crates/pgevolve-core/tests/property_tests.rs).

Rewrites

Tests (whole subsection): tier-1: crates/pgevolve-core/src/plan/rewrite/tests.rs (covers concurrent index, FK NOT VALID, CHECK NOT VALID, set-not-null pattern, atomic-vs-online gating).

Plan-time lint gate

Step grouping

Tests (whole subsection): tier-1: crates/pgevolve-core/src/plan/grouping.rs::tests.

Plan format (pgevolve_core::plan::{serialize, deserialize})

Tests (whole section): tier-1: crates/pgevolve-core/src/plan/serialize.rs::tests, plan/deserialize.rs::tests, plan/plan.rs::tests.

Executor (pgevolve::executor)

Tests (whole section): tier-4: crates/pgevolve/tests/executor_smoke.rs, chaos_apply.rs; tier-5: drift_recovery_property in crates/pgevolve/tests/pg_property_tests.rs.

Shadow validation (pgevolve validate --shadow)

Tests (whole section): tier-4: crates/pgevolve/tests/shadow_validate.rs, shadow_validate_flag.rs, shadow_validate_views.rs, shadow_backend.rs.