Skip to content

Fix upgrade test#2403

Open
jrgemignani wants to merge 1 commit intoapache:masterfrom
jrgemignani:fix_upgrade_template_regression_test
Open

Fix upgrade test#2403
jrgemignani wants to merge 1 commit intoapache:masterfrom
jrgemignani:fix_upgrade_template_regression_test

Conversation

@jrgemignani
Copy link
Copy Markdown
Contributor

@jrgemignani jrgemignani commented Apr 21, 2026

The age_upgrade regression test (added in #2364, improved in #2377, #2397)
was designed to validate the upgrade template (age----y.y.y.sql) by
creating graph data before the upgrade and verifying it survived afterward.
This approach had two fundamental problems:

  1. It did not detect incomplete upgrade templates. The test verified that
    graph data (vertices, edges, checksums, GIN indexes) survived ALTER
    EXTENSION UPDATE, but never checked whether new SQL objects (functions,
    views, relations, indexes, types, operators, casts, constraints) were
    actually created by the template. A developer could add a new function
    to sql/ and sql_files, forget to add it to the upgrade template, and
    all tests would pass — the function existed via the fresh CREATE
    EXTENSION install that ran before the upgrade test, but would be
    missing for users who upgraded via ALTER EXTENSION UPDATE.

  2. The data-integrity checks relied on cypher queries (MATCH/RETURN) within
    the same backend session after DROP EXTENSION + CREATE EXTENSION. This
    caused intermittent failures on some PostgreSQL versions where AGE's
    internal type cache (agtype OID) was not properly refreshed after the
    extension was dropped and recreated, resulting in 'type with OID 0
    does not exist' errors. The data-integrity aspect was also redundant —
    ALTER EXTENSION UPDATE runs DDL statements and does not touch heap data,
    so data survival is guaranteed by PostgreSQL and not a meaningful test.

The fix replaces the entire test with a comprehensive catalog comparison:

  1. Snapshot the ag_catalog schema from the fresh install across seven
    PostgreSQL system catalogs:
    • pg_proc: functions, aggregates, procedures (name, args, and
      properties: volatility, strictness, kind, return type, setof)
    • pg_class: tables, views, sequences, indexes (name, kind)
    • pg_type: types (name, type category)
    • pg_operator: operators (name, left/right operand types)
    • pg_cast: casts involving AGE types (source, target, context)
    • pg_opclass: operator classes (name, access method)
    • pg_constraint: constraints (name, type, table, referenced table)
  2. DROP EXTENSION, CREATE EXTENSION at the synthetic initial version,
    then ALTER EXTENSION UPDATE to the current version via the stamped
    upgrade template.
  3. Snapshot the catalog again after upgrade.
  4. Compare: any object present in the fresh snapshot but missing after
    upgrade means the template is incomplete. Any object present after
    upgrade but not in the fresh snapshot means the template creates
    something unexpected. Function properties (volatility, strictness,
    prokind, return type) are also compared for functions that exist in
    both — catching cases where a CREATE OR REPLACE in the template
    changes a function's signature or behavior.

Additional improvements from code review feedback:

  • Graph cleanup in Step 1 uses a DO block with PERFORM and suppressed
    NOTICEs to produce deterministic output regardless of prior test state.
  • The pg_class snapshot includes indexes (relkind 'i') in addition to
    tables, views, and sequences.
  • Diagnostic output includes relkind/typtype suffixes for actionable diffs.
  • Summary uses boolean equality checks (funcs_match, rels_match, etc.)
    instead of absolute counts, so the expected output does not need
    updating when new objects are added to AGE. Developers who correctly
    add objects to both sql/ and the template will never need to modify
    this test or its expected output.

This approach:

  • Catches the actual failure mode: incomplete upgrade templates.
  • Covers all SQL object categories: functions (including aggregates),
    relations, types, operators, casts, operator classes, and constraints.
  • Detects property changes on existing functions (volatility, strictness,
    kind, return type changes).
  • Uses only plain SQL catalog queries — no cypher, no .so cache issues.
  • Works reliably across all PostgreSQL versions.
  • Reports the exact missing/extra/changed object in the diff output.
  • Is maintenance-free: no expected output changes needed when AGE grows.

Makefile: updated step 5 comment to reflect catalog comparison approach.

All 33 regression tests pass.

Co-authored-by: Claude noreply@anthropic.com

modified: Makefile
modified: regress/expected/age_upgrade.out
modified: regress/sql/age_upgrade.sql

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the age_upgrade regression test to validate upgrade templates by comparing the installed catalog objects from a fresh install vs. an upgrade-from-initial install, avoiding cypher-based integrity checks and related type-cache flakiness.

Changes:

  • Replace graph/data creation + cypher-based data integrity assertions with catalog snapshots and diffs for ag_catalog.
  • Update expected regression output to reflect the new catalog-comparison workflow.
  • Adjust Makefile commentary to describe the new test approach.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
regress/sql/age_upgrade.sql Reworks the upgrade regression test to snapshot and compare pg_proc / pg_class entries for ag_catalog before vs. after upgrade.
regress/expected/age_upgrade.out Updates expected output to match the new SQL and result sets (0-row diffs + summary counts).
Makefile Updates documentation comment for the upgrade test to reflect catalog comparison behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread regress/sql/age_upgrade.sql Outdated
Comment thread regress/sql/age_upgrade.sql Outdated
Comment thread regress/sql/age_upgrade.sql Outdated
@jrgemignani jrgemignani force-pushed the fix_upgrade_template_regression_test branch from ab23426 to 84e2954 Compare April 22, 2026 17:13
The age_upgrade regression test (added in apache#2364, improved in apache#2377, apache#2397)
was designed to validate the upgrade template (age--<VER>--y.y.y.sql) by
creating graph data before the upgrade and verifying it survived afterward.
This approach had two fundamental problems:

1. It did not detect incomplete upgrade templates. The test verified that
   graph data (vertices, edges, checksums, GIN indexes) survived ALTER
   EXTENSION UPDATE, but never checked whether new SQL objects (functions,
   views, relations, indexes, types, operators, casts, constraints) were
   actually created by the template. A developer could add a new function
   to sql/ and sql_files, forget to add it to the upgrade template, and
   all tests would pass — the function existed via the fresh CREATE
   EXTENSION install that ran before the upgrade test, but would be
   missing for users who upgraded via ALTER EXTENSION UPDATE.

2. The data-integrity checks relied on cypher queries (MATCH/RETURN) within
   the same backend session after DROP EXTENSION + CREATE EXTENSION. This
   caused intermittent failures on some PostgreSQL versions where AGE's
   internal type cache (agtype OID) was not properly refreshed after the
   extension was dropped and recreated, resulting in 'type with OID 0
   does not exist' errors. The data-integrity aspect was also redundant —
   ALTER EXTENSION UPDATE runs DDL statements and does not touch heap data,
   so data survival is guaranteed by PostgreSQL and not a meaningful test.

The fix replaces the entire test with a comprehensive catalog comparison:

  1. Snapshot the ag_catalog schema from the fresh install across seven
     PostgreSQL system catalogs:
       - pg_proc: functions, aggregates, procedures (name, args, and
         properties: volatility, strictness, kind, return type, setof)
       - pg_class: tables, views, sequences, indexes (name, kind)
       - pg_type: types (name, type category)
       - pg_operator: operators (name, left/right operand types)
       - pg_cast: casts involving AGE types (source, target, context)
       - pg_opclass: operator classes (name, access method)
       - pg_constraint: constraints (name, type, table, referenced table)
  2. DROP EXTENSION, CREATE EXTENSION at the synthetic initial version,
     then ALTER EXTENSION UPDATE to the current version via the stamped
     upgrade template.
  3. Snapshot the catalog again after upgrade.
  4. Compare: any object present in the fresh snapshot but missing after
     upgrade means the template is incomplete. Any object present after
     upgrade but not in the fresh snapshot means the template creates
     something unexpected. Function properties (volatility, strictness,
     prokind, return type) are also compared for functions that exist in
     both — catching cases where a CREATE OR REPLACE in the template
     changes a function's signature or behavior.

Additional improvements from code review feedback:

  - Graph cleanup in Step 1 uses a DO block with PERFORM and suppressed
    NOTICEs to produce deterministic output regardless of prior test state.
  - The pg_class snapshot includes indexes (relkind 'i') in addition to
    tables, views, and sequences.
  - Diagnostic output includes relkind/typtype suffixes for actionable diffs.
  - Summary uses boolean equality checks (funcs_match, rels_match, etc.)
    instead of absolute counts, so the expected output does not need
    updating when new objects are added to AGE. Developers who correctly
    add objects to both sql/ and the template will never need to modify
    this test or its expected output.

This approach:
  - Catches the actual failure mode: incomplete upgrade templates.
  - Covers all SQL object categories: functions (including aggregates),
    relations, types, operators, casts, operator classes, and constraints.
  - Detects property changes on existing functions (volatility, strictness,
    kind, return type changes).
  - Uses only plain SQL catalog queries — no cypher, no .so cache issues.
  - Works reliably across all PostgreSQL versions.
  - Reports the exact missing/extra/changed object in the diff output.
  - Is maintenance-free: no expected output changes needed when AGE grows.

Makefile: updated step 5 comment to reflect catalog comparison approach.

All 33 regression tests pass.

Co-authored-by: Claude <noreply@anthropic.com>

modified:   Makefile
modified:   regress/expected/age_upgrade.out
modified:   regress/sql/age_upgrade.sql
@jrgemignani jrgemignani reopened this Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants