Make topological_sort's level-0 seed insert idempotent#18
Merged
Conversation
The full pipeline finally ran end-to-end through analysis, module, symbol, and declaration loads (~50 min), reaching topological_sort -- which crashed: psycopg.errors.UniqueViolation: duplicate key value violates unique constraint "level_pkey" Every other INSERT in this file (load_module, load_symbol, both dependency inserts, load_declaration) uses ON CONFLICT DO NOTHING so re-runs against a DB with prior data are safe. The iterative level insert already guards against this via 'WHERE NOT EXISTS (SELECT 1 FROM level l ...)', but the level-0 seed insert had no such guard, so it collided on symbols that already had a level from an earlier successful run. Add ON CONFLICT DO NOTHING to match.
There was a problem hiding this comment.
Pull request overview
This PR makes database/jixia_db.py’s topological_sort() resilient to incremental re-runs by making the level seeding/incremental inserts idempotent (avoiding UniqueViolation on already-seen symbols).
Changes:
- Add
ON CONFLICT DO NOTHINGto the level-0 seed insert intolevel. - Add
ON CONFLICT DO NOTHINGto the iterative level insert intolevel.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
182
to
186
| WHERE NOT EXISTS (SELECT 1 FROM dependency e WHERE e.source = v.name) | ||
| ON CONFLICT DO NOTHING | ||
| """) | ||
| while cursor.rowcount: | ||
| logger.info("topological sort: %d rows affected", cursor.rowcount) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
With #17 (defer DB connection), the pipeline finally ran end-to-end through the analysis, module, symbol, and declaration loads (~50 min of DB writes, all succeeded) and reached the final step, `topological_sort()`, which crashed:
```
psycopg.errors.UniqueViolation: duplicate key value violates unique constraint "level_pkey"
DETAIL: Key (symbol_name)=(["ContinuousLinearMap", "adjoint", "isBoundedBilinearMap_real"]) already exists.
```
Root cause
Every other INSERT in `jixia_db.py` (`load_module`, `load_symbol`, both dependency inserts, `load_declaration`) uses `ON CONFLICT DO NOTHING`, since `module`/`symbol`/`dependency`/`declaration` accumulate across incremental runs. The iterative level insert in `topological_sort` already guards against re-processing via `WHERE NOT EXISTS (SELECT 1 FROM level l WHERE l.symbol_name = e.source)` — but the level-0 seed insert has no equivalent guard, so it collided on a symbol that already had a `level` row from an earlier successful indexing pass.
What
Add `ON CONFLICT DO NOTHING` to both level inserts, matching the established pattern in the rest of the file.