Add traceability audit template for cross-document specification drift#35
Merged
Alan-Jowett merged 8 commits intomicrosoft:mainfrom Mar 20, 2026
Merged
Conversation
Add a new template that audits requirements, design, and validation documents for specification drift — gaps, contradictions, and divergence that accumulate as documents evolve independently. New components: - Persona: specification-analyst — adversarial toward completeness claims - Taxonomy: specification-drift (D1-D7) — classifies drift types with D8+ reserved for future code/test compliance audits - Protocol: traceability-audit — 6-phase cross-document comparison - Template: audit-traceability — single-shot, design doc optional Also extends the document-lifecycle pipeline with stage 4 (audit) and adds a use case section to README.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new PromptKit “traceability audit” capability to help users detect and classify cross-document specification drift across requirements, design, and validation artifacts, extending the document-lifecycle pipeline with an audit stage.
Changes:
- Introduces new components:
specification-analystpersona,traceability-auditreasoning protocol,specification-drifttaxonomy (D1–D7), andaudit-traceabilitytemplate. - Extends
manifest.yamlto register the new components and adds a 4th pipeline stage todocument-lifecycle. - Updates
README.mdto document the new components and add a usage example.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| templates/audit-traceability.md | New template orchestrating the traceability audit workflow and output mapping. |
| protocols/reasoning/traceability-audit.md | New 6-phase methodology for bidirectional traceability + consistency checks. |
| taxonomies/specification-drift.md | New D1–D7 classification scheme for drift findings (plus reserved ranges). |
| personas/specification-analyst.md | New persona defining an evidence-driven, completeness-adversarial auditor stance. |
| manifest.yaml | Registers the new persona/protocol/taxonomy/template and adds pipeline stage 4. |
| README.md | Documents the new components and adds a “Specification Traceability Audit” use case. |
Document the phased evolution from cross-document spec audits (shipped) through bidirectional code/spec audits, invariant extraction, and standards/protocol analysis. Includes vision for continuous semantic integration as a CI consumer of PromptKit prompts. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This reverts commit 9afefe7.
Walks through an auth service example: three documents authored weeks apart, with accumulated drift (missing test coverage, scope creep, constraint violation, stale REQ-ID references). Shows before/after comparison and explains why each component contributes to the result. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Curated gallery of real-world scenarios showing when and why to use each template. Each scenario is a short paragraph: the problem, which components are assembled, and what you get. Covers 9 existing templates and 4 roadmap items. Bridges the gap between one-line descriptions in the README and full case study walkthroughs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix ordering conflict: use severity-first ordering with taxonomy ranking as tiebreaker within severity tiers (protocol, template, taxonomy) - Fix pipeline stage 4 consumes: remove optional design-document - Fix input_contract type: use validation-plan (matches pipeline stage) - Fix hyphen: cross document -> cross-document in persona Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The CLI assembler now includes taxonomies in assembled prompts, matching the bootstrap.md composition order (Identity -> Protocols -> Taxonomy -> Format -> Task). Templates declare taxonomies via a 'taxonomies' field in YAML frontmatter; the assembler resolves them from the manifest. This fixes the gap where taxonomy definitions (e.g., specification-drift D1-D7 labels) were not included when using 'promptkit assemble'. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…accidental files - Fix README pipeline diagram: design is optional, not 'all three' - Fix case study CLI command: use @alan-jowett/promptkit package name - Fix scenarios.md: correct the 'design drift' scenario to use all three docs (design is optional, not validation) - Remove accidentally committed create_pptx*.py files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Mar 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new template that audits requirements, design, and validation documents for specification drift — gaps, contradictions, and divergence that accumulate as documents evolve independently.
New Components
Drift Taxonomy (D1–D7)
D8–D13 are reserved for future \�udit-code-compliance\ and \�udit-test-compliance\ templates.
Pipeline Extension
Extends \document-lifecycle\ with stage 4:
\
requirements → design → validation → audit (NEW)
\\
Other Changes
Validation