-
Notifications
You must be signed in to change notification settings - Fork 0
feat: Add CoVe commands, template sync, and Python SOLID checklist #24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
3a90875
feat: Add CoVe commands, template sync, and Python SOLID checklist
stranma f06cf95
fix: Guard PR step against empty sync_branch on early exit
stranma 16d79a2
fix: Make hook and skill tests pass on Windows
stranma 5615fd1
fix: Handle non-git context in hook executable test
stranma File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| --- | ||
| allowed-tools: Read, Glob, Grep, Bash, Agent | ||
| description: Isolated Chain-of-Verification (CoVe) -- verification step runs in a separate agent to prevent confirmation bias. | ||
| --- | ||
|
|
||
| <!-- Inspired by serpro69/claude-starter-kit; based on Meta's CoVe paper (arxiv.org/abs/2309.11495) --> | ||
|
|
||
| # Isolated Chain-of-Verification (CoVe) | ||
|
|
||
| Apply the 4-step CoVe process with **isolated verification** -- Step 3 runs in a separate agent that cannot see the baseline response, preventing confirmation bias. | ||
|
|
||
| ## Step 1: Generate Baseline Response | ||
|
|
||
| Answer the user's question fully, as you normally would. Write out your complete response under a heading: | ||
|
|
||
| ```markdown | ||
| ## Baseline Response | ||
| [your full answer here] | ||
| ``` | ||
|
|
||
| ## Step 2: Plan Verification Questions | ||
|
|
||
| Review your baseline response and generate a numbered list of fact-check questions. Focus on claims that could be wrong -- file paths, function signatures, API behavior, version numbers, configuration syntax, behavioral assertions. | ||
|
|
||
| ```markdown | ||
| ## Verification Questions | ||
| 1. [Is the file path X correct?] | ||
| 2. [Does function Y actually accept parameter Z?] | ||
| 3. [Is it true that library A supports feature B?] | ||
| ... | ||
| ``` | ||
|
|
||
| Generate 3-8 questions depending on response complexity. | ||
|
|
||
| ## Step 3: Isolated Verification (Agent) | ||
|
|
||
| Launch a general-purpose Agent to answer the verification questions **independently**. The agent must NOT see your baseline response -- only the verification questions. This prevents confirmation bias. | ||
|
|
||
| Provide the agent with: | ||
| - The numbered list of verification questions from Step 2 | ||
| - Instructions to use Read, Grep, Glob, and Bash to find evidence | ||
| - Instructions to answer each question with CONFIRMED or INCORRECT plus evidence | ||
|
|
||
| Example agent prompt: | ||
| ``` | ||
| Answer each of these fact-check questions by investigating the codebase. For each question, respond with CONFIRMED or INCORRECT and cite your evidence. | ||
|
|
||
| Questions: | ||
| 1. [question 1] | ||
| 2. [question 2] | ||
| ... | ||
| ``` | ||
|
|
||
| ## Step 4: Generate Final Verified Response | ||
|
|
||
| Review the agent's verification results and revise your baseline response, incorporating all corrections. If no errors were found, state that the baseline was verified and present it as final. | ||
|
|
||
| ```markdown | ||
| ## Verified Response | ||
| [corrected answer, incorporating all verification results] | ||
| ``` | ||
|
|
||
| If any corrections were made, add a brief summary: | ||
|
|
||
| ```markdown | ||
| ## Corrections Made | ||
| - [what changed and why] | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,60 @@ | ||
| --- | ||
| allowed-tools: Read, Glob, Grep, Bash | ||
| description: Chain-of-Verification (CoVe) prompting for high-stakes accuracy. Generates a response, self-verifies with fact-check questions, then revises. | ||
| --- | ||
|
|
||
| <!-- Inspired by serpro69/claude-starter-kit; based on Meta's CoVe paper (arxiv.org/abs/2309.11495) --> | ||
|
|
||
| # Chain-of-Verification (CoVe) | ||
|
|
||
| Apply the 4-step CoVe process to reduce hallucinations and factual errors in your response to the user's question. | ||
|
|
||
| ## Step 1: Generate Baseline Response | ||
|
|
||
| Answer the user's question fully, as you normally would. Write out your complete response under a heading: | ||
|
|
||
| ```markdown | ||
| ## Baseline Response | ||
| [your full answer here] | ||
| ``` | ||
|
|
||
| ## Step 2: Plan Verification Questions | ||
|
|
||
| Review your baseline response and generate a numbered list of fact-check questions. Focus on claims that could be wrong -- file paths, function signatures, API behavior, version numbers, configuration syntax, behavioral assertions. | ||
|
|
||
| ```markdown | ||
| ## Verification Questions | ||
| 1. [Is the file path X correct?] | ||
| 2. [Does function Y actually accept parameter Z?] | ||
| 3. [Is it true that library A supports feature B?] | ||
| ... | ||
| ``` | ||
|
|
||
| Generate 3-8 questions depending on response complexity. | ||
|
|
||
| ## Step 3: Answer Verifications | ||
|
|
||
| Answer each verification question independently. Use tools (Read, Grep, Glob, Bash) to check facts against the actual codebase, documentation, or runtime behavior. Do not rely on your baseline response -- verify from source. | ||
|
|
||
| ```markdown | ||
| ## Verification Results | ||
| 1. [CONFIRMED / INCORRECT] -- [evidence] | ||
| 2. [CONFIRMED / INCORRECT] -- [evidence] | ||
| ... | ||
| ``` | ||
|
|
||
| ## Step 4: Generate Final Verified Response | ||
|
|
||
| Revise your baseline response, incorporating all corrections from Step 3. If no errors were found, state that the baseline was verified and present it as final. | ||
|
|
||
| ```markdown | ||
| ## Verified Response | ||
| [corrected answer, incorporating all verification results] | ||
| ``` | ||
|
|
||
| If any corrections were made, add a brief summary: | ||
|
|
||
| ```markdown | ||
| ## Corrections Made | ||
| - [what changed and why] | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,169 @@ | ||
| # Inspired by serpro69/claude-starter-kit template-sync approach | ||
| name: Template Sync | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| dry_run: | ||
| description: "Show changes without creating a PR" | ||
| type: boolean | ||
| default: false | ||
| template_repo: | ||
| description: "Upstream template repository (owner/repo)" | ||
| type: string | ||
| default: "stranma/claude-code-python-template" | ||
| template_branch: | ||
| description: "Upstream template branch" | ||
| type: string | ||
| default: "master" | ||
| schedule: | ||
| - cron: "0 9 * * 1" # Weekly on Monday at 09:00 UTC | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| sync: | ||
| name: Sync from upstream template | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Configure git | ||
| run: | | ||
| git config user.name "github-actions[bot]" | ||
| git config user.email "github-actions[bot]@users.noreply.github.com" | ||
|
|
||
| - name: Determine template repo | ||
| id: config | ||
| run: | | ||
| REPO="${{ inputs.template_repo || 'stranma/claude-code-python-template' }}" | ||
| BRANCH="${{ inputs.template_branch || 'master' }}" | ||
| echo "repo=${REPO}" >> "$GITHUB_OUTPUT" | ||
| echo "branch=${BRANCH}" >> "$GITHUB_OUTPUT" | ||
| echo "Syncing from ${REPO}@${BRANCH}" | ||
|
|
||
| - name: Add upstream remote and fetch | ||
| run: | | ||
| git remote add upstream "https://github.com/${{ steps.config.outputs.repo }}.git" || true | ||
| git fetch upstream "${{ steps.config.outputs.branch }}" | ||
|
|
||
| - name: Compute template diff | ||
| id: diff | ||
| env: | ||
| UPSTREAM_BRANCH: ${{ steps.config.outputs.branch }} | ||
| run: | | ||
| # Paths managed by the template (synced from upstream) | ||
| # Defined once here; reused in the apply step via GITHUB_OUTPUT | ||
| TEMPLATE_PATHS=".claude/agents/ .claude/commands/ .claude/hooks/ .claude/rules/ .claude/skills/ .devcontainer/ .github/workflows/ docs/DEVELOPMENT_PROCESS.md" | ||
| echo "template_paths=${TEMPLATE_PATHS}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| # Get changed files between local and upstream | ||
| CHANGED=$(git diff --name-only HEAD "upstream/${UPSTREAM_BRANCH}" -- ${TEMPLATE_PATHS} 2>/dev/null || true) | ||
|
|
||
| if [ -z "$CHANGED" ]; then | ||
| echo "No template changes found" | ||
| echo "has_changes=false" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "Template changes detected:" | ||
| echo "$CHANGED" | ||
| echo "has_changes=true" >> "$GITHUB_OUTPUT" | ||
| # Store diff summary for PR body | ||
| DIFF_STAT=$(git diff --stat HEAD "upstream/${UPSTREAM_BRANCH}" -- ${TEMPLATE_PATHS} 2>/dev/null || true) | ||
| { | ||
| echo "diff_stat<<EOF" | ||
| echo "$DIFF_STAT" | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Show diff (dry run) | ||
| if: steps.diff.outputs.has_changes == 'true' && (inputs.dry_run == true || inputs.dry_run == 'true') | ||
| run: | | ||
| echo "=== DRY RUN: Changes that would be synced ===" | ||
| echo "${{ steps.diff.outputs.diff_stat }}" | ||
|
|
||
| - name: Apply template changes | ||
| id: apply | ||
| if: steps.diff.outputs.has_changes == 'true' && inputs.dry_run != true && inputs.dry_run != 'true' | ||
| env: | ||
| UPSTREAM_BRANCH: ${{ steps.config.outputs.branch }} | ||
| TEMPLATE_PATHS: ${{ steps.diff.outputs.template_paths }} | ||
| run: | | ||
| SYNC_BRANCH="template-sync/$(date +%Y%m%d)" | ||
|
|
||
| # Check if branch already exists | ||
| if git rev-parse --verify "refs/heads/${SYNC_BRANCH}" > /dev/null 2>&1; then | ||
| echo "Sync branch ${SYNC_BRANCH} already exists, updating" | ||
| git checkout "${SYNC_BRANCH}" | ||
| else | ||
| git checkout -b "${SYNC_BRANCH}" | ||
| fi | ||
|
|
||
| # Checkout template-managed files from upstream | ||
| for path in ${TEMPLATE_PATHS}; do | ||
| git checkout "upstream/${UPSTREAM_BRANCH}" -- "${path}" 2>/dev/null || true | ||
| done | ||
|
|
||
| # Stage and commit | ||
| git add -A | ||
| if git diff --cached --quiet; then | ||
| echo "No changes to commit after checkout" | ||
| echo "changes_applied=false" >> "$GITHUB_OUTPUT" | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "changes_applied=true" >> "$GITHUB_OUTPUT" | ||
|
|
||
| git commit -m "chore: sync template from upstream | ||
|
|
||
| Source: ${{ steps.config.outputs.repo }}@${{ steps.config.outputs.branch }}" | ||
|
|
||
| git push -u origin "${SYNC_BRANCH}" | ||
|
|
||
| echo "sync_branch=${SYNC_BRANCH}" >> "$GITHUB_ENV" | ||
|
|
||
| - name: Create pull request | ||
| if: steps.apply.outputs.changes_applied == 'true' && inputs.dry_run != true && inputs.dry_run != 'true' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| run: | | ||
| # Check for existing PR from this branch | ||
| EXISTING_PR=$(gh pr list --head "${{ env.sync_branch }}" --json number --jq '.[0].number' 2>/dev/null || true) | ||
|
|
||
| if [ -n "$EXISTING_PR" ]; then | ||
| echo "PR #${EXISTING_PR} already exists for this sync branch" | ||
| exit 0 | ||
| fi | ||
|
|
||
| gh pr create \ | ||
| --title "chore: sync upstream template changes" \ | ||
| --body "$(cat <<'EOF' | ||
| ## Template Sync | ||
|
|
||
| Automated sync of template-managed files from upstream. | ||
|
|
||
| **Source:** ${{ steps.config.outputs.repo }}@${{ steps.config.outputs.branch }} | ||
|
|
||
| ### Changed files | ||
| ``` | ||
| ${{ steps.diff.outputs.diff_stat }} | ||
| ``` | ||
|
|
||
| ### What to review | ||
| - Check if any synced files conflict with project-specific customizations | ||
| - Template-managed paths: `.claude/`, `.devcontainer/`, `.github/workflows/`, `docs/DEVELOPMENT_PROCESS.md` | ||
| - Project-specific files (`apps/`, `libs/`, `tests/`, `pyproject.toml`, `README.md`) are NOT touched | ||
|
|
||
| ### How to resolve conflicts | ||
| If a synced file conflicts with local changes, edit the file on this branch to keep your customizations, then merge. | ||
| EOF | ||
| )" | ||
|
|
||
| - name: Summary | ||
| if: steps.diff.outputs.has_changes == 'false' | ||
| run: echo "Already up to date with upstream template. No sync needed." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.