Skip to content

feat: add --publish flag for understand-quickly registry integration#185

Open
amacsmith wants to merge 2 commits into
The-Pocket:mainfrom
amacsmith:feat/uq-publish
Open

feat: add --publish flag for understand-quickly registry integration#185
amacsmith wants to merge 2 commits into
The-Pocket:mainfrom
amacsmith:feat/uq-publish

Conversation

@amacsmith
Copy link
Copy Markdown

@amacsmith amacsmith commented May 10, 2026

Why

looptech-ai/understand-quickly is a public registry of code-knowledge graphs that ships an MCP server and a stable registry.json API. PocketFlow-Tutorial-Codebase-Knowledge already produces per-codebase knowledge artifacts (abstractions, ordered chapters, relationships) — exactly the producer shape the registry is designed for.

Wiring a --publish flag means any user who generates a tutorial can land in the registry with one flag, and AI agents (Claude, Codex, Cursor via MCP) can discover and consume their tutorial-knowledge graph immediately.

  • Discoverability. Every published tutorial appears at https://looptech-ai.github.io/understand-quickly/.
  • No infrastructure. Tutorials stay in the user's output/<project>/; the registry only stores pointers and fetches from raw.githubusercontent.com.
  • Agent-consumable. Schema validation, drift detection (when metadata.commit is set), and a turnkey MCP server.

What changes

  • A new utils/uq_publish.py module (stdlib-only — urllib, subprocess, json).
  • A --publish flag in main.py (opt-in).
  • A small block in CombineTutorial.post (in nodes.py) that, when --publish is set, builds and writes the projection.
  • README "Publish to understand-quickly" paragraph.
  • Two unit tests in tests/test_uq_publish.py (uses stdlib unittest, no new test deps).

When --publish is set, after the existing markdown tutorial is written:

  1. A generic@1 JSON projection is built — abstractions become nodes, AnalyzeRelationships.details plus the chapter ordering become edges.
  2. Embedded metadata.{tool, tool_version, generated_at, commit, project_name, repo_url, summary}.
  3. Written to <output>/<project>/tutorial.json.
  4. If $UNDERSTAND_QUICKLY_TOKEN is set, fires a repository_dispatch at looptech-ai/understand-quickly so the registry's sync workflow re-pulls the entry. If unset, no network call.

Diff stats

 README.md                |  20 +++++
 main.py                  |  12 +++
 nodes.py                 |  27 +++++++
 tests/test_uq_publish.py |  74 +++++++++++++++++
 utils/uq_publish.py      | 207 +++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 340 insertions(+)

The utils/uq_publish.py module is self-contained and only loaded when --publish is set (lazy import inside CombineTutorial.post).

No-op default

Without --publish, behavior is identical to today — no extra files, no new code paths. With --publish but no UNDERSTAND_QUICKLY_TOKEN, only the local tutorial.json is written. Failures inside the publish block are caught and logged; tutorial generation never fails because of publish.

Token setup

Fine-grained GitHub PAT, single permission:

  • Repository access: looptech-ai/understand-quickly only.
  • Permissions: Repository dispatches: write. Nothing else.

The recommended CI step is the looptech-ai/uq-publish-action@v0.1.0 Marketplace Action:

- uses: looptech-ai/uq-publish-action@v0.1.0
  with:
    graph-path: 'output/<project>/tutorial.json'
    format: 'generic@1'
    token: ${{ secrets.UNDERSTAND_QUICKLY_TOKEN }}

Schema fit

generic@1 only requires nodes and edges arrays, so the chapter/abstraction shape lands today with no schema PR. If you'd later prefer a richer tutorial@1 schema (chapter ordering, prerequisite edges, source-file citations), the registry has a format-authoring path.

Test plan

  • python3 -m unittest tests.test_uq_publish -v — both tests green.
  • python3 -m py_compile main.py nodes.py utils/uq_publish.py — all clean.
  • (Maintainer) python main.py --repo <url> --publish with UNDERSTAND_QUICKLY_TOKEN set and the repo registered fires the dispatch and the registry's sync.yml runs within ~1 minute.
  • (Maintainer) python main.py --repo <url> --publish with the token unset writes output/<project>/tutorial.json and prints an informational line; no network call.

Notes

  • This is opt-in for early adopters; nothing in PocketFlow's existing surface needs to break.
  • Once a few users land in the registry, we can add The-Pocket/PocketFlow-Tutorial-Codebase-Knowledge to the verified-publisher allowlist for auto-merge.
  • Licensing for users. Submitting via --publish is governed by the Understand-Quickly Data License 1.0. It is opt-in, gated on the user setting UNDERSTAND_QUICKLY_TOKEN.

Links


Current live state (sweep 2026-05-11)

The registry and its publishing surface are tagged and live:

Nothing in this PR depends on any pre-release surface — all referenced artifacts are pinned versions.

Adds a `--publish` flag (opt-in) that, after tutorial generation, emits
a small `generic@1` knowledge-graph projection of the tutorial
(abstractions as nodes; AnalyzeRelationships output + chapter ordering
as edges) at `<output>/<project>/tutorial.json` with metadata.{tool,
tool_version, generated_at, commit}.

When `UNDERSTAND_QUICKLY_TOKEN` is set, also fires a
`repository_dispatch` at `looptech-ai/understand-quickly` so the
registry resyncs the entry. Without the token, only the local file is
written — no network call, no failure.

New module `utils/uq_publish.py` is stdlib-only (urllib, subprocess,
json) — no new dependencies. Includes two unit tests using stdlib
`unittest`.

Spec: https://github.com/looptech-ai/understand-quickly/blob/main/docs/spec/code-graph-protocol.md
Action: https://github.com/looptech-ai/uq-publish-action
Copilot AI review requested due to automatic review settings May 10, 2026 07:49
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an opt-in --publish integration that emits a generic@1 knowledge-graph JSON alongside the generated tutorial and (optionally) triggers an understand-quickly registry resync via GitHub repository_dispatch.

Changes:

  • Introduces --publish CLI flag and plumbs it through the flow to trigger publishing after tutorial generation.
  • Adds utils/uq_publish.py to build/write a generic@1 graph and optionally dispatch a registry sync event when UNDERSTAND_QUICKLY_TOKEN is set.
  • Adds README docs + basic unit tests for graph generation and the “no token” publish path.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
utils/uq_publish.py Implements graph projection, file writing, and optional GitHub dispatch to the registry.
nodes.py Calls the publish logic in CombineTutorial.post when publish_to_uq is enabled.
main.py Adds --publish flag and stores it in shared state (publish_to_uq).
README.md Documents the opt-in publish flow and CI action usage.
tests/test_uq_publish.py Adds unit tests for graph emission and local write without dispatch when token is unset.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread utils/uq_publish.py Outdated
Comment on lines +4 to +5
(nodes = chapters, edges = chapter relationships) and, if a token is set,
fires a `repository_dispatch` at the registry.
Comment thread utils/uq_publish.py Outdated
"label": abstr.get("name", f"abstraction {i}"),
"kind": "abstraction",
"description": abstr.get("description", ""),
"files": list(abstr.get("files", [])),
Comment thread utils/uq_publish.py Outdated
Comment on lines +84 to +87
"description": abstr.get("description", ""),
"files": list(abstr.get("files", [])),
"chapter_index": chapter_order.index(i) if i in chapter_order else None,
})
Comment thread utils/uq_publish.py Outdated
Comment on lines +182 to +191
repo_slug = _detect_repo_slug(source_dir or Path.cwd(), repo_url)
if not repo_slug:
print("[uq-publish] could not detect github repo slug — skipping dispatch.",
file=log)
return {"dispatched": False, "metadata": metadata}

try:
status = dispatch(
repo_slug, token=token, schema=graph.get("schema", "generic@1"),
graph_path=str(output_path), commit=metadata.get("commit"),
Comment thread README.md Outdated

## 📡 Publish to understand-quickly (opt-in)

Add `--publish` to land the generated tutorial in [`looptech-ai/understand-quickly`](https://github.com/looptech-ai/understand-quickly), a public registry of code-knowledge graphs that ships an MCP server. The flag emits a small `generic@1` JSON projection of the tutorial (chapters as nodes, relationships as edges) at `<output>/<project>/tutorial.json` with `metadata.{tool, tool_version, generated_at, commit}`. If `UNDERSTAND_QUICKLY_TOKEN` is set, it also fires a `repository_dispatch` so the registry resyncs the entry.
Comment thread tests/test_uq_publish.py Outdated
Comment on lines +18 to +19
{"name": "Flow", "description": "Pipeline orchestrator", "files": ["flow.py"]},
{"name": "Node", "description": "Unit of work", "files": ["nodes.py"]},
- Resolve abstraction file indices to repo-relative paths via files_data
  (passed from shared['files']); fall back to 'file_indices' when absent so
  consumers don't get raw integers under a 'files' key.
- Precompute chapter_index lookup dict to avoid O(n^2) chapter_order.index().
- Dispatch payload sends repo-relative POSIX graph_path; skip dispatch
  cleanly when the graph isn't inside source_dir.
- Clarify module docstring (abstractions/chapters) and README that commit
  is only populated when a local git repo is available.
- Update test fixtures to use integer indices, matching production data.
@amacsmith
Copy link
Copy Markdown
Author

Addressed @copilot review feedback:

  • utils/uq_publish.py:
    • build_generic_graph now accepts files_data (the (path, content) tuples from shared['files']) and resolves abstraction file indices to repo-relative paths in the exported files field. Without files_data the field is renamed file_indices so consumers don't mistake integers for paths.
    • Replaced chapter_order.index(i) per node with a precomputed {abstraction_index: chapter_index} dict (O(n) build, O(1) lookups).
    • publish() dispatch payload now sends a repo-relative POSIX graph_path (e.g. output/demo/tutorial.json); if the graph isn't inside source_dir we stamp/write locally and skip dispatch with a clear message — registry can only fetch via raw.githubusercontent.com.
    • Module docstring reworded ("abstractions/chapters").
  • nodes.py: CombineTutorial.post now passes shared['files'] through as files_data.
  • README.md: clarified that metadata.commit is populated when a local git repo is available (i.e. for --dir; not always for remote --repo crawls).
  • tests/test_uq_publish.py: fixtures now use integer indices to match production data; added a test asserting that file_indices is exposed (not files) when files_data is omitted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants