feat(extensions): support multiple active catalogs simultaneously

[Copilot]

• Analyze existing codebase and understand current ExtensionCatalog implementation
• Add CatalogEntry dataclass and catalog stack logic to extensions.py
• Add CLI commands to __init__.py (catalogs, catalog add/remove)
• Update extension_add / extension_search / extension_info
• Add 13 new tests
• Update docs (RFC, user guide, API reference)
• Fix ruff F541: remove extraneous f prefix from strings with no placeholders

Original prompt

---

This section details on the original issue you should resolve

<issue_title>feat(extensions): support multiple active catalogs simultaneously</issue_title>
<issue_description>## Summary

The current extension catalog system supports only a single active catalog at a time — selectable via SPECKIT_CATALOG_URL or defaulting to the built-in catalog.json. This creates a fundamental conflict: catalog.json is intentionally empty (organizations curate their own approved extensions), while catalog.community.json exists as a shared discovery resource, and organizations also need to point at their own private/internal catalogs. Users currently have no way to benefit from all three simultaneously.

Problem Statement

The RFC describes a "Dual Catalog System" but the two catalogs serve different purposes and neither is composable with the other today:

Catalog	Purpose	Current Behavior
catalog.json	Org-curated approved extensions	Empty by design; sole default search target
catalog.community.json	Community discovery resource	Never searched by CLI commands
Custom URL	Private/internal org catalog	Replaces default entirely via env var

Real-world usage requires all three active at once: search the community catalog for discoverability, restrict installs to org-approved entries, and also pull from an internal catalog — with clear precedence rules between them.

Proposed Solution

Introduce a catalog stack — an ordered list of catalogs the CLI merges and searches across. Each catalog entry has a url, an optional name, a priority, and an install_allowed flag.

Default built-in stack (no config required)

When no .specify/extension-catalogs.yml exists, the CLI uses a built-in default stack:

1. catalog.json (org-curated, install_allowed: true, priority 1)
2. catalog.community.json (community discovery, install_allowed: false, priority 2)

This means specify extension search works out of the box and surfaces community extensions, while specify extension add is still restricted to whatever is in catalog.json — preserving the existing curation/trust model.

New config file: .specify/extension-catalogs.yml (project-scoped)

catalogs:
  - name: "org-approved"
    url: "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.json"
    priority: 1          # Highest — only approved entries can be installed
    install_allowed: true

  - name: "internal"
    url: "https://internal.company.com/spec-kit/catalog.json"
    priority: 2
    install_allowed: true

  - name: "community"
    url: "https://raw.githubusercontent.com/github/spec-kit/main/extensions/catalog.community.json"
    priority: 3          # Lowest — discovery only, not installable
    install_allowed: false

An equivalent user-level config lives at ~/.specify/extension-catalogs.yml for user-wide defaults. When a project-level config is present, it takes full control and the built-in defaults are not applied.

Resolution order

When a user runs specify extension search or specify extension add <name>, the CLI:

1. Loads all configured catalogs in priority order
2. Merges results, with higher-priority catalogs winning on conflicts (same extension id)
3. Respects install_allowed: false — extensions from discovery-only catalogs are shown in search results but cannot be installed directly

CLI additions

# List active catalogs
specify extension catalogs

# Add a catalog (project-scoped)
specify extension catalog add --name "internal" https://internal.company.com/spec-kit/catalog.json

# Remove a catalog
specify extension catalog remove internal

# Show which catalog an extension came from
specify extension info jira
# → Source catalog: org-approved

Backward compatibility

• Built-in default stack includes catalog.community.json as install_allowed: false — no config needed to get community discoverability
• SPECKIT_CATALOG_URL env var still works: treated as a single install_allowed: true catalog, replacing both defaults for full backward compat
• Explicit .specify/extension-catalogs.yml overrides all defaults entirely

Acceptance Criteria

• specify extension catalogs lists all active catalogs with name, URL, priority, and install_allowed
• When no catalog config exists, catalog.community.json is included in the default stack as install_allowed: false at priority 2
• specify extension search aggregates results across all active catalogs, annotating each result with its source catalog
• specify extension add respects install_allowed: false and rejects installs from discovery-only catalogs with a clear message: "'linear' is available in the community catalog but not in your approved catalog. Add it to .specify/extension-catalogs.yml with install_allowed: true to enable installation."
• .specify/extension-catalogs.yml and `~/.specify/extension-cata...

• Fixes feat(extensions): support multiple active catalogs simultaneously #1707

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

[mnriem]

@copilot Address the following

Downloading ruff (10.7MiB)
Downloaded ruff
Installed 1 package in 2ms
F541 [*] f-string without any placeholders
--> src/specify_cli/init.py:1921:23
|
1919 | config_path = specify_dir / "extension-catalogs.yml"
1920 | if not config_path.exists():
1921 | console.print(f"[red]Error:[/red] No catalog config found. Nothing to remove.")
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1922 | raise typer.Exit(1)
|
help: Remove extraneous f prefix

F541 [*] f-string without any placeholders
--> src/specify_cli/init.py:2219:21
|
2217 | console.print(f"\n [yellow]⚠[/yellow] Not directly installable from '{catalog_name}'.")
2218 | console.print(
2219 | f" Add to an approved catalog with install_allowed: true, "
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2220 | f"or use: specify extension add --from "
2221 | )
|
help: Remove extraneous f prefix

F541 [*] f-string without any placeholders
--> src/specify_cli/init.py:2220:21
|
2218 | console.print(
2219 | f" Add to an approved catalog with install_allowed: true, "
2220 | f"or use: specify extension add --from "
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2221 | )
2222 | console.print()
|
help: Remove extraneous f prefix

Found 3 errors.
[*] 3 fixable with the --fix option.

[Copilot]

@copilot Address the following

Downloading ruff (10.7MiB)
...

Fixed in 7279f1d — removed the extraneous f prefix from the three strings that had no placeholders (catalog_remove error message and the two-part string in extension_search).

[Copilot]

Pull request overview

This PR implements multi-catalog support for the Spec Kit extension system, replacing the single active catalog model with a composable "catalog stack." By default, both the org-approved catalog.json (installable) and catalog.community.json (discovery-only) are active simultaneously, enabling out-of-the-box community discoverability while preserving org curation controls.

Changes:

• Catalog stack engine: New CatalogEntry dataclass, get_active_catalogs(), _load_catalog_config(), _fetch_single_catalog(), and _get_merged_extensions() methods in extensions.py — implementing the full resolution order (env var → project config → user config → built-in defaults)
• CLI commands: Three new commands added to __init__.py: specify extension catalogs, specify extension catalog add, and specify extension catalog remove; plus updated extension_add, extension_search, and extension_info to surface _install_allowed / _catalog_name metadata
• Tests and docs: 13 new tests in TestCatalogEntry and TestCatalogStack; comprehensive updates to RFC, User Guide, and API Reference

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/specify_cli/extensions.py	Core multi-catalog logic: CatalogEntry dataclass, get_active_catalogs, _load_catalog_config, _fetch_single_catalog, _get_merged_extensions, updated search and get_extension_info, extended clear_cache
src/specify_cli/__init__.py	New catalog_app typer sub-app; new extension_catalogs, catalog_add, catalog_remove commands; _install_allowed enforcement in extension_add; catalog metadata display in extension_search and extension_info
tests/test_extensions.py	13 new tests covering CatalogEntry, get_active_catalogs, _load_catalog_config, merge conflict resolution, and _install_allowed propagation; updated test_search_all_extensions to use single-catalog config
extensions/RFC-EXTENSION-SYSTEM.md	Rewrote "Custom Catalogs" section to document the implemented catalog stack (was "FUTURE FEATURE")
extensions/EXTENSION-USER-GUIDE.md	Updated "Finding Extensions" and "Extension Catalogs" sections; new catalog management documentation
extensions/EXTENSION-API-REFERENCE.md	Added CatalogEntry API docs; updated ExtensionCatalog methods; added new CLI command docs

Comments suppressed due to low confidence (1)

src/specify_cli/init.py:1830

• At line 1829, catalog._load_catalog_config(config_path) is called after get_active_catalogs() has already succeeded. However, if SPECKIT_CATALOG_URL is set AND a .specify/extension-catalogs.yml exists with an invalid URL (HTTP non-localhost), then:

1. get_active_catalogs() returns early using the env var (succeeds), so no ValidationError is raised there.
2. But the _load_catalog_config(config_path) call on line 1829 then independently validates the URLs in the file, and will throw an unhandled ValidationError.

This ValidationError is not caught in extension_catalogs(), causing an unhandled exception crash instead of a clean error message. The call on line 1829 should be wrapped in a try/except, or the intent of this second _load_catalog_config call should be to check only file existence (not URL validity).

    config_path = project_root / ".specify" / "extension-catalogs.yml"
    if config_path.exists() and catalog._load_catalog_config(config_path) is not None:
        console.print(f"[dim]Config: {config_path.relative_to(project_root)}[/dim]")

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The config source detection in extension_catalogs (lines 1828–1836) doesn't account for the user-level config path ~/.specify/extension-catalogs.yml. When a user-level config is active (i.e., no project config and no SPECKIT_CATALOG_URL), the function falls through to the else branch and displays "Using built-in default catalog stack." — which is incorrect. The footer message should also check whether a user-level config is active, similar to how the project-level config is detected.

This issue also appears on line 1828 of the same file.

[Copilot]

The docstring for _load_catalog_config says it returns None when the file "contains no valid catalog entries", but when a ValidationError is raised inside the method (from _validate_catalog_url), it is not caught by the except (yaml.YAMLError, OSError) handler on line 1021. As a result, an invalid URL in the config file causes a ValidationError to propagate to callers (including get_active_catalogs), rather than returning None as the documentation implies.

This is an inconsistency between the documented behavior and actual behavior. The docstring should be updated to document the Raises: ValidationError case, or alternatively, ValidationError should be added to the exception handler if the intent is truly to return None on all errors.

Suggested change

	or contains no valid catalog entries.
	or contains no valid catalog entries.
	Raises:
	ValidationError: If any catalog entry has an invalid URL.

[Copilot]

The test_search_all_extensions test was correctly updated to write a single-catalog config (to prevent the community catalog from being fetched). However, several other existing tests in TestExtensionCatalog that are not part of this diff (test_search_by_query, test_search_by_tag, test_search_verified_only, test_get_extension_info) don't write a catalog config and will now use the new default two-catalog stack. This causes them to attempt a network fetch of the community catalog during test runs. In an offline CI environment, this produces spurious stderr warnings. These tests should also be updated to use a single-catalog config (as done for test_search_all_extensions) to preserve test isolation.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/specify_cli/extensions.py:1021

• The except clause on line 1021 only catches yaml.YAMLError and OSError, but int(item.get("priority", idx + 1)) on line 1016 can raise ValueError if the priority field contains a non-integer value in the YAML (e.g., priority: "not-a-number"). Similarly, bool(item.get("install_allowed", True)) on line 1017 can raise TypeError in edge cases. These exceptions would propagate as unhandled Python errors through _load_catalog_config() and get_active_catalogs(), resulting in a raw traceback for the user instead of a graceful fallback.

The except handler should also catch (ValueError, TypeError) to handle malformed YAML values gracefully, consistent with how YAML parsing errors are handled.

                    priority=int(item.get("priority", idx + 1)),
                    install_allowed=bool(item.get("install_allowed", True)),
                ))
            entries.sort(key=lambda e: e.priority)
            return entries if entries else None
        except (yaml.YAMLError, OSError):

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

There's an inconsistency in the default value for install_allowed between _load_catalog_config (line 1017 in extensions.py) and the catalog_add CLI command (line 1845-1848 in __init__.py).

In _load_catalog_config, install_allowed defaults to True when the key is missing from YAML: bool(item.get("install_allowed", True)). But the catalog_add CLI command defaults to False for install_allowed.

This means that a user who manually edits the YAML to add a catalog without specifying install_allowed will get a catalog with install_allowed=True, but a user who uses the CLI catalog add command without specifying --install-allowed will get a catalog with install_allowed=False. This inconsistency can be confusing and lead to unexpected behavior depending on how the catalog is configured.

This issue also appears on line 1016 of the same file.

Suggested change

	install_allowed=bool(item.get("install_allowed", True)),
	install_allowed=bool(item.get("install_allowed", False)),