From 289b4490f951d5621f0153c1c0c894c5aaa92c8f Mon Sep 17 00:00:00 2001 From: Juan David Date: Sat, 21 Mar 2026 18:40:09 -0500 Subject: [PATCH 1/4] docs(FR-00001): add user story and tech spec for --config flag Co-Authored-By: Claude Sonnet 4.6 --- docs/tickets/features/FR-00001-TD.md | 33 ++++++++++++++++++++++++++++ docs/tickets/features/FR-00001-US.md | 23 +++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 docs/tickets/features/FR-00001-TD.md create mode 100644 docs/tickets/features/FR-00001-US.md diff --git a/docs/tickets/features/FR-00001-TD.md b/docs/tickets/features/FR-00001-TD.md new file mode 100644 index 0000000..a5f9325 --- /dev/null +++ b/docs/tickets/features/FR-00001-TD.md @@ -0,0 +1,33 @@ +# Technical Document: FR-00001 — Add --config flag to storage CLI + +## Overview + +The core-storage CLI currently resolves configuration through a layered system: package defaults → `cwd/settings.toml` → `~/.config/core-storage/settings.toml` → env vars → CLI flags. The project-root layer is implicit and CWD-dependent, making it fragile when the user is not in the expected directory. This ticket adds a `--config ` flag that lets the user point explicitly at a settings file, removing the CWD dependency and giving a single, predictable way to switch configurations. + +## Approach + +A new `explicit_config` parameter is added to `LayerDiscovery.discover_layers()`. When set, it inserts the file as a named layer (`"explicit-config"`) immediately above the XDG user config, and skips project-root auto-discovery entirely. This preserves the full layer priority order: explicit config overrides user config, but env vars and CLI flags still override explicit config. + +The change is threaded through `get_settings()` as a `config_file` parameter and exposed at the CLI via a Typer `--config`/`-c` option with `exists=True`, which delegates the "file not found" validation to Typer before any backend setup runs. + +## Architecture / Design Decisions + +- **Priority slot**: explicit config sits above user config but below env vars and CLI flags. This means per-invocation overrides (env vars, flags) always win, which is consistent with the existing precedence model. +- **Skip project-root when explicit config is given**: combining both would be surprising — if you pass `--config`, you are declaring your config source explicitly and auto-discovery adds noise. +- **`exists=True` on the Typer option**: offloads path validation to Typer, keeping `configure_backend` free of manual file checks and producing a consistent error message format. + +## Implementation Plan + +1. Add `explicit_config: Path | None = None` to `LayerDiscovery.discover_layers()`: skip project-root step when set, append an `"explicit-config"` `LayerSource` after user-config. +2. Add `config_file: Path | None = None` to `get_settings()` and pass it as `explicit_config` to `LayerDiscovery`. +3. Add `--config`/`-c` Typer option to the `configure_backend` CLI callback with `exists=True`; pass resolved path to `get_settings(config_file=...)`. + +## Testing Strategy + +- Unit tests for `LayerDiscovery`: explicit config appears in layers, project-root is skipped, explicit config sits above user-config by index. +- Unit tests for `get_settings()`: config file is applied, config file beats project-root, CLI overrides beat config file. +- CLI tests: `--config` with a valid file exits 0 and uses the settings from that file; `--config` with a nonexistent path exits non-zero. + +## Risks / Unknowns + +- None. The change is additive and all existing behaviour is preserved when `--config` is absent. diff --git a/docs/tickets/features/FR-00001-US.md b/docs/tickets/features/FR-00001-US.md new file mode 100644 index 0000000..415dd9d --- /dev/null +++ b/docs/tickets/features/FR-00001-US.md @@ -0,0 +1,23 @@ +# User Story: FR-00001 — Add --config flag to storage CLI + +## Story + +As a developer using the core-storage CLI, I want to pass an explicit settings file path via a `--config` flag so that I can switch between storage configurations without depending on my current working directory or modifying global user settings. + +## Acceptance Criteria + +- [ ] The CLI accepts a `--config ` / `-c ` flag that points to a settings TOML file. +- [ ] When `--config` is provided, the specified file is loaded as a configuration layer above the XDG user config (`~/.config/core-storage/settings.toml`). +- [ ] When `--config` is provided, project-root auto-discovery (`cwd/settings.toml`) is skipped. +- [ ] Environment variables and other CLI flags (e.g. `--backend`, `--sqlite-path`) still override values from `--config`. +- [ ] If the path passed to `--config` does not exist, the CLI exits with a non-zero code and a clear error message before attempting to connect to any backend. +- [ ] All existing CLI behaviour is unchanged when `--config` is not provided. + +## Out of Scope + +- Adding a `--config` flag to the cache CLI (no CLI exists yet for that package). +- Supporting multiple `--config` files in a single invocation. + +## Notes + +- Related to the broader question of how cache and storage share configuration at the CLI level. From 3365bb2d8b6093ea9f5b6274ce4fd0747ced4069 Mon Sep 17 00:00:00 2001 From: Juan David Date: Sat, 21 Mar 2026 18:40:39 -0500 Subject: [PATCH 2/4] feat(FR-00001): add --config flag to core-storage CLI Adds --config / -c option to pass an explicit settings file path, bypassing cwd-based project-root auto-discovery. The explicit config layer sits above XDG user config but below env vars and CLI flags, preserving the existing priority model. Co-Authored-By: Claude Sonnet 4.6 --- .../storage/src/core_storage/cli/__init__.py | 15 +++++- .../src/core_storage/settings/layers.py | 45 +++++++++++------ .../src/core_storage/settings/resolver.py | 9 +++- .../tests/unit/settings/test_layers.py | 50 +++++++++++++++++++ .../tests/unit/settings/test_resolver.py | 31 ++++++++++++ packages/storage/tests/unit/test_cli.py | 24 +++++++++ 6 files changed, 158 insertions(+), 16 deletions(-) diff --git a/packages/storage/src/core_storage/cli/__init__.py b/packages/storage/src/core_storage/cli/__init__.py index e96fc11..93b715a 100644 --- a/packages/storage/src/core_storage/cli/__init__.py +++ b/packages/storage/src/core_storage/cli/__init__.py @@ -35,6 +35,16 @@ @app.callback() def configure_backend( ctx: typer.Context, + config: Path | None = typer.Option( + None, + "--config", + "-c", + help="Path to a settings.toml file. Overrides project-root and user config.", + exists=True, + file_okay=True, + dir_okay=False, + readable=True, + ), backend_type: str | None = typer.Option( None, "--backend", @@ -98,7 +108,10 @@ def configure_backend( cli_overrides["backend.redis.password"] = redis_password try: - storage_settings = get_settings(cli_overrides=cli_overrides or None) + storage_settings = get_settings( + config_file=config, + cli_overrides=cli_overrides or None, + ) ctx.obj = Store(create_backend(storage_settings)) except StorageError as storage_error: typer.echo(f"Error: {storage_error}", err=True) diff --git a/packages/storage/src/core_storage/settings/layers.py b/packages/storage/src/core_storage/settings/layers.py index 30de2a4..2d27573 100644 --- a/packages/storage/src/core_storage/settings/layers.py +++ b/packages/storage/src/core_storage/settings/layers.py @@ -52,6 +52,7 @@ def discover_layers( app_name: str, project_root: Path | None = None, env_prefix: str | None = None, + explicit_config: Path | None = None, _environ: dict[str, str] | None = None, ) -> list[LayerSource]: """Return all applicable layers ordered lowest to highest priority. @@ -60,9 +61,14 @@ def discover_layers( app_name: Used to construct the XDG config path. project_root: Explicit project root for the project-level ``settings.toml``. Falls back to ``Path.cwd()``. + Ignored when ``explicit_config`` is provided. env_prefix: When provided, environment variables matching ``SECTION__FIELD`` are injected as the highest-priority layer. + explicit_config: Path to a settings file passed explicitly (e.g. + via ``--config``). When provided, project-root + auto-discovery is skipped and this file is loaded + above the XDG user config. _environ: Override ``os.environ`` for testing. """ discovered_layers: list[LayerSource] = [] @@ -77,20 +83,21 @@ def discover_layers( ) ) - # 2. Project root settings.toml - try: - root = project_root if project_root is not None else Path.cwd() - project_settings_path = root / "settings.toml" - if project_settings_path.exists(): - discovered_layers.append( - LayerSource( - name="project-root", - filepath=project_settings_path, - is_namespaced=True, + # 2. Project root settings.toml (skipped when explicit_config is given) + if explicit_config is None: + try: + root = project_root if project_root is not None else Path.cwd() + project_settings_path = root / "settings.toml" + if project_settings_path.exists(): + discovered_layers.append( + LayerSource( + name="project-root", + filepath=project_settings_path, + is_namespaced=True, + ) ) - ) - except OSError: - pass # cwd inaccessible (e.g. deleted in container) — skip + except OSError: + pass # cwd inaccessible (e.g. deleted in container) — skip # 3. XDG user config user_settings_path = get_user_settings_file(app_name) @@ -103,7 +110,17 @@ def discover_layers( ) ) - # 4. Environment variables + # 4. Explicit config (--config flag) — overrides user config + if explicit_config is not None: + discovered_layers.append( + LayerSource( + name="explicit-config", + filepath=explicit_config, + is_namespaced=True, + ) + ) + + # 5. Environment variables if env_prefix: env_data = parse_env_vars(prefix=env_prefix, environ=_environ) if env_data: diff --git a/packages/storage/src/core_storage/settings/resolver.py b/packages/storage/src/core_storage/settings/resolver.py index 056650d..fd64a40 100644 --- a/packages/storage/src/core_storage/settings/resolver.py +++ b/packages/storage/src/core_storage/settings/resolver.py @@ -13,17 +13,23 @@ def get_settings( project_root: Path | None = None, + config_file: Path | None = None, cli_overrides: dict[str, Any] | None = None, _environ: dict[str, str] | None = None, ) -> StorageSettings: """Return resolved settings by applying all layers in priority order. Priority: package defaults < project settings.toml < - ~/.config/core-storage/settings.toml < env vars < cli_overrides. + ~/.config/core-storage/settings.toml < config_file < + env vars < cli_overrides. Args: project_root: Override the project root used to find ``settings.toml``. Defaults to ``Path.cwd()``. + Ignored when ``config_file`` is provided. + config_file: Explicit path to a settings file (e.g. from ``--config``). + When provided, project-root auto-discovery is skipped and + this file is loaded above the XDG user config. cli_overrides: Dotted-path overrides from CLI flags, e.g. ``{"backend.type": "redis"}``. _environ: Override ``os.environ`` (for testing). @@ -32,6 +38,7 @@ def get_settings( app_name=APP_NAME, project_root=project_root, env_prefix=ENV_PREFIX, + explicit_config=config_file, _environ=_environ, ) validated_settings = ConfigBuilder.build( diff --git a/packages/storage/tests/unit/settings/test_layers.py b/packages/storage/tests/unit/settings/test_layers.py index e613171..eb899e0 100644 --- a/packages/storage/tests/unit/settings/test_layers.py +++ b/packages/storage/tests/unit/settings/test_layers.py @@ -82,6 +82,56 @@ def test_discover_layers_includes_xdg_user_config( assert "user-config" in names +def test_discover_layers_explicit_config_added_as_layer( + tmp_path: Path, +) -> None: + config_file = tmp_path / "custom.toml" + config_file.write_text('[backend]\ntype = "json"\n') + layers = LayerDiscovery.discover_layers( + app_name="core-storage", explicit_config=config_file + ) + names = [layer.name for layer in layers] + assert "explicit-config" in names + + +def test_discover_layers_explicit_config_skips_project_root( + tmp_path: Path, +) -> None: + project_settings = tmp_path / "settings.toml" + project_settings.write_text('[backend]\ntype = "sqlite"\n') + config_file = tmp_path / "custom.toml" + config_file.write_text('[backend]\ntype = "json"\n') + layers = LayerDiscovery.discover_layers( + app_name="core-storage", + project_root=tmp_path, + explicit_config=config_file, + ) + names = [layer.name for layer in layers] + assert "project-root" not in names + assert "explicit-config" in names + + +def test_discover_layers_explicit_config_sits_above_user_config( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + xdg_dir = tmp_path / "config" + app_dir = xdg_dir / "core-storage" + app_dir.mkdir(parents=True) + (app_dir / "settings.toml").write_text('[backend]\ntype = "sqlite"\n') + monkeypatch.setenv("XDG_CONFIG_HOME", str(xdg_dir)) + + config_file = tmp_path / "custom.toml" + config_file.write_text('[backend]\ntype = "json"\n') + + layers = LayerDiscovery.discover_layers( + app_name="core-storage", explicit_config=config_file + ) + names = [layer.name for layer in layers] + user_idx = names.index("user-config") + explicit_idx = names.index("explicit-config") + assert explicit_idx > user_idx + + def test_discover_layers_handles_oserror_for_project_root( tmp_path: Path, monkeypatch: pytest.MonkeyPatch ) -> None: diff --git a/packages/storage/tests/unit/settings/test_resolver.py b/packages/storage/tests/unit/settings/test_resolver.py index 5cc2ceb..82c66f6 100644 --- a/packages/storage/tests/unit/settings/test_resolver.py +++ b/packages/storage/tests/unit/settings/test_resolver.py @@ -54,3 +54,34 @@ def test_get_settings_cli_overrides_beat_project_settings(tmp_path: Path) -> Non cli_overrides={"backend.type": "redis"}, ) assert settings.backend.type == BackendType.redis + + +def test_get_settings_config_file_applied(tmp_path: Path) -> None: + config_file = tmp_path / "custom.toml" + config_file.write_text('[backend]\ntype = "json"\n') + settings = get_settings(config_file=config_file, _environ={}) + assert settings.backend.type == BackendType.json + + +def test_get_settings_config_file_beats_project_root(tmp_path: Path) -> None: + project_settings = tmp_path / "settings.toml" + project_settings.write_text('[backend]\ntype = "sqlite"\n') + config_file = tmp_path / "custom.toml" + config_file.write_text('[backend]\ntype = "json"\n') + settings = get_settings( + project_root=tmp_path, + config_file=config_file, + _environ={}, + ) + assert settings.backend.type == BackendType.json + + +def test_get_settings_cli_overrides_beat_config_file(tmp_path: Path) -> None: + config_file = tmp_path / "custom.toml" + config_file.write_text('[backend]\ntype = "json"\n') + settings = get_settings( + config_file=config_file, + _environ={}, + cli_overrides={"backend.type": "redis"}, + ) + assert settings.backend.type == BackendType.redis diff --git a/packages/storage/tests/unit/test_cli.py b/packages/storage/tests/unit/test_cli.py index 0ee7aea..67cddd6 100644 --- a/packages/storage/tests/unit/test_cli.py +++ b/packages/storage/tests/unit/test_cli.py @@ -531,6 +531,30 @@ def test_exists_storage_error_exits_1() -> None: # --------------------------------------------------------------------------- +def test_config_flag_loads_settings(tmp_path: object) -> None: + import tempfile + from pathlib import Path + + with tempfile.TemporaryDirectory() as tmpdir: + config_file = Path(tmpdir) / "custom.toml" + json_path = Path(tmpdir) / "store.json" + config_file.write_text( + f'[backend]\ntype = "json"\n[backend.json]\npath = "{json_path}"\n' + ) + result = runner.invoke( + app, + ["--config", str(config_file), "list", "ns"], + ) + assert result.exit_code == 0 + + +def test_config_flag_missing_file_exits_nonzero() -> None: + result = runner.invoke( + app, ["--config", "/nonexistent/settings.toml", "list", "ns"] + ) + assert result.exit_code != 0 + + def test_sqlite_path_flag_used() -> None: import tempfile from pathlib import Path From 93a206fdd8fc3783b5218216f852b365fc4e74bd Mon Sep 17 00:00:00 2001 From: Juan David Date: Mon, 23 Mar 2026 21:14:33 -0500 Subject: [PATCH 3/4] =?UTF-8?q?=E2=9C=85=20test(FR-00001):=20add=20smoke?= =?UTF-8?q?=20tests=20for=20core-storage=20--config=20flag?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit End-to-end CLI tests covering: --help baseline, --config loading, set/get/exists/list/delete/clear round-trips, --config overrides project-root auto-discovery, and invalid path rejection. Workflow triggers on packages/storage/** changes only. Co-Authored-By: Claude Sonnet 4.6 --- .github/workflows/smoke-test-storage.yml | 44 ++++++ .../storage/tests/smoke/run-smoke-tests.sh | 138 ++++++++++++++++++ 2 files changed, 182 insertions(+) create mode 100644 .github/workflows/smoke-test-storage.yml create mode 100755 packages/storage/tests/smoke/run-smoke-tests.sh diff --git a/.github/workflows/smoke-test-storage.yml b/.github/workflows/smoke-test-storage.yml new file mode 100644 index 0000000..f55a63b --- /dev/null +++ b/.github/workflows/smoke-test-storage.yml @@ -0,0 +1,44 @@ +name: Smoke Tests (storage) + +on: + push: + branches: ["**"] + paths: + - "packages/storage/**" + - ".github/workflows/smoke-test-storage.yml" + workflow_dispatch: + inputs: + verbose: + description: 'Verbose test output' + required: false + default: false + type: boolean + +jobs: + smoke-test: + name: Smoke Tests (storage) + runs-on: ubuntu-latest + timeout-minutes: 10 + + steps: + - name: Checkout code + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Install uv + run: pip install uv + + - name: Install dependencies + run: uv sync --dev --all-packages + + - name: Run smoke tests (standard mode) + if: inputs.verbose == false && github.event_name != 'push' + run: bash packages/storage/tests/smoke/run-smoke-tests.sh + + - name: Run smoke tests (verbose mode) + if: inputs.verbose == true || github.event_name == 'push' + run: bash packages/storage/tests/smoke/run-smoke-tests.sh --verbose diff --git a/packages/storage/tests/smoke/run-smoke-tests.sh b/packages/storage/tests/smoke/run-smoke-tests.sh new file mode 100755 index 0000000..b0c1f0b --- /dev/null +++ b/packages/storage/tests/smoke/run-smoke-tests.sh @@ -0,0 +1,138 @@ +#!/usr/bin/env bash +# Smoke tests for core-storage CLI — validates the --config flag end-to-end. +# Usage: bash tests/smoke/run-smoke-tests.sh [--verbose] + +set -euo pipefail + +VERBOSE=false +for arg in "$@"; do + [[ "$arg" == "--verbose" ]] && VERBOSE=true +done + +PASS=0 +FAIL=0 +TMPDIR_ROOT=$(mktemp -d) +trap 'rm -rf "$TMPDIR_ROOT"' EXIT + +log() { [[ "$VERBOSE" == true ]] && echo "$@" || true; } +pass() { echo " ✓ $1"; PASS=$((PASS + 1)); } +fail() { echo " ✗ $1" >&2; FAIL=$((FAIL + 1)); } + +run_test() { + local description="$1"; shift + log " running: core-storage $*" + if "$@" > /dev/null 2>&1; then + pass "$description" + else + fail "$description" + fi +} + +run_test_expect_fail() { + local description="$1"; shift + log " running (expect failure): core-storage $*" + if ! "$@" > /dev/null 2>&1; then + pass "$description" + else + fail "$description" + fi +} + +# --------------------------------------------------------------------------- + +echo "=== Smoke Tests: core-storage ===" +log "" + +# --- Baseline --- +echo "--- Baseline ---" +run_test "--help exits 0" core-storage --help + +# --- --config flag: basic loading --- +echo "" +echo "--- --config flag ---" + +CONFIG="$TMPDIR_ROOT/custom.toml" +JSON_STORE="$TMPDIR_ROOT/store.json" +cat > "$CONFIG" < "$PROJECT_DIR/settings.toml" < "$EXPLICIT_CONFIG" < /dev/null && + [[ ! -f '$PROJECT_STORE' ]] +" + +# --- invalid --config path is rejected --- +echo "" +echo "--- --config validation ---" + +run_test_expect_fail "nonexistent --config path exits nonzero" \ + core-storage --config /nonexistent/path.toml list ns + +# --------------------------------------------------------------------------- + +echo "" +echo "Results: ${PASS} passed, ${FAIL} failed" + +if [[ "$FAIL" -gt 0 ]]; then + exit 1 +fi From 0f5d545b53f7176ddca90faa5de1fe7989ed9421 Mon Sep 17 00:00:00 2001 From: Juan David Date: Mon, 23 Mar 2026 21:24:50 -0500 Subject: [PATCH 4/4] fix(ci): add venv to PATH before running storage smoke tests Co-Authored-By: Claude Sonnet 4.6 --- .github/workflows/smoke-test-storage.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/smoke-test-storage.yml b/.github/workflows/smoke-test-storage.yml index f55a63b..2a93f5b 100644 --- a/.github/workflows/smoke-test-storage.yml +++ b/.github/workflows/smoke-test-storage.yml @@ -35,6 +35,9 @@ jobs: - name: Install dependencies run: uv sync --dev --all-packages + - name: Add venv to PATH + run: echo "$(pwd)/.venv/bin" >> $GITHUB_PATH + - name: Run smoke tests (standard mode) if: inputs.verbose == false && github.event_name != 'push' run: bash packages/storage/tests/smoke/run-smoke-tests.sh