Skip to content

Add analyze subset command#1301

Merged
gayanW merged 1 commit into
mainfrom
AIENG-367-1
May 18, 2026
Merged

Add analyze subset command#1301
gayanW merged 1 commit into
mainfrom
AIENG-367-1

Conversation

@gayanW
Copy link
Copy Markdown
Contributor

@gayanW gayanW commented May 15, 2026

Add new command analyze subset which can be used to demonstrate PTS effectiveness.

Usage:

$ smart-tests analyze subset --help
Usage: smart-tests analyze subset [OPTIONS] <SUBSET_ID>

Arguments:
  SUBSET_ID
      Subset ID to analyze

Options:
  --baseline-subset-id INT
      Baseline subset ID to compare against (auto-detected if not provided) [optional]

Example:

$ smart-tests analyze subset 2019138
→ Fetching subset data...
✓ Found 67 tests in subset

──────────────────────────────────────────────────────────────────────
Select Expected Tests
──────────────────────────────────────────────────────────────────────

Code files modified:
  - smart_tests/commands/inspect/model.py

Which tests do you expect PTS to prioritize?

? Select expected tests? Yes

Use Tab/Space to toggle selection, Enter when done

? Search and select expected tests: ['file=tests/commands/inspect/test_model.py']

✓ Selected 1 expected test(s):
  • file=tests/commands/inspect/test_model.py

──────────────────────────────────────────────────────────────────────
Step 2: Comparison Results
──────────────────────────────────────────────────────────────────────

Δ Rank      Subset Rank  Test Name                                  Reason                                                 Density
--------  -------------  -----------------------------------------  ---------------------------------------------------  ---------
↑42                   1  file=tests/commands/inspect/test_model.py  Changed file: smart_tests/commands/inspect/model.py   0.840091


✅ Tests Promoted:
- test_model.py promoted by 42 positions to rank #1

✅ Tests Prioritized:
- Expected test prioritized at rank #1 out of 67 (top 1.5%)

✅ Test Selection:
- PTS successfully picked expected tests with high confidence
- test_model.py: density 0.84 (very strong confidence)

@gayanW
Copy link
Copy Markdown
Contributor Author

gayanW commented May 18, 2026

Light review should be fine, since I tested all the execution paths locally.

@gayanW gayanW merged commit 7a912fa into main May 18, 2026
3 checks passed
@gayanW gayanW deleted the AIENG-367-1 branch May 18, 2026 07:04
@github-actions github-actions Bot mentioned this pull request May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants