feat: add LISA test suite support#126
Draft
reubeno wants to merge 1 commit intomicrosoft:mainfrom
Draft
Conversation
Add LISA test framework support as a second test suite type alongside pytest. LISA test suites specify git sources for the framework and runbook repos (pinned to commit SHAs), pip pre-install overrides, extras, and extra-args with placeholder substitution. Add LisaConfig, GitSourceConfig, and LisaRunbookConfig types with validation (including commit SHA format checking). Extend TestSuiteConfig.Validate() for LISA type-specific rules including cross-subtable mismatch detection. Add lisarunner.go implementing venv setup, git repo cloning, pip installation, SSH key generation, and LISA invocation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
liunan-ms
reviewed
Apr 30, 2026
|
|
||
| if err := prereqs.RequireExecutable(env, "git", nil); err != nil { | ||
| return fmt.Errorf("git is required to clone LISA repos:\n%w", err) | ||
| } |
Contributor
There was a problem hiding this comment.
ensureAdminKeyPair uses ssh-keygen but the runner only checks for python3 and git. ssh-keygen check should be added here.
liunan-ms
reviewed
Apr 30, 2026
|
|
||
| // Type indicates the test framework to use. | ||
| Type TestType `toml:"type" json:"type" jsonschema:"required,enum=pytest,title=Type,description=Type of test framework (pytest)"` | ||
| Type TestType `toml:"type" json:"type" jsonschema:"required,enum=pytest lisa,title=Type,description=Type of test framework (pytest or lisa)"` |
Contributor
There was a problem hiding this comment.
"pytest lisa" will be treated as a single value, two enum entries (enum=pytest, enum=lisa) are needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add LISA test framework support as a second test suite type alongside pytest. LISA test suites specify git sources for the framework and runbook repos (pinned to commit SHAs), pip pre-install overrides, extras, and extra-args with placeholder substitution.
Add LisaConfig, GitSourceConfig, and LisaRunbookConfig types with validation (including commit SHA format checking). Extend TestSuiteConfig.Validate() for LISA type-specific rules including cross-subtable mismatch detection.
Add lisarunner.go implementing venv setup, git repo cloning, pip installation, SSH key generation, and LISA invocation.