Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
92c09c3
fix
alexandraBara May 4, 2026
c45ddc1
-Added support for paginated Redfish GET requests
amd-aharding May 5, 2026
27bd68f
-Converted Redfish constants to unix format
amd-aharding May 5, 2026
60c5059
-Added follow_next_link and max_pages to README
amd-aharding May 6, 2026
ade2800
-Updated const comments
amd-aharding May 6, 2026
ff7dbc1
updates
alexandraBara May 7, 2026
89d3ffc
updates
alexandraBara May 7, 2026
eae6dda
added serialization
alexandraBara May 7, 2026
e6f5a82
appending exception traceback
alexandraBara May 8, 2026
1e7ef9c
utest
alexandraBara May 8, 2026
e8b29da
fix for workflow
alexandraBara May 8, 2026
b333e44
Merge pull request #3 from amd/aharding/rf-pagination
alexandraBara May 11, 2026
bc500d4
added session_id
alexandraBara May 11, 2026
6489f1c
network enhancements
alexandraBara May 13, 2026
b7c2913
utest fix
alexandraBara May 15, 2026
825b2e0
fix
alexandraBara May 15, 2026
79a51ae
Merge pull request #195 from amd/alex_network
alexandraBara May 18, 2026
4151ccd
docs: Update plugin documentation [automated]
github-actions[bot] May 19, 2026
c791c3d
Merge pull request #196 from amd/automated-plugin-docs-update
alexandraBara May 19, 2026
309e219
codeowners update
alexandraBara May 19, 2026
4f2f277
exiting with NOT_RAN when cards not present
alexandraBara May 19, 2026
73cb9df
rocm versioning update
alexandraBara May 19, 2026
c8d0ea6
Merge pull request #197 from amd/alex_nic_update
alexandraBara May 20, 2026
e3ceca6
docs: Update plugin documentation [automated]
github-actions[bot] May 21, 2026
acf89d0
merged develop
alexandraBara May 26, 2026
29d92df
Merge pull request #198 from amd/automated-plugin-docs-update
alexandraBara May 27, 2026
dfeec7e
Merge branch 'development' into alex_rocm_versioning
alexandraBara May 27, 2026
dff0232
Merge branch 'development' into alex_filemodel_update
alexandraBara May 27, 2026
f2bf3db
readme fix
alexandraBara May 27, 2026
03aac37
Merge pull request #200 from amd/alex_rocm_versioning
alexandraBara May 27, 2026
508ef2a
utest fix
alexandraBara May 27, 2026
bc0160e
enhancements
alexandraBara May 27, 2026
f03e79c
Merge branch 'development' into alex_filemodel_update
alexandraBara May 27, 2026
47c1131
docs: Update plugin documentation [automated]
github-actions[bot] May 28, 2026
0e854c5
Merge pull request #199 from amd/alex_filemodel_update
alexandraBara May 28, 2026
5c0f5ea
Merge branch 'development' into alex_pcie_fix
alexandraBara May 28, 2026
59146e9
Merge pull request #201 from amd/alex_pcie_fix
alexandraBara May 28, 2026
8f2476e
Merge branch 'development' into automated-plugin-docs-update
alexandraBara May 28, 2026
9b727e0
Merge pull request #202 from amd/automated-plugin-docs-update
alexandraBara May 28, 2026
641082e
docs: Update plugin documentation [automated]
github-actions[bot] May 28, 2026
59199cf
Merge pull request #203 from amd/automated-plugin-docs-update
alexandraBara May 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1 +1 @@
* @landrews-amd @alexandraBara @jaspals3123
*@alexandraBara
11 changes: 9 additions & 2 deletions .github/workflows/code_quality_checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,19 @@ jobs:
container: python:3.9

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for pre-commit to work

# python:3.9 image has no git; pre-commit requires it.
- name: Install git
run: |
apt-get update
apt-get install -y --no-install-recommends git

- name: Configure git for container
run: |
git config --global --add safe.directory /__w/node-scraper/node-scraper
git config --global --add safe.directory "$GITHUB_WORKSPACE"
git config --global user.email "ci@github.com"
git config --global user.name "CI Bot"
- name: setup environment and run pre-commit hooks
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/functional-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
container: python:3.9

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Install xmllint
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-trusted-publisher.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:

steps:
- name: Checkout code
uses: actions/checkout@v3
uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history and tags
token: ${{ secrets.GITHUB_TOKEN }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/unit-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
container: python:3.9

steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Install xmllint
run: |
Expand Down
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ system debug.
## Table of Contents
- [Installation](#installation)
- [Install from PyPI](#install-from-pypi)
- [Install From Source](#install-from-source)
- [Install from Source](#install-from-source)
- [CLI Usage](#cli-usage)
- [Execution Methods](#execution-methods)
- [Example: Remote Execution](#example-remote-execution)
Expand Down Expand Up @@ -38,7 +38,7 @@ Use a virtual environment if you prefer. After installation, confirm the CLI is
node-scraper --help
```

### Install From Source
### Install from Source
Node Scraper requires Python 3.9+ for installation. After cloning this repository,
call dev-setup.sh script with 'source'. This script creates an editable install of Node Scraper in
a python virtual environment and also configures the pre-commit hooks for the project.
Expand Down Expand Up @@ -491,7 +491,12 @@ The RedfishEndpointPlugin collects Redfish URIs (GET responses) and optionally r
}
```

**`collection_args`**
- **`uris`**: List of Redfish paths (e.g. `/redfish/v1/`, `/redfish/v1/Systems/1`) to GET and store.
- **`follow_next_link`**: Optional (default `false`). When `true`, the collector follows `Members@odata.nextLink` pagination for each URI and merges all pages into a single response.
- **`max_pages`**: Optional (default `200`). Safety cap on the number of pages to follow per URI when `follow_next_link` is enabled.

**`analysis_args`**
- **`checks`**: Optional. Map of URI to expected values or constraints for analysis. Supports exact match (e.g. `"PowerState": "On"`), `anyOf`, `min`/`max`, etc.

#### **'summary' sub command**
Expand Down
27 changes: 16 additions & 11 deletions docs/PLUGIN_DOC.md

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions nodescraper/base/inbandcollectortask.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@

from nodescraper.connection.inband import InBandConnection
from nodescraper.connection.inband.inband import BaseFileArtifact, CommandArtifact
from nodescraper.constants import DEFAULT_EVENT_REPORTER
from nodescraper.enums import EventPriority, OSFamily, SystemInteractionLevel
from nodescraper.generictypes import TCollectArg, TDataModel
from nodescraper.interfaces import DataCollector, TaskResultHook
Expand All @@ -52,6 +53,8 @@ def __init__(
max_event_priority_level: Union[EventPriority, str] = EventPriority.CRITICAL,
parent: Optional[str] = None,
task_result_hooks: Optional[list[TaskResultHook]] = None,
event_reporter: str = DEFAULT_EVENT_REPORTER,
session_id: Optional[str] = None,
**kwargs,
):
super().__init__(
Expand All @@ -62,6 +65,8 @@ def __init__(
connection=connection,
parent=parent,
task_result_hooks=task_result_hooks,
event_reporter=event_reporter,
session_id=session_id,
)
if self.system_info.os_family not in self.SUPPORTED_OS_FAMILY:
raise SystemCompatibilityError(
Expand Down
27 changes: 27 additions & 0 deletions nodescraper/base/redfishcollectortask.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
from typing import Generic, Optional, Union

from nodescraper.connection.redfish import RedfishConnection, RedfishGetResult
from nodescraper.constants import DEFAULT_EVENT_REPORTER
from nodescraper.enums import EventPriority
from nodescraper.generictypes import TCollectArg, TDataModel
from nodescraper.interfaces import DataCollector, TaskResultHook
Expand All @@ -47,6 +48,8 @@ def __init__(
max_event_priority_level: Union[EventPriority, str] = EventPriority.CRITICAL,
parent: Optional[str] = None,
task_result_hooks: Optional[list[TaskResultHook]] = None,
event_reporter: str = DEFAULT_EVENT_REPORTER,
session_id: Optional[str] = None,
**kwargs,
):
super().__init__(
Expand All @@ -56,6 +59,8 @@ def __init__(
max_event_priority_level=max_event_priority_level,
parent=parent,
task_result_hooks=task_result_hooks,
event_reporter=event_reporter,
session_id=session_id,
**kwargs,
)

Expand All @@ -77,3 +82,25 @@ def _run_redfish_get(
if log_artifact:
self.result.artifacts.append(res)
return res

def _run_redfish_get_paged(
self,
path: str,
max_pages: int = 200,
log_artifact: bool = True,
) -> RedfishGetResult:
"""
Run a Redfish GET and follow Members@odata.nextLink pagination, merging all pages into a single response.

Args:
path (str): Redfish URI path.
max_pages (int, optional): safety cap on the number of pages to follow. Defaults to 200.
log_artifact (bool, optional): whether we should log the merged result. Defaults to True.

Returns:
RedfishGetResult: path, success, merged data (or error), status_code.
"""
res = self.connection.run_get_paged(path, max_pages=max_pages)
if log_artifact:
self.result.artifacts.append(res)
return res
4 changes: 3 additions & 1 deletion nodescraper/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import os
import platform
import sys
import uuid
from typing import Optional

import nodescraper
Expand Down Expand Up @@ -200,7 +201,7 @@ def _add_cli_root_globals(


def build_global_argument_parser(*, add_help: bool = True) -> argparse.ArgumentParser:
"""Globals only (no subcommands), for host CLIs such as amd-error-scraper ``error-scraper``."""
"""Globals only (no subcommands), for host CLIs."""
plugin_reg = PluginRegistry()
config_reg = _config_registry_with_all_plugins(plugin_reg)
parser = argparse.ArgumentParser(
Expand Down Expand Up @@ -642,6 +643,7 @@ def main(
timestamp=timestamp,
sname=sname,
host_cli_args=host_cli_args,
session_id=str(uuid.uuid4()),
)

log_system_info(log_path, system_info, logger)
Expand Down
7 changes: 1 addition & 6 deletions nodescraper/cli/host_cli_embed.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,7 @@ def apply_host_cli_args_to_parsed_args(
parsed_args: argparse.Namespace,
host_ns: Optional[argparse.Namespace],
) -> None:
"""Copy host profile fields from an embedding host onto parsed top-level args.

Used when ``main(..., host_cli_args=...)`` is invoked (e.g. from the
error-scraper wrapper) so ``--connection-config`` profile data loaded by the
host is visible to :func:`get_system_info` and the rest of the CLI.
"""
"""Copy host profile fields from an embedding host onto parsed top-level args."""
if host_ns is None:
return
for attr in (
Expand Down
4 changes: 4 additions & 0 deletions nodescraper/cli/invocation.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ class PluginRunInvocation:
timestamp: str
sname: str
host_cli_args: Optional[argparse.Namespace] = None
session_id: Optional[str] = None


def run_plugin_queue_with_invocation(
Expand All @@ -84,6 +85,7 @@ def run_plugin_queue_with_invocation(
timestamp: str,
sname: str,
host_cli_args: Optional[argparse.Namespace] = None,
session_id: Optional[str] = None,
) -> list[PluginResult]:
"""Constructs the plugin executor, binds invocation context, and runs the plugin queue."""
inv = PluginRunInvocation(
Expand All @@ -96,6 +98,7 @@ def run_plugin_queue_with_invocation(
timestamp=timestamp,
sname=sname,
host_cli_args=host_cli_args,
session_id=session_id,
)
plugin_executor = PluginExecutor(
logger=logger,
Expand All @@ -104,6 +107,7 @@ def run_plugin_queue_with_invocation(
system_info=system_info,
log_path=log_path,
plugin_registry=plugin_reg,
session_id=session_id,
)
with plugin_run_invocation_scope(inv):
return plugin_executor.run_queue()
Expand Down
10 changes: 10 additions & 0 deletions nodescraper/connection/redfish/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,12 @@
RedfishConnectionError,
RedfishGetResult,
)
from .redfish_constants import (
RF_MEMBERS,
RF_MEMBERS_COUNT,
RF_MEMBERS_NEXT_LINK,
RF_ODATA_ID,
)
from .redfish_manager import RedfishConnectionManager
from .redfish_oem_diag import (
collect_oem_diagnostic_data,
Expand All @@ -45,4 +51,8 @@
"RedfishPath",
"collect_oem_diagnostic_data",
"get_oem_diagnostic_allowable_values",
"RF_MEMBERS",
"RF_MEMBERS_COUNT",
"RF_MEMBERS_NEXT_LINK",
"RF_ODATA_ID",
]
48 changes: 48 additions & 0 deletions nodescraper/connection/redfish/redfish_connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
from requests import Response
from requests.auth import HTTPBasicAuth

from .redfish_constants import RF_MEMBERS, RF_MEMBERS_COUNT, RF_MEMBERS_NEXT_LINK
from .redfish_path import RedfishPath

DEFAULT_REDFISH_API_ROOT = "redfish/v1"
Expand Down Expand Up @@ -183,6 +184,53 @@ def run_get(self, path: Union[str, RedfishPath]) -> RedfishGetResult:
status_code=None,
)

def run_get_paged(
self,
path: Union[str, RedfishPath],
max_pages: int = 200,
) -> RedfishGetResult:
"""Run a Redfish GET and transparently follow Members@odata.nextLink pagination.

Each subsequent page's Members list is appended to the first page's Members list
so the caller receives a single merged response body. The Members@odata.nextLink key
and Members@odata.count are updated to reflect the merged result. If there is no
Members@odata.nextLink in the first response this behaves identically to run_get.
max_pages is a safety cap on the number of pages to fetch (default 200).
"""
first = self.run_get(path)
if not first.success or first.data is None:
return first

# Short-circuit when there is nothing to page through.
if RF_MEMBERS_NEXT_LINK not in first.data:
return first

merged_members: list = list(first.data.get(RF_MEMBERS) or [])
merged_data: dict = dict(first.data)
pages_fetched = 1
next_link: Optional[str] = first.data.get(RF_MEMBERS_NEXT_LINK)
last_status_code = first.status_code

while next_link and pages_fetched < max_pages:
page_result = self.run_get(next_link)
last_status_code = page_result.status_code
if not page_result.success or page_result.data is None:
break
merged_members.extend(page_result.data.get(RF_MEMBERS) or [])
next_link = page_result.data.get(RF_MEMBERS_NEXT_LINK)
pages_fetched += 1

merged_data[RF_MEMBERS] = merged_members
merged_data[RF_MEMBERS_COUNT] = len(merged_members)
merged_data.pop(RF_MEMBERS_NEXT_LINK, None)

return RedfishGetResult(
path=first.path,
success=True,
data=merged_data,
status_code=last_status_code,
)

def copy(self) -> "RedfishConnection":
"""Return a new connection with the same config and its own session (for concurrent use)."""
return RedfishConnection(
Expand Down
38 changes: 38 additions & 0 deletions nodescraper/connection/redfish/redfish_constants.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
###############################################################################
#
# MIT License
#
# Copyright (c) 2026 Advanced Micro Devices, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
###############################################################################
"""Redfish field name constants shared across the Redfish package(s)."""

# Resource collection property which identifies members of the collection.
RF_MEMBERS = "Members"

# Resource collection property which defines the total number of resources/members.
RF_MEMBERS_COUNT = "Members@odata.count"

# Resource collection property which points to the next set of partial members from the originating operation.
RF_MEMBERS_NEXT_LINK = "Members@odata.nextLink"

# Resource identifier property (optional for registry resources, required for all other resources and resource collections).
RF_ODATA_ID = "@odata.id"
4 changes: 1 addition & 3 deletions nodescraper/connection/redfish/redfish_oem_diag.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
from nodescraper.enums import TaskState

from .redfish_connection import RedfishConnection, RedfishConnectionError
from .redfish_constants import RF_ODATA_ID
from .redfish_path import RedfishPath

_module_logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -65,9 +66,6 @@ def _log_collect_diag_response(
)


# Redfish JSON key for resource link
RF_ODATA_ID = "@odata.id"

# @Redfish.AllowableValues: Redfish annotation for the list of allowable values for a string
REDFISH_ANNOTATION_ALLOWABLE_VALUES = "Redfish.AllowableValues"

Expand Down
2 changes: 2 additions & 0 deletions nodescraper/constants.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,5 @@
#
###############################################################################
DEFAULT_LOGGER = "nodescraper"

DEFAULT_EVENT_REPORTER = "NODE_SCRAPER"
Loading
Loading