Skip to content

FEAT Add PromptIntel remote dataset loader#1400

Open
anandansundar wants to merge 5 commits intoAzure:mainfrom
anandansundar:feature/promptintel-dataset-loader
Open

FEAT Add PromptIntel remote dataset loader#1400
anandansundar wants to merge 5 commits intoAzure:mainfrom
anandansundar:feature/promptintel-dataset-loader

Conversation

@anandansundar
Copy link

Adds _PromptIntelDataset class that fetches prompts from the PromptIntel API and transforms them into PyRIT SeedDataset format (SeedPrompt + SeedObjective pairs grouped by prompt ID). Supports pagination, severity/category/search filters, and max_prompts limit. Includes 23 unit tests.

Description

Tests and Documentation

Copy link
Author

@anandansundar anandansundar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initial dataset loader from prompt intel

@romanlutz romanlutz changed the title Add PromptIntel remote dataset loader FEAT Add PromptIntel remote dataset loader Feb 25, 2026
Copy link
Contributor

@romanlutz romanlutz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome contribution @anandansundar ! This looks very good. A few small things:

Did you run the integration test for datasets test_fetch_dataset_integration for this dataset?

Also make sure to rerun the notebook 1_loading_datasets.py since it will otherwise not show this new option.

Did you run pre-commit run --all-files?

Finally, you need to accept the CLA (see the CLA bot's message) 🙂

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new _PromptIntelDataset class to fetch adversarial prompts from the PromptIntel API and load them into PyRIT's SeedDataset format. The implementation includes comprehensive filtering options (severity, category, search), pagination support, and a max_prompts limit. The PR includes 23 unit tests covering initialization, data transformation, pagination, API errors, and filter parameters.

Changes:

  • Adds _PromptIntelDataset remote dataset loader with API integration for fetching adversarial prompts
  • Includes filtering by severity, category, and search terms with proper validation
  • Adds comprehensive test suite covering all major functionality

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 10 comments.

File Description
pyrit/datasets/seed_datasets/remote/promptintel_dataset.py New dataset loader class that fetches prompts from PromptIntel API with pagination and filtering support
pyrit/datasets/seed_datasets/remote/init.py Registers the new _PromptIntelDataset class in the module exports
tests/unit/datasets/test_promptintel_dataset.py Comprehensive test suite with 23 tests covering initialization, data transformation, pagination, errors, and filters

@anandansundar anandansundar reopened this Feb 25, 2026
anandansundar pushed a commit to anandansundar/PyRIT that referenced this pull request Feb 25, 2026
@romanlutz romanlutz requested a review from Copilot February 25, 2026 19:27
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

anandansundar pushed a commit to anandansundar/PyRIT that referenced this pull request Feb 25, 2026
Adds _PromptIntelDataset class that fetches prompts from the PromptIntel API and transforms them into PyRIT SeedDataset format (SeedPrompt + SeedObjective pairs grouped by prompt ID). Supports pagination, severity/category/search filters, and max_prompts limit. Includes 23 unit tests.
@anandansundar anandansundar force-pushed the feature/promptintel-dataset-loader branch from 761a66b to b75f34f Compare February 25, 2026 22:41
Copy link
Contributor

@romanlutz romanlutz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Just want to try it out myself before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants