Is your feature request related to a problem? Please describe.
PyRIT does not currently expose a paired prompt-injection dataset designed to test the boundary between legitimate agent workflows and adversarial instructions. Attack-only collections are useful for recall, but they do not let users build or inspect false-positive-aware test surfaces where benign and attack prompts intentionally share the same scenario and vocabulary.
Describe the solution you'd like
Add a remote loader for the CC BY 4.0 Agentic Prompt-Injection Boundary Pairs dataset.
The proposed loader would:
- load the published train, validation, test, or combined surface;
- default to attack examples for red-team use while allowing benign or paired loading;
- preserve
pair_id, scenario_id, attack_family, pair_family, source_context, risk_domain, target_boundary, split, and label metadata;
- support typed filters for label, split, attack family, and source context;
- keep scenario-isolated split semantics intact;
- include mocked unit tests, documentation, and bibliography updates.
The dataset contains 1,200 English records organized as 600 controlled benign/attack pairs across 50 enterprise and agentic scenarios and 12 attack families. JSONL and Parquet artifacts, deterministic build scripts, validation, checksums, and a 30% stratified manual-review record are public.
Describe alternatives you've considered, if relevant
Users can download the JSONL files directly, but that bypasses PyRIT dataset discovery, metadata preservation, filtering, and consistent SeedDataset handling. A generic file loader would also lose the paired-boundary semantics unless every consumer rebuilt the same mapping.
Additional context
Dataset: https://huggingface.co/datasets/3nesdeniz/agentic-prompt-injection-boundary-pairs
Source and build pipeline: https://github.com/3nesdeniz/agentic-prompt-injection-boundary-pairs
I maintain the dataset and can keep the loader schema synchronized with tagged dataset releases.
Is your feature request related to a problem? Please describe.
PyRIT does not currently expose a paired prompt-injection dataset designed to test the boundary between legitimate agent workflows and adversarial instructions. Attack-only collections are useful for recall, but they do not let users build or inspect false-positive-aware test surfaces where benign and attack prompts intentionally share the same scenario and vocabulary.
Describe the solution you'd like
Add a remote loader for the CC BY 4.0 Agentic Prompt-Injection Boundary Pairs dataset.
The proposed loader would:
pair_id,scenario_id,attack_family,pair_family,source_context,risk_domain,target_boundary,split, and label metadata;The dataset contains 1,200 English records organized as 600 controlled benign/attack pairs across 50 enterprise and agentic scenarios and 12 attack families. JSONL and Parquet artifacts, deterministic build scripts, validation, checksums, and a 30% stratified manual-review record are public.
Describe alternatives you've considered, if relevant
Users can download the JSONL files directly, but that bypasses PyRIT dataset discovery, metadata preservation, filtering, and consistent
SeedDatasethandling. A generic file loader would also lose the paired-boundary semantics unless every consumer rebuilt the same mapping.Additional context
Dataset: https://huggingface.co/datasets/3nesdeniz/agentic-prompt-injection-boundary-pairs
Source and build pipeline: https://github.com/3nesdeniz/agentic-prompt-injection-boundary-pairs
I maintain the dataset and can keep the loader schema synchronized with tagged dataset releases.