diff --git a/index.toml b/index.toml index 2473a6f1..565560ad 100644 --- a/index.toml +++ b/index.toml @@ -260,4 +260,15 @@ completion_time = "20 min" created_at = 2026-03-30 dependencies = ["haystack-ai", "turboquant-vllm", "transformers-haystack"] featured = false -python_version = "3.12" \ No newline at end of file +python_version = "3.12" + +[[tutorial]] +title = "Securing Pipelines Against Runaway Costs and PII Leaks" +description = "Add deterministic governance to Haystack pipelines using the tealtiger-haystack integration to prevent PII leaks, secret exposure, and runaway LLM costs" +level = "intermediate" +weight = 13 +notebook = "50_Securing_Pipelines_Against_Runaway_Costs_and_PII_Leaks.ipynb" +aliases = [] +completion_time = "20 min" +created_at = 2026-06-17 +dependencies = ["tealtiger-haystack"] diff --git a/tutorials/50_Securing_Pipelines_Against_Runaway_Costs_and_PII_Leaks.ipynb b/tutorials/50_Securing_Pipelines_Against_Runaway_Costs_and_PII_Leaks.ipynb new file mode 100644 index 00000000..0e21fc97 --- /dev/null +++ b/tutorials/50_Securing_Pipelines_Against_Runaway_Costs_and_PII_Leaks.ipynb @@ -0,0 +1,388 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Tutorial: Secure Haystack Pipelines Against Runaway Costs and PII Leaks\n", + "\n", + "- **Level**: Intermediate\n", + "- **Time to complete**: 20 min\n", + "- **Components Used**: [`OpenAIChatGenerator`](https://docs.haystack.deepset.ai/docs/openaichatgenerator), [`Pipeline`](https://docs.haystack.deepset.ai/docs/pipeline), [`TealTigerGovernanceComponent`](https://haystack.deepset.ai/integrations/tealtiger)\n", + "- **Prerequisites**: OpenAI API key\n", + "- **Goal**: After completing this tutorial, you will have learned how to add deterministic governance to Haystack pipelines using the `tealtiger-haystack` integration, preventing PII leaks, secret exposure, and runaway LLM costs — all without adding another LLM to the governance path." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Overview\n", + "\n", + "Once pipelines can query databases and call tools, new risks appear:\n", + "- **PII flowing to the model** — customer SSNs, credit card numbers in retrieved documents\n", + "- **Secrets in generated code** — API keys, passwords in context windows\n", + "- **Cost runaway** — infinite agent loops burning through budget\n", + "\n", + "This tutorial shows how to prevent all three using the [`tealtiger-haystack`](https://pypi.org/project/tealtiger-haystack/) integration — a deterministic governance component that adds <2ms overhead and requires no additional LLM calls.\n", + "\n", + "We'll use the official `TealTigerGovernanceComponent` which plugs directly into Haystack pipelines as a first-class component." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Setup" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%%bash\n", + "pip install tealtiger-haystack haystack-ai" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from getpass import getpass\n", + "\n", + "if \"OPENAI_API_KEY\" not in os.environ:\n", + " os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OpenAI API key: \")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Part 1: Zero-Config Observe Mode\n", + "\n", + "The simplest way to add governance is **observe mode** — it allows everything through but tracks costs, detects PII, and produces structured audit entries. No policies needed." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from haystack import Pipeline\n", + "from haystack.components.generators.chat import OpenAIChatGenerator\n", + "from haystack.components.builders import ChatPromptBuilder\n", + "from haystack.dataclasses import ChatMessage\n", + "from haystack_integrations.components.connectors.tealtiger import TealTigerGovernanceComponent\n", + "\n", + "# Create a pipeline with TealTiger governance in OBSERVE mode (default)\n", + "observe_pipeline = Pipeline()\n", + "\n", + "# Add the governance component — zero config, just add it\n", + "observe_pipeline.add_component(\"governance\", TealTigerGovernanceComponent())\n", + "observe_pipeline.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", + "\n", + "# Connect governance output to LLM input\n", + "observe_pipeline.connect(\"governance.text\", \"llm.prompt\")\n", + "\n", + "print(\"Pipeline with TealTiger governance (OBSERVE mode) created.\")\n", + "print(\"All requests pass through, but cost + PII are tracked automatically.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Testing Observe Mode — Clean Input\n", + "\n", + "A normal query with no PII passes through and gets tracked." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "result = observe_pipeline.run({\n", + " \"governance\": {\"text\": \"What was Apple's Q4 2024 revenue?\"}\n", + "})\n", + "\n", + "# The governance decision shows what was detected\n", + "decision = result[\"governance\"][\"decision\"]\n", + "print(f\"Action: {decision['action']}\")\n", + "print(f\"PII detected: {decision['pii_detected']}\")\n", + "print(f\"Cost tracked: ${decision['cost_tracked']:.6f}\")\n", + "print(f\"Evaluation time: {decision['evaluation_time_ms']:.2f}ms\")\n", + "print(f\"Correlation ID: {decision['correlation_id']}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Testing Observe Mode — Input with PII\n", + "\n", + "Even with PII present, observe mode **allows the request through** but records what was found." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "result = observe_pipeline.run({\n", + " \"governance\": {\n", + " \"text\": \"Customer John Smith, SSN: 000-00-0000, has a balance of $4,200. Summarize.\"\n", + " }\n", + "})\n", + "\n", + "decision = result[\"governance\"][\"decision\"]\n", + "print(f\"Action: {decision['action']}\") # Still ALLOW in observe mode\n", + "print(f\"PII detected: {decision['pii_detected']}\") # Shows SSN was found\n", + "print(f\"Cumulative cost: ${decision['cumulative_cost']:.6f}\")\n", + "print(\"\\nNote: In OBSERVE mode, PII is reported but never blocked.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Part 2: Enforce Mode — Blocking PII and Secrets\n", + "\n", + "Now let's switch to **ENFORCE mode** with policies. This will actively block requests containing PII or secrets before they reach the LLM." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from tealtiger import TealEngine\n", + "from haystack_integrations.components.connectors.tealtiger import TealTigerGovernanceComponent\n", + "from haystack_integrations.components.connectors.tealtiger.governance_component import GovernanceDenyError\n", + "\n", + "# Configure the governance engine with policies\n", + "engine = TealEngine(policies=[\n", + " {\"type\": \"pii_block\", \"categories\": [\"ssn\", \"credit_card\"]},\n", + " {\"type\": \"cost_limit\", \"max_per_session\": 5.00},\n", + "])\n", + "\n", + "# Create an enforcing pipeline\n", + "enforce_pipeline = Pipeline()\n", + "enforce_pipeline.add_component(\n", + " \"governance\",\n", + " TealTigerGovernanceComponent(engine=engine, mode=\"ENFORCE\", raise_on_deny=True),\n", + ")\n", + "enforce_pipeline.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", + "enforce_pipeline.connect(\"governance.text\", \"llm.prompt\")\n", + "\n", + "print(\"Enforcing pipeline created with PII blocking + $5 cost limit.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test: Clean Input Passes Through" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "result = enforce_pipeline.run({\n", + " \"governance\": {\"text\": \"What is the capital of France?\"}\n", + "})\n", + "\n", + "decision = result[\"governance\"][\"decision\"]\n", + "print(f\"Action: {decision['action']}\") # ALLOW\n", + "print(f\"PII detected: {decision['pii_detected']}\") # Empty\n", + "print(\"Clean input passed governance successfully.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test: PII is Blocked Before Reaching the LLM" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "try:\n", + " result = enforce_pipeline.run({\n", + " \"governance\": {\n", + " \"text\": \"Customer SSN: 000-00-0000, credit card: 4111-1111-1111-1111. Process refund.\"\n", + " }\n", + " })\n", + " print(\"ERROR: Should have been blocked!\")\n", + "except GovernanceDenyError as e:\n", + " print(f\"BLOCKED by governance.\")\n", + " print(f\"Reason: {e.decision['reason']}\")\n", + " print(f\"Reason codes: {e.decision['reason_codes']}\")\n", + " print(f\"\\nThe LLM never saw this data — blocked at the governance layer.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Test: Secrets are Blocked" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "try:\n", + " result = enforce_pipeline.run({\n", + " \"governance\": {\n", + " \"text\": \"The database uses password: EXAMPLE_SECRET_TOKEN_12345 and connects to db.internal.example.com.\"\n", + " }\n", + " })\n", + " print(\"ERROR: Should have been blocked!\")\n", + "except GovernanceDenyError as e:\n", + " print(f\"BLOCKED: {e.decision['reason']}\")\n", + " print(f\"Secrets never reach the LLM context window.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Part 3: Cost Tracking and Budget Enforcement\n", + "\n", + "TealTiger tracks cumulative cost per session. When the budget limit is exceeded, requests are blocked before incurring more spend." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create a pipeline with a very low budget to demonstrate cost blocking\n", + "tight_budget_engine = TealEngine(policies=[\n", + " {\"type\": \"cost_limit\", \"max_per_session\": 0.001}, # $0.001 budget\n", + "])\n", + "\n", + "budget_pipeline = Pipeline()\n", + "budget_pipeline.add_component(\n", + " \"governance\",\n", + " TealTigerGovernanceComponent(engine=tight_budget_engine, mode=\"ENFORCE\"),\n", + ")\n", + "budget_pipeline.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", + "budget_pipeline.connect(\"governance.text\", \"llm.prompt\")\n", + "\n", + "# First request may pass (under budget)\n", + "result = budget_pipeline.run({\"governance\": {\"text\": \"Hello\"}})\n", + "print(f\"Request 1 — Cost so far: ${result['governance']['decision']['cumulative_cost']:.6f}\")\n", + "\n", + "# Subsequent requests will be blocked once budget is exhausted\n", + "try:\n", + " for i in range(5):\n", + " result = budget_pipeline.run({\"governance\": {\"text\": f\"Request {i+2}\"}})\n", + "except GovernanceDenyError as e:\n", + " print(f\"\\nBUDGET EXCEEDED — blocked at request.\")\n", + " print(f\"Reason: {e.decision['reason']}\")\n", + " print(\"\\nThis prevents infinite agent loops from burning through your budget.\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Part 4: Inspecting the Audit Trail\n", + "\n", + "Every governance evaluation produces a structured audit entry with correlation IDs, timestamps, and findings — useful for compliance and debugging." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Run a few requests through observe mode to build up audit history\n", + "audit_pipeline = Pipeline()\n", + "governance_component = TealTigerGovernanceComponent()\n", + "audit_pipeline.add_component(\"governance\", governance_component)\n", + "audit_pipeline.add_component(\"llm\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\n", + "audit_pipeline.connect(\"governance.text\", \"llm.prompt\")\n", + "\n", + "# Process several requests\n", + "test_inputs = [\n", + " \"What is machine learning?\",\n", + " \"Customer email: test@example.com needs help with billing.\",\n", + " \"Normal technical question about Python async.\",\n", + "]\n", + "\n", + "for text in test_inputs:\n", + " result = audit_pipeline.run({\"governance\": {\"text\": text}})\n", + "\n", + "# Get the full audit trail\n", + "audit_trail = governance_component.get_decisions()\n", + "\n", + "print(f\"Total governance evaluations: {len(audit_trail)}\")\n", + "print(\"\\n--- Audit Trail ---\")\n", + "for i, entry in enumerate(audit_trail):\n", + " print(f\"\\n[{i+1}] Correlation ID: {entry['correlation_id'][:8]}...\")\n", + " print(f\" Action: {entry['action']}\")\n", + " print(f\" PII found: {len(entry['pii_detected'])} items\")\n", + " print(f\" Cost: ${entry['cost_tracked']:.6f}\")\n", + " print(f\" Eval time: {entry['evaluation_time_ms']:.2f}ms\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "In this tutorial, you learned how to:\n", + "\n", + "1. **Add zero-config governance** using `TealTigerGovernanceComponent()` in OBSERVE mode\n", + "2. **Block PII and secrets** before they reach the LLM using ENFORCE mode with policies\n", + "3. **Enforce cost budgets** to prevent runaway agent loops\n", + "4. **Inspect structured audit trails** with correlation IDs for compliance\n", + "\n", + "All governance is **deterministic** (no LLM in the decision path), adds **<2ms latency**, and works entirely **in-process** with no external service dependencies.\n", + "\n", + "### Next Steps\n", + "\n", + "- [TealTiger Haystack Integration](https://haystack.deepset.ai/integrations/tealtiger) — Full integration docs\n", + "- [tealtiger-haystack on PyPI](https://pypi.org/project/tealtiger-haystack/) — Package reference\n", + "- [TealTiger GitHub](https://github.com/agentguard-ai/tealtiger) — Source code and examples" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "name": "python", + "version": "3.11.0" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +}