diff --git a/docs/configuration/pgdog.toml/general.md b/docs/configuration/pgdog.toml/general.md index 21aae155..f0bfc2e8 100644 --- a/docs/configuration/pgdog.toml/general.md +++ b/docs/configuration/pgdog.toml/general.md @@ -175,6 +175,18 @@ that servers don't stay blocked forever due to healthcheck false positives. Default: **`300_000`** (5 minutes) +### `ban_replica_lag` + +Ban a replica from serving read queries if its replication lag (in milliseconds) exceeds this threshold. See [replication failover](../../features/load-balancer/replication-failover.md). + +Default: **disabled** (no time-based lag ban) + +### `ban_replica_lag_bytes` + +Ban a replica from serving read queries if its replication lag (in bytes) exceeds this threshold. + +Default: **disabled** (no byte-based lag ban) + ### `shutdown_timeout` How long to wait for active clients to finish transactions when shutting down. This ensures that PgDog redeployments disrupt as few @@ -391,6 +403,18 @@ Enable the query parser in single-shard deployments and record its decisions. Ca Default: **`false`** (disabled) +### `cross_shard_disabled` + +Disable cross-shard queries globally. When enabled, queries touching more than one shard are rejected. + +Default: **`false`** (cross-shard queries allowed) + +### `expanded_explain` + +When enabled, PgDog appends its own routing annotations to `EXPLAIN` output, showing how the query was routed (e.g., which sharding key matched, which parameter supplied the value, or whether the query was broadcast to all shards). + +Default: **`false`** (disabled) + ### `two_phase_commit` Enable [two-phase commit](../../features/sharding/2pc.md) for write, cross-shard transactions. @@ -403,6 +427,33 @@ Enable automatic conversion of single-statement write transactions to use [two-p Default: **`true`** (enabled) +### `two_phase_commit_wal_dir` + +Directory where the [two-phase commit](../../features/sharding/2pc.md) write-ahead log is stored. + +!!! note "Requires restart" + This setting cannot be changed at runtime. + +Default: **`./pgdog_wal`** + +### `two_phase_commit_wal_segment_size` + +Maximum size, in bytes, of a single 2pc WAL segment file before it is rotated. + +Default: **`16_777_216`** (16 MiB) + +### `two_phase_commit_wal_fsync_interval` + +How long, in milliseconds, the 2pc WAL writer waits to coalesce concurrent appends into a single fsync. Setting this to `0` disables waiting for additional appends; records already queued in the channel when the writer wakes are still batched into one fsync. Higher values trade per-transaction commit latency for fewer fsyncs under load. + +Default: **`2`** (2ms) + +### `two_phase_commit_wal_checkpoint_interval` + +How often, in seconds, to write a checkpoint record to the 2pc WAL and garbage-collect old segments. + +Default: **`60`** (60s) + ### `query_cache_limit` Limit on the number of statements saved in the statement cache used to accelerate query parsing. The saved statements are visible by running the `SHOW QUERY_CACHE` in the [admin database](../../administration/index.md). @@ -428,6 +479,23 @@ Available options: Default: **`auto`** +### `query_parser_engine` + +Underlying parser implementation used to analyze SQL queries. + +Available options: + +- `pg_query_protobuf` (default): the standard [`pg_query`](https://github.com/pganalyze/pg_query) parser +- `pg_query_raw` (experimental): native Rust bindings to PostgreSQL's parser, skipping the protobuf serialization layer. Requires a larger thread stack. PgDog will automatically raise [`memory.stack_size`](memory.md) to at least 32 MiB per Tokio worker when this engine is selected. + +Default: **`pg_query_protobuf`** + +### `regex_parser_limit` + +Maximum size, in bytes, of a query that the regex pre-parser will inspect before deferring to the full parser. Larger queries skip the regex fast path. + +Default: **`1_000`** + ### `system_catalogs` Changes how system catalog tables (like `pg_database`, `pg_class`, etc.) are treated by the query router. Default behavior is to assume they are the same on all shards and send queries referencing them to a random shard. This makes tools like `psql` work out of the box. @@ -457,6 +525,24 @@ Available options: `text` format is required when migrating from `INTEGER` to `BIGINT` primary keys during resharding. +### `resharding_parallel_copies` + +How many parallel `COPY` workers to launch during [resharding](../../features/sharding/resharding/index.md), irrespective of the number of available replicas. + +Default: **`1`** + +### `resharding_copy_retry_max_attempts` + +Maximum number of retries for a failed table copy during resharding (per-table). Retries use exponential backoff starting at [`resharding_copy_retry_min_delay`](#resharding_copy_retry_min_delay), doubling each attempt and capped at 32×. + +Default: **`5`** + +### `resharding_copy_retry_min_delay` + +Base delay, in milliseconds, between table copy retries during resharding. Each successive attempt doubles the delay, capped at 32×. + +Default: **`1_000`** (1s) + ### `reload_schema_on_ddl` !!! warning @@ -522,6 +608,23 @@ Default: **`abort`** ## Logging +### `log_format` + +Format to use for PgDog application logs. + +Available options: + +- `text` (default): human-readable text logs +- `json`: structured JSON logs suitable for ECS/Datadog ingestion + +Default: **`text`** + +### `log_level` + +Log filter directives using the same syntax as the `RUST_LOG` environment variable (e.g. `info`, `debug`, `pgdog=debug,hyper=warn`). + +Default: **`info`** + ### `log_connections` If enabled, log every time a user creates a new connection to PgDog. @@ -534,6 +637,18 @@ If enabled, log every time a user disconnects from PgDog. Default: **`true`** (enabled) +### `log_dedup_window` + +Window, in milliseconds, over which to deduplicate identical log messages. Identical messages (same level, target, and body) that exceed [`log_dedup_threshold`](#log_dedup_threshold) within this window are suppressed and replaced with a single summary line at the end of the window. Set to `0` to disable throttling. + +Default: **`0`** (disabled) + +### `log_dedup_threshold` + +Number of identical log messages allowed within [`log_dedup_window`](#log_dedup_window) before further duplicates are suppressed. Set to `0` to disable throttling. + +Default: **`0`** (disabled) + ## Statistics ### `stats_period` @@ -569,3 +684,22 @@ Default: **`5_000`** (5s) Maximum amount of time allowed for the [replication delay](../../features/load-balancer/replication-failover.md) query to return a result. Default: **`5_000`** (5s) + +## Unique IDs + +### `unique_id_function` + +Function used by PgDog to generate unique IDs. + +Available options: + +- `standard` (default): 64-bit function using the entire 64-bit range +- `compact`: 53-bit function whose output fits in a JavaScript-safe integer + +Default: **`standard`** + +### `unique_id_min` + +Minimum value returned by the unique ID generator. Useful for reserving a range of low IDs for fixtures or system rows. + +Default: **`0`** diff --git a/docs/configuration/pgdog.toml/otel.md b/docs/configuration/pgdog.toml/otel.md new file mode 100644 index 00000000..c5eaae5b --- /dev/null +++ b/docs/configuration/pgdog.toml/otel.md @@ -0,0 +1,84 @@ +--- +icon: material/chart-line +--- + +# OTEL + +OTEL is a standard for sending telemetry from applications to backends that support it, like Grafana or Datadog. PgDog supports OTEL out of the box: + +=== "pgdog.toml" + ```toml + [otel] + datadog_api_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + endpoint = "https://otlp.us5.datadoghq.com/v1/metrics" + ``` +=== "Helm chart" + ```yaml + otel: + datadogApiKey: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + endpoint: "https://otlp.us5.datadoghq.com/v1/metrics" + ``` + +### `endpoint` + +Endpoint where PgDog will push OTEL metrics. + +### `datadog_api_key` + +If using Datadog, configuring the API key here will automatically set +the authentication header. + +To obtain a Datadog API key, go to your Organization settings / API keys. + +### `namespace` + +By default, all metrics will be sent under the `pgdog` namespace, e.g.: `pgdog.sv_idle`. + +### `headers` + +HTTP headers sent with each OTLP push request. Use this to configure authentication or any custom +headers required by your OTLP backend. + +In `pgdog.toml`, headers are configured as a sub-table: + +=== "pgdog.toml" + ```toml + [otel.headers] + DD-API-KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + X-Custom = "foo" + ``` +=== "Helm chart" + ```yaml + otel: + headers: + DD-API-KEY: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" + X-Custom: "foo" + ``` + +Headers can also be set via the `OTEL_EXPORTER_OTLP_HEADERS` environment variable, as a +comma-separated list of `key=value` pairs. + +### `push_interval` + +How often, in milliseconds, to push metrics to the OTLP endpoint. + +_Default:_ `10000` + +## Environment variables + +PgDog honors the standard OpenTelemetry environment variables. When set, they override the +corresponding `pgdog.toml` values (or supply a value when the setting is omitted). + +| Variable | Description | +| --- | --- | +| `OTEL_EXPORTER_OTLP_ENDPOINT` | OTLP metrics ingest URL. Equivalent to `endpoint`. | +| `OTEL_EXPORTER_OTLP_HEADERS` | Comma-separated `key=value` pairs added to OTLP push requests. Equivalent to `[otel.headers]`. | +| `OTEL_METRIC_EXPORT_INTERVAL` | Push interval in milliseconds. Equivalent to `push_interval`. | +| `OTEL_SERVICE_NAME` | Sets the `service.name` resource attribute. Takes precedence over any value set via `OTEL_RESOURCE_ATTRIBUTES`. _Default:_ `pgdog`. | +| `OTEL_RESOURCE_ATTRIBUTES` | Comma-separated `key=value` pairs added as resource attributes on every metric (e.g. `env=prod,region=us-east-1`). Values may be percent-encoded. | +| `PGDOG_OTEL_NAMESPACE` | Metric name prefix. Equivalent to `namespace`. | +| `DD_API_KEY` | Datadog API key. Equivalent to `datadog_api_key`. | + +PgDog automatically sets the following resource attributes on every exported metric: +`service.name` (defaults to `pgdog`), `service.instance.id`, and `host.name`. Use +`OTEL_RESOURCE_ATTRIBUTES` to add or override any of these. diff --git a/docs/enterprise_edition/index.md b/docs/enterprise_edition/index.md index a8c85e43..cba6ef1f 100644 --- a/docs/enterprise_edition/index.md +++ b/docs/enterprise_edition/index.md @@ -65,7 +65,7 @@ If you're using our [Helm chart](../installation.md#kubernetes), you just need t ```yaml image: repository: ghcr.io/pgdogdev/pgdog-enterprise - tag: v2026-04-06 + tag: v2026-05-01 ``` For deploying the [control plane](control_plane/index.md), you have two options: diff --git a/docs/examples/control_plane/docker-compose.yaml b/docs/examples/control_plane/docker-compose.yaml index 1787e8b9..0b71baaa 100644 --- a/docs/examples/control_plane/docker-compose.yaml +++ b/docs/examples/control_plane/docker-compose.yaml @@ -24,7 +24,7 @@ services: - redis_data:/data migrate: - image: ghcr.io/pgdogdev/pgdog-enterprise/control:v2026-04-30 + image: ghcr.io/pgdogdev/pgdog-enterprise/control:v2026-05-01 environment: <<: *control-env command: control migrate @@ -33,7 +33,7 @@ services: condition: service_healthy init: - image: ghcr.io/pgdogdev/pgdog-enterprise/control:v2026-04-30 + image: ghcr.io/pgdogdev/pgdog-enterprise/control:v2026-05-01 environment: <<: *control-env command: control onboard --email demo@pgdog.dev --password demopass --token 644b527c-b9d6-4fb2-9861-703bad871ec0 --name Demo @@ -42,7 +42,7 @@ services: condition: service_completed_successfully control: - image: ghcr.io/pgdogdev/pgdog-enterprise/control:v2026-04-30 + image: ghcr.io/pgdogdev/pgdog-enterprise/control:v2026-05-01 ports: - "8099:8080" environment: @@ -55,7 +55,7 @@ services: condition: service_started pgdog: - image: ghcr.io/pgdogdev/pgdog-enterprise:v2026-04-30 + image: ghcr.io/pgdogdev/pgdog-enterprise:v2026-05-01 command: ["pgdog", "--config", "/etc/pgdog/pgdog.toml", "--users", "/etc/secrets/pgdog/users.toml"] ports: - "6432:6432" diff --git a/docs/features/metrics.md b/docs/features/metrics.md index 847dbf73..fbda6c94 100644 --- a/docs/features/metrics.md +++ b/docs/features/metrics.md @@ -71,7 +71,9 @@ OTEL is a standard for publishing metrics to compatible systems, like Grafana, P ### Configuration -OTEL export is disabled by default. To enable it, configure the collector endpoint and the necessary credentials. If you're using Datadog, you can set the API key as a separate setting: +OTEL export is disabled by default. To enable it, configure the collector endpoint and the necessary credentials. If you're using Datadog, you can set the API key as a separate setting. + +See [`[otel]` configuration](../configuration/pgdog.toml/otel.md) for the full list of settings, including custom headers, push interval, and supported `OTEL_*` environment variables. === "pgdog.toml" ```toml