Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 28 additions & 8 deletions docs-mintlify/admin/connect-to-data/data-sources/ksqldb.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -338,10 +338,11 @@ cubes:
time_dimension: CUBE.created_at
granularity: second
partition_granularity: day
# ksqlDB does not support NOW(); use CURRENT_TIMESTAMP with INTERVAL arithmetic instead.
build_range_start:
sql: "SELECT date_trunc('day', DATE_SUB(NOW(), INTERVAL '5 hour'))"
sql: "SELECT date_trunc('day', CURRENT_TIMESTAMP - INTERVAL '5 hour')"
build_range_end:
sql: "SELECT DATE_ADD(NOW(), INTERVAL '15 minute')"
sql: "SELECT CURRENT_TIMESTAMP + INTERVAL '15 minute'"
refresh_key:
every: 1 minute
update_window: 1 hour
Expand Down Expand Up @@ -521,11 +522,12 @@ cube("order_events_stream", {
time_dimension: CUBE.created_at,
granularity: `second`,
partition_granularity: `day`,
// ksqlDB does not support NOW(); use CURRENT_TIMESTAMP with INTERVAL arithmetic instead.
build_range_start: {
sql: `SELECT date_trunc('day', DATE_SUB(NOW(), INTERVAL '5 hour'))`,
sql: `SELECT date_trunc('day', CURRENT_TIMESTAMP - INTERVAL '5 hour')`,
},
build_range_end: {
sql: `SELECT DATE_ADD(NOW(), INTERVAL '15 minute')`,
sql: `SELECT CURRENT_TIMESTAMP + INTERVAL '15 minute'`,
},
refresh_key: {
every: `1 minute`,
Expand Down Expand Up @@ -615,6 +617,22 @@ with:

<Warning>

**The ksqlDB stream (or table) name and the backing Kafka topic name
MUST be identical, including case.** Kafka streams mode will fail if
they differ in any way.

Take this into account when creating the stream — explicitly set the
topic name to match the stream name (and vice versa). For example:

```sql
CREATE STREAM ORDER_EVENTS_STREAM (...)
WITH (KAFKA_TOPIC='ORDER_EVENTS_STREAM', VALUE_FORMAT='JSON', ...);
```

The match is **case-sensitive**: `OrderEvents` and `ORDEREVENTS` are
treated as different names and will cause the build to fail with
`Topic table ... is not found`.

This is a known limitation of Kafka streams mode. It does not occur
when the ksqlDB object name and the Kafka topic name are the same,
which is the default behavior when ksqlDB creates a stream or table
Expand Down Expand Up @@ -707,10 +725,11 @@ pre_aggregations:
time_dimension: CUBE.created_at
granularity: second
partition_granularity: day
# ksqlDB does not support NOW(); use CURRENT_TIMESTAMP with INTERVAL arithmetic instead.
build_range_start:
sql: "SELECT date_trunc('day', DATE_SUB(NOW(), INTERVAL '5 hour'))"
sql: "SELECT date_trunc('day', CURRENT_TIMESTAMP - INTERVAL '5 hour')"
build_range_end:
sql: "SELECT DATE_ADD(NOW(), INTERVAL '15 minute')"
sql: "SELECT CURRENT_TIMESTAMP + INTERVAL '15 minute'"
refresh_key:
every: 1 minute
update_window: 1 hour
Expand Down Expand Up @@ -744,11 +763,12 @@ pre_aggregations: {
time_dimension: CUBE.created_at,
granularity: `second`,
partition_granularity: `day`,
// ksqlDB does not support NOW(); use CURRENT_TIMESTAMP with INTERVAL arithmetic instead.
build_range_start: {
sql: `SELECT date_trunc('day', DATE_SUB(NOW(), INTERVAL '5 hour'))`,
sql: `SELECT date_trunc('day', CURRENT_TIMESTAMP - INTERVAL '5 hour')`,
},
build_range_end: {
sql: `SELECT DATE_ADD(NOW(), INTERVAL '15 minute')`,
sql: `SELECT CURRENT_TIMESTAMP + INTERVAL '15 minute'`,
},
refresh_key: {
every: `1 minute`,
Expand Down
76 changes: 76 additions & 0 deletions docs-mintlify/admin/connect-to-data/data-sources/presto.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,82 @@ To enable SSL-encrypted connections between Cube and Presto, set the
configure custom certificates, please check out [Enable SSL Connections to the
Database][ref-recipe-enable-ssl].

## Custom headers

The Presto driver supports forwarding custom HTTP headers on every request to
the Presto coordinator. This is useful for setting headers like
`X-Presto-Source`, `X-Presto-Client-Tags`, or any other custom header your
Presto coordinator expects.

Custom headers can't be configured via environment variables. Instead, use the
[`driver_factory`](/reference/configuration/config#driver_factory) configuration
option to pass a `headers` object to the driver:

<CodeGroup>

```python title="Python"
from cube import config

@config('driver_factory')
def driver_factory(ctx: dict) -> dict:
return {
'type': 'prestodb',
'headers': {
'X-Presto-Source': 'cube',
'X-Presto-Client-Tags': 'user=alice@example.com'
}
}
```

```javascript title="JavaScript"
module.exports = {
driverFactory: ({ dataSource }) => ({
type: "prestodb",
headers: {
"X-Presto-Source": "cube",
"X-Presto-Client-Tags": "user=alice@example.com"
}
})
};
```

</CodeGroup>

In multitenant deployments, you can use the [security
context](/embedding/authentication/security-context) to pass per-tenant headers,
for example to forward a user token from the API request down to Presto:

<CodeGroup>

```python title="Python"
from cube import config

@config('driver_factory')
def driver_factory(ctx: dict) -> dict:
security_context = ctx['securityContext']
return {
'type': 'prestodb',
'headers': {
'X-Presto-Client-Tags': f"user={security_context['user_id']}",
'X-Custom-User-Token': security_context['token']
}
}
```

```javascript title="JavaScript"
module.exports = {
driverFactory: ({ securityContext }) => ({
type: "prestodb",
headers: {
"X-Presto-Client-Tags": `user=${securityContext.user_id}`,
"X-Custom-User-Token": securityContext.token
}
})
};
```

</CodeGroup>

[aws-s3]: https://aws.amazon.com/s3/
[google-cloud-storage]: https://cloud.google.com/storage
[presto]: https://prestodb.io/
Expand Down
80 changes: 80 additions & 0 deletions docs-mintlify/admin/connect-to-data/data-sources/trino.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,86 @@ To enable SSL-encrypted connections between Cube and Trino, set the
configure custom certificates, please check out [Enable SSL Connections to the
Database][ref-recipe-enable-ssl].

## Custom headers

The Trino driver supports forwarding custom HTTP headers (e.g., `X-Trino-Source`,
`X-Trino-Routing-Group`, `X-Trino-Client-Tags`, or any other custom header) on
every request to the Trino coordinator. See the [Trino client
protocol][trino-docs-client-protocol] for the list of headers accepted by Trino.

Custom headers can't be configured via environment variables. Instead, use the
[`driver_factory`](/reference/configuration/config#driver_factory) configuration
option to pass a `headers` object to the driver:

<CodeGroup>

```python title="Python"
from cube import config

@config('driver_factory')
def driver_factory(ctx: dict) -> dict:
return {
'type': 'trino',
'headers': {
'X-Trino-Source': 'cube',
'X-Trino-Routing-Group': 'etl',
'X-Trino-Client-Tags': 'user=alice@example.com'
}
}
```

```javascript title="JavaScript"
module.exports = {
driverFactory: ({ dataSource }) => ({
type: "trino",
headers: {
"X-Trino-Source": "cube",
"X-Trino-Routing-Group": "etl",
"X-Trino-Client-Tags": "user=alice@example.com"
}
})
};
```

</CodeGroup>

In multitenant deployments, you can use the [security
context](/embedding/authentication/security-context) to pass per-tenant headers, for example
to forward a user token from the API request down to Trino:

<CodeGroup>

```python title="Python"
from cube import config

@config('driver_factory')
def driver_factory(ctx: dict) -> dict:
security_context = ctx['securityContext']
return {
'type': 'trino',
'headers': {
'X-Trino-Client-Tags': f"user={security_context['user_id']}",
'X-Custom-User-Token': security_context['token']
}
}
```

```javascript title="JavaScript"
module.exports = {
driverFactory: ({ securityContext }) => ({
type: "trino",
headers: {
"X-Trino-Client-Tags": `user=${securityContext.user_id}`,
"X-Custom-User-Token": securityContext.token
}
})
};
```

</CodeGroup>

[trino-docs-client-protocol]: https://trino.io/docs/current/develop/client-protocol.html

[aws-s3]: https://aws.amazon.com/s3/
[google-cloud-storage]: https://cloud.google.com/storage
[ref-caching-using-preaggs-build-strats]: /docs/pre-aggregations/using-pre-aggregations#pre-aggregation-build-strategies
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ auth providers, BI APIs targeted by SLS, and other upstream services), see
With Dedicated Infrastructure and Bring Your Own Cloud on AWS, Cube supports
establishing **AWS PrivateLink** connections from your AWS accounts to the Cube
API endpoints. This lets your applications, internal BI tools, and end-user
browsers reach the Cube HTTP and SQL APIs entirely over private AWS networking
— never touching the public internet. When private connectivity is in place,
the public API endpoints can be disabled completely on request.
browsers reach the Cube HTTP, SQL, and AI APIs entirely over private AWS
networking — never touching the public internet. When private connectivity is
in place, the public API endpoints can be disabled completely on request.

<Note>

Expand All @@ -41,10 +41,12 @@ Cube runs as two cooperating planes:
private connectivity — it is a SaaS UI like any other.
- The **data plane** runs your Cube deployments and serves all Cube HTTP and
SQL **data API** traffic — REST/GraphQL queries from your applications,
live data calls issued by the Cube UI while rendering charts, and SQL
connections from BI tools. The data plane is the part that talks to your
databases and returns query results, and the part this document teaches you
how to expose privately.
live data calls issued by the Cube UI while rendering charts, SQL
connections from BI tools, and AI traffic to the [Chat API][chat-api] and
other AI endpoints (including data-plane-hosted AI Engineer agents and
external agentic clients [calling the Chat API as a tool][agent-to-agent]).
The data plane is the part that talks to your databases and returns query
results, and the part this document teaches you how to expose privately.

[AWS PrivateLink][aws-privatelink] lets a service running in one VPC be
consumed from another VPC over the AWS internal network, without VPC peering
Expand Down Expand Up @@ -330,6 +332,8 @@ the VPC's CIDR.
[cube-region]: /admin/deployment/infrastructure#understanding-cube-cloud-region
[aws-private-link]: /admin/deployment/dedicated/aws/private-link
[aws-vpc-peering]: /admin/deployment/dedicated/aws/vpc-peering
[chat-api]: /reference/embed-apis/chat-api
[agent-to-agent]: /recipes/ai/agent-to-agent
[aws-privatelink]: https://docs.aws.amazon.com/vpc/latest/privatelink/what-is-privatelink.html
[aws-endpoint-service]: https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html
[aws-interface-endpoint]: https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html
Expand Down
16 changes: 8 additions & 8 deletions packages/cubejs-backend-native/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion rust/cube/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
target
target
.zed
Loading