Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/docs/concepts/catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,9 @@ CREATE CATALOG my_jdbc WITH (
'warehouse' = 'hdfs:///path/to/warehouse'
);
```

The JDBC catalog also persists Paimon views. View metadata is stored in an automatically created
`paimon_views` table, and table/view names share a single namespace per database (a name cannot be
used by both a table and a view at the same time). See [Views](./views) for details on the JDBC
catalog upgrade requirements, single-process locking semantics, and behavior of `DROP DATABASE`
against view-only databases.
34 changes: 33 additions & 1 deletion docs/docs/concepts/views.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,10 +36,11 @@ View metadata is persisted only when the catalog implementation supports it:
- **Hive metastore catalog** – view metadata is stored together with table metadata inside the
metastore warehouse.
- **REST catalog** – view metadata is kept in the REST backend and exposed through the catalog API.
- **JDBC catalog** – view metadata is stored in the catalog database in the `paimon_views`
metadata table. The table is created automatically when the JDBC catalog is initialized.

File-system catalogs do not currently support views because they lack persistent metadata storage.


### Representation structure

| Field | Type | Description |
Expand All @@ -50,13 +51,44 @@ File-system catalogs do not currently support views because they lack persistent
Multiple representations can be stored for the same version so that different engines can access the
view using their native dialect.

### JDBC catalog notes

The JDBC catalog stores views in a dedicated `paimon_views` table that is created on first
initialization. A few things are worth knowing when running on top of an existing JDBC catalog:

- **Required permissions on upgrade.** Upgrading to a Paimon release with view support requires
`CREATE TABLE` permission on the catalog database the first time the catalog is opened, so that
the `paimon_views` table can be created. Operators who tightened privileges to CRUD-only after
the initial deployment should either restore `CREATE TABLE` permission temporarily or create the
`paimon_views` table manually beforehand.
- **Table and view share the same identifier namespace.** A name cannot be used by both a table
and a view in the same database. `createTable`, `renameTable`, `createView` and `renameView` all
validate this invariant under the catalog lock; concurrent operations targeting the same
identifier will see exactly one winner.
- **Single-process atomicity does not depend on `lock.enabled`.** The JDBC catalog also keeps a
per-JVM stripe lock keyed by `(catalog key, database, object name)`, so the table-vs-view name
uniqueness invariant holds within one JVM even when `lock.enabled = false`. Setting
`lock.enabled = true` (with `lock.type = jdbc`) is still recommended for multi-process
deployments because the stripe lock only serializes operations within the same JVM.
- **Database visibility.** A database that contains only views (and no tables or properties) is
reported by `listDatabases` and `SHOW DATABASES`. `DROP DATABASE ... CASCADE` removes both the
tables and the views in that database; `DROP DATABASE` without `CASCADE` will reject databases
that still hold any view.
- **Cross-database rename.** `renameView(from, to)` and `renameTable(from, to)` raise an
`IllegalArgumentException` (`Database X does not exist.`) when the target database is missing,
matching the BadRequest semantics of the REST catalog.

## Operations

### Create or replace view

Use `CREATE VIEW` or `CREATE OR REPLACE VIEW` to register a view. Paimon assigns a UUID, writes the
first metadata file, and records version `1`.

The catalog stores the view definition. It does not resolve or validate referenced tables when the
view is created, so missing or cross-database table references are checked by the compute engine when
the view is queried.

```sql
CREATE VIEW sales_view AS
SELECT region, SUM(amount) AS total_amount
Expand Down
17 changes: 17 additions & 0 deletions docs/docs/flink/sql-ddl.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,23 @@ You can also perform logical isolation for databases under multiple catalogs by

Additionally, when creating a JdbcCatalog, you can specify the maximum length for the lock key by configuring "lock-key-max-length," which defaults to 255. Since this value is a combination of {catalog-key}.{database-name}.{table-name}, please adjust accordingly.

JDBC catalog supports persistent Paimon views. View metadata is stored in the automatically created
`paimon_views` table in the catalog database. The view SQL is stored as catalog metadata and is not
resolved when the view is created.

Within a single database, a name cannot be used by both a table and a view; all writers
(`createTable`, `renameTable`, `createView`, `renameView`) reject conflicting identifiers. See
[Views](../concepts/views#jdbc-catalog-notes) for upgrade-time permissions, single-process locking
semantics, and database visibility details.

```sql
CREATE VIEW sales_view AS SELECT name, amount FROM sales WHERE amount > 100;

SHOW VIEWS;

DROP VIEW sales_view;
```

You can define any default table options with the prefix `table-default.` for tables created in the catalog.

## Create Table
Expand Down
10 changes: 9 additions & 1 deletion docs/docs/spark/sql-ddl.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,14 @@ Paimon JDBC Catalog in Spark needs to correctly add the corresponding jar packag
| mysql | mysql-connector-java | [Download](https://mvnrepository.com/artifact/mysql/mysql-connector-java) |
| postgres | postgresql | [Download](https://mvnrepository.com/artifact/org.postgresql/postgresql) |

JDBC catalog supports persistent Paimon views. View metadata is stored in the automatically created
`paimon_views` table in the catalog database.

Within a single database, a name cannot be used by both a table and a view; all writers
(`createTable`, `renameTable`, `createView`, `renameView`) reject conflicting identifiers. See
[Views](../concepts/views#jdbc-catalog-notes) for upgrade-time permissions, single-process locking
semantics, and database visibility details.

```bash
spark-sql ... \
--conf spark.sql.catalog.paimon=org.apache.paimon.spark.SparkCatalog \
Expand Down Expand Up @@ -375,7 +383,7 @@ CREATE TABLE target_tbl LIKE source_tbl;
## View

Views are based on the result-set of an SQL query, when using `org.apache.paimon.spark.SparkCatalog`, views are managed by paimon itself.
And in this case, views are supported when the `metastore` type is `hive` or `rest`.
And in this case, views are supported when the `metastore` type is `hive`, `rest` or `jdbc`.

### Create Or Replace View

Expand Down
Loading
Loading