Starburst Delta Lake connector#

The Starburst Delta Lake connector is an extended version of the Delta Lake connector with configuration and usage identical.

The following improvements are included:

Requirements#

To connect to Databricks Delta Lake, you need:

Extensions#

The connector includes all the functionality described in the Delta Lake connector as well as these features:

Configuration#

The configuration of the Starburst Delta Lake connector is identical to the Delta Lake connector, with the addition of the following catalog session properties for the parquet writer:

Parquet catalog session properties#

Property name

Description

parquet_optimized_writer_enabled

Enables the experimental, native Parquet writer.

SQL support#

The connector supports all of the SQL statements listed in the Delta Lake connector documentation.

The following improvements are included:

SQL security#

You must set the delta.security property in your catalog properties file to sql-standard in order to use SQL security operation statements. See SQL standard based authorization for more information.

Performance#

The connector includes a number of performance improvements, detailed in the following sections:

Accelerated Parquet reader#

The connector includes a Parquet reader with numerous performance improvements for reading data files using the Parquet format.

The reader is enabled by default. You can disable it, and use the default reader from Trino in your catalog properties file:

delta.accelerated-parquet-reader.enabled=false

Alternatively, you can disable it for a specific user session with the catalog session property accelerated_parquet_reader_enabled.

Starburst Cached Views#

The connector supports table scan redirection to improve performance and reduce load on the data source.

Security#

The connector includes a number of security-related features, detailed in the following sections.

Authorization#

The connector supports standard Hive security for authorization under the delta.security configuration property. For more information, see the Delta Lake connector authorization configuration options.

Built-in access control#

If you have enabled built-in access control for SEP, you must add the following configuration to all Delta Lake catalogs:

delta.security=starburst