Release 435-e LTS (29 Feb 2024)#

Starburst Enterprise platform (SEP) 435-e LTS is the follow up release to the 435-e STS release and the 429-e LTS release.

This release is a promotion of the original 435-e STS release in February 2023 into a long term support (LTS) release.

The 435-e release includes all improvements from the following Trino releases:

It contains all improvements from Starburst Enterprise releases since the 429-e LTS release:

Highlights since 429-e#

Breaking changes#

  • This release includes some changes to how Ranger policies are evaluated that may require policy modifications. If you use Apache Ranger for role-based access control, read the migration guide.

  • This release removes RubiX, Apache Sentry, and Hive Alluxio from the Hive connector. If your existing deployment relies on these technologies and you wish to use the legacy Hive connector, contact Starburst Support for assistance.

  • The query.max-writer-tasks-count configuration property has been renamed to query.max-writer-task-count, and its corresponding session property max_writer_tasks_count renamed to max_writer_task_count. These properties must be renamed in your cluster configuration or the cluster fails to start.

  • The built-in access control REST API has been updated. Grants for functions are now created for all functions in a catalog (catalog.*), all functions in a catalog’s schema (catalog.schema.*), or for a specific function (catalog.schema.function). API requests that only describe the function name must be updated or validation fails with an error.

  • Older versions of the Trino CLI and JDBC driver now fail when attempting to run RESET/SET SESSION AUTHORIZATION statements. To use these statements, upgrade to the latest versions of these clients. As a best practice, always use the client version matching the SEP cluster version.

  • The hive.metastore-timeout catalog configuration property has been removed in favor of the hive.metastore.thrift.client.connect-timeout and hive.metastore.thrift.client.read-timeout properties. This property must be removed from your catalog configuration files to allow successful startup.

  • The experimental.late-materialization.enabled and experimental.work-processor-pipelines configuration properties have been removed. These properties must be removed from your cluster configuration to allow successful cluster startup.

  • This release introduces some security fixes in Trino that restrict users’ ability to create new Iceberg materialized views with Glue while using the iceberg.security=system catalog configuration setting. Querying new Iceberg materialized views that fit this criteria fail and return a Materialized view does not have an owner error. See Using BIAC with Iceberg materialized views for steps to mitigate this issue.

  • The optimizer.mark-distinct-strategy configuration property has been removed. You must remove this property from the cluster configuration in favor of the optimizer.distinct-aggregations-strategy configuration property or the cluster fails to start.

  • The internal-communication.shared-secret, managed-statistics.http-client.key-store-path, and managed-statistics.http-client.key-store-password catalog configuration properties are no longer required to enable managed statistics for a catalog. You must remove these properties from your catalog configuration files or the cluster fails to start. The internal-communication.shared-secret property is still valid as a general configuration property and can remain in non-catalog configuration files.

  • This release removed support for registering external tables with CREATE TABLE and the location table property. Use the register_table procedure instead. The delta.legacy-create-table-with-existing-location.enabled configuration property and corresponding legacy_create_table_with_existing_location_enabled session property are removed as well.

435-e initial changes#

General#

Security#

Hive connector#

  • Added public preview support for querying views created in Amazon Athena.

  • Fixed an issue with accessing security DEFINER views as INVOKER mode when {doc}}Hive metastore impersonation<object-storage/file-system-hdfs> is enabled.

Hudi connector#

  • Released as a general availability feature.

  • Added a configuration option to enforce that a filter on a partition key be present in the query.

Iceberg connector#

  • Fixed data loss on contended commit to a table under some circumstances.

MongoDB connector#

SingleStore connector#

435-e.1 changes (29 Feb 2024)#

  • Updated Kubernetes external secret operator.

  • Fixed query failure when invoking the json_table function.

  • Fixed query hang when a SQL routines dereferences a row field.

  • Fixed potential incorrect results when using the ST_Centroid() and ST_Buffer() functions for tiny geometries.

  • Fixed query failure when using char types with the reverse() function.

  • Fixed query failure when querying data with deeply nested rows.

  • Fixed showing stats (EXPLAIN ANALYZE or SHOW STATS) for security definer views on partitioned table, when HMS impersonation is enabled.

  • Fixed incorrect results for DATETIMEOFFSET values before the year 1400.

435-e.2 changes (11 Mar 2024)#

  • Fixed UI authentication for large authentication tokens.

  • Fixed access issue with security DEFINER views when both materialized views and HMS impersonation are enabled.

  • Fixed a bug where column statistics created in SEP would not be visible in Hive when using CDP 7.

435-e.3 changes (28 Mar 2024)#

  • Added support for setting endpoint and region in STS clients in Lake Formation.

  • Added AWS endpoint configuration for Lake Formation client.

  • Fixed an issue which caused the sync_partition_metadata operation to fail when partition paths had case changes.

  • Restored support for SymlinkTextInputFormat for text formats.

  • Fixed reading Delta Lake files with encoded characters on Azure.

  • Fixed failure when reading certain Avro data with UNION data types.

  • Fixed incompatibility with GCS for the Lakehouse connectors.

435-e.4 changes (17 Apr 2024)#

  • Enabled PyStarburst dataframe API by default.

  • Fixed possible worker crashes when running aggregation queries due to out-of-memory error.

  • Fixed incorrect results when querying a table being modified concurrently.

  • Fixed embedded cache service with password authenticator.

  • Fixed handling of union options in Hive and Avro to allow coercion to a single type.

  • Fixed dataframe improperly handling SQL function invocations.

  • Fixed a bug that caused the creation of materialized views to fail when using MySQL as the cache service backend database if materialized_view_definitions is longer than 64K characters.

435-e.5 changes (20 May 2024)#

  • Fixed potential query failure due to worker nodes running out of memory in concurrent scenarios.

  • Fixed failure with predicate pushdown on BigQuery views.

  • Fixed incorrect result with deletion vector on Delta partitioned table.

  • Fixed last openRecordGroup not processed in FlatArrayBuilder.

  • Fixed correctness bug in constant literal distinct aggregation.

  • Fixed Prometheus whiteListObjectNames being overwritten when KEDA is enabled.

  • Fixed underestimation of memory usage when writing strings to Parquet files.

  • Fixed failure when translating Hive views that contain EXISTS clauses.

  • Fixed complex predicate handling with table scan redirection.

435-e.6 changes (14 Jun 2024)#

  • Fixed potential failure when reading ORC files larger than 2GB.

  • Fixed potential correctness issue on receivers refresh that could cause query hanging. Applies to the Teradata Direct connector.

  • Backported IMDSv2 service metadata access.

435-e.7 changes (28 Jun 2024)#

  • Fixed incorrect results when specifying a value for the cassandra.partition-size-for-batch-select configuration property.

  • Fixed failure when writing to tables with Iceberg VARBINARY values.

  • Fixed rare query failure for array types when the data dictionary is encoded.

435-e.8 changes (11 Jul 2024)#

  • Added encoding to error code in OAuth2 callback handler.

  • Fixed reading empty files from S3 and GCS.

  • Fixed issue syncing partition metadata which could cause data deletion.

435-e.9 changes (29 Jul 2024)#

  • Fixed bug preventing use of Starburst security in Delta Lake connector.

  • Fixed error when writing a large amount of data in S3 file system.

435-e.10 changes (14 Aug 2024)#

  • Fixed failure when executing vacuum procedure on tables without old transaction logs.

  • Fixed optimizer timeout for certain queries involving aggregations and CASE expressions.

  • Fixed failure when adding new columns with a decimal type.

  • Fixed failure to read Hive tables migrated to Iceberg with Apache Spark.

  • Fixed issue that caused the error ‘Multiple masks on a single column are not supported’ to occur unintentionally.

435-e.11 changes (30 Aug 2024)#

  • Fixed failure when a user-defined type name contains uppercase characters.

  • Fixed query failure when file-based network topology is configured with the node-scheduler.network-topology.file configuration property.

  • Fixed support for migration of Ranger policies in security zones.

  • Fixed performance issues with partitioned tables when using Lake Formation integration in Hive connector.

435-e.12 changes (13 Sep 2024)#

  • Fixed a bug that caused cluster metrics to be created with incorrect intervals and subsequently led to loss of cluster metrics data.

  • Fixed Run and troubleshoot feature when insights.authorized-groups configuration property contains authorized groups.

  • Fixed numeric overflow during managed statistics computation for large tables in Teradata mode session.

435-e.13 was skipped

435-e.14 changes (18 Oct 2024)#

  • Fixed OpenX JSON decoding a JSON array line that resulted in data being written to the wrong output column.

  • Fixed reading large Prometheus responses.

  • Fixed failures for count(*) queries with predicates containing non-ASCII strings. Applies to the Elasticsearch connector.

435-e.15 changes (4 Nov 2024)#

  • Use hive.metastore.partition-batch-size.max config property value in sync_partition_metadata procedure. The default batch size is changed to 100 from 1000.

  • Updated Iceberg connector migration procedure to use nullable columns by default.

435-e.16 changes (14 Nov 2024)#

  • Fixed memory leak in InMemoryEventClient within cache service.

435-e.17 changes (27 Nov 2024)#

  • Fixed incorrect results for queries filtering on a partition columns and the NAME column mapping is used.