Release 435-e LTS (29 Feb 2024)#
Starburst Enterprise platform (SEP) 435-e LTS is the follow up release to the 435-e STS release and the 429-e LTS release.
This release is a promotion of the original 435-e STS release in February 2023 into a long term support (LTS) release.
The 435-e release includes all improvements from the following Trino releases:
It contains all improvements from Starburst Enterprise releases since the 429-e LTS release:
Highlights since 429-e#
Added support for Java 21.
Released SCIM in general availability.
Added public preview support for querying views created in Amazon Athena.
Added Schema discovery as a public preview
Added support for the
queryText
selector rule in resource groups to allow the use of a regular expression to match against the SQL query string.Added public preview support for SQL routines.
Breaking changes#
This release includes some changes to how Ranger policies are evaluated that may require policy modifications. If you use Apache Ranger for role-based access control, read the migration guide.
This release removes RubiX, Apache Sentry, and Hive Alluxio from the Hive connector. If your existing deployment relies on these technologies and you wish to use the legacy Hive connector, contact Starburst Support for assistance.
The
query.max-writer-tasks-count
configuration property has been renamed toquery.max-writer-task-count
, and its corresponding session propertymax_writer_tasks_count
renamed tomax_writer_task_count
. These properties must be renamed in your cluster configuration or the cluster fails to start.The built-in access control REST API has been updated. Grants for functions are now created for all functions in a catalog (
catalog.*
), all functions in a catalog’s schema (catalog.schema.*
), or for a specific function (catalog.schema.function
). API requests that only describe the function name must be updated or validation fails with an error.Older versions of the Trino CLI and JDBC driver now fail when attempting to run
RESET/SET SESSION AUTHORIZATION
statements. To use these statements, upgrade to the latest versions of these clients. As a best practice, always use the client version matching the SEP cluster version.The
hive.metastore-timeout
catalog configuration property has been removed in favor of thehive.metastore.thrift.client.connect-timeout
andhive.metastore.thrift.client.read-timeout
properties. This property must be removed from your catalog configuration files to allow successful startup.The
experimental.late-materialization.enabled
andexperimental.work-processor-pipelines
configuration properties have been removed. These properties must be removed from your cluster configuration to allow successful cluster startup.This release introduces some security fixes in Trino that restrict users’ ability to create new Iceberg materialized views with Glue while using the
iceberg.security=system
catalog configuration setting. Querying new Iceberg materialized views that fit this criteria fail and return aMaterialized view does not have an owner
error. See Using BIAC with Iceberg materialized views for steps to mitigate this issue.The
optimizer.mark-distinct-strategy
configuration property has been removed. You must remove this property from the cluster configuration in favor of theoptimizer.distinct-aggregations-strategy
configuration property or the cluster fails to start.The
internal-communication.shared-secret
,managed-statistics.http-client.key-store-path
, andmanaged-statistics.http-client.key-store-password
catalog configuration properties are no longer required to enable managed statistics for a catalog. You must remove these properties from your catalog configuration files or the cluster fails to start. Theinternal-communication.shared-secret
property is still valid as a general configuration property and can remain in non-catalog configuration files.This release removed support for registering external tables with
CREATE TABLE
and thelocation
table property. Use theregister_table
procedure instead. Thedelta.legacy-create-table-with-existing-location.enabled
configuration property and correspondinglegacy_create_table_with_existing_location_enabled
session property are removed as well.
435-e initial changes#
General#
Added Schema discovery as a public preview
Added public preview support for Neo4j connector.
Added keyboard navigation to additional pages on the Starburst Enterprise web UI.
Updated contents of the Run and troubleshoot archive to include OpenTelemetry traces.
Fixed incorrect behavior of the
ELSE
clause in SQL routines with a singleIF
orAND
condition.
Security#
Added location privilege support for Ranger policies.
Hive connector#
Added public preview support for querying views created in Amazon Athena.
Fixed an issue with accessing security
DEFINER
views asINVOKER
mode when {doc}}Hive metastore impersonation<object-storage/file-system-hdfs>
is enabled.
Hudi connector#
Released as a general availability feature.
Added a configuration option to enforce that a filter on a partition key be present in the query.
Iceberg connector#
Fixed data loss on contended commit to a table under some circumstances.
MongoDB connector#
Added support for Kerberos credential pass-through.
SingleStore connector#
Added support for parallel read operations.
435-e.1 changes (29 Feb 2024)#
Updated Kubernetes external secret operator.
Fixed query failure when invoking the
json_table
function.Fixed query hang when a SQL routines dereferences a row field.
Fixed potential incorrect results when using the
ST_Centroid()
andST_Buffer()
functions for tiny geometries.Fixed query failure when using
char
types with thereverse()
function.Fixed query failure when querying data with deeply nested rows.
Fixed showing stats (
EXPLAIN ANALYZE
orSHOW STATS
) for security definer views on partitioned table, when HMS impersonation is enabled.Fixed incorrect results for
DATETIMEOFFSET
values before the year 1400.
435-e.2 changes (11 Mar 2024)#
Fixed UI authentication for large authentication tokens.
Fixed access issue with security
DEFINER
views when both materialized views and HMS impersonation are enabled.Fixed a bug where column statistics created in SEP would not be visible in Hive when using CDP 7.
435-e.3 changes (28 Mar 2024)#
Added support for setting endpoint and region in STS clients in Lake Formation.
Added AWS endpoint configuration for Lake Formation client.
Fixed an issue which caused the
sync_partition_metadata
operation to fail when partition paths had case changes.Restored support for
SymlinkTextInputFormat
for text formats.Fixed reading Delta Lake files with encoded characters on Azure.
Fixed failure when reading certain Avro data with
UNION
data types.Fixed incompatibility with GCS for the Lakehouse connectors.
435-e.4 changes (17 Apr 2024)#
Enabled PyStarburst dataframe API by default.
Fixed possible worker crashes when running aggregation queries due to out-of-memory error.
Fixed incorrect results when querying a table being modified concurrently.
Fixed embedded cache service with password authenticator.
Fixed handling of union options in Hive and Avro to allow coercion to a single type.
Fixed dataframe improperly handling SQL function invocations.
Fixed a bug that caused the creation of materialized views to fail when using MySQL as the cache service backend database if
materialized_view_definitions
is longer than 64K characters.
435-e.5 changes (20 May 2024)#
Fixed potential query failure due to worker nodes running out of memory in concurrent scenarios.
Fixed failure with predicate pushdown on BigQuery views.
Fixed incorrect result with deletion vector on Delta partitioned table.
Fixed last openRecordGroup not processed in FlatArrayBuilder.
Fixed correctness bug in constant literal distinct aggregation.
Fixed Prometheus whiteListObjectNames being overwritten when KEDA is enabled.
Fixed underestimation of memory usage when writing strings to Parquet files.
Fixed failure when translating Hive views that contain
EXISTS
clauses.Fixed complex predicate handling with table scan redirection.
435-e.6 changes (14 Jun 2024)#
Fixed potential failure when reading ORC files larger than 2GB.
Fixed potential correctness issue on receivers refresh that could cause query hanging. Applies to the Teradata Direct connector.
Backported IMDSv2 service metadata access.
435-e.7 changes (28 Jun 2024)#
Fixed incorrect results when specifying a value for the
cassandra.partition-size-for-batch-select
configuration property.Fixed failure when writing to tables with Iceberg
VARBINARY
values.Fixed rare query failure for array types when the data dictionary is encoded.
435-e.8 changes (11 Jul 2024)#
Added encoding to error code in OAuth2 callback handler.
Fixed reading empty files from S3 and GCS.
Fixed issue syncing partition metadata which could cause data deletion.
435-e.9 changes (29 Jul 2024)#
Fixed bug preventing use of Starburst security in Delta Lake connector.
Fixed error when writing a large amount of data in S3 file system.
435-e.10 changes (14 Aug 2024)#
Fixed failure when executing
vacuum
procedure on tables without old transaction logs.Fixed optimizer timeout for certain queries involving aggregations and
CASE
expressions.Fixed failure when adding new columns with a decimal type.
Fixed failure to read Hive tables migrated to Iceberg with Apache Spark.
Fixed issue that caused the error ‘Multiple masks on a single column are not supported’ to occur unintentionally.
435-e.11 changes (30 Aug 2024)#
Fixed failure when a user-defined type name contains uppercase characters.
Fixed query failure when file-based network topology is configured with the
node-scheduler.network-topology.file
configuration property.Fixed support for migration of Ranger policies in security zones.
Fixed performance issues with partitioned tables when using Lake Formation integration in Hive connector.
435-e.12 changes (13 Sep 2024)#
Fixed a bug that caused cluster metrics to be created with incorrect intervals and subsequently led to loss of cluster metrics data.
Fixed Run and troubleshoot feature when
insights.authorized-groups
configuration property contains authorized groups.Fixed numeric overflow during managed statistics computation for large tables in Teradata mode session.
435-e.13 was skipped
435-e.14 changes (18 Oct 2024)#
Fixed OpenX JSON decoding a JSON array line that resulted in data being written to the wrong output column.
Fixed reading large Prometheus responses.
Fixed failures for
count(*)
queries with predicates containing non-ASCII strings. Applies to the Elasticsearch connector.