Release 443-e LTS (31 May 2024)#
Starburst Enterprise platform (SEP) 443-e LTS is the follow up release to the 438-e STS release and the 435-e LTS release.
This release is a promotion of the original 443-e STS release in May 2024 into a long term support (LTS) release.
It contains all improvements from Starburst Enterprise releases since the 435-e LTS release:
The 443-e release includes all improvements from the following Trino releases:
This release is a long term support (LTS) release.
Highlights since 435-e#
Promoted Managed statistics, SAML 2.0 authentication, AWS Lake Formation access control support, and Shared queries to general availability.
Added support for Apache Ozone.
Improved the Starburst Enterprise web UI with accessibility enhancements.
Added resource group ID to query information in the Starburst Enterprise web UI.
Added location privilege support for built-in access control.
Added support for Vault by Hashicorp and AWS Secrets Manager as external configuration providers.
Breaking changes#
As of SEP 438-e, Starburst Warp Speed uses a new file system caching mechanism. To use Starburst Warp Speed, you must add the
--enable-preview
flag to yourjvm.config
configuration. Additionally, the following catalog configuration properties have been removed:warp-speed.workerdb.db.path
warp-speed.file-system-reserve-percentage
warp-speed.call-home.enable
You must remove these configuration properties from your cluster configuration or the cluster fails to start.
The Starburst Elasticsearch connector no longer supports Elasticsearch version 6.x or OpenSearch 1.x. Update Elasticsearch to version 7.x or 8.x to continue using the connector. To connect to OpenSearch, use the OpenSearch connector.
SEP now requires JDK 21 to run. See the Java runtime environment requirements for more information.
The
hive.cache.enabled
configuration property has been deprecated in favor offs.cache.enabled
. Remove and replace the deprecated property from all Hive catalog configurations.The
legacy
value forhive.security
has been removed, the new default value isallow-all
. See Authorization for more information.The following Hive authorization configuration properties have been removed. These properties must be removed from all configurations or the cluster does not start:
hive.allow-drop-table
hive.allow-rename-table
hive.allow-add-column
hive.allow-drop-column
hive.allow-rename-column
hive.allow-comment-table
hive.allow-comment-column
Removed the
service-database.connection-pool.enabled
configuration property from the cache service. You must remove this configuration property or the cluster fails to start.The cache service now supports the same authentication methods as the SEP backend service database. As part of this change, the following cache service configuration properties have changed and must be updated in your configuration:
service-database.user
toinsights.jdbc.user
.service-database.password
toinsights.jdbc.password
.service-database.jdbc-url
toinsights.jdbc.url
.service-database.connection-pool.max-size
toinsights.jdbc.connection-pool.max-size
.service-database.connection-pool.idle-timeout
toinsights.jdbc.connection-pool.idle-timeout
.
Read more about the requirements for the cache service storage in the documentation.
If you are using MySQL as the externally-managed database for the cache service, you must append the parameter
sessionVariables=sql_mode=ANSI
to the connection string you use in theinsights.jdbc.url
property or the cluster fails to start.This release removes the
snowflake_distributed
connector. You must remove or migrate existing Snowflake catalogs that use the distributed connector to the parallel connector or the cluster fails to start.If you are using Oracle as the externally-managed database for the Backend service and upgrading to this version from SEP version 435-e or earlier, you must take additional steps prior to upgrading SEP. Contact Starburst Support for assistance.
443-e initial changes#
General#
Added public preview support for MaxCompute connector.
Enabled PyStarburst dataframe API by default.
Added support for Vault by Hashicorp and AWS Secrets Manager as external configuration providers.
Added support for updating and creating specific views or materialized views with data products in the SEP REST API.
Added support for automatic internal transport layer security (TLS) for managed statistics.
Added support for using a stored procedure to manually refresh tables in a table scan redirection.
Added keyboard shortcuts to the query name links in Saved queries and the assign button in Roles and Privileges in the Starburst Enterprise web UI to improve accessibility.
Added a toggle switch in the What can they do? dialog and in the Switch role dialog in the Starburst Enterprise web UI to improve accessibility.
Improved subquery cache hits by removing redundant predicates on data columns from cache key.
Changed the
experimental.thread-per-driver-scheduler-enabled
property to be disabled by default.Increased character limit from 40 to 255 in the SEP REST API.
Fixed a bug that caused the creation of materialized views to fail when using MySQL as the cache service backend database if
materialized_view_definitions
is longer than 64K characters.Fixed issue where a dynamic row filtering fallback mechanism could cause invalid results.
Db2 connector#
Added support for variable-precision timestamps to the nanosecond.
Delta Lake connector#
Improved speed at which tables and views are listed.
DynamoDB connector#
Added limited support for partial predicate pushdown.
Fixed unbounded
VARCHAR
handling.
Hive connector#
Added support for comments on partitioned columns in the File and Thrift Hive metastores.
Improved speed at which tables and views are listed.
Fixed bug that caused
DESCRIBE materialized_view
to fail.
Iceberg connector#
Improved speed at which tables and views are listed.
443-e.1 changes (31 May 2024)#
Fixed failure when translating Hive views that contain
EXISTS
clauses.Fixed under-accounting of memory usage when writing strings to Parquet files.
Fixed potential failure when reading ORC files larger than 2GB.
Fixed startup failure when fault-tolerant execution is enabled with Google Cloud Storage exchange.
Fixed potential loss of a query completion event when multiple queries fail at the same time.
Fixed potential failure when queries contain filtered aggregations.
Fixed under-accounting of memory usage when writing strings to Parquet files.
Fixed complex predicate handling with table scan redirection.
Fixed last openRecordGroup not processed in FlatArrayBuilder.
Fixed potential query hang when there is an error processing data.
Fixed incorrect results for distinct count aggregations over a constant value.
443-e.2 was skipped
443-e.3 changes (14 Jun 2024)#
Fixed potential correctness issue on receivers refresh that could cause query hanging. Applies to the Teradata Direct connector.
Backported IMDSv2 service metadata access.
443-e.4 changes (28 Jun 2024)#
Fixed incorrect results when specifying a value for the
cassandra.partition-size-for-batch-select
configuration property.Fixed failure when reading Parquet files without
field-id
on structured types.Fixed failure when writing to tables with Iceberg
VARBINARY
values.Fixed rare query failure for array types when the data dictionary is encoded.
Fixed failure when partition column name contains uppercase in
UNLOAD
.
443-e.5 was skipped
443-e.6 changes (11 Jul 2024)#
Added flag for cleaning the storage when the system is loaded when using Starburst Warp Speed.
Added encoding to error code in OAuth2 callback handler.
Fixed reading empty files from S3 and GCS.
Fixed issue syncing partition metadata which could cause data deletion.
Fixed a bug preventing use of Starburst security in the Delta Lake connector.
443-e.7 changes (29 Jul 2024)#
Fixed error when writing a large amount of data in S3 file system.
Fixed failure when reading tables with
NULL
on partition columns while theoptimize_metadata_queries
session property is enabled.
443-e.8 changes (14 Aug 2024)#
Fixed failure when executing
vacuum
procedure on tables without old transaction logs.Fixed potential failure for queries involving
GROUP BY
,UNNEST
, and filters over expressions that may produce an error for certain inputs.Fixed optimizer timeout for certain queries involving aggregations and
CASE
expressions.Fixed failure when adding new columns with a decimal type.
Fixed failure to read Hive tables migrated to Iceberg with Apache Spark.
Fixed issue that caused the error ‘Multiple masks on a single column are not supported’ to occur unintentionally.
443-e.9 changes (30 Aug 2024)#
Fixed failure when a user-defined type name contains uppercase characters.
Fixed query failure when file-based network topology is configured with the
node-scheduler.network-topology.file
configuration property.Fixed support for migration of Ranger policies in security zones.
Fixed performance issues with partitioned tables when using Lake Formation integration in Hive connector.
Fixed numeric overflow during managed statistics computation for large tables in Teradata connector.
Fixed an issue that affected managed statistics collection on wide Teradata tables in specific circumstances.
443-e.10 changes (13 Sep 2024)#
Fixed a bug that caused cluster metrics to be created with incorrect intervals and subsequently led to loss of cluster metrics data.
Fixed memory tracking issue for aggregations that could cause worker crashes with out-of-memory errors.
Fixed Run and troubleshoot feature when
insights.authorized-groups
configuration property contains authorized groups.