Jekyll2024-03-28T14:04:04-05:00https://docs.starburst.io/blog/feed.xmlStarburst developer blogYour hub to all knowledge about Starburst products.We moved2023-04-06T00:00:00-05:002023-04-06T00:00:00-05:00https://docs.starburst.io/blog/we-moved<p>Our blog moved to the Starburst blog! You can now find all old posts and many
new posts in the <a href="https://www.starburst.io/blog/technical">Technical blog</a>.</p>
<p>Enjoy the wealth of knowledge and updates, new features, updates, and
information from many additional authors.</p>
<p><em>Manfred</em></p>Our blog moved to the Starburst blog! You can now find all old posts and many new posts in the Technical blog. Enjoy the wealth of knowledge and updates, new features, updates, and information from many additional authors. ManfredStarburst Enterprise LTS backport releases2023-04-03T00:00:00-05:002023-04-03T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-407-e.html">407-e.3</a></li>
<li><a href="../latest/release/release-402-e.html">402-e.7</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.12</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.16</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 410-e STS2023-03-15T00:00:00-05:002023-03-15T00:00:00-05:00https://docs.starburst.io/blog/sep-410-e<p>Starburst Enterprise platform (SEP) 410-e STS has been released as a follow-up
to the <a href="/blog/2023-02-28-sep-407-e.html">407-e LTS release</a>.</p>
<!--more-->
<p>The release is based on latest Trino release and therefore includes the new
features from Trino 408, 409, and 409 since the last SEP release. Following are
some the highlights of these releasees:</p>
<p><a href="../latest/release/release-408.html">Trino 408</a></p>
<ul>
<li>New Apache Ignite connector!</li>
<li>Add support for writing decimal types to BigQuery.</li>
<li>Improve performance when reading structural types from Parquet files in Delta
Lake.</li>
</ul>
<p><a href="../latest/release/release-409.html">Trino 409</a></p>
<ul>
<li>Support for nested fields in <code class="language-plaintext highlighter-rouge">DROP COLUMN</code>.</li>
<li>Support for sorted tables in Iceberg.</li>
<li>Support for time type in Cassandra.</li>
<li>Faster aggregations containing <code class="language-plaintext highlighter-rouge">DISTINCT</code>.</li>
<li>Faster <code class="language-plaintext highlighter-rouge">LIKE</code> with dynamic patterns.</li>
</ul>
<p><a href="../latest/release/release-410.html">Trino 410</a></p>
<ul>
<li>Support for the <code class="language-plaintext highlighter-rouge">sheet</code> table function in Google Sheets.</li>
<li>Better file pruning in Iceberg.</li>
</ul>
<p>In SEP, we’ve added support for managed statistics with the Snowflake connector,
and add PingIdentity as supported provider for OAuth 2.0 authentication. Have a
look at the <a href="../latest/release/release-410-e.html">release notes for 410-e</a> for
further features and improvements.</p>
<p>Especially take note of the <em>breaking changes</em> and adapt your catalog and config
properties files accordingly as part of your update.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 410-e STS has been released as a follow-up to the 407-e LTS release.Starburst Enterprise LTS backport releases2023-03-15T00:00:00-05:002023-03-15T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-407-e.html">407-e.2</a></li>
<li><a href="../latest/release/release-402-e.html">402-e.6</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.11</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.15</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 407-e LTS2023-02-28T00:00:00-06:002023-02-28T00:00:00-06:00https://docs.starburst.io/blog/sep-407-e<p>Another quarter, another <a href="../latest/versions.html#lts-sts">long term support
(LTS)</a> release! The new Starburst Enterprise
407-e LTS release includes the latest batch of features for our customers.
There’s never a better time to upgrade your cluster, so read on for details of
what’s included.</p>
<!--more-->
<p>The 407-e LTS release introduces the new <a href="../latest/optimizer/managed-statistics.html">managed
statistics</a> feature, which allows
Starburst Enterprise to collect table and column statistics from select data
sources that currently expose limited or no statistics. This feature allows
queries to Oracle, PostgreSQL, and Teradata to take advantage of query
optimizations that may dramatically improve query runtime and efficiency.</p>
<p><a href="../latest/admin/fault-tolerant-execution.html">Fault-tolerant execution</a>
now supports write operations with MongoDB and BigQuery. The feature is now also
compatible with HDFS storage as an external spooling storage option.</p>
<p>As of the 407-e LTS release, Starburst Warp Speed (previously “Smart indexing
and caching”) is available as a GA feature with the Starburst Enterprise Elite
license. Warp Speed adds an indexing and caching layer for object storage
catalogs to dramatically improve query performance on your data lakehouse. This
LTS release also adds REST endpoints for Warp Speed, and introduces the new
index and cache resiliency feature as a public preview.</p>
<p>The <a href="../latest/starburst-rest-api.html">Starburst Enterprise REST API</a> now
includes endpoints that allow you to programmatically manage <a href="../latest/security/biac-overview.html">built-in access
control</a> for your Starburst Enterprise
cluster. See the updated <a href="../latest/api/index.html">API documentation</a> for all
of the new and existing endpoints available.</p>
<p>We’ve expanded support for <a href="../latest/security/aws-lake-formation.html">AWS Lake Formation access
control</a> with more
granular control options, such as Glue user impersonation and selecting
AWS roles per-catalog.</p>
<p>These are just some of the highlights that have made it into this quarterly LTS
release. For a complete list of features and changes, read the <a href="../latest/release/release-407-e.html">407-e LTS
release notes</a>.</p>
<p>As always, we’re continuing to add and improve features for the next release,
and we eagerly await your feedback on the release. Cheers!</p>
<p><em>Joe and Manfred</em></p>Another quarter, another long term support (LTS) release! The new Starburst Enterprise 407-e LTS release includes the latest batch of features for our customers. There’s never a better time to upgrade your cluster, so read on for details of what’s included.Starburst Enterprise LTS backport releases2023-02-21T00:00:00-06:002023-02-21T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-402-e.html">402-e.5</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.10</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.14</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.18</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise LTS backport releases2023-02-14T00:00:00-06:002023-02-14T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-402-e.html">402-e.4</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.9</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.13</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.17</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 406-e STS2023-02-01T00:00:00-06:002023-02-01T00:00:00-06:00https://docs.starburst.io/blog/sep-406-e<p>Starburst Enterprise platform (SEP) 406-e STS has been released as a follow-up
to the <a href="/blog/2023-01-03-sep-405-e.html">405-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-406.html">Trino 406</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for JDBC catalog in Iceberg connector.</li>
<li>Support for fault-tolerant execution in BigQuery connector.</li>
<li>Support for exchange spooling on HDFS.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">CHECK</code> constraints in <code class="language-plaintext highlighter-rouge">INSERT</code>.</li>
<li>Improved performance for Parquet in Delta Lake, Hive, Hudi and Iceberg
connectors.</li>
</ul>
<p>In SEP, we’ve added a new Azure Cosmos DB connector available as a public
preview. We’ve also added a new feature to the Starburst Enterprise web UI to
save your query editor worksheets, and added more support to Starburst
Enterprise built-in access control. For more information, refer to the <a href="../latest/release/release-406-e.html">release
notes for 406-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 406-e STS has been released as a follow-up to the 405-e STS release.Starburst Enterprise LTS backport releases2023-01-20T00:00:00-06:002023-01-20T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-402-e.html">402-e.3</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.8</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.12</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.16</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 405-e STS2023-01-03T00:00:00-06:002023-01-03T00:00:00-06:00https://docs.starburst.io/blog/sep-405-e<p>Starburst Enterprise platform (SEP) 405-e STS has been released as a follow-up
to the <a href="/blog/2022-12-23-sep-403-e.html">403-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-405.html">Trino 405</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for <code class="language-plaintext highlighter-rouge">ALTER COLUMN ... SET DATA TYPE</code> statement.</li>
<li>Support for Apache Arrow when reading from BigQuery.</li>
<li>Support for views in Delta Lake connector.</li>
<li>Support for the Iceberg REST catalog.</li>
<li>Support for Protobuf encoding in Kafka connector.</li>
<li>Support for fault-tolerant execution in MongoDB connector.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">DELETE</code> and query pushdown in Redshift connector.</li>
<li>Performance improvements when reading Parquet data.</li>
</ul>
<p>On the SEP side we included a lot of minor improvements and bug fixes for smart
indexing and caching. For more information, refer to the <a href="../latest/release/release-405-e.html">release notes for
405-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 405-e STS has been released as a follow-up to the 403-e STS release.Starburst Enterprise 403-e STS2022-12-23T00:00:00-06:002022-12-23T00:00:00-06:00https://docs.starburst.io/blog/sep-403-e<p>Starburst Enterprise platform (SEP) 403-e STS has been released as a follow-up
to the <a href="/blog/2022-11-17-sep-402-e.html">402-e LTS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-403.html">Trino 403</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for <code class="language-plaintext highlighter-rouge">DELETE</code> in MongoDB connector.</li>
<li>Faster aggregations when input data does not contain nulls.</li>
<li>Faster data transfers when fault-tolerant execution is enabled.</li>
<li>Faster <code class="language-plaintext highlighter-rouge">SHOW SCHEMAS</code> in BigQuery connector.</li>
<li>Faster <code class="language-plaintext highlighter-rouge">expire_snapshots</code> in Iceberg connector.</li>
</ul>
<p>SEP highlights include:</p>
<ul>
<li>Multiple improvements to the
<a href="../latest/client/query-editor.html">Query editor</a>, such as live syntax
checks, improved readability for single-row query results, and an option to
download diagnostic information with query execution to help with
troubleshooting.</li>
<li>Improvements to the <a href="../latest/security/biac-audit-log.html">built-in access control audit
logs</a>.</li>
</ul>
<p>This release also features a brand-new <strong>Dark mode</strong> option in the <a href="../latest/overview/sep-ui.html">SEP web
UI</a>, which we encourage you to check out.</p>
<p>Light mode (default):</p>
<p><img src="../assets/img/general/12-23-lightmode.png" alt="Screenshot of the default light mode for the SEP web UI" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>Dark mode:</p>
<p><img src="../assets/img/general/12-23-darkmode.png" alt="Screenshot of the new dark mode for the SEP web UI" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>For more information, refer to the <a href="../latest/release/release-403-e.html">release notes for
403-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 403-e STS has been released as a follow-up to the 402-e LTS release.Starburst Enterprise LTS backport releases2022-12-08T00:00:00-06:002022-12-08T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-402-e.html">402-e.2</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.7</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.11</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.15</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise LTS backport releases2022-12-01T00:00:00-06:002022-12-01T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-402-e.html">402-e.1</a></li>
<li><a href="../latest/release/release-393-e.html">393-e.6</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.10</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.14</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p>The 364-e LTS version has reached the end of its support lifetime as of November
30th, and therefore was not included in this round of backports. We strongly
suggest that any users of 364-e upgrade to a supported LTS version to continue
receiving product support through regular updates.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Building lakehouse with dbt and Trino2022-11-30T00:00:00-06:002022-11-30T00:00:00-06:00https://docs.starburst.io/blog/dbt0-introduction<p>In this series, we demonstrate how to build data pipelines using dbt and Trino
with data directly from your operational systems. They can use a variety of
sources including relational databases, NoSQL databases, and other systems. The
resulting data is stored in the data lake using the open source Iceberg table
format.</p>
<!--more-->
<p>The series is centered around a data-driven e-commerce business. This business
is optimizing its revenue based on customer behavior. Their architecture
consists of a PostgreSQL database driving the e-commerce processes on their
website, and an Elasticsearch database gathering all customer behavior on their
website in the form of clickstream data.</p>
<p>The webshop has all the typical entities and data points such as users,
sessions, products, categories, carts, orders, payments, and deliveries.</p>
<p>The clickstream contains information about every click performed by every user
on the webshop and its affiliate websites.</p>
<p>In the series, you learn how to set up dbt and Trino, and start using it to
solve real business cases. You set up reports that offer insights on improving
the customer journey. For example, a report to show sales by channel or a report
with referral traffic to determine which channel is most effective to attract
new sales.</p>
<h2 id="why-dbt">Why dbt?</h2>
<p><a href="https://www.getdbt.com/">dbt™</a> is a transformation workflow that lets teams
quickly and collaboratively deploy analytics code following software engineering
best practices like modularity, portability, CI/CD, and documentation. Anyone
who knows SQL can build production-grade data pipelines.</p>
<p><img src="../assets/img/blog/dbt.png" alt="how-dbt-works image" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<h2 id="why-trino">Why Trino?</h2>
<p><a href="https://trino.io">Trino</a> can read data from numerous data sources, and supports
writing to modern table formats such as Iceberg. SQL support and high
performance enable these data federation capabilities of Trino to make all your
data instantly available without the need to copy it into a proprietary data
format.</p>
<h2 id="why-iceberg">Why Iceberg?</h2>
<p><a href="https://iceberg.apache.org/">Iceberg</a> brings a lot of the ACID goodies from
relational databases to the lakehouse table.</p>
<ul>
<li>Instead of using proprietary data formats inaccessible by other tools, your
data is stored in open formats like Parquet, ORC, or AVRO files, and is
immediately accessible through a multitude of tools like Pandas, Dask, Spark,
and Trino.</li>
<li>Iceberg enables fast querying as it knows how your data is organized with its
support for partitioning, bucketing and other improvements.</li>
<li>Iceberg allows for flexibility such as adding or removing columns through
schema evolution.</li>
<li>Iceberg allows for time travel on your data. It offers you a way to see how
your data looked at a particular point in time or even revert it to an earlier
state through the concept of snapshots.</li>
</ul>
<h2 id="roadmap">Roadmap</h2>
<p>In this series, you can learn how to set up Trino and dbt for fast and solid
data pipelines including machine learning use cases:</p>
<ul>
<li><strong>Setting up Trino for dbt</strong> - Register all your data sources to
immediately start using them in your data pipelines.</li>
<li><strong>First dbt-trino data pipeline</strong> - Learn how to set up dbt and create your
first dbt data pipeline.</li>
<li><strong>Refresh your data faster using incremental models</strong> - Instead of loading all
your data every time dbt runs, learn how to use incremental refresh to reduce
spent and compute time.</li>
</ul>
<h2 id="target-audience">Target audience</h2>
<p>This series is targeted at analytics engineers and data engineers who would like
to improve their data pipelines with dbt and Trino tools. The beginner friendly
tutorials enable you to use the federation and lakehouse architecture
capabilities of Trino, and cover several aspects like setting up a Trino
instance locally with all underlying data sources, presenting how to use dbt
with Trino, and many more advanced features of dbt.</p>
<h2 id="demo-repository">Demo repository</h2>
<p>All files and code referenced in the article is available on GitHub. Clone or
fork <a href="https://github.com/starburstdata/dbt-demo">the repository
https://github.com/starburstdata/dbt-demo</a>
to follow along and explore capabilities of dbt and Trino.</p>
<h2 id="contents">Contents</h2>
<ul>
<li><a href="/blog_dbt/2022-11-30-dbt1-trino-setup.html">Setting up Trino for dbt</a></li>
<li><a href="/blog_dbt/2022-11-30-dbt2-first-dbt-data-pipeline.html">First dbt-trino data pipeline</a></li>
<li><a href="/blog_dbt/2022-11-30-dbt3-refresh-your-data-faster.html">Refresh your data faster using incremental models</a></li>
</ul>In this series, we demonstrate how to build data pipelines using dbt and Trino with data directly from your operational systems. They can use a variety of sources including relational databases, NoSQL databases, and other systems. The resulting data is stored in the data lake using the open source Iceberg table format.Starburst Enterprise 402-e LTS2022-11-17T00:00:00-06:002022-11-17T00:00:00-06:00https://docs.starburst.io/blog/sep-402-e<p>Another quarter, another <a href="../latest/versions.html#lts-sts">long term support (LTS)</a>
release! The new Starburst Enterprise 402-e LTS includes the latest batch of
features for our customers. There’s never a better time to upgrade your cluster,
so read on for details of what’s included.</p>
<!--more-->
<p>The 402-e LTS features <a href="../latest/connector/starburst-warp-speed.html">smart indexing and
caching</a> that can greatly improve
performance for object storage catalogs like Hive and Iceberg. This is a huge
feature that brings some highly requested performance improvements to Starburst
Enterprise, and we’ll have much more to say about it in future blog posts and
announcements. Stay tuned!</p>
<p>Starburst Enterprise’s <a href="../latest/security/biac-overview.html">built-in access
control</a> keeps improving with new
features, such as <a href="../latest/security/biac-masks-filters.html">row filtering and column
masking</a> configurable directly in
the Starburst web interface. We’ve also added a new access log to the <a href="../latest/security/biac-audit-log.html">audit
logs</a> that shows events of users
accessing data and Starburst product features.</p>
<p>The Starburst Enterprise UI’s built-in <a href="../latest/client/query-editor.html">query
editor</a> features several new quality of life
improvements. We’ve added support for writing and executing multiple queries in
a single editor tab, and improved SQL syntax support with the prettify feature.</p>
<p><a href="../latest/admin/fault-tolerant-execution.html">Fault-tolerant execution</a>
support is expanding as well; you can now run write operations for MySQL,
PostgreSQL, and SQL Server with fault-tolerant execution. If you haven’t yet
configured fault-tolerant execution in your Starburst Enterprise cluster, check
out our new guide on Enabling fault-tolerant execution for queries in SEP.</p>
<p>These are just some of the highlights that have made it into this quarterly LTS
release. For a complete list of features and changes, read the <a href="../latest/release/release-402-e.html">402-e LTS
release notes</a>.</p>
<p>With this release, we’ve also improved our <a href="../latest/index.html">Starburst Enterprise reference
documentation</a> with a reworked navigation. Don’t worry,
despite the new look all pages still share the same URLs so none of your
bookmarks should be affected by the rework. Please take a look!</p>
<p>As always, we’re continuing to add and improve features for the next release,
and we eagerly await your feedback on the release. Cheers!</p>
<p><em>Joe and Manfred</em></p>Another quarter, another long term support (LTS) release! The new Starburst Enterprise 402-e LTS includes the latest batch of features for our customers. There’s never a better time to upgrade your cluster, so read on for details of what’s included.Starburst Enterprise LTS backport releases2022-11-17T00:00:00-06:002022-11-17T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-393-e.html">393-e.5</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.9</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.13</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.18</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 401-e STS2022-11-03T00:00:00-05:002022-11-03T00:00:00-05:00https://docs.starburst.io/blog/sep-401-e<p>Starburst Enterprise platform (SEP) 401-e STS has been released as a follow-up
to the <a href="/blog/2022-10-17-sep-399-e.html">399-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-400.html">Trino 400</a> and
<a href="../latest/release/release-401.html">Trino 401</a>, which include the following
noteworthy changes:</p>
<ul>
<li>Support for TRUNCATE in BigQuery connector.</li>
<li>Support for the Pinot proxy.</li>
<li>Improved latency when querying Iceberg tables with many files.</li>
<li>Support for writes on GCS in Delta Lake connector.</li>
<li>Support for IBM Cloud Object Storage in Hive connector.</li>
<li>New query table function for MongoDB connector.</li>
<li>Support for writes with fault tolerant execution in MySQL, PostgreSQL and SQL
Server connectors.</li>
</ul>
<p>In SEP, we’ve improved the new <a href="../latest/security/biac-audit-log.html">built-in access control access
log</a> with an option to download the log
as a CSV file.</p>
<p>For more information, refer to the <a href="../latest/release/release-401-e.html">release notes for 401-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 401-e STS has been released as a follow-up to the 399-e STS release.Starburst Enterprise LTS backport releases2022-10-27T00:00:00-05:002022-10-27T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-393-e.html">393-e.4</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.8</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.12</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.17</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 399-e STS2022-10-17T00:00:00-05:002022-10-17T00:00:00-05:00https://docs.starburst.io/blog/sep-399-e<p>Starburst Enterprise platform (SEP) 399-e STS has been released as a follow-up
to the <a href="/blog/2022-10-04-sep-398-e.html">398-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-399.html">Trino 399</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Faster joins.</li>
<li>Faster reads of decimal values in Parquet data.</li>
<li>Support for writing <code class="language-plaintext highlighter-rouge">array</code>, <code class="language-plaintext highlighter-rouge">row</code>, and <code class="language-plaintext highlighter-rouge">timestamp</code> columns in BigQuery.</li>
<li>Support for predicate pushdown involving datetime types in MongoDB.</li>
</ul>
<p>In SEP, we’ve added a new access logging feature to the <a href="../latest/security/biac-audit-log.html">built-in access control
audit log</a>, which gives admins more
insights into how the cluster is being used.</p>
<p>For more information, refer to the <a href="../latest/release/release-399-e.html">release notes for 399-e</a>.</p>
<p><em>Jessie, Shea, and Manfred</em></p>Starburst Enterprise platform (SEP) 399-e STS has been released as a follow-up to the 398-e STS release.Starburst Admin 1.6.0 release2022-10-12T00:00:00-05:002022-10-12T00:00:00-05:00https://docs.starburst.io/blog/starburst-admin-update<p>Announcing the Starburst Admin 1.6.0 release, which is a follow-on to
release 1.5.1.</p>
<!--more-->
<p>Starburst Admin is the best tool choice for folks looking to install
Starburst Enterprise platform (SEP) on bare metal or in your virtual machines.</p>
<p>In this release, we’ve added a new automated memory configuration feature to
Starburst Admin, to make it simpler for customers to configure
SEP’s memory consumption parameters.</p>
<p>You can use this new feature by enabling the <code class="language-plaintext highlighter-rouge">memory_auto_config</code> section in
the <code class="language-plaintext highlighter-rouge">vars.yml</code> file for the <code class="language-plaintext highlighter-rouge">push-configs</code> playbook.</p>Announcing the Starburst Admin 1.6.0 release, which is a follow-on to release 1.5.1.Starburst Enterprise LTS backport releases2022-10-06T00:00:00-05:002022-10-06T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-393-e.html">393-e.3</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.7</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.11</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.15</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Shea and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Moonlighting for Trino2022-10-05T00:00:00-05:002022-10-05T00:00:00-05:00https://docs.starburst.io/blog/moonlighting-trino<p>Starburst was founded all around the <a href="https://www.trino.io">Trino project and
community</a>, and it shows! The passion to work on Trino,
Java, and other open source projects permeates everything we do. Let me tell you
about some of these efforts. Specifically, I wanted to let you know about the
results of my latest efforts of working late in the night under the moon - the
second edition of Trino: The Definitive Guide is out!</p>
<p><strong><a href="https://www.starburst.io/info/oreilly-trino-guide/">Get a free copy from Starburst now!</a></strong></p>
<!--more-->
<h2 id="engineers-advocating">Engineers advocating</h2>
<p>Before we dive into the book work, I wanted to point out a few efforts that show
what’s going on at Starburst:</p>
<p><a href="https://github.com/nineinchnick">Jan</a> works on the Trino developer experience
team for a reason. He hacks on improvements on Trino and for Trino developers
all the time. Just have a look at <a href="https://trino.io/episodes/37.html">Trino Community Broadcast episode
37</a>, where we talked about some of the
metrics he is gathering, that we then use to help with PR triage, reviews, and
merging. Jan created <a href="https://github.com/nineinchnick">Trino connectors for git and REST
APIs</a>, that are ready for your use.</p>
<p>Our developer teams in Poland are spreading the word about Trino and showcasing
their work on the core engine, connector, integrations, related topics like
CI/CD, DevOps and more. <a href="https://github.com/ppalucha">Pawel</a> presented at
<a href="https://yavaconf.com/en/">Ya!VaConf</a> and <a href="https://github.com/s2lomon">Szymon</a>
talked about Trino and data mesh at <a href="https://summit.datamass.io">Datamass Gdansk
Summit</a>. And last, but definitely not least,
<a href="https://github.com/wendigo">Mateusz</a> talked about his his experiences joining a
huge open source project at <a href="https://2022.confitura.pl">Confitura</a>, one of the
biggest Java conferences in Poland with over 1000 attendees.</p>
<h2 id="java-and-python">Java and Python</h2>
<p><img src="/assets/img/logo/java.png" style="float: left; margin-right: 20px; margin-bottom: 20px; width: 100%; max-width: 350px;" /></p>
<p>Recently <a href="https://github.com/Randgalt">Jordan</a> put together an awesome <a href="/blog/2022-09-20-bej0-introduction.html">blog
post series about the bleeding edge of Java 19</a>. That level of understanding and skill directly
benefits Trino, since we are working on updating a lot of the codebase to Java
17, and we plan to follow up with more modernization later.</p>
<p>Beyond our ever-present developer teams, product managers like
<a href="https://www.linkedin.com/in/bzhan/">Brian</a> constantly work for the Trino
project. Brian lives and breathes Trino in the communities in Silicon Valley,
researches what use cases we should focus on next, what features we need to
improve, and what new integrations and tools we should support. Help Brian out
by answering some questions about how you use <a href="https://trino.io/blog/2022/09/20/python-progress.html">Trino and
Python</a> and what
improvements you would like to see next.</p>
<h2 id="trino-summit">Trino Summit</h2>
<p>My Trino developer relations team member and Trino Community Broadcast co-host
<a href="https://github.com/bitsondatadev">Brian</a>, is full of creative ideas about
Trino. Just have a look at our <a href="https://trino.io/blog/2022/09/22/trino-summit-2022-teaser.html">Trino Summit invite
video</a> if you
have any doubts about his passion and creativity.</p>
<h2 id="trino-the-definitive-guide">Trino: The Definitive Guide</h2>
<p><img src="/assets/img/ttdg2-cover.png" style="float: right; margin-left: 20px; margin-bottom: 20px; width: 100%; max-width: 350px;" /></p>
<p>Getting back to the book though, it has been my late night activity over the
last couple months. Java 17, Trino 392, lakehouse architecture, views, more
connectors, better SQL support - so many changes! And <a href="https://simpligility.ca/2022/10/trino-the-definitive-guide-2nd-edition/">now it is
done</a>.
The book production process is finished, so binaries are available and
hardcopies are coming soon. In fact, you should <a href="https://trino.io/blog/2022/09/22/trino-summit-2022-teaser.html">join Matt, Martin, and myself
at the Trino
Summit</a>. As
Brian says, it will be legendary. We will be at the Starburst booth for a book
signing event, and in general just be in attendance for the rest of the
conference catching up about everything Trino.</p>
<p>The Trino and larger data community knows about the version from the
<a href="https://trino.io/blog/2022/10/03/the-definitive-guide-2.html">announcement on the Trino
site</a> and
community chat. Best of all, <a href="https://www.starburst.io/blog/second-edition-of-trino-the-definitive-guide/">Starburst is sponsoring the book for
everyone</a>.
So <a href="https://www.starburst.io/info/oreilly-trino-guide/">get a free digital copy of Trino: The Definitive Guide from
Starburst</a>, use snippets in
the <a href="https://github.com/trinodb/trino-the-definitive-guide">book examples
repository</a> to play
around with Trino and SQL, and let me know of any issues you find.</p>
<h2 id="join-starburst">Join Starburst</h2>
<p>And one last tip: If you are interested to work with Pawel, Szymon, Mateusz,
Brian, Jan, Brian, myself and many, many other great people at Starburst, you should
check out our careers page. We need your help and <a href="https://www.starburst.io/careers/" target="_blank">we are
hiring!</a></p>
<p><em>Manfred</em></p>Starburst was founded all around the Trino project and community, and it shows! The passion to work on Trino, Java, and other open source projects permeates everything we do. Let me tell you about some of these efforts. Specifically, I wanted to let you know about the results of my latest efforts of working late in the night under the moon - the second edition of Trino: The Definitive Guide is out! Get a free copy from Starburst now!Starburst Enterprise 398-e STS2022-10-04T00:00:00-05:002022-10-04T00:00:00-05:00https://docs.starburst.io/blog/sep-398-e<p>Starburst Enterprise platform (SEP) 398-e STS has been released as a follow-up
to the <a href="/blog/2022-09-29-sep-397-e.html">397-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-398.html">Trino 398</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>New Hudi connector.</li>
<li>Improved performance for Parquet data in Delta Lake, Hive, and Iceberg
connectors.</li>
<li>Support for column comments in Accumulo connector.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">timestamp</code> type in Pinot connector.</li>
</ul>
<p>For more information, refer to the <a href="../latest/release/release-398-e.html">release notes for
398-e</a>.</p>
<p><em>Jessie and Manfred</em></p>Starburst Enterprise platform (SEP) 398-e STS has been released as a follow-up to the 397-e STS release.Starburst Enterprise 397-e STS2022-09-29T00:00:00-05:002022-09-29T00:00:00-05:00https://docs.starburst.io/blog/sep-397-e<p>Starburst Enterprise platform (SEP) 397-e STS has been released as a follow-up
to the <a href="/blog/2022-09-26-sep-396-e.html">396-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-397.html">Trino 397</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>S3 Select pushdown for JSON data in Hive connector.</li>
<li>Faster <code class="language-plaintext highlighter-rouge">date_trunc</code> predicates over partition columns in Iceberg connector.</li>
<li>Reduced query latency with Glue catalog in Iceberg connector.</li>
</ul>
<p>For more information, refer to the <a href="../latest/release/release-397-e.html">release notes for
397-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 397-e STS has been released as a follow-up to the 396-e STS release.Starburst Enterprise 396-e STS2022-09-26T00:00:00-05:002022-09-26T00:00:00-05:00https://docs.starburst.io/blog/sep-396-e<p>Starburst Enterprise platform (SEP) 396-e STS has been released as a follow-up
to the <a href="/blog/2022-09-22-sep-395-e.html">395-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-396.html">Trino 396</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Improved performance when processing strings.</li>
<li>Faster writing of array, map and row types to Parquet.</li>
<li>Support for pushing down complex join criteria to connectors.</li>
<li>Support for column and table comments in BigQuery connector.</li>
</ul>
<p>For more information, refer to the <a href="../latest/release/release-396-e.html">release notes for
396-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 396-e STS has been released as a follow-up to the 395-e STS release.Starburst Enterprise LTS backport releases2022-09-26T00:00:00-05:002022-09-26T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-393-e.html">393-e.2</a></li>
<li><a href="../latest/release/release-380-e.html">380-e.6</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.10</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.14</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 395-e STS2022-09-22T00:00:00-05:002022-09-22T00:00:00-05:00https://docs.starburst.io/blog/sep-395-e<p>Starburst Enterprise platform (SEP) 395-e STS has been released as a follow-up
to the <a href="/blog/2022-09-12-sep-394-e.html">394-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-395.html">Trino 395</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Faster <code class="language-plaintext highlighter-rouge">INSERT</code> queries.</li>
<li>Better performance for large clusters.</li>
<li>Improved memory efficiency for aggregations and fault tolerant execution.</li>
<li>Faster aggregations over <code class="language-plaintext highlighter-rouge">DECIMAL</code> columns.</li>
<li>Support for dynamic function resolution.</li>
</ul>
<p>In SEP we are continuing to improve data products and the SEP web UI with bug
fixes and minor enhancements.</p>
<p>For more information, refer to the <a href="../latest/release/release-395-e.html">release notes for
395-e</a>.</p>
<p><em>Shea and Manfred</em></p>Starburst Enterprise platform (SEP) 395-e STS has been released as a follow-up to the 394-e STS release.Bleeding edge Java2022-09-20T00:00:00-05:002022-09-20T00:00:00-05:00https://docs.starburst.io/blog/bej0-introduction<p>In honor of the release of <a href="https://www.oracle.com/java/" target="_blank">Java 19</a>,
we present this series of blog posts on how to use the latest <em>bleeding
edge</em> features available in the Java programming language. Here at
<a href="https://www.starburst.io/careers/" target="_blank">Starburst</a>, Java is our primary language.
<a href="https://trino.io/" target="_blank">Trino</a> is written in Java as are most of our backend
services. Unlike many companies, Starburst is not stuck on an older version of
Java.</p>
<!--more-->
<p>Trino, in fact, was recently updated to require Java 17. The Java language has
evolved so dramatically since its inception that code written 20 years ago
hardly resembles modern code written today.</p>
<p>Many of the Java libraries we use today were written a long time ago and must
also support old versions of Java lest they wreak havoc with their user
communities. <em><strong>Imagine if you could rewrite some of your favorite libraries
with a fresh usage of modern Java?</strong></em> In this series, we do exactly that. The
goal is not to replace existing libraries - longstanding libraries shouldn’t be
replaced merely for aesthetic reasons. Our purpose is take a familiar use-case
and see how it might be implemented using the very latest, even experimental,
features in Java.</p>
<p>Let’s take a common use-case that nearly all Java developers have to deal with -
<a href="https://www.json.org/json-en.html" target="_blank">JSON</a>. The standard library for JSON in Java
is <a href="https://github.com/FasterXML/jackson" target="_blank">Jackson</a>. It would be futile to
attempt to emulate the performance and features of Jackson. Instead, let’s have
a simple goal of being able to write a Java object into valid JSON text and
then read valid JSON text and map it to a Java object, so serializing and
deserializing.</p>
<h2 id="roadmap">Roadmap</h2>
<p>In this series you can learn about numerous modern Java features:</p>
<ul>
<li><strong>Records</strong> - effectively use Java records for concise code that eliminates
boilerplate as well as enabling pattern matching and de-structuring</li>
<li><strong>Sealed interfaces</strong> - close a hierarchy of related classes to express a
pre-defined set of related types</li>
<li><strong>Interfaces as pseudo companion classes</strong> - a Java <code class="language-plaintext highlighter-rouge">class</code> definition is a
low level concept and is usually not necessary at the top level</li>
<li><strong>Record patterns</strong> - records are not just data carriers. They enable
functional idioms that in the past were only available in languages such as
Scala and Haskell.</li>
<li><strong>Pattern matching for switch</strong> - the union of record patterns and switch
makes code much easier to reason about and to write</li>
</ul>
<p>In addition, here are some goals and design decisions:</p>
<ul>
<li><strong>No magic</strong> - no hacks or tricks should be used. Only standard, modern Java.</li>
<li><strong>No annotations</strong> - serialization libraries have tended to rely on Java’s
annotation processing but this can be error prone and difficult to maintain
and debug</li>
<li><strong>No special libraries or hacks for type information</strong> - surprisingly,
standard Java retains enough type information to write a serialization library
that supports generics</li>
<li><strong>No dependencies</strong> - the library should be self-contained and not rely on any
third party libraries</li>
</ul>
<h2 id="target-audience">Target audience</h2>
<p>This series is targeted at Java developers who are comfortable with Java 8, but
have not used or are not comfortable with newer features in Java. The code
samples in this series are not meant for production, but we would appreciate any bug
reports. A more full-featured version of the JSON library <a href="https://github.com/starburstdata/developer-blog-assets/tree/main/bleeding-edge-java/json" target="_blank">is available on
GitHub</a>.</p>
<h2 id="all-posts">All posts</h2>
<ul>
<li><a href="/blog_bej/2022-09-20-bej1-design.html">Designing a JSON model</a> - Use some newer features
of Java to model the JSON RFC.</li>
<li><a href="/blog_bej/2022-09-20-bej2-serialization.html">Serialization</a> - Serializing is simpler than deserializing.
Let’s start with that.</li>
<li><a href="/blog_bej/2022-09-20-bej3-printing.html">Printing</a> -
Converting a stream of JSON tokens into JSON text.</li>
<li><a href="/blog_bej/2022-09-20-bej4-parsing.html">Parsing</a> -
Parsing JSON text into tokens.</li>
<li><a href="/blog_bej/2022-09-20-bej5-deserialization.html">Deserialization</a> - The
harder part, deserialize JSON into Java objects.</li>
<li><a href="/blog_bej/2022-09-20-bej6-deserialization.html">Deserialization of complex types</a> - Finalizing the system into a usable,
extensible library.</li>
<li><a href="/blog_bej/2022-09-20-bej7-conclusion.html">Conclusion</a> - Summarize what we achieved together.</li>
</ul>
<h2 id="were-hiring">We’re hiring</h2>
<p>Want to be able to use the latest features of Java? <a href="https://www.starburst.io/careers/" target="_blank">We’re
hiring!</a></p>
<hr />
<p>Jordan Zimmerman is a Senior Software Engineer working on
<a href="https://www.starburst.io/platform/starburst-galaxy/" target="_blank">
<img src="../assets/img/logo/sg-light-background.png" alt="Starburst Galaxy" />
</a>.</p>In honor of the release of Java 19, we present this series of blog posts on how to use the latest bleeding edge features available in the Java programming language. Here at Starburst, Java is our primary language. Trino is written in Java as are most of our backend services. Unlike many companies, Starburst is not stuck on an older version of Java.Starburst Enterprise 394-e STS2022-09-12T00:00:00-05:002022-09-12T00:00:00-05:00https://docs.starburst.io/blog/sep-394-e<p>Starburst Enterprise platform (SEP) 394-e STS has been released as a follow-up
to the <a href="/blog/2022-08-31-sep-393-e.html">393-e LTS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-394.html">Trino 394</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>JSON output format for <code class="language-plaintext highlighter-rouge">EXPLAIN</code>.</li>
<li>Improved performance for <code class="language-plaintext highlighter-rouge">LIKE</code> expressions.</li>
<li>Query table function support in the BigQuery connector.</li>
<li><code class="language-plaintext highlighter-rouge">INSERT</code> support in the BigQuery connector.</li>
<li>TLS support in the Pinot connector.</li>
</ul>
<p>SEP highlights:</p>
<ul>
<li>Add support for running multiple queries in a single
<a href="../latest/client/query-editor.html">Query editor</a> tab.</li>
<li>Improve the query editor’s prettify tool to support Trino SQL syntax.</li>
</ul>
<p>For more information, refer to the <a href="../latest/release/release-394-e.html">release notes for
394-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 394-e STS has been released as a follow-up to the 393-e LTS release.September 2022 Starburst Enterprise LTS backport releases2022-09-09T00:00:00-05:002022-09-09T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-380-e.html">380-e.5</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.9</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.13</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Terry, Joe, and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available.Starburst Enterprise 393-e LTS2022-08-31T00:00:00-05:002022-08-31T00:00:00-05:00https://docs.starburst.io/blog/sep-393-e<p>Another quarter, another <a href="../latest/versions.html#lts-sts">long term support (LTS)</a>
release! The new Starburst Enterprise 393-e LTS includes the latest batch of
features for our customers. There’s never a better time to upgrade your cluster,
so read on for details of what’s included.</p>
<!--more-->
<p>The 393-e LTS marks the first LTS release that requires Java 17, bringing better
performance, efficiency, and an improved codebase for our engineers to work
with. This upgrade allows us to build new features that Java 11 would not be
able to support, so we can consider the latest and greatest technologies when
designing new features.</p>
<p>The <a href="../latest/connector/iceberg.html">Iceberg connector</a> has been steadily
improving over time, and as of this LTS release Iceberg catalogs now fully
support CRUD operations as well as the Iceberg v2 specification. This includes
support of the newly-added <a href="../latest/sql/merge.html"><code class="language-plaintext highlighter-rouge">SQL MERGE</code></a> statement,
which has also been added to other connectors including <a href="../latest/connector/delta-lake.html">Delta
Lake</a> and <a href="../latest/connector/hive.html">Hive</a>.</p>
<p>Starburst Enterprise’s <a href="../latest/security/biac-overview.html">built-in access control</a>
has been getting steady improvements as well, with support for explicit deny
policies and granular control over the newly-added <a href="../latest/functions/table.html">table
functions</a>. In addition, the web interface now
features an option to download the audit log into a comma-separated values
(<code class="language-plaintext highlighter-rouge">.csv</code>) file for further review.</p>
<p>These are just some of the highlights that have made it into this quarterly LTS
release. For a complete list of features and changes, read the <a href="../latest/release/release-393-e.html">393-e LTS
release notes</a>.</p>
<p>If you’re upgrading to 393-e from Starburst Enterprise 389-e or earlier, you
should read our new <a href="../latest/appendix/to-sep-390-plus.html">migration guide</a>
with step-by-step instructions on how to prepare your cluster for Java 17.</p>
<p>As always, we’re continuing to add and improve features for the next STS
release, and we eagerly await your feedback on the release. Cheers!</p>
<p><em>Joe and Manfred</em></p>Another quarter, another long term support (LTS) release! The new Starburst Enterprise 393-e LTS includes the latest batch of features for our customers. There’s never a better time to upgrade your cluster, so read on for details of what’s included.Starburst Admin 1.5.1 release2022-08-17T00:00:00-05:002022-08-17T00:00:00-05:00https://docs.starburst.io/blog/starburst-admin-updates<p>Announcing the recent Starburst Admin releases 1.5.0 and 1.5.1, which
follow-on to release 1.4.0. In this post, we cover what’s new since 1.4.0.</p>
<!--more-->
<p>Starburst Admin is the best tool choice for folks looking to install
Starburst Enterprise platform (SEP) on bare metal or in your virtual machines. Recent
updates include:</p>
<ul>
<li>Adjust Java version check to support Java 17 for release version 390-e and
higher.</li>
<li>Update default JVM configuration for Java 17 for release versions 390-e
and higher.</li>
<li>Add the ability to install Starburst Admin as a non-root user.</li>
</ul>
<h2 id="java-17-changes">Java 17 changes</h2>
<p>Java requirements change based on the SEP version you are
installing. If you are not sure which Java version is required, refer the
<a href="../latest/installation/deployment.html#requirements">requirements topic</a>.</p>
<p>There are additional configuration properties included in the JVM
configuration. Use the configuration file included with this
new release of Starburst Admin to ensure your installation completes
successfully.</p>
<h2 id="install-as-a-non-root-user">Install as a non-root user</h2>
<p>You can now install using Starburst Admin as a non-root user. This
requires specific changes to the <code class="language-plaintext highlighter-rouge">playbooks/vars.yml</code> file.</p>Announcing the recent Starburst Admin releases 1.5.0 and 1.5.1, which follow-on to release 1.4.0. In this post, we cover what’s new since 1.4.0.Starburst Enterprise 392-e STS2022-08-08T00:00:00-05:002022-08-08T00:00:00-05:00https://docs.starburst.io/blog/sep-392-e<p>Starburst Enterprise platform (SEP) 392-e STS has been released as a follow-up
to the <a href="/blog/2022-07-26-sep-391-e.html">391-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-392.html">Trino 392</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for dynamic filtering with fault-tolerant query execution.</li>
<li>Support for correlated subqueries in <code class="language-plaintext highlighter-rouge">DELETE</code> queries.</li>
<li>Support for Amazon S3 Select pushdown for JSON files.</li>
<li>Support for Avro format in Iceberg connector.</li>
<li>Faster queries when filtering by <code class="language-plaintext highlighter-rouge">__time</code> column in Druid.</li>
</ul>
<p>SEP highlights:</p>
<ul>
<li>Add predicate pushdown support to the Starburst Teradata connectors for SQL
statements using <code class="language-plaintext highlighter-rouge">IN</code>.</li>
<li>Add a function to the Starburst Teradata connectors that flushes the JDBC
metadata cache.</li>
</ul>
<p>For more information, refer to the <a href="../latest/release/release-392-e.html">release notes for
392-e</a>.</p>
<p><em>Terry, Joe, and Manfred</em></p>Starburst Enterprise platform (SEP) 392-e STS has been released as a follow-up to the 391-e STS release.Use Metabase as a Starburst BI tool2022-08-03T00:00:00-05:002022-08-03T00:00:00-05:00https://docs.starburst.io/blog/metabase-client<p><a href="https://www.metabase.com">Metabase</a> can now be used to connect to
SEP, Starburst Galaxy, and Trino as a BI tool and
client.</p>
<p>Metabase excels at providing BI insights to users without needing to write a
single line of SQL, making the tool a perfect fit for the Trino
query engine that connects you to all of your data, wherever it lives.</p>
<!--more-->
<p>With the newly-developed <a href="https://github.com/starburstdata/metabase-driver/">Starburst partner driver for
Metabase</a>, you can easily
connect Metabase Cloud, On-Prem, or Open Source to your cluster. This
lets you take advantage of Metabase’s data exploration tools with the
scalability and performance of a world-class query engine.</p>
<p><img src="../assets/img/general/metabase-dashboard.png" alt="Example of an interactive dashboard in Metabase" class="img-fluid img-screenshot" description="" width="800px" style="" /></p>
<p>The Metabase Starburst driver is released as an open-source project.
You can contribute code directly to the driver’s GitHub repository, or report
issues and make feature requests on the driver <a href="https://github.com/starburstdata/metabase-driver/issues">issue
tracker</a>.</p>
<p>To get started using Metabase with your Starburst or Trino
cluster, check out the <a href="../data-consumer/clients/metabase.html">Metabase client guide</a>
for setup instructions and more information.</p>
<p><em>Joe and Piotr</em></p>Metabase can now be used to connect to SEP, Starburst Galaxy, and Trino as a BI tool and client. Metabase excels at providing BI insights to users without needing to write a single line of SQL, making the tool a perfect fit for the Trino query engine that connects you to all of your data, wherever it lives.August 2022 Starburst Enterprise LTS backport releases2022-08-01T00:00:00-05:002022-08-01T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Another batch of backport releases for our long-term support (LTS)
versions of Starburst Enterprise platform (SEP) is now available.</p>
<p><strong>Important</strong>: In these releases, we’ve patched a bug that causes the potential
for incorrect results from certain queries that have had joins reordered in
specific ways, in combination with certain data. A small percentage of queries
are affected, but due to the complexity of the conditions that trigger the bug,
it is not possible to predict which queries will be affected and thus we
recommend that <em>all</em> customers upgrade all clusters.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-380-e.html">380-e.3</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.7</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.11</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.15</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Terry and Manfred</em></p>Another batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise platform (SEP) is now available. Important: In these releases, we’ve patched a bug that causes the potential for incorrect results from certain queries that have had joins reordered in specific ways, in combination with certain data. A small percentage of queries are affected, but due to the complexity of the conditions that trigger the bug, it is not possible to predict which queries will be affected and thus we recommend that all customers upgrade all clusters.Starburst Enterprise 391-e STS2022-07-26T00:00:00-05:002022-07-26T00:00:00-05:00https://docs.starburst.io/blog/sep-391-e<p>Starburst Enterprise platform (SEP) 391-e STS has been released as a follow-up
to the <a href="/blog/2022-07-21-sep-390-e.html">390-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-391.html">Trino 391</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for AWS Athena partition projection.</li>
<li>Faster writing of Parquet data in Iceberg and Delta Lake.</li>
<li>Support for reading BigQuery external tables.</li>
<li>Support for table and column comments in BigQuery.</li>
</ul>
<p>SEP highlights:</p>
<ul>
<li>Add a feature to clone data products.</li>
<li>Add support for multiple databases in the Snowflake connector.</li>
</ul>
<p>As a reminder, Trino 390 introduced a raised requirement for the Java runtime
version. Starburst Enterprise platform (SEP) 391-e therefore requires a 64-bit version of
Java 17, with a <a href="../latest/installation/deployment.html#jvm-config">minimum required version of
17.0.3</a>.</p>
<p>For more information, refer to the <a href="../latest/release/release-391-e.html">release notes for
391-e</a>.</p>
<p><em>Shea and Manfred</em></p>Starburst Enterprise platform (SEP) 391-e STS has been released as a follow-up to the 390-e STS release.Starburst Enterprise 390-e STS2022-07-21T00:00:00-05:002022-07-21T00:00:00-05:00https://docs.starburst.io/blog/sep-390-e<p>Starburst Enterprise platform (SEP) 390-e STS has been released as a follow-up
to the <a href="/blog/2022-07-13-sep-389-e.html">389-e STS release</a>.</p>
<p>Trino 390 introduces a raised requirement for the Java runtime version.
Starburst Enterprise platform (SEP) 390-e therefore requires a 64-bit version of Java 17,
with a minimum required version of 17.0.3.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-390.html">Trino 390</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for setting comments on views.</li>
<li>Improved UNNEST performance.</li>
<li>Support for Databricks runtime 10.4 LTS in Delta Lake connector.</li>
</ul>
<p>SEP highlights:</p>
<ul>
<li>Added a feature to download the built-in access control Audit log into a
comma-separated values (<code class="language-plaintext highlighter-rouge">.csv</code>) file.</li>
<li>Add support for access control for table functions with the built-in access
control.</li>
</ul>
<p>For more information, refer to the <a href="../latest/release/release-390-e.html">release notes for
390-e</a>.</p>
<p><em>Terry and Manfred</em></p>Starburst Enterprise platform (SEP) 390-e STS has been released as a follow-up to the 389-e STS release. Trino 390 introduces a raised requirement for the Java runtime version. Starburst Enterprise platform (SEP) 390-e therefore requires a 64-bit version of Java 17, with a minimum required version of 17.0.3.Starburst Enterprise 389-e STS2022-07-13T00:00:00-05:002022-07-13T00:00:00-05:00https://docs.starburst.io/blog/sep-389-e<p>Starburst Enterprise platform (SEP) 389-e STS has been released as a follow-up
to the <a href="/blog/2022-07-07-sep-388-e.html">388-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-389.html">Trino 389</a>, which
includes the following noteworthy change:</p>
<ul>
<li>Improved performance for <code class="language-plaintext highlighter-rouge">row</code> type and aggregation.</li>
<li>Faster joins when spilling to disk is disabled.</li>
<li>Improved performance when writing non-structural types to Parquet.</li>
<li>New <code class="language-plaintext highlighter-rouge">raw_query</code> table function for full query pass-through in Elasticsearch.</li>
</ul>
<p>The highlight of the 389-e release is that data product users can now change the
type of any
<a href="../latest/data-products/manage-data-products.html#edit-or-remove-a-published-data-product">dataset</a>.</p>
<p>For more information, refer to the <a href="../latest/release/release-389-e.html">release notes for
389-e</a>.</p>
<p><em>Rose and Manfred</em></p>Starburst Enterprise platform (SEP) 389-e STS has been released as a follow-up to the 388-e STS release.Starburst Enterprise 388-e STS2022-07-07T00:00:00-05:002022-07-07T00:00:00-05:00https://docs.starburst.io/blog/sep-388-e<p>Starburst Enterprise platform (SEP) 388-e STS has been released as a follow-up
to the <a href="/blog/2022-06-30-sep-387-e.html">387-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-388.html">Trino 388</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for JSON output in <code class="language-plaintext highlighter-rouge">EXPLAIN</code>.</li>
<li>Improved performance for row data types.</li>
<li>Support for OAuth 2.0 refresh tokens.</li>
<li>Support for table and column comments in Delta Lake.</li>
</ul>
<p>In 388-e, we’ve released public preview support for <a href="../latest/admin/fault-tolerant-execution.html">fault-tolerant
execution</a>. Note that this
release also ends support for MapR-based Hive metastores and the MapR
filesystem.</p>
<p>For more information, refer to the <a href="../latest/release/release-388-e.html">release notes for
388-e</a>.</p>
<p><em>Joe and Manfred</em></p>Starburst Enterprise platform (SEP) 388-e STS has been released as a follow-up to the 387-e STS release.Announcing the 2022 Trino Summit2022-07-06T00:00:00-05:002022-07-06T00:00:00-05:00https://docs.starburst.io/blog/trino-summit<p>It’s happening! Starburst is excited to act as the lead sponsor for the 2022
Trino Summit.</p>
<!--more-->
<p>Join us online or in-person on November 10th, 2022 at the Commonwealth Club in
San Francisco as we gather the minds of those with one common passion:
<strong>Trino</strong>. You can find all the relevant information on the <a href="https://trino.io/blog/2022/06/30/trino-summit-call-for-speakers.html">Trino
blog</a>,
including details about registration, call for speakers, and sponsorship
opportunities.</p>
<p>In addition to hearing from some outstanding speakers, this year’s Trino Summit
will bring together Trino experts and community members for a day of insightful
discussion and Trino celebration.</p>
<p>We believe in Trino as a technology, so much so that we based our business on
it. We also believe that a healthy community fosters innovation and
collaboration; therefore, Starburst is committed to growing and enriching the
Trino community.</p>
<p>Starburst wants to help facilitate a successful 2022 Trino Summit and offer our
support to the Trino Software Foundation so that we can work together to plan
the best community event possible. If you or your employer have any interest
in helping us throw this awesome event, please <a href="mailto:events@starburst.io">contact our events team for more
information</a>.</p>
<p>We know you are excited. We are too. To hold you over until the summit, go
explore <a href="https://www.starburst.io/platform/starburst-galaxy/">Starburst Galaxy</a>,
the cloud native and fully managed high performant query engine.</p>
<p>We look forward to seeing you in November, either in-person or online!</p>
<p><em>Monica</em></p>It’s happening! Starburst is excited to act as the lead sponsor for the 2022 Trino Summit.July 2022 Starburst Enterprise LTS backport releases2022-07-01T00:00:00-05:002022-07-01T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Hey all, another release batch for our long-term support (LTS)
versions of Starburst Enterprise is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-380-e.html">380-e.2</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.6</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.10</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.14</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Rose and Manfred</em></p>Hey all, another release batch for our long-term support (LTS) versions of Starburst Enterprise is now available.Starburst Enterprise 387-e STS2022-06-30T00:00:00-05:002022-06-30T00:00:00-05:00https://docs.starburst.io/blog/sep-387-e<p>Starburst Enterprise platform (SEP) 387-e STS has been released as the follow up
release to <a href="/blog/2022-06-21-sep-386-e.html">386-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-387.html">Trino 387</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for writing ORC Bloom filters for <code class="language-plaintext highlighter-rouge">varchar</code> columns.</li>
<li>Support for querying Pinot via the gRPC endpoint.</li>
<li>Support for predicate pushdown on string columns in Redis.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">OPTIMIZE</code> on Iceberg tables with non-identity partitioning.</li>
</ul>
<p>Be aware that Trino 387 introduces a raised requirement for the Java runtime
version. SEP 387-e therefore requires Java 11.0.15 or higher.</p>
<p>For more information, refer to the <a href="../latest/release/release-387-e.html">release notes for
387-e</a>.</p>
<p><em>Shea and Manfred</em></p>Starburst Enterprise platform (SEP) 387-e STS has been released as the follow up release to 386-e STS release.Starburst Galaxy adds support for Azure features2022-06-29T00:00:00-05:002022-06-29T00:00:00-05:00https://docs.starburst.io/blog/sg-update-synapse-serviceprince<p>Starburst Galaxy is delighted to announce the addition of <a href="https://azure.microsoft.com/en-us/services/synapse-analytics/">Azure
Synapse</a> as a
supported data source. To connect your existing Synapse account, click
<strong>Catalogs</strong> in the navigation menu, then click <strong>Configure a catalog</strong>. Select
the <strong>Azure Synapse</strong> tile, then provide the connection details for your
account. Follow the steps in <a href="../starburst-galaxy/catalogs/synapse.html">Azure Synapse
catalogs</a>.</p>
<p>Starburst Galaxy also now supports authenticating your Azure Data Lake Storage
(ADLS) catalogs with an Azure service principal.</p>
<!--more-->
<p>Service principals are Azure’s way to provide fine-grained access to resources.
This change adds a second authentication method for ADLS data sources after the
existing Azure access keys method.</p>
<p>Start with <a href="../starburst-galaxy/security/external-azure.html">External security in
Azure</a> to prepare your Azure
account to use a service principal. Then see <a href="../starburst-galaxy/cloud-settings/azure.html">Cloud settings for
Azure</a> to configure Starburst
Galaxy with a local alias for the service principal and validate a test
connection with it. Thereafter, you can use the alias to validate connections
with multiple ADLS catalogs.</p>
<p>We hope you find these new features useful, and look forward to your feedback.
In the meantime we have a few other exciting new features already lined up. Stay
tuned.</p>
<p><strong>Barton and Kamran</strong></p>Starburst Galaxy is delighted to announce the addition of Azure Synapse as a supported data source. To connect your existing Synapse account, click Catalogs in the navigation menu, then click Configure a catalog. Select the Azure Synapse tile, then provide the connection details for your account. Follow the steps in Azure Synapse catalogs. Starburst Galaxy also now supports authenticating your Azure Data Lake Storage (ADLS) catalogs with an Azure service principal.Starburst Enterprise 386-e STS2022-06-21T00:00:00-05:002022-06-21T00:00:00-05:00https://docs.starburst.io/blog/sep-386-e<p>Starburst Enterprise platform (SEP) 386-e STS has been released as the follow up
release to <a href="/blog/2022-06-14-sep-385-e.html">385-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-386.html">Trino 386</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Improved performance for fault-tolerant query execution</li>
<li>Faster queries on Delta Lake</li>
</ul>
<p>For more information, head over to the <a href="../latest/release/release-386-e.html">release notes for
386-e</a>.</p>
<p><em>Shea and Manfred</em></p>Starburst Enterprise platform (SEP) 386-e STS has been released as the follow up release to 385-e STS release.Starburst Enterprise 385-e STS2022-06-14T00:00:00-05:002022-06-14T00:00:00-05:00https://docs.starburst.io/blog/sep-385-e<p>Starburst Enterprise platform (SEP) 385-e STS has been released as the follow up
release to <a href="/blog/2022-06-01-sep-382-e.html">382-e STS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-385.html">Trino 385</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>New <code class="language-plaintext highlighter-rouge">json_array</code> and <code class="language-plaintext highlighter-rouge">json_object</code> functions.</li>
<li>Support for time travel syntax in the Iceberg connector.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">timestamp(p)</code> type in MariaDB connector.</li>
<li>Performance improvements in Iceberg connector.</li>
</ul>
<p>It also includes the changes from <a href="../latest/release/release-384.html">Trino
384</a> and <a href="../latest/release/release-383.html">Trino
383</a>:</p>
<ul>
<li>Support for new pass-through query table function for Druid, MariaDB, MySQL,
Oracle, PostgreSQL, Redshift, SingleStore, and SQL Server.</li>
<li>New <code class="language-plaintext highlighter-rouge">json_exists</code>, <code class="language-plaintext highlighter-rouge">json_query</code>, and <code class="language-plaintext highlighter-rouge">json_value</code> functions.</li>
<li>Support for table comments in Delta Lake.</li>
<li>Support IAM roles for exchange spooling on S3.</li>
<li>Improved performance for aggregation queries.</li>
</ul>
<p>The 383-e STS and 384-e STS releases were skipped.</p>
<p>The highlight of the 385-e release is the <a href="../latest/security/immuta-overview.html">Immuta access control
support</a>. Immuta access control has
been elevated from public preview to general availability, and is now ready for
production.</p>
<p>For more information, head over to the <a href="../latest/release/release-385-e.html">release notes for
385-e</a>.</p>
<p><em>Terry and Manfred</em></p>Starburst Enterprise platform (SEP) 385-e STS has been released as the follow up release to 382-e STS release.June 2022 Starburst Enterprise LTS backport releases2022-06-07T00:00:00-05:002022-06-07T00:00:00-05:00https://docs.starburst.io/blog/sts-backports<p>Hey all, another release batch for our long-term support (LTS)
versions of Starburst Enterprise is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-380-e.html">380-e.1</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.5</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.9</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Rose and Manfred</em></p>Hey all, another release batch for our long-term support (LTS) versions of Starburst Enterprise is now available.Starburst Enterprise 382-e STS2022-06-01T00:00:00-05:002022-06-01T00:00:00-05:00https://docs.starburst.io/blog/sep-382-e<p>Starburst Enterprise platform (SEP) 382-e STS is available for you all. And
again we bring some exciting new features from Trino and our Starburst
improvements.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-382.html">Trino 382</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Exchange spooling on Google Cloud Storage.</li>
<li>Support for reading wildcard tables in BigQuery connector.</li>
<li>Support for adding columns in Delta Lake connector.</li>
<li>Support updating Iceberg table partitioning.</li>
<li>Improved <code class="language-plaintext highlighter-rouge">INSERT</code> performance in MySQL, Oracle and PostgreSQL connectors.</li>
<li>Basic authentication in Prometheus connector.</li>
</ul>
<p>Specifically you can now test out <a href="../latest/admin/fault-tolerant-execution.html">fault-tolerant
execution</a> with exchange spooling
on all three major cloud providers. We have been <a href="https://trino.io/blog/2022/05/05/tardigrade-launch.html">working tirelessly on this
feature set in Trino</a>,
and are looking forward to offer this as a fully supported features to our SEP
users in the future. Go ahead and try it out, and let us know of any features
you miss, or issues you run into.</p>
<p>Since 380-e our built-in access control and data products are fully supported
features. However we certainly did not stop to improve. Notably the
documentation for the <a href="../latest/data-products/api.html">REST API</a> for data
products <a href="../latest/api/index.html">looks much nicer now and is also much more detailed and
informative</a>. And if you are a heavy user of built-in
access control, you might want to read about the new support for SQL <code class="language-plaintext highlighter-rouge">DENY</code>
statements.</p>
<p>As query editor user you are going to love the ability to run a query based on
the position of your cursor. You can just type a query and press run. No more
need to highlight the full query!</p>
<p>Kubernetes users can now use the <a href="../latest/k8s/requirements.html">newer 1.22 and 1.23
versions</a> for their deployment, while users of
the unsupported 1.18 have to upgrade.</p>
<p>As usual head over to the <a href="../latest/release/release-382-e.html">release notes and documentation for
382-e</a> for more details.</p>
<p><em>Manfred and Matt</em></p>Starburst Enterprise platform (SEP) 382-e STS is available for you all. And again we bring some exciting new features from Trino and our Starburst improvements.Starburst Enterprise 381-e STS2022-05-24T00:00:00-05:002022-05-24T00:00:00-05:00https://docs.starburst.io/blog/sep-381-e<p>Starburst Enterprise platform (SEP) 381-e STS has been released, a rapid
follow-up to our <a href="/blog/2022-05-23-sep-380-e.html">380-e LTS release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-381.html">Trino 381</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Support for <code class="language-plaintext highlighter-rouge">UPDATE</code> in the Iceberg connector.</li>
<li>Experimental support for table functions.</li>
<li>Support for exchange spooling on Azure Blob Storage.</li>
<li>Support for reading snapshot tables and materialized views in the BigQuery
connector.</li>
</ul>
<p>For more information, head over to the <a href="../latest/release/release-381-e.html">release
notes for 381-e</a>.</p>
<p><em>Joe and Matt</em></p>Starburst Enterprise platform (SEP) 381-e STS has been released, a rapid follow-up to our 380-e LTS release.Starburst Enterprise 380-e LTS2022-05-23T00:00:00-05:002022-05-23T00:00:00-05:00https://docs.starburst.io/blog/sep-380-e<p>Did you know that the 23rd of May is world turtle day?! We think that is
turtle-y awesome, and it’s a perfect excuse to celebrate our latest <a href="../latest/versions.html#lts-sts">long term
support (LTS)</a> release! With the new Starburst
Enterprise 380-e LTS, we are shipping the latest and greatest features for our
customers, who also want the availability of security and correctness backports
for the next year. It is upgrade time, and there are many good reasons to get
onto 380-e.0.</p>
<!--more-->
<p>In the recent STS releases <a href="/blog/2022-05-04-sep-379-e.html">379-e</a> and
<a href="/blog/2022-05-02-sep-378-e.html">378-e</a>, we added a number of delightful
features and updates. For your convenience, we have summarized all the changes
implemented since the 370-e LTS release on the 380-e.0 release page within the
following sections:</p>
<ul>
<li><a href="../latest/release/release-380-e.html#release-380-highlights">Highlights since 370-e</a></li>
<li><a href="../latest/release/release-380-e.html#release-380-breaking">Breaking changes since 370-e</a></li>
<li><a href="../latest/release/release-380-e.html#release-380-changes">380-e initial changes</a></li>
</ul>
<p>One of the most exciting updates is that <a href="../latest/data-products/index.html">data
products</a> and <a href="../latest/security/biac-overview.html">built-in access
control</a> are now available for production
use! Even better, built-in access control now allows you to edit roles and add
privileges for data products from the Starburst Enterprise web UI:</p>
<p><img src="../assets/img/general/ui-controls.png" alt="User interface controls in Starburst Enterprise web UI" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>The good news keeps on going for data products, the new <a href="../latest/data-products/api.html">REST
API</a> is out and ready. Other improvements
include audit log information for built-in access control, Markdown support in
the description and comments, and an updated edit workflow for data products.</p>
<p>For Kubernetes users we added the <a href="../latest/k8s/sep-configuration.html#keda-sep-scaler">Kubernetes-based Event Driven Autoscaler
(KEDA)</a>. This allows you
to configure SEP with an external KEDA scaler to adjust the number of workers
automatically based on JVM performance metrics available through JMX.</p>
<p>We are steadily improving features for the next STS release, feel free to reach
out to us with your feedback. Enjoy world turtle day!</p>
<p><em>Rose and Russell</em></p>Did you know that the 23rd of May is world turtle day?! We think that is turtle-y awesome, and it’s a perfect excuse to celebrate our latest long term support (LTS) release! With the new Starburst Enterprise 380-e LTS, we are shipping the latest and greatest features for our customers, who also want the availability of security and correctness backports for the next year. It is upgrade time, and there are many good reasons to get onto 380-e.0.May 2022 Starburst Enterprise LTS backport releases2022-05-20T00:00:00-05:002022-05-20T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Hey all, another release batch for our long-term support (LTS)
versions of Starburst Enterprise is now available.</p>
<!--more-->
<p>As usual, we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-370-e.html">370-e.4</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.8</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.12</a></li>
<li><a href="../latest/release/release-356-e.html">356-e.15</a></li>
</ul>
<p>The related Helm charts and other resources are all available as part of this
update.</p>
<p><em>Joe and Manfred</em></p>Hey all, another release batch for our long-term support (LTS) versions of Starburst Enterprise is now available.New release of the dbt-trino adapter2022-05-09T00:00:00-05:002022-05-09T00:00:00-05:00https://docs.starburst.io/blog/dbt-trino<p><a href="https://www.getdbt.com/">dbt labs</a> released version 1.1 of the <a href="https://github.com/dbt-labs/dbt-core">dbt-core
project</a> in late April, 2022. This did not
catch the maintainers of the
<a href="https://github.com/starburstdata/dbt-trino">dbt-trino</a> project by surprise. We
have been hard at work adding features and fixing bugs, and are proud to
announce version 1.1 of the dbt-trino adapter.</p>
<!--more-->
<p>Install the new version, and any necessary dependencies, with the following
command:</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>dbt-trino<span class="o">==</span>1.1.0
</code></pre></div></div>
<p>The most important new feature in dbt-trino 1.1 is support for <a href="../latest/security/oauth2.html">OAuth2
authentication</a> using a web browser. This lets
you log in to a cluster using the credentials from an OAuth2-compatible single
sign-on system. To use this feature, install the <code class="language-plaintext highlighter-rouge">keyring</code> package to cache the
OAuth2 token over multiple dbt invocations, and make sure you have the <code class="language-plaintext highlighter-rouge">trino</code>
package’s external auth token cache feature.</p>
<div class="language-shell highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install</span> <span class="s1">'trino[external-authentication-token-cache]'</span>
</code></pre></div></div>
<p>Other new features of dbt-trino 1.1 include:</p>
<ul>
<li>Improved data type mapping.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">on_table_exists</code> in table materialization.</li>
<li>Support for <code class="language-plaintext highlighter-rouge">view_security</code> to define the security mode for views.</li>
</ul>
<p>For those developing dbt-trino or extensions based on it, the 1.1 release
includes:</p>
<ul>
<li>Migration of <a href="https://github.com/starburstdata/dbt-trino">the GitHub project</a>
to a new adapter testing framework.</li>
<li>Adding a PostgreSQL Docker container for testing.</li>
</ul>
<p>See these pages for more on using dbt with Trino or Starburst.</p>
<ul>
<li><a href="../data-consumer/clients/dbt.html">Starburst documentation</a></li>
<li><a href="https://github.com/starburstdata/dbt-trino#readme">GitHub project’s README</a></li>
</ul>
<p><em>Barton, Przemek, Michiel, and Manfred</em></p>dbt labs released version 1.1 of the dbt-core project in late April, 2022. This did not catch the maintainers of the dbt-trino project by surprise. We have been hard at work adding features and fixing bugs, and are proud to announce version 1.1 of the dbt-trino adapter.Batch processing, Iceberg, and Delta Lake for Starburst Galaxy2022-05-04T00:00:00-05:002022-05-04T00:00:00-05:00https://docs.starburst.io/blog/sg-update<p>Starting in the first week of May, Starburst Galaxy includes two exciting new
features:</p>
<ul>
<li><a href="#great-lakes">Great Lakes connectivity</a></li>
<li><a href="#batch-mode">Batch mode clusters</a></li>
</ul>
<p>These features greatly expand the supported use cases for Starburst Galaxy, and
bring new performance benefits to everyone.</p>
<!--more-->
<h2 id="beyond-hive-with-delta-lake-or-iceberg-">Beyond Hive with Delta Lake or Iceberg <a name="great-lakes"></a></h2>
<p><a href="../starburst-galaxy/sql/great-lakes.html">Great Lakes connectivity</a> abstracts
the details of using different table formats and file types when using certain
write access statements for object storage systems. This connectivity is built
into Starburst Galaxy, and is available to all users.</p>
<p>You can now use any of the following object storage catalogs with the modern
table formats Delta Lake and Iceberg.</p>
<ul>
<li><a href="../starburst-galaxy/catalogs/s3.html">Amazon S3</a></li>
<li><a href="../starburst-galaxy/catalogs/adls.html">Azure Data Lake Storage</a></li>
<li><a href="../starburst-galaxy/catalogs/gcs.html">Google Cloud Storage</a></li>
</ul>
<p>This allows you to seamlessly migrate from the legacy Hive system to Iceberg or
Delta Lake. You can migrate all from within Starburst Galaxy, one table at the
time if desired. Once you are using Iceberg or Delta Lake, all the advantages of
these modern systems are available, such as improved performance, snapshots, and
more. At the same time, you can query all tables in your queries just as
before.</p>
<p>Great Lakes connectivity lets you configure the table format with a single
<code class="language-plaintext highlighter-rouge">type</code> parameter for <code class="language-plaintext highlighter-rouge">CREATE TABLE</code> or <code class="language-plaintext highlighter-rouge">CREATE TABLE AS</code> statements. Find out
more details for the different formats in the documentation:</p>
<p><a href="../starburst-galaxy/sql/great-lakes.html#delta-lake-type"><img src="../assets/img/logo/delta-lake.png" /></a>
<a href="../starburst-galaxy/sql/great-lakes.html#iceberg-type"><img src="../assets/img/logo/iceberg.png" /></a>
<a href="../starburst-galaxy/sql/great-lakes.html#hive-type"><img src="../assets/img/logo/hive.png" /></a></p>
<p>For example, use a <code class="language-plaintext highlighter-rouge">CREATE TABLE</code> statement like the following to specify using
the Iceberg table format and the Apache Parquet file format:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">TABLE</span> <span class="n">customer</span> <span class="p">(</span>
<span class="n">name</span> <span class="nb">varchar</span><span class="p">,</span>
<span class="n">address</span><span class="p">,</span> <span class="nb">varchar</span><span class="p">)</span>
<span class="k">WITH</span> <span class="p">(</span>
<span class="k">type</span><span class="o">=</span><span class="s1">'iceberg'</span><span class="p">,</span>
<span class="n">format</span><span class="o">=</span><span class="s1">'parquet'</span><span class="p">);</span>
</code></pre></div></div>
<h2 id="batch-and-etl-processing-for-everyone-">Batch and ETL processing for everyone <a name="batch"></a></h2>
<p>In an ideal world, analytics is instant and data is clean and consistent at the
source. In the real world however the amount of data you deal with every day is
huge. The data is distributed, includes inconsistencies, and running any
analytics can take a long time. That is why Extract, Transform, Load (ETL)
processes are still important. Processing these long-running queries is often a
critical process and failures have to be avoided.</p>
<p>With the new batch mode for Starburst Galaxy clusters, you now get access to a
very convenient way to activate fault-tolerant query execution using a simple
<a href="../starburst-galaxy/clusters/index.html#batch">cluster type selection</a>.</p>
<p><img src="../assets/img/galaxy/cluster-batch-mode.png" alt="Cluster batch mode selection" class="img-fluid img-screenshot" description="" width="365px" style="" /></p>
<p>With batch mode enabled, all queries on the cluster benefit from fault-tolerant
execution. Problems such as network issues accessing the source data, memory
overflows, or even partial cluster outages no longer cause the query to fail.
Even parts of a query can be reprocessed as necessary.</p>
<p>With this feature enabled on your cluster, usage of <a href="../data-consumer/clients/dbt.html">tools such as
dbt</a> can become a regular occurrence. This
allows you to automate further parts of your analytics pipeline. With Starburst
Galaxy you don’t have to worry about managing the infrastructure or operating
the clusters, and your query processing can be automated as well. You can spend
more time on understanding the data, such as with the help of the <a href="../starburst-galaxy/query/query-editor.html">built-in
query editor</a>.</p>
<p><a href="../starburst-galaxy/index.html">Try it out</a>. You are going to love it! And keep
an eye on the <a href="../starburst-galaxy/release-notes.html">release notes for Starburst
Galaxy</a>, because more great features are
on the way.</p>
<p><strong>Barton and Manfred</strong></p>Starting in the first week of May, Starburst Galaxy includes two exciting new features: Great Lakes connectivity Batch mode clusters These features greatly expand the supported use cases for Starburst Galaxy, and bring new performance benefits to everyone.Starburst Enterprise 379-e STS2022-05-04T00:00:00-05:002022-05-04T00:00:00-05:00https://docs.starburst.io/blog/sep-379-e<p>Starburst Enterprise platform (SEP) 379-e STS has been released, a rapid
follow-up to our <a href="/blog/2022-05-02-sep-378-e.html">378-e release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-379.html">Trino 379</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Added a new <a href="../latest/connector/mariadb.html">MariaDB connector</a>.</li>
<li>Performance improvements for <code class="language-plaintext highlighter-rouge">JOIN</code>, <code class="language-plaintext highlighter-rouge">UNION</code> and <code class="language-plaintext highlighter-rouge">GROUP BY</code>.</li>
<li>Support for Google Cloud Storage in the
<a href="../latest/connector/delta-lake.html">Delta Lake connector</a>.</li>
<li>Support for Pinot 0.10.</li>
</ul>
<p>In SEP, the following improvements were added to the
<a href="../latest/client/query-editor.html">Query editor</a>:</p>
<ul>
<li>Added autocomplete functionality for SQL functions.</li>
<li>Added inline documentation.</li>
<li>Added an option to download a full result set.</li>
</ul>
<p>In addition, the following improvement was added to SEP:</p>
<ul>
<li>Added user interface entities to built-in access control
<a href="../latest/security/biac-privileges.html">roles and privileges</a>.</li>
</ul>
<p>For more information, head over to the <a href="../latest/release/release-379-e.html">release
notes</a>.</p>
<p><em>Joe and Matt</em></p>Starburst Enterprise platform (SEP) 379-e STS has been released, a rapid follow-up to our 378-e release.Starburst Enterprise 378-e STS2022-05-02T00:00:00-05:002022-05-02T00:00:00-05:00https://docs.starburst.io/blog/sep-378-e<p>Starburst Enterprise platform (SEP) 378-e STS has been released, a rapid
follow-up to our <a href="/blog/2022-04-21-sep-376-e.html">376-e release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-378.html">Trino 378</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>New <code class="language-plaintext highlighter-rouge">to_base32</code> and <code class="language-plaintext highlighter-rouge">from_base32</code> functions.</li>
<li>New <code class="language-plaintext highlighter-rouge">expire_snapshots</code> and <code class="language-plaintext highlighter-rouge">delete_orphan_files</code> table procedures for
Iceberg.</li>
<li>Faster planning of queries with <code class="language-plaintext highlighter-rouge">IN</code> predicates.</li>
<li>Faster query planning for Hive, Delta Lake, Iceberg, MySQL, PostgreSQL, and
SQL Server connectors.</li>
</ul>
<p>In addition, the changes from <a href="../latest/release/release-377.html">Trino 377</a> are new:</p>
<ul>
<li>Add support for standard SQL <code class="language-plaintext highlighter-rouge">trim</code> syntax.</li>
<li>Better performance for Glue metastore.</li>
<li>Join pushdown for SQL Server connector.</li>
</ul>
<p>Specifically important to note is that a <a href="../latest/security/internal-communication.html">shared secret for internal
authentication</a> is now required
if you have any authentication configured.</p>
<p>In SEP, the following changes occurred:</p>
<ul>
<li>Improve predicate pushdown for <code class="language-plaintext highlighter-rouge">WHERE</code> clauses with the
<a href="../latest/connector/starburst-teradata.html">Teradata connector</a>.</li>
<li>You can now synchronize <code class="language-plaintext highlighter-rouge">EXTERNAL</code> <a href="../latest/security/ranger-overview.html">Ranger
groups</a> only when using
<code class="language-plaintext highlighter-rouge">ranger.user-group-source=STARBURST</code>.</li>
</ul>
<p>For more information, head over to the <a href="../latest/release/release-378-e.html">release
notes</a>.</p>
<p><em>Rose and Matt</em></p>Starburst Enterprise platform (SEP) 378-e STS has been released, a rapid follow-up to our 376-e release.April 2022 Starburst Enterprise LTS backport releases2022-04-25T00:00:00-05:002022-04-25T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Welcome back folks - checkout the latest batch of backport releases for our
long-term support (LTS) versions of Starburst Enterprise.</p>
<!--more-->
<p>We encourage everyone to check the release notes, and update at your earliest
convenience:</p>
<ul>
<li><a href="../latest/release/release-370-e.html">370-e.3</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.4</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.11</a></li>
<li><a href="../latest/release/release-356-e.html">356-e.14</a></li>
</ul>
<p>As usual the related Helm charts and other resources are all available as part
of this update.</p>
<p><strong>Rose and Manfred</strong></p>Welcome back folks - checkout the latest batch of backport releases for our long-term support (LTS) versions of Starburst Enterprise.Starburst Enterprise 376-e STS2022-04-21T00:00:00-05:002022-04-21T00:00:00-05:00https://docs.starburst.io/blog/sep-376-e<p>Starburst Enterprise platform (SEP) 376-e STS has been released, a rapid
follow-up to our <a href="/blog/2022-04-06-sep-375-e.html">375-e release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-376.html">Trino 376</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Better performance when reading Parquet data.</li>
<li>Join pushdown for MySQL.</li>
<li>Aggregation pushdown for Oracle.</li>
<li>Support table and column comments in the ClickHouse connector.</li>
<li>Support for adding and deleting schemas in the Accumulo connector.</li>
<li>Support system truststore in the CLI and the JDBC driver.</li>
<li>Add two-way TLS/SSL certificate validation with LDAP authentication.</li>
</ul>
<p>In SEP, the following changes occurred:</p>
<ul>
<li>JDBC connection pooling is now available in the
<a href="../latest/connector/starburst-netezza.html">Netezza connector</a>.</li>
<li>You can now leave out some optional fields for Ranger users when using
<a href="../latest/security/ranger-overview#sep-user-synchronization">SEP user synchronization</a>.</li>
</ul>
<p>For more information, head over to the <a href="../latest/release/release-376-e.html">release
notes</a>.</p>
<p><em>Joe and Matt</em></p>Starburst Enterprise platform (SEP) 376-e STS has been released, a rapid follow-up to our 375-e release.New AWS Marketplace offering for EKS users2022-04-08T00:00:00-05:002022-04-08T00:00:00-05:00https://docs.starburst.io/blog/eks-marketplace<p>What a great day it is for our AWS Marketplace users! Building on the success of
our CFT-based marketplace offering and the superior deployment platform EKS, we
bring you the brand new <a href="../ecosystems/amazon/aws-marketplace/aws-marketplace-eks.html">AWS Marketplace EKS
offering</a>.</p>
<!--more-->
<p>With EKS offered in the AWS marketplace, customers get to enjoy streamlined
configuration and installation, automated deployment, and billing through AWS.
The new EKS offering also includes a pay-as-you-go option that is extremely
convenient to get started and scale your usage on demand. It also takes into
account any downtimes in which you decide to shut off your clusters.</p>
<p>The whole marketplace offering is based on the mature tooling for enterprise
customer, who manage the deployment in the Kubernetes clusters themselves. The
documentation for these used Helm charts including requirements, examples, and
many useful tips and tricks is available in our <a href="../latest/k8s.html">documentation as
usual</a>.</p>
<p>The contract billing option even supports using the more performant and
cost-effective ARM-64 processor option.</p>
<p>We encourage all new marketplace customers to use the new EKS offering. Users of
the <a href="../ecosystems/amazon/aws-marketplace/aws-marketplace-cft.html">CFT
offering</a> might
want to give it a try and consider migrating.</p>
<p>Reach out if you need more help, or want to provide some feedback.</p>
<p><em>Jessie and Kevin</em></p>What a great day it is for our AWS Marketplace users! Building on the success of our CFT-based marketplace offering and the superior deployment platform EKS, we bring you the brand new AWS Marketplace EKS offering.Starburst Enterprise 375-e STS2022-04-06T00:00:00-05:002022-04-06T00:00:00-05:00https://docs.starburst.io/blog/sep-375-e<p>Starburst Enterprise platform (SEP) 375-e STS has been released, a rapid
follow-up to our <a href="/blog/2022-03-29-sep-374-e.html">374-e release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-375.html">Trino 375</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Improve performance of queries that contain filter predicates on aggregation
results.</li>
<li>Add access control to <code class="language-plaintext highlighter-rouge">drop_extended_stats</code> and <code class="language-plaintext highlighter-rouge">vacuum</code> procedures in the
Delta Lake connector.</li>
<li>Add support for predicate pushdown on boolean columns in the MongoDB
connector.</li>
</ul>
<p>In SEP, the following changes occurred:</p>
<ul>
<li>The <code class="language-plaintext highlighter-rouge">memsql</code> connector name is deprecated in SEP and
Trino, and is replaced with <code class="language-plaintext highlighter-rouge">singlestore</code>. Catalog property files
that use the <a href="../latest/connector/memsql.html">SingleStore connector</a> should
begin using these as soon as possible.</li>
<li>Starburst Hive is updated to version 3.1.2-e.9.</li>
</ul>
<p>For more information, head over to the <a href="../latest/release/release-375-e.html">release
notes</a>.</p>
<p><em>Annie and Matt</em></p>Starburst Enterprise platform (SEP) 375-e STS has been released, a rapid follow-up to our 374-e release.Starburst Admin 1.3.1 release2022-04-05T00:00:00-05:002022-04-05T00:00:00-05:00https://docs.starburst.io/blog/starburst-admin-update<p>We are delighted to announce the release of Starburst Admin 1.3.1, a follow-on
to release 1.3.0. In this post, we cover what’s new since 1.3.0.</p>
<!--more-->
<p>Starburst Admin is the best tool choice for folks looking to install Starburst
Enterprise platform (SEP) on bare metal or in your virtual machines. Recent
updates include:</p>
<ul>
<li>Allow user to specify a custom <code class="language-plaintext highlighter-rouge">vars.yml</code> file for Ansible playbooks to use.</li>
<li>Allow users to specify the location of custom application configuration files.</li>
<li>Enhanced clean up from previous installations.</li>
<li>Remove the deprecated <code class="language-plaintext highlighter-rouge">discovery-server.enabled</code> configuration property from
the <code class="language-plaintext highlighter-rouge">config.properties</code> file.</li>
</ul>
<h2 id="custom-varsyml-file">Custom <code class="language-plaintext highlighter-rouge">vars.yml</code> file</h2>
<p>To improve the flexibility of Starburst Admin, customers are now able to specify
a custom <code class="language-plaintext highlighter-rouge">vars.yml</code> file. This file contains all settings used by the install
playbooks to do things like define the version and define where the
configuration settings files can be found. No changes are required for you if
you are not yet ready to customize.</p>
<p>More advanced installations may customize the <code class="language-plaintext highlighter-rouge">vars.yml</code> file to tune the
installation process to meet their needs. These customizations can be tedious to
copy from one version install to the next. To make things easier, we’ve
introduced a new variable named <code class="language-plaintext highlighter-rouge">vars_yml</code>. This variable is used to specify
the location of a custom <code class="language-plaintext highlighter-rouge">vars.yml</code> file to override the default <code class="language-plaintext highlighter-rouge">vars.yml</code> file
included with the download.</p>
<p>An example of how to use the <code class="language-plaintext highlighter-rouge">vars_yml</code> variable is included in the Starburst
Admin install instructions. Also included there are instructions for setting the
<code class="language-plaintext highlighter-rouge">local_vars</code> variable in your customized <code class="language-plaintext highlighter-rouge">vars.yml</code> file to optionally specify a
custom location for the coordinator and worker configuration files.</p>
<h2 id="removing-old-installs">Removing old installs</h2>
<p>Some customers who installed a previous version of Starburst Enterprise platform
(SEP) were experiencing issues when installing newer versions. In RPM
installations for example, there were occasionally empty plugin directories that
caused the application to crash when it tried to load an empty plugin. For some
tarball installs old installs were not removed, which lead to VMs and bare metal
servers running out of disk space.</p>
<p>The <code class="language-plaintext highlighter-rouge">install.yml</code> playbook has now been amended to ensure that after installing
the new version:</p>
<ul>
<li>any empty plugin directories are removed for rpm installs</li>
<li>previous installation directories are removed for tarball installs</li>
</ul>
<h2 id="deprecated-configuration-property">Deprecated configuration property</h2>
<p>The <code class="language-plaintext highlighter-rouge">files/coordinator/config.properties.j2</code> file included with the Starburst
Admin download has default settings for the coordinator’s <code class="language-plaintext highlighter-rouge">config.properties</code>
file. Previous versions of SEP required that the <code class="language-plaintext highlighter-rouge">discovery-server.enabled</code>
configuration property be set. This property was deprecated in version 360,
however, and the property has now been removed from the <code class="language-plaintext highlighter-rouge">config.properties</code>
file.</p>
<p>Keep an eye on our Starburst Admin release notes for more details and future
updates. And of course, we’d love to hear from you - let us know how we can
improve <code class="language-plaintext highlighter-rouge">starburst-admin</code> to suit your needs!</p>
<p><em>Kevin, Frank and Annie</em></p>We are delighted to announce the release of Starburst Admin 1.3.1, a follow-on to release 1.3.0. In this post, we cover what’s new since 1.3.0.Starburst Enterprise 374-e STS2022-03-29T00:00:00-05:002022-03-29T00:00:00-05:00https://docs.starburst.io/blog/sep-374-e<p>Starburst Enterprise platform (SEP) 374-e STS has been released, a rapid
follow-up to our <a href="/blog/2022-03-17-sep-373-e.html">373-e release</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-374.html">Trino 374</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Add support for query parameters in CREATE SCHEMA.</li>
<li>Add support for TRUNCATE TABLE in the Cassandra connector.</li>
<li>Add support for CREATE SCHEMA and DROP SCHEMA in the MongoDB connector.</li>
</ul>
<p>SEP 374-e contains updates to our ‘Starburst Enterprise web UI’. For more
information, head over to the <a href="../latest/release/release-374-e.html">release
notes</a>.</p>
<p><em>Annie and Matt</em></p>Starburst Enterprise platform (SEP) 374-e STS has been released, a rapid follow-up to our 373-e release.Simplifying policy enforcement for your data mesh with Starburst Enterprise and Immuta2022-03-23T00:00:00-05:002022-03-23T00:00:00-05:00https://docs.starburst.io/blog/immuta-sep<p>This blog was co-authored by <a href="https://www.linkedin.com/in/meetsumit/">Sumit
Sarkar</a>, Sr. Director, Product Marketing
at Immuta.</p>
<p>In today’s global economy, it’s impossible to understate the importance of being
able to access data efficiently and securely, regardless of where it resides. As
external data sharing becomes increasingly common, data use agreements and
regulatory requirements, including <a href="https://www.immuta.com/articles/data-localization/">data
localization</a> laws, make it
difficult for data platform teams to keep up and manage a compliant system.</p>
<!--more-->
<p>Starburst and Immuta minimize complexity so the right people are able to access
the right data at the right time, no matter where the data lives. As a federated
query engine, <a href="https://www.starburst.io/platform/starburst-enterprise/">Starburst
Enterprise</a> acts as a
single access point across even the most complex data architectures. Immuta, the
leader in automated <a href="https://www.immuta.com/capabilities/data-access-control/">data access
control</a>, centralizes
data access policy management and enforcement so that policies are dynamically
and consistently applied at query time. The latest release of Starburst
Enterprise ships with Immuta, making it easier than ever to provide <a href="https://blog.starburst.io/rapid-controlled-access-to-data-with-starburst-and-immuta">rapid,
controlled access to
data</a>
across data mesh architectures.</p>
<h2 id="starburst-adds-support-for-data-access-control-with-immuta">Starburst adds support for data access control with Immuta</h2>
<p>Starburst is the analytics engine built for <a href="https://www.immuta.com/articles/what-is-a-data-mesh/">data
mesh</a>, allowing
organizations to use data from multiple data warehouses, which in turn enables
domain-driven data ownership and architecture. This core data mesh principle
reduces pipeline complexity and makes data infrastructure more flexible and
agile. However, the decentralized nature of data mesh architectures can make
consistent data access control and policy enforcement difficult to manage at
scale.</p>
<p>Read more about <a href="https://www.immuta.com/articles/sql-is-your-data-mesh-api/">SQL is your data mesh API</a></p>
<p>To proactively solve this issue and avoid unauthorized data access, Starburst
has added simplified, built-in support for Immuta
<a href="https://www.immuta.com/data-governance-deployment/">SaaS</a> and
<a href="https://www.immuta.com/self-managed-deployment/">self-managed</a> deployment
methods. Starburst users can now seamlessly get up and running with automated
data access control across their data mesh, with minimal overhead or maintenance
required. As a result, users across any domain can query data from any platform
through Starburst Enterprise, and Immuta acts as an invisible layer that
enforces data access policies at runtime.</p>
<p>Immuta and Starburst Enterprise separate policy and data platform. As a result
users can access data more quickly, while enforcing business and regulatory
policies with the most comprehensive and easy-to-use access controls in place.
This allows data teams to maximize the power and flexibility of data mesh
without losing time or data security.</p>
<p>How is this done? When plugged into Starburst Enterprise, Immuta enables users with:</p>
<ul>
<li>
<p><strong><a href="https://www.immuta.com/capabilities/sensitive-data-discovery/">Sensitive data discovery and
classification</a></strong>,
which includes more than 60 prebuilt classifiers, as well as the ability to
build custom tags. Tags can also map to specific data privacy laws and be
implemented through global starter policies. This streamlines certification
workflows by making it easy for stakeholders across functions to inspect and
approve tags.</p>
</li>
<li>
<p><strong>Scalable <a href="https://www.immuta.com/capabilities/attribute-based-access-control/">attribute-based access control
(ABAC)</a></strong>,
which leverages metadata and user/subject attributes to make context-based
access decisions at query time. Data teams can author policies in plain English
so stakeholders have visibility into how data is being protected without
requiring SQL expertise or additional data engineering resources.</p>
</li>
<li>
<p><strong>Advanced policy building and <a href="https://www.immuta.com/capabilities/dynamic-data-masking/">dynamic data
masking</a></strong> through
techniques like privacy-enhancing technologies (PETs), including differential
privacy, conditional masking, and k-anonymization, as well as the ability to
easily limit data use for specific purposes to ensure secure data
collaboration. Immuta allows users to implement both global and local
policies, so that data owners and data platform owners can control access to
data.</p>
</li>
<li>
<p><strong><a href="https://www.immuta.com/capabilities/data-policy-enforcement/">Data policy enforcement and
auditing</a></strong>, which
automates access control implementation at query time with no impact to user
workflows, and provides unified data access logs that show how data is
requested, accessed, and used. Immuta also allows organizations using service
accounts from BI dashboards to impersonate end users in order to restrict
access with no impact to the user or risk of a data leak.</p>
</li>
</ul>
<p>Together, these capabilities allow Starburst Enterprise users to scale policy
management through automation, increase agility across teams by empowering more
stakeholders to manage policy, and easily prove compliant data use at any time.</p>
<p>Let’s take a closer look at how to get up and running with Starburst Enterprise
and Immuta.</p>
<h2 id="implementing-immuta-access-control-with-starburst-enterprise">Implementing Immuta access control with Starburst Enterprise</h2>
<p>The <a href="../latest/security/biac-overview.html">built-in access control of Starburst
Enterprise</a> is role-based and provides a
baseline for data security and privacy management. For organizations scaling
data use or leveraging sensitive data, particularly for data sharing, Immuta
adds a robust layer of advanced protection and control.</p>
<p>To get started, Starburst Enterprise users need an active Immuta account and API
key. Immuta manages access control for existing users and groups, regardless of
the type of user authentication used with Starburst Enterprise.</p>
<p>Once Starburst Enterprise is registered as a data source in Immuta, you’re able
to perform the following tasks</p>
<ul>
<li>Configure catalogs for your cluster’s data sources with no process changes.</li>
<li>Configure a catalog for an Immuta connector.</li>
<li>Create an entry for each schema that is managed by Immuta. Immuta creates a
view to represent the schema in its domain.</li>
<li>Configure an Immuta event listener in Starburst Enterprise.</li>
<li>Configure file-based access control in Starburst Enterprise.</li>
</ul>
<p>These steps ensure that queries are validated by Immuta’s dynamic access
control, then passed back to Starburst Enterprise for analysis and query
processing. Let’s take a closer look at how it works in practice.</p>
<h2 id="query-changes">Query changes</h2>
<p>When adding schemas to Immuta, it’s important to differentiate between the
Immuta schema names and the underlying Starburst Enterprise schema names.
Similarly, tables that exist in multiple Starburst Enterprise catalogs should
each have a unique name to make it easy to distinguish which are under Immuta’s
control.</p>
<p>Integrating Immuta with Starburst Enterprise allows you to run the same SQL
queries you run directly with Starburst Enterprise. However, you must use the
catalog name immuta and the schema names recognized by Immuta to ensure the
Immuta policies are taken into account. For instance, you can use the <a href="../latest/client/query-editor.html">Starburst
Enterprise query editor</a> to browse the
immuta catalog and identify specific schemas and tables, with no change to the
catalog.schema.table hierarchy. An example query for a table labeled customers
might look like this:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="n">myimmuta</span><span class="p">.</span><span class="n">orcl_accounting</span><span class="p">.</span><span class="n">customers</span>
<span class="k">LIMIT</span> <span class="mi">100</span><span class="p">;</span>
</code></pre></div></div>
<p>Admin users are able to see the underlying schema names and reach the same table with the following query:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="n">oracle</span><span class="p">.</span><span class="n">accounting</span><span class="p">.</span><span class="n">customers</span>
<span class="k">LIMIT</span> <span class="mi">100</span><span class="p">;</span>
</code></pre></div></div>
<h2 id="configuring-an-immuta-catalog">Configuring an Immuta catalog</h2>
<p>Immuta now ships with Starburst Enterprise. To configure the Immuta connector
and activate the plugin, you must create a configuration file in your cluster’s
<code class="language-plaintext highlighter-rouge">etc/catalog</code> folder. This catalog properties file name <code class="language-plaintext highlighter-rouge">myimmuta.properties</code>
determines the name of the catalog to use for querying data via Immuta. In the
preceding example query the name <code class="language-plaintext highlighter-rouge">myimmuta</code> is used. The file needs to specify
usage of the <code class="language-plaintext highlighter-rouge">immuta</code> connector with <code class="language-plaintext highlighter-rouge">connector-name</code> property and include the
URL of your Immuta server and the API key:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>connector-name=immuta
immuta-endpoint=your-Immuta-site's-URL
immuta-apikey=your-API-key
</code></pre></div></div>
<p>For example:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>connector-name=immuta
immuta-endpoint=https://www.immuta.example.com/
immuta-apikey=e16456a3395d42988c2da6f06ccf1f58
</code></pre></div></div>
<h2 id="configuring-an-immuta-event-listener">Configuring an Immuta event listener</h2>
<p>Immuta audit event listeners enable you to audit queries on your Starburst
Enterprise cluster so you can see what data is used and prove compliance with
internal and external rules and regulations. Event listeners must be configured
in a file named <code class="language-plaintext highlighter-rouge">event-listener.properties</code> in the <code class="language-plaintext highlighter-rouge">etc</code> folder and the
<code class="language-plaintext highlighter-rouge">immuta-catalogs</code> value must match the name of the catalog.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>event.listener.name=immuta-event-listener
immuta-endpoint=https://www.immuta.example.com/
immuta-apikey=e16456a3395d42988c2da6f06ccf1f58
immuta.catalogs=myimmuta
</code></pre></div></div>
<h2 id="managing-system-access-control">Managing system access control</h2>
<p>There are two processes that fall outside of Immuta’s control, and are
controlled in Starburst Enterprise:</p>
<ul>
<li>Prevent direct access to catalogs for standard users trying to bypass Immuta.</li>
<li>Protecting Immuta-managed tables from inadvertent secrets disclosure.</li>
</ul>
<p>You can control these with the file-based access control and a JSON rules file
in Starburst Enterprise:</p>
<p>The first step here is adding a configuration file called
<code class="language-plaintext highlighter-rouge">access-control.properties</code> to the <code class="language-plaintext highlighter-rouge">etc</code> folder with the following two lines:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>access-control.name=file
security.config-file=etc/access-rules.json
</code></pre></div></div>
<p>Below shows a starting point for an access-rules.json file:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"catalogs"</span><span class="p">:</span><span class="w"> </span><span class="p">[</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"user"</span><span class="p">:</span><span class="w"> </span><span class="s2">"super_admin"</span><span class="p">,</span><span class="w">
</span><span class="nl">"catalog"</span><span class="p">:</span><span class="w"> </span><span class="s2">"immuta|postgres|redshift|mysql|oracle|system"</span><span class="p">,</span><span class="w">
</span><span class="nl">"allow"</span><span class="p">:</span><span class="w"> </span><span class="s2">"all"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"role"</span><span class="p">:</span><span class="w"> </span><span class="s2">"IT"</span><span class="p">,</span><span class="w">
</span><span class="nl">"catalog"</span><span class="p">:</span><span class="w"> </span><span class="s2">"immuta"</span><span class="p">,</span><span class="w">
</span><span class="nl">"allow"</span><span class="p">:</span><span class="w"> </span><span class="s2">"all"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"group"</span><span class="p">:</span><span class="w"> </span><span class="s2">"query_user"</span><span class="p">,</span><span class="w">
</span><span class="nl">"catalog"</span><span class="p">:</span><span class="w"> </span><span class="s2">"immuta"</span><span class="p">,</span><span class="w">
</span><span class="nl">"allow"</span><span class="p">:</span><span class="w"> </span><span class="s2">"read-only"</span><span class="w">
</span><span class="p">},</span><span class="w">
</span><span class="p">{</span><span class="w">
</span><span class="nl">"catalog"</span><span class="p">:</span><span class="w"> </span><span class="s2">"postgres|redshift|mysql|oracle|system"</span><span class="p">,</span><span class="w">
</span><span class="nl">"allow"</span><span class="p">:</span><span class="w"> </span><span class="s2">"none"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">]</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>This rules file grants anyone with the <code class="language-plaintext highlighter-rouge">super_admin</code> role full access to both
Immuta-managed and non-Immuta-managed catalogs. It also ensures <code class="language-plaintext highlighter-rouge">IT</code> has full
access to Immuta-managed objects but no other catalogs, and it gives read-only
access to Immuta-managed schemas and tables to <code class="language-plaintext highlighter-rouge">query_user</code> roles. This
configuration prevents users from running
<a href="../latest/sql/describe.html">DESCRIBE</a> on Immuta views and avoids secret
disclosures in masking policies. Finally, the rules file ensures that non-Immuta
catalog access defaults to none for everyone except <code class="language-plaintext highlighter-rouge">super_admin</code>.</p>
<p>With Immuta integrated with Starburst Enterprise, you’re able to seamlessly
create data access policies once and enforce them across all data, so the right
people are able to access the right data at the right time.</p>
<p>Ready to start building a data product with Starburst and Immuta? <a href="https://www.starburst.io/learn/events-webinars/datanova/video/?wistia=fk4uij4rwk">Check out
this exclusive
video</a>
featuring Starburst’s Director of Engineering, Colleen Tartow, and Head of Data
Products, Vishal Singh to learn more.</p>
<p>Ready to try it out yourself? Use the <a href="../latest/security/immuta-overview.html">documentation for the Immuta integration
with Starburst Enterprise</a> for all the
details.</p>
<p>Alex Breshears is a Product Manager at Starburst.</p>This blog was co-authored by Sumit Sarkar, Sr. Director, Product Marketing at Immuta. In today’s global economy, it’s impossible to understate the importance of being able to access data efficiently and securely, regardless of where it resides. As external data sharing becomes increasingly common, data use agreements and regulatory requirements, including data localization laws, make it difficult for data platform teams to keep up and manage a compliant system.Support for SQL Server added to Starburst Galaxy2022-03-22T00:00:00-05:002022-03-22T00:00:00-05:00https://docs.starburst.io/blog/sg-update<p><img style="float: right;" src="../assets/img/logo/sqlserver.png" alt="SQL Server" /></p>
<p>We are delighted to announce that <a href="../starburst-galaxy/catalogs/sqlserver.html">Microsoft SQL
Server</a> joins the list of data
sources supported by Starburst Galaxy.</p>
<!--more-->
<h2 id="sql-server-on-all-ecosystems">SQL Server on all ecosystems</h2>
<p>Support in Starburst Galaxy for a data source compatible with <a href="https://www.microsoft.com/en-us/sql-server/sql-server-2019">Microsoft SQL
Server</a> is now
available on three ecosystems:</p>
<ul>
<li>
<p><a href="https://aws.amazon.com/rds/sqlserver/">Amazon RDS for SQL Server</a></p>
</li>
<li>
<p><a href="https://azure.microsoft.com/en-us/products/azure-sql/database/">Azure SQL Database</a></p>
</li>
<li>
<p><a href="https://cloud.google.com/sql-server">SQL Server on Google Cloud</a></p>
</li>
</ul>
<p>To add an existing SQL Server database to your cluster, start by configuring a
<a href="../starburst-galaxy/catalogs/sqlserver.html">SQL Server catalog</a>.</p>We are delighted to announce that Microsoft SQL Server joins the list of data sources supported by Starburst Galaxy.March 2022 Starburst Enterprise LTS backport releases2022-03-18T00:00:00-05:002022-03-18T00:00:00-05:00https://docs.starburst.io/blog/lts-backports<p>Heads up everyone - another release batch for our long-term support (LTS)
versions of Starburst Enterprise is available.</p>
<!--more-->
<p>Especially 370-e users are unblocked by the fix for the log file issues. As
usual we encourage you all to check the release notes, and update at your
earliest convenience:</p>
<ul>
<li><a href="../latest/release/release-370-e.html">370-e.2</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.6</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.10</a></li>
<li><a href="../latest/release/release-356-e.html">356-e.13</a></li>
</ul>
<p>The related Helm charts include an update of Ranger. All resources are available
as part of the update.</p>
<p><em>Manfred</em></p>Heads up everyone - another release batch for our long-term support (LTS) versions of Starburst Enterprise is available.Starburst Enterprise 373-e STS2022-03-17T00:00:00-05:002022-03-17T00:00:00-05:00https://docs.starburst.io/blog/sep-373-e<p>Starburst Enterprise platform (SEP) 373-e STS has been released. What a rapid
follow up to <a href="/blog/2022-03-11-sep-372-e.html">372-e just last week</a>.</p>
<!--more-->
<p>The release is based on <a href="../latest/release/release-373.html">Trino 373</a>, which
includes the following noteworthy changes:</p>
<ul>
<li>Improve query performance by better estimating partitioned tables statistics
in Hive.</li>
<li>Support for access to S3 via an HTTP proxy in
<a href="../latest/connector/iceberg.html">Iceberg</a>.</li>
</ul>
<p>In <a href="../latest/release/release-373-e.html">SEP 373-e</a>, we updated
the Ranger Helm chart to use Ranger 2.1.0-e.37.</p>
<p><em>Annie and Matt</em></p>Starburst Enterprise platform (SEP) 373-e STS has been released. What a rapid follow up to 372-e just last week.Starburst donates the Delta Lake connector to Trino2022-03-15T00:00:00-05:002022-03-15T00:00:00-05:00https://docs.starburst.io/blog/delta-lake<p>Starburst was founded around the <a href="../index.html#trino">Trino open source
project</a>. Many contributors and maintainers of the project
are part of our teams. Our products, <a href="../starburst-enterprise/index.html">Starburst Enterprise platform
(SEP)</a> and <a href="../starburst-galaxy/index.html">Starburst
Galaxy</a>, both use Trino as their beating heart
and essential engine. And in the same way, Trino is at the heart of our efforts
to improve our products. <a href="https://delta.io/">Delta Lake</a> support is the latest
example.</p>
<!--more-->
<p>{Starburst Enterprise platform (SEP) customers have been enjoying ability to use
the Delta Lake connector to query their object storage <a href="https://docs.starburst.io/latest/release/release-332-e.html">for nearly two
years</a>. In that
timeframe, we had over 25 engineers learn the codebase and contribute
improvements. At this stage the connector is production-tested and full of great
features. Now we are giving it back to the Trino community to work with us to
make it even better. It’s a real privilege to work with such dedicated
communities.</p>
<p>Everyone can now use the open source projects from <a href="https://delta.io/">Delta
Lake</a> and <a href="https://trino.io/">Trino</a> to fully unlock the data
in their lakehouse.</p>
<h2 id="technical-details">Technical details</h2>
<p>Delta Lake is an open source storage format that provides ACID guarantees on
object storage. With the new connector, we support use cases such as reading
data, writing data, and more.</p>
<p>Let’s take a quick look at the capabilities:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">CREATE SCHEMA</code> - Create schemas for both new tables, and existing tables that
you may have created with another engine.</li>
<li><code class="language-plaintext highlighter-rouge">CREATE TABLE (AS)</code> - Create empty tables or pre-load them with the result of
an other query. With the power of Trino this query can retrieve
data from any other configured catalog, and therefore data source, and to
populate the table.</li>
<li>Read operations - Retrieve and analyze your data with raw SQL or
your <a href="../data-consumer/clients/index.html">client tool of choice</a>.</li>
<li><code class="language-plaintext highlighter-rouge">INSERT/UPDATE/DELETE</code> - Load data, remove old records, and keep current
with changes.</li>
<li><code class="language-plaintext highlighter-rouge">VACUUM and OPTIMIZE</code> - Manage your data in Delta Lake to keep it performant
even after heavy use.</li>
</ul>
<h2 id="next-steps">Next steps</h2>
<p>The connector and the <a href="https://trino.io/docs/current/connector/delta-lake.html">documentation are in
Trino</a>, so
you can all start using it! Get the <a href="https://trino.io/docs/current/release/release-373.html">latest Trino 373
release</a>, and you’ve got
everything you need. However, this is just the beginning.</p>
<p>Our next steps include rolling the connector into
<a href="../starburst-galaxy/index.html">Starburst Galaxy</a> to empower Delta Lake users
even more. From here on out we are committed to keeping it current and improving
it, and we have a few things in mind already. And of course, we are also looking
forward to see what the community wants and contributes.</p>
<p>If you want to find out more, make sure to join us on <a href="https://trino.io/episodes/34.html">Trino Community Broadcast
episode 34 on Thursday this week</a>.</p>
<p><em>Starburst engineering and product teams</em></p>Starburst was founded around the Trino open source project. Many contributors and maintainers of the project are part of our teams. Our products, Starburst Enterprise platform (SEP) and Starburst Galaxy, both use Trino as their beating heart and essential engine. And in the same way, Trino is at the heart of our efforts to improve our products. Delta Lake support is the latest example.Starburst Enterprise 372-e STS2022-03-11T00:00:00-06:002022-03-11T00:00:00-06:00https://docs.starburst.io/blog/sep-372-e<p>When the time span between releases is longer, the change set of the release
becomes larger. This is certainly true with the Trino 372 release and our new
Starburst Enterprise 372-e STS. A whole lot of new goodness is coming your way
today.</p>
<!--more-->
<p><a href="../latest/release/release-372.html">Trino 372</a> includes few noteworthy changes.
Martin Traverso specifically mentioned the following improvements:</p>
<ul>
<li>New <code class="language-plaintext highlighter-rouge">trim_array</code> function.</li>
<li>Support for reading ZSTD-compressed Avro files.</li>
<li>Support for column comments in <a href="../latest/connector/iceberg.html">Iceberg</a>.</li>
<li>Support for Kerberos authentication in <a href="../latest/connector/kudu.html">Kudu connector</a>.</li>
</ul>
<p>Depending on your specific use case there are many other improvements mentioned
in the release notes. <a href="../latest/release/release-372.html">Go check it out</a>!</p>
<p>When looking at the <a href="../latest/release/release-372-e.html">release notes and Starburst Enterprise 372-e
itself</a>, the most visible change is the new
login screen. No longer is it a plain dialog. Users of the <a href="../latest/overview/sep-ui.html">Starburst Enterprise
web UI</a> now get a great design, just like
<a href="../starburst-galaxy/index.html">Starburst Galaxy</a> users. And you can even
customize it with your own image.</p>
<p><img src="../assets/img/general/sep-ui-login.png" alt="Login screen for Starburst Enterprise web UI" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>Another impactful change concerns users of the SQL Server connector. <a href="../latest/connector/sqlserver.html#sqlserver-tls">TLS is now
enabled by default</a> due to an
update of the embedded JDBC driver. As a result you need to potentially update
your catalog properties file. Even better, use a globally trusted certificate on
the SQL Server host, and enjoy the improved security. And by the way, this is a
result of our preparation to fully support Java 17. Stay tuned for more details
about that.</p>
<p>When it comes to security we also <a href="../security/starburst-enterprise.html">clarified our work on the Apache Ranger and
Hive Metastore Service Helm charts</a>.</p>
<p>Back to the release though - it includes a couple of other improvements for
connectors, such as the Delta Lake connector. The <a href="../latest/k8s/release-notes.html">SEP Helm
chart</a>, got a little update to avoid a
Kubernetes throttling issue. Early users of our <a href="../latest/security/biac-overview.html">built-in access
control</a> should also update and provide
us with further feedback.</p>
<p>We continue to be hard at work improving performance, security, and usability.</p>
<p><em>Manfred and Matt</em></p>When the time span between releases is longer, the change set of the release becomes larger. This is certainly true with the Trino 372 release and our new Starburst Enterprise 372-e STS. A whole lot of new goodness is coming your way today.More SSH tunneling for Starburst Galaxy2022-03-03T00:00:00-06:002022-03-03T00:00:00-06:00https://docs.starburst.io/blog/sg-update<p>It was just a little while ago that we <a href="/blog/2022-02-18-sg-update.html">released SSH tunnel support</a> for PostgreSQL and MySQL catalogs in Starburst Galaxy.
And today it is time to add on to that.</p>
<!--more-->
<h2 id="redshift-catalogs">Redshift catalogs</h2>
<p>Support for Redshift catalogs just launched recently, and of course it can’t be
left behind PostgreSQL and MySQL. So with this release we are adding the ability
for <a href="../starburst-galaxy/security/ssh-tunnels.html">SSH tunnel usage</a> with the
<a href="../starburst-galaxy/catalogs/redshift.html">Redshift catalog</a>.</p>
<h2 id="hive-metastore-service">Hive Metastore Service</h2>
<p>The Hive Metastore Service (HMS) is crucial for the functionality of object
storage catalogs, since it contains all the metadata about the objects. And just
like the data itself often needs more security, so does the metadata. That can
be achieved with locking the in a private network. With this update <a href="../starburst-galaxy/security/ssh-tunnels.html">SSH
tunnels</a> come to the rescue. You
can now use them to connect to the HMS in <a href="../starburst-galaxy/catalogs/s3.html">Amazon
S3</a>, <a href="../starburst-galaxy/catalogs/adls.html">Azure Data Lake
Storage</a>, and <a href="../starburst-galaxy/catalogs/gcs.html">Google Cloud
Storage</a> catalogs.</p>
<h2 id="other-improvements">Other improvements</h2>
<p>Numerous backend improvements also rolled out, but you won’t notice those
directly. However some of the UI tweaks on the query editor come in handy.
Specifically the tree view now shows tables and views in the same node.</p>
<p>That is it for today. More to come soon.</p>
<p><em>Bob and Manfred</em></p>It was just a little while ago that we released SSH tunnel support for PostgreSQL and MySQL catalogs in Starburst Galaxy. And today it is time to add on to that.February 2022 Starburst Enterprise LTS backport releases2022-02-23T00:00:00-06:002022-02-23T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>It’s that time again! We’ve released the latest batch of backport releases for
our Starburst Enterprise platform (SEP) LTS users.</p>
<!--more-->
<p>Read more about each LTS update in their respective release notes:</p>
<ul>
<li><a href="../latest/release/release-356-e.html">356-e.12</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.9</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.5</a></li>
<li><a href="../latest/release/release-370-e.html">370-e.1</a></li>
</ul>
<p>This batch includes the first update to our most recent LTS,
<a href="/blog/2022-02-17-sep-370-e.html">370-e</a>.</p>
<p>As usual the related Helm charts and other resources are all available as part
of the update.</p>
<p><em>Joe and Manfred</em></p>It’s that time again! We’ve released the latest batch of backport releases for our Starburst Enterprise platform (SEP) LTS users.Starburst Enterprise 371-e STS2022-02-22T00:00:00-06:002022-02-22T00:00:00-06:00https://docs.starburst.io/blog/sep-371-e<p>Didn’t we just release 370-e LTS? <a href="/blog/2022-02-17-sep-370-e.html">We sure did</a>. But that doesn’t mean we are resting.</p>
<!--more-->
<p>With the new Starburst Enterprise 371-e STS release you get access to a number
of improvements from <a href="../latest/release/release-371.html">Trino 371</a></p>
<p>And of course, we also have some <a href="../latest/release/release-371-e.html">great updates from our Starburst Enterprise
features in this release</a>. Vertica, and
PostgreSQL connector users get better statistics, and therefore query
performance. And Kafka connector users can now use Kerberos authentication to
the schema registry.</p>
<p>Users of <a href="../latest/data-products/index.html">data products</a> and <a href="../latest/security/biac-overview.html">built-in
access control system</a> can now rejoice
since data products are now securable entities. You can therefore make sure no
other users delete your valuable work creating a data product.</p>
<p>We continue to work towards getting both of these features ready for any
production usage. Stay tuned for the next STS release with more updates and
improvements.</p>
<p><em>Manfred and Matt</em></p>Didn’t we just release 370-e LTS? We sure did. But that doesn’t mean we are resting.Redshift catalogs and SSH tunnels for Starburst Galaxy2022-02-18T00:00:00-06:002022-02-18T00:00:00-06:00https://docs.starburst.io/blog/sg-update<p>Usage of Starburst Galaxy is really picking up. We had a great demo session at
the recent Datanova event, where the attendees spun up hundreds of clusters. The
new users love the experience, and bring new demands to us. Our
always-productive product and engineering teams are ready to help, and today we
are providing two great new features in the latest production deployment of
Starburst Galaxy.</p>
<!--more-->
<h2 id="redshift-catalogs">Redshift catalogs</h2>
<p>We are moving another step forward in our mission to enable data analysis and
unlock all your data, no matter where it is stored. It is time to query all the
data in <a href="https://aws.amazon.com/redshift/">Amazon Redshift</a> with our new support
for <a href="../starburst-galaxy/catalogs/redshift.html">Redshift catalogs</a>.
Configuration is similar to the existing catalogs: create the new catalog, add
to a cluster, and start writing SQL to query your Redshift cluster. You can even
combine it with data from another catalog in a federated query.</p>
<h2 id="ssh-tunnels">SSH tunnels</h2>
<p>A common problem faced by customers are locked-away databases. Sure, your
PostgreSQL or MySQL database is deployed and managed in the cloud. But that
doesn’t mean just anybody can access it. It is safely tucked away in your
virtual private cloud (VPC), and any attempt to query it with a catalog from
Starburt Galaxy is bound to fail.</p>
<p>And now we have a solution for that. SSH tunnels to the rescue! By placing a
bastion host server within the VPC of your database and cloud provider, you can
create an <a href="../starburst-galaxy/security/ssh-tunnels.html">SSH tunnel</a>. The
bastion host is configured to allow Starburst Galaxy access from outside. All
network traffic goes through SSH to the bastion host, and to the database
host inside the VPC. With permission for the bastion host to connect to the
database, you can now query the database even though it is located inside the
VPC, and get your query results in Starburst Galaxy running outside the VPC.
SSH ensures all traffic is encrypted.</p>
<p>With this release SSH tunneling through bastion hosts is supported for MySQL and
PostgreSQL catalogs.</p>
<p>We have a lot more in the pipeline, so watch this space for more announcements.</p>
<p><em>Bob and Manfred</em></p>Usage of Starburst Galaxy is really picking up. We had a great demo session at the recent Datanova event, where the attendees spun up hundreds of clusters. The new users love the experience, and bring new demands to us. Our always-productive product and engineering teams are ready to help, and today we are providing two great new features in the latest production deployment of Starburst Galaxy.Starburst Enterprise 370-e LTS2022-02-17T00:00:00-06:002022-02-17T00:00:00-06:00https://docs.starburst.io/blog/sep-370-e<p>Yes, it is this time of year again for us. <a href="../latest/versions.html#lts-sts">Long term support
(LTS)</a> release time! With the new Starburst
Enterprise 370-e LTS, we are shipping the latest and greatest features for our
customers, who also want the availability of security and correctness backports
for the next year. It is upgrade time, and there are many good reasons to get
onto 370-e.</p>
<!--more-->
<p>In the recent STS releases <a href="/blog/2022-01-18-sep-368-e.html">368-e</a> and
<a href="/blog/2022-02-01-sep-369-e.html">369-e</a> we added a number of great new
features and updates. The most noticeable change is the new look and feel of the
<a href="../latest/overview/sep-ui.html">Starburst Enterprise web UI</a>. Make sure to
<a href="../latest/admin/query-logger.html">configure a query logger database</a> to take
advantage of query editor tab persistence, query history, telemetry data, and
others. You can also start using the <a href="../latest/security/biac-overview.html">built-in access control
system</a> to grant privileges to perform
actions on entities to your users, and deny access to others.</p>
<p>The most groundbreaking feature is <a href="../latest/data-products/index.html">data
products</a>. It allows you to abstract the
complexity of your underlying catalogs, schemas, tables, and all those pesky
details. You can define a data product for a
<a href="../latest/data-products/domain-management.html">domain</a>, allow users to rate
it, find it, and use it for their own reporting, dashboards, and visualizations.</p>
<p><img src="../assets/img/general/dp-overview.png" alt="Data products in Starburst Enterprise" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>As data product owner, you can hide the details and allow your users to
concentrate on what is useful and interesting to them. And of course, you can
take advantage of the knowledge of other data engineers by using their data
products. This enables great collaboration of domain experts, and unlocks even
more understanding of all the data in your business.</p>
<p>We are steadily improving features for the next STS release, and look forward to
hear your feedback. Explore and enjoy!</p>
<p><em>Manfred and Matt</em></p>Yes, it is this time of year again for us. Long term support (LTS) release time! With the new Starburst Enterprise 370-e LTS, we are shipping the latest and greatest features for our customers, who also want the availability of security and correctness backports for the next year. It is upgrade time, and there are many good reasons to get onto 370-e.Increased CPU utilization in Trino2022-02-03T00:00:00-06:002022-02-03T00:00:00-06:00https://docs.starburst.io/blog/cpu-utilization<p>While working on performance enhancements at Starburst, engineering focuses on
reducing the amount of time that a CPU has to work on a given query. This is
referred to as <em>CPU time</em>. CPU time is a stable metric that reflects real
performance in the case of concurrent queries.</p>
<p>As the CPU time for an individual query drops so can Trino’s ability to utilize
the CPUs fully; this can be due to scheduling as an example. The result is that
a reduction in CPU time does not always translate to a reduction in latency or
wall time. After a year of major performance enhancements we decided to focus
our efforts on increasing CPU utilization and reducing query wall time.</p>
<!--more-->
<h2 id="cpu-utilization-enhancements">CPU utilization enhancements</h2>
<p>The most significant change is that Trino’s
<a href="../370-e/admin/properties-query-management.html">query.execution-policy</a> now
defaults to <code class="language-plaintext highlighter-rouge">phased</code> rather than <code class="language-plaintext highlighter-rouge">all-at-once</code>. The all-at-once approach
scheduled all query stages in a single shot with the goal of simplicity and
reduced latency. The phased execution policy was later added as a configuration
option which would schedule only the stages of a query which can can make
progress.</p>
<p>Recently, Karol Sobczak made the observation that the phased execution policy
can schedule stages that can create subsequent stages which can’t make progress.
That issue defeats the purpose of the phased execution policy. Fixing this logic
resulted in reduced latency, and the ability to set the phased execution policy
as default.</p>
<h2 id="other-improvements">Other improvements</h2>
<p>Other significant changes include adaptively setting <code class="language-plaintext highlighter-rouge">task.concurrency</code> to the
number of physical cores on a node and increasing the default value of
<code class="language-plaintext highlighter-rouge">hive.split-loader-concurrency</code>. We have observed that hyper-threaded cores do
not translate to improved query performance. Increasing the split loader
concurrency helps the engine process partitions and small files more quickly.</p>
<h2 id="benchmark-results">Benchmark results</h2>
<p>We were impressed with a 20% reduction in wall time for TPC-H partitioned data,
and customers should be able to expect an average reduction of 13% in wall time
based on our internal benchmarking. We have seen improvements as high as 50%
for TPC-H query 12 on partitioned data.</p>
<p>The benchmark results were obtained by running the TPC-H and TPC-DS benchmarks
with one coordinator and six worker nodes. The data was queried by the Hive
connector with partitioned and unpartitioned data at 1TB scale.</p>
<p><img src="../assets/img/general/cpu-utilization-tpc-h-benchmark-results.png" alt="TPC-H benchmark results for CPU utilization enhancements" class="img-fluid " description="" width="px" style="" /></p>
<p>While TPC-H and TPC-DS are both decision support benchmarks TPC-H is said to be
more representative of ad hoc queries which tend to be simpler.</p>
<p><img src="../assets/img/general/cpu-utilization-tpc-ds-benchmark-results.png" alt="TPC-DS benchmark results for CPU utilization enhancements" class="img-fluid " description="" width="px" style="" /></p>
<h2 id="ready-for-you">Ready for you</h2>
<p>The best part about these changes mentioned above is that they do not require
any changes. They will be available out of the box in the upcoming LTS release.
Just remember that the software makes use of existing configurations so you
might need to unset <code class="language-plaintext highlighter-rouge">query.execution-policy</code> and <code class="language-plaintext highlighter-rouge">task.concurrency</code>.</p>
<p>Enjoy the even faster query processing.</p>
<p><em>Starburst Performance Team</em></p>While working on performance enhancements at Starburst, engineering focuses on reducing the amount of time that a CPU has to work on a given query. This is referred to as CPU time. CPU time is a stable metric that reflects real performance in the case of concurrent queries. As the CPU time for an individual query drops so can Trino’s ability to utilize the CPUs fully; this can be due to scheduling as an example. The result is that a reduction in CPU time does not always translate to a reduction in latency or wall time. After a year of major performance enhancements we decided to focus our efforts on increasing CPU utilization and reducing query wall time.Starburst Enterprise 369-e STS2022-02-01T00:00:00-06:002022-02-01T00:00:00-06:00https://docs.starburst.io/blog/sep-369-e<p>We are not resting, on the contrary. Today we are bringing you another STS
release packed with great new features to try and use. And we are solidifying
everything for the upcoming LTS release as well.</p>
<!--more-->
<h2 id="trino">Trino</h2>
<p>Back to the 369-e STS release first though. It is based on Trino 369, which
added a couple of amazing new features, including:</p>
<ul>
<li>Experimental support for task-level retries.</li>
<li>Support for groups in <a href="../latest/release/release-369.html#security">OAuth2 claims</a>.</li>
<li><a href="../latest/sql/comment.html">Column comments</a> in ClickHouse connector.</li>
<li>Write <a href="../latest/release/release-369.html#hive-connector">Bloom filters</a> in ORC files.</li>
<li>Procedure for <a href="../latest/release/release-369.html#iceberg-connector">optimizing Iceberg tables</a>.</li>
</ul>
<p>This release also <a href="https://github.com/trinodb/trino/issues/10710">fixes an S3/Parquet file data
issue</a> for users of <a href="/blog/2021-12-29-sep-367-e.html">367-e</a> and <a href="/blog/2022-01-18-sep-368-e.html">368-e</a>, who used AWS S3 with streaming enabled with the Hive, Iceberg, or Delta
Lake connectors, so make sure you update if this applies to you.</p>
<p>If you are interested in the upcoming support for query and task-level retries,
and our work towards even better support for ETL and batch operations, you can
learn more in <a href="https://trino.io/episodes/32.html">episode 32 of the Trino Community Broadcast about Project
Tardigrade</a>.</p>
<h2 id="starburst-enterprise-platform-sep">Starburst Enterprise platform (SEP)</h2>
<p>Looking at the <a href="../latest/release/release-369-e.html">Starburst Enterprise platform (SEP) 369-e STS release
notes</a> itself, the biggest standout is the
preview of our <a href="../latest/security/biac-overview.html">new built-in access
control</a>.</p>
<p>Users of Starburst Galaxy are already used to a <a href="../starburst-galaxy/security/index.html">powerful, integrated access
control system</a>. And of course, we are
not leaving Starburst Enterprise platform (SEP) users behind. In fact, the new built-in
access control is even more powerful, and includes features such as <a href="../latest/security/biac-privileges.html#entity-category-tables">table-level
access control</a>.
Try it out!</p>
<p><img src="../assets/img/general/sep-biac.png" alt="Starburst Enterprise built-in access control" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>The release also includes numerous improvements for the <a href="../latest/connector/starburst-delta-lake.html">Starburst Delta Lake
connector</a> and Kerberos
authentication support for the <a href="../latest/connector/starburst-memsql.html">SingleStore
connector</a>. And our connector pages
include colorful logos now as well!</p>
<h2 id="next-up">Next up</h2>
<p>We hope you enjoy testing this new release. See you next time on the LTS
announcement.</p>
<p><em>Manfred and Matt</em></p>We are not resting, on the contrary. Today we are bringing you another STS release packed with great new features to try and use. And we are solidifying everything for the upcoming LTS release as well.Starburst Enterprise 368-e STS2022-01-18T00:00:00-06:002022-01-18T00:00:00-06:00https://docs.starburst.io/blog/sep-368-e<p>Today we are starting the year 2022 off with a great new release of Starburst
Enterprise platform (SEP). Believe us, you do want to try out the refreshed
Starburst Enterprise web UI.</p>
<!--more-->
<p>Following up on the <a href="/blog/2021-12-29-sep-367-e.html">367-e STS release from the end of 2021</a>, we have another set of changes ready for you. Today we
are releasing <a href="../368-e/index.html">Starburst Enterprise 368-e STS</a>.</p>
<p>As usual this release is based on Trino, and Martin Traverso announced the
following major improvements:</p>
<ul>
<li>Improve performance of <code class="language-plaintext highlighter-rouge">DECIMAL</code> type</li>
<li><a href="../latest/release/release-368.html#iceberg-connector">File-based access
control</a> for Apache
Iceberg connector</li>
<li>Support for <code class="language-plaintext highlighter-rouge">TIME</code> type in <a href="../latest/release/release-368.html#singlestore-memsql-connector">SingleStore
connector</a></li>
<li>Support for <code class="language-plaintext highlighter-rouge">BINARY</code> type in <a href="../latest/release/release-368.html#phoenix-connector">Apache Phoenix
connector</a></li>
</ul>
<p>On the Starburst Enterprise side we <a href="../latest/release/release-368-e.html">added a few more
improvements</a>. The biggest change is our
rebranded and refreshed <a href="../latest/overview/sep-ui.html">Starburst Enterprise platform (SEP) web
UI</a>. Over the last few releases we have added
more features and polished everything.</p>
<p>The Starburst Enterprise platform (SEP) web UI offers access to the query
editor, data products, query history, cluster information, usage metrics and
others.</p>
<p><img src="../assets/img/general/sep-ui-cluster-overview.png" alt="Starburst Enterprise web UI" class="img-fluid img-screenshot" description="" width="px" style="" /></p>
<p>As a data consumer you can write and test your queries right in the <a href="../latest/client/query-editor.html">query
editor</a> without the needs to configure
another tool. And once the query runs you can analyze its processing metrics and
more information in the <a href="../latest/insights/query-details.html">query details</a>.
As a data engineer, you can quickly and easily test that new catalog you just
added, and confirm access to the nested schemas and tables works as expected.
And as a platform administrator you can check <a href="../latest/insights/cluster-history.html">cluster
status</a>, <a href="../latest/insights/query-overview.html">query
history</a>, and <a href="../latest/insights/usage-metrics-overview.html">usage
metrics</a> to figure out when it
is time to add another worker to that busy cluster.</p>
<p>These are just some of the use cases you can play around with in the user
interface. We definitely want you to check it out. Did we miss something? Is
something not working? <a href="../support.html">Contact us</a> so
we can get it improved in time for the upcoming LTS release.</p>
<p><em>Manfred and Matt</em></p>
<p><strong>Update, 2022-01-27:</strong></p>
<p>In 368-e and 367-e, writing to Hive, Iceberg, or Delta tables on S3 or
S3-compatible storage with a streaming upload enabled can write corrupted data
files without reporting an error back to the users.</p>
<p>In 368-e streaming uploads are enabled by default. As a results users may
encounter this without taking any action to enable streaming in their
configuration.</p>
<p>In 367-e streaming upload are disabled by default. However it is a common
practice to enable streaming.</p>
<p>As a fix we recommend to set <code class="language-plaintext highlighter-rouge">hive.s3.streaming.enabled=false</code> and
<code class="language-plaintext highlighter-rouge">trino.s3.streaming.enabled=false</code> in all catalog property files using S3 with
the Hive, Iceberg or Delta Lake connector.</p>Today we are starting the year 2022 off with a great new release of Starburst Enterprise platform (SEP). Believe us, you do want to try out the refreshed Starburst Enterprise web UI.A Trino and dbt community success story2022-01-03T00:00:00-06:002022-01-03T00:00:00-06:00https://docs.starburst.io/blog/dbt-trino<p>The community around the open source project Trino is at the heart of Starburst.
The creators and many maintainers are on our teams. Our products, Starburst
Enterprise platform (SEP) and Starburst Galaxy are based on Trino. And today, I
tell you a story that shows how it takes a many people across the globe to
provide all the great features of Trino to our community of users and fellow
contributors.</p>
<!--more-->
<p>Let’s start our story in Europe. In Vienna, Austria to be precise. Marius got
interested into Trino there and started to learn more about it by trying it out.
It did not take long and he showed up on the <a href="https://trino.io/community.html">community
channels</a>.</p>
<p>Marius bumped into Brian and Manfred on the slack channels, and after a few
chats online and in video calls he joined us as guest in our <a href="https://trino.io/episodes/20.html">Trino Community
Broadcast episode 20</a>. We talked about his
experience as beginner, and his advances to documentation and code contributor.</p>
<p>In <a href="https://trino.io/episodes/21.html">episode 21</a>, Brian and Manfred learned
more about dbt and Trino. We also learned that back then we still
had to use the legacy Presto support. And the next day we had a proof of concept
implementation for Trino dbt support in our hands. Marius was at it
again!</p>
<p>With this implementation ready to go, Starburst and <a href="https://www.getdbt.com/">dbt
labs</a> started to collaborate towards a fully supported
release. And a few months later the benefits showed up for everyone.</p>
<p>Brian and Manfred recorded <a href="https://trino.io/episodes/30.html">episode 30 of the Trino Community
Broadcast</a> about real world usage of dbt at
Trino at Talkdesk in November. We were also joined by our dbt
champion at Starburst, Przemek, and learned about his work to make the
integration fully tested and feature rich.</p>
<p>Marius and Przemek worked with Starburst teams in the USA and Poland to
get the integration further improved. This resulted in a fully tested and
supported release 1.0.0 of
<a href="https://github.com/starburstdata/dbt-trino">dbt-trino</a> in December 2021. And of
course, this integration is open source and available for you all to test and
use. Watch the episodes and get started with <a href="../data-consumer/clients/dbt.html">our dbt client
documentation</a>.</p>
<p>Of course our thanks go out to all the other helpers on the way, including
<a href="https://github.com/leniartek">Piotr</a>,
<a href="https://www.linkedin.com/in/yuanamychen/">Amy</a>,
<a href="https://twitter.com/victorcouste">Victor</a>, and
<a href="https://twitter.com/jecabeda">José</a>. And if you got <a href="../data-consumer/clients/index.html">another client
integration</a>, or other feature to test or
contribute, don’t hesitate to reach out.</p>
<p><em>Manfred, Brian, and Marius</em></p>The community around the open source project Trino is at the heart of Starburst. The creators and many maintainers are on our teams. Our products, Starburst Enterprise platform (SEP) and Starburst Galaxy are based on Trino. And today, I tell you a story that shows how it takes a many people across the globe to provide all the great features of Trino to our community of users and fellow contributors.When it comes to security...2021-12-31T00:00:00-06:002021-12-31T00:00:00-06:00https://docs.starburst.io/blog/security<p>We’ve got you covered. The recent <a href="../security/log4shell.html">log4shell issue</a>
is the latest noise in the information security world. And as is often the case
with these issues, if you have your software development and information
security processes under control, there is more hype than reality around the
threats.</p>
<!--more-->
<p>You can see from our swift responses to the log4shell issue in the Trino
community](https://trino.io/blog/2021/12/13/log4shell-does-not-affect-trino.html),
and our own releases of Starburst Enterprise platform (SEP) LTS backports](/blog/2021-12-17-lts-backports.html) and <a href="/blog/2021-12-29-sep-367-e.html">a new STS release</a> that we have years of experience with security in our
teams. We were essentially not affected, but took the opportunity to trim our
binaries and tighten our processes yet again.</p>
<p>And to give you and idea, what we do to ensure your data secure we added a new
security section. It now details many of our processes and procedures:</p>
<ul>
<li><a href="../security/corporate.html">Corporate information security policies and procedures</a></li>
<li><a href="../security/development.html">Software development and security practices</a></li>
<li><a href="../security/starburst-enterprise.html">Security and Starburst Enterprise platform (SEP)</a></li>
<li><a href="../security/starburst-galaxy.html">Security and Starburst Galaxy</a></li>
</ul>
<p><a href="../security/contact.html">Contact us</a> to find out more.</p>
<p><em>Manfred and Keavy</em></p>We’ve got you covered. The recent log4shell issue is the latest noise in the information security world. And as is often the case with these issues, if you have your software development and information security processes under control, there is more hype than reality around the threats.Starburst Enterprise 367-e STS2021-12-29T00:00:00-06:002021-12-29T00:00:00-06:00https://docs.starburst.io/blog/sep-367-e<p>Promise! This is the last Starburst Enterprise platform (SEP) release of the
year 2021.</p>
<!--more-->
<p>With our <a href="/blog/2021-12-17-lts-backports.html">recent LTS backport release train</a> in your hands, we just had to cut a release of a
new STS to finish up the work around <a href="../security/log4shell.html">log4shell</a>.</p>
<p>Of course the release includes more than just upgrades and removal of unused
log4j binaries. In fact, if you look at the <a href="../latest/release/release-367-e.html">367-e release
notes</a> and the <a href="../latest/release/release-365-e.html">365-e release
notes</a> you see a number of new features
and changes that are definitely worth trying out. Specifically our <a href="../latest/insights.html">new default
web interface for SEP</a> includes numerous
things to play with:</p>
<ul>
<li>Simplified <a href="../latest/insights/index.html#configuration">configuration</a>.</li>
<li>Add <a href="../latest/insights/query-details.html">live query plan</a> display.</li>
<li>Add new <a href="../latest/data-products/index.html"><em>Data products</em></a> as
experimental feature.</li>
<li>Increased scope of <a href="../latest/admin/telemetry.html">telemetry data</a>.</li>
<li>New filters in <a href="../latest/insights/query-overview.html">query
history</a>.</li>
</ul>
<p>Even if you are still running the <a href="../364-e/index.html">364-e LTS release</a> in
productions, I strongly suggest you play with the new 367-e release. And of
SEP even more for the next releases. There is plenty coming
in 2022.</p>
<p><em>Manfred and Matt</em></p>
<p><strong>Update, 2022-01-27:</strong></p>
<p>In 368-e and 367-e, writing to Hive, Iceberg, or Delta tables on S3 or
S3-compatible storage with a streaming upload enabled can write corrupted data
files without reporting an error back to the users.</p>
<p>In 368-e streaming uploads are enabled by default. As a results users may
encounter this without taking any action to enable streaming in their
configuration.</p>
<p>In 367-e streaming upload are disabled by default. However it is a common
practice to enable streaming.</p>
<p>As a fix we recommend to set <code class="language-plaintext highlighter-rouge">hive.s3.streaming.enabled=false</code> and
<code class="language-plaintext highlighter-rouge">trino.s3.streaming.enabled=false</code> in all catalog property files using S3 with
the Hive, Iceberg or Delta Lake connector.</p>Promise! This is the last Starburst Enterprise platform (SEP) release of the year 2021.December 2021 LTS backport releases2021-12-17T00:00:00-06:002021-12-17T00:00:00-06:00https://docs.starburst.io/blog/lts-backports<p>We did it again! A bunch of new backport releases for our Starburst Enterprise
platform (SEP) LTS users are available.</p>
<!--more-->
<p>Following are the new available versions linking to the relevant release notes
entries:</p>
<ul>
<li><a href="../latest/release/release-350-e.html">350-e.18</a></li>
<li><a href="../latest/release/release-356-e.html">356-e.11</a></li>
<li><a href="../latest/release/release-360-e.html">360-e.8</a></li>
<li><a href="../latest/release/release-364-e.html">364-e.4</a></li>
</ul>
<p>Of course the related Helm charts and other resources are all available as part
of the update.</p>
<p>Beyond the usual backports for correctness issues, this release train had a
focus point around the log4shell vulnerabilities. Despite the fact that
Trino and Starburst Enterprise <a href="https://trino.io/blog/2021/12/13/log4shell-does-not-affect-trino.html">are not
affected</a>,
we decided to eliminate any potential risk of exposure. Now log4j binaries are
either gone completely or updated to unaffected version.</p>
<p>All the details can be found in the release notes and the dedicated <a href="../security/log4shell.html">security
advisory for log4shell</a> in our brand new <a href="../security/index.html">security
section</a>.</p>
<p><em>Manfred and Matt</em></p>We did it again! A bunch of new backport releases for our Starburst Enterprise platform (SEP) LTS users are available.