starburst galaxy > data engineering > optimization performance and quality > workload optimization > Query result caching

Query result caching #

Query result caching is an industry standard mechanism used to accelerate query execution. When a query is executed, the result is cached for a specified period of time. By caching the results of queries, Starburst Galaxy can rely on pre-computed results and can avoid unnecessarily processing queries if the results can be read from the cache. Retrieving a result directly from the cache substantially reduces query time.

For clusters created in AWS regions only, query result caching is enabled by default when you create a new cluster and select an AWS region.

Note: Cached results can contain personally identifiable information (PII) or other sensitive data. However, cached data is encrypted and is not directly consumable. All cached data is deleted 24 hours after it is written to cloud storage. You can manually delete a cluster’s cached results from the options menu in the Clusters pane.

In Starburst Galaxy, specify query result caching by selecting toggle_on Cache query results for improved query performance in the Create cluster or Edit cluster dialogs.

Additionally, set a duration for Cache reuse period by entering an integer and selecting Minutes or Hours from the drop-down menu. The cache reuse period must be between 5 minutes and 12 hours, inclusive. Run a query to cache its results. There is a 1MB size limit for cached results.

Note: Cache entries are not shared between users. Each logged-in user for a Galaxy account has their own query result cache.

For all subsequent runs of the same query as the same user on the same cluster within the cache reuse period, Galaxy fetches the cached results of the original query. Cached results are static and do not update dynamically based on changes in the source data.

The query insights pane includes a Cached results column that shows a circled checkmark for queries run on a cluster that has result set caching enabled.

Note: You might initially experience a delay in viewing query results from the cache after enabling it on an existing cluster. This delay is a one-time event, and subsequent queries do not encounter such delays once the cache is active. Cached results are isolated per cluster and cannot be fetched from a different cluster.

Is the information on this page helpful?

Yes

Cancel

Query result caching