Starburst Documentation
Starburst Galaxy
  • Overview
  • Get started
  •   Query
  •   Catalogs
  •   Catalog explorer
  •   Clusters
  •   SQL
  •   Admin
  •   Access control
  •   Cloud settings
  •   Security
  •   Single sign-on
  • API
  •   Tutorials
  •   Troubleshooting
  • Release notes
  • Status

    • starburst galaxy > catalogs > Catalogs overview

    Catalogs overview #

    A catalog contains the configuration that allows Starburst Galaxy to access a data source.

    To query a data source in Galaxy, configure a catalog for it, and include that catalog in a cluster. Once the catalog is defined and used in a cluster, you can query the data source by accessing the catalog and its nested schemas and tables.

    Data sources and clusters must be located in the same cloud provider and region to enable optimal performance and avoid unnecessary data transfer costs.

    Access to create and manage catalogs is provided through the Catalogs item in the navigation menu.

    Data sources #

    Starburst Galaxy facilitates access to numerous different data sources. Configuration for object storage systems, data warehouses, relational databases, and other systems varies by cloud and hosting provider. If your data source has secured or locked down network access, you may need to configure its network to admit one or more of Starburst Galaxy’s outgoing IP blocks as shown on the IP allow list.

    The following sections provide links to the configuration pages for the data source catalogs supported by Starburst Galaxy.

    Object storage #

    • Amazon S3, object storage on Amazon Web Services, combined with Amazon Glue, your own Hive Metastore Service, or the Starburst Galaxy metastore
    • Azure Data Lake Storage, object storage on Microsoft Azure, combined with your own Hive Metastore Service, or the Starburst Galaxy metastore

    • Google Cloud Storage, object storage on Google Cloud, combined with your own Hive Metastore Service, or the Starburst Galaxy metastore

      Select object storage

    Warehouses and databases #

    • MySQL, relational database in numerous variants on Amazon Web Services, Google Cloud, or Microsoft Azure
    • PostgreSQL, relational database in numerous variants on Amazon Web Services, Google Cloud, or Microsoft Azure
    • SQL Server, relational database in numerous variants on Amazon Web Services, Google Cloud, or Microsoft Azure
    • Amazon Redshift, a fast, fully managed, petabyte-scale data warehouse service.
    • Azure Synapse, an analytics service that brings together data integration, enterprise data warehousing, and big data analytics.
    • Google BiqQuery, a serverless, scalable, cost-effective multicloud data warehouse.
    • Snowflake, a cloud-based data platform.

    • MongoDB or MongoDB Atlas data platform.

      Select a warehouse or database

    Sample datasets #

    Starburst Galaxy also provides access to a number of full datasets. You can create a catalog for these datasets, and use them for a number of purposes:

    • Easy demonstration of Starburst Galaxy features without the need to configure an external data source.
    • Availability of a full dataset to query, learn SQL, and experiment with different clients.
    • Performance and other benchmark tests with well known data and standardized queries.

    The following dataset catalogs are available:

    AWS COVID-19 data lake.

    See the Introductory project tutorials for examples of using this dataset.

    Sample dataset

    Provides data in two tables that represent space mission data.

    TPC-H

    Provides a set of schemas to support the TPC Benchmark™ H database, which is a benchmark used to measure the performance of highly-complex decision support databases.

    TPC-DS

    Provides a set of schemas to support the TPC Benchmark™ DS database, which is a benchmark used to measure the performance of complex decision support databases.

    Select a dataset

    Manage catalogs #

    You can create, view, and management catalogs in the View catalogs pane. This pane also provides access to the features of the catalog explorer, which allow you to browse through the metadata of a catalog.

      View catalogs

    Create a catalog #

    To create a new catalog, click Create catalog. The new catalog appears in the list of catalogs.

    List of catalogs #

    The list of catalogs displays the following information about each catalog:

    • Name: The name of the catalog.
    • Kind: The data source type. For example, Amazon S3.
    • Description: The description provided for the catalog, if any.
    • Cloud: The cloud service provider.
    • Region: The cloud service provider region.

    Use beside the column names to sort catalogs in the list.

    Search catalogs #

    Use the search field to find catalogs by name, kind, and description.

    Edit a catalog #

    You can edit a catalog’s configuration details on the Edit catalog page. To access the editing page, click or, to open the same page from the options menu, click , then Edit configuration. The same editing options are available during the catalog creation process.

    Change owner #

    To change the owner of the catalog to a different role, click , then Change owner.

    Delete a catalog #

    To delete a catalog, click , then Delete catalog.

    Add a catalog to a cluster #

    Once you have configured a catalog, add it to a cluster to query the data source. You have the option to add a catalog to a cluster during the catalog configuration process or later by using the Edit cluster option.