Starburst Documentation
Starburst Galaxy
  •  Get started
  •  Working with data
  •  Data engineering
  •  Developer tools
  •  Cluster administration
  •  Security and compliance
  •  Troubleshooting
  • Galaxy status
  •  Reference

    • starburst galaxy > working with data > create catalogs > non object storage > Lakekeeper catalog

    Lakekeeper catalog #

    Use a Lakekeeper catalog to configure access to a Lakekeeper warehouse. Lakekeeper uses Apache Iceberg’s REST API to provide access to Iceberg tables on Azure Data Lake Storage.

    Follow these steps to create a catalog for Lakekeeper:

    1. In the navigation menu, select Data, then Catalogs.
    2. Click Create catalog.
    3. On the Create a catalog pane, click the Lakekeeper icon.

    Select a data source

    1. Configure the catalog as prompted in the dialog.
    2. Test the connection.
    3. Connect the catalog.
    4. Set any required permissions.
    5. Add the new catalog to a cluster.

    The following sections provide more detail for creating Lakekeeper catalog connections.

    Define catalog name and description #

    The Catalog name is visible in the query editor and other clients. It is used to identify the catalog when writing SQL or showing the catalog and its nested schemas and tables in client applications.

    The name is displayed in the query editor, and in the output of a SHOW CATALOGS command. It is used to fully qualify the name of any table in SQL queries following the catalogname.schemaname.tablename syntax. For example, you can run the following query in the sample cluster without first setting the catalog or schema context: SELECT * FROM tpch.sf1.nation;.

    The Description is a short, optional paragraph that provides further details about the catalog. It appears in the Starburst Galaxy user interface and can help other users determine what data can be accessed with the catalog.

    Enter catalog name and description

    Authentication to ADLS #

    To authenticate to ADLS, provide an ADLS storage account name, which you can find in your list of Resources in the Azure services section when you log into the Azure portal.

    Select a service principal alias from the drop-down list of configured service principals. If you have not yet configured a service principal for this Starburst Galaxy account, click Configure an Azure service principal to do so now. For more information about Azure service principals, see Azure service principals.

    Lakekeeper configuration #

    To configure the connection to your Lakekeeper catalog, provide the following details:

    • Lakekeeper REST URL: Endpoint URL of the Lakekeeper service.
    • Lakekeeper warehouse: Lakekeeper warehouse to connect to.
    • Lakekeeper Azure tenant ID: Lakekeeper OAuth2 tenant ID.
    • REST catalog OAuth2 client ID: Client ID used to access Lakekeeper.
    • REST catalog OAuth2 secret: REST catalog OAuth2 secret.
    • REST catalog OAuth2 scope client ID: Lakekeeper application ID to include in scopes requested from the IdP.

    Refer to Lakekeeper documentation for Entra-ID (Azure) on how to obtain these credentials. REST catalog OAuth2 client ID and REST catalog OAuth2 secret would typically refer to the Azure app created as “App 3” in the linked documentation. REST catalog OAuth2 scope client ID is “App 2.”

    Your browser might try to fill credential fields with your browser login name and password. If so, overwrite these fields with the correct values.

      Lakekeeper configuration

    Test the connection #

    Once you have configured the connection details, click Test connection to confirm data access is working. If the test is successful, you can save the catalog.

    If the test fails, look over your entries in the configuration fields, correct any errors, and try again. If the test continues to fail, Galaxy provides diagnostic information that you can use to fix the data source configuration in the cloud provider system.

    Connect catalog #

    Click Connect catalog, and proceed to set permissions where you can grant access to certain roles.

    Set permissions #

    This optional step allows you to configure read-only access or full read and write access to the catalog.

    Use the following steps to assign read-only access to all roles:

    1. Select the Read-only catalog switch to grant a set of roles read-only access to the catalog’s schemas, tables, and views.
    2. Next, use the drop-down menu in the Role-level permissions section to specify the roles that have read-only access.
    3. Click Save access controls.

    You can specify read-only access and read-write access separately for different sets of roles. That is, one set of roles can get full read and write access to all schemas, tables, and views in the catalog, while another set of roles gets read-only access.

    Use the following steps to assign read/write access to some or all roles:

    1. Leave the Read-only catalog switch cleared.
    2. In the Role-level permissions section:
      • Expand the drop-down menu in the Roles with read and write access field and select one or more roles to grant read and write access to.
      • Expand the drop-down menu in the Roles with read access field and select one or more roles from the list to grant read-only access to.
    3. Click Save access controls.

      Set permissions for read and write screenshot

    Add to cluster #

    You can add your catalog to a cluster later by editing a cluster. Click Skip to proceed to the catalogs page.

    Use the following steps to add your catalog to an existing cluster or create a new cluster in the same cloud region:

    • In the Add to cluster section, expand the menu in the Select cluster field.
    • Select one or more existing clusters from the drop down menu.
    • Click Create a new cluster to create a new cluster in the same region, and add it to the cluster selection menu.

    • Click Add to cluster to view your new catalog’s configuration.

        Add to cluster