COVID-19 data lake #

The COVID-19 data lake connects to data found in the Registry of Open Data on AWS. This is a free use of an existing S3 data source that does not require access to an AWS account.

The data lives in S3. Amazon makes public access available without charge and without needing an AWS account. To access this data, connect to the COVID-19 data lake dataset catalog instead of an Amazon S3 data source catalog since the data is publicly accessible.

There is no actual data in the dataset catalog. The dataset catalog provides a connection point to allow you to run CREATE TABLE AS statements and create the data lake tables. Visit the data lake analytics tutorial to utilize the COVID-19 data lake catalog and run an example of these statements.

Create a catalog #

To create a catalog with the Covid-19 data lake, select Data > Catalogs in the navigation menu, and click Create catalog. Click on the Covid-19 data lake button in the Select a dataset section.

Define catalog name and description #

The Catalog name is visible in the query editor and other clients. It is used to identify the catalog when writing SQL or showing the catalog and its nested schemas and tables in client applications.

The name is displayed in the query editor and in the output of a SHOW CATALOGS command. It is used to fully qualify the name of any table in SQL queries following the catalogname.schemaname.tablename syntax. For example, you can run the following query in the sample cluster without first setting the catalog or schema context: SELECT * FROM tpch.sf1.nation;.

The Description is a short, optional paragraph that provides further details about the catalog. It appears in the Starburst Galaxy user interface and can help other users determine what data can be accessed with the catalog.

In the Region field, select the cloud vendor region where the sample data resides from the drop-down list of supported regions.

Connect catalog #

Click Connect catalog, and proceed to set permissions where you can grant access to certain roles.

Set permissions #

Use the following steps to assign read-only access to a specified set of roles:

1. Select the Read-only catalog switch to grant a set of roles read-only access to the catalog’s schemas, tables, and views. For catalogs that are pre-set to read-only, this switch is already set and is disabled.
2. Next, use the drop-down menu in the Role-level permissions section to specify the roles that have read-only access.
3. Click Save access controls.

Add to cluster #

You can add your catalog to a cluster later by editing a cluster. Click Skip to proceed to the catalogs page.

Use the following steps to add your catalog to an existing cluster or create a new cluster in the same cloud region:

• In the Add to cluster section, expand the menu in the Select cluster field.
• Select one or more existing clusters from the drop down menu.
• Click Create a new cluster to create a new cluster in the same region, and add it to the cluster selection menu.

• Click Add to cluster to view your new catalog’s configuration.

   

The Pending changes to clusters dialog appears when you try to add a catalog to a running cluster.

• In the Pending changes to cluster dialog, click Return to catalogs to edit the catalog or create a new catalog.
• Click Go to clusters to confirm the addition of the catalog to the running cluster.

• On the Clusters page, click the Update icon beside the running cluster, to add the catalog.

   

Next steps #

Now that you have created a catalog and added it to a cluster in Starburst Galaxy, follow the tutorial to query the COVID-19 data lake and learn more about the analytics capabilities of Starburst Galaxy.

SQL support #

The catalog provides read access to data and metadata in the COVID-19 dataset. It supports the following features:

• Globally available statements
• Read operations