External security in AWS #

You need to configure sufficient permissions to allow Starburst Galaxy access to your data sources in AWS. The details depend on the connected data source.

The following sections detail the different methods, and explain the necessary steps to configure security and permissions in AWS. You can then use the provided details in your catalog configuration in Starburst Galaxy.

AWS access and secret key #

You can use AWS access and secret keys to configure access to data in S3 and the metadata in Amazon Glue in your S3 catalogs.

Use the following steps to create a user with the necessary access:

  • Access the IAM service configuration from the Security, Identity, & Compliance section of the list of all services.
  • Select Users from the Access management menu.
  • Click on the Add users button.
  • Provide a user name.
  • Select Access key - Programmatic access in Select AWS credential type.
  • Click on the Next: Permissions button.
  • Select Attach existing policies directly in Set permissions.
  • Locate one or multiple of the following policies and select them with the checkbox beside the name:
    • AmazonS3ReadOnlyAccess, if the role is aimed for read access to S3 only.
    • AmazonS3FullAccess, if the role is aimed for read and write access to S3.
    • AWSGlueConsoleFullAccess, if the role is aimed to provide access to AWS Glue.
  • Click on the Next: Tags button.
  • Optionally configure any tags.
  • Click on the Next: Review button.
  • Review and click on the Create user button.

The success page provides the Access key ID directly on screen and allows you to access the Secret access key with the Show link. You need to make sure to copy both values for your use. The secret access key value is only available at this stage.

If you do not have the secret access key for the user, you need to use Create new access key in the Security credentials tab of the user Summary.

Now you can use the access key and secret key to configure the authentication for S3 and AWS Glue in your S3 catalogs.

Cross account IAM role #

You can use an AWS cross account IAM role to configure access to data in S3 and the metadata in Amazon Glue in your S3 catalogs. Create one role for all S3 catalogs and Amazon Glue instances, or split it up into multiple roles, as desired.

Use the following steps to create the role or roles with suitable permissions in your AWS console:

  • Access the IAM service configuration from the Security, Identity, & Compliance section of the list of all services.
  • Select Roles from the Identity and Access management menu.
  • Click on the Create role button.
  • Use AWS account in the Select type of trusted entity section.
  • Select Another AWS account in the An AWS account section.
  • Copy the value Starburst AWS account ID from the Starburst Galaxy user interface, and paste it in the Account ID field.
  • Enable Require external ID.
  • Copy the value External ID from the Starburst Galaxy user interface, and paste it in the External ID field.
  • Click on the Next: Permissions button.
  • Locate one or multiple of the following policies and select them with the checkbox beside the name:
    • AmazonS3ReadOnlyAccess, if the role is aimed for read access to S3 only.
    • AmazonS3FullAccess, if the role is aimed for read and write access to S3.
    • AWSGlueConsoleFullAccess, if the role is aimed to provide access to AWS Glue.
  • Click on the Next: Tags button.
  • Optionally configure any tags.
  • Click on the Next: Review button.
  • Add a meaningful Role name, such as starburst-galaxy-access, and provide details in the Description.
  • Click on the Create role button to finish.

Once the role is created, you can access the necessary details or update the role as desired:

  • Use the Search filter to locate the role by name.
  • Click on the role name in the list of roles.
  • Copy the value of the Role ARN from the Summary section to provide the value in Starburst Galaxy. The ARN value looks similar to arn:aws:iam::youraccountid:role/starburst-galaxy-access.

Now you can use the Role ARN value to configure the authentication for S3 and AWS Glue in your S3 catalogs.