Installing Starburst Enterprise in a Kubernetes cluster#

Now that you satisfied all the requirements you are ready to proceed with installing Starburst Enterprise platform (SEP).

Overview#

Installing charts with Helm on your cluster follows the same steps:

  • Establish access to the Helm chart repository

  • Configure credentials to access the Docker registry and create the needed YAML file

  • Create a YAML file specific to the chart and cluster, for example sep-prod-cluster.yaml

  • Ensure your Helm/kubectl configuration points at the correct cluster with kubectl cluster-info

  • Run Helm to install the chart

  • Access the cluster and check for success

Each Helm chart includes a values.yaml file that sets a reasonable set of default values. This default setup does not include catalog definitions and others that are necessary for your cluster.

You have to change your values file to add or update any configuration and run a Helm upgrade to apply the changes to the cluster.

Iterate on the configuration in the values YAML file with minimal setup until you have a working system. Depending on your cluster you have to adjust memory requirements for the worker and coordinator and other settings. Inspect the message with kubectl or Octant to determine details.

After you achieved a running cluster, ensure to store the values YAML file as a reference and then add more details as required for the specific cluster need.

SEP installation#

The SEP installation is managed with the starburst-enterprise Helm chart. Use a minimal values YAML file with the registry credentials in the current directory to install SEP with the following command:

helm upgrade  my-sep starburstdata/starburst-enterprise \
  --install \
  --version 355.0.0 \
  --values ./registry-credentials.yaml
  --values ./sep-prod-cluster.yaml

The version value is available from the Helm repository.

The default values result in one coordinator and two worker nodes in the cluster. We strongly suggest that you install the default cluster, then make small, focused customizations to suit your organization.

Create this one file before you begin#

No matter what other configurations you need for your deployments, create a registry-credentials.yaml file for reuse across all clusters and charts. This helps to ensure a smooth install and deployment experience.

In your registry-credentials.yaml file, add the following:

registryCredentials:
  enabled: true
  registry: harbor.starburstdata.net/starburstdata
  username: <yourusername>
  password: <yourpassword>

The contents of this file override the default, empty values and ensure the Helm charts can be download the required Docker containers. Typically, you must use your username and password for the Starburst Harbor instance. If you want to use your organization’s internal Docker registry, you can.

If you have multiple clusters, this same file is used for all of them. You can also use the same file for the optional Ranger and Hive Metastore Service charts.

Installation checklist#

The following checklist describes the installation process:

  1. Gather repository credentials for the Helm chart repository and the Docker registry.

  2. Create the registry-credentials.yaml file to override the default, empty values.

  3. Create your correctly-sized Kubernetes cluster .

  4. Ensure your Helm/kubectl configuration points at the correct cluster with kubectl cluster-info.

  5. Create the YAML configuration file with all the custom configurations for your cluster, for example sep-prod-cluster.yaml.

  6. Run Helm to install the default chart, as well as any override YAML files using the --values argument, as in the following example:

helm upgrade sep-prod-cluster starburstdata/starburst-enterprise \
  --install \
  --version 350.0.0 \
  --values ./registry-credentials.yaml
  --values ./sep-prod-cluster.yaml
  1. Determine the IP address or the DNS hostname of the coordinator and verify the coordinator is running by accessing the Web UI. You can use the same details to connect with the CLI or the JDBC driver.

Next steps:

Updating to a new release#

If you have created focused, well-managed override files following our best practices guide, the upgrade process is a straightforward Helm-based process. As with any enterprise-scale application, we do recommend that you test upgrades from one release to another using a test cluster. This allows you to catch any configuration changes and update Helm charts before deploying into production:

  1. Review the Helm charts release notes for any relevant changes that affect your override files. For instance, there may be new configuration options to add, or deprecated properties to remove.

  2. Review the SEP release notes for new capabilities and breaking changes.

  3. Run Helm with the updated version with the updated YAML configuration files, as in the following example:

helm upgrade my-sep-test-cluster starburstdata/starburst-enterprise \
  --install \
  --version 350.5.0 \
  --values ./registry-credentials.yaml
  --values ./sep-prod-cluster.yaml

Hive Metastore Service installation#

The Hive Metastore Service installation is managed with the starburst-hive Helm chart. Installation follows the same approach as the SEP chart with a minimal values YAML file with the registry credentials and custom values file for your HMS configuration, for example hms-prod-cluster.yaml, in the current directory:

helm upgrade  my-sep starburstdata/starburst-hive \
  --install \
  --version 355.0.0 \
  --values ./registry-credentials.yaml
  --values ./hms-prod-cluster.yaml

The Configuring the Hive Metastore Service in Kubernetes page provides details for the configuration of the Hive Metastore Service.

Ranger installation#

The Ranger installation is managed with the starburst-ranger Helm chart. Installation follows the same approach as the SEP chart with a minimal values YAML file with the registry credentials and custom values file for your Ranger configuration, for example ranger-prod-cluster.yaml, in the current directory:

helm upgrade  my-sep starburstdata/starburst-ranger\
  --install \
  --version 355.0.0 \
  --values ./registry-credentials.yaml
  --values ./ranger-prod-cluster.yaml

The Configuring Starburst Enterprise with Ranger in Kubernetes provides details for the configuration of Apache Ranger.

Node assignment#

All charts you to configure criteria to define which node and pod in the cluster is suitable to use for running the relevant container. The following configurations are available, and by default not defined:

nodeSelector: {}
affinity: {}
tolerations: []

Example configurations are available in the k8s documentation. Specific usage and values highly depend on your k8s cluster configuration.

Further resources: