Operating Starburst Enterprise in a Kubernetes cluster#

Congratulations! Your cluster is up and running. Now it is time to test it out by connecting your clients like the CLI or some BI application using the JDBC driver.

With that usage you gain insights on the correct configuration, data sources and sizing of the cluster for the desired usage and performance. This in turn leads you to further work such as updating configuration, scaling your cluster or even upgrading to a newer release.

This document covers these and other aspects associated to running your SEP cluster, or even clusters.

Running#

Once you have successfully installed SEP, you can confirm that the coordinator pod is running with tools such as kubectl or Octant.

You can also check the log in the server pod. A successful start of the coordinator, and also each worker, shows with a log entry similar to the following snippet:

INFO  main    io.trino.server.Server  ======== SERVER STARTED ========

With these checks performed, you can access the Web UI on the coordinator. Determine the FQDN and port of the coordinator as configured in the expose section.

A default deployment exposes the UI via HTTP and does not require a password. Production deployments should use HTTPS and have authentication configured.

The main page of the Web UI displays the number of connected workers. Confirm that the number is identical to the number specific in the worker section. The default number of workers is two.

Connecting tools#

With the knowledge that you can access the Web UI, and therefore the coordinator, and that some workers are running, you are ready to connect client tools for running queries.

Use the protocol, FQDN and port with the --server option for starting the CLI to access SEP:

trino --server https://trino.example.com:9999

The same URL can be used to configure the JDBC driver or ODBC driver connection and connection from any other client.

Updates#

From time to time you need to make incremental changes to your configuration or move to a new release. To do so:

  • Create or update the correct YAML files with the new or changed configuration properties.

  • Run the helm upgrade command with the updated version of the chart.

  • Verify that the desired changes are applied successfully. We suggest making changes to one area of the application at a time.

  • Repeat the process with the next desired change.

Typically Helm applies the relevant changes in a granular fashion. For example, if you change the worker configuration, no coordinator changes are performed and the coordinator continues to run while workers are updated.

If you need to ensure that pods are recreated completely, you can use the --recreate-pods option for your helm command. This essentially restarts all workers as well as the coordinator and therefore restarts the cluster. This includes a downtime for users.

Scaling#

You can scale your SEP cluster manually by adding more workers. This is done by updating the worker replicas and updating your deployment. Sufficient resources available in the cluster is a requirement.

Alternatively, you can enable autoscaling in your worker configuration and updating the deployment. Sufficient resources available in the cluster is a requirement.

You can also increase the memory resources allocated to the workers and updating your deployment, if sufficiently sized pods are available and you recreate pods as part of the update.