Key use cases#

This topic describes the three main ways to use Starburst control plane: load balancing, workload isolation, and high availability. For each use case, it lists what to consider, what you need, and what limits apply.

Load balancing#

Load balancing is a workload management strategy that lets you scale a workload beyond what a single SEP cluster can handle by spreading queries across multiple clusters.

If your workload has sudden spikes in demand, is high in volume, or is sensitive to latency, run multiple identical clusters in a single cluster group. Each query then has the resources it needs, which helps you protect query performance and meet your performance service-level agreements (SLAs). Adding clusters also reduces the load on any single coordinator.

Use query routing and smart load balancing to automatically distribute queries to clusters within a cluster group based on the existing workload of each cluster to handle high concurrency demands.

Considerations#

Before you set up load balancing, think about the following:

  • How many clusters a cluster group needs. Use at least two active and healthy clusters, so that control plane has a choice. Each cluster runs up to its query limit at once.

  • How large each cluster should be. Clusters in a group can have different sizes. Set the query limit of each cluster to match its capacity, so that control plane does not send it more queries than it can run. See Set query limits.

  • Which strategy to use. With RANDOM, the default, control plane picks a random cluster from the clusters that have capacity. With QUERY_COUNT_BASED, control plane picks the cluster with the lowest load, which is the number of queries that run on it. If the smart load balancing statement queue is disabled, for example in standalone mode, queries can also queue on the cluster, and control plane counts those too. See Set the load balancing strategy.

  • What limit to set. The query limit of a cluster group is the number of queries that can run at the same time on each cluster in the group. It is a whole number from 1 to 10,000, and new groups start with a limit of 100. A cluster without a limit of its own uses the limit of its group. A limit that you set on a single cluster overrides the limit of the group. See Set query limits.

  • How long a query can wait. When every cluster is at its limit, a statement waits in the statement queue. By default, it expires after 10 minutes. Usually, control plane starts a queued statement almost immediately when a cluster has capacity. In rare cases, for example when there is a problem, it can take up to 10 seconds. Your client applications should tolerate a short delay.

Requirements#

To balance the load, you need:

  • Clusters in the cluster group that are configured identically. However, clusters can vary in size. See Cluster requirements.

  • If the clusters in a group vary in size, a cluster-level query limit on each cluster that matches its capacity. Smart load balancing then sends each cluster a suitable load. See Set query limits.

  • At least two active and healthy clusters in the cluster group.

  • Smart load balancing turned on, so that the limits and the load-aware selection apply. It is on by default with auto tag-based routing and cluster authentication. See Smart load balancing.

Set the load balancing strategy#

Use the gateway.load-balancing.strategy property in the control plane’s config.properties file to choose how control plane distributes queries across the healthy clusters in a cluster group.

Property

Description

Default

gateway.load-balancing.strategy

Algorithm that distributes queries across healthy clusters in a cluster group. Choose between the following strategies:

  • RANDOM: Distribute queries randomly across healthy clusters

  • QUERY_COUNT_BASED: Route based on active query count

RANDOM

With smart load balancing on, both strategies skip the clusters that are at their query limit. If every cluster is at its limit, the statement waits in the queue. With smart load balancing off, control plane applies the strategy that you pick to all healthy clusters. See Smart load balancing.

Limitations#

  • The query limits, and the load-aware selection that depends on them, apply only when smart load balancing is on. The smart load balancing statement queue is on by default with auto tag-based routing, and it is disabled in standalone mode. Without the queue, control plane ignores the limits. See How load balancing works.

  • Load balancing works within one cluster group. Control plane does not spread queries across groups.

  • Control plane picks a cluster when a client submits a query. It does not move a query that is running.

  • A statement waits in the queue for up to the maximum queued time, 10 minutes by default. The queue holds up to 100,000 statements, and control plane rejects a new statement when the queue is full.

  • QUERY_COUNT_BASED counts queries. It does not take the size or the resource use of a query into account.

  • A statement that a client sends directly to a cluster, through the address of the cluster, is not load balanced.

Workload isolation#

Workload isolation is a workload management strategy that keeps dissimilar workloads from interfering with each other by running them on separate SEP clusters. You put each workload in its own cluster group, and tag-based query routing sends each query to the right group based on criteria you specify.

Isolating workloads keeps one workload from slowing down another, and lets you size and configure each cluster group for its workload. You can use it to meet different performance or resource availability SLAs, to prioritize critical workloads, or to separate the workloads of different teams.

Considerations#

  • How to divide your workloads. Decide the criteria that separate your workloads, for example the team that submits the query or the application. Each group of workloads needs a cluster group. With auto tag-based routing, control plane routes on the identity of the user, so you do not need to change your applications. You might divide workloads based on factors like:

    • Different SLAs

    • Different workloads (interactive analysis, extract, transform, and load (ETL) processing)

    • Different teams (marketing, finance)

  • Which workloads go to the default cluster group. A query that matches no routing rule goes to the default cluster group. Decide which workloads you want there.

  • How much capacity each group needs. You can set a limit on the number of queries that run at once for each cluster group and for each cluster. When every cluster in a group is at its limit, new queries wait in the statement queue. See Set query limits.

  • Workload isolation and load balancing. A cluster group can contain a single cluster. To scale an isolated workload horizontally, add more clusters to its group, and control plane then load balances the workload’s queries across them.

  • Background work. Control plane routes the scheduled refresh of a materialized view with the same routing rules. If the refreshes are large, give them their own cluster group. See How routing works.

Workload-optimized clusters#

After you isolate workloads to specific cluster groups, size and configure the clusters within each group to meet the requirements of the workload. For example, for a customer-facing application that requires low latency, enable Warp Speed on the clusters in that cluster group.

Requirements#

Before you can isolate workloads, you need:

  • Two or more cluster groups, each with at least one active and healthy cluster. See Manage cluster groups.

  • Clusters in each group that are interchangeable: the same execution modes and SEP version. In standalone mode, each cluster must also have the same catalogs, because each cluster keeps its own. See Cluster requirements.

  • Routing rules that send queries to the cluster groups, and, for custom tags, tagging rules. See Create routing rules and Create tagging rules.

  • For managed mode: a role with the privilege MANAGE_QUERY_ROUTING_RULES to create the rules. To choose a cluster group for a rule, you need one of the privileges MANAGE_QUERY_ROUTING_RULES, MANAGE_SECURITY, or CREATE_CLUSTER.

Limitations#

  • If you delete the default routing rule, control plane rejects a query that matches no other rule.

  • A statement that a client sends directly to a cluster, through the address of the cluster, is not routed by your rules, and control plane does not load balance it.

High availability#

High availability keeps your queries running when part of your deployment fails. Control plane offers it for two parts of the deployment: control plane itself, and the coordinators of your clusters. You can enable high availability for control plane, for your cluster coordinators, or for both.

High availability of control plane. You run several control plane instances. They share one backend database, and a load balancer in front of them sends each new connection to a healthy instance. If an instance fails, the load balancer sends new connections to the other instances.

High availability of coordinators. Each cluster has one active coordinator. To make the cluster highly available, you add one or more standby coordinators. Control plane checks the health of every coordinator. When the active coordinator fails, control plane promotes a healthy standby coordinator, and it sends the query traffic to it. You can also trigger a failover yourself. See Trigger a hard failover.

Considerations#

Before you set up high availability, think about the following.

For high availability of control plane:

  • How many instances of control plane to run. Run at least two. If you use Kubernetes, spread the instances across nodes, so that the failure of one node does not stop all of them.

  • How your clients behave. Clients that poll for results, such as the Trino CLI, retry on their own when a control plane instance fails. The web UI resumes a query only if the browser tab stays open.

For high availability of coordinators:

  • How many coordinators each cluster needs. Each cluster needs at least two coordinators of the same size. Each standby coordinator is an extra coordinator that you deploy and run.

  • How long a failover takes. Control plane promotes a standby coordinator after three failed health checks over 10 seconds, so a failover takes about 10 to 15 seconds. If a coordinator becomes unresponsive without disconnecting, a failover can take up to two minutes. To shorten this time, set clusters.http-client.request-timeout.

  • How your clients behave. Control plane does not retry a query that is running when a coordinator fails. Configure your client application to retry failed queries.

  • Where the clusters announce themselves. Coordinators and workers announce themselves to control plane, through the discovery.uri property. In managed mode, every cluster uses discovery.

Requirements#

For high availability of control plane:

  • A backend database that all the instances share.

  • A load balancer or reverse proxy in front of the instances.

  • The same version of control plane on every instance.

  • The same signing secret on every instance, so that each instance validates web UI sessions.

For the steps, see Configure control plane high availability.

For high availability of coordinators:

  • At least two coordinators in each environment, with the same size.

  • The same node.environment value on all nodes of a cluster.

  • Network access in both directions between control plane and every coordinator, for announcements and health checks.

  • The same configuration on every coordinator, including catalogs, connectors, and security settings.

  • The discovery.uri property on the coordinators and the workers, set to the address of control plane.

For the steps, see Configure coordinator high availability.

Limitations#

High availability of control plane

  • A query survives the failure of an instance only if the client can retry against another instance. A query that you run in the web UI is lost if you refresh or close the tab.

High availability of coordinators

  • Coordinator high availability works within one cluster. Each cluster keeps its own active coordinator, and control plane does not move a query to another cluster.

  • Control plane does not retry running queries when a coordinator fails.

  • A coordinator that control plane removed from the pool returns only after you restart it.

  • If you deploy SEP with Helm and set internalTls: true, the SEP init container overrides the discovery.uri that you set. The Helm chart does not support coordinator high availability with this setting. This requires custom setup.