Key concepts#

This topic defines the main terms in the Starburst control plane documentation, and shows how the parts of a deployment fit together.

Terminology#

Control plane#

Control plane is the single access layer for all activity in a multi-cluster SEP deployment. Your users and applications connect to control plane, not to individual clusters. Control plane receives each query, chooses a cluster group for it, and passes it to a cluster in that group by using routing rules.

Control plane contains the services that work across your clusters, such as query routing and smart load balancing. For the full list and the status of each service, see Components and their status.

Cluster#

A cluster is a SEP cluster: a coordinator and workers that run queries. Control plane does not run queries itself. It sends each query to a cluster, and the cluster runs it.

Each cluster belongs to a cluster group. A cluster can also have a standby coordinator, which makes the cluster available when its active coordinator fails.

Cluster group#

A cluster group is a set of clusters that control plane treats as a destination for a set of queries. The clusters in a group are interchangeable, so that a query returns the same result on any of them. They have the same execution modes and SEP version, though they can differ in size. In standalone mode, they also have the same catalogs, because each cluster keeps its own. See Cluster requirements.

Use cluster groups for two purposes:

  • Isolate workloads. Put each workload in its own cluster group, so that workloads do not interfere with each other.

  • Balance the load. Put identical clusters in one cluster group for horizontal scalability. Control plane load balances the queries of the group across those clusters with smart load balancing.

Exactly one cluster group is the default. Control plane sends a query that matches no routing rule to the default cluster group.

Tag-based query routing#

Control plane uses tags to decide which cluster group receives a query. A tag is a label that control plane attaches to a query. You write routing rules that match the tags of a query to a cluster group.

System tags#

With auto tag-based routing, control plane adds system tags to each query automatically. The tags come from two sources:

  • The HTTP request: system.request.source, from the X-Trino-Source header, and system.request.clientTag, from the X-Trino-Client-Tags header.

  • The signed-in user: the user name (system.identity.user), the groups of the user (system.identity.group), and the roles of the user (system.identity.role).

Custom tags#

You create custom tags with tagging rules. A tagging rule reads the system tags of a query and produces a new tag. For example, a rule can set the tag tier to premium when the user belongs to the group premium_users. A routing rule can then send queries with tier set to premium to a particular cluster group.

Deployment modes#

Control plane runs with clusters configured for one of two modes:

  • Managed mode. Control plane holds the catalogs for all of your clusters. Starburst access control manages access to them from the control plane, unless you specify the catalog as delegated for built-in access control. In the configuration of each SEP cluster, you set deployment.mode to MANAGED, and you register the cluster in control plane with discovery turned on. For the steps, see Connect a cluster to control plane.

    Managed mode also requires control-plane.enabled=true in the control plane’s config.properties file, which turns on centralized catalogs. See Install and configure control plane for more information.

  • Standalone mode (not recommended for new deployments). Each cluster keeps its own catalogs and its own built-in access control. Standalone is the default deployment mode of a SEP cluster, so you do not need to set deployment.mode. For more information, see Control plane in standalone mode.