Manage coordinators#
Use the following steps to manage coordinators in a high availability (HA) deployment.
Add a standby coordinator#
To add a standby coordinator:
Deploy a Starburst Enterprise platform (SEP) coordinator with the same environment configuration.
Configure the coordinator to announce itself to the Starburst control plane.
The coordinator automatically registers as
STANDBY.Health monitoring starts automatically.
Remove a coordinator#
To remove a coordinator from the HA pool:
Ensure at least one other healthy coordinator is running in the environment.
Stop the SEP coordinator, for example by scaling down a Kubernetes pod or stopping the service.
The coordinator record remains in the database with UNKNOWN status. To return
the coordinator to the HA pool, restart the coordinator. After health checks
succeed, the status changes to HEALTHY and the coordinator resumes in the
STANDBY role.
Trigger a hard failover#
A hard failover promotes a standby coordinator immediately. The cluster must use discovery, and a healthy standby coordinator must be ready.
To trigger a hard failover in the UI, see Trigger a hard failover. Only the owner of the cluster can trigger a hard failover, in the UI or with the API. If the action is unavailable in the UI, a tooltip says why.
To trigger a hard failover with the public API, send a POST request to
/public/api/v1/cluster/{clusterId}:failover. To check whether the cluster is
ready first, send a GET request to /public/api/v1/cluster/{clusterId}/status.
The response has the following fields:
failoverReadiness.code, with the valueReadyorNotReady.healthStatusof the active coordinator and of the standby coordinators, with the valueHealthy,NotHealthy, orUnknown.
If the control plane rejects the failover with HTTP status 409, the code
field of the response says why:
Code |
Meaning |
|---|---|
|
The cluster has no active coordinator. |
|
No healthy standby coordinator can take over. |
|
A failover is already running for the cluster. |
|
The cluster does not use discovery. |
|
The state of the cluster changed while the control plane processed the request. |
Note
The API returns these values in PascalCase. Earlier releases returned READY,
NOT_READY, HEALTHY, NOT_HEALTHY, UNKNOWN, NO_ACTIVE_COORDINATOR,
NO_HEALTHY_STANDBY, FAILOVER_IN_PROGRESS, and CONFLICT. Update any script
that compares them before you upgrade.
Trigger a failover by stopping the active coordinator#
To trigger a failover without a hard failover, stop the active SEP coordinator by scaling down a Kubernetes pod or stopping the service. The control plane detects the failure and promotes a healthy standby. This takes longer than a hard failover, because the control plane waits for three failed health checks. See How coordinator high availability works.