Create an AKS cluster #

This page describes how to configure a new AKS cluster that ensures all SEP resources are co-located and follow best practices.

Prerequisites #

Ensure that you have the following tools, policies, and certificates before creating a Kubernetes cluster for SEP in AKS:

  • helm
  • kubectl
  • Azure CLI (az)
  • Azure resource group for the SEP nodes
  • Virtual network assigned for the resource group
  • IAM policies for ADLS, S3, as desired
  • CA-signed certificate for HTTPS/TLS (for a domain such as starburst.example.com) if using AD/LDAP authentication

Create your Azure cluster #

It is strongly recommended to have your SEP coordinator and workers share the same resource group. The following example az aks create command creates the “sep-example” cluster in the “example-rg” resource group:

$ az aks create --kubernetes-version 1.20.7 --name sep-example --resource-group example-rg \
  --vnet-subnet-id /subscriptions/1234abcd-a1b2-c3d4-e5f6-example/resourceGroups/example-rg/providers/Microsoft.Network/virtualNetworks/example-network/subnets/default \
  --service-cidr 10.10.0.0/16 \
  --dns-service-ip 10.10.0.10 \
  --docker-bridge-address 172.16.0.1/16 \
  --location eastus \
  --zones 1 \
  --network-plugin azure \
  --node-vm-size standard_ds2_v2 \
  --enable-aad \
  --aad-admin-group-object-ids aabbccdd-1a2b-3c4d-5d6f-example \
  --assign-identity /subscriptions/1234abcd-a1b2-c3d4-e5f6-example/resourcegroups/example-rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/example-identity-policy \
  --enable-cluster-autoscaler \
  --node-count 1 \
  --min-count 1 \
  --max-count 3 \
  --nodepool-name systempool \
  --node-osdisk-size 64

Establish nodepools #

The best practice is to create one nodepool for your SEP coordinator and another for worker nodes. The following az aks nodepool add command creates a nodepool for a coordinator:

$ az aks nodepool add --cluster-name sep-example --resource-group example-rg \
  --name sep-coordinator \
  --labels apps=sep-coordinator \
  --priority Spot \
  --node-vm-size standard_d8s_v3 \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --enable-cluster-autoscaler \
  --node-count 1 \
  --min-count 1 \
  --max-count 2 \
  --node-osdisk-size 64 \
  --node-osdisk-type Ephemeral \
  --no-wait

The following command creates a scaling nodepool for a minimum of two workers:

$ az aks nodepool add --cluster-name sep-example --resource-group example-rg \
  --name sep-workers \
  --labels apps=sep-workers \
  --priority Spot \
  --node-vm-size standard_d8s_v3 \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --enable-cluster-autoscaler \
  --node-count 2 \
  --min-count 2 \
  --max-count 4 \
  --node-osdisk-size 64 \
  --node-osdisk-type Ephemeral \
  --no-wait