Create an AKS cluster#

This page describes how to configure a new AKS cluster that ensures all SEP resources are co-located and follow best practices.

Warning

SEP has specific requirements for sizing, placement, and sharing of resources. You must ensure that your AKS cluster meets all requirements described in our cluster requirements section.

Prerequisites#

Ensure that you have the following tools, policies, and certificates before creating a Kubernetes cluster for SEP in AKS:

  • helm

  • kubectl

  • Azure CLI (az)

  • Azure resource group for the SEP nodes

  • Virtual network assigned for the resource group

  • IAM policies for ADLS, S3, as desired

  • CA-signed certificate for HTTPS/TLS (for a domain such as starburst.example.com) if using AD/LDAP authentication

Create your Azure cluster#

It is strongly recommended to have your SEP coordinator and workers share the same resource group. The following example az aks create command creates the sep-example cluster in the example-rg resource group:

$ az aks create --kubernetes-version <supported-version> --name sep-example --resource-group example-rg \
  --vnet-subnet-id /subscriptions/1234abcd-a1b2-c3d4-e5f6-example/resourceGroups/example-rg/providers/Microsoft.Network/virtualNetworks/example-network/subnets/default \
  --service-cidr 10.10.0.0/16 \
  --dns-service-ip 10.10.0.10 \
  --docker-bridge-address 172.16.0.1/16 \
  --location eastus \
  --zones 1 \
  --network-plugin azure \
  --node-vm-size standard_ds2_v2 \
  --enable-aad \
  --aad-admin-group-object-ids aabbccdd-1a2b-3c4d-5d6f-example \
  --assign-identity /subscriptions/1234abcd-a1b2-c3d4-e5f6-example/resourcegroups/example-rg/providers/Microsoft.ManagedIdentity/userAssignedIdentities/example-identity-policy \
  --enable-cluster-autoscaler \
  --node-count 1 \
  --min-count 1 \
  --max-count 3 \
  --nodepool-name systempool \
  --node-osdisk-size 64

You must adjust parameters such as the VM size for the nodes, the available disk space, node counts, and specific identifiers to your planned deployment.

Establish nodepools#

The best practice is to create one nodepool for your SEP coordinator and another for worker nodes. The following az aks nodepool add command creates a nodepool for a coordinator:

$ az aks nodepool add --cluster-name sep-example --resource-group example-rg \
  --name sep-coordinator \
  --labels apps=sep-coordinator \
  --node-vm-size standard_d8s_v3 \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --enable-cluster-autoscaler \
  --node-count 1 \
  --min-count 1 \
  --max-count 2 \
  --node-osdisk-size 64 \
  --node-osdisk-type Ephemeral \
  --no-wait

The following command creates a scaling nodepool for a minimum of two workers:

$ az aks nodepool add --cluster-name sep-example --resource-group example-rg \
  --name sep-workers \
  --labels apps=sep-workers \
  --node-vm-size standard_d8s_v3 \
  --eviction-policy Delete \
  --spot-max-price -1 \
  --enable-cluster-autoscaler \
  --node-count 2 \
  --min-count 2 \
  --max-count 4 \
  --node-osdisk-size 64 \
  --node-osdisk-type Ephemeral \
  --no-wait

You must adjust parameters such as the VM size for the nodes, the available disk space, node counts, and specific identifiers to your planned deployment.