Getting started

• 1: Install EKS Anywhere
• 2: Create local cluster
• 3: Create production cluster

1 - Install EKS Anywhere

EKS Anywhere will create and manage Kubernetes clusters on multiple providers. Currently we support creating development clusters locally with Docker and production clusters using VMware vSphere. Other deployment targets will be added in the future, including bare metal support in 2022.

Creating an EKS Anywhere cluster begins with setting up an Administrative machine where you will run Docker and add some binaries. From there, you create the cluster for your chosen provider. See Create cluster workflow for an overview of the cluster creation process.

To create an EKS Anywhere cluster you will need eksctl and the eksctl-anywhere plugin. This will let you create a cluster in multiple providers for local development or production workloads.

Administrative machine prerequisites

• Docker 20.x.x
• Mac OS (10.15) / Ubuntu (20.04.2 LTS)
• 4 CPU cores
• 16GB memory
• 30GB free disk space

NOTE: If you are using Ubuntu use the Docker CE installation instructions to install Docker and not the Snap installation.

NOTE: If you are using Mac OS Docker Desktop 4.4.2 or newer "deprecatedCgroupv1": true must be set in ~/Library/Group\ Containers/group.com.docker/settings.json.

NOTE: Currently newer versions of Ubuntu (21.10) and other linux distributions with cgroup v2 enabled are not supported.

Install EKS Anywhere CLI tools

Via Homebrew (macOS and Linux)

You can install eksctl and eksctl-anywhere with homebrew . This package will also install kubectl and the aws-iam-authenticator which will be helpful to test EKS Anywhere clusters.

brew install aws/tap/eks-anywhere

Manually (macOS and Linux)

Install the latest release of eksctl. The EKS Anywhere plugin requires eksctl version 0.66.0 or newer.

curl "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" \
    --silent --location \
    | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin/

Install the eksctl-anywhere plugin.

export EKSA_RELEASE="0.8.2" OS="$(uname -s | tr A-Z a-z)" RELEASE_NUMBER=10
curl "https://anywhere-assets.eks.amazonaws.com/releases/eks-a/${RELEASE_NUMBER}/artifacts/eks-a/v${EKSA_RELEASE}/${OS}/amd64/eksctl-anywhere-v${EKSA_RELEASE}-${OS}-amd64.tar.gz" \
    --silent --location \
    | tar xz ./eksctl-anywhere
sudo mv ./eksctl-anywhere /usr/local/bin/

Upgrade eksctl-anywhere

If you installed eksctl-anywhere via homebrew you can upgrade the binary with

brew update
brew upgrade eks-anywhere

If you installed eksctl-anywhere manually you should follow the installation steps to download the latest release.

You can verify your installed version with

eksctl anywhere version

Deploy a cluster

Once you have the tools installed you can deploy a local cluster or production cluster in the next steps.

• Create local cluster
• Create production cluster

2 - Create local cluster

EKS Anywhere docker provider deployments

EKS Anywhere supports a Docker provider for development and testing use cases only. This allows you to try EKS Anywhere on your local system before deploying to a supported provider.

To install the EKS Anywhere binaries and see system requirements please follow the installation guide .

Steps

1. Generate a cluster config

   CLUSTER_NAME=dev-cluster
   eksctl anywhere generate clusterconfig $CLUSTER_NAME \
      --provider docker > $CLUSTER_NAME.yaml
   

The command above creates a file named eksa-cluster.yaml with the contents below in the path where it is executed. The configuration specification is divided into two sections:

• Cluster

• DockerDatacenterConfig

  apiVersion: anywhere.eks.amazonaws.com/v1alpha1
  kind: Cluster
  metadata:
  name: dev-cluster
  spec:
  clusterNetwork:
     cni: cilium
     pods:
        cidrBlocks:
        - 192.168.0.0/16
     services:
        cidrBlocks:
        - 10.96.0.0/12
  controlPlaneConfiguration:
     count: 1
  datacenterRef:
     kind: DockerDatacenterConfig
     name: dev-cluster
  externalEtcdConfiguration:
     count: 1
  kubernetesVersion: "1.21"
  managementCluster:
     name: dev-cluster
  workerNodeGroupConfigurations:
  - count: 1
     name: md-0
  ---
  apiVersion: anywhere.eks.amazonaws.com/v1alpha1
  kind: DockerDatacenterConfig
  metadata:
  name: dev-cluster
  spec: {}
  

  Some key considerations and configuration parameters:

  • Apart from the base configuration, you can add additional optional configuration to enable supported functionalities
    • OIDC
    • etcd
    • proxy
    • gitops

For full EKS Anywhere configuration reference for a VMware vSphere cluster and explanation on each parameter in the configuration generated above refer vSphere configuration

1. Create a cluster

   eksctl anywhere create cluster -f $CLUSTER_NAME.yaml
   

   Example command output

   Performing setup and validations
   ✅ validation succeeded {"validation": "docker Provider setup is valid"}
   Creating new bootstrap cluster
   Installing cluster-api providers on bootstrap cluster
   Provider specific setup
   Creating new workload cluster
   Installing networking on workload cluster
   Installing cluster-api providers on workload cluster
   Moving cluster management from bootstrap to workload cluster
   Installing EKS-A custom components (CRD and controller) on workload cluster
   Creating EKS-A CRDs instances on workload cluster
   Installing AddonManager and GitOps Toolkit on workload cluster
   GitOps field not specified, bootstrap flux skipped
   Deleting bootstrap cluster
   🎉 Cluster created!
   

2. Use the cluster

   Once the cluster is created you can use it with the generated KUBECONFIG file in your local directory

   export KUBECONFIG=${PWD}/${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig
   kubectl get ns
   

   Example command output

   NAME                                STATUS   AGE
   capd-system                         Active   21m
   capi-kubeadm-bootstrap-system       Active   21m
   capi-kubeadm-control-plane-system   Active   21m
   capi-system                         Active   21m
   capi-webhook-system                 Active   21m
   cert-manager                        Active   22m
   default                             Active   23m
   eksa-system                         Active   20m
   kube-node-lease                     Active   23m
   kube-public                         Active   23m
   kube-system                         Active   23m
   

   You can now use the cluster like you would any Kubernetes cluster. Deploy the test application with:

   kubectl apply -f "https://anywhere.eks.amazonaws.com/manifests/hello-eks-a.yaml"
   

   Verify the test application in the deploy test application section . See the Cluster management section with more information on common operational tasks like scaling and deleting the cluster.

3 - Create production cluster

EKS Anywhere supports a vSphere provider for production grade EKS Anywhere deployments. EKS Anywhere allows you to provision and manage Amazon EKS on your own infrastructure.

This document walks you through setting up EKS Anywhere in a way that:

• Deploys an initial cluster on your vSphere environment. That cluster can be used as a self-managed cluster (to run workloads) or a management cluster (to create and manage other clusters)
• Deploys zero or more workload clusters from the management cluster

If your initial cluster is a management cluster, it is intended to stay in place so you can use it later to modify, upgrade, and delete workload clusters. Using a management cluster makes it faster to provision and delete workload clusters. Also it lets you keep vSphere credentials for a set of clusters in one place: on the management cluster. The alternative is to simply use your initial cluster to run workloads.

Prerequisite Checklist

EKS Anywhere needs to be run on an administrative machine that has certain machine requirements . An EKS Anywhere deployment will also require the availability of certain resources from your VMware vSphere deployment .

Steps

The following steps are divided into two sections:

• Create an initial cluster (used as a management or self-managed cluster)
• Create zero or more workload clusters from the management cluster

Create an initial cluster

Follow these steps to create an EKS Anywhere cluster that can be used either as a management cluster or as a self-managed cluster (for running workloads itself).

1. Generate an initial cluster config (named mgmt for this example):

   CLUSTER_NAME=mgmt
   eksctl anywhere generate clusterconfig $CLUSTER_NAME \
      --provider vsphere > eksa-mgmt-cluster.yaml
   

2. Modify the initial cluster config (eksa-mgmt-cluster.yaml) as follows:

   • Refer to vsphere configuration for information on configuring this cluster config for a vSphere provider.
   • Create at least two control plane nodes, three worker nodes, and three etcd nodes for a production cluster, to provide high availability and rolling upgrades.
   • Optionally, configure the cluster for OIDC , etcd , proxy , gitops and/or a container registry mirror .

3. Set Credential Environment Variables

   Before you create the initial cluster, you will need to set and export these environment variables for your vSphere user name and password. Make sure you use single quotes around the values so that your shell does not interpret the values:

   export EKSA_VSPHERE_USERNAME='billy'
   export EKSA_VSPHERE_PASSWORD='t0p$ecret'
   

4. Set License Environment Variable

   If you are creating a licensed cluster, set and export the license variable (see License cluster if you are licensing an existing cluster):

   export EKSA_LICENSE='my-license-here'
   

5. Create the initial cluster

   After you have created your eksa-mgmt-cluster.yaml and set your credential environment variables, you will be ready to create the cluster:

   eksctl anywhere create cluster -f eksa-mgmt-cluster.yaml
   

6. Once the cluster is created you can use it with the generated KUBECONFIG file in your local directory:

   export KUBECONFIG=${PWD}/${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig
   

7. Check the cluster nodes:

   To check that the cluster completed, list the machines to see the control plane, etcd, and worker nodes:

   kubectl get machines -A
   

   Example command output

   NAMESPACE   NAME                PROVIDERID        PHASE    VERSION
   eksa-system mgmt-b2xyz          vsphere:/xxxxx    Running  v1.21.2-eks-1-21-5
   eksa-system mgmt-etcd-r9b42     vsphere:/xxxxx    Running  
   eksa-system mgmt-md-8-6xr-rnr   vsphere:/xxxxx    Running  v1.21.2-eks-1-21-5
   ...
   

   The etcd machine doesn’t show the Kubernetes version because it doesn’t run the kubelet service.

8. Check the initial cluster’s CRD:

   To ensure you are looking at the initial cluster, list the CRD to see that the name of its management cluster is itself:

   kubectl get clusters mgmt -o yaml
   

   Example command output

   ...
   kubernetesVersion: "1.21"
   managementCluster:
     name: mgmt
   workerNodeGroupConfigurations:
   ...
   

   Note

   The initial cluster is now ready to deploy workload clusters. However, if you just want to use it to run workloads, you can deploy pod workloads directly on the initial cluster without deploying a separate workload cluster and skip the section on running separate workload clusters.

Create separate workload clusters

Follow these steps if you want to use your initial cluster to create and manage separate workload clusters.

1. Generate a workload cluster config:

   CLUSTER_NAME=w01
   eksctl anywhere generate clusterconfig $CLUSTER_NAME \
      --provider vsphere > eksa-w01-cluster.yaml
   

   Refer to the initial config described earlier for the required and optional settings. The main differences are that you must have a new cluster name and cannot use the same vSphere resources.

2. Create a workload cluster

   To create a new workload cluster from your management cluster run this command, identifying:

   • The workload cluster yaml file
   • The initial cluster’s credentials (this causes the workload cluster to be managed from the management cluster)

   eksctl anywhere create cluster \
       -f eksa-w01-cluster.yaml  \
       --kubeconfig mgmt/mgmt-eks-a-cluster.kubeconfig
   

   As noted earlier, adding the --kubeconfig option tells eksctl to use the management cluster identified by that kubeconfig file to create a different workload cluster.

3. Check the workload cluster:

   You can now use the workload cluster as you would any Kubernetes cluster. Change your credentials to point to the new workload cluster (for example, mgmt-w01), then run the test application with:

   export CLUSTER_NAME=mgmt-w01
   export KUBECONFIG=${PWD}/${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig
   kubectl apply -f "https://anywhere.eks.amazonaws.com/manifests/hello-eks-a.yaml"
   

   Verify the test application in the deploy test application section .

4. Add more workload clusters:

   To add more workload clusters, go through the same steps for creating the initial workload, copying the config file to a new name (such as eksa-w02-cluster.yaml), modifying resource names, and running the create cluster command again.

See the Cluster management section with more information on common operational tasks like scaling and deleting the cluster.