Open in app

Sign in

Write

Sign in

AWS EKS Anywhere — Local Kubernetes On Premise

Sankar Panneerselvam
6 min readSep 9, 2021

Amazon EKS Anywhere is a new deployment option for Amazon EKS that enables you to easily create and operate Kubernetes clusters on-premises. EKS Anywhere provides an installable software package for creating and operating Kubernetes clusters on-premises and automation tooling for cluster life-cycle support. In this post I will walk you through the setups for setting up your own local EKS Cluster. EKS Anywhere is released on Sept 8th 2021 as Opensource. AWS offers a broad range of Kubernetes solutions are below is a quick summary of the K8 offerings and their capabilities.

Source: AWS

Prerequisites

  1. Morning cup of Coffee and a dose of Super excitement
  2. Docker Version 20 locally installed ( I have Docker desktop running on my Mac OS running Big Sur -Intel Chipset with 8 GB of RAM).
  3. Latest version of EKSCTL (version 0.66) or newer. If not installed locally run the following command on your Mac command line
curl "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" \
    --silent --location \
    | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin/

4. EKS Anywhere Plugin

export EKSA_RELEASE="0.5.0" OS="$(uname -s | tr A-Z a-z)"
curl "https://anywhere-assets.eks.amazonaws.com/releases/eks-a/1/artifacts/eks-a/v${EKSA_RELEASE}/${OS}/eksctl-anywhere-v${EKSA_RELEASE}-${OS}-amd64.tar.gz" \
    --silent --location \
    | tar xz ./eksctl-anywhere
sudo mv ./eksctl-anywhere /usr/local/bin/

Creating your first EKS Cluster

  1. Run the following command to generate your cluster YAML.
eksctl anywhere generate clusterconfig myfirstekscluster --provider docker > myfirstekscluster.yaml

2. Verify your EKS Cluster configuration YAML

apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
  name: myfirstekscluster
spec:
  clusterNetwork:
    cni: cilium
    pods:
      cidrBlocks:
      - 192.168.0.0/16
    services:
      cidrBlocks:
      - 10.96.0.0/12
  controlPlaneConfiguration:
    count: 1
  datacenterRef:
    kind: DockerDatacenterConfig
    name: myfirstekscluster
  externalEtcdConfiguration:
    count: 1
  kubernetesVersion: "1.21"
  workerNodeGroupConfigurations:
  - count: 1

---
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: DockerDatacenterConfig
metadata:
  name: myfirstekscluster
spec: {}

---

2. Create your Cluster by running eksctl create cluster command as below

eksctl anywhere create cluster -f myfirstekscluster.yaml

3. Verify your Cluster Creation. I have Docker Desktop on my local mac and here is a sample snapshot of the creation logs.

4. Cluster creation takes time and I noticed you need you to bump up the RAM requirements in Docker Desktop to 6GB to keep this process going smooth. This did bring my Macbook to its knees given the limited 8GB RAM but I was able to get past Memory allocation issues (after multiple retries) to create the EKS Anywhere Cluster

8 GB MacBook sucks big time with Docker Desktop!!
Snapshot of my RAM during EKS Anywhere cluster creation.

5. Finally light at the end of the tunnel

eksctl anywhere create cluster -f myfirstekscluster.yaml
Performing setup and validations
Warning: The docker infrastructure provider is meant for local development and testing only
✅ Docker Provider setup is valid
Creating new bootstrap cluster
Installing cluster-api providers on bootstrap cluster
Provider specific setup
Creating new workload cluster
Installing networking on workload cluster
Installing storage class on workload cluster
Installing cluster-api providers on workload cluster
Moving cluster management from bootstrap to workload cluster
Installing EKS-A custom components (CRD and controller) on workload cluster
Creating EKS-A CRDs instances on workload cluster
Installing AddonManager and GitOps Toolkit on workload cluster
GitOps field not specified, bootstrap flux skipped
Writing cluster config file
Deleting bootstrap cluster
🎉 Cluster created!

6. Lets get our ducks in row. Exporting KUBECONFIG

export KUBECONFIG=myfirstekscluster-eks-a-cluster.kubeconfig

7. Check the namespaces

$ kubectl get ns
NAME                                STATUS   AGE
capd-system                         Active   8m3s
capi-kubeadm-bootstrap-system       Active   9m30s
capi-kubeadm-control-plane-system   Active   8m32s
capi-system                         Active   9m46s
capi-webhook-system                 Active   9m53s
cert-manager                        Active   12m
default                             Active   18m
eksa-system                         Active   5m54s
etcdadm-bootstrap-provider-system   Active   9m15s
etcdadm-controller-system           Active   8m57s
kube-node-lease                     Active   18m
kube-public                         Active   18m
kube-system                         Active   18m

8. Hello World EKS Kubernetes Anywhere.

kubectl apply -f "https://anywhere.eks.amazonaws.com/manifests/hello-eks-a.yaml"
deployment.apps/hello-eks-a created
service/hello-eks-a created
kubectl logs -l app=hello-eks-a
2021/09/09 17:25:48 [notice] 1#1: using the "epoll" event method
2021/09/09 17:25:48 [notice] 1#1: nginx/1.21.1
2021/09/09 17:25:48 [notice] 1#1: built by gcc 10.3.1 20210424 (Alpine 10.3.1_git20210424)
2021/09/09 17:25:48 [notice] 1#1: OS: Linux 5.10.47-linuxkit
2021/09/09 17:25:48 [notice] 1#1: getrlimit(RLIMIT_NOFILE): 1048576:1048576
2021/09/09 17:25:48 [notice] 1#1: start worker processes
2021/09/09 17:25:48 [notice] 1#1: start worker process 37
2021/09/09 17:25:48 [notice] 1#1: start worker process 38
2021/09/09 17:25:48 [notice] 1#1: start worker process 39
2021/09/09 17:25:48 [notice] 1#1: start worker process 40
$ kubectl port-forward deploy/hello-eks-a 8000:80
Forwarding from 127.0.0.1:8000 -> 80
Forwarding from [::1]:8000 -> 80

9. Test your Hello World App running in EKS Anywhere.

curl localhost:8000
⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢

Thank you for using

███████╗██╗ ██╗███████╗
██╔════╝██║ ██╔╝██╔════╝
█████╗ █████╔╝ ███████╗
██╔══╝ ██╔═██╗ ╚════██║
███████╗██║ ██╗███████║
╚══════╝╚═╝ ╚═╝╚══════╝

█████╗ ███╗ ██╗██╗ ██╗██╗ ██╗██╗ ██╗███████╗██████╗ ███████╗
██╔══██╗████╗ ██║╚██╗ ██╔╝██║ ██║██║ ██║██╔════╝██╔══██╗██╔════╝
███████║██╔██╗ ██║ ╚████╔╝ ██║ █╗ ██║███████║█████╗ ██████╔╝█████╗
██╔══██║██║╚██╗██║ ╚██╔╝ ██║███╗██║██╔══██║██╔══╝ ██╔══██╗██╔══╝
██║ ██║██║ ╚████║ ██║ ╚███╔███╔╝██║ ██║███████╗██║ ██║███████╗
╚═╝ ╚═╝╚═╝ ╚═══╝ ╚═╝ ╚══╝╚══╝ ╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝╚══════╝

You have successfully deployed the hello-eks-a pod hello-eks-a-9644dd8dc-ff2g8

For more information check out
https://anywhere.eks.amazonaws.com

⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢⬡⬢

Roadblocks Encountered and Resolutions

Here are some of my lessons learned.

  1. cluster-api failures most likely due to network issues. I encountered Cluster creation issues outlined below while running the cluster creation with VPN ON.

Resolution — Disconnect from VPN and retry works fine.

Installing cluster-api providers on workload cluster
Error: failed to create cluster: error waiting for capi-kubeadm-bootstrap-controller-manager in namespace capi-kubeadm-bootstrap-system: error executing wait: error: the server doesn’t have a resource type “deployments”

2. Cluster deletion is bit wonky. Even with correct Cluster name I was forced to use -f option to provide config-file. Attached is a sample command for a dev-cluster.yaml that was created earlier. Looks like a bug to me since KUBECONFIG=${CLUSTER_NAME}/${CLUSTER_NAME}-eks-a-cluster.kubeconfig is explicitiy adding -eks-a-cluster for lookup.

Resolution — Manually enter the cluster YAML file

 eksctl anywhere delete cluster ${CLUSTER_NAME}
Error: clusterconfig file dev/dev-eks-a-cluster.yaml for cluster: dev not found, please provide the clusterconfig path manually using -f <config-file>
eksctl anywhere delete cluster -f dev-cluster.yaml

3. Delete EKS Anywhere cluster is not a straigtforward. I had to manually look at Docker logs to see what’s really happening which needs some fine tuning. I had to prune all images and restart which was bit of a pain in the neck.

eksctl anywhere delete cluster -f dev-cluster.yaml
Performing provider setup and validations
Creating management cluster
Installing cluster-api providers on management cluster
Moving cluster management from workload cluster
Deleting workload cluster
Error: failed to delete cluster: error deleting cluster dev-cluster apply: Error from server (NotFound): clusters.cluster.x-k8s.io "dev-cluster" not found

Conclusion

EKS Anywhere will be a game changer for Hybrid Cloud strategy. I am more excited to get started exploring Hybrid Cloud Telco and Media Industry usecases for EKS Anywhere — Enterprise IT, OSS/BSS, NWDAF, AI/ML and Core Networks will be transformed using EKS Anywhere. In my next post I will get into details of running Telco and Media Network workloads on premise closer (Near Edge) to solve Industry 4.0 Edge Computing use cases.

PS: The views and opinions expressed in this article are mine and do not necessarily reflect the official policy or position of my employer Amazon AWS.

Docker
AWS
Aws Eks
Kubernetes
Aws Fargate

--

--

Sankar Panneerselvam

Written by Sankar Panneerselvam

5 Followers

Amazonian, MBA, AI/ML, Blockchain, Web3, Cloud Evangelist and EDM Music Enthusiast — Nerd by profession, Hacker at Heart.

Help

Status

About

Careers

Blog

Privacy

Terms

Text to speech

Teams