|
| 1 | +# Argo Rollout with AWS Gateway API Controller for Amazon VPC Lattice |
| 2 | + |
| 3 | +This guide walks through the process of setting up Amazon VPC Lattice with Gateway API Controller in a Kubernetes environment running on AWS, along with Argo Rollouts for advanced deployment strategies. The implementation is based on a polyglot microservice application, demonstrating service-to-service connectivity, traffic management, and canary deployments. |
| 4 | + |
| 5 | +## Table of Contents |
| 6 | + |
| 7 | +- [What is Amazon VPC Lattice?](#what-is-amazon-vpc-lattice) |
| 8 | +- [What is AWS Gateway API Controller for Lattice?](#what-is-aws-gateway-api-controller-for-lattice) |
| 9 | +- [Architecture Overview](#architecture-overview) |
| 10 | +- [Prerequisites](#prerequisites) |
| 11 | +- [Setup Instructions](#setup-instructions) |
| 12 | + - [Create Cluster](#create-cluster) |
| 13 | + - [Prepare for Lattice Gateway API Controller](#prepare-for-lattice-gateway-api-controller) |
| 14 | + - [Set up environment variables and configure security groups](#set-up-environment-variables-and-configure-security-groups-so-that-they-allow-all-pods-communicating-with-vpc-lattice-to-allow-traffic-from-the-vpc-lattice-managed-prefix-lists) |
| 15 | + - [Setup IAM Permissions](#setup-iam-permissions) |
| 16 | + - [Deploy Gateway API CRDs](#deploy-gateway-api-crds) |
| 17 | + - [Install the Gateway API Controller using Helm](#install-the-gateway-api-controller-using-helm) |
| 18 | + - [Deploy Gateway Class](#deploy-gateway-class) |
| 19 | + - [Install Argo Rollout](#install-argo-rollout) |
| 20 | + - [Install Gateway API Plugin in ArgoRollout](#install-gateway-api-plugin-in-argorollout) |
| 21 | + - [Create Gateway](#create-gateway) |
| 22 | + - [Deploy Applications](#deploy-applications) |
| 23 | +- [References](#references) |
| 24 | + |
| 25 | + |
| 26 | +## What is Amazon VPC Lattice? |
| 27 | + |
| 28 | +Amazon VPC Lattice is an application networking service that connects, secures, and monitors services across multiple AWS accounts and VPCs. It simplifies service-to-service connectivity without complex networking configurations, automatically scales to handle varying traffic loads, and provides consistent security controls across all service communications. |
| 29 | + |
| 30 | +## What is AWS Gateway API Controller for Lattice? |
| 31 | + |
| 32 | +AWS Gateway API Controller for Lattice is an implementation of the Kubernetes Gateway API that integrates with Amazon VPC Lattice. It provides a Kubernetes-native way to manage service networking using standard Kubernetes APIs. The controller watches for Gateway API resources and automatically provisions corresponding VPC Lattice objects, eliminating the need to write custom code or manage sidecar proxies. |
| 33 | + |
| 34 | +## Architecture Overview |
| 35 | + |
| 36 | +The following diagram illustrates the architecture of our polyglot microservice application using AWS VPC Lattice with Gateway API Controller and Argo Rollouts: |
| 37 | + |
| 38 | + |
| 39 | + |
| 40 | +This implementation demonstrates how to use AWS VPC Lattice and Gateway API Controller to connect and manage traffic between microservices written in different programming languages, while leveraging Argo Rollouts for advanced deployment strategies like canary deployments. |
| 41 | + |
| 42 | +A key component of this architecture is the Argo Rollouts Gateway API plugin, which enables Argo Rollouts to interact directly with HTTProute resources. This integration allows for dynamic traffic shifting during canary deployments by modifying the HTTProute configurations. When a rollout is initiated, the plugin updates the weight distributions in the HTTProute file, gradually shifting traffic from the stable version to the canary version based on the defined rollout strategy. This provides fine-grained control over traffic management without requiring manual intervention or custom scripting. |
| 43 | + |
| 44 | +## Prerequisites |
| 45 | + |
| 46 | +Before starting the setup, ensure you have: |
| 47 | + |
| 48 | +- AWS CLI configured with appropriate permissions |
| 49 | +- `kubectl` installed |
| 50 | +- `eksctl` installed |
| 51 | +- `helm` installed |
| 52 | +- `jq` installed |
| 53 | + |
| 54 | +## Setup Instructions |
| 55 | + |
| 56 | +### Create Cluster |
| 57 | + |
| 58 | +Create an EKS cluster using eksctl: |
| 59 | + |
| 60 | +```bash |
| 61 | +eksctl create cluster --name <clustername> --enable-auto-mode --region <regioncode> |
| 62 | +``` |
| 63 | + |
| 64 | +### Prepare for Lattice Gateway API Controller |
| 65 | + |
| 66 | +#### Set up environment variables and configure security groups so that they allow all Pods communicating with VPC Lattice to allow traffic from the VPC Lattice managed prefix lists |
| 67 | + |
| 68 | +```bash |
| 69 | +export AWS_REGION=<RegionCode> |
| 70 | +export CLUSTER_NAME=<ClusterName> |
| 71 | +export ACCOUNT_ID=<YourAWSAccountID> |
| 72 | +export EKSVPC=<VPCIdOfEKSCluster> |
| 73 | +export LATTICE_SVC_NET=<NameForLatticeService> |
| 74 | + |
| 75 | +# EXAMPLE Variable |
| 76 | +# export AWS_REGION=ap-southeast-1 |
| 77 | +# export CLUSTER_NAME=lattice-canary |
| 78 | +# export ACCOUNT_ID=<YourAWSAccountID> |
| 79 | +# export EKSVPC=<VPCIdOfEKSCluster> |
| 80 | +# export LATTICE_SVC_NET=latcansvcnet |
| 81 | + |
| 82 | +CLUSTER_SG=$(aws eks describe-cluster --name $CLUSTER_NAME --output json| jq -r '.cluster.resourcesVpcConfig.clusterSecurityGroupId') |
| 83 | +PREFIX_LIST_ID=$(aws ec2 describe-managed-prefix-lists --query "PrefixLists[?PrefixListName=="\'com.amazonaws.$AWS_REGION.vpc-lattice\'"].PrefixListId" | jq -r '.[]') |
| 84 | +aws ec2 authorize-security-group-ingress --group-id $CLUSTER_SG --ip-permissions "PrefixListIds=[{PrefixListId=${PREFIX_LIST_ID}}],IpProtocol=-1" |
| 85 | +PREFIX_LIST_ID_IPV6=$(aws ec2 describe-managed-prefix-lists --query "PrefixLists[?PrefixListName=="\'com.amazonaws.$AWS_REGION.ipv6.vpc-lattice\'"].PrefixListId" | jq -r '.[]') |
| 86 | +aws ec2 authorize-security-group-ingress --group-id $CLUSTER_SG --ip-permissions "PrefixListIds=[{PrefixListId=${PREFIX_LIST_ID_IPV6}}],IpProtocol=-1" |
| 87 | +``` |
| 88 | + |
| 89 | +#### Setup IAM Permissions: |
| 90 | + |
| 91 | +```bash |
| 92 | +curl https://raw.githubusercontent.com/aws/aws-application-networking-k8s/main/files/controller-installation/recommended-inline-policy.json -o recommended-inline-policy.json |
| 93 | + |
| 94 | +aws iam create-policy \ |
| 95 | + --policy-name VPCLatticeControllerIAMPolicy \ |
| 96 | + --policy-document file://recommended-inline-policy.json |
| 97 | + |
| 98 | +export VPCLatticeControllerIAMPolicyArn=$(aws iam list-policies --query 'Policies[?PolicyName==`VPCLatticeControllerIAMPolicy`].Arn' --output text) |
| 99 | +``` |
| 100 | + |
| 101 | +#### Deploy Gateway API CRDs |
| 102 | + |
| 103 | +Download and apply the Gateway API Custom Resource Definitions: |
| 104 | + |
| 105 | +```bash |
| 106 | +wget https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml |
| 107 | + |
| 108 | +kubectl create -f standard-install.yaml |
| 109 | +``` |
| 110 | + |
| 111 | +Download and apply the namespace configuration: |
| 112 | + |
| 113 | +```bash |
| 114 | +wget https://raw.githubusercontent.com/aws/aws-application-networking-k8s/main/files/controller-installation/deploy-namesystem.yaml |
| 115 | + |
| 116 | +kubectl create -f deploy-namesystem.yaml |
| 117 | +``` |
| 118 | + |
| 119 | +Create a service account for the controller: |
| 120 | + |
| 121 | +```bash |
| 122 | +cat >gateway-api-controller-service-account.yaml <<EOF |
| 123 | +apiVersion: v1 |
| 124 | +kind: ServiceAccount |
| 125 | +metadata: |
| 126 | + name: gateway-api-controller |
| 127 | + namespace: aws-application-networking-system |
| 128 | +EOF |
| 129 | +kubectl apply -f gateway-api-controller-service-account.yaml |
| 130 | +``` |
| 131 | + |
| 132 | +Create the trust relationship for IAM role: |
| 133 | + |
| 134 | +```bash |
| 135 | +cat >trust-relationship.json <<EOF |
| 136 | +{ |
| 137 | + "Version": "2012-10-17", |
| 138 | + "Statement": [ |
| 139 | + { |
| 140 | + "Sid": "AllowEksAuthToAssumeRoleForPodIdentity", |
| 141 | + "Effect": "Allow", |
| 142 | + "Principal": { |
| 143 | + "Service": "pods.eks.amazonaws.com" |
| 144 | + }, |
| 145 | + "Action": [ |
| 146 | + "sts:AssumeRole", |
| 147 | + "sts:TagSession" |
| 148 | + ] |
| 149 | + } |
| 150 | + ] |
| 151 | +} |
| 152 | +EOF |
| 153 | +``` |
| 154 | + |
| 155 | +Create and configure the IAM role: |
| 156 | + |
| 157 | +```bash |
| 158 | +aws iam create-role --role-name VPCLatticeControllerIAMRole --assume-role-policy-document file://trust-relationship.json --description "IAM Role for AWS Gateway API Controller for VPC Lattice" |
| 159 | + |
| 160 | +aws iam attach-role-policy --role-name VPCLatticeControllerIAMRole --policy-arn=$VPCLatticeControllerIAMPolicyArn |
| 161 | + |
| 162 | +export VPCLatticeControllerIAMRoleArn=$(aws iam list-roles --query 'Roles[?RoleName==`VPCLatticeControllerIAMRole`].Arn' --output text) |
| 163 | + |
| 164 | +aws eks create-pod-identity-association --cluster-name $CLUSTER_NAME --role-arn $VPCLatticeControllerIAMRoleArn --namespace aws-application-networking-system --service-account gateway-api-controller |
| 165 | +``` |
| 166 | + |
| 167 | +### Install the Gateway API Controller using Helm: |
| 168 | + |
| 169 | +```bash |
| 170 | +aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws |
| 171 | + |
| 172 | +helm install gateway-api-controller \ |
| 173 | + oci://public.ecr.aws/aws-application-networking-k8s/aws-gateway-controller-chart \ |
| 174 | + --version=v1.1.2 \ |
| 175 | + --set=serviceAccount.create=false --namespace aws-application-networking-system \ |
| 176 | + --set=log.level=info \ |
| 177 | + --set=awsRegion=$AWS_REGION \ |
| 178 | + --set=clusterVpcId=$EKSVPC \ |
| 179 | + --set=awsAccountId=$ACCOUNT \ |
| 180 | + --set=clusterName=$CLUSTER_NAME \ |
| 181 | + --set=defaultServiceNetwork=$LATTICE_SVC_NET |
| 182 | +``` |
| 183 | + |
| 184 | +### Deploy Gateway Class |
| 185 | + |
| 186 | +Download and apply the Gateway Class configuration: |
| 187 | + |
| 188 | +```bash |
| 189 | +wget https://raw.githubusercontent.com/aws/aws-application-networking-k8s/main/files/controller-installation/gatewayclass.yaml |
| 190 | + |
| 191 | +kubectl create -f gatewayclass.yaml |
| 192 | +``` |
| 193 | + |
| 194 | +### Install Argo Rollout |
| 195 | + |
| 196 | +You can use the instructions mentioned [here](https://argo-rollouts.readthedocs.io/en/stable/installation/#controller-installation) |
| 197 | + |
| 198 | +Create a namespace and install Argo Rollouts: |
| 199 | + |
| 200 | +```bash |
| 201 | +kubectl create namespace argo-rollouts |
| 202 | + |
| 203 | +kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml |
| 204 | +``` |
| 205 | + |
| 206 | +Patch the deployment to include AWS region: |
| 207 | + |
| 208 | +```bash |
| 209 | +kubectl patch deployment argo-rollouts -n argo-rollouts \ |
| 210 | + --type='json' \ |
| 211 | + -p='[{"op": "add", "path": "/spec/template/spec/containers/0/env", "value": [{"name": "AWS_REGION", "value": "'$AWS_REGION'"}]}]' |
| 212 | +``` |
| 213 | + |
| 214 | +Install the Argo Rollouts CLI, as mentioned [here](https://argo-rollouts.readthedocs.io/en/stable/installation/#controller-installation) |
| 215 | + |
| 216 | +```bash |
| 217 | +brew install argoproj/tap/kubectl-argo-rollouts |
| 218 | +``` |
| 219 | + |
| 220 | +### Install Gateway API Plugin in ArgoRollout |
| 221 | + |
| 222 | +You can follow the instruction [here](https://rollouts-plugin-trafficrouter-gatewayapi.readthedocs.io/en/latest/installation/#installing-the-plugin-via-https) |
| 223 | + |
| 224 | +Apply the configuration and restart the deployment: |
| 225 | + |
| 226 | +```bash |
| 227 | +kubectl apply -f configmap-gatewayapicontrollerforargo.yaml |
| 228 | +# Give Access to Argo Rollouts for the Gateway/HTTP Route. Do Note thatthese permissions are not very strict. You should lock them down according to your needs. |
| 229 | +kubectl create -f ClusterRoleForArgoGatewayAPI.yaml |
| 230 | +# Restart the Argo Rollout Controller |
| 231 | +kubectl rollout restart deployment -n argo-rollouts argo-rollouts |
| 232 | +``` |
| 233 | + |
| 234 | +### Create Gateway |
| 235 | + |
| 236 | +Create the Gateway and related resources for 'proddetail' service: |
| 237 | + |
| 238 | +```bash |
| 239 | +kubectl create -f gateway.yaml |
| 240 | +kubectl create -f httproute.yaml |
| 241 | +kubectl create -f proddetails-lat-svc.yaml |
| 242 | +kubectl create -f proddetail-tgp.yaml |
| 243 | + |
| 244 | +# Check VPC Lattice generated DNS Address for HTTPRoute 'latcan-app-default'. Lattice provide the name of lattice service as httproutename-namespace, hence latcan-app-default |
| 245 | +latcanFQDN=$(kubectl get httproute latcan-app-default -o json | jq -r '.metadata.annotations."application-networking.k8s.aws/lattice-assigned-domain-name"') |
| 246 | +echo $latcanFQDN |
| 247 | +``` |
| 248 | + |
| 249 | +### Deploy Applications |
| 250 | + |
| 251 | +Deploy the frontend, product catalog, and proddetail rollout: |
| 252 | + |
| 253 | +```bash |
| 254 | +# Note: Take the domain name of latcan-app-default, as in output of $latcanFQDN and put into the variable of prodcatalogdep.yaml file AGG_APP_URL value |
| 255 | +kubectl create -f frontenddep.yaml |
| 256 | +kubectl create -f frontendsvc.yaml |
| 257 | +kubectl create -f prodcatalogdep.yaml |
| 258 | +kubectl create -f prodcatalogsvc.yaml |
| 259 | +kubectl create -f proddetail-rollout.yaml |
| 260 | +``` |
| 261 | +Check the Argo Rollout Status |
| 262 | +```bash |
| 263 | +kubectl argo rollouts get rollout proddetail -n workshop |
| 264 | + |
| 265 | +Name: proddetail |
| 266 | +Namespace: workshop |
| 267 | +Status: ✔ Healthy |
| 268 | +Strategy: Canary |
| 269 | + Step: 9/9 |
| 270 | + SetWeight: 100 |
| 271 | + ActualWeight: 100 |
| 272 | +Images: nicksrj/product_detail:1.0 (stable) |
| 273 | +Replicas: |
| 274 | + Desired: 1 |
| 275 | + Current: 1 |
| 276 | + Updated: 1 |
| 277 | + Ready: 1 |
| 278 | + Available: 1 |
| 279 | + |
| 280 | +NAME KIND STATUS AGE INFO |
| 281 | +⟳ proddetail Rollout ✔ Healthy 19s |
| 282 | +└──# revision:1 |
| 283 | + └──⧉ proddetail-75b89f79bd ReplicaSet ✔ Healthy 19s stable |
| 284 | + └──□ proddetail-75b89f79bd-hk98b Pod ✔ Running 19s ready:1/1 |
| 285 | + |
| 286 | +``` |
| 287 | + |
| 288 | +If you hit the ELB DNS created using frontend_node svc, you will get the image which shows "Vendors : ABC.com" |
| 289 | + |
| 290 | +Update the image for the rollout: |
| 291 | + |
| 292 | +```bash |
| 293 | +kubectl argo rollouts set image proddetail "*=975050187486.dkr.ecr.ap-southeast-1.amazonaws.com/niksrj/proddetail" -n workshop |
| 294 | + |
| 295 | +kubectl argo rollouts get rollout proddetail -n workshop |
| 296 | +Name: proddetail |
| 297 | +Namespace: workshop |
| 298 | +Status: ॥ Paused |
| 299 | +Message: CanaryPauseStep |
| 300 | +Strategy: Canary |
| 301 | + Step: 1/9 |
| 302 | + SetWeight: 20 |
| 303 | + ActualWeight: 20 |
| 304 | +Images: nicksrj/product_detail:1.0 (stable) |
| 305 | + nicksrj/product_detail:2.0 (canary) |
| 306 | +Replicas: |
| 307 | + Desired: 1 |
| 308 | + Current: 2 |
| 309 | + Updated: 1 |
| 310 | + Ready: 2 |
| 311 | + Available: 2 |
| 312 | + |
| 313 | +NAME KIND STATUS AGE INFO |
| 314 | +⟳ proddetail Rollout ॥ Paused 5m54s |
| 315 | +├──# revision:2 |
| 316 | +│ └──⧉ proddetail-5d66bbc795 ReplicaSet ✔ Healthy 73s canary |
| 317 | +│ └──□ proddetail-5d66bbc795-t9gsx Pod ✔ Running 73s ready:1/1 |
| 318 | +└──# revision:1 |
| 319 | + └──⧉ proddetail-75b89f79bd ReplicaSet ✔ Healthy 5m54s stable |
| 320 | + └──□ proddetail-75b89f79bd-hk98b Pod ✔ Running 5m54s ready:1/1 |
| 321 | +``` |
| 322 | +Now if you hit the same ELB DNS , you will get the image which shows "Vendors : ABC.com,XYZ.com". So basically XYZ.com is coming from the 2.0 image of product_detail. |
0 commit comments