Skip to content

Commit 67c635c

Browse files
authored
Adding AWS Gateway API in Provider List along with example (#129)
Adjusting the space between table and Notes Signed-off-by: nikitsrj <nikitsrj@gmail.com>
1 parent 7f6da34 commit 67c635c

14 files changed

+644
-0
lines changed

docs/provider-status.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ For convenience we are including here a list of those actually tested with the p
2323
| [NGINX Gateway](https://github.com/nginxinc/nginx-gateway-fabric) | Unknown | 0.8.0 | 0.2.0 | [Example](https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/tree/main/examples/nginx) |
2424
| [Traefik](https://doc.traefik.io/traefik/providers/kubernetes-gateway/) | 3.1.3 | 1.1 | 0.4.0 | [Example](https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/tree/main/examples/traefik) |
2525
| [Linkerd](https://linkerd.io/) | 2.13.0 | Unknown | 0.2.0 | [Example](https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/tree/main/examples/linkerd) |
26+
| [AWS Gateway API Controller for Amazon VPC Lattice](https://www.gateway-api-controller.eks.aws.dev/latest//) | 1.1.2 | 1.3.0 | 0.6.0 | [Example](https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/tree/main/examples/aws-gateway-api-controller-lattice ) |
2627

2728
Note that these examples are included just for completeness. You should be able
2829
to use any solution that implements the Gateway API.
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
---
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
kind: ClusterRole
4+
metadata:
5+
name: gateway-controller-role
6+
namespace: argo-rollouts
7+
rules:
8+
- apiGroups:
9+
- "*"
10+
resources:
11+
- "*"
12+
verbs:
13+
- "*"
14+
---
15+
apiVersion: rbac.authorization.k8s.io/v1
16+
kind: ClusterRoleBinding
17+
metadata:
18+
name: gateway-admin
19+
roleRef:
20+
apiGroup: rbac.authorization.k8s.io
21+
kind: ClusterRole
22+
name: gateway-controller-role
23+
subjects:
24+
- namespace: argo-rollouts
25+
kind: ServiceAccount
26+
name: argo-rollouts
Lines changed: 322 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,322 @@
1+
# Argo Rollout with AWS Gateway API Controller for Amazon VPC Lattice
2+
3+
This guide walks through the process of setting up Amazon VPC Lattice with Gateway API Controller in a Kubernetes environment running on AWS, along with Argo Rollouts for advanced deployment strategies. The implementation is based on a polyglot microservice application, demonstrating service-to-service connectivity, traffic management, and canary deployments.
4+
5+
## Table of Contents
6+
7+
- [What is Amazon VPC Lattice?](#what-is-amazon-vpc-lattice)
8+
- [What is AWS Gateway API Controller for Lattice?](#what-is-aws-gateway-api-controller-for-lattice)
9+
- [Architecture Overview](#architecture-overview)
10+
- [Prerequisites](#prerequisites)
11+
- [Setup Instructions](#setup-instructions)
12+
- [Create Cluster](#create-cluster)
13+
- [Prepare for Lattice Gateway API Controller](#prepare-for-lattice-gateway-api-controller)
14+
- [Set up environment variables and configure security groups](#set-up-environment-variables-and-configure-security-groups-so-that-they-allow-all-pods-communicating-with-vpc-lattice-to-allow-traffic-from-the-vpc-lattice-managed-prefix-lists)
15+
- [Setup IAM Permissions](#setup-iam-permissions)
16+
- [Deploy Gateway API CRDs](#deploy-gateway-api-crds)
17+
- [Install the Gateway API Controller using Helm](#install-the-gateway-api-controller-using-helm)
18+
- [Deploy Gateway Class](#deploy-gateway-class)
19+
- [Install Argo Rollout](#install-argo-rollout)
20+
- [Install Gateway API Plugin in ArgoRollout](#install-gateway-api-plugin-in-argorollout)
21+
- [Create Gateway](#create-gateway)
22+
- [Deploy Applications](#deploy-applications)
23+
- [References](#references)
24+
25+
26+
## What is Amazon VPC Lattice?
27+
28+
Amazon VPC Lattice is an application networking service that connects, secures, and monitors services across multiple AWS accounts and VPCs. It simplifies service-to-service connectivity without complex networking configurations, automatically scales to handle varying traffic loads, and provides consistent security controls across all service communications.
29+
30+
## What is AWS Gateway API Controller for Lattice?
31+
32+
AWS Gateway API Controller for Lattice is an implementation of the Kubernetes Gateway API that integrates with Amazon VPC Lattice. It provides a Kubernetes-native way to manage service networking using standard Kubernetes APIs. The controller watches for Gateway API resources and automatically provisions corresponding VPC Lattice objects, eliminating the need to write custom code or manage sidecar proxies.
33+
34+
## Architecture Overview
35+
36+
The following diagram illustrates the architecture of our polyglot microservice application using AWS VPC Lattice with Gateway API Controller and Argo Rollouts:
37+
38+
![Architecture Diagram](lattice-argrollout.jpg)
39+
40+
This implementation demonstrates how to use AWS VPC Lattice and Gateway API Controller to connect and manage traffic between microservices written in different programming languages, while leveraging Argo Rollouts for advanced deployment strategies like canary deployments.
41+
42+
A key component of this architecture is the Argo Rollouts Gateway API plugin, which enables Argo Rollouts to interact directly with HTTProute resources. This integration allows for dynamic traffic shifting during canary deployments by modifying the HTTProute configurations. When a rollout is initiated, the plugin updates the weight distributions in the HTTProute file, gradually shifting traffic from the stable version to the canary version based on the defined rollout strategy. This provides fine-grained control over traffic management without requiring manual intervention or custom scripting.
43+
44+
## Prerequisites
45+
46+
Before starting the setup, ensure you have:
47+
48+
- AWS CLI configured with appropriate permissions
49+
- `kubectl` installed
50+
- `eksctl` installed
51+
- `helm` installed
52+
- `jq` installed
53+
54+
## Setup Instructions
55+
56+
### Create Cluster
57+
58+
Create an EKS cluster using eksctl:
59+
60+
```bash
61+
eksctl create cluster --name <clustername> --enable-auto-mode --region <regioncode>
62+
```
63+
64+
### Prepare for Lattice Gateway API Controller
65+
66+
#### Set up environment variables and configure security groups so that they allow all Pods communicating with VPC Lattice to allow traffic from the VPC Lattice managed prefix lists
67+
68+
```bash
69+
export AWS_REGION=<RegionCode>
70+
export CLUSTER_NAME=<ClusterName>
71+
export ACCOUNT_ID=<YourAWSAccountID>
72+
export EKSVPC=<VPCIdOfEKSCluster>
73+
export LATTICE_SVC_NET=<NameForLatticeService>
74+
75+
# EXAMPLE Variable
76+
# export AWS_REGION=ap-southeast-1
77+
# export CLUSTER_NAME=lattice-canary
78+
# export ACCOUNT_ID=<YourAWSAccountID>
79+
# export EKSVPC=<VPCIdOfEKSCluster>
80+
# export LATTICE_SVC_NET=latcansvcnet
81+
82+
CLUSTER_SG=$(aws eks describe-cluster --name $CLUSTER_NAME --output json| jq -r '.cluster.resourcesVpcConfig.clusterSecurityGroupId')
83+
PREFIX_LIST_ID=$(aws ec2 describe-managed-prefix-lists --query "PrefixLists[?PrefixListName=="\'com.amazonaws.$AWS_REGION.vpc-lattice\'"].PrefixListId" | jq -r '.[]')
84+
aws ec2 authorize-security-group-ingress --group-id $CLUSTER_SG --ip-permissions "PrefixListIds=[{PrefixListId=${PREFIX_LIST_ID}}],IpProtocol=-1"
85+
PREFIX_LIST_ID_IPV6=$(aws ec2 describe-managed-prefix-lists --query "PrefixLists[?PrefixListName=="\'com.amazonaws.$AWS_REGION.ipv6.vpc-lattice\'"].PrefixListId" | jq -r '.[]')
86+
aws ec2 authorize-security-group-ingress --group-id $CLUSTER_SG --ip-permissions "PrefixListIds=[{PrefixListId=${PREFIX_LIST_ID_IPV6}}],IpProtocol=-1"
87+
```
88+
89+
#### Setup IAM Permissions:
90+
91+
```bash
92+
curl https://raw.githubusercontent.com/aws/aws-application-networking-k8s/main/files/controller-installation/recommended-inline-policy.json -o recommended-inline-policy.json
93+
94+
aws iam create-policy \
95+
--policy-name VPCLatticeControllerIAMPolicy \
96+
--policy-document file://recommended-inline-policy.json
97+
98+
export VPCLatticeControllerIAMPolicyArn=$(aws iam list-policies --query 'Policies[?PolicyName==`VPCLatticeControllerIAMPolicy`].Arn' --output text)
99+
```
100+
101+
#### Deploy Gateway API CRDs
102+
103+
Download and apply the Gateway API Custom Resource Definitions:
104+
105+
```bash
106+
wget https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml
107+
108+
kubectl create -f standard-install.yaml
109+
```
110+
111+
Download and apply the namespace configuration:
112+
113+
```bash
114+
wget https://raw.githubusercontent.com/aws/aws-application-networking-k8s/main/files/controller-installation/deploy-namesystem.yaml
115+
116+
kubectl create -f deploy-namesystem.yaml
117+
```
118+
119+
Create a service account for the controller:
120+
121+
```bash
122+
cat >gateway-api-controller-service-account.yaml <<EOF
123+
apiVersion: v1
124+
kind: ServiceAccount
125+
metadata:
126+
name: gateway-api-controller
127+
namespace: aws-application-networking-system
128+
EOF
129+
kubectl apply -f gateway-api-controller-service-account.yaml
130+
```
131+
132+
Create the trust relationship for IAM role:
133+
134+
```bash
135+
cat >trust-relationship.json <<EOF
136+
{
137+
"Version": "2012-10-17",
138+
"Statement": [
139+
{
140+
"Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
141+
"Effect": "Allow",
142+
"Principal": {
143+
"Service": "pods.eks.amazonaws.com"
144+
},
145+
"Action": [
146+
"sts:AssumeRole",
147+
"sts:TagSession"
148+
]
149+
}
150+
]
151+
}
152+
EOF
153+
```
154+
155+
Create and configure the IAM role:
156+
157+
```bash
158+
aws iam create-role --role-name VPCLatticeControllerIAMRole --assume-role-policy-document file://trust-relationship.json --description "IAM Role for AWS Gateway API Controller for VPC Lattice"
159+
160+
aws iam attach-role-policy --role-name VPCLatticeControllerIAMRole --policy-arn=$VPCLatticeControllerIAMPolicyArn
161+
162+
export VPCLatticeControllerIAMRoleArn=$(aws iam list-roles --query 'Roles[?RoleName==`VPCLatticeControllerIAMRole`].Arn' --output text)
163+
164+
aws eks create-pod-identity-association --cluster-name $CLUSTER_NAME --role-arn $VPCLatticeControllerIAMRoleArn --namespace aws-application-networking-system --service-account gateway-api-controller
165+
```
166+
167+
### Install the Gateway API Controller using Helm:
168+
169+
```bash
170+
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
171+
172+
helm install gateway-api-controller \
173+
oci://public.ecr.aws/aws-application-networking-k8s/aws-gateway-controller-chart \
174+
--version=v1.1.2 \
175+
--set=serviceAccount.create=false --namespace aws-application-networking-system \
176+
--set=log.level=info \
177+
--set=awsRegion=$AWS_REGION \
178+
--set=clusterVpcId=$EKSVPC \
179+
--set=awsAccountId=$ACCOUNT \
180+
--set=clusterName=$CLUSTER_NAME \
181+
--set=defaultServiceNetwork=$LATTICE_SVC_NET
182+
```
183+
184+
### Deploy Gateway Class
185+
186+
Download and apply the Gateway Class configuration:
187+
188+
```bash
189+
wget https://raw.githubusercontent.com/aws/aws-application-networking-k8s/main/files/controller-installation/gatewayclass.yaml
190+
191+
kubectl create -f gatewayclass.yaml
192+
```
193+
194+
### Install Argo Rollout
195+
196+
You can use the instructions mentioned [here](https://argo-rollouts.readthedocs.io/en/stable/installation/#controller-installation)
197+
198+
Create a namespace and install Argo Rollouts:
199+
200+
```bash
201+
kubectl create namespace argo-rollouts
202+
203+
kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
204+
```
205+
206+
Patch the deployment to include AWS region:
207+
208+
```bash
209+
kubectl patch deployment argo-rollouts -n argo-rollouts \
210+
--type='json' \
211+
-p='[{"op": "add", "path": "/spec/template/spec/containers/0/env", "value": [{"name": "AWS_REGION", "value": "'$AWS_REGION'"}]}]'
212+
```
213+
214+
Install the Argo Rollouts CLI, as mentioned [here](https://argo-rollouts.readthedocs.io/en/stable/installation/#controller-installation)
215+
216+
```bash
217+
brew install argoproj/tap/kubectl-argo-rollouts
218+
```
219+
220+
### Install Gateway API Plugin in ArgoRollout
221+
222+
You can follow the instruction [here](https://rollouts-plugin-trafficrouter-gatewayapi.readthedocs.io/en/latest/installation/#installing-the-plugin-via-https)
223+
224+
Apply the configuration and restart the deployment:
225+
226+
```bash
227+
kubectl apply -f configmap-gatewayapicontrollerforargo.yaml
228+
# Give Access to Argo Rollouts for the Gateway/HTTP Route. Do Note thatthese permissions are not very strict. You should lock them down according to your needs.
229+
kubectl create -f ClusterRoleForArgoGatewayAPI.yaml
230+
# Restart the Argo Rollout Controller
231+
kubectl rollout restart deployment -n argo-rollouts argo-rollouts
232+
```
233+
234+
### Create Gateway
235+
236+
Create the Gateway and related resources for 'proddetail' service:
237+
238+
```bash
239+
kubectl create -f gateway.yaml
240+
kubectl create -f httproute.yaml
241+
kubectl create -f proddetails-lat-svc.yaml
242+
kubectl create -f proddetail-tgp.yaml
243+
244+
# Check VPC Lattice generated DNS Address for HTTPRoute 'latcan-app-default'. Lattice provide the name of lattice service as httproutename-namespace, hence latcan-app-default
245+
latcanFQDN=$(kubectl get httproute latcan-app-default -o json | jq -r '.metadata.annotations."application-networking.k8s.aws/lattice-assigned-domain-name"')
246+
echo $latcanFQDN
247+
```
248+
249+
### Deploy Applications
250+
251+
Deploy the frontend, product catalog, and proddetail rollout:
252+
253+
```bash
254+
# Note: Take the domain name of latcan-app-default, as in output of $latcanFQDN and put into the variable of prodcatalogdep.yaml file AGG_APP_URL value
255+
kubectl create -f frontenddep.yaml
256+
kubectl create -f frontendsvc.yaml
257+
kubectl create -f prodcatalogdep.yaml
258+
kubectl create -f prodcatalogsvc.yaml
259+
kubectl create -f proddetail-rollout.yaml
260+
```
261+
Check the Argo Rollout Status
262+
```bash
263+
kubectl argo rollouts get rollout proddetail -n workshop
264+
265+
Name: proddetail
266+
Namespace: workshop
267+
Status: ✔ Healthy
268+
Strategy: Canary
269+
Step: 9/9
270+
SetWeight: 100
271+
ActualWeight: 100
272+
Images: nicksrj/product_detail:1.0 (stable)
273+
Replicas:
274+
Desired: 1
275+
Current: 1
276+
Updated: 1
277+
Ready: 1
278+
Available: 1
279+
280+
NAME KIND STATUS AGE INFO
281+
⟳ proddetail Rollout ✔ Healthy 19s
282+
└──# revision:1
283+
└──⧉ proddetail-75b89f79bd ReplicaSet ✔ Healthy 19s stable
284+
└──□ proddetail-75b89f79bd-hk98b Pod ✔ Running 19s ready:1/1
285+
286+
```
287+
288+
If you hit the ELB DNS created using frontend_node svc, you will get the image which shows "Vendors : ABC.com"
289+
290+
Update the image for the rollout:
291+
292+
```bash
293+
kubectl argo rollouts set image proddetail "*=975050187486.dkr.ecr.ap-southeast-1.amazonaws.com/niksrj/proddetail" -n workshop
294+
295+
kubectl argo rollouts get rollout proddetail -n workshop
296+
Name: proddetail
297+
Namespace: workshop
298+
Status: ॥ Paused
299+
Message: CanaryPauseStep
300+
Strategy: Canary
301+
Step: 1/9
302+
SetWeight: 20
303+
ActualWeight: 20
304+
Images: nicksrj/product_detail:1.0 (stable)
305+
nicksrj/product_detail:2.0 (canary)
306+
Replicas:
307+
Desired: 1
308+
Current: 2
309+
Updated: 1
310+
Ready: 2
311+
Available: 2
312+
313+
NAME KIND STATUS AGE INFO
314+
⟳ proddetail Rollout ॥ Paused 5m54s
315+
├──# revision:2
316+
│ └──⧉ proddetail-5d66bbc795 ReplicaSet ✔ Healthy 73s canary
317+
│ └──□ proddetail-5d66bbc795-t9gsx Pod ✔ Running 73s ready:1/1
318+
└──# revision:1
319+
└──⧉ proddetail-75b89f79bd ReplicaSet ✔ Healthy 5m54s stable
320+
└──□ proddetail-75b89f79bd-hk98b Pod ✔ Running 5m54s ready:1/1
321+
```
322+
Now if you hit the same ELB DNS , you will get the image which shows "Vendors : ABC.com,XYZ.com". So basically XYZ.com is coming from the 2.0 image of product_detail.
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: argo-rollouts-config # must be named like this
5+
namespace: argo-rollouts # must be in this namespace
6+
data:
7+
trafficRouterPlugins: |-
8+
- name: "argoproj-labs/gatewayAPI"
9+
location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.6.0/gatewayapi-plugin-linux-amd64"
10+

0 commit comments

Comments
 (0)