From 21174153433a43d5ded4eb0adcd683bd1764b393 Mon Sep 17 00:00:00 2001 From: Gaurav Agerwala Date: Sun, 7 Dec 2025 09:13:59 -0800 Subject: [PATCH] Update design for PR #2775: feat(loadgenerator): allow customizing frontend protocol --- ...n-workflow-1-local-development-workflow.md | 4 +- ...sign-workflow-2-gke-deployment-workflow.md | 6 +- ...design-workflow-3-helm-chart-deployment.md | 11 +-- ...-kustomize-customization-and-deployment.md | 7 +- ...5-terraform-infrastructure-provisioning.md | 2 +- docs/pr-analysis-2775.md | 86 +++++++++++++++++++ 6 files changed, 105 insertions(+), 11 deletions(-) create mode 100644 docs/pr-analysis-2775.md diff --git a/.exp/design-workflow-1-local-development-workflow.md b/.exp/design-workflow-1-local-development-workflow.md index 2fa3ae092cc..7d3c4e3c97f 100644 --- a/.exp/design-workflow-1-local-development-workflow.md +++ b/.exp/design-workflow-1-local-development-workflow.md @@ -23,7 +23,7 @@ The workflow is initiated by running `skaffold dev` from the project root. Acces - **Dockerfiles**: Service-specific Dockerfiles in `src//Dockerfile` (e.g., multi-stage Go builds for Go services, Node.js for JS services). Skaffold builds these locally using Docker CLI and Buildkit for efficiency. -- **Kubernetes Manifests**: YAML files in `kubernetes-manifests/` defining Deployments, Services, Pods, etc., for all 11 microservices, Redis (cart store), and loadgenerator. Image references (e.g., `image: frontend`) are updated by Skaffold with generated tags. +- **Kubernetes Manifests**: YAML files in `kubernetes-manifests/` defining Deployments, Services, Pods, etc., for all 11 microservices, Redis (cart store), and loadgenerator. The loadgenerator now supports `FRONTEND_PROTO` (default: http) and `FRONTEND_ADDR` environment variables to allow customizing the protocol and address for load testing the frontend (e.g., https for secure setups), as updated in src/loadgenerator/Dockerfile by PR #2775. Image references (e.g., `image: frontend`) are updated by Skaffold with generated tags. - **Kustomize**: Used by Skaffold to build manifests, allowing bases and patches (e.g., via profiles adding components like network policies). @@ -47,9 +47,11 @@ sequenceDiagram S->>B: docker build -t : src/ B->>B: Build image from Dockerfile end + Note over B: Loadgenerator image now supports FRONTEND_PROTO env var for custom protocol (PR #2775) S->>K: kustomize build kubernetes-manifests/ | kubectl apply -f - K->>K: Create Deployments, Services, Pods Note over S,K: Pods pull local images and start + Note over K: Loadgenerator configurable via FRONTEND_PROTO, FRONTEND_ADDR env vars in manifests D->>K: kubectl port-forward svc/frontend 8080:8080 Note over D: Access app at http://localhost:8080 ``` diff --git a/.exp/design-workflow-2-gke-deployment-workflow.md b/.exp/design-workflow-2-gke-deployment-workflow.md index 6d8df079df8..2ec42f05380 100644 --- a/.exp/design-workflow-2-gke-deployment-workflow.md +++ b/.exp/design-workflow-2-gke-deployment-workflow.md @@ -37,17 +37,19 @@ sequenceDiagram participant K as Kustomize participant C as GKE Cluster (kubectl) U->>S: skaffold run --default-repo= - Note over S: Parse skaffold.yaml
Configs: app, loadgenerator + Note over S: Parse skaffold.yaml
Configs: app, loadgenerator (now supports protocol customization) loop For each service artifact S->>B: Build image from src//Dockerfile B->>B: Use local Docker or GCB profile B->>R: Tag & push /: end + Note over B: Loadgenerator image includes support for FRONTEND_PROTO env var (PR #2775) Note over S: Update image tags in manifests S->>K: kustomize build kubernetes-manifests/ K->>C: kubectl apply -f rendered.yaml C->>C: Deploy pods, services, etc. Note over C: Includes Redis if not customized + Note over C: Loadgenerator configurable for https via env vars (PR #2775) U->>C: kubectl get service frontend-external C->>U: External IP for access ``` @@ -85,7 +87,7 @@ sequenceDiagram - **Profiles in Skaffold**: Activate via `-p `, e.g., `-p gcb` for remote builds, `-p network-policies` to patch manifests with additional Kustomize paths. - **Kustomize Overlays**: Uncomment or add components in `kubernetes-manifests/kustomization.yaml` for features like service mesh (Istio), alternative databases (AlloyDB/Spanner), or observability (Cloud Operations). - **Image Tagging**: Git commit-based for traceability; customizable via `tagPolicy` in `skaffold.yaml`. -- **Load Generator**: Separate Skaffold config (`loadgenerator`) requiring `app`; deploys `loadgenerator.yaml` manifest. +- **Load Generator**: Separate Skaffold config (`loadgenerator`) requiring `app`; deploys `loadgenerator.yaml` manifest. Supports `FRONTEND_PROTO` (default: http) and `FRONTEND_ADDR` env vars to customize protocol for frontend load testing (e.g., https for ingress), updated in Dockerfile by PR #2775. - **Multi-Architecture**: Builds for amd64 and arm64, useful for GKE Autopilot or diverse node pools. ### Error Handling and Best Practices diff --git a/.exp/design-workflow-3-helm-chart-deployment.md b/.exp/design-workflow-3-helm-chart-deployment.md index 2ba4e5325e6..7dfad440408 100644 --- a/.exp/design-workflow-3-helm-chart-deployment.md +++ b/.exp/design-workflow-3-helm-chart-deployment.md @@ -32,7 +32,7 @@ The chart is marked as experimental in the README, encouraging feedback via GitH - **README.md**: Provides basic and advanced installation examples, links to blog posts on advanced scenarios (e.g., Spanner integration, gRPC health probes). ### Deployed Resources -- **Core**: Deployments and Services for 11 microservices (adService, cartService, etc.) + loadGenerator. +- **Core**: Deployments and Services for 11 microservices (adService, cartService, etc.) + loadGenerator (configurable via Helm values for FRONTEND_PROTO and FRONTEND_ADDR to support custom protocols like https, per PR #2775). - **Optional**: - In-cluster Redis StatefulSet if `cartDatabase.inClusterRedis.create: true`. - NetworkPolicies per service if enabled. @@ -55,14 +55,15 @@ sequenceDiagram U->>H: helm upgrade --install [options] [values overrides] H->>H: Load chart from OCI registry or local path H->>H: Merge default values.yaml with user overrides - H->>H: Render templates (e.g., service Deployments, conditional policies) + H->>H: Render templates (e.g., service Deployments, conditional policies incl. loadgenerator env) H->>S: Apply rendered YAMLs (e.g., Deployments, Services, Redis if enabled) S->>R: Create/Update Kubernetes objects S->>R: Schedule Pods, pull images, run init containers if needed - Note over R: Microservices start; gRPC health checks; inter-service communication begins - R->>S: Pods become ready; Services get endpoints + Note over R: Microservices start, gRPC health checks, inter-service communication begins, loadgenerator supports custom protocol (PR #2775) + R->>S: Pods become ready, Services get endpoints + Note over R: Loadgenerator configurable via FRONTEND_PROTO value S->>H: Confirmation of resource creation - H->>U: Helm release status (success/failure); NOTES for frontend access + H->>U: Helm release status (success/failure), NOTES for frontend access ``` ### Component Creation Flowchart diff --git a/.exp/design-workflow-4-kustomize-customization-and-deployment.md b/.exp/design-workflow-4-kustomize-customization-and-deployment.md index ee7831cb24b..38144bc8e01 100644 --- a/.exp/design-workflow-4-kustomize-customization-and-deployment.md +++ b/.exp/design-workflow-4-kustomize-customization-and-deployment.md @@ -18,7 +18,7 @@ The design supports extensibility: new components can be added following Kustomi ### Base Layer - **Location**: `kustomize/base/` - **Contents**: Individual YAML files for each microservice (e.g., `frontend.yaml`, `cartservice.yaml`) defining Deployments, Services, ConfigMaps, etc. The `base/kustomization.yaml` aggregates these as resources. -- **Role**: Provides the default, vanilla deployment of Online Boutique services, including in-cluster Redis for carts and public frontend exposure. +- **Role**: Provides the default, vanilla deployment of Online Boutique services, including in-cluster Redis for carts and public frontend exposure. The loadgenerator service now supports environment variables `FRONTEND_PROTO` (default: "http") and `FRONTEND_ADDR` to customize the protocol and address used for load testing the frontend, enabling scenarios like HTTPS ingress (via PR #2775 updating src/loadgenerator/Dockerfile). Components or direct patches can set these env vars in the loadgenerator Deployment. ### Top-Level Configuration - **File**: `kustomize/kustomization.yaml` @@ -105,15 +105,18 @@ flowchart TD Comp1["Example: Cymbal Branding Component
Patches frontend env vars"] Comp2["Example: Service Mesh Istio
Adds Istio resources, patches"] Comp3["Example: Database (e.g., Spanner)
Patches cartservice config"] + CompLG["Example: Loadgenerator Customization (new)
Patches loadgenerator Deployment with FRONTEND_PROTO and FRONTEND_ADDR env vars for custom protocol (e.g., https)"] Final["Final Customized Manifests
Ready for kubectl apply"] Base --> Comp1 Comp1 --> Comp2 Comp2 --> Comp3 - Comp3 --> Final + Comp3 --> CompLG + CompLG --> Final style Base fill:#f9f style Final fill:#bbf + style CompLG fill:#90ee90 ``` ## Additional Design Aspects diff --git a/.exp/design-workflow-5-terraform-infrastructure-provisioning.md b/.exp/design-workflow-5-terraform-infrastructure-provisioning.md index a26219102b6..4accf2b1177 100644 --- a/.exp/design-workflow-5-terraform-infrastructure-provisioning.md +++ b/.exp/design-workflow-5-terraform-infrastructure-provisioning.md @@ -44,7 +44,7 @@ This workflow is particularly useful for quickly spinning up a production-like e ### Application Deployment Components - **Kustomize (`kustomize/`)**: - `kustomization.yaml`: References base manifests and enabled components. - - `base/`: Kubernetes YAMLs for all 11 microservices + loadgenerator and in-cluster Redis. + - `base/`: Kubernetes YAMLs for all 11 microservices + loadgenerator (supports FRONTEND_PROTO and FRONTEND_ADDR env vars for custom frontend targeting, PR #2775) and in-cluster Redis. - `components/memorystore/`: Patches cartservice Deployment (sets REDIS_ADDR env), deletes in-cluster Redis resources. - **Local Executions**: - `gcloud` module: Fetches cluster credentials. diff --git a/docs/pr-analysis-2775.md b/docs/pr-analysis-2775.md new file mode 100644 index 00000000000..c6a4ae1d661 --- /dev/null +++ b/docs/pr-analysis-2775.md @@ -0,0 +1,86 @@ +# PR #2775: Workflow Design Impact Analysis + +## Affected Workflows +- **Local Development Workflow (1)**: Impacted because Skaffold builds the loadgenerator image from the updated `src/loadgenerator/Dockerfile` during `skaffold dev`, and deploys `loadgenerator.yaml`. The new env var support enables customized local load testing. Evidence: workflows.json relevant_files; PR changes Dockerfile used in build loop. +- **GKE Deployment Workflow (2)**: Skaffold `run` builds and pushes the updated image to registry for GKE deployment. New feature allows protocol customization in GKE setups with ingress. Evidence: similar to workflow 1; separate loadgenerator config in skaffold.yaml. +- **Helm Chart Deployment (3)**: Helm chart templates include `loadgenerator.yaml`, using the image with new ENTRYPOINT logic. Can be configured via values for env vars. Evidence: helm-chart/templates/loadgenerator.yaml exists; design doc updated to note. +- **Kustomize Customization and Deployment (4)**: Base includes `loadgenerator.yaml`; PR testing procedure explicitly uses Kustomize patch to set new env vars in loadgenerator Deployment. Enables new component-like customizations. Evidence: PR description; kustomize/base/loadgenerator.yaml. +- **Terraform Infrastructure Provisioning (5)**: Indirect impact as it provisions infra for subsequent app deployment using base manifests with loadgenerator. Design doc mentions it in base. Evidence: doc reference. +- **Cloud Build CI/CD Pipeline (6)**: Uses cloudbuild.yaml and skaffold to build images including loadgenerator, affected by Dockerfile change. No design doc yet. +- **Release Process (7)**: Builds tagged images and updates release manifests including loadgenerator. Dockerfile change affects released image. No design doc yet. +- **Adding New Microservice (8)**: Updates to skaffold, kustomize, helm would need to account for loadgenerator changes when integrating new services, but primarily affects existing loadgen. No design doc yet. + +## Local Development Workflow (1) Analysis +### Summary of design changes +The PR introduces support for customizing the frontend protocol in the loadgenerator by adding `${FRONTEND_PROTO:-http}` to the locust --host in Dockerfile ENTRYPOINT. This allows setting env var in deployment manifests or Skaffold profiles for flexible load testing (e.g., https for local ingress simulations). Implemented via shell variable in ENTRYPOINT. Benefits: Enhances local dev realism for secure setups; backwards compatible. Implications: Update manifests or use profiles to leverage; hot reload now rebuilds with new logic. + +The initial deployment sequence diagram was updated with new notes highlighting the change (additions in green via notes). + +```mermaid +sequenceDiagram + participant D as Developer (Old & New) + participant S as Skaffold + participant B as Docker Build + participant K as K8s Cluster + Note over D: PR #2775 Change + Note over B: Addition (green): Build loadgenerator with new ENTRYPOINT supporting FRONTEND_PROTO + loop For each artifact (incl. loadgen yellow: enhanced) + S->>B: docker build src/loadgenerator + B->>B: Use updated Dockerfile + end + S->>K: Deploy incl. loadgen Deployment (yellow: supports new env) + Note over K: New: Can set env for https protocol +``` + +## GKE Deployment Workflow (2) Analysis +### Summary of design changes +Similar to workflow 1, but for GKE with image push. New image pushed to registry includes the customization, allowing GKE deployments to configure loadgen for external HTTPS frontends. Implemented same way. Benefits: Supports production-like load testing in GKE. Updated diagram notes reflect build/push enhancement. + +```mermaid +sequenceDiagram + participant U as User + participant S as Skaffold + participant B as Builder + participant R as Registry + Note over B: Addition (green): Loadgen image with protocol var support + loop Build & Push + S->>B: Build from updated Dockerfile + B->>R: Push enhanced image + end + Note over R: New feature available in deployed images (yellow change) +``` + +## Helm Chart Deployment (3) Analysis +### Summary of design changes +Helm renders loadgenerator template using image with new feature. Design updated to note config via values. Allows setting env vars in values.yaml for protocol. Benefits: Parameterized deploys with load testing customization. No structural change to flow, but enhanced capability. + +```mermaid +sequenceDiagram + participant H as Helm + participant S as K8s + Note over H: Render templates (yellow): loadgenerator.yaml with optional new env vars + H->>S: Apply incl. loadgen Deployment + Note over S: Addition (green): Support for FRONTEND_PROTO in pod spec via values +``` + +## Kustomize Customization and Deployment (4) Analysis +### Summary of design changes +Enables new patches for loadgenerator Deployment to set FRONTEND_PROTO and FRONTEND_ADDR, as demoed in PR. Base role and composition flowchart updated to include example customization component/patch for protocol. How: Image ENTRYPOINT change allows env-driven host. Benefits: Better ingress/HTTPS support in customized deploys. Implications: Can create component for it. + +Updated flowchart shows addition of Loadgenerator Customization example in green. + +```mermaid +flowchart TD + Base["Base (unchanged)"] + Comps["Existing Components (unchanged)"] + NewComp["New Example: Loadgenerator Customization Component/Patch
green addition: Set env vars for protocol"] + Final["Final Manifests yellow: enhanced loadgen config"] + Base --> Comps --> NewComp --> Final + style NewComp fill:#90ee90 + style Final fill:#ffff00 +``` + +## Other Workflows +For workflows 5-8, no design documents were available for detailed analysis or updates, but they are impacted similarly through image builds or manifest updates including loadgenerator. Recommend creating design docs for comprehensive coverage. + +[PR #2775](https://github.com/GoogleCloudPlatform/microservices-demo/pull/2775) \ No newline at end of file