Skip to content

feat(lambdas): add batch SSM parameter fetching to reduce API calls#5017

Open
thomasnemer wants to merge 1 commit intogithub-aws-runners:mainfrom
thomasnemer:batch-ssm-parameter-get
Open

feat(lambdas): add batch SSM parameter fetching to reduce API calls#5017
thomasnemer wants to merge 1 commit intogithub-aws-runners:mainfrom
thomasnemer:batch-ssm-parameter-get

Conversation

@thomasnemer
Copy link

@thomasnemer thomasnemer commented Feb 4, 2026

This PR intends to reduce SSM AWS API calls by doing the following:

Add getParameters() function to aws-ssm-util that fetches multiple SSM parameters in a single API call with automatic chunking (max 10 per call per AWS API limits).

Apply batch fetching to:

  • auth.ts: fetch App ID and Private Key in one call (2 calls → 1)
  • ConfigLoader.ts: fetch multiple matcher config paths in one call
  • ami.ts: batch resolve SSM parameter values for AMI lookups

Also remove redundant appId SSM fetch in scale-up.ts that was only used for logging.

Add getParameters() function to aws-ssm-util that fetches multiple SSM
parameters in a single API call with automatic chunking (max 10 per call
per AWS API limits).

Apply batch fetching to:
- auth.ts: fetch App ID and Private Key in one call (2 calls → 1)
- ConfigLoader.ts: fetch multiple matcher config paths in one call
- ami.ts: batch resolve SSM parameter values for AMI lookups

Also remove redundant appId SSM fetch in scale-up.ts that was only
used for logging.
@thomasnemer thomasnemer requested a review from a team as a code owner February 4, 2026 13:03
@Brend-Smits Brend-Smits requested a review from Copilot February 5, 2026 12:33
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes AWS SSM API usage by implementing batch parameter fetching to reduce the number of API calls. The changes introduce a new getParameters() function that retrieves multiple SSM parameters in a single API call (with automatic chunking for AWS's 10-parameter limit) and applies this optimization across multiple Lambda functions.

Changes:

  • Added getParameters() function to aws-ssm-util for batch SSM parameter fetching with automatic chunking
  • Replaced sequential SSM calls with batch fetching in authentication, configuration loading, and AMI housekeeping
  • Removed redundant appId SSM fetch in scale-up.ts that was only used for logging

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
lambdas/libs/aws-ssm-util/src/index.ts Implements new getParameters() function for batch SSM parameter retrieval
lambdas/functions/webhook/src/ConfigLoader.ts Replaces sequential parameter fetching with batch call for matcher configs
lambdas/functions/webhook/src/ConfigLoader.test.ts Updates tests to mock getParameters() instead of individual getParameter() calls
lambdas/functions/control-plane/src/scale-runners/scale-up.ts Removes redundant appId SSM fetch used only for logging
lambdas/functions/control-plane/src/github/auth.ts Batches App ID and Private Key fetching into single SSM call
lambdas/functions/control-plane/src/github/auth.test.ts Updates tests to verify batch parameter fetching behavior
lambdas/functions/ami-housekeeper/src/ami.ts Refactors to use batch parameter fetching for AMI SSM lookups
lambdas/functions/ami-housekeeper/src/ami.test.ts Updates tests to use GetParametersCommand instead of GetParameterCommand

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

} else {
this.configLoadingErrors.push(
`Failed to load parameter for matcherConfig from path ${path}: Parameter not found`,
);
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code continues processing remaining parameters even after encountering a missing parameter, which may lead to incomplete configuration. Consider returning early or throwing an error when a required parameter is not found to avoid partially constructed configurations.

Suggested change
);
);
// Stop further processing to avoid partially constructed matcherConfig
return;

Copilot uses AI. Check for mistakes.
Comment on lines +74 to +84
const paramNames = [process.env.PARAMETER_GITHUB_APP_ID_NAME, process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME];
const params = await getParameters(paramNames);

const appIdValue = params.get(process.env.PARAMETER_GITHUB_APP_ID_NAME);
const privateKeyBase64 = params.get(process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME);

if (!appIdValue) {
throw new Error(`Parameter ${process.env.PARAMETER_GITHUB_APP_ID_NAME} not found`);
}
if (!privateKeyBase64) {
throw new Error(`Parameter ${process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME} not found`);
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The array may contain undefined values if the environment variables are not set. This will cause the getParameters call to fail or return unexpected results. Add validation to ensure both environment variables exist before creating the array.

Suggested change
const paramNames = [process.env.PARAMETER_GITHUB_APP_ID_NAME, process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME];
const params = await getParameters(paramNames);
const appIdValue = params.get(process.env.PARAMETER_GITHUB_APP_ID_NAME);
const privateKeyBase64 = params.get(process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME);
if (!appIdValue) {
throw new Error(`Parameter ${process.env.PARAMETER_GITHUB_APP_ID_NAME} not found`);
}
if (!privateKeyBase64) {
throw new Error(`Parameter ${process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME} not found`);
const appIdParamName = process.env.PARAMETER_GITHUB_APP_ID_NAME;
const appKeyParamName = process.env.PARAMETER_GITHUB_APP_KEY_BASE64_NAME;
if (!appIdParamName) {
throw new Error('Environment variable PARAMETER_GITHUB_APP_ID_NAME is not set');
}
if (!appKeyParamName) {
throw new Error('Environment variable PARAMETER_GITHUB_APP_KEY_BASE64_NAME is not set');
}
const paramNames = [appIdParamName, appKeyParamName];
const params = await getParameters(paramNames);
const appIdValue = params.get(appIdParamName);
const privateKeyBase64 = params.get(appKeyParamName);
if (!appIdValue) {
throw new Error(`Parameter ${appIdParamName} not found`);
}
if (!privateKeyBase64) {
throw new Error(`Parameter ${appKeyParamName} not found`);

Copilot uses AI. Check for mistakes.
// Handle wildcard patterns by first discovering matching parameters, then
// fetching their values
let wildcardValues: Promise<(string | undefined)[]> = Promise.resolve([]);
let wildcardValuesPromise: Promise<(string | undefined)[]> = Promise.resolve([]);
Copy link

Copilot AI Feb 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable name 'wildcardValuesPromise' is inconsistent with the original 'wildcardValues'. While the rename improves clarity, it creates inconsistency with the pattern used for 'explicitValuesPromise'.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant