docs: add documentation for running aws integ tests locally #4830
Conversation
PavelSafronov
changed the title
dariakp
changed the title
baileympearson
changed the title
baileympearson
changed the title
| set -euxo pipefail # Exit on error, undefined variable, and fail on pipe errors | ||
|
|
||
| # NOTE: This script assumes that you've created an AWS SSO session already, as outlined in | ||
| # ./test/readme.md, section `AWS Profile`, and you have an AWS profile |
There was a problem hiding this comment.
| # ./test/readme.md, section `AWS Profile`, and you have an AWS profile | |
| # ../test/readme.md, section `AWS Profile`, and you have an AWS profile |
|
|
||
| Setup an AWS_PROFILE locally to be able to use AWS and to run AWS tests locally. | ||
|
|
||
| 1. Get SSO sign-in info from AWS |
There was a problem hiding this comment.
Are these steps just a duplicate of the docs here? https://wiki.corp.mongodb.com/spaces/DRIVERS/pages/239737385/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets
If so, I'd suggest just linking these docs instead
There was a problem hiding this comment.
Actually, we do link to these docs (indirectly) in the secrets section of the readme
There was a problem hiding this comment.
The instructions are indirectly linked, so it did take a bit of experimentation to figure out what was necessary or not. I think it may be helpful to be explicit about these instructions, hence why I included them.
What about adding a link to the wiki, but keeping this section as "quick setup" instructions?
| source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh | ||
|
|
||
| # clear AWS_PROFILE, so it doesn't interfere with env-creds | ||
| unset AWS_PROFILE |
There was a problem hiding this comment.
Is this environment variable used by any AWS auth variants in CI?
aWs there a particular reason this change was needed? iirc the AWS credential provider SDK fetches credentials from the environment first. So it's unclear how having the AWS_PROFILE set would interfere with AWS environment variable credential fetching.
There was a problem hiding this comment.
This variable is not used by any AWS auth variants in our CI.
The tests fetch the credentials from the environment, and this is fine in CI where we use env access/secret keys directly: we use the env creds to generate temporary/EC2/whatever credentials, then the new access/secret/token values overwrite the env vars, and the SDK picks up the newest values.
But running these locally with a profile configured I saw a different behavior: we use the profile to generate temporary/EC2/whatever credentials, then the new access/secret/token values are added to the env, in addition to the profile. When this happens, our tests fail with an auth error (you can verify this by commenting out the line unset AWS_PROFILE) :
> mongodb@7.0.0 check:aws
> nyc mocha --config test/mocha_mongodb.js test/integration/auth/mongodb_aws.test.ts test/integration/auth/mongodb_aws.prose.test.ts
[baseline-browser-mapping] The data in this module is over two months old. To ensure accurate Baseline data, please update: `npm i baseline-browser-mapping@latest -D`
@aws-sdk/credential-provider-node - defaultProvider::fromEnv WARNING:
Multiple credential sources detected:
Both AWS_PROFILE and the pair AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY static credentials are set.
This SDK will proceed with the AWS_PROFILE value.
However, a future version may change this behavior to prefer the ENV static credentials.
Please ensure that your environment only sets either the AWS_PROFILE or the
AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY pair.
1) "before all" hook: testConfigBeforeHook in "{root}"
0 passing (1s)
1 failing
1) "before all" hook: testConfigBeforeHook in "{root}":
MongoServerError: Authentication failed.
at Connection.sendCommand (src/cmap/connection.ts:168:71)
at processTicksAndRejections (node:internal/process/task_queues:103:5)
at async Connection.command (src/cmap/connection.ts:168:71)
at async MongoDBAWS.auth (src/cmap/auth/mongodb_aws.ts:17:45)
at async performInitialHandshake (src/cmap/connect.ts:7:53)
at async connect (src/cmap/connect.ts:2:1757)
Compare this to a run where the AWS_PROFILE is unset:
> mongodb@7.0.0 check:aws
> nyc mocha --config test/mocha_mongodb.js test/integration/auth/mongodb_aws.test.ts test/integration/auth/mongodb_aws.prose.test.ts
[baseline-browser-mapping] The data in this module is over two months old. To ensure accurate Baseline data, please update: `npm i baseline-browser-mapping@latest -D`
{
topology: 'Single',
version: '8.3.0-alpha0-4662-g7385039',
node: 'v24.12.0',
os: 'darwin',
alpineLinux: false,
cryptdUri: undefined,
pid: 15263,
auth: true,
tls: false,
csfle: {
enabled: false,
mongodbClientEncryption: {
cryptoCallbacks: [Getter],
MongoCryptContext: [class MongoCryptContext],
MongoCrypt: [Function]
},
version: '7.0.0',
libmongocrypt: '1.15.1'
},
cryptSharedVersion: null,
cryptSharedLibPath: undefined,
serverApi: undefined,
atlas: false,
aws: true,
awsSdk: undefined,
azure: false,
kerberos: false,
ldap: false,
socks5: false,
compressor: undefined,
zstdVersion: '7.0.0'
}
====================================================================================================
Flaky test: "Change Streams should properly handle a changeStream event being processed mid-close when invoked with promises" is not run at all
====================================================================================================
MONGODB-AWS Prose Tests
1. Custom Credential Provider Authenticates
✔ authenticates with a user provided credentials provider (578ms)
2. Custom Credential Provider Authentication Precedence
Case 2: Custom Provider Takes Precedence Over Environment Variables
✔ authenticates with a user provided credentials provider (480ms)
MONGODB-AWS
when the AWS SDK is not present
when attempting AWS auth
✔ throws an error
when the AWS SDK is present
✔ should authorize when successfully authenticated (560ms)
✔ should allow empty string in authMechanismProperties.AWS_SESSION_TOKEN to override AWS_SESSION_TOKEN environment variable
✔ should store a MongoDBAWS provider instance per client (526ms)
ConversationId
✔ should use conversationId returned by saslStart in saslContinue
with missing aws token
✔ should not throw an exception when aws token is missing (622ms)
EC2 with missing credentials
- should respect the default timeout of 10000ms
↬ requires an AWS EC2 environment
when using AssumeRoleWithWebIdentity
when no AWS region settings are set
- uses the default region
- fromNodeProviderChain called once
when only AWS_STS_REGIONAL_ENDPOINTS is set
- uses the default region
- fromNodeProviderChain called once
when only AWS_REGION is set
- uses the default region
- fromNodeProviderChain called once
when AWS_STS_REGIONAL_ENDPOINTS is set to regional and region is legacy
- uses the region from the environment
- fromNodeProviderChain called once
when AWS_STS_REGIONAL_ENDPOINTS is set to regional and region is new
- uses the region from the environment
- fromNodeProviderChain called once
when AWS_STS_REGIONAL_ENDPOINTS is set to legacy and region is legacy
- uses the region from the environment
- fromNodeProviderChain called once
when AWS_STS_REGIONAL_ENDPOINTS is set to legacy and region is new
- uses the default region
- fromNodeProviderChain called once
AWS KMS Credential Fetching
when the AWS SDK is not installed
✔ fetching AWS KMS credentials throws an error
when the AWS SDK is installed
✔ does not return any extra keys for the `aws` credential provider
when a credential provider is not provided
✔ KMS credentials are successfully fetched.
when a credential provider is provided
✔ KMS credentials are successfully fetched.
12 passing (3s)
15 pending
wrote xunit.xml
=============================== Coverage summary ===============================
Statements : 40.05% ( 4184/10445 )
Branches : 23.8% ( 1081/4541 )
Functions : 26.96% ( 518/1921 )
Lines : 40.43% ( 4096/10131 )
================================================================================
| @@ -0,0 +1,33 @@ | |||
| #!/usr/bin/env bash | |||
| set -euxo pipefail # Exit on error, undefined variable, and fail on pipe errors | |||
|
|
|||
There was a problem hiding this comment.
Generally, if we add a script that we plan to use for local testing, we also ensure that it is the same script used to run tests in CI. This ensures the script is always up-to-date.
- Is there something this script does that isn't captured by our testing readme's instructions?
- Do you think it's worth refactoring CI to use this script?
There was a problem hiding this comment.
I don't feel too strongly about including this script. The way we run tests in CI makes sense and I don't think we need to change it.
This was mostly my own experimentation to figure out a way to simply run AWS tests locally. Can definitely drop it from the PR.
baileympearson
added
the
Primary Review
2 participants
Description
Summary of Changes
Add documentation explaining how to run AWS integration tests locally.
Double check the following
npm run check:lint)type(NODE-xxxx)[!]: descriptionfeat(NODE-1234)!: rewriting everything in coffeescript