chore(deps): bump peter-evans/create-pull-request from 7.0.8 to 7.0.9… #749
mongodb-devtools-bot[bot]accuracy-tests.yml
on: push
Run Accuracy Tests
12m 52s
Annotations
6 errors
|
Run Accuracy Tests
Process completed with exit code 1.
|
|
tests/accuracy/listCollections.test.ts > 'Azure - gpt-4o' > 'Is there a shows collection in my Mon…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 2 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly
|
|
tests/accuracy/listCollections.test.ts > 'Azure - gpt-4o' > 'List all the collections in my MongoD…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 3 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 3 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly
|
|
tests/accuracy/dropIndex.vectorSearchDisabled.test.ts > 'Azure - gpt-4o' > 'Delete the index called year_1 from m…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly f
|
|
tests/accuracy/dropCollection.test.ts > 'Azure - gpt-4o' > 'Remove books collection from which ev…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 2 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 2 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly
|
|
tests/accuracy/createDeployment.test.ts > 'Azure - gpt-4o' > 'Setup a local MongoDB cluster, do not…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly f
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
accuracy-test-summary
|
48.6 KB |
sha256:a3828ff8997b4fbc0d3cefe254d553fa4e59582168c5db32c0e797059757ccde
|
|
|
run-accuracy-tests-permissions-ab5c15b3dc68b4d3ec804d1f40ee36a9
|
140 Bytes |
sha256:06c0bd30ef356cd55e31f2e30d1ebadf51087dd6ee93de4e7a3857f7a0e51704
|
|