chore: split connect and switch-connection tool MCP-301 (#749) #747
accuracy-tests.yml
on: push
Run Accuracy Tests
13m 5s
Annotations
6 errors
|
Run Accuracy Tests
Process completed with exit code 1.
|
|
tests/accuracy/insertMany.test.ts > 'Azure - gpt-4o' > 'In my namespace \'mflix.movies\', ins…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 5 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 5 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly
|
|
tests/accuracy/explain.test.ts > 'Azure - gpt-4o' > 'Will counting documents, where releas…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly f
|
|
tests/accuracy/dropIndex.test.ts > 'Azure - gpt-4o' > 'Create a vector search index on \'mfl…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly f
|
|
tests/accuracy/dropIndex.test.ts > 'Azure - gpt-4o' > 'First create a text index on field \'…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 2 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 2 seconds. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly
|
|
tests/accuracy/dropDatabase.test.ts > 'Azure - gpt-4o' > 'If there is a mflix database in my cl…':
tests/accuracy/sdk/agent.ts#L68
AI_RetryError: Failed after 3 attempts. Last error: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.
❯ _retryWithExponentialBackoff node_modules/ai/src/util/retry-with-exponential-backoff.ts:111:13
❯ fn node_modules/ai/src/generate-text/generate-text.ts:345:34
❯ node_modules/ai/src/telemetry/record-span.ts:18:22
❯ generateText node_modules/ai/src/generate-text/generate-text.ts:281:12
❯ Object.prompt tests/accuracy/sdk/agent.ts:68:44
❯ tests/accuracy/sdk/describeAccuracyTests.ts:124:28
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { reason: 'maxRetriesExceeded', errors: [ { stack: 'AI_APICallError: Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/response-handler.ts:57:16\n at processTicksAndRejections (node:internal/process/task_queues:103:5)\n at postToApi (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/provider-utils/src/post-to-api.ts:118:28)\n at OpenAIChatLanguageModel.doGenerate (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/@ai-sdk/openai/src/chat/openai-chat-language-model.ts:321:9)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:389:32)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at _retryWithExponentialBackoff (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/util/retry-with-exponential-backoff.ts:96:12)\n at fn (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:345:34)\n at file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/telemetry/record-span.ts:18:22\n at generateText (file:///home/runner/work/mongodb-mcp-server/mongodb-mcp-server/node_modules/ai/src/generate-text/generate-text.ts:281:12)', message: 'Your requests to gpt-4o for gpt-4o in South Central US have exceeded the token rate limit for your current OpenAI S0 pricing tier. This request was for ChatCompletions_Create under Azure OpenAI API version 2024-12-01-preview. Please retry after 1 second. To increase your default rate limit, visit: https://aka.ms/oai/quotaincrease.', name: 'AI_APICallError', cause: undefined, url: 'https://gai-405.openai.azure.com/openai//deployments/gpt-4o/chat/completions?api-version=2024-12-01-preview', requestBodyValues: { model: 'gpt-4o', logit_bias: undefined, logprobs: undefined, top_logprobs: undefined, user: undefined, parallel_tool_calls: undefined, max_tokens: undefined, temperature: undefined, top_p: undefined, frequency_penalty: undefined, presence_penalty: undefined, response_format: undefined, stop: undefined, seed: undefined, verbosity: undefined, max_completion_tokens: undefined, store: undefined, metadata: undefined, prediction: undefined, reasoning_effort: undefined, service_tier: undefined, prompt_cache_key: undefined, safety_identifier: undefined, messages: [ { role: 'system', content: 'The keywords "MUST", "MUST NOT", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119\nYou are an expert AI assistant with access to a set of tools for MongoDB database operations.\nYou MUST use the most relevant tool to answer the user\'s request\nWhen calling a tool, you MUST strictly f
|
Artifacts
Produced during runtime
| Name | Size | Digest | |
|---|---|---|---|
|
accuracy-test-summary
|
47.4 KB |
sha256:bd10e690f1547682f1b28f0681d3020645b2e695a13c34d3abe9140040988279
|
|
|
run-accuracy-tests-permissions-9bb4f13708c94cabdab616ebdb062ef8
|
140 Bytes |
sha256:9fdfb2d22728f0e62d6d566420d383df98688329ecb097bf6f1c502fbeb9e481
|
|