Cucumber JSON files are not matching with the cucumber-json-schema

[ramakkonatam]

Framework - Webdriverio - Cucumber - Typescript
Reporter - Wdio-cucumber-Json-reporter
CI - Jenkins
Integration tools - Zephyr
package.json

The issue we are encountering pertains to the generated JSON files from our test executions. Our objective is to upload the JSON results into the Zephyr tool from Jenkins using either the Zephyr plugin or the Zephyr API. Unfortunately, in both scenarios, we encounter a JSON parse error.

To address this challenge, we have thoroughly examined the Cucumber JSON files generated under the framework against the official JSON schema provided by Cucumber (https://github.com/cucumber/cucumber-json-schema/blob/main/schema.json). Regrettably, the JSON files do not align with the Cucumber JSON schema. Consequently, the Zephyr API, which validates the uploaded JSON files against the same schema, triggers a JSON parse error.

Now, the question arises: How can we ensure that the generated JSON files conform to the Cucumber JSON Schema? Is there a possibility to customize the JSON files to meet the required specifications? Additionally, we are puzzled as to why the WebDriverIO-Cucumber-generated JSON files fail to match the JSON schema (https://github.com/cucumber/cucumber-json-schema/blob/main/schema.json).

We are seeking guidance on aligning the JSON files with the Cucumber JSON schema and understanding whether there is flexibility to customize the JSON files. Furthermore, we would appreciate insights into why the JSON files generated by WebDriverIO-Cucumber do not adhere to the specified JSON schema (https://github.com/cucumber/cucumber-json-schema/blob/main/schema.json).
schema.json

[christian-bromann]

Thanks for reporting!

Please note that the reporter hasn't received much maintenance over the last time due to limited maintainer resources.

We greatly appreciate any contributions that help resolve the bug. While we understand that active contributors have their own priorities, we kindly request your assistance if you rely on this bug being fixed. We encourage you to take a look at our contribution guidelines or join our friendly Discord development server, where you can ask any questions you may have. Thank you for your support, and cheers!

[ramakkonatam]

Hi @christian-bromann
Thanks for the reply.
Could you please clarify whether the WDIO-Cucumber framework supports uploading cucumber.json files to Zephyr Scale Cloud? This information is crucial for us to make an informed decision about whether to proceed with migrating from Mocha to Cucumber.
Here I am attaching the framework-generated .json file and the
FrameworkGenerated.json
expected JSON schema(https://github.com/cucumber/cucumber-json-schema/blob/main/schema.json)
schema.json

Framework generated JSON
[ { "keyword": "Feature", "type": "feature", "description": "", "line": 2, "name": "The Internet Guinea Pig Website", "uri": "/Desktop/webdriverio-web-mobile-boilerplate-main/tests/features/demo-login.feature", "tags": [ { "location": { "line": 1, "column": 1 }, "name": "@demo", "id": "8" } ], "elements": [ { "keyword": "Scenario", "type": "scenario", "description": "", "name": "As a user, I can log into the secure area", "tags": [ { "name": "@demo", "astNodeId": "8" } ], "id": "the-internet-guinea-pig-websiteas-a-user,-i-can-log-into-the-secure-area", "steps": [ { "arguments": [], "keyword": "Before", "name": "", "result": { "status": "passed", "duration": 1000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "Given", "name": "I am on the login page", "result": { "status": "passed", "duration": 3622000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "When", "name": "I login to the application with valid_user_credentials", "result": { "status": "passed", "duration": 1025000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "Then", "name": "I should see a flash message saying You logged into a secure area!", "result": { "status": "passed", "duration": 39000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "After", "name": "", "result": { "status": "passed", "duration": 647000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } } ] }, { "keyword": "Scenario", "type": "scenario", "description": "", "name": "As a user, I can log into the secure area", "tags": [ { "name": "@demo", "astNodeId": "8" } ], "id": "the-internet-guinea-pig-websiteas-a-user,-i-can-log-into-the-secure-area", "steps": [ { "arguments": [], "keyword": "Before", "name": "", "result": { "status": "passed", "duration": 1000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "Given", "name": "I am on the login page", "result": { "status": "passed", "duration": 3381000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "When", "name": "I login to the application with invalid_user_credentials", "result": { "status": "passed", "duration": 684000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "Then", "name": "I should see a flash message saying Your username is invalid!", "result": { "status": "passed", "duration": 35000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } }, { "arguments": [], "keyword": "After", "name": "", "result": { "status": "passed", "duration": 725000000 }, "line": null, "match": { "location": "can not be determined with webdriver.io" } } ] } ], "id": "the-internet-guinea-pig-website", "metadata": { "browser": { "name": "chrome", "version": "120.0.6099.109" }, "device": "Device name not known", "platform": { "name": "osx", "version": "Version not known" } } } ]

[christian-bromann]

supports uploading cucumber.json files to Zephyr Scale Cloud?

Afaik it doesn't. But support can be surely added.

[ramakkonatam]

Thanks @christian-bromann
Your answer has helped us to postpone the framework migration from Mocha to Cucumber. We eagerly await this support, as it would provide significant business value, particularly in terms of uploading the results to Zephyr.
If possible, may I kindly inquire about the tentative timeline for the support of Zephyr?

[christian-bromann]

If possible, may I kindly inquire about the tentative timeline for the support of Zephyr?

As mentioned above, there is no active development on this package due to a shortage of maintainers. If you have interest in support for this we would appreciate any contribution. Thank you.