cucumber-json does not contain all scenarios if testing multiple URLs in one feature

[benjaminknauer]

We have the requirement to test multiple clients with different URLs. For this we have a given step which overwrites the base URL in the cypress object. If a cy.visit() is then executed on a "new" domain, the Cypress Runner is restarted.

See also:
https://docs.cypress.io/guides/references/best-practices.html#Setting-a-global-baseUrl

"As soon as it encounters a cy.visit (), Cypress then switches to the url of the main window to the url specified in your visit. This can result in a 'flash' or 'reload' when your tests first start."

This behavior means that all scenarios that were executed before the restart are not listed in the generated cucumber-json file.

Current behavior

cucumber-json file does not contain all scenarios if multiple URLs are tested in one feature.

Desired behavior

All scenarios should be present in cucumber-json file

Test code to reproduce

https:/benjaminknauer/cypress-cucumber-preprocessor-cucumber-json-bug

Feature: Test with multiple domains

Scenario: Test google
When you visit "https://www.google.de"
Then the url includes "google"

Scenario: Test ebay
When you visit "https://www.ebay.de"
Then the url includes "ebay"

--> Only Test ebay is present in cucumber-json/test.cucumber.json

[benjaminknauer]

Any comment on this? 🤔

[lgandecki]

Sorry, me and my team didn't work on nor uses this feature, it was provided by the community. We are definitely open to take a PR but can't really provide much more support than that at this moment.

[a8trejo]

Ran into this exact same problem and followed the next approach:

-Separated the feature files, one per different domain

-In some of the features where same domain was used but needed to navigate to different URLs same domain to validate a text, we used cy.request instead of cy.visit and validated the response body.

That was the only way the report were generated correctly.

Hope it helps your case as well

[zamioku]

I'm facing exactly the same issue on Cypress version 6.1.0

@benjaminknauer @a8trejo thank you so much for your info! I couldn't figure out why my Api test results were ok but Ui ones were not.

Has anyone found a workaround to stop Cypress from reloading every time?

[benjaminknauer]

Problem exists in this code fragment:
https:/TheBrainFamily/cypress-cucumber-preprocessor/blob/ddcb25a4522ad388a19e95c2e206a7cb52884b3e/lib/createTestFromScenario.js#L103

window.testState is cleared after runner restart.

possible fixes:

• generate Cucumber json after each scenario
  https:/TheBrainFamily/cypress-cucumber-preprocessor/blob/ddcb25a4522ad388a19e95c2e206a7cb52884b3e/lib/createTestFromScenario.js#L164
• do not persist test state on window. use some sort of other storage

[benjaminknauer]

I updated my test repo https:/benjaminknauer/cypress-cucumber-preprocessor-cucumber-json-bug to the newest version (4.0.3). Still not fixed :(

[badeball]

Due to personal reasons, the previous maintainers of this package are stepping down and handing the reigns over to me, a long-time contributor to the project and a user of it myself. This is a responsibility I'm very excited about. Furthermore, I'd like to thank @lgandecki ++ for all the work that they've done so far.

Read more about the transfer of ownership here.

The repository has however moved and all outstanding issues are being closed. This is not a reflection of the perceived importance of your reported issue. However, if after upgrading to the new version, you still find there to be an issue, feel free to open up another ticket or comment below. Please make sure to read CONTRIBUTING.md before doing so.