Use ‘Flavors’ to Run the Best Tests for an Environment

A Test Result Is Not Just a Test Result

Key Takeaways

  • Launchable now supports unique test scenarios with a new concept called flavors.

  • This feature lets you select the right tests to run based on the changes being tested and the environment they are being run in.

Lots of teams run the same tests across several different environments. For example, a UI test suite might be run in several browsers in parallel. Or perhaps you need to build a slightly different version of a mobile app for different locales and need to run the same tests across all of them.

In these scenarios, a test result is not just a test result: it is the combination of the test and the environment that it was run in. A test might pass in one environment but fail in another.

Launchable now supports these scenarios with a new concept called flavors.

When you submit test results using `launchable record tests`, you can now submit additional metadata in the form of key-value pairs using the `--flavor` option.

For example:

// after running tests in Chrome
launchable record tests --build [BUILD NAME] --flavor browser=chrome cypress ./report-chrome.xml
// after running tests in Firefox
launchable record tests --build [BUILD NAME] --flavor browser=firefox cypress ./report-chrome.xml

And so on. (You can submit multiple key-value pairs, too.)

Later, when you want to request a subset of tests, you can include the same key-value pairs to get a subset of tests specifically selected for that flavor.

For example:

// before running tests in Chrome, get a subset
find ./cypress/integration | launchable subset --build [BUILD NAME] --confidence 90% --flavor browser=chrome cypress > subset-chrome.txt
// run the Chrome subset
cypress run --spec "$(cat subset-chrome.txt)" --reporter junit --reporter-options "mochaFile=report/test-output-chrome-[hash].xml"
...
// before running tests in Firefox, get a subset
find ./cypress/integration | launchable subset --build [BUILD NAME] --confidence 90% --flavor browser=firefox cypress > subset-chrome.txt
// run the Chrome subset
cypress run --spec "$(cat subset-chrome.txt)" --reporter junit --reporter-options "mochaFile=report/test-output-firefox-[hash].xml"

This feature lets you select the right tests to run based on the changes being tested and the environment they are being run in.

Seeking Your Expert Feedback on Our AI-Driven Solution

Quality a focus? Working with nightly, integration or UI tests?
Our AI can help.