Unreliable tests are a pervasive problem eating into dev velocity
Classifying tests as flakes takes time
Flaky tests originate from multiple reasons such as test, infrastructure, memory failures and so on. However, each test failure requires an additional time to investigate to find the right cause is.
It takes remembering test suite history to finally classify a test as a flake. Thus, in a large test suite classifying flakes becomes a non trivial problem.
Flakes require test re-runs that slow down dev velocity
Once teams have identified flakes, they have policies to deal with these flakes. These could include running tests multiple times till you get a pass or moving them out in their own runs or testing them later in the pipeline.
Each of the options slows the developers and teams down. Something as simple as re-run the tests till you get a pass can drastically slow teams down.
As entropy sets in, there are more and more flaky tests. Then a hero steps in - analyzes the failures, fixes some flakes. They get some wins and then entropy kicks in all over again.
-- GG, CTO Web Retailer
Finding and fixing flakes is a never ending cycle
As projects & tests grow decay sets in and results in more flakes. Keeping on top of the flakes is a headache that every dev team has to constantly work through
Launchable solution: a flakiness dashboard
Get on top of flaky tests in a test suite quickly
Quickly get to and eliminate flakes
Identify if a test runs failed because of flaky tests helping developers hone in on the real problems and NOT flakes
Identify the top flaky tests in a test suite so that teams can prioritize fixing the right flakes
Get started in less than 30 minutes
Get your API key. Connect your builds. Connect your tests.