Flagging flaky tests to speed up test

failure analysis for dev teams

Launchable selects the right tests to run for each code change in real-time.

Flaky tests

Unreliable tests are a pervasive problem eating into dev velocity

Flaky test classification takes time

Classifying tests as flakes takes time

Flaky tests originate from multiple reasons such as test, infrastructure, memory failures and so on. However, each test failure requires an additional time to investigate to find the right cause is.

It takes remembering test suite history to finally classify a test as a flake. Thus, in a large test suite classifying flakes becomes a non trivial problem.

Flaky tests rerun

Flakes require test re-runs that slow down dev velocity

Once teams have identified flakes, they have policies to deal with these flakes. These could include running tests multiple times till you get a pass or moving them out in their own runs or testing them later in the pipeline.

Each of the options slows the developers and teams down. Something as simple as re-run the tests till you get a pass can drastically slow teams down.

As entropy sets in, there are more and more flaky tests. Then a hero steps in - analyzes the failures, fixes some flakes. They get some wins and then entropy kicks in all over again.

-- GG, CTO Web Retailer

Finding and fixing flakes is a never ending cycle

As projects & tests grow decay sets in and results in more flakes. Keeping on top of the flakes is a headache that every dev team has to constantly work through

Launchable solution: a flakiness dashboard

Get on top of flaky tests in a test suite quickly

flaky tests dashboard

Quickly get to and eliminate flakes

  • Identify if a test runs failed because of flaky tests helping developers hone in on the real problems and NOT flakes
  • Identify the top flaky tests in a test suite so that teams can prioritize fixing the right flakes
Getting started for flaky tests

Get started in less than 30 minutes

Get your API key. Connect your builds. Connect your tests.

Sit back and let Launchable do the analysis.

Signup for flakiness dashboard beta

Get rid of your flaky tests