Flagging flaky tests to speed up test

failure analysis for dev teams

Flaky Tests Insights

Unreliable tests are a pervasive problem eating into development velocity

Flaky test classification takes time

Classifying tests as flaky takes time

Flaky tests originate from multiple sources such as poorly written tests, infrastructure issues, memory failures, and so on. However, each test failure requires additional time to identify the actual cause.

Classifying flaky tests requires remembering the history of a test suite history. Thus, in a large test suite, classifying flakes becomes a non trivial problem.

Flaky tests rerun

Flakes require test re-runs that slow down dev velocity

Once teams have identified flakes, they have policies to deal with them. These could include running tests multiple times until they pass, moving them out in their own runs, or testing them later in the pipeline.

Each option slows the developers and teams down. Something as simple as re-running the tests till you get a pass can drastically reduce velocity.

As entropy sets in, there are more and more flaky tests. Then a hero steps in - analyzes the failures, fixes some flakes. They get some wins and then entropy kicks in all over again.

- GG, CTO of a Web Retailer

Finding and fixing flakes is a never ending cycle

As projects and tests grow, decay sets in, along with flaky tests. Keeping on top of flaky tests is a headache that every development team has to constantly work through.

Launchable's solution:

Flaky Tests Insights

flaky tests dashboard

Quickly identify and eliminate flakes

  • Identify the top flaky tests in a test suite to prioritize fixing the right tests
  • Identify if a test run failed because of flaky tests to hone in on the real problems
Getting started for flaky tests

Get started in less than 30 minutes

Get your API key. Connect your builds. Connect your test results. Sit back, and let Launchable do the analysis.

Sign up for Launchable to use the Flaky Tests Insights

Get rid of your flaky tests!

Your cart
    Checkout