The key challenge: Triaging test failures is difficult and time consuming
Triaging test failures is a necessary part of any SDLC, but it comes at a cost
Handling test failures via triage is something all software teams have to do at some point. Triaging test failures is the process of finding who is responsible for the test, assigning them a task to resolve the test, then verifying the test is fixed in the next test cycle.
Triaging test failures takes work. The complexity increases as the codebase and teams grow. The problem is not specific to an industry; it is pervasive across all industry verticals. This responsibility sometimes falls on developers, other times on QA teams, but both sides agree: They need assistance in this process to help reduce the cognitive load and relative effort that it requires.
Solution: Intelligently group failures to reduce cognitive load for root cause analysis
With Launchable’s Intelligent Test Failure Diagnostics, we are able to intelligently identify and summarize failures across tests. We use GenAI to help assist in making this possible by analyzing stderr and test result output to simplify error messages.
Inside of our web app, you will find an ‘Issues’ tab for each test session. These issues will be a summarization of all test failures inside of that test session. If there were related failures in the run, they will be placed under a common issue that succinctly describes the failure. This will assist both developers and QA personnel with quickly performing root cause analysis on failing tests. Along with Launchable’s identification of test characteristics, such as flaky test identification, this should greatly reduce the time it takes to find the root cause. Test failure triaging now becomes instantaneous with this issue simplification.
In addition to this, Launchable provides daily reports sent via email that contain test issue summaries organized by branch. This makes it easy to see which issues are affecting various branches in your development cycle.
Benefit: Test failure triaging overall time is reduced and streamlined
By using Intelligent Test Failure diagnostics, test failure triage processes can be greatly simplified. This feature can help to summarize and provide daily updates of issues on a branch by branch basis, empowering development and QA teams to move quickly to address issues.