Managing complex test session layouts

The Test Session is one of Launchable's core concepts. When you record test results, those results are recorded to a test session. When you request a subset of tests from Launchable, the subset is linked to a test session, too. This concept is useful because tests might run several times against the same build; it helps disambiguate those runs.

Normally, the Launchable CLI manages session creation and usage in the background. However, if your build and test processes are split across multiple machines, or if your tests are parallelized across multiple machines, you'll need to create test sessions yourself.

Build and test processes happen on different machines

Normally, the Launchable CLI handles creating, saving, and retrieving a session ID in the background. When you run launchable subset or launchable record tests, the CLI checks for an existing file in ~/.launchable. This file is written when you run launchable record build.

However, if you need to record tests (launchable record tests) or request a subset (launchable subset) on a different machine than the one where launchable record build ran, the ~/.launchable file won't be present, and you'll need to manually create a test session using the launchable record session command at the beginning of your test process.

This command outputs a string that you can store and then pass into the --session option in launchable subset and launchable record tests.

Here's some pseudocode to illustrate:

1# machine 1
2
3    # build step
4    launchable record build --name <BUILD NAME> [OPTIONS]
5
6    # build software
7    bundle install
8
9# machine 2
10
11    # before running tests, save a session token
12    # you'll use this token again later
13    launchable record session --build <BUILD NAME> > launchable-session.txt
14
15    # get a subset, if applicable
16    launchable subset --session $(cat launchable-session.txt) [OPTIONS]
17
18    # run tests
19    bundle exec rails test [OPTIONS]
20
21# machine 3
22
23    # record test results
24    launchable record tests --session $(cat launchable-session.txt) [OPTIONS]

Combining test reports from multiple runs

Some pipelines execute multiple test runs against a build, outputting distinct test report(s) across several machines. Depending on your layout (see Test Session), you may want to combine these into a single test session. This section explains how to do this.

To tie multiple launchable record tests invocations to a single test session:

1. Use the launchable record session command to create a session ID

2. Pass this ID into the corresponding --session parameter in launchable record tests (note: pseudocode):

1## build step
2
3# before building software, send commit and build info
4# to Launchable
5launchable record build --build <BUILD NAME> [OPTIONS]
6
7# build software the way you normally do, for example
8bundle install
9
10## test step
11
12# before running tests, save a session token
13# you'll use this token to group the test reports together later
14launchable record session --build <BUILD NAME> > launchable-session.txt
15
16    # start multiple test runs
17
18        # machine 1
19
20            # run tests
21            bundle exec rails test
22
23            # send test results to Launchable from machine 1
24            # Note: You need to configure the line to always run whether test run succeeds/fails.
25            #       See each integration page.
26            launchable record tests --session $(cat launchable-session.txt) [OPTIONS]
27
28        # machine 2
29
30            # run tests
31            bundle exec rails test
32
33            # send test results to Launchable from machine 2
34            # Note: You need to configure the line to always run whether test run succeeds/fails.
35            #       See each integration page.
36            launchable record tests --session $(cat launchable-session.txt) [OPTIONS]
37
38        ## repeat as needed...
39
40## finish multiple test runs

You can read more about launchable record session in the CLI reference.