Reduce Test Runtime

Case study

Reducing test runtime for a Ruby on Rails application

Manba is a small two-man startup working to create a community around Manga (Japanese comic books). Today Manba is focused on attracting readers who love comics to their website, which is built with Ruby on Rails.

Key challenge: Long cycle time. Even though Manba is small tests still take over 20 minutes to run. This makes it hard to push out small changes quickly.

Employees Mode of Development Scenario Languages & Tooling
2 Web application Pre-merge tests Ruby, Ruby on Rails, Minitest

Introduction

Meet Masayuki Oguni - the man behind the startup Manba. Oguni-san is the CEO and the lead developer there. Manba is a website for Manga readers. Today, Manba is focused on attracting readers who love comics to their website (if you like Manga, check out Manba's website).

As the founder/CEO, he has his hands full. He codes, he markets, he runs his company. 

The struggle: long cycle times

Manba is built with Ruby on Rails and deployed to Heroku. Oguni-san loves to do quick experiments with new Heroku features. He loves to make small changes and push them out quickly. 

However, every change takes 22 minutes to build and test (4 minutes to build and 18 minutes to run tests). While 22 minutes doesn’t seem a lot, it adds up, especially with multiple code pushes a day. 

I hate waiting! The only good thing about waiting for tests is that I can read comics. 😀

Masayuki Oguni

CEO & Lead Developer

When your hands are full, every minute counts!

Shifting right and saving time

Oguni-san used Launchable to split his test runs. With every commit, Launchable selects the most important tests for the code changes. He lets the nightly test runs capture any tests that might've been missed earlier. Initially, he worried that Launchable might skip some important tests but within a few weeks he quickly built enough confidence with the system that he no longer worries about it.

My test runtime went down 90 percent! Deployment to Heroku went from 30 to 10 minutes. It is great, just great!

Masayuki Oguni

CEO & Lead Developer

He was looking to reduce the cycle time by 20% when he first came across Launchable. After model training, he saw that a 76% reduction in test run time would result in a 60% reduction in cycle time. His usage of CircleCI went down as well.

A drop in testing time means that Oguni-san can make smaller pull requests and he no longer batches things. He can iterate much faster with high confidence. 


So where does Manba go from here?

Oguni-san’s motivation to write more tests has gone up because he no longer views them as a negative (adding to his cycle time). He’s also shipping more features for his customers.

Of course, there is a downside:

No more comic reading between tests for me! 😄

Masayuki Oguni

CEO & Lead Developer


Parting thoughts

If you have a new project on a small team (or at a startup) it is an ideal time to bring in Launchable. At this early phase, tests run times are not as much of a problem but delivering features fast is. As engineers struggle to deliver code at high quality, Launchable can provide that extra time buffer that a team always wishes they had but could never find. 

Launchable supports multiple languages and test frameworks and it is very easy for our team at Launchable to add more. It takes only minutes to integrated Launchable with your build pipeline. We also make it really easy for small teams to adopt us with a free tier for small teams.

Interested in taking Launchable for a spin?

Eliminate slow test cycles. Test what matters.

Your cart
    Checkout