Reducing test runtime for a Ruby on Rails application
Manba is a small two-man startup working to create a community around Manga (Japanese comic books). Today Manba is focused on attracting readers who love comics to their website, which is built with Ruby on Rails.
Key challenge: Long cycle time. Even though Manba is small tests still take over 20 minutes to run. This makes it hard to push out small changes quickly.
Employees
Mode of Development
Scenario
Languages & Tooling
2
Web application
Pre-merge tests
Ruby, Ruby on Rails, Minitest
Introduction
Meet Masayuki Oguni - the man behind the startup Manba. Oguni-san is the CEO and the lead developer there. Manba is a website for Manga readers. Today, Manba is focused on attracting readers who love comics to their website (if you like Manga, check out Manba's website).
As the founder/CEO, he has his hands full. He codes, he markets, he runs his company.
The struggle: long cycle times
Manba is built with Ruby on Rails and deployed to Heroku. Oguni-san loves to do quick experiments with new Heroku features. He loves to make small changes and push them out quickly.
However, every change takes 22 minutes to build and test (4 minutes to build and 18 minutes to run tests). While 22 minutes doesn’t seem a lot, it adds up, especially with multiple code pushes a day.
I hate waiting! The only good thing about waiting for tests is that I can read comics. 😀
Masayuki Oguni
CEO & Lead Developer
When your hands are full, every minute counts!
Shifting right and saving time
Oguni-san used Launchable to split his test runs. With every commit, Launchable selects the most important tests for the code changes. He lets the nightly test runs capture any tests that might've been missed earlier. Initially, he worried that Launchable might skip some important tests but within a few weeks he quickly built enough confidence with the system that he no longer worries about it.
My test runtime went down 90 percent! Deployment to Heroku went from 30 to 10 minutes. It is great, just great!
Masayuki Oguni
CEO & Lead Developer
He was looking to reduce the cycle time by 20% when he first came across Launchable. After model training, he saw that a 76% reduction in test run time would result in a 60% reduction in cycle time. His usage of CircleCI went down as well.
A drop in testing time means that Oguni-san can make smaller pull requests and he no longer batches things. He can iterate much faster with high confidence.
So where does Manba go from here?
Oguni-san’s motivation to write more tests has gone up because he no longer views them as a negative (adding to his cycle time). He’s also shipping more features for his customers.
Of course, there is a downside:
No more comic reading between tests for me! 😄
Masayuki Oguni
CEO & Lead Developer
Parting thoughts
If you have a new project on a small team (or at a startup) it is an ideal time to bring in Launchable. At this early phase, tests run times are not as much of a problem but delivering features fast is. As engineers struggle to deliver code at high quality, Launchable can provide that extra time buffer that a team always wishes they had but could never find.
