Sai Venkatesh works for a virtualization technology company based out of California, let's call it RocketSoft. RocketSoft delivers multiple products including on-premises products and SaaS solutions. Sai came to RocketSoft through an acquistion and his prior role was as founder/CTO of the acquired company. Today, he is the VP of Engineering for a group that has about 1500 developers.
At RocketSoft, Sai was not surprised to see that software delivery times were high, a direct outcome of a complex codebase. Integration of the code bases of the two companies will further compounding the problem.
He is acutely aware that the developers are unhappy because of the slow velocity. He is starting to see a steady attrition especially in Silicon Valley, where developers are at a premium.
He yearns to bring the velocity, productivity from his startup days to RocketSoft, thus making his developers happy and plug the attrition risk.
Development times are slow and we are not productive… these things make people dissatisfied. It’s not just about productivity it’s about satisfaction which drives retention.
Sai Venkatesh, VP of Engineering
The velocity, attrition, complex code base is hurting RocketSoft's ability to innovate.
Side note: “Slow Velocity” is a conclusion Sai reached based on grumblings he hears from developers. Launchable was able to back this gut with data. His lighthouse team of 50 developers was pushing only 6 changes per day and it took 5 patches to get 1 change in.
This was an eye opener for Sai.
RocketSoft is throwing a lot to drive velocity. They have allocated a couple of engineers to drive pipeline automations and embarked a move to Bazel to speed their builds which is in year 2 of a multi year project. The hypothesis is that if builds go faster, developers can push more commits and thus drive innovation.
The problem is that these efforts will not impact the biggest bottleneck which are tests. In their on-premises product line, the basic acceptance tests takes 3 days!
Launchable started with the smaller lighthouse SaaS product line (mentioned earlier) which has about 50-75 developers.
Today, the team runs 199 builds per week. Each build runs for 51 minutes with a test run of 22 minutes. Launchable brough its Predictive Test Selection to smartly identify the tests likely to fail and we prioritized them first.
We analyzed their code, tests to produce the following graph.
This graph shows how quickly can Launchable find a failing build. In other words, if I was a developer pushing my change in and wanted to know if I am going to break the build then Launchable can find a breaking build with 90% confidence with 10% of tests. I can pick the test that fails the build and go about debugging it very quickly.
So what does this mean in absolute terms? Or how quickly is how quickly?
How about a 90% reduction in test time or a 40% reduction in the entire build time.
How about a 90% reduction in test time or a 40% reduction in the entire build time!
Find issues earlier. Innovate faster This is clear that developers are finding failures 40% faster. They can fix issues and go back to pushing more.
Happier developers Developers are now pushing more code in. Fast turnaround time leaves developers happier.
The goal for RocketSoft has been to get to their on-premises application where the acceptance tests takes 3 days. They are looking to bring down the feedback time of this test harness from 3 days to 2 and ultimately to 1 day.
In conclusion, Sai has brought in the power of AI and ML for one of his teams; relatively effortlessly. The SaaS team is innovating faster and other teams are clamoring to bring Launchable to their projects.
Pre-merge tests. Java, Python, Gerrit and Bazel