Predicting quarterly bookings for HashiCorp
Keeping the team ahead of surprises from sales
Role: Product Designer
Background
HashiCorp is one of the fastest-growing tech companies around today. In order to manage that growth effectively, the finance team needed an objective bookings forecast to complement what they were hearing from Sales. They had been using spreadsheets to create forecasts based on historical patterns, but the high-level approach couldn’t facilitate detailed analysis and the complexity involved led to infrequent updates—limiting the team’s ability to look ahead.
We partnered with HashiCorp to develop an automated, reliable bookings forecast for the finance team. As this features was being produced as a partnership, success would be simple to measure—we wouldn’t be done until the team was satisfied.
Creating a bookings forecast was not as straightforward as it was for cash because the data science capabilities at our disposal were somewhat constrained. For example, we could not predict the date that a deal would close, but we could predict the likelihood that a deal would close by a certain date.
We needed to figure out how to deliver sufficient value given the tools at our disposal.
How might we…
Provide the HashiCorp team with the most useful bookings forecast possible?
Research and iteration
Upon launching the project, we used regular checkins with the HashiCorp team to present ideas and get feedback on both the UX and the data science work as it progressed.
To start, we explored how the predictive capabilities at our disposal could be used. Given that the forecast was being built for 13-week quarters with fixed date ranges, we agreed it would make sense try assigning a probability to the likelihood that a given deal would close by the end of a given quarter.
Using Rivet’s cash forecast as a starting point, we focused UX testing on the information that unique to the bookings forecast. In addition to the forecast itself, we knew the team was interested in segmentation like new business vs. renewals, as well as comparisons to what their Sales team was forecasting.
We started by testing a top-down forecast that presented a likely range of outcomes based on monte carlo simulations. While technically sound, this proved to be too similar to how HashiCorp had been forecasting in-house. They were already using Rivet’s cash forecast and expressed interest in opportunity-level predictions, so we looked into options.
We needed to figure out how to summarize the thousands of sales opportunities – and their varying degrees of probability – into a digestible forecast that the finance team could a) consume quickly, and b) self-serve any common follow-up questions. Additionally, the raw probabilities produced by our ML were confusing to understand.
Solution
To make the probabilities more intuitive, we created a formula that translated those probabilities into a simple “Rivet Score” with a 0-100 scale. In addition to providing opportunity-level predictions, the Rivet Score was also be used to define forecast scenarios: as the baseline forecast was “all opportunities with a Rivet Score greater than 80,” we used higher thresholds to define additional “upside” and “stretch” scenarios. Furthermore, the same scoring methodology could be applied to the forecast from the Sales team, giving finance their own objective insight into what Sales was calling.
“Rivet is an unbiased view of what to expect versus what the sales team is telling us.”
— Director of Corporate FP&A, HashiCorp
Results and reflections
The Rivet Score concept resonated with the HashiCorp team. Even when presented to users who weren’t familiar with Rivet, the 0-100 scale was quickly understood across both individual opportunities and aggregated forecasts. The team particularly appreciated that the machine-generated scores gave them a means to compare forecasts from both Sales and Rivet in an apples-to-apples manner.
The success of the effort was validated in several ways. In addition to renewing their annual contract, the HashiCorp team expressed trust and satisfaction with the feature by asking if we could next a) extend the forecast’s horizon from 13 weeks to a full year, and b) include scenario-modeling levers like retention rate or size of sales staff.
