Lean Analytics

Lean Analytics is the latest addition to the Lean Series. The book has been a year in the making, and authors Ben Yoskovitz and Alistair Croll—themselves successful founders with several exits under their belts—spent much of that time speaking with founders, investors, and analysts to understand a really basic, but seldom-asked, question: What's normal? As it turns out, normal is a hard question. Normal depends on what kind of business you're in, and what stage of that business you're at. If you're working on the Sticky Engine of Growth, you're focused on very different metrics from those that you care about in the Viral Engine of Growth. Similarly, a two-sided marketplace cares about different things from a traditional e-commerce product.

Start with metrics in mind

To help with this, the book looks at dozens of metrics—such as churn, customer lifetime value, viral coefficient, acquisition cost, uptime, and engagement—and suggests where that metric should be before you can move on to the next stage of your business. Here's what they have to say about churn rates in SaaS businesses:

The best SaaS sites or applications usually have churn ranging from 1.5% to 3% a month. For other sites, it’ll vary depending on how you define “disengaged.” Mark MacLeod, Chief Corporate Development Officer at Freshbooks, says that you need to get below a 5% monthly churn rate before you know you’ve got a business that’s ready to scale. Remember, though, that if you’re surprising your subscribers in a bad way (i.e. billing them for something they didn’t know they’d ordered) then churn will spike during your first billing period, sometimes to 50%, so you should factor this into your calculations.

Matrix Partners' David Skok agrees with the 5% churn threshold, but only for early stage companies, and says that you have to see a clear path to getting churn below 2% if you want to scale significantly.

“In the early days of a SaaS business, churn really doesn’t matter that much. Let’s say you lose 3% of your customers every month. When you only have a hundred customers, losing three of them is not that terrible. You can easily go and find another three to replace them. However as your business grows in size, the problem becomes different. Imagine that you have become really big, and now have a million customers. Three percent churn means that you are losing 30,000 customers every month. That turns out to be a much harder number to replace.”

Not all SaaS companies are the same, of course. Certain products or services are very sticky, in part because of the lock-in users experience. Photo upload sites and online backup services, for example, are hard to leave—because there’s a lot of data in place. So churn for those product categories may be lower. On the other hand, in an industry with relatively low switching costs, churn will be substantially higher.

Social sites may have some tricks at their disposal, too. If users try to leave Facebook, they’re reminded that some of their close friends will miss them—along with pictures of those friends. This is an example of how an emotional tweak was supported later by the data: once implemented, this last-ditch guilt trip reduced deactivations by 7%, which at the time meant millions of users stayed on Facebook.

If you’re going to offer users an incentive to stick around—such as a free month or an upgrade to a new phone—you’ll have to weigh the cost of doing so against the cost of acquiring another customer. Of course, if word gets out that you’re incenting disgruntled users to stick around then many customers may threaten to leave just to receive the discount. And getting the word out is what the Internet is for.

Bottom line: Try to get down to 5% churn a month before looking at other things to optimize. If churn is higher than that, chances are you’re not sticky enough. If you can get churn to around 2.5% you’re doing exceptionally well.

Knowing what normal looks like is essential. If you don't know what normal is, you can't tell if your efforts are paying off. You don't know if you're at a point of diminishing returns and should focus on something else.

Finding your One Metric That Matters

But it's not enough just to know "normal". It's also vital to know what the most important metric is to your business right now. That's because one of the most precious resources a startup has is focus, and spreading your attention across dozens of metrics gets in the way of learning. Ben and Alistair call this focusing on the One Metric That Matters (OMTM), and it's a core theme of the book. The following table shows some examples of an OMTM based on stage and model:

Many Mores

The book goes into detail about how founders can move the needle a different stages of growth. For example, in the revenue stage—where the company is busy growing revenues and pouring a percentage of them back into user acquisition—there are several places where analytics and iteration can help increase revenues.

Sergio Zyman, Coca-Cola’s CMO, once said that marketing is about selling more stuff to more people more often for more money more efficiently.

Business growth comes from improving one of these five “knobs”:

More stuff means adding products or services, preferably those you know your customers want so you don’t waste time building things they won’t use or buy. For intrapreneurs, this means applying Lean methods to new product development, rather than starting an entirely new company.

More people means adding users, ideally through virality or word of mouth, but also through paid advertising. The best way to add users is when it’s an integral part of product use—such as Dropbox, Hotmail , or a project management tool that invites outside users—since this happens automatically and implies an endorsement from the inviting user.

More often means stickiness (so people come back), reduced churn (so they don’t leave) and repeated use (so they use it more frequently). Early on, stickiness tends to be a key knob on which to focus, because until your core early adopters find your product superb, it’s unlikely you can achieve good viral marketing.

More money means upselling and maximizing the price users will pay, or the revenue from ad clicks, or the amount of content they create, or the number of in-game purchases they make.

More efficiently means reducing the cost of delivering and supporting your service, but also lowering the cost of customer acquisition by doing less paid advertising and more word of mouth.

In the Revenue Stage, you need to figure out which “more” increases your revenues per engaged customer the most:

If you’re dependent on physical, per-transaction costs (like direct sales, or shipping products to a buyer, or signing up merchants) then more efficiently will figure prominently on either the supply or demand side of your business model.

If you’ve found a high viral coefficient, then more people makes sense, because you’ve got a strong force multiplier added to every dollar you pour into customer acquisition.

If you’ve got a loyal, returning set of customers who buy from you every time, then more often makes sense, and you’re going to emphasize getting them to come back more frequently.

If you’ve got a one-time, big-ticket transaction, then more money will help a lot, because you’ve only got one chance to extract revenue from the customer and need to leave as little money as possible on the table.

If you’re a subscription model, and you’re fighting churn, then upselling customers to higher-capacity packages with broader features is your best way of growing existing revenues, so you’ll spend a lot of time on more stuff.

Any founder knows that it's really, really hard work to identify the riskiest part of the business, then find the simplest way to validate or repudiate your business model with that risk in mind. To do this, you need metrics, and a mindset that turns everything into a learning an experience. As I said in the foreword to the book,

Ben and Alistair have done the incredibly hard work of surveying the best thinking on the metrics and analytics, gathering in-depth examples, and breaking new ground in presenting their own frameworks for figuring out what metrics matter, and when. Their work collecting industry-wide benchmarks to use for a variety of key metrics is worth the price of admission all by itself.

While Lean Analytics applies to startups, it's also valuable for companies selling to business customers, and to intrapreneurs within large organizations trying to change the status quo. That's because the cycle of learning and measurement is universal. But unlike startups, Intrapreneurs have to work within existing systems and ultimate hand over their successful new products to the host organization, which presents some unique challenges.

If you work in a company of any significant size, you owe your org chart to an enterprising General Superintendent of the railroad era named Daniel C. McCallum. In the 1850s, railroads were a booming business. Unfortunately for investors, they didn’t scale well. Small railroads turned a profit; big ones didn’t.

McCallum noticed this, and divided his railroad into smaller sections, each run by subordinates who reported back a standard set of information he defined. McCallum’s line—as well as other lines that copied this approach—thrived. McCallum’s model, inspired by his time as a soldier and the regimented hierarchies he had learned there, was then applied to other fields.

McCallum was the first management scientist, introducing controls, structure, and regulations in order to reduce risk and increase predictability at scale. Companies like Google and Apple know this, creating their own advanced research groups such as the Google X Lab.

Intrapreneurs aren’t trying to solve for safety and predictability. Their job is to take risks, and to uncover the non-obvious and the unpredictable. If you’re trying to provoke change and disrupt the status quo, then the organizations McCallum introduced are your kryptonite. You need to shield yourself, just as the engineers within the Skunk Works did decades ago. But you also need to coexist with the organization, because unlike an independent startup, the fruits of your labors must integrate with your host company.

What you make may cannibalize the existing business, or threaten employees’ jobs. People will behave irrationally. When Marc Andreesen famously said “software eats everything,” one of the things in its diet was jobs. When a software company introduces a SaaS version of their application, salespeople who make a living selling enterprise licenses get angry.

Inertia is real. If you’re asking people to change how they work, you’ll need to give them reason to do so. Consider an Apple store: there’s no central cash register, and they’ll e-mail you a receipt. It takes a fraction of the time to purchase something; but convincing an existing retailer to change to this model will require retraining and modifying store layout.

If you do your job well, you’ll disrupt the ecosystem. A traditional music label has relationships with distributors and stores. That made it hard to move into online music distribution, leaving the opportunity open for online retailers.

Your innovation will live or die in the hands of others. While it’s easy to be myopic about your work—and disdainful of what the rest of the company is doing—the two are one and the same. “When problems crop up it is easy to see things from your own point of view,” says Richard Templar in The Rules Of Work, “Once you make the leap to corporate speak it gets easier to stop doing this and start seeing problems from the company’s point of view.”

In their book Confronting Reality, Larry Bossidy and Ram Charan list the six habits of highly unrealistic leaders: Filtered information; selective hearing; wishful thinking; fear; emotional overinvestment; and unrealistic expectations from capital markets.

Intrapreneurs need the opposite attributes to thrive—and many of those attributes are driven by data and iteration. You need access to the real information, and you need to go where the data takes you, avoiding confirmation bias. You need to set aside your own assumptions and preconceived notions, and you need to combine high standards with low expectations.

Data Yields Insights

In many of the book's case studies—there are thirty in all—analysis of the data yields an insight that unlocks product-market fit, such as the discovery of an adjacent market or the effectiveness of a different pricing model. Consider what happened to ClearFit, a SaaS recruiting software company, as described in the book.

ClearFit is a SaaS provider of recruitment software aimed at helping small businesses find job candidates and predict their success. When they started, founders Ben Baldwin and Jamie Schneiderman offered a $99/month (per job posting) package. “We kept hearing over and over that monthly subscriptions was the key to growing a successful SaaS business,” says Ben. “So that’s the direction we took, but it didn’t work as planned.”

Two things confused ClearFit’s customers: the price point and the monthly subscription. Ben and Jamie wanted to price ClearFit below what customers paid for job boards (typically $300+ per job posting), but customers were so used to that price point they were skeptical of ClearFit’s value at $99/month.

Ben says, “We don’t compete with job boards, we partner with them, but at the time it seemed reasonable to have a lower price point to garner attention.” Customers didn’t understand why they would pay a subscription fee for something that they would most likely use sporadically. “When a company needs to hire, they want to do it fast and they’re willing to invest at that moment in time,” says Ben. “Our customers are too small to have dedicated HR staff or recruiters that are constantly looking for talent, and their hiring needs go up and down frequently."

Ben and Jamie decided to abandon their low monthly subscription and switch to a model that their customers understood: a per job fee. ClearFit launched its new price point at $350 for a single job (for 30 days) and almost immediately saw three times the sales. The increase in volume and the higher price point improved revenue 10x.

“When we increased the price,” Ben says, “it was an important signal to our customers. They understood the model and could more easily compare the value against other solutions they use. Even though what we do is different than a job board, we wanted our customers to feel comfortable with purchasing from us, and we wanted to fit into how they budget for recruiting.”

In ClearFit’s case, innovating on the business model didn’t make sense. Ben says, “People don’t do subscriptions for haircuts, hamburgers and hiring. You have to understand your customer, who they are, how and why they buy, and how they value your product or service.”

ClearFit’s switch to a per job posting model may go against the popular grain of subscription-based SaaS businesses, but the company continues to see great success with 30% month-over-month revenue growth.

ClearFit initially focused on a subscription model for revenue, but customers misinterpreted their low pricing as a sign of a weak offering. They switched to a paid listing model, and tripled sales while improving revenue tenfold. Ultimately, the problem wasn’t the business model, it was the pricing and the messages it sent to prospects.

The company learned that just because SaaS is a recurring service doesn’t mean it needs to be priced that way. If your product is ephemeral—like a transient job posting—it might be better to offer more transactional pricing. Pricing is a tricky beast. You need to test different price points qualitatively (by getting feedback from customers) and quantitatively. Don’t assume a low price is the answer; customers might not attribute enough value to your offering.

This prescriptive, data-informed approach shows founders what to do, when. While every business is unique, the book provides a starting point for building your own model, creating your own metrics, and deciding when it's time to grow, when it's time to pivot, and when it's time to quit.

Ask Good Questions

As Ben and Alistair say in the conclusion:

"There’s never been a better time to know your market. Your customers leave a trail of digital breadcrumbs with every click, tweet, vote, like, share, check-in, and purchase, from the first time they hear about you until the day they leave you forever, whether they’re online or off. If you know how to collect those breadcrumbs, you have unprecedented insight into their needs, their quirks, and their lives.

This insight is forever changing what it means to be a business leader. Once, a leader convinced others to act in the absence of information. But today, there’s simply too much information available. We don’t need to guess—we need to know where to focus. We need a disciplined approach to growth that identifies, quantifies, and overcomes risk every step of the way. Today’s leader doesn’t have all the answers. Instead, today’s leader knows what questions to ask.

Go and ask good questions."

You can order Lean Analytics today.