I recently purchased Alistar Croll & Benjamin Yoskovitz‘ book Lean Analytics. It’s a highly recommended read to basically anyone in the tech industry. One thing this book does very well is to describe other models and how the lean analytics approach relates to them (this is required to do in research but unfortunately a rare sight in more mainstream literature). The book focuses on various startup types and stages, and describe which metrics are relevant to whom at what time. This post comprises what I find to be the most useful insights.
Starting with the startup
The two authors note that business models and marketing models are seen as substitutes when they are not: “Freemium isn’t a business model – it’s a marketing tactic” (p. 67). It is thus important to make proper distinctions between acquisition channel, selling tactic, revenue source, product type, and delivery model. These five things can be compiled in many different ways, so you should use them as a flipbook to build the right combination in your startup, see Figure 7.1 below:
Building on Dave McClure’s AAARR metrics (or the so called pirate metrics), users can add values in 5 ways:
You should think about getting the users to do as many of these things as possible.
A startup stage model
- Empathy: Talk to many potential users and actual users in order to get feedback on the product and gain empathy for the users. At this early stage it’s critical to talk to them and get qualitative data. It’s recommended to talk to at least 15 different people. And as suggested by many researchers, it is recommended to measure actual behavior and not just rely on self-reported data. Instead of asking a potential customer if she would buy the product, ask for the money right away (this is actually what Kickstarter does). It’s also recommended to bring prototypes in order to attain better feedback.
- Stickiness: After truly understanding the user, the next step is to build a sticky product – i.e. one that engages users. The product needs to be so good that users keep coming back. This will require many experiments and iterations. All development activities must evolve around the One Metric That Matters (see below). A sticky product requires a set of features (but not too many). Seven questions can help prioritize between features: (a) Why will it make things better? (b) Can you measure the effects of the feature? (c) How long will the feature take to build? (d) Will the feature overcomplicate things? (e) how risky is the new feature? (f) how innovative is it? (g) Do users say they want it?
- Virality: When the product is sticky, it’s time to acquire new users. This virality engine has three drivers: (a) inherent virality – virality is an automatical byproduct of product usage. (b) Artificial virality – forced virality built into rewards meachnisms. (c) Word-of-mouth virality – existing users tell other users about it. It is crucial to understand that virality rarely happens by itself – it needs to be designed into the product.
- Revenue: Now it’s time to start examining if the product can be sufficiently monetized to build a sustainable business. The focus expands from building a product to building a business. In this phase, focus is on metrics such as customer lifetime value and customer acquisition costs.
- Scale: When your company is part of a larger ecosystem, you are probably ready to scale. This is a very hard part due to what Michael Porter calls the “hole in the middle” problem: When you are small you can compete on differentiated niche. When you are big you can compete on cost and margin. But being mid-sized is hard because you can’t focus, and you can’t dominate the ecosystem. In this phase it is important to create barriers of entry for potential competitors and thus establish an unfair advantage for your company. In this phase it is necessary to have more than one important metric that matters. A hierarchy of metrics is necessary in a full scale company, but it is important to remain focused on as few metrics as possible.
Lying vs data
As entrepreneurs, we all need small lies. That creates reality distortion fields, necessary to pursue new ideas. But our lies need to be counterbalanced by data. More importantly, lies don’t help us learn. We need data and analytics to do this. But don’t get lost in the data. “Analytics is about tracking the metrics that are critical to your business.” (p. 9). This means there are no universal set of right analytics. As Avinash have been saying for years: They are highly context dependent. This makes it harder to write about, but this book does a pretty good job by distinguishing between different types of startups (although this list isn’t exhaustive): e-commerce, SaaS, mobile apps, media sites, UGC sites, and two-sided market places.
In the early phases of the startup, you should define the One Metric That Matters (OMTM). This metric should be the one that is most crucial to find out if you are on the right way toward stardom: i.e. number of signups, retention etc. The OMTM will help guide all decisions and experiments in the startup as the ultimate goal is to improve this one metric. This metric will change over time. Good metrics share these four traits:
- Ratios or rates
- Change the way we behave
Growth Engines: More more more more more
In order to grow your brand, it is relevant to focus on one of Eric Ries’ three engines of growth:
- Sticky engine (retention). In order to find out when it’s time to start driving a revenue for the startup, it is crucial to measure and understand how sticky your product is. Engagement, number of visits and churn rates serve as good metrics for this. Stickiness is measured differently for different types of products/sites. For media and UGC sites, 17 minutes daily usage serves as the threshold for stickiness.
- Virality engine. Getting virality coefficient > 1.01 is the holy grail for viral growth. Having such coefficient will in theory create eternal growth as each new user will recruit more than one other user.
- Paid engine. Inbound and outbound acquisition channels that require direct or indirect paid involvement.
Or seen another way: Coca-Cola CMO Sergio Zyman has defined marketing as “selling more stuff to more people more often for more money more efficiently” (p. 64). This means business growth can come from five different types of more:
- More stuff
- More people
- More often
- More money (from each customer by upselling)
- More efficiently
Read more about lean analytics on the book website.