Ditching the vanity

I started a new product recently, actually two products. The whole process of coming up with ideas and starting the implementation is fascinating and familiar to every person who has launched something new.

Each product after launch is going through the iteration cycle. I'm sure you are familiar with that cycle of feedback collection, iteration and improvement.

Getting insights from the user and understanding their usability path is crucial for newly launched products. The speed of responding to the changes and user requests is the main differentiator between excellent execution and goog execution. So here's a question: what is the similarity between a startup, small business and big corporation? Sure these organizations are very different. The big difference between them has started this startup culture. However, they have one significant similarity. They are tracking their metrics vigorously.

Let's back up here a bit. What is the metric?

Metrics

I don't like using the strict dictionary definition, so here's my definition of metrics:

Metric is a number that is measuring the state of the product in achieving its initial goal.

The words that are new here are "achieving its initial goal." Metrics are mainly used to track the consequences of the decision; new features usabilities the growth of the company at a given period. As important as that is, this approach to measuring the product is pretty detached from the core: solving a problem for a specific group of users. Don't get me wrong here. It's good to have many metrics for different reasons, to track the sales, to see the return on investment for a specific feature. Still, there should be one metric that is responsible for validating the completeness of the initial goal.

In other words, when the founders can say that they fulfilled the mission of their product? How the vision came to life? Is it 10 emails from customers that say that their product changed their life or highest rank on G2Crowd, maybe getting on the first page of ProductHunt? It is very sentimental; however, let's take a more unbiased approach here. Number don't lie (well, at least they try to ūüôÉin the right hands).

All this introduction was done to show you the difference between the vanity metrics and real actionable metrics.

Vanity metrics

Vanity... yup, we all act on it sometimes.
Sometimes products need time to reach their core audience and stick to their routine, and tracking actionable metrics during that period is very crucial. But sometimes we go after some other things like the number of active users.

‚Äć
Here are some vanity metrics that don't need the attention they get.

Verious user counts

These metrics are the number of active users, subscriber count, number of downloads, even the number of unique users, number of users on the waiting list.

Let's see how these metrics are holding the test.

I had 1000 visitors last week

Well, great, what does it tell you? Does this number help you to achieve some goal? What are you doing with this kind of attention? How do you use this spotlight in favour of your product? These are not rhetorical questions. They can have zillions of answers depending on the business model and product type, and the main thing is the number of visitors doesn't matter, as long as you don't know what to do with them. They can be scraping bots, and you probably would never know are the actual users or not.

Metrics that say nothing

- Hey I have an idea, instead of tracking the things that our users are doing on our platform, let's track how much time they are spending on the platform.

These metrics are the session duration, bounce rate, number of pages per session.

The common thing about this section of vanity metrics is that these metrics are not worth tracking. For example, why session duration is a vanity metric.

This could've been funny, but it's not. Average session duration is a faulty metric, as it is highly dependent on your analytics tool. So each analytics tool is tracking the session duration (and the other vanity metrics too) in their way.

I've been a long user of Google analytics, and for a very long time, I saw that I had a session duration of 1 second, or even 0 seconds, which is pretty strange. Depending on a faulty tool to track a vital metric is not the best idea, especially taking into account that even if it was correctly tracked, it still is not saying anything new.

If you have a 10min session duration, is it because your content/product is fantastic, or is it because the user spent much time searching for something on your website? The same is true for pages per session metric.

Actionable metrics

Based on the naming, it's pretty easy to understand what are the differences between actionable metrics and vanity metrics. The first type is dealing with the ego of the founder. The second is providing insight on the product performance and the alignment to its core purpose. Although there are many articles on how to determine the actionable metrics and actual lists of metrics you need to follow. I decided to do a different kind of separation.

Money

Show me the money, baby.

What can be better than getting the validation of steady revenue? Having stable monthly recurring revenue, that leads to beautiful annual revenue is the only traction metric that should matter. Tracking revenue sources:

- the best/worst performing pricing plans,

- the best/worst performing coupon code,

- customers lifetime value,

- the returning paying customers,

- how many freebie customers are converting to paying customers at a given period?

Unit economics

Unit economics is often overlooked practice for tracking the actual value of the product. Unit economics is helping to measure how much is your customer acquisition cost (the costs you make to onboard one paying customer) is higher than the lifetime value of the paying customer. The golden rule that is often referred to on the internet is that your lifetime value should be 3 times higher than the customer acquisition cost for your business to be viable.
You can find a lot of sophisticated formulas for calculating these both metrics. Still, the simple truth is that customer acquisition cost is simply the sum of all your costs (natural right?). And the lifetime value is simply the sum that the customer is paying to you for a given time.

Notice that I'm putting the emphasis on the paying customers.

Business models are changing and evolving. Marketplaces are becoming communities, and SaaS products are reintroducing lifetime pricing plans and the most valuable information right now is the niche experience of creator. However, the unchangeable thing is that the purpose of business is to get profit to the founder and bringing ease to the customers (hopefully to create some value). So paying attention to the paying customers is crucial to see what your users are ready to pay.

Non-money

This metrics are the ones that can and will lead to the money question, but they are not entierly related to money. They set the tone and foreshadow the the money metric. What are non money actionable metrics?

Replies

For example, for my newsletter, this is the metric that matters. How many people reply to me after I send the new issue. How many of them answer when to ask them about something. For non-newsletter products, it can be people answering your introduction emails when you, as a CEO, introduce your product during the onboarding process, when customers book the introduction call in your onboarding message. In other words, how engaged are they with your content and product itself?

Onboarding completeness

Do you have a dedicated onboarding process for you new customers? If no, than you need one (I know I'm guilty of this). Having a smooth and easy to go through introduction is crucial to kickoff the product on the right foot. What can you track about this:

- How many new customers went through it,

- On what step most of the customers drop out,

- What Call-to-actions the customers are using during onboarding process (e.g. if you ask to setup payment options, how many of new customers are actually doing it).

Connections made

If you are creating a marketplace, two-sided N-sided no matter how many, the primary and only goal for the product is to connect. Let's look at a booking (anything booking) platform as an example, and see what can you track for it. The best metrics would be:

- the number of bookings,

- the number of cancelled bookings,

- the number of closed bookings.

All this metrics are simply showing is how much the platform is used for it's purpose.

In the startup world, the buzzword right now is "traction" many founders are running after the proof of traction. The proof that users are engaging with your product. The proof that you are the horse to bet on, the proof that this and only this product is worth the investment. Vanity metrics are good at creating the buzz but are deadly for reaching the real goals of the product. Various user counts and session durations are the deadly sins for getting that sweet, sweet traction. They sound impressive, but at the same time, they say nothing.

Metric first and foremost should be tied to the purpose of the product and tracking that metrics will show the traction much better than number of downloads per week.

Good luck and track the right metrics.