Welcome back to my weekly newsletter. This is going to be a series of articles covering how we came up with the idea of Calm Sleep, scaled to over a million users in less than a year, and all that with less than $1,000 worth of total investment. 🌱

I’m Akshay Pruthi, an entrepreneur who loves to build products from the ground up. Over the past 6 years, I’ve built multiple products from scratch and scaled them to millions of users. This is my attempt to share our most important lessons from building one of those apps - Alora (Android) and Alora (iOS) 🤗

Every week I will publish an article about a different challenge we faced while building Calm Sleep.

Last week, we discussed how we coped with the inevitable element of any start-up: Its highs and lows and what’s a value metric.

What’s in this week’s read

1. The results of launching a playlist feature for new users.

2. The instrument we launched to get user insights and collect feedback whenever we wanted.

3. A small yet important change led to an increase in our notification open rate.

4. My 2 cents on Sahil Lavingia’s tweet - ‘80% of the product is onboarding!’

5. As a product lead of the app, how do I categorize my users into different buckets to meet the demands of these different cohorts?

In one of my previous articles, I had mentioned why we launched a playlist feature and how we hoped this playlist feature will help convert more users.

🤭And here is what we found:

1. Only 30% of the total users attempted the playlist feature.

2. After the playlist feature launch, the payment conversion rate from the usual 1.7% dropped to 1.4%.

3. Sound completion rates for people using the playlist increased by 15%.

4. A very small percentage of people were using the playlist for all 7 days. Most drop-offs occurred after the 1st day of usage.

Introducing the playlist feature increased the sound completion rates. But it came at the cost of a decrease in payment conversion. And this made it clear that the playlist feature wasn’t working out for us. The primary reason we anticipated was that people didn’t want to commit to a 7-day timeline. Additionally, we assumed that the feature wasn’t doing them any good as they weren’t able to explore which sound works best for them.

User Feedback: Why should you be obsessed about it?

If you are a regular reader of my newsletter, you might have realized how much I dig user feedback. If I don’t gather enough user feedback, be it quantitative or qualitative, I feel that I am shooting things in the air without a strong thesis around it.

You might have heard or read this often:

Customers don’t know what they want or No good comes out of speaking to them.

But I believe that just like everything else, there are two sides of this coin too...wherein some preach the aforesaid statement, others argue that you should build what your customers have been asking for.

From my start-up exposure, I have gauged that, especially in the nascent state, the latter ideology is prevalent where a few customer reviews or feedback come & product leaders/founders incorporate it in the next app update.

If you think about it, what customers really know is the problem they are going through. And they are looking for tools to help resolve their problems. If product founders/managers really listen to them, the answer of ‘what can we do better or ‘what can we launch next’ lies in it. Don’t believe me? Look at this interesting video explaining why people bought milkshakes.

First and foremost, if you aren’t aware of the JTBD framework, you should thoroughly read this article. This framework can help you better align with what you're building V/S what your users really want.

Let’s jump to some of the trap questions you can get to while understanding the users’ problems.

1. You got one idea that’s constantly circling in your head. You reach out to your users and start asking their opinion on the idea seeking validation. Result: This often results in useless feedback as you are not asking about the user's problem but trying to validate what you think is the solution to their problem.

2. Rushing into building what you perceive is the resolution only because some customers agree with your idea. Result: You don’t solve the problem for the majority of users but possibly for the minority. That too can be a false positive.

You will find yourself in this trap multiple times while building a product. I can be so certain because back then I sure did. So, I developed a framework that helps me make better decisions when it comes to user feedback minus getting trapped. Stay clear of these 3 simple yet sweeping questions:

1. Was it your idea or their problem? If you find yourself asking: A. What do you think about this idea? B. Do you face problems doing this often? (this = activity you want to validate as your idea) Stop. Rather ask: A. Can you walk me through how you do [Activity]? B. Why do you do this [Activity] often? C. Tell me about the last time you did [Activity]. How was the experience?
   

2. Currently, How are they solving their problem? The best proof of a genuine problem is that users have invested energy/cash to fix it. Maybe they have made their own workarounds, but are still troubled. Ask: A. How do you currently solve this? [Problem] B. Have you looked for any solutions to solve this? Did you pay for anything?
   

3. Lastly, ask yourself, do you have enough data points backing the problem?

I try to look for multiple data points. For example,

1. Interviewed 100 users and over 70% are facing this [Problem]. [This] is how they currently solve it and [This much] they currently pay for it.

2. The analytics tells us that X% of users opted for this option over other options.

3. How many users have mentioned this problem statement in the reviews and feedback that we receive?

At Calm Sleep, we launched a Poll feature so that we can have a continuous stream of feedback. We can launch this anytime we want, to any cohort of users we want, controlled from our backend systems.

Jumping to another interesting data point we observed was the notification pop-up delivery. Every night, an a sleep reminder pop-up notification goes to our users. Currently, we had set the timing of that notification as 9:30 PM.

From data, we observed the majority of our users sleep by 11 PM. So, they used to see the pop-up notification, jump on the app and straight away exit the app without consuming any sounds.

We decided to do a small change by moving the notification time to 10:30 PM. We did 10:30 PM, 30 minutes before the time they actually hit the bed because we want users to develop sleep hygiene.

Result:

The notification time change improved App Open to SoundPlayStart Conversion: From 71% ----> 75%😀

80% of the product is onboarding!

Imagine, you went to buy a refrigerator but there is no human assistance to guide you on how the refrigerator works. Scale this use case to any new activity in the world. Whenever we try out something new, we need some assistance. If the assistance is missing, we end up getting frustrated and move on to the next thing that turns up.

When the users come to your app for the first time, a good percentage of them are constantly asking this question in their head:

Does this app help me solve my problem?

Unfortunately, you get limited time to convey to your users that it DOES solve their problem & how!

If you aren’t able to, the probability of them coming to the app reduces by a great percentage.

If you look at the data below, 25% of mobile apps are only used once and this hasn’t changed much for the past ten years.

This is why, while designing the entire onboarding experience, you need to constantly ask this question to yourself:

Is your app assisting your users to help them find the solution to THEIR problem?

And if the answer is NO, make amends.

Summing up, onboarding is the heart of your product. Period.

----------

Let’s try to understand how one creates and recreates onboarding experiences to keep improving user retention.

If you think about the journey of your users on the app, you will be able to broadly categorize them into “new users” and “old users”

The onboarding experience is the key for new users, while for old users, nourishing the value proposition is the key. As a product person, you need to think about the two sets differently.

For now, I will cover what I think about onboarding and my thought process behind it. In my upcoming articles, I will talk more about the “old user” journey and how to optimize that to reach better retention.

When you think of onboarding, you need to think of a *Jargon Alert* Activation Metric.

The activation metric is a measure of how many actions a user takes to receive value out of a product. Essentially, it is what a user has to do before they derive the value from the product.

Having an activation metric grants you the ability to guide the users to reach the value proposition faster. This has a direct impact on user retention and in turn the engagement on the app.

Onboarding is the guiding tool for the user to reach the activation metric faster. Think of the same example I gave before.

When you enter a shop to buy a refrigerator, there’s someone to receive you, understand your problem(the kind of refrigerator you want to buy), and guide you to the right options. If the recommended refrigerator meets your criteria, you will purchase it.

Now scenario 2, imagine, there’s no one to guide you. You will end up looking around for a bit and leave because you couldn’t derive the value out of the service.

Similarly, when the user enters your app, you need to reinforce the value proposition with communication and guide them to reach the activation metric.

Personally, I have realized that when you are handling multiple things at the same time, you often forget to recreate experiences for your new users. As a habit, I try to relook at my onboarding design every 2 months. This helps me refresh my thoughts and look at them from a different lens.

Every time I go over the onboarding experience, I ensure that it answers one and only one question - “Is my app able to guide the user to reach the value proposition before they exit the app?”

In my upcoming articles, I will talk about how I redesigned my onboarding and what were the results 🙃 Till then, keep Calm :)

If you enjoy reading my articles, Buy me a coffee here