How to supercharge the product metrics of your Slack bot

Tweet about this on TwitterShare on LinkedInShare on FacebookEmail this to someone

A lot of startups and large companies are now exploring the world of chat bots, those smart assistants that can interact with you through a messaging application. Even though this new concept of interaction design represents a great opportunity for anyone that wants to create a product or a service, chatbots are so early that there are no standards or best practices out there to define the right product metrics. Since we’ve been experimenting with bots for a little more than 4 months, I wanted to share with you how you can apply some of your regular application metrics to your bot, and how you should think about new actions and events to track .

stencil.twitter-post

What are the questions you should always ask yourself to understand your digital product?

Depending on your goals, whether you want to diagnose a problem or take a decision about a new feature, you can break down the types of metrics in a variety of ways. For the sake of simplicity, I’m going to divide them into user acquisition, engagement, conversion/retention and money. If you are familiar with those metrics, feel free to jump on the next section.

User acquisition:

  • How many users do you have?
  • How and why do you have more users overtime?
  • How many active users do you have? What is an active user?
  • Where are your users coming from?
  • Which channels are the most effective in getting users?

Engagement:

  • How many users are using which features? How often?
  • How many users completed this workflow?

Conversion / Retention:

  • What is your conversion rate?
  • What does converting mean? Free to paid? Installing your app? Signed up?
  • How effectively does your product convert?
  • How often do your users come back?
  • How many of your users come back after x days?
  • What is your churn rate?

Money:

  • How does your product make money? How much does it make?
  • What is your average revenue per user?
  • What is your customer cost of acquisition?
  • What are your development costs? What are your hosting costs?
  • Do you have high sales or support cost?

If you want to dive deeper into product analytics, you can have a look at this presentation of Dave McClure from 500 Startups: Startup Metrics for pirates

What product metrics can you quickly start applying to your chatbot?

stencil.twitter-post(1)Now, using the same categories of user acquisition, engagement, conversion/retention and money, let’s see how we can translate that into the bot world. Except for the money category that I did not implement, I’ll mention metrics that I’ve found valuable while building an experimental Slackbot.

User acquisition:

  • How many users do you have?
  • How and why do you have more users overtime?
  • How many active users do you have? What is an active user?
    • Is an active user interacting with my bot every few days or every few hours?
    • Are you interested in team usage or individual usage?
  • Where are your users coming from?
    • Is it because other users are publicly referring to your bot?
    • Is it because of your onboarding messaging?
  • Which channels are the most effective in getting users?
    • Is it word of mouth? A user is mentioning your bot a in a group conversation and other users see it?
    • Is it an onboarding messaging? A bot is newly installed and broadcast a message in a channel?
    • Is it an install button on a website or email campaign?

Engagement:

  • How many users are using this feature or this one? How often?
  • How many users are using a specific keyword or expressing a specific intent? How often?
    • Is it because they have the “wow” moment of the first time interacting with a bot?
    • Are your users asking for help?
  • How many users completed this workflow?
  • When engaged in a conversation, how many users complete it?
  • Do your conversations have a positive/negative/neutral sentiment?
  • How long does it takes for your user to read/reply? This is great to understand the complexity of what you are asking (your messaging).

Conversion / Retention:

  • What is your conversion rate?
  • What does converting means? Free to paid? Installing your app? Signed up?
    • What does converting means? Bot installed in a team? Bot authenticated? First message received?
  • How effectively does your product convert?
    • If following a SaaS model, did a subscription get started?
    • If your bot is selling items, did a sale occur?
  • How often do your users come back?
  • How many of your users come back after x days?
    • What is your ratio of daily active users to monthly active users? Depending on what your bot is doing, this should be pretty high. Think about the friends you are texting every week vs the others: which ones would you qualify as active?
  • What is your churn rate?
    • Did your bot get muted?
    • Did your bot get deactivated?
    • Did your bot get uninstalled?
    • Is your bot too slow to respond?
    • Are your users forgetting about your bot over time? This will probably be the most important metric to track in terms of retention.

Now that you have a good understanding of what to measure you might ask yourself how to implement it… fear not! I’ll soon write a small article about how to get setup easily with Segment.io!

Tweet about this on TwitterShare on LinkedInShare on FacebookEmail this to someone

You should also consider reading

Antonin Lapiche Written by:

Antonin Lapiche is the rapid prototyping guy of the Product Design team. His passion for new technology makes him a self learner in order to experiment anything new out there. His focus is on using a set of new technologies to solve customer pain points. His philosophy: K.I.S.S and “a prototype is worth a thousand meetings”. Antonin is the founder of thesiliconvalleystable.com and proud father of Captain Feedback. He holds a Master in CS from EFREI Engineering School and a Master in Management from Audencia Nantes School of Management. You can engage with him (or just send him funny GIFs) on twitter @airlikidh2.