code.isdangero.us

Basic funnel tracking in Rails

February 13, 2011 - 2 Comments - elite command analytics ruby rails

This is the first in a series of articles I intend to publish about the technology, philosophy, and process behind my latest Rails project, Elite Command. Elite Command is a turn-based, multiplayer strategy game, and a lot of interesting approaches were taken in developing for its unique requirements.

Before I get too into the grittier aspects of the game, I’m going to cover something which applies to any web application which strives for commercial success: analytics.

Having designed and maintained various analytics platforms over the last three years, I’ve learned a thing or two about best practices and capable architectures for supporting the requirements of a robust analytics infrastructure. I’ve also learned a lot about the types of metrics which are most useful in a business sense. With this knowledge, adding analytics to Elite Command was a matter of choosing the 20% effort which would yield 80% of the useful results. That’s why the first analytics tool I baked in was funnel tracking.

Note that my code is using MongoDB with Mongoid for the ORM, but that this is all just as applicable to whatever data store you may be using.

What is this funnel of which you speak?

Funnels are incredibly useful for exactly one thing: tracking conversion rates of users through a flow. That’s why it’s called a funnel. In any given user flow, there is a chance for users to drop off, leaving fewer and fewer remaining users who actually reach the final step. A funnel allows you to pinpoint exactly where you’re losing users, and thus gives you some evidence on which to base targeted improvements with the goal of converting more users each step of the way.

This is extremely valuable when trying to build a user base. The first funnel you would want to measure is naturally the experience for brand-new users. From the moment they arrive at your landing page from a given source (such as an ad) to reaching the sign up page, to signing up, to engaging with your application, you want to know where users are giving up and leaving, and you need hard data to support your theories and measure improvement. This is especially important when users are coming from ads, as you want to make sure that you are maximizing the cost-effectiveness of paying for clicks or impressions.

The source and the cookie

The first thing you want is some kind of trigger for slapping the user with a unique identifier when they come from a trackable source. For this purpose, Elite Command looks for a query string such as ?src=cid_0_2 (which is indeed included in the link in this article) and saves that, along with a unique tracking ID, to the user’s cookies. The src string can be anything, but I recommend coming up with a standard format which makes sense for your needs. Mine is “[source name] _ [creative number] _ [release number]”, allowing me to measure the effectiveness of ad channels, individual ads, and the actual release of Elite Command. Here’s the code for creating these cookies:

The tid is the unique tracking ID, which comes from a class method on User. Here’s the code for that:

The UUID class comes from the UUID gem and generates a Universally-Unique Identifier, simply a string which is by all practical means completely unique within the scope of the application.

Now we have a cookie which uniquely identifies the user and tells us the source from which they came.

The User model

Cookies are great, but if a user signs up, we may want to know throughout the application’s lifetime which source they came from. We can use this to make funnels which stretch across multiple uses of the application. For example, we may want to create funnels which track whether users respond to emails sent by the app. To do this, it just makes sense to save our tid and src to the user record itself. This is pretty simple. First, just add the fields to your model (here using Mongoid):

Notice that we have the ensure_tid callback. This attaches a unique identifier to every user, even if they did not get tagged through one of our sources. This simply means we can track them within the same system.

Now every new User has a tid, and they also have a src if they came from one of our predefined sources. Now it’s time to put this all to use and start recording some data.

The UserAction model

Now we need a way to record when an action is taken, along with the tid and src, if available, of that user. Behold, the UserAction model:

That’s all there is to it! Now we can call UserAction#record whenever we want. You can pass in both a User instance (which can be nil) and a cookies object, and #record will do the right thing with it.

Funnels!

Now we can create a Funnel model. This model consists of a series of ordered steps (or UserAction#name strings) and a name. We also need a way to calculate the numbers of unique tids recorded at each step. Without further ado, here it is:

There you have it! I won’t go into details on creating the admin UI for creating and viewing these funnels, but it should be pretty straightforward. Funnel#step_results(src) returns an array of pairs, each pair consisting of the name of the step and the number of unique tids remaining at that step. There are many ways to display this data. I settled for a simple column of numbers and percentages with CSS-generated bars representing the portion of users remaining at each step. Whatever lets you see the data.

What next?

Using this data, I’ve made several ad runs using Facebook Ads and tracked the effectiveness of the new user experience. The goal is to get users not only to sign up but to become active users of the game. With each ad run, I can see exactly where the flow could use the most improvement. This is where I get creative, imagining what might entice the users to keep going forward. I implement and launch the improvements, do another ad run, and then look at the funnel numbers for that source. From this, I can determine whether the improvements worked or not and continue iterating toward an optimal new user experience.

There is certainly room for improvement. For one thing, it would be nice to track source by HTTP referrer for untracked users coming from an unknown source. It would also be nice to calculate the funnel in a more strict manner, ensuring not only unique tids at each step, but also enforcing the order of the steps. And, as I’m sure goes without saying, there is plenty of room to improve on performance; the funnel numbers should be calculated using map-reduce, and there are some missing indices as well. But this works for the immediate need, and I’ve been able to use this system to improve retention of new users considerably.

Let me know what you think and if you have any ideas for improving the system. Also, be sure to give Elite Command a whirl and tell me what you think, and don’t forget to invite your friends! I hope this article proves useful to someone else’s project.

self.steps.map do |step|
   q = UserAction.where(conds.merge(name: step))

if last_user_tids.nil?
    last_user_tids = q.distinct(:user_tid)
  else
    last_user_tids = q.where(:user_tid.in => last_user_tids).distinct(:user_tid)
  end
end

[step, last_user_tids.size]

[
  ['step1', 54],
  ['step2', 32],
  ['step3', 27],
  ['step4', 24]
]