When developing your business, online or offline, it’s important to test your ideas and objectively evaluate the results. Offline you can do this with order tracking, Microsoft Excel, customer surveys, etc., but online we have more sophisticated tools. This post will provide you with a definitive way to track iterations and split testing for any web or mobile application using MixPanel.
Why MixPanel?
After reviewing Google Analytics and MixPanel (I did not review KISSmetrics) I found MixPanel to be much more flexible. Google Analytics only allowed 5 custom variables, and, it was confusing as to how/when they would be set and how long they would stick. MixPanel provided an easy way for me to track customers, iterations, split tests, and then view the data by cohort.
How does it work?
Users will be assigned an iteration and split test when they visit your site for the first time. When you up the iteration number, the split test value will be chosen at random for the new iteration. We’ll use themes to show them different interfaces, and, we’ll use the CakePHP MixPanel plugin to track them (note: I’m not the original author of the plugin but I added new features and support for CakePHP 1.3. Kudos to Noahm for the original.)
1. Install the plugin
# CakePHP 1.3 git submodule add git@github.com:dkullmann/CakePHP-Mixpanel.git plugins/mixpanel cd plugins/mixpanel git checkout 1.3 # CakePHP 2.1 git submodule add git@github.com:dkullmann/CakePHP-Mixpanel.git Plugins/Mixpanel
2. Setup your config
# Sample from my config file which is environments.php:
'Iteration.iteration' => 1,
'Iteration.split_tests' => array('A', 'B'),
'Iteration.split_notes' => 'Different webinar pops',
3. Add the MixPanel code to your controller
By adding the following code to our AppController, we’ll have enabled iteration tracking, and, tagging users by their ID and name in MixPanel if they are authenticated.
# app_controller or AppController depending on your version of CakePHP
public $components = array('Mixpanel.mixpanel');
public function beforeFilter() {
$this->_setupMixpanel();
$this->_setupIteration();
}
protected function _setupMixpanel() {
$user = $this->Auth->user();
if (!empty($user)) {
$this->Mixpanel->identify($user['User']['id']);
$this->Mixpanel->name_tag($user['User']['username']);
}
if (Configure::read('debug')) {
$this->Mixpanel->settings['properties']['test'] = 1;
} else {
$this->Mixpanel->settings['properties']['test'] = 0;
}
}
protected function _setupIteration() {
$iterationSettings = Configure::read('Iteration');
if (empty($iterationSettings)) {
return false;
}
$iterationCookie = $this->Cookie->read('Iteration');
if (empty($iterationCookie) || $iterationCookie['iteration'] < $iterationSettings['iteration']) {
$iterationCookie['iteration'] = $iterationSettings['iteration'];
$iterationCookie['split_test'] = $iterationSettings['split_tests'][ rand(0, count($iterationSettings['split_tests']) - 1) ];
$this->Cookie->write('Iteration', $iterationCookie);
}
$this->theme = 'SplitTest' . $iterationCookie['split_test'];
$this->Mixpanel->register($iterationCookie);
}
4. Add the Mixpanel Helper
# in your app_controller/AppController
public $helpers = array('Mixpanel.mixpanel');
# In your layout
<!-- Mixpanel should go before $scripts_for_layout so you can add mpq/mixpanel events in scripts --> <?php echo $this->Mixpanel->embed(); ?> <?php echo $scripts_for_layout; ?>
5. Results
After you’ve setup your funnels in MixPanel it’s very easy to see which split test is winning, check out my MixPanel funnel here.
In addition, whenever you up the “iteration” value in your config the visitors will get assigned a new split test.
6. Gotchas!
The only caveat here is that if you call mpq / mixpanel events directly from the JS API they wont have the “settings” values. If you want a value to persist throughout all mixpanel API calls make sure to set it with MixpaneLComponent::register();
Leave questions and comments below!