It is ironic how things in life come back to you in a spiral manner sometimes. The same happened to me in relation to the KPI topic. I have explored it briefly in 2007, and today I am able to share a few more good practices that any emarketer will find effective.
So what are the top 3 things you need to know, or rather do to make the most of your KPIs?
1) Clearly distinguish a KPI from all other metrics you collect. Many of times, it gets confusing with all the data we pull from a web analytics tool, to what focus on, because every count brings out a unique information piece. Simultaneously, the raw data is much easier to grasp rather than the one hidden through a formula. Thus, any metric can become a part or a standalone KPI depending on your objective. You probably would think now – “Well, that’s not making it any easier to understand!”. Which is exactly the same thought that tortured me for about a week till I discovered work by Steve Jackson so generously shared with all of us in his Cult of Analytics book. I felt like Amerigo Vespucci on that day – cause I cracked (found) the definition of a KPI. His 4 attributes on p. 50 served me very well to progress further while developing new and refining the old. “Every KPI should have the following attributes:
1) The metric has a timescale associated to it (is reported weekly, monthly, quarterly).
2) The metric has a benchmark.
3) The metric has a reason to be reported to the actor.
4) The metric has an associated action that can rectify the situation.
…Most of the times it is a ratio.” Now, that makes it much easier to set the KPI definition in stone. And though, you can still show your site performance from the user interaction through visits, clickthroughs, add-to-cart clicks and ultimately all the way down to the placed orders, adding the ratios contributes so much color to the overall picture. If both are displayed separately – it creates more unnecessary questions, while placed together (general sequential metrics and ratios) allows for focusing on the right piece of data (a ratio KPI) while the raw data next to it, validates its accuracy.
2) Slice and dice your KPI – aka segment it by campaigns, traffic source, etc. This advice is not new and has been declared many times to anyone who faced data analysis. At the same time, it is also not very much followed. Similar to the exercise prescription in addition to the diet, data segmentation gets a lower follow through. But, if you do it once, you will never take data any other way! By segmenting, you are able to find trends since you put data in a context.
3) Get to know your KPIs better on all levels to learn what is normal and what is not in terms of their behavior. In this respect, you view your KPIs as predictable subjects. In the same manner as criminal investigators or psychologists observe people and get to know what behavior is normal for a given individual and what is out of the line, you can practice the same with your KPIs to get the most out of your reporting. Avinash Kaushik has a great insight on how to do just that – use the statistical tools of upper and lower controls to define the normal playfield for your data.
The only question that I am yet to resolve is – what are the best practices of calculating those controls for various KPIs and metrics? Some suggest to use 3 standard deviations to calculate controls, some make sense to use just 1 (as in Visits per Page as a Lower Limit possible). If I use 3, I will be expanding a range of behavior too wide if my data fluctuates frequently. If I use 1, it creates a too narrow field. I hope to get the answer very soon and for now I plan to watch all three scenarios.