Metrics Are Unit Tests for Your Business

It recently occurred to me that the unit tests I wrote as a software engineer are very similar to the company metrics most startup teams track. Because metrics are so important for startups, I thought it might be useful to briefly describe unit testing, draw an analogy between unit testing and metrics, and discuss how some of the advantages of unit tests also apply to the world of metrics.

If you're not familiar with the concept, a unit test is a small piece of code which verifies that another piece of code is correct and meets all expectations. For example, if you're developing an address book website, one portion of your program might be a function called isPhoneNumberValid. This function takes a user-entered phone number and returns GOOD if the phone number is valid and BAD if it's invalid. A unit test for isPhoneNumberValid might check that "310-555-1234" and "1-800-TMOBILE" are GOOD phone numbers while "310-555" and "TMOBILE" are BAD. If isPhoneNumberValid returns an unexpected result (e.g. it says "310-555" is GOOD), the unit test fails and the developer is notified.

The obvious purpose of unit tests is to ensure that your code does what you think it does. Checking if a phone number is valid requires considering many special cases, and it's quite possible that your first draft of the code contains a bug -- or several bugs. Some engineers stop here: they write a few unit tests to make sure their code works, then they go off to write the next piece of code. However, unit tests have several important benefits in addition to checking correctness:

In many ways, metrics are for companies what unit tests are for software. Companies keep track of all kinds of data: number of active users, average amount per purchase, average number of purchases per user, hours of website downtime per year, and so on. On the surface, these numbers are simple indicators of company health. You have 10 million active users? Great! The average purchase amount is only $4.37? Uh-oh. When you look beyond the surface, however, metrics also offer the secondary benefits of unit tests. Specifically:

Finally, in the same way that unit tests can reveal when new bugs are introduced in a piece of code, metrics can reveal when you make a misstep. If you are measuring the number of people who visit your site every week, then a sudden 15% drop indicates that one of your recent features is probably broken or not having its intended effect. 

Metrics have many benefits beyond assessing the current health of a company, and as a results it's hard to overstate their value. Figure out what's important to your business, design a handful of metrics that correspond to your priorities, and then use those metrics to run focused experiments that move you in the right direction. Happy measuring!

Tags: Operating Tips

Share this post


Subscribe by Email
Copyright © 2013-2017 Leo Polovets