Ed Colet
July 30, 2002

Comparing "apples vs. oranges" to "apples vs. apples"

If you do a particular race more than once, it's natural to compare your race times from one year to another. How did your Ironman USA 2002 time compare to 2001, or perhaps to it's inaugural year, 1999? Although a particular race route and distance may remain the same from year to year, all is not exactly the same each year. The weather is never exactly the same, the competitors are never exactly the same, and even your own fitness level varies from year to year. Despite these differences, it seems reasonable to compare times. But is an 11:52 in one year always better than a 12:34 of another year? How can raw times be meaningfully compared? And what if you haven't done the same race twice? There still is a natural urge to compare your race times. How about your Ironman USA time vs Ironman Austria? Or Ironman Hawaii compared to another Ironman? In these comparisons, the courses, the weather, and the competitors are even less likely to be the same relative to each other. Do things get impossibly complicated if you want to compare races of different distances such as an olympic distance race vs a half-Ironman? What could you make of your race times at Westchester versus Eagleman for example? Well, it's about time (pun intended) that we make use of a way to make such comparisons more meaningful.

Despite apparent differences, meaningful comparisons are possible through the use of standardized scores. It's likely that standardized scores played a big role in your life when you were close to graduating from High School, and looking towards going to College; and for some us, when you graduated college and headed onto graduate school. Years ago, when you sat down to take your SATs your raw score on the test was converted to an SAT score. An SAT score is an example of a standardized score. Obviously, an SAT test never contains the exact same questions from year to year, and it can be taken at various times of the year, and different people take it each year. Colleges then used these SAT scores as a major factor in deciding whether to send you an acceptance letter or not. By design, a 720 on the math (or verbal, or other) section in one year is equivalent to a 720 on the math (or verbal, or other) section from any other year. And an SAT score of 500 is by definition the average or mean score. Standardized scores are designed to enable comparisons despite differences in the test questions, test administration dates, etc.

How is a standardized score determined? Imagine that you answered 70/100 questions correctly on your SAT test. So, 70 is your raw score. Is that a good score? That depends on what the average was. If the average was 60, then 70 is above average and so it can't be too bad. But you still don't know if it's just a bit above average or far above average. That will depend on the distribution of scores. If the plot of the distribution of scores were "tall and thin", centered around 60, with almost all scores falling between 55 and 65, then a score of 70 is actually very good as it represents one of the few highest scores. On the other hand, if the distribution was more "short and fat", centered around 60, with almost all scores between 40 and 80, then a score of 70 is actually fairly mediocre. The spread of scores around the mean is called the standard deviation, and it's a measure of how spread out, or how varied are the scores. A high standard deviation means scores are quite spread out. A low standard deviation means scores are tightly packed around the mean. So a raw score of 70 being good or bad really depends on what the average and the variation of scores (determined by others taking the test) turns out to be. Depending on this, a raw score of 70 may turn out to be a standardized SAT score of 510 or an impressive 780.

Armed with the raw score, the mean score, and the standard deviation, it's easy to convert a raw score into a standardized score. You simply take the difference between the raw score and the mean, and divide this result by the standard deviation. This is your standardized score. Computed this way, it is also known as a z-score. A z-score of 0 means your raw score is exactly the average. By definition, a z-score distribution is centered around zero with a standard deviation of 1.0. (The SAT's convert this score again just to have the average be 500, with a standard deviation of 100, so that things become easy to interpret).

What's this got to do with triathlon race results? If you imagine the above distributions of scores, they could just as easily represent the distributions of race times from a race. But instead of the x-axis showing SAT scores, imagine that it shows race finishing times. The y-axis will represent the number of people coming across in that race time. In fact, most races will have such Normal distributions. In terms of times, the front runners are few and far between, the middle of the packers arrive at a more frequent rate, and then things taper down again as the stragglers roll in. At IM USA 2002, approximately a full hour separates the first from the tenth finisher while the space of an hour in the middle of the pack sees over 300 finishers.

In a race, the shape of the distribution of finishers over time usually shows the peak of the distribution being close to the mean, and the spread of people's times around the mean is the standard deviation. Knowing the mean and standard deviation, along with an original race time (raw score) lets you convert a race time to a z-score. Simply take the difference between your race time and the mean and divide this by the standard deviation.

If you have the mean, the standard deviation, along with your race time for each race, you can compute a z-score for each of your races. Because z-scores are standardized (recall that by definition, a score of 0 is equal to the average), you can now meaningfully compare performances -- regardless of race distances, race venues, etc. Technically speaking, comparisons of z-scores are valid because the transformation to z-scores means both distributions will now have the same mean and standard deviation, whereas a comparison of raw time scores isn't valid because the original distributions had different means and standard deviations. With standardized scores, it's no longer a case of comparing "apples vs. oranges", but "apples vs. apples".

An additional benefit is that you can even compute separate z-scores for the swim, the bike, and the run portions of a race, and compare these to each other to determine your strengths and weakness relative to the field. This would actually be more useful and better than simply comparing your rank for each leg of race because even a small and trivial difference in time can mean a large difference in place/rank if you're close to being in the middle of the pack. For example, is placing 45^th in the bike mean you're really a better cyclist than runner because your rank in the run was 52^nd? Tough to tell. But if you compare z-score values instead of ranks, it becomes an easier conclusion to draw.