Analysis

Update (April 16th): Since originally posting I have added an age-weighted component to the numbers, so that newer results have a larger influence than older ones. As you can see from the changes in the numbers, this adds another interesting dimension to the numbers in this post.

I love getting feedback on my analysis and predictions – very often, they trigger some new, interesting way of looking at the data. For example, Linsey Corbin made the following remark to me:

I wish there was a way that your predictions could show consistency. One thing I pride myself on is being fairly consistent across the board.

Thanks for the suggestion, Linsey (and great to see you back to racing)! I have been looking at different ways of attacking this question, here is what I was able to come up with. I will continue to monitor these numbers for upcoming races, maybe and I’ll include them in future predictions.

Deviation

In statistics, there are a number of way to measure how “consistent” a set of data is. The most common way to express variability in data sets is the “Standard Deviation“. StdDev basically measures the distance of data points from the average value – the more “outliers” there are and the further off they are, the higher the standard deviation.

This was my first try of analyzing consistency. The data analysis part is pretty simple, as the function is built into all kinds of programs. However, the results were not very helpful: In essence it helped identify athletes that had one or more sub-standard results, e.g because of walking large parts of the marathon in a race. For example, Lucy Gossage showed up as an inconsistent athlete with a large deviation, but that was almost exclusively a result of her marathon walk resulting in an 11:32 finish in Kona 2014. It also didn’t value “good” results: The difference of a good result to an average – maybe 30 minutes or so – is much smaller than that of a bad result – walking easily adds an hour to the overall time.

Identifying Non-standard Results and Quantifying Consistency

Even when looking at the deviation of results of each athlete did not lead to a good measure, it formed the basis for another way of looking at the data. In the familiar “bell shape” curve of the normal distribution, 68% of results fall within one standard deviation around the average. When looking at the difference between an athlete’s “expected time” and their actual finishing time, roughly 68% of the results are within 20 minutes of the expected time. Based on this I classify results within 20 minutes of the expected finishing time as “normal”, and any result quicker as “better” results and anything slower and DNFs as “sub-par” results.

I can then aggregate all the results of an athlete into a figure like this:

Linsey Corbin: 83% +17% -0% (18)

Older results have less of a meaning than newer, so adding in an aging component gives the following numbers:

Linsey Corbin: 79% +21% -0% (18)

Each part has the following meaning:

• Linsey Corbin: Name of the athlete
• 79%: Fraction of normal race results
• +21%: Fraction of “better than expected” race results
• -0%: Fraction of “sub-par” race results (including DNFs)
  (Note: Technically, Linsey has at least one DNF in her Ironman races – she didn’t finish IM Texas in 2011. This is a limitation in my data – I have only been including DNF’s since 2014.)
• (18): Total number of Ironman-distance results (including DNFs)

Examples

Here are some more numbers from well known athletes – put into different groups. (As I have updated my algorithm a bit since posting for the first time, I am also including the originally posted numbers in [square brackets].)

Stable Athletes

• Andy Potts: 100% +0% -0% (13) [originally posted: 100% +0% -0% (13)]
• Yvonne Van Vlerken: 84% +0% -16% (23) [originally posted: 91% +0% -9% (23)]
• Lucy Gossage: 92% +0% -8% (12) [originally posted: 91% +0% -9% (11)]
• Sebastian Kienle: 85% +12% -3% (11) [originally posted: 82% +9% -9% (11)]

Normal Stability

• Jodie Swallow: 55% +0% -45% (10) [originally posted: 78% +0% -22% (9) – she has since DNF’d in South Africa]
• Caroline Steffen: 92% +8% -0% (20) [originally posted: 75% +25% -0% (20)]
• Meredith Kessler: 65% +14% -20% (23) [originally posted: 70% +17% -13% (23)]
• Andreas Raelert: 48% +0% -52% (19) [originally posted: 63% +0% -37% (19)]
• Luke McKenzie: 51% +30% -19% (26) [originally posted: 62% +23% -15% (26)]

Lower Stability

• Sarah Piampiano: 41% +47% -12% (14) [originally posted: 50% +43% -7% (14)]
• Luke Bell: 23% +5% -72% (26) [originally posted: 38% +12% -50% (26)]
• Dede Griesbauer: 41% +18% -40% (26) [originally posted: 32% +32% -36% (25)]
• Tim O’Donnell: 14% +63% -23% (11) [originally posted: 27% +45% -27% (11)]
• Pete Jacobs: 5% +16% -79% (26) [originally posted: 15% +42% -42% (26)]

Then there are athletes that have a lower fraction of “normal” results. Here it’s also interesting to look at the upside (e.g. Sarah Piampiano, Tim O’Donnell) or downside potential (e.g. Luke Bell). Some athletes’ results are very hard to predict from previous numbers – for example Dede Griesbauer and Pete Jacobs have had a good fraction of great results but also slower, disappointing results.

After the fast times at the end of 2015 there has been some discussion about continental and national “records” over the Ironman-distance. Because of doubts about the accuracy of courses, comparing times from different courses is always a bit tricky, but here is an overview of the data I was able to compile.

Please let me know if I missed some older results that are better than the continental and national records in this list!

Continental Records

Female Athletes

Male Athletes

National Records

Sometimes, the nation of an athlete is not clear – often athletes are listed with their country of residence (e.g. foreign athletes staying in Boulder), and some athletes have dual citizenships. Please let me know if I have mis-attributed a fast result by an athlete to the wrong country!

Female Athletes

Male Athletes

Notes

There are a few records that need some explanations.

Female African & South African Record

I have listed Dianne McEwan (now Dianne Emery who became a mom in January) as the African record holder, but she herself considers Annah Watkinson’s 9:31 from IM Austria 2015 as the record. Annah has been racing as an age-grouper then, and because of the different race dynamics from the Pro race I’m not counting her result. But Annah has turned Pro this season, so there’s a good chance we will see a new African record this year, maybe as early as IM South Africa!

Male Canadian Record

Lionel Sanders sent me the following tweet after I posted the fastest times:

Lionel is right that Peter finished in 7:51:56 in 1999. However, it is accepted that the marathon in Klagenfurt was short (Peter ran a 2:35:21!) – probably by more than 1k. With Peter being a great athlete and fast runner, one could speculate if he could have finished faster than Brent’s 7:55:48, but I’ve decided to not accept his time as a record.

Male Dutch Record

Jefry Visier, the Operational Director of Challenge Almere, was going through older Almere results and found four times (three by Jan Van der Marel and one by Frank Heldoorn) that were quicker than the one I had from Bas Diederen.

This is an except from my free “2015 TriRating Report“. If you’re interested in more information about the 2015 long-distance Triathlon season, you should definitely check it out!

Overall Money List

First, here is an overview of the races I have included in my money list:

This does not include the $1 million prize that Daniela Ryf collected for winning the “Triple Crown”.

The next table shows the Top 20 athletes – both from the men and women – that have earned the most prize money in the 2015 calendar year from all the races listed above:

Meredith Kessler has made it into third spot without any money from the “big money races” in Kona or Zell Am See.

Ironman Money List

Here are the Top 15 money earners from Ironman races (excluding Kona):

70.3 Money List

Here are the Top 15 money earners from 70.3 races (including the 70.3 Champs):

It’s interesting to note that Tim Don and Meredith Kessler have almost made it to the top of the list without any money from the 70.3 Championships.

I have been publishing Race Predictions for a few years now, so it’s about time to have a look at how “good” my predictions are. When I started in 2011, there were quite a few changes in the algorithm and the parameters to deal with a number of edge cases. During 2015 there have not been any changes, so this is a good data pool for validation.

Data Used

In 2015 I have published predictions for 36 Professional Ironman-distance races, 31 Ironman-branded races by WTC and 5 more Challenge races.There have been a total of 1098 finishes, 688 by male athletes (62.7%) and 410 by females (37.3%). These were posted by 600 different athletes, 382 male (63.7%) and 218 females (36.3). In addition there were 349 DNFs, 244 by males (70%) and 105 by females (30%).

Using my algorithm and the available start lists, I have seeded the participants in each of the races, and predicted 930 finishing times (84.7% of all finishers). There are some cases when I didn’t predict the finishing times, for example when an athlete didn’t have any prior IM-distance finishes or when there was a late entry (and therefore the athlete not included in the start list).

Predicting the Winners

Here’s a look at the places the eventual race winners have been seeded based on previous results and the start lists:

With 36 IM-distance races, there are 72 winners (one each for the male and female race). My algorithm has correctly predicted the winner in 26 races (36%), and another 26 winners were seeded in #2 or #3 (winning frequency of an athlete seeded on the podium: 72%). Only three winners have been seeded higher than 8th: Kirill Kotshegarov was seeded 10th at IM Chattanooga, Mel Hauschildt was seeded 11th at IM Melbourne, and Matt Hanson was seeded 12th at IM Texas. There was also one unrated (and therefore unseeded) winner in 2015: Jesse Thomas won IM Wales in his debut Ironman.

The numbers would be even better when only considering the athletes that finished a race. Only including athletes that actually started increases the frequency of picking the right winner to 39% (and one of the podium picks to win the race to 80%), also discarding athletes not finishing would have yielded 42% and 83% of the winners.

Time Predictions

In my pre-race posts, the finish times are predicted for each athlete that has raced an Ironman race before. The algorithm considers the previous finishing times of an athlete and the course that the race is going to be held on.

The following graph compares the actual finish times to the predicted finish times (each data point is one dot on the graph). Dots towards the upper left are results where the actual finish was faster than predicted, dots towards the lower right are results that are slower than predicted.

The graph shows actual and predicted times between 8 and 12 hours (only 11 faster results/predictions and 15 slower ones are missing).

I have added a “trend line” that shows the best fit of all the data points, highlighting the fact that most of the data points are pretty close to the “diagonal” (where actual = predicted). Between 8 hours and 10 hours the algorithmic predictions are pretty good on average (maybe predictions are a bit too fast around 8 hours). Towards 10 hours finishing time and especially over 10 hours the predictions are too fast: This is caused by “explosions” that lead to very slow times even for athletes that have been predicted to be relatively fast. To put it another way: Finishing times over 10 hours are most often bad races that are pretty much unpredictable using only data.

Here is another way of looking at how far off the time predictions have been from the actual results:

The graph shows the number of results in one minute bins of difference between predicted and actual finishing times. Data points towards the right are faster than predicted, they are slower than predicted to the left. Again a trend line smoothes out the statistical “noise”.

A few observations:

• In a range roughly between -40 minutes and +40 minutes the graph is pretty symmetric and is very close the normal distribution.
• As noted above, there is relatively large number of “explosions” with large negative differences, resulting in a non-symmetrical distribution on the edges of the graph. (There are 49 results that are more than 60 minutes slower than predicted, but only 10 that are more than 60 minutes faster.)

On average, the predictions are -4.7 minutes off the actual finish time (i.e. the actual finish is  slower by close to five minutes). An average close to 0 means that on average the predictions are closer to the actual finish. The standard deviation is 31.8, this means that 68% of the time differences are between -36.5 and 27.1 minutes (-4.7 +/- 31.8 minutes). Usually, a smaller deviation corresponds to a “better” prediction.

The standard “statistical” way of measuring the dependence between two data sets is correlation. Correlation is +1 in the case of a perfect linear relationship, −1 in the case of a perfect inverse relationship, and some value between −1 and 1 in all other cases, indicating the degree of linear dependence between the variables. A value around zero indicates that there is less of a relationship (closer to uncorrelated). The closer the coefficient is to either −1 or 1, the stronger the correlation between the variables. The predictions and actual finishing times have a correlation of 0.77, indicating a pretty strong dependence between the two data sets.

Comparing Other Prediction Strategies

When trying to predict a completely random event (the classical example is a “perfect dice”), the correlation between the actual events and the predictions won’t be very high. When only working off previous results, anything that happens before a race – for example a good block of training, a slight injury – will influence how fast an athlete is able to go on race day, as will random events during the race (e.g. getting punched during the swim, technical issues on the bike). Therefore a “perfect prediction” (resulting in a correlation of 1) is impossible, and in order to determine whether a correlation of 0.77 indicates a “good predictor” or not, one has to compare the results of my algorithm to other predictions.

I am not aware of anyone other than me publishing time predictions for Ironman races on a regular basis. (Please let me know if there is!) Therefore, I am comparing my predictions to a few much simpler strategies:

1. Last Finish: “You are only as good as your last race” (prediction = last IM-distance finish)
2. Best Finish: “My best time is a sub-x” (prediction = fastest IM-distance finish)
3. Average Finish: “I usually finish around y” (prediction = average IM-distance finish)
4. Average Last Year Finish: “This season is going great” (prediction = average IM-distance finish in the last twelve months)

Here’s a comparison of the correlation of these different methods and my comments:

Summary

The Prediction Algorithm I use to calculate the expected times in my pre-race posts provides better predictions than simpler prediction strategies. My model certainly has limitations, but the large number of “successful” winner predictions and the high correlation show that the time predictions and the conclusions drawn from them are pretty much valid. I think my analysis is quite good at telling the “data part of the story”.

While the “data part” is an important (and impartial) part of the story, it is still only a part of the story. A coach or teammate that has been able to observe an athlete getting ready for a race has additional (and more current) information available – even if that is not always fully objective.

The tension between past performances, the uncertainty of a future performance, the challenges athletes face in their training and the hard work they put in to be better in their next race .. that’s why I still love following the races!