Update and Reflection (October 2015):

WOW! This was almost two years ago. It's pretty funny looking back at these old blog posts. This is probably the first actual "data science" project I ever worked on, and now I get paid to do silly things like this which is pretty cool. So yeah, if you're looking to get into software or data science, you have to find random little projects to work through. It might be embarrassing to look back, but in the end if you're learning stuff that's all that matters. Thanks for letting me scrape your website for all of these years, guys! Cheers!

I've added some additional comments at the bottom of the article.

Original Article:

Project Ares held their first full tournament yesterday.  This included 16 teams playing a variety of maps, and it lasted a whopping 8 hours!

Anyway, I thought it would be fun to compare all of the teams competing using some of the statistical data available on the Project Ares site to see how well of a performance predictor it is.  There are three main statistics: KD, KK and OD.  They are defined as follows:

• KD = kills/deaths
• KK = kills/times killed by another player
• OD = objectives completed / deaths

KD and KK are directly related to fighting skills, where as OD has more to do with completing objectives (capturing wools and leaking cores).

I compared teams by taking the average of each of these three quantities for each team.

Results

Predicted Bracket
Actual Bracket (link now broken, October 2015)

The KD and KK ratios give identical predictions.  The OD ratio, on the other hand, is a very poor indicator of performance.

Using the team's average KD (or KK) alone, I was able to predict 12 out of 15 matches (accuracy of 80%), including the semifinals and tournament winner and most match outcomes (there were only three upsets: two pictured and one when YoloSwag beat Impact in Round II).  Of course, minecraft gameplay is complex and the three statistics alone do not represent all of the skills required to win a match, but it does seem to be a pretty strong indicator of overall performance.

Looking at the numbers, it is quite clear that the winning team, Badlion, is totally stacked.  Their average KD was 3 times that of the overall average.

Other Stuff

I decided this would be a neat idea late last night after the tourney ended, so the way I built the bracket is a bit of a mess (a combination of an extremely inefficient Mathematica script and google docs).  But, I plan on playing with this more and automating it better for future use.

More Update Stuff (Still October 2015)

So what's changed? An accuracy of 80% using only player's KD ratios is pretty crazy. In fact, since the first tournament, it is no longer possible to make such a "good" prediction. I think there are two reasons why KD has lost its predictive power:

1. Since the first tournament, players have begun boosting their stats to get on teams. This includes playing differently than you would in a tournament (e.g. cautiously farming), making them less reflective of actual ability and more of patience.
2. People strat the hell out of maps now. Players knew maps during the first tournament, but now teams are in general much more organized and practice a lot more, trying to exploit every little opportunity each map has to offer.

I think point 1 isn't as important, because most good players know that stats aren't necessarily an indicator of skill level. At the end of the day everyone does a little sword fighting, but map-specific knowledge and team organization still plays an enormous role in winning matches.

Now that Overcast Network (which was named "Project Ares" when this post was originally written) has introduced a more sophisticated ranking system, I'm looking forward to squeezing some information out of that data next time around.