A while ago, I wrote an article investigating how a player’s experience affects their skill in Dota 2. Some of the keenest readers pointed out a couple of flaws in my methods, some of which I already considered and one that caught me off-guard. In this article, I would like to bring out some new data and address those criticisms.
Examining the MMR
The key pitfall of my previous analysis was not considering the effect of the matchmaking system. You see, although the Dota wiki does say this:
“…over time, win-loss ratios will naturally settle around 50/50 for all but the very best players…”
It is clear that, overall, the win rate is not an accurate proxy for a player’s skill. Following this line of reasoning, the best proxy for skill is in fact the matchmaking rating (MMR). More games played should translate to a higher MMR, right?
While there is not too much data on the MMR around, I was willing to put in the effort for the sake of sound science. With this in mind, I collected data on the MMR and amount of matches played by 46 players out of the European leader board and Dotabuff. Here is the end product – a small sample, but with quite a bit of variation.
The next step was to plot the data. I put the amount of matches played on the horizontal axis and the MMR rating on the vertical axis.
And what do we see? Unfortunately nothing. I had high hopes for this one, but it is quite apparent that there is no statistically significant relation. While kind of a let down, it is at least in line with the previous article. Mind you, this is the data for top players with 2,000 to 10,000 matches played (1-5k hours), and it is expected that the learning curve flattens out with time.
The real learning curve probably looks something like this.
As a result, the regression in the 2,000<10,000 matches played region actually shows no significant relation. This could be because at this level of skill, there can only be so much variation in MMR over the course of games played. The best of the best don’t really have to improve. But unfortunately, low MMR player data is not easily available.
If you would like to contribute to figuring this out, add your data to this sheet (can be anonymous), and I will run the numbers once we reach 50 players with less than 4,000 matches of experience (no smurfs, please).
Brief account of early analysis
A couple of my friends filled in their Dota 2 stats, and here are the early results for the mid MMR player analysis.
Even with a few mid-MMR players, the early learning curve starts to become more apparent. The logarithmic pattern is a guess — I would not like to start drawing conclusions just yet. It is, however, obvious that players with <3,000 matches played are more likely to have a lower MMR.
Conclusion
The goal of my last two articles has been to figure out if extra experience increases a Dota 2 player’s skill. I’ve considered a couple of proxies from top players (Win rate, KDA ratio, MMR) and they all lead to the same conclusion – there is no significant relationship between the time spent playing and the player’s skill. When also considering average players however, one can see a hint of a steep learning curve pattern.
Data from the top players teaches us that after the first 4,000 matches, simply pouring more time into Dota is not the way to become a better player. Whether it is talent, team work, methodical study, or organized practice, it is crucial to keep in mind that, aside from time, there are many other much more important factors improving your play.
Given that people pitch in with their numbers, future articles will consider the mid-MMR learning curve in depth. I am also interested in examining what differences there are in the distribution of Solo MMR and Team MMR. Is it the case that players learn how to play alone faster than mastering teamwork? Can we distinguish players in types according to individualistic or team preferences? There are many good questions to have a look at.
I hope this addresses the criticisms brought forward by the Dota 2 community. If you have ideas to share – leave us a comment!
Review Yobo
Published: Mar 2, 2016 01:24 pm