K let me try and explain why we think our system is worth it 
CSports.net is an automated ranking system that attempts to rank worldwide players using a uniform and logical ladder methodology. CSports.net currently covers 3,100 modifications on 100 first person shooter titles. We normally process around 3 million player hours a day and 25 million unique player names a month. This equates to about 90 million player hours a month and about 9 billion records.
Given the magnitude of data and the type of information that it is possible to get from a public server, our methodology has to be simple, logically and conceptually believable, and applicable to 99% of the modifications out there. The methodology is based on that used in a traditional sports ladder. If you beat someone above you, you go up. In practice the way that we do this is as follows:
Every 3 minutes we look at each server and calculate the number of points each player scored during the 3 minute period. For each player we work out how many players who rank above them on the ladder they have beaten and we raise their rank by that number. In addition at the end of each day we apply what we call “rank gravity� to all players that have not played for a while. Those who have not played for more than seven days accelerate on each additional inactive day towards the bottom of the table.
The result is a dynamic ladder which is always trying to settle out to the “right” rankings. It is a natural equilibrium based system where player names strive to attain a natural equilibrium corresponding to a players ability. It is also a greasy pole in that it is hard to hold on to a high level with gravity and other player movements pulling you down. At times this equilibrium system can seem a little “offâ€?, especially at the beginning of a ladder (where players are in order of appearance and the system has not started to achieve a semblance of equilibrium) and when a player starts out or changes his name (the player hasn’t reached is equilibrium position). However rank gravity normally sees to it that new players/names can rise up the ladder to their natural level reasonable quickly. All in all the rankings seem to work really well even on a large scale.
One of the main benefits of a ladder system is that with one exception, it does not need to take into account individual scoring methodologies for each modification. However if a negative score is better then of course the methodology doesn’t work. But this is rare.
The way to get to the top of any particular ladder and stay there is to play often (at least once a week) and consistently beat players above you on the ladder. Of course there are other ways of getting to the top of a ladder that broadly speaking fall into the categories of cheating or gaming of the system. If this is your biscuit there is isn’t much we can do to stop you, although it rather defeats the object of the ladders (additional fun, interest and motivation for everyone) and eventually you get caught out.
One comment that always amuses me whenever I see it, is that “The CSports.net ranking is not accurate�. Reasons given vary from “I am ranked far too low, how can that be right� to “but you haven’t taken into account my role within the game�. Invariably this comment completely misses the point which is:
- By definition a ladder based ranking system cannot be “accurate� for the following reasons:
a. Any ladder system takes an unequal number of sample points for each player so it tends to favour those who play more.
b. At any one time there are new players joining a ladder who are always going to enter at a position that does not reflect their ability.
A ladder is a dynamic entity that is always trying to reach the “right� answer but is never able to, because of new players and the ups and downs of each individual’s form over time. What should be important to each player is the progression of their rank over time, rather than their absolute rank, hence our emphasis on rank graphs.
-
“Accuracy� is in the eye of the beholder. Conceptually the only way to rank players by ability is for someone or something to observe each match that takes place in the world and rank the players according to a whole host of criteria (eg role, weapon handling skill, movement, use of strategies and tactics, score, accuracy, tiredness, benefit or not from luck etc). But most of these measures are entirely subjective and their combination is absolutely subjective. Any such ranking system, were it possible, would always suffer from the criticisms of a different point of view on the values of the variables and their combination.
-
You cannot make subjective judgements using a computer. Even if you could measure all the necessary variables their mathematical combination would at worst be a complicated, unintelligible random number generator and at best reflect the subjective views of its coder.
-
On public servers there is only one available measure on which to judge players and that is their score over time. Even if all of the above wasn’t a problem then technically you can’t get the necessary measurements on this scale without a great deal of coding of a server side app for each game/mod and then you would need to get the compliance of 100s of thousands of server operators.
-
The CSports.net ladders are supposed to be fun. This is quite often forgotten Incidentally one result of this philosophy is the implementation of “rank gravityâ€? which can’t be defended as a valid metric of someone’s ability. But it is necessary to keep automatic ladders fresh, relevant and “fair”.
At the same time there are a number of steps we take to aid “accuracy�
a. We make sure that our system sticks to mathematically to the algorithm that we have developed over the last 6 years and has now not needed tweaking for 2 years.
b. We keep snapshot times as low as possible, the more sample points we can get the quicker equilibrium is reached.
c. We try to sample as many servers as we can.
d. We credit the mod/game developers rather than ourselves as the best judge of what makes a good player in a game ie the published score value.
e. When alerted we do our best to expurgate the records of obvious bots and cheats.
f. When alerted we do our best to exclude from the ranking common names.
At the end of the day there are only five of us at CSports.net and we try our best to provide a consistent, logical and competent ranking system that is useful and fun to use. How well founded and useful the resultant ladders are depends very much upon the communities that we serve. If the people within those games communities feed back the problems (bots, common names, odd mods, missed mods, missed servers, missed players etc.), then we can adjust what we do to make the system more useful and as a result more fun for that community. Please enjoy what has become our lives’ work.
