Community Question: Measuring Player Skill


(badman) #1

Splash Damage has released several team-based multiplayer shooters over the years, and they’ve all featured various ways of showing off how skilled a player you are. For starters, experience points (XP) has been front and center on all of our in-game scoreboards, while player rank icons were featured in Wolfenstein: Enemy Territory’s Campaign mode and persistent ranks first made their debut in Enemy Territory: QUAKE Wars. This brings us to this week’s Community Question:

What do you think is the most important stat for player skill?

Please cast your vote in the poll above. We’d also like to know why you think your stat of choice is the right one to go for.

We’re looking forward to your replies!


(Humate) #2

A high accuracy is useless if it doesn’t lead to your opponents dying when you need them to. Staying alive or building battlesense or a k/d ratio is completely meaningless if that doesn’t lead to valuable contributions that in turn lead to winning the match.

A score used to determine skill is useless, if the playerbase doesnt agree with it.


(Ashog) #3

Are we in a hurry somewhere? Let give the game gather statistics for half a year, then come up with statistically weighted win/loss criterium. Will take anyway at least half a year to get the game fully patched. If the game is bound to live for several years, half a year is not a big deal to wait, especially if other stats are easily available meanwhile. Also what’s about all of this striving to get rated by all costs by players nowadays? I don’t mind being not rated, as far as I get fun from actually playing rather than spending hours staring at own stats.

The other thing is that I smell something. I smell SD going for permaunlocks and that’s why Badman is asking this sort of questions. This is indeed a grave news if it is so…


(tokamak) #4

Arguments concerning the support of a proposition always border on the appeal to popularity fallacy. What people value isn’t necessarily valuable when the indicator needs to be tared on ‘winning the match’.

Now you can say ‘who cares? I there’s one statistic that has the more support than all the others then why take it away from people?’

That’s the elementary ‘first past the post problem’ explained here:

//youtu.be/s7tWHJfhiyo

The most popular choice isn’t automatically the majority choice. This is the reason why giving the public a single choice results in so much dissatisfaction.

Anyway, to break it down further:

There are two camps in this debate. Those that value prestige and those that value effect. Those that value prestige want to give the highest score to players that make you go ‘wow!’ when you see them perform in frag movies. Players that value effect want the player that plays vigorously to win and the player that plays vigorously to improve his score to be completely indiscernible from each other.

These two camps will never agree on an indicator and trying to align these two preferences under the same score is impossible so I’m not going to bother (and thankfully at least one of them has stopped bothering as well).

That means that as far as the ‘effect’ players are concerned we can break the issue down by simply listing the options we have.

Let me break down the problem here:

We have five options:

- Don’t track anything at all old-school, has it’s merits but ultimately unfufilling. People have ego’s and ego’s want to compare themselves to others.
- Track everything and let players decide what they value.
This turns the game into a sandbox because you now established that winning isn’t the most important thing to accomplish any longer. “Woot I got the highest accuracy” “Who cares? I ran the longest distance!” “So what I fired the most shots!”

  • Establish a singular statistic as a skill indicator
    Same problem as above. The moment you do this you establish that winning no longer is the most important thing. Kills per minute the prevalent indicator? Then suddenly the whole game is about killing as efficiently as possible. ANY singular statistic becomes the main purpose of the game.
    -An average score based on several statistics Again, same issue as above, only harder to manipulate because players will have an issue keeping track of several directions to play in. Bottom line is that fragging and staying alive is all that matters, doing objectives becomes an annoying distraction that only serves to lower your score.
    Collect all tangible beneficial actions that we know will contribute to winning and express them in xp/min
    Are we certain that that revive was necessarily the best thing that player could do? Not under the old systems. That’s the flaw and that’s what this system is currently attacked on. Not everything the (current!) xp system rewards is valuable. A fair point but not unsolvable. Rather than switching the indicator it’s it’s base dataset that needs tweaking. Not just the weights between actions but a more sophisticated context-sensitive aproach. Additionally match outcomes should skew it to get an even more representative score.

This score is the hardest to accomplish because the underlying clockwork needs to be detailed and sophisticated. I believe that’s worth the while because the xp system doesn’t only judges a player, it also rewards the player for doing well. So any advancement in this field is a double win.


(tokamak) #5

I don’t think you fully appreciate the amount of matches required to achieve significance. I’m not a statistics wonder so it would probably take me a page by itself to explain it but trust me when I say that in order for w/l ever to become significant you need more games than someone will be able to complete in several lifetimes.

The other thing is that I smell something. I smell SD going for permaunlocks and that’s why Badman is asking this sort of questions. This is indeed a grave news if it is so…

That’s why I hope we will have a poll about character progression very soon in which SD already supplies the context (F2P or full game purchase). I really would like to weigh in on that.


(Ashog) #6

I don’t agree. Half a year of games should be well enough. The game needs only to take into account the current win/loss ratings of other people who played in your and in enemy team and calculate a modifier for your rating after each game. Remember Starcraft 2 could determine your skill and place you in that or another league division just after several games?


(tokamak) #7

That’s because Starcraft works with 1v1 pairings based on a hidden ELO rating. Then, for SC’s team games you’ll see that the league ratings become increasingly useless. A bronze team of 4 bronze 1v1 players have no issue reaching diamond in 4v4. This is simply because the system struggles to ‘guess’ the right league for you. Your 4v4 rating simply isn’t significant enough for it.

Now, that’s when you play with the same 4 people. Imagine how it turns out if you’re constantly playing 8v8 or 12v12 with completely different people on each side.


(Humate) #8
  • Track everything and let players decide what they value.
    This turns the game into a sandbox because you now established that winning isn’t the most important thing to accomplish any longer. “Woot I got the highest accuracy” “Who cares? I ran the longest distance!” “So what I fired the most shots!”

“Guys, did you see my game winning tac shield? cmon guys it was aaaaaallllllll me… you MFers better recognise aiiight”

“I’m really disgusted with SD’s score system, it doesnt realise the gpmg nest I built was the sole reason we won the game”

“I feel so clever using my platypus as a distraction, pity SD cant see the impact of this genius move.”

“You know if it wasnt for my moral support in the passenger seat of the bumblebee, you would never have gotten those kills”

“Guys, you see that medcrate youre hugging? Yeh thats mine, thank me later… no really, you better thank me”


(tokamak) #9

That’s why you let the victory be a modifier to your xp score, that way even intangible contributions are weighed.


(DarkangelUK) #10

“That’s why when XP fails we pretend it didn’t”


(BioSnark) #11

I can see the cycle is going to run again, so, to help it along, it doesn’t matter if the playerbase can’t agree on it if it’s it’s being used as a metric for an automated balancing or matchmaking system. The worst thing about xp other than that it’s massively flawed is that it’s visible and visibly open to manipulation to any end.


(Dormamu) #12

SD doesn’t need need to implement a new state of the art ranking system who will be accepted by all or the majority, they only need to create the tools and lay out the foundation, everything else will be made by the community (so i hope and pray :D).
Assuming SD will give us the tolls to make new modes/maps :smiley: this will mean that the new modes/maps and the action you do on those modes/maps will not be bound by the majority choice system, but by the things that are important in those specific modes/maps. Let’s say a new mode will need a system who ranks the teams by the time it takes them to complete a map, or by KDR, or WLR, etc.
I think this is the most important part of their new game, to allow modification and freedom, to allow the community to create and implement a system they think is best for that particular mode or type of game or for different people. Why should we need the same system for 1vs1, 3vs3, 6vs6, 16vs16, CTF, TDM, OBJ, Stopwatch, Pub, Competition, etc.? Why not allow it to grow (game/system/mod) in to something amazing or something different from your starting idea?
Choosing one path can limit your unannounced game ®evolution :D.


(Humate) #13

Thats not what is being discussed. :slight_smile:


(BioSnark) #14

Sorry. ELO was being discussed somewhere else. The hundreds of posts in this thread get all blurry sometimes.


(Humate) #15

Thats cool :slight_smile:
Its a silly discussion anyway.


(edxot) #16

[QUOTE=edxot;405061]if i had to make some automated system to find out who was the most important player in some match of etqw 12v12 i would choose it this way: it would be the player with most kills in the defending team, or the player with most xp in the attacking team (one of those 2 is the best, depending on the team who actually won).

but there are a few situations where this does not work so well … so i would includ this other thing:

a player can also be chosen from the losing team in any of these (uncommon, i.e., unbalanced teams) situations:
he was attacking and lost, but he got the most kills
he was defending and lost, but he got the most xp

PS: remember, 12v12 matches, not 5v5 etqwpro games (about those i have no guesses).[/QUOTE]

forget about this idea


(XA417YP) #17

In addition, we’ve had statistics sites filled with data, allowing players to examine many different aspects of their performance and in-game achievements.
Don’t care about the “major statistic” and comparison with other players, but to know your own stats is vital.
Also it’s more important to have some stats site at release date, then perfect stats coverage 3 months later.


(murka) #18

I do hope that future games allow plugins to be made about stats, be it parsing the log file or sending packets to servers and getting detailed information.


(Ashog) #19

yeah, that’d be great!


(ailmanki) #20

Sounds like a winner! That would be very attractive for server admins, even if its a ranked server - at least something they can tune.