Community Question: Measuring Player Skill


(Ruben0s) #61

I don’t think that it’s possible to measure skill.

but I would do something based on:

  1. damage dealt / damage received per minute. (this is way more accurate then the K/D ratio, since you can steal a lot of kills)
  2. K/D ratio

and tons of other things that totally depend on the game and some that you can’t measure(player movement, how you handle difficult situations, working as a team, finishing objectives under pressure( for example the enemy will respawn really fast, will you suicide for the objective and help the team) etc.).

I have no idea how you can measure skill as in doing an objective, because it depends most of your time on your teammates.

PS: I don’t give a thing about a stat called “skill”.


(Dormamu) #62

Other

In W:ET i was interested from Player Stats about Accuracy, Headshots and Team Damage, but most important if i got in the Roll of Honor / stats for the entire map played :smiley:

I would like SD to make and implement for their next game some kind of ELO/FIDE/USCF rating system with its own classification of players/skill. The player will have a number/name/etc about overall skill of a player with some W:ET fluff (Player Stats, Roll of Honor per map/ campaign ).
I think you already have some matchmaking system in work for your next game, if is something like the ELO/FIDE/USCF why not make it visible so everybody can see his class (Class J,K,L…,B,A, NM, OLM, SM - USCF classification) you will use the matchmaking and make it also a Skill rating system.
i’m more interested in the rating system because i would like to bet some money on the upcoming matches of your next game, you could also implement a betting system and some new currency to be used in game to purchase fluff/guns/abilities/etc. If this work you can always implement some system to exchange the in game/betting currency to real money :eek: , with this you will bring along the people who only like to bet to your FPS game :smiley:
So we have: matchmaking => skill rating => betting => in game money => YOU JUST WON A S*** LOAD OF MONEY!!! :smiley:


(phoen1x) #63

Something like quakelive’s score system. K/D really say nothing about player skill. Dunno about ELO, tried playing League of legends and trust me that ELO system is complete FAILURE.


(tangoliber) #64

I loved that Brink removed KDR stats, because it encouraged people to play selflessly. People are more willing to die 20 times in a row trying to rush in and complete an objective…there was no motivation to just deathmatch.

I don’t care much about scoring systems… I doubt they can ever be accurate. Most players will know that they don’t actually represent usefulness to the team and not even peek at the rankings…but some players will always take them too seriously. :slight_smile:


(Exedore) #65

The reasons why we want to know what you all think about this go far beyond personal fulfilment. Essentially, we’re asking your help in distilling KPIs (Key Performance Indicators), which can ultimately help balance the game, and also make sure the game is fair. We know what we think is important, but as the game is ultimately for the players, it’s valuable to have your input as well.

As many of the comments here have indicated, if players don’t think a rating system is solid, it diminishes the urge to excel. And screw that!
:flamethrower:


(light_sh4v0r) #66

A combination of things.
Battlesense I thought was always pretty accurate, so I would take that and combine it with some objective and k/m stats.


(phoen1x) #67

Why would u combine it with objectives completed? Most important stats would be damage dealt/taken per time (damage dealt to turrets (if there will be any) should not count), accuracy, k/d and… um… i don’t know what else u could measure. If possible measure bullet dodging :smiley:


(shirosae) #68

Depends on the game mechanics, so I’d take the stuff suggested at this stage kinda lightly.

That said, you’re probably looking for a combination of mechanisms. In terms of a player exerting influence over the match, I think you’re looking at:
K/D Ratio
Damage dealt/taken ratio
Damage survived (like battlesense)
Time spent in proximity to objectives.
Time spent interacting with players near/performing/defending the objectives.

In the fantasy realm in my head, each player would build a temporary web of score incomes depending on how their interactions with other players went.

If you revived a player, you’d gain a short period where their actions would syphon score to you. So if you revive key players who are doing their job, you gain a fraction of their score. If you keep pew pewing, you get another stream of incoming points based on the influence those enemy players were having. So you gain more points for shooting a player who has killed lots of people, or done lots of objectives, or revived lots of players.


(tokamak) #69

The combination is already available in the poll, it’s called score/min. I’ve always argued for xp/min and I always will. The premise for this is that the xp If the xp/min isn’t accurate then that means the xp-distribution needs to be looked at. It’s the only way to encompass all the measurable actions and weigh them together.

Anything else is incomplete by definition. An entire team can focus on getting as many kills or doing as much damage as possible and still not bring them closer to the objective.

I’m actually pretty disappointed with the poll results so far. Why would people even play a shooter that requires more than just fragging and/or surviving if that’s the only thing they wish to be lauded for? Why value just one of these qualities if you can have them all?

I believe this is perfectly possible. And not just interactions with players but interactions with the environment as well. Killing a player near his goal should reward you more xp than killing someone far away.

That’s another issue with just looking at damage or just looking at kills. Not every kill, mission, buff or revive is equally important and a system that is blind to that has perverse incentives and skews the players priorities in a match.

I also think more actions need to be included. Sometimes people overestimate how difficult it is to reward certain actions.I remember Rahdo was working on a way to reward smoke grenades. It was a really complex system that measured the amount of line-of sight of an enemy player got obscured by the smoke grenade and whether or not he would still fire at it etc etc.

Something like this often has a way easier solution. Just award xp for the amount of missions fulfilled within the effective radius of the grenade! If a player revives someone or hacks/repairs/destroys something while in the 10 feet radius of an active smoke grenade then the thrower gets an xp bonus for that.

Simplicity is always lurking around the corner for these things.

There’s just so much garbage in that formula I don’t even know where to start.
" (K/D Ratio x objectives completed + overall XP) / hours played = gameplay score"

The only inherent value in this is the overall xp. And is already directly or indirectly including the only value that objectives completed and K/D have.

Objectives completed is part of a team-effort rather than an individual achievement. The due covering your ass while you’re doing the final objective is most of the time worth more than what you’re doing. Your actual stake in the objective is supposed to be covered in the xp you gained from it, with an elegant and exhaustive xp-distribution system the dude covering you would also benefit here.
Giving any more value to merely completing objectives creates a game of soloists, lemmings and even saboteurs.

K/D ratio is even more worthless. It shows combat potency and nothing else. Yes, it’s the single most important value in a TDM, but in an xp-based assault mode it says very little. And once again a high K/D often correlates to a high XP/min. After all, a player that doesn’t die a lot compared to how he kills often finds him in a situation freed up to do more for his team. K/D is a only a means, XP/min is the end.

Besides, it’s easy to think up moments where keeping up a high K/D is detrimental to the goal of the game. Sometimes you need to sacrifice yourself in order to get further. Sometimes really valuable missions that will end in certain death don’t score you any kills to offset that death. A medic could go in, revive his entire team and die at the end which would result in a lower K/D score.

In the end K/D by itself is worthless. Once again, its a means. Not an end.

The same reasoning goes for battlesense. Just like K/D, having a high battlesense only means you’re more able to be value to your team. That true value however, is only expressed in xp/min (and a part of your xp/min is the xp you gained from battlesense).


(Indloon) #70

Agreed, the kills can be stolen easily and can be also abused by “Kills boosting”.

In ETpro mod,people look at the damage received and damage given, that shows how professional or noob you are.

ELO rating system is not a failure. Just read here about it: http://en.wikipedia.org/wiki/Elo_rating_system

Now let me quote something from Wiki:

It is also used as a rating system for multiplayer competition in a number of video games

Perfect!


(DarkangelUK) #71

Was wondering how long it would take tok to barge his way in and tell everyone how it should be.

Before we even start, xp/min is so open to abuse and false results that Stevie Wonder to could see that. Take any ET game and see the medics raging away on top with that… so lets stop pretending its a good idea and get your head out of your ass.

[QUOTE=tokamak;402885]The combination is already available in the poll, it’s called score/min. I’ve always argued for xp/min and I always will. The premise for this is that the xp If the xp/min isn’t accurate then that means the xp-distribution needs to be looked at. It’s the only way to encompass all the measurable actions and weigh them together.

Anything else is incomplete by definition. An entire team can focus on getting as many kills or doing as much damage as possible and still not bring them closer to the objective.[/quote]
Be definition, the entire team could focus on XP farming and still not bring them closer to the objective. The only possible way that even work (which will never be considered btw) is to remove XP from everything except the core objectives that drive the game along. So again, nonsense.

I’m actually pretty disappointed with the poll results so far. Why would people even play a shooter that requires more than just fragging and/or surviving if that’s the only thing they wish to be lauded for? Why value just one of these qualities if you can have them all?

You’re quite clearly failing to understand the question. It asks what should be used to calculate skill, not what should be used to motivate the player.

There’s just so much garbage in that formula I don’t even know where to start.
" (K/D Ratio x objectives completed + overall XP) / hours played = gameplay score"

Why value just one of these qualities if you can have them all?

First the disclaimer, I quite clearly pulled that off the top of my head to give the example that everything should be considered to give an overall ranking as there is no single entity that can be categorised as ‘skill’. Secondly, you said it yourself, why value one when all could be utilised, and for all to be utilised they have to be measured and calculated… XP/min does nothing of the sort. You’re catering to farmers, those that can play the XP game and that’s just a load of crap.

The rest of your post is just nonsense, you say all qualities should be valued then go on to say why a quality shouldn’t be measured… wp.

Again, skill is an amalgamated metric and needs to be pulled from multiple resources. XP/min is about the worst measurement I can think of to do that… ever.


(Humate) #72

I’m actually pretty disappointed with the poll results so far. Why would people even play a shooter that requires more than just fragging and/or surviving if that’s the only thing they wish to be lauded for? Why value just one of these qualities if you can have them all?

In ET and more so in ETQW, there are two types of actions that contribute to a game:

  1. Value actions
    2.**Skill based actions

Value actions - are essentially classed based actions that either directly influence the objective, or allows others to directly influence the objective
**Skill based actions - are actions that prevent value based actions from occurring in the first place, that either directly influence the objective, or allows others to directly influence the objective.

Now to use SC2s system, because I know you like it - once you get to Master/GM level servers, value actions only occur if your skill based actions allow you to. In other words, you cant lemming. You cant revive first and shoot second. You cant build the bridge, unless the other team is dead first.
Bronze to Diamond - Not an issue, you can finish maps without ever needing to shoot anyone. You can all play the primary objective role, and go mass soldier on a destroy objective, as the opposing team is too terrible to exploit class imbalance.

Now none of this actually means that the “scoring system” should reflect skill.
On the contrary, keep it as XP. Heck even have shiny ranks on the side of players names, it doesnt matter.

**Artillery/Airstrikes and spam not included


(Breo) #73

Measuring player skill is a combination of stats.

  • High KDR with low WLR
  • High KPR with low rof weapon
  • High ingame rank, it should be reset every month or something
  • High SPM, allthough it depends on the reward system
  • High battlesense, it should be hard to earn for example less points when clusterpowned
  • High amount of completed objectives

Also I think DPR (distance per round) is fun to add to reveal campers :smiley:


(ailmanki) #74

Kills per minute, if a ELO like system can be done - which works as expected … then that would be great. I think latest vsp stats has a working implementation of that.

A difficult thing to measure I think is the quality of a teamplayer. In ET for example, a medic supporting an engi doing a objective - should gain more XP then another one. Basically the hole XP system is the indicator itself. Detect situations where someone does something for his team…

Another example, 2 engies repairing same tank - get today the half amount of XP each - more or less. Maybe it would be rewarding if - through working together they get a bonus - instead of punishing them… And the medic supporting them also. Similar to two killing the same guy - I heard often enough “you stole my kill” instead of “thanks for support”
Overall I think it is very difficult to make this happen, seems like the game has to be done for this. E.g. I have no idea how to detect if a fops did a good job for his team or not - since often they block their own team; but get a lot of kills.


(tokamak) #75

The teams are too large for ELO to be an accurate personal indicator.

[QUOTE=DarkangelUK;402889]Before we even start, xp/min is so open to abuse and false results that Stevie Wonder to could see that. Take any ET game and see the medics raging away on top with that… so lets stop pretending its a good idea and get your head out of your ass.

(Snip)

By definition, the entire team could focus on XP farming and still not bring them closer to the objective[/QUOTE]

Like I said, that’s an indicator that something is wrong with the xp-distribution. Not with taking it as a measure for skill. It speaks of a profound lack of imagination to say that xp-distribution can’t accurately reflect a player’s worth.

You’re quite clearly failing to understand the question. It asks what should be used to calculate skill, not what should be used to motivate the player.

Yeah I actually do fail to see why what should be used to calculate skill and what should be used to motivate the player shouldn’t be the exact same thing.

If there’s a difference between both then that means players are motivated to do something else than to display skill. Maybe we don’t agree on what constitutes as skill but my definition of skill is the degree in which a player adds value to a team. If yours is something else then maybe you’re simply playing the wrong game.


(DarkangelUK) #76

[QUOTE=tokamak;402903]
Like I said, that’s an indicator that something is wrong with the xp-distribution. Not with taking it as a measure for skill. It speaks of a profound lack of imagination to say that xp-distribution can’t accurately reflect a player’s worth[/QUOTE]

I’m going to be completely honest here, believe me if you want to or not, but I knew that comment was going to be the next post you make. Suggest an already broken feature, then blame the devs when it doesn’t work. Here’s a clue, basing it on XP is broken and won’t work. Give me a detailed XP distribution method for your suggestion and i’ll give you easy ways to break it and cause the system to fall flat on it’s face… the easy part is that it doesn’t take much imagination to break and unimaginative and already inherently broken suggestion… lucky for me, eh.

You also don’t seem to know what adds value to a team, by your definition its who ever has the most XP. Let me refer you to every game in existence that rewards menial repetitive tasks with XP and causes farmers to exploit that process to reach the heights of the XP table.


(Humate) #77

Maybe we don’t agree on what constitutes as skill but my definition of skill is the degree in which a player adds value to a team. If yours is something else then maybe you’re simply playing the wrong game.

Class based actions and skill based actions both add value… but anyone can do the former, you cant say the same for the latter. Thats part of the ET models charm, it doesnt exclude anyone.


(DarkangelUK) #78

[QUOTE=tokamak;402903]
Yeah I actually do fail to see why what should be used to calculate skill and what should be used to motivate the player shouldn’t be the exact same thing.[/QUOTE]

Take your wanabe developer hat off and think like a gamer then you’ll understand. Motivation for skill mirroring an exploitable path to get there is the wrong way to go, by that method of thinking, everyone who wants to reach the heights of the skill ladder will use the dun dun dun path of least resistance to get there, and that ties in with the in-game motivations of XP exploiting to reach that goal.

Once again I’m back to doubting if you’ve even played for any substantial period the games you proclaim to know anything about. XP whoring was always an issue in ET and ETQW, and to a degree Brink so they could brandish their 10 level 20 chars.


(tokamak) #79

[QUOTE=DarkangelUK;402905]I’m going to be completely honest here, believe me if you want to or not, but I knew that comment was going to be the next post you make. Suggest an already broken feature, then blame the devs when it doesn’t work. Here’s a clue, basing it on XP is broken and won’t work. Give me a detailed XP distribution method for your suggestion and i’ll give you easy ways to break it and cause the system to fall flat on it’s face… the easy part is that it doesn’t take much imagination to break and unimaginative and already inherently broken suggestion… lucky for me, eh[/QUOTE].
Like I said, this is short-sighted and unimaginative. You can put all kinds of diminishing returns, context sensitivity and positive and negative feedbacks within the distribution to ensure of that.

You also don’t seem to know what adds value to a team, by your definition its who ever has the most XP. Let me refer you to every game in existence that rewards menial repetitive tasks with XP and causes farmers to exploit that process to reach the heights of the XP table.

Now you’re going in circles.

Class based actions and skill based actions both add value… but anyone can do the former, you cant say the same for the latter. Thats part of the ET models charm, it doesnt exclude anyone.

The value of the ‘skill’ based actions, IE stuff like accuracy, K/D and battlesense is highly situational and frequently detrimental to the team. Someone who refuses to fire from a distance because it messes up his accuracy, someone who camps on a hill with a sniper rifle or refuses to jump in for an important team-member because it messes up his K/D or battlesense is being a burden.


(Humate) #80

The manipulation of stats, doesnt mitigate what skill actually is,
The stats are merely indicators, nothing more nothing less.