Community Question: Measuring Player Skill

(DarkangelUK) #281

[QUOTE=tokamak;403539]
Really if you want me to take this argument seriously, then by all means show me one example where a player needs to be rewarded for doing something else than contributing to the win. When does this even become relevant let alone preferable?[/quote]
All you’re rewarding, as you say, are tangible actions… skill isn’t purely based on tangible actions. You’ve had examples of distinguishable gameplay with relative outcome. Both achieving the same outcome where one is efficient and one is messy. Nothing you’ve mentioned takes that into account because of your apparently made up version of what’s considered skill. We’re measuring individual skill here, it sounds like you only class it as worthy if the outcome was a win, anything else was futile and pointless because it didn’t result in a victory.

Besides, W/L alone encourages team-stacking like no tomorrow. Players will be consistently keeping score of which team is about to win and will try to join the favoured team as soon as possible and failing that, leave the game (it doesn’t prevent the loss, but a game that you’re certainly going to lose is a waste of time in your W/L agenda). XP/min doesn’t have this problem because it accounts for how long you are playing, that’s the weight in your average. It doesn’t count matches, it counts time played. Trying to disconnect and hop to servers where you’re about to win doesn’t help because the time played in that victory is to short to matter enough.

Sorry but a large win modifier won’t encourage team stacking? Steam rolling the opposite team, easy rack of XP then a big reward at the end is just as much encouragement, if not more as the progress rewarded during the match for easy kills, easy objectives will rack up. If anything the XP min will encourage the user to join the winning team as they’re already going to get a little XP for joining the losing team as it is, why not grab as much as you can by joining the winning team?

All possible tangible actions, and yeah the balance between these things definitely needs some thought. Brink messed it up but I think ETQW did a pretty good job at giving each class an equal opportunity to gain the most xp as well as having the right values for each individual action.

Medics at the top most of the time doesn’t sound like ETQW did a pretty good job at all.

The moment a player pursuing the skill indicator leads him to contribute to a victory in the best way he can, and the moment a player contributing the best way he can leads to being recognised by the skill indicator, that’s when this mission is accomplished.
I define skill as doing whatever is the best thing you can do in order to win. This means that no matter how impressive and skilful an action is, if it doesn’t bring your team closer to a win, it should be discounted or even punished.

Your system doesn’t take all factors into account, which has been said I don’t know how many times now and you’re still failing to see this. Again we’ll go with the dyno plant example. 2 guys, 1 goes the long way, announces his presence, gets into gun fights, plants the dyno and wins the game in 2mins losing health and team mates along the way. 2 uses movement agility to by-pass all of that, doesn’t alert the enemy, no gun fights, no damage taken, catches the enemy off guard and wins in a 1 min. You don’t class movement as tangible yet this skill led the payer to contribute to the victory the best way he could and was less of a liability to the team. Player 1 is ranked higher than player 2, isn’t the opposite if what you just classed as a skilled player?

When does an action bring you closer to a team? Whenever it leads to more favourable player/player and player/objective interaction. Why these interactions? Because these interactions are often the most valuable things you can do in order to win a match. What about the few cases they aren’t favourable to win the match? Those cases are discouraged by the outcome modifier.

Aren’t we talking about ranking individuals? 2 players on opposite teams play exactly the same way, earn the exact same XP during gameplay, the rest of the team let one down and he loses therefore doesn’t get the outcome modifier. Now you’re using other peoples performance to determine an individuals skill. Really?

(ailmanki) #282

Well I manage a server with enemy-territory, and I thought also about writing a lua stats modul for it…

While the mentioned skills, like timing (waiting somehwhere), movement, evading enemies, are skills, which can have a great importance, they mostly can be countered by good teamplay. So I wanted to make a system, which would favor teamplay, moving together and similar things.
Unfortunately lua is very limited in ET, and I can only see kills, deaths, and some pickups - currently even objectives are not really easy to get infos from. And I can see weaponstats, like bullets fired, teamkills and so on.

Now my plan is, I record for each kill, the position, and create something similar as Lithium mod made, with this I can programmatically tell where the front is - if there is any. Now, it should be possible to tell if somebody is going behind front - and such things, and reward that appropriately. Its also possible to tell if a sniper is sniping. Now not having real access to objective info makes this a lot easier, since there are so many different maps - there would be a need to configure each one. So I can keep it relative simple and only look for kills.
Armed with this kind of statistics, it should be possible to create a hole range of “rewards” for different actions - still difficult but kinda interesting. The fun part is, each reward would also give a XP multiplier for the current actions. So at end of round new awards can be shown…

While defining the skill for shuffling, I would use the xp/min and some ELO like system for kills. Skill should be between a normalized value 0-1. Starting at 0.5. edit (and I guess a map modifier is needed - defining difficulty for attackers)

This is still no solution for various things, as I wrote earlier - I think the game would have to be done from scratch for XP to meet all requirements perfectly.

(Stroggafier) #283

In my sophomore year, the math prof gave long and complex assignments consisting of multiple problems that needed solving. She marked (scored) the work with either 0 , 1, or 2.

Although fail, pass or perfect, might seem woefully inadequate to assess complex abilities (skill), it turned out very accurate. How? one might ask. The answer is that these assignments were given weekly, and that over a large sample size, simple systems are very good at predicting complex associations.

Apply this concept to games - i.e. no complex system of calculations is needed to assess skills, as the sheer number of games played will result in a constant striation using a simple skill measure.

(tokamak) #284

Beautiful. I love that. And that may be exactly why the flat xp distribution may not have to be overly detailed after all. The one necessity is repetition and a shooter has an enormous amount of interactions that all stack up.

(ailmanki) #285

[QUOTE=Stroggafier;403547]In my sophomore year, the math prof gave long and complex assignments consisting of multiple problems that needed solving. She marked (scored) the work with either 0 , 1, or 2.

Although fail, pass or perfect, might seem woefully inadequate to assess complex abilities (skill), it turned out very accurate. How? one might ask. The answer is that these assignments were given weekly, and that over a large sample size, simple systems are very good at predicting complex associations.

Apply this concept to games - i.e. no complex system of calculations is needed to assess skills, as the sheer number of games played will result in a constant striation using a simple skill measure.[/QUOTE]
The complexity going on she solved with her brains and put it into simple 0, 1 or 2.

(Humate) #286

I define skill as doing whatever is the best thing you can do in order to win.
.

Unfortunately, this is why your score falls apart.
I would have a re-think about what it actually is, if you want people to take your score seriously. :slight_smile:

(SockDog) #287

[QUOTE=Stroggafier;403547]In my sophomore year, the math prof gave long and complex assignments consisting of multiple problems that needed solving. She marked (scored) the work with either 0 , 1, or 2.

Although fail, pass or perfect, might seem woefully inadequate to assess complex abilities (skill), it turned out very accurate. How? one might ask. The answer is that these assignments were given weekly, and that over a large sample size, simple systems are very good at predicting complex associations.

Apply this concept to games - i.e. no complex system of calculations is needed to assess skills, as the sheer number of games played will result in a constant striation using a simple skill measure.[/QUOTE]

I don’t think three posts could have been more ideally positioned. :smiley:

(MoonOnAStick) #288

I don’t know SockDog. If there is one thing I’ve learned from this thread it’s that you can only trust the numbers (youtube).

(tokamak) #289

Yeah yeah been there done that. I’m a game-theory nutter. It’s a great documentary. It came up in our environmental economics course which was all about making seemingly intangible values tangible.

I wrote a paper about Barry Schwartz who drives the point home much better than The Trap:

//youtu.be/lA-zdh_bQBo

The thing is, xp system doesn’t treat people like just self-serving automatons. We treat them like self-serving automatons that want to be recognised and appreciated.

We’re not just handing out meaningless currency that they can spent like with money. Being wealthy doesn’t make you virtuous because you could’ve gotten it through any means. XP however has the chance to become virtue expressed in currency.

(I’ll get back to the rest of the comments for now I’m having waaay too much fun with SPSS for my summer job :-/)

(Humate) #290

^ But the recognition and appreciation isn’t warranted.

(DarkangelUK) #291

When did the subject of the thread change from accurately gauging an individuals skill level to stroking their ego giving them a pat on the back?

(edxot) #292

i think there is one improvement for the current ladder system that would work nice. people will try to climb the ladder by playing the maps where they can score better, and leave server whenever one of the other maps arrived.

my suggestion is this: create 2 ladders for each map (one when people are attacking, and one defending). use this information to calculate a global ladder. i mean, the average position among all previous ladders.

this way, people will have to do something else than just flying. because in some maps there are no flyers.

(Humate) #293

Player recognition and appreciation is what tokamaks score is about, for the most part.
But what he doesnt realise or… may realise and just not willing to concede is, recognition and appreciation by the game is meaningless when the playerbase doesnt agree with the game’s assessment of its players. Creating your own score,that specifically pats you on the back is a bit like creating a robot that tells you it loves you every morning. Maybe some people are cool with a game telling them they are awesome, i dunno : /

(DarkangelUK) #294

Sounds like he’s doing it purely for selfish reasons and not actually for accurately measuring player skill… like the thread title asks.

(edxot) #295

concerning etqw i think there are some server side configurations that must be equal in all servers.

in some servers, you get xp for choosing the right mission. it must be equal in all servers to make things fair.

allowing admins to reduce number of slots isnt fair either.

(Humate) #296

All in good fun though! Regardless of intent, its a good read. :slight_smile:

(edxot) #297

some way to avoid auto-decoy macros would also be very apreciated. thanks in advance from all rocket noobs.

(Setlec) #298

the perfect way to mesure a player is to vote to that player. say that a game of 12 players, likeit was on the pinktacobar’s etqw was online, whom participate for a whole campaign (3 maps) vote for someone of their choosing for being the best player. the criteria belongs to the voters and not by the KDR/KPM/SPM/WLR/WTFotheracronymyouwant.

I doubt that a soldier would gain his medals by his ‘Kills per Mission’ during a war time.

(DarkangelUK) #299

A popularity contest? That’s not accurate at all o_O

(Setlec) #300

popularity contest has nothing to do this… if you go because someone is popular then that would mean that you aren’t smart enough to judge someone by his action… unless your team mates are truly zombi like smart.