Community Question: Measuring Player Skill

(Humate) #41

If skill doesn’t manifest itself in impact then it’s absolutely without any value.

In a match (pub) that is of absolutely no value, so is a skill score.

(tokamak) #42

Well if 99.9% of all matches played as well as any system to track the player’s performance within it is without any value then what are we discussing here? Isn’t that just the easy way out?

The discussion is regarding measuring player skill, the outcome has no reflection on an individuals skill when the entire team have an effect on that outcome

And the player is a part of that entire team, therefore the outcome should factor into his skill score. It shouldn’t be the skill score (like w/l or ELO), but it needs to have weight in it. The handlings that reward xp are the smallest scale in which we can measure skill, but as you rightly pointed out, not everything that rewards xp is equally valuable at any time.

When is a handling more valuable? When it enhances the team’s chance of winning. When is a handling not worth it’s base xp rate? When a player is better of doing something more important. How can we know if all these things ultimately where contributing or taking away from the overall performance? By checking whether or not the team actually won the round or not. Is this always the case? No, but it’s the case most of the time so after a few rounds doing useful xp handlings will quickly elevate your score while doing pointless xp-handlings will quickly bring it down.

When the xp you accumulated during a match is modified by the end result then all your performance into the match needs to be relevant to the outcome. Players are free to do as they see fit with only one condition: they need to increase the probability of a victory in the best they can. It doesn’t matter if they do it through fragging, making a select number of high-priority kills, complete objectives, sabotage or support, if it adds to winning then they’ll be rewarded for it.

(DarkangelUK) #43

We’ve been here before dude and you came up trumps, I thought maybe you’d actually have progressed but I guess not. Broken, wrong, incorrect, inaccurate, easily abused, doesn’t reflect skill etc etc. It’s only those new to the discussion that you need to rehash the same old crap.

(tokamak) #44

Fine, fine… those two posts deserve to be addressed.

A player, particularly a new player, needs to get some reward for just doing class stuff like repairing a rear-line artillery deployable. If he spawns beside it this is probably a productive use of his time, but if he has to walk there from the frontlines, it is not. The system has no real way of judging fringe situation and can’t accurately assess his action.

Under my system it can. Players generating xp inefficiently are a burden and will end up losing more often than those who are valuable assets, this is expressed in their modified xp/min. The beginners still get their flat rewards but if they want to have that victory modifier applied more often they’ll have to start working on improving themselves. The current system offers no such incentive, mine does.

And even if it wasn’t the case:

However, we certainly know that a player whos first experience of the repair tool is that it will never benefit him will be less keen to use it again. XP as skill system impairs the function of XP as tutorial/instant gratification system.

Framing xp as a learning tool with only temporary value is just disingenuous, that was never what xp was intended for. XP was meant as a positive feedback for doing well in the game, that’s why it’s so ideal as indicator as well.

No, you’re showing a lack of understanding here. A barebones elo system is indeed less appropriate for very large teams but xp/min does not have the opposite problem. Xp/min can’t be accurate at all unless you mandate and enforce certain player counts.

You forget how malleable this number is. Because it already includes the entire player worth you can keep on adding context sensitive modifiers. If on average 6v6 matches generate say, 35% less xp than 8v8 matches then you apply a 35% enhancer for playing on a less populated server. Actually this shouldn’t just be a score enhancer, it should be a currency boost as well just to make sure players don’t see large servers as xp-farms.

It circumvents having to dictate a particular ideal server size. Even if you want an ideal server size enforced or encouraged you just make the enhancer smaller the further away the admins venture from what the developer thinks is ideal. This makes the enforcement a gradual decision rather than a binary choice.

You are repeatedly deflecting criticism of the system with criticism of other systems, or by saying the same flaws are present in both. This is intellectually dishonest.

I’m merely pointing out double standards. But you’re right, I shouldn’t have to do that as I just offered a solution to something no other system can deal with.

AlsoDAUK said:

All you’re rewarding, as you say, are tangible actions… skill isn’t purely based on tangible actions

That’s another one. He didn’t read it well because putting the tangible actions in the context of the outcome of the match means the intangible (the way the tangible actions are carried out) is measured as well.

That’s the only solution I’ve seen in this thread to deal with this without having to invoke subjectivity and popular opinion (like Sockdog’s jury system).

My point was asking how does your system punish losers which is a feature necessary for a skill rating (like elo). Without the punishment multiplier it doesn’t, and with the punishment multiplier it over-relies on it. Whether you see it or not, you’re guilty of what you claim is an error of relying on W/L.

How much an ELO rating weighs into this is a choice you can make as a developer. I don’t think there’s an objective answer to this. You can just tweak it depending on how much of an issue team stacking is.

And you want to add more measurements and more externalities.

More measurements that account for those externalities yes.

No different from a punishment multiplier.

It’s very different because the W/L ratio doesn’t factor in how much time you spent on each team. You could’ve just entered the server and lost before you spawned for it to be counted.

You’re wrong, it still encourages you to stop “wasting” time and switch to the winning team or even leave the server as soon as possible, regardless of how much time you’ve already sunk into the game.

A simple solution to that would be to keep counting the time after you left until the end of the match. That may punish disconnecter too harshly though. Wouldn’t want people with ruined kidneys because they preferred to stay behind their pc instead of taking a leak.

So you are pretty confident that you can ignore the fact the system doesn’t measure actions that don’t generate XP and is blind to their relative contribution. Why?

Actions that don’t generate xp but are still useful to the match still enhance your chances of getting that victory modifier. In the same way, lots of intangible stuff is frequently expressed in a higher capacity to score xp. High accuracy, mobility, situation awareness all lead to more kills, more objectives and more awesomeness. There’s no need to track these things because when they matter, they’ll be contributing towards a higher xp or a higher chance to win, either way this system picks up both and compares them.

DAUK continues from here:

Nothing you’ve mentioned takes that into account because of your apparently made up version of what’s considered skill. We’re measuring individual skill here, it sounds like you only class it as worthy if the outcome was a win, anything else was futile and pointless because it didn’t result in a victory.

Actually, that’s what W/L would do. My system would only reward you with less if your outcome didn’t result in a win. You would still be inclined to keep racking up that xp because in the end it’s only a chance of getting that modifier applied after a match.

What it certainly doesn’t do however, is reward players for pursuing their own agenda and achievements. A player hell-bent on improving his accuracy or w/l or laps around the valley will score incredibly low in this system.

Sorry but a large win modifier won’t encourage team stacking? Steam rolling the opposite team, easy rack of XP then a big reward at the end is just as much encouragement

It certainly encourages players to stay on the winning side of the team. I already explained how to deal with defectors above. Still, players are human and being on the losing side will sometimes be frustrating enough to just take that hit on your xp/min and leave the match and go do some yoga or punch a wall. In that case, when the losing team is losing players you can do that cute ETQW proposition of rewarding a player to move to the opposite team, not for a flat reward but by letting him keep his victory modifier in the end.

Medics at the top most of the time doesn’t sound like ETQW did a pretty good job at all.

ETQW did a fairly good job, Brink dropped the ball. Either way, getting the xp system as accurate as possible is something to strive for, this system only makes it even more important to get it right.

Your system doesn’t take all factors into account, which has been said I don’t know how many times now and you’re still failing to see this. Again we’ll go with the dyno plant example. 2 guys, 1 goes the long way, announces his presence, gets into gun fights, plants the dyno and wins the game in 2mins losing health and team mates along the way. 2 uses movement agility to by-pass all of that, doesn’t alert the enemy, no gun fights, no damage taken, catches the enemy off guard and wins in a 1 min.

If they both managed to successfully plant the dynamite then they’re equally successful players. The latter can be considered more prestigious, but again, prestige alone is a pretty hollow pursuit.

You can say that the latter has higher chance of succeeding. I would agree, in that case if you repeated this scenario a few times then you’ll see that player A would lose more often than player B which will show in their score.

Aren’t we talking about ranking individuals? 2 players on opposite teams play exactly the same way, earn the exact same XP during gameplay, the rest of the team let one down and he loses therefore doesn’t get the outcome modifier. Now you’re using other peoples performance to determine an individuals skill. Really?

Really. Two equal players will receive the same modifiers over time because their contribution to winning is equal. This remains a team game with rather large teams, so the only way to do it right is to tie the individual contribution to the collective result rather than exclusively looking at either one of them.

(DarkangelUK) #45

Sorry I skipped all that, it’s broken and doesn’t work. Inherent flaws and all that jazz.

(tokamak) #46

That’s okay.

(DJScream) #47

I’m beginning to hope SD doesn’t use too much resources on this and just focus on getting the gameplay absolutely perfect this time. Public games keep the normal XP thing and clan matches/duels are decided with wins as always.

I have a very hard time believing the public skill measuring will actually work in massive public games with people coming and going and the normal cowards stacking defensing teams etc. As said before the XP exploiting will just be so easy to do. There’s always some guy that finds a good way to get lots of XP and then everyone follows.

I haven’t seen any real solutions in here yet to believe in it, only some theories that don’t exactly tell me anything in practice. I of course don’t mind if someone proves me wrong and finds a solution for those massive public games.

(tokamak) #48

Do you have any specific points of contention regarding the theories posted here?

(acQu) #49

Hm, wasn’t ET Battlesense just kills mapped to XP ? Hm, must watch.

Just wanted to post here, because it is zi super awesome thread :smiley:

(tokamak) #50

Battlesense was awarded for damaging and or being damaged every period of time. In short it mostly rewarded players who were constantly in the fray of battle. Players who kill from a distance or who focus on covert missions or objectives will score considerably lower on battlesense.

(Senethro) #51

ugh what the hell

am i still expected to have a train of thought on this matter? how many weeks has it been

perhaps your FACE is context sensitive, and the context is my fist!

(edxot) #52

[QUOTE=Senethro;405437]ugh what the hell

am i still expected to have a train of thought on this matter? how many weeks has it been

perhaps your FACE is context sensitive, and the context is my fist![/QUOTE]

VOTE

1 : hangover

2 : still drunk

3 : born troll

4 : none of the above

5 : a combination (please specify)

(ailmanki) #53

5=1+3
:stroggtapir:

(Senethro) #54

Oh I’m sorry, I like replies to my posts to come sooner than 7 weeks later. Its something of an ambush to wait until a guys forgotten everything about a discussion and then do the quote butchery on his text. A cursory glance over tokas post shows that toka has forgotten much as well and has readopted a bunch of wrong position due to only reading the most recent posts.

This is the overlap between the fieldwork season and the report writing season for me. I really don’t have time for this or Tokas antics. His system still over relies on W/L, he is completely unaware how recent games with stats systems have treated time spent on both teams (i.e. solved his criticism LAST YEAR), and will continue to pile on layers of meaningless numbers (%age multipliers of XP based on team size) to cover the holes in the previous layer.

ollies outy

(tokamak) #55

I’m not expecting you to be completely up to speed with an old discussion. And although I would appreciate some comments on any points you see fit you certainly don’t need to respond right away. You two put effort in those old responses and it it’s not decent to have them be left on the table (especially considering the vacuum that followed really didn’t get filled with any substance at all).

That said, I don’t feel any need to take those broad objections seriously. Just take your time to point out any specific problems whenever you feel like.

(_KaszpiR_) #56

Suggestion:
Good old time spent to fulfill mission objectives, like in official league matches.

For me the most important was battlesense, it was in ETQW.

But frankly speaking each class scores differently, let me show the example on TeamFortress 2 - playing as soldier is just very hard without medic on certaain maps, while scoring as engineer is quite straighforward - but for example spy gets double scores for every kill.

I think sticking in certain radius around next objective is important. Some classess don’t have to score just because they kill people or make objectives - in example someone someone just can lay suppressive fire to block enemy team from advancing or is a spawnpoint for other teammates/recon.

(DarkangelUK) #57

I chose a combination of metrics. I wrote up some silly convoluted formula quite a while ago when discussing this about Brink, gonna post it here too. It’s not exactly accurate, but at least gets my idea across.

Couldn’t they create some uber math formula for determining a skill score or something?

(K/D Ratio x objectives completed + overall XP) / hours played = gameplay score

So it computes overall performance, examples over an 18hr period of Brinkage…

So an average player that does objectives and can hold his own in a firefight would get…
(0.9KD x 100 + 12000) / 18hrs = 671 score

Now take someone that can’t aim, has played for the same time but can do objectives well…
0.3KD x 180 + 13000 / 18 = 725 score

Since they’re both lending to the game there’s not exactly a great spread between overall score

Now take someone who doesn’t play objectives much at all and just wants to kill stuff and has a cracking aim, but isn’t exactly about the teamplay or objective. He’s not gonna get as much XP or help the team as much as he could, but on shooting skill alone he has a decent score…
2.7KD x 30 + 9000 / 18 = 504

Take someone **** in all areas, reaching the lower newbie scale
0.4KD x 30 + 4000 / 18 = 222

Then someone awesome in all areas, upper pro scale
2.2KD x 190 + 19000 / 18 = 1078

Even though you don’t kill much, the completed objectives brings the score up. But a good KD ratio with little completed objectives will get your score decent, but could benefit from helping the team.

All of the above is totally made up on the spot and probably worthless for this exercise.

(Indloon) #58

It could follow SplatterLadder kind rate calculating which only uses score( but can use kills and deaths values ) to calculate and it is effected by players playing in the server, depending on their rate.

(Cep) #59

This is a good idea and one I would vote for it rewards team play over kills but still rewards players for keeping enemy numbers down. I reckon deaths should be penalised againsr kills with a greater ratio. For example a kill is +25 points yet a death is -100 the reason for this is to stop the lemming runs that Brink had in abundance

(Humate) #60

Cool question badman.
I dont believe theres a concrete stat or formula for skill.
But these are things that are common with the top players of say a game like ETQW.

  1. Kills per Minute
  2. K/D
  3. Head shot ratio
  4. Accuracy

Again these stats are basic pointers.
Theres also a lot of subtle stuff, that cant really be tracked.