Community Question: Measuring Player Skill


(MoonOnAStick) #401

[QUOTE=shirosae;404218]This isn’t like some resignation that we need to rely on people to balance games. It’s an active choice; I want people to balance games. Some people are losers who don’t get that games are most fun when balanced. No amount of XP reward or game director is going to change that. At least this way, the game starts to suck when stuff is unbalanced. If you had an automated system in place which was effective at balancing the game so that matches were always tied or nearly tied, would there be any reason for players to even try to balance the game?

I think the most you can do is try to design your game mechanics so that there’s the least benefit from doing obnoxious stuff.[/QUOTE]
Agreed. Maybe there is scope for some subtle extra help beyond base turrets, but excessive rubber-banding is likely to be a big turn-off.

Random selection of post game stats sounds good. How about a gentle hint on how balanced the teams were? An explicit xp factor (on a scale of 1-5 say) indicating the relative team experience*. Or a post game heat map showing player kill locations? Fairly easy to knock-up and would highlight the time spent at each objective/spawn (there was a good looking ET spectator mod in a recent thread…)

I’m definitely sympathetic to the goal of preventing 20 minute spawn camps, which are more of a problem in ET than some other FPS game types but I think education, rather than intervention, is a better way forward.

*obviously this brings us full circle: How do you measure team ability? Although I facetiously suggested a regression model earlier, qw-style xp (or simply total game time) would probably be well enough correlated for pub games.


(Humate) #402

In short, at the end of the match you get to vote for your Most Valued Player and maybe even a least valued one. This would give some real life recognition to individuals from their team or even the opposing team. IMO it allows you to recognise even the smallest action and reinforce it with a player. Say for example that when you was doing a plant and this guy just comes out of nowhere an keeps the enemy off your back long enough to plant and win the match. You can recognise that, say how much it helped. Now that guy may have been nowhere on the leaderboards but he gains recognition for a positive action during the game.

Would players buy into it or just abuse it. I’m not totally sure but if we’re going for utopian ideas then this is something I’d love to see built upon the old GG at the end of a match.

The wonderful bonding experience must arise and develop naturally SockDog. :magicpony:

Shuffle is probably only useful to break an already obvious team stack or poor team configuration and it does so by only randomly moving players

Shuffling could be done by random, ranks and xp if I remember correctly.
But the effect of it was random.

If you had an automated system in place which was effective at balancing the game so that matches were always tied or nearly tied, would there be any reason for players to even try to balance the game?

And we have a winner.

edit: You must spread some Reputation around before giving it to shirosae again :confused:


(SockDog) #403

[QUOTE=MoonOnAStick;404235]I’m definitely sympathetic to the goal of preventing 20 minute spawn camps, which are more of a problem in ET than some other FPS game types but I think education, rather than intervention, is a better way forward.

*obviously this brings us full circle: How do you measure team ability? Although I facetiously suggested a regression model earlier, qw-style xp (or simply total game time) would probably be well enough correlated for pub games.[/QUOTE]

In the view of random pub games do you have any ideas that could educate players? ETQW tried with XP bribes to swap teams, shuffle votes were more often than not ignored etc.

And if you strip away the meddling you are left with a Team ability system. This is I think the crux of this topic, do you focus and reward the individual or the team? Which impacts and promotes better team play? Using a team to base score, rewards, xp etc IMO promotes people to work together for a team goal, there is no reason to showboat for some personal glory stat or farm xp while screwing over your team. That of course doesn’t mean that you can’t reward players on a personal level with things like stats, kudos and highlights/achievements.

We must force the puny humans for their brains and will are weak.! :slight_smile:

Shuffling could be done by random, ranks and xp if I remember correctly.
But the effect of it was random.

Again, it’s not really a solution is it, just an attempt to say you tried something even if it’s just as likely to give a bad result. It also brings up the question of people wanting to play with friends and quitting/resisting shuffles because of it.

And we have a winner.
edit: You must spread some Reputation around before giving it to shirosae again :confused:

Conversely is there any reason to play a game if the random nature of the pub servers means more often than not you’ll get much less enjoyment from the game than you would in a match where handicapping of some description is in place. Again the goal is to create and environment that is challenging for both teams by bridging large gaps in the skill of each team.

I don’t miss the point being made, I just think it’s being made while ignoring the many other factors we currently accept because that’s the way they’ve always been. If I was to got back 20 years and talking about MP games and how you could join a server and be ****ed over x times out of 10 games just because of random team skill levels would you be so eager to accept it and consider that a fair way to do things?

That all said I do see the issues and concerns and think that it may do better as an option or mode of play in itself. Hopefully as development tools become more powerful we’ll see a lot more smaller indie devs mix up this whole thing and experiment a bit more freely. Of course if SD was to do it that would be great also. :slight_smile:


(Humate) #404

We must force the puny humans for their brains and will are weak.!

I was looking through some old game vids, and I found a blackops one, where I had only recorded the end of match result, along with the chat. I had the entire enemy team, as well as the players speccing me calling me a hacker in chat. But there was one player that expressed his delight in seeing a k/d of 30 -0, and I thought that was pretty cool. Not because he complimented me, but that he could have easily went along with what everyone else was saying. :magicpony:

Conversely is there any reason to play a game if the random nature of the pub servers means more often than not you’ll get much less enjoyment from the game than you would in a match where handicapping of some description is in place. Again the goal is to create and environment that is challenging for both teams by bridging large gaps in the skill of each team.

I understand the goal, I just disagree with the method.

Again, it’s not really a solution is it, just an attempt to say you tried something even if it’s just as likely to give a bad result. It also brings up the question of people wanting to play with friends and quitting/resisting shuffles because of it.

It does a reasonable job I think, but as I said - prefer to just not take the game too seriously, and offer the enemy team some lee-way.Obviously the impact of taking the foot off the pedal varies from player to player, but I found that to be the best solution. The second thing is, there were craploads of faction loyalists not willing to switch teams, that would F2 shuffles even when they were getting smashed. Lets just say it didnt do any favours for GDF.


(SockDog) #405

Mmmm makes me wonder about devaluing. If we always had good games and always have good sportmanlike players would we appreciate it less than we do on the rare occasions now? :slight_smile:

I understand the goal, I just disagree with the method.

Fair enough. I disagree with the general apathy in the MP community to do anything about it. :slight_smile:

It does a reasonable job I think, but as I said - prefer to just not take the game too seriously, and offer the enemy team some lee-way.Obviously the impact of taking the foot off the pedal varies from player to player, but I found that to be the best solution. The second thing is, there were craploads of faction loyalists not willing to switch teams, that would F2 shuffles even when they were getting smashed. Lets just say it didnt do any favours for GDF.

I’m going to be a little picky here but I find it a bit contradictory to say you don’t take the game too seriously and can live with poor team composition but the thought of a consistent challenging game over a period of 20minutes isn’t serious enough. It’s a perspective shift, granted, maybe even a different game (mode?) but ultimately you could get a good 20 minutes out of that on a more consistent basis than you would in playing five games traditionally.

And yes, there are many reasons why shuffles, pleas, bribes and any other means of cajoling players to balance things out just failed on a consistent basis. Not trying to win an argument here with this question but how do you feel teams should be balanced in a consistent way? I think we can all agree that the current methods are broken and even the best skill point analysis isn’t going to be reading a player’s motivations and intentions at the point of joining a game.


(Humate) #406

When I say I dont take the game seriously, I mean I dont usually put a lot of effort in pub matches. Win or loss doesnt bother me. I can live with poor team compositions - on both sides of the spectrum. The team dominating, and the team getting dominated - happy to play on both. I can also live with evenly balanced teams with games going down to the wire. Yet I feel if it does evolve into a tit for tat, I would rather it be the result of two teams going at it, than the game offering its 2 cents to the losing team midway through. Again just my opinion, I respect that you see it differently.

Not trying to win an argument here with this question but how do you feel teams should be balanced in a consistent way?

Good because I’m not arguing with you. :slight_smile: As for your question, I dont have a problem with balance and the nature of how pub games are played. But if I did, I would use Biosnark’s idea. Thats allowing the dominated team, to forfeit the match with their own vote, and have the game enforce a shuffle on the start of the following map. But I would use a skill rating, instead of XP/Ranks/Random. This means dominating teams that want to avoid the shuffle to play with their friends or play the faction of their choice, will less likely spawn camp/trap the opposition. The dominated teams, have to decide whether playing with their friends or their faction is more important, than an evenly matched game.


(edxot) #407

this talk has got a little confusing because you ask 2 questions, player skill onthefly and ranking system. people will answer one of those while thinking about one game (may it be ET, ETQW or brink).

concerning etqw

ok, let face this fact: you wanted a game where teamplay would be important. being the best shooter would not be the most important factor to win.
you did not fail, but now you want the other way around and see who is the ultimate frag machine ? WHY ?

unbalanced games are bad if you are thinking about kills and victories. but when you are focusing on xp, its quite the oposite. its easier to get lots of xp when your team is weaker than the other.

is xp a good measure for skill ? no way, and i can give a few examples of that, but this one is the first to pop into my mind:
i played et for ages. when i started, i had a pretty bad computer: laptop with onboard gpu and wheel mouse. so during that period i only could get xp in easy tasks and i learned them pretty well for the usual 6 maps of africa and europe campaign (mostly 12 v 12).
a few years later, i already had a good computer, and i got into this strange africa campaign. when it started i had to choose my class and i saw my team (defending) already had 11 medics ! meaning this is going to be the easiest game ever. i would normally choose medic but as a good xp whore, i remember you get more xp if you do things your team isn’t doing. so i went in a mixture of 2 classes: field ops and engineer (spawning one life with engy and the next with field ops). the results were better than i would ever imagine. i was used to play those 90 minutes and getting around 900 xp in the end (the usual for the top player in that campaign). but this time i got more than 1100 xp !!! never again i would make such score in that campaign. 900 maybe (depending on the teams), but never more than 1000. it was my fault ? no way. the medics did the job, not me. so xp can be really missleading.

concerning etqw again:
as i said before, playing for xp can save you form the frustation of having snipers, field ops, or vehicle whores getting all the kills. of from losing to an obviously stacked team. it may have some problems but i believe it is the best thing to track (among all other suggested). but you must find a way to force people to play ground and without vehicles, i mean, play all maps and not just their favorite ones. i remember when i was trying to climb the etqw ladder i would never play those infantry maps like salvage or volcano. and the guy in 1st place was even worse, he played only 3 maps.

i think you could give my idea some try before going to next game: 24 ranks (invisible or not) - 12 maps times 2 sides. forcing people to play all maps and both sides. you just need to fix the aging system (it must be 24 times slower). and you also need to calculate the global rank based on the positions in the other ranks (those would be xp/min ranks for some map/side).

if my english in not good enough to understand what i am saying, please say something, i will try to explain using other words.


(tokamak) #408

Of course they would abuse it. Friends will vote for friends, clans will vote for clans. You’re adding highly subjective element into the mix that won’t only lead to heaps of grief, it will also cause people to completely disassociate from this popularity contest (the dude playing with the most friends will be the top player) making any rating worthless. You know that, everyone knows that.

But, for the sake of argument: Let us ignore the tedious process bothering people at the end of a game, the protests that arise and the problem of incomplete tallies because of all the people taking a piss or getting a coffee at the end. let us ignore the human factor and say that the votes are determined by completely honest 100% objective human referees with a high understanding of the game. As you say, complete ideal circumstances.

Then the whole thing still doesn’t fly because these people operate from an incredibly narrow point of view. They can only judge what they see. Your skill score suddenly stands or falls by the amount of observers present at your feats. You hacked an entire deployable park and backstabbed the dude disarming the charge on the final objective? Tough! The guy who revived four team-mates gets the vote because his work had an audience.

And then there’s the utility vs prestige issue which is something I think is the crux of this debate. I value utility but some folks here value prestige. That’s the only reason why anyone would ever like to see stuff like k/d or accuracy incorporated, that’s the only reason anyone ever wants to see human judges holding scoreboards after a massacre.

You want to see the most spectacular player rewarded, I want to see the most effective player rewarded. These CAN overlap but that’s not a necessity.

Don’t get me wrong, human judges WOULD indeed be the best way to rate the entertainment value of a player, there’s very little that would ever beat that. But when you truly want to value a player’s contribution to the outcome of a match then you’re going to need an objective, score based on tangible handlings.


(tokamak) #409

Quirky stories each round are already a thing in CS: GO

MVP= Dude that was the most important. This is usually the guy with the most kills unless someone completed the objective.

Under the MVP are the funny stories. In these examples there’s only one per round, but multiple are possible.

Now of course, an ET-style game is far more nuanced than a CS game where the kills and the objective are truly the only things to value. An ET game also has much more possibilities for interesting tales.

Anyway we need this. Regardless of the type of skill indicator, this is something that makes seriously addictive. We need one per match and we need a general collection in player profiles and a (automatic) collection of the most extreme stories each day, which then proceed to go on into the hall of fame.

Seriously the promise of reaching that hall of fame alone is already enough to make players give everything to reach it.


(Humate) #410

Anyway we need this.

No we dont,


(tokamak) #411

Yes we do,


(Humate) #412

No we dont


(tokamak) #413

Well, can’t argue with that. :smiley:


(shirosae) #414

[QUOTE=SockDog;404224]In short, at the end of the match you get to vote for your Most Valued Player and maybe even a least valued one. This would give some real life recognition to individuals from their team or even the opposing team. IMO it allows you to recognise even the smallest action and reinforce it with a player. <snippety, but I did read this stuff>

Would players buy into it or just abuse it. I’m not totally sure but if we’re going for utopian ideas then this is something I’d love to see built upon the old GG at the end of a match.[/quote]

I wouldn’t be opposed to this. Sure, people would abuse it, but if it doesn’t directly affect game mechanics then I don’t see it doing much more damage than the usual drama anyway.

For me, the fun is somewhere in that struggle, trying to leverage all of my ability into all of the interplay of game mechanisms happening all at once. I need to try to play to the best of my ability, cause it’s in that stretch that the enjoyment comes. I can handle a little randomness, like some spread on weapons tied to movement/stances, but very quickly it overpowers that struggle.

If you could have a system that would dynamically alter things to offset player ability, the best way to play would be to play intentionally badly, let the dynamic system settle, and then use that temporarily altered gameplay to push at a critical moment. And then what? Add some random seeding to offset the balance rate? Make the balance swing further back? I just want to play a game on a level field.

[QUOTE=SockDog;404224]I get that and it’s probably the biggest hurdle for something like this to overcome, it’s making me think more and more that this would need to be trailed alongside something else. Where people can opt into the experience rather than feel force and rail against it regardless of how it plays.

I also don’t think this should be something that allows a poor team to win. It’s more a handicap system based on live data. Reactive matchmaking if we want to make up names. It should in the most part bring teams to a closer level so that a game can be played out without either side being impacted by real world practicalities.[/quote]

Don’t get me wrong, though. As an idea I’d be interested in seeing how it played out, and maybe testing it. It’s just that a system like that wouldn’t hold my interest for very long.

Isn’t that basically what people have been suggesting with ELO, except that they want a ‘win’ to be modified by comparing total team score, rather than altering the game as it’s being played. That hinders people who want to stat-pad by stacking teams by killing what they earn for a win, but doesn’t screw with balance.

I think gameplay unlocks are a load of pants, frankly. ETQWpro was kinda okay though, in that the unlocks were pleasant tweaks without being overpowered. I guess that was partly due to most promod servers running the reduced spread, making assault rifles the best overall weapons, and alternatives niche weapons.

I guess if you wanted to take something minor like that, and weight the upgrade points against team composition, so you got unlocks faster against a good team, it might work out okay.

So a quantised system? How would you ensure that most games were balanced in the saddle points, and didn’t continually tip back and forth between two balance levels?

How about you have a system that measures a bunch of key skill indicators as the player stays on the server as described in the OP, and then used that to weight the number of votes required to call a server shuffle ordered by ELO rating?

If the server skill ratings are unbalanced, and the sum ELO ratings of both teams are lopsided, it takes fewer people to shuffle by ELO. If people leave because they can’t bully the server into staying stacked, then so be it; they’ll be leaving a lot of games and not having fun.

The biggest issue I guess are people trying to pull sour grapes shuffles, though if the system requires ELO to be unbalanced by so much, you could guard against that nonsense by pre-shuffling by ELO so it’s already balanced. Maybe that’s a feature of sorts? You can guard against sour grapes shuffling by preshuffling to sort ELO scores out, so if you’re willing to shuffle you can stick it to the bad losers.

It also screws with the ‘won/played’ metric. I guess you then need to start allocating fractions of wins based on time spent on each team, stuff accomplished whilst on each team etc.

Ironically I see the same problem, I just think the solution is to encourage players to not be dicks, rather than to filter them so that their dickishness has no effect on the game. Or rather, to give the players who aren’t dicks easier ways of finding each other by shuffling stackers into ragequitting.

Yeah, I’d be interested in seeing how it worked out. If SD are trying to develop a PC FPS as a service rather than fire and forget game, it’d be interesting to see a test bed game mode in which they tinkered with this stuff on a game with real humans playing continually.


(Patriotqube) #415

By looking at the polls made by SD it seems they all have a clear message :slight_smile:

We <3 the ET style so ad that to next game, if it aint broke dont fix


(tokamak) #416

I wonder how much of that vote is for ‘ET-style’ and how much is for ‘battlesense’.


(DarkangelUK) #417

I think most of that is just because it says ‘ET’ in the option, probably could’ve said ‘ET phone home’ and it still would’ve got votes.


(Ashog) #418

Win/Loss ratio. The rest doesn’t matter for me. After certain amount of statistics accumulation for each player I am sure that with this parameter it will be clearly visible who is a good player who is not. Takes longer time to gather this kind of statisctics though to be meaningful.


(SockDog) #419

This promotes team stacking and quitting.


(tokamak) #420

And even if it didn’t then the amount of matches required before any significance is reached is impossible. Win Loss ratio is already a questionable statistic for 1v1 games like chess and Starcraft (due to the match making influence) but within the context of 8v8 or 12v12 it becomes completely meaningless.