I hope there is an mptest with enough time for people really play it (I interpreted wils dev diary to hint toward that, but maybe it’s wishful thinking on my part.)
I know there is a school of thought that says public betas are pretty much a waste. The signal to noise ratio of bug reports is very bad, and you end up wasting time that would otherwise be spent just making the product better. There is also the argument that a buggy beta can turn people off the game, and even if the final game is good, lessen the “wow” factor when it is released.
Those arguments have some merit, but for a complex MP FPS like et:qw, there is a huge downside. Balance issues require often only show up with a lot of play time. When everyone is running around like a newb, you can’t tell if a trick that is dominating is really out of balance, or the other team simply hasn’t found the counter. That means you need time for players to really get comfortable with the game. If that were the only problem, you could still do it with a closed beta, but…
Balance problems also tend to be a result of thinking outside the box, which really handicaps closed betas. When you play a game, you pick up habits from the people around you. If you only play in one small group, you are far less likely to come up with completely unexpected tactics, because you learn from those who have been on the team and doing the same things since day 1. This is a particular problem for developers, because they think they know how the game is ‘supposed’ to be played.
It is a well established fact that players will always come up with completely unexpected ways of using the game mechanics. Some of them will be fun and add to the game, while others will hurt gameplay.
Finally, open betas get the product tested in a far wider variety of systems than any QA lab can. Given how much of the tech in et:qw is new, this seems like a good thing.[/quote]
I’m not really convinced by the school of thought that beta’s are a waste either, espeically if you keep your employed/contracted playtesters on for at least part of the period (say, a few weeks at the beginning and again towards the end).
Even if you totally ignore the players the employees will have a much wider experience ‘in the wild’. If you combine a general forum for players to bounce things off each other with a formal bugreport submission page, a lot of the noise would be filtered out - certainly in the beta’s I’ve participated in anything I’ve found got discussed in the forum and the actual bugreport was a summary of conclusions from that. Sure, I’d bet you still get more crap than a county of farmers can make use of, but it’d be reduced, and the quality of the good stuff would be better.
I vaguely recall it as a common theme in the games that I’ve been involved in testing in, the devs have later stated their mistake with the beta was not giving it enough time.