Jul 21, 2010
For many of us the mystery of what goes on behind the gilded doors of GW are filled with rumor and conjecture. I have always been surprised by how few Interviews and discussions are conducted by members of GW design teams. It leaves the public with so many unanswered questions on how GW goes about designing their games. Luckly GW does use play-testers that lay outside their corporate structure. One such play-tester is Mike Major a tournament organizer for Astronomi-con he has been gracious enough to pull back a bit of the veil that shrouds GW thought process when designing our favorite games. You can find his name on many projects the latest of which was the Battle Mission book. Let us see what he has to say!
Apparently there is a lot of rumor, mystery and shadow about Games Workshop’s playtesting of their products.
This, I simply cannot understand. Some days I swear the amount of conspiracy theory stuff just gets downright silly. I mean a generic ‘playtest’ army? Why would GW bother with such silliness? All it would get you is results which reflect battles against an army that doesn’t exist. They’d be worthless.
So, let me speak to you from some experience. There will be no dark secrets here. No violation of the non-disclosure agreement I signed. Nothing like that. This isn’t about GW’s up and coming products this is about the experience of playtest and what it was like from when I first started with it around ten years ago to the recent past.
I’m sorry to pop your bubble, but it wasn’t/isn’t particularly mysterious.
GW playtest started off as a huge secret and originally it was only for Fantasy. By the time I came along they’d branched into 40k some but we were still a bit ‘less’ in some ways than the WFB crowd. Still there were things to test and test we did.
In those early, secret days we ‘Techpriests’ would go to a website, download the latest version of a playtest codex, make some lists and play them against ‘regular’ armies of various sorts. It was in many ways fairly informal. We’d test what we wanted to test and then write up reports and sometimes make recommendations (this unit really needs to be cheaper, or more expensive, this unit needs to be better or no one will take it etc.) Sometimes the designers would listen to us. Sometimes they wouldn’t. Sometimes, in all honesty, it was a good thing that they didn’t. Sometimes it was a very bad thing that they didn’t.
Later, once GW published all our names in the back of Chapter Approved, things were much less secretive. We were, as a group, all very careful to never release information that was covered under our NDAs, such as what armies we were playtesting etc. My own personal response to queries on what we were working on tended to be, “Something pretty cool. You’re going to really like it. If I told you any more I’d have to kill you.” Once in awhile it would be more like, “Something pretty cool, but it still needs some polish. We’re working on it.”
We were all volunteers. We were selected for our experience in 40k and for our ability to try and be as balanced in our critique as we could. It was very much an ‘old boys’ crowd – selected in many ways because of who we knew as much as anything else but no one got in who wasn’t qualified that I saw. All in all it mostly functioned pretty well although we did have some folks who didn’t work out. One fellow was (and is) a very very smart bug player but in playtesting he could never get out of the ‘how will this work with my bug army’ approach. That made his critiques of very limited usefulness and eventually he got let go.
GW didn’t pay us. We might, sometimes, get the occasional perk, such as a copy of a ‘dex we worked on signed by the guys who wrote it, however such things were infrequent. By and large it was a fair bit of work for very little reward other than the satisfaction of making the hobby better. As such we tried. REALLY tried to make sure things were as good as they could be. Remember, though, GW didn’t always listen. Sometimes they did. Sometimes they didn’t for good reasons. Sometimes they didn’t for bad reasons. Sometimes they didn’t and it wound up being very much for the best. We ourselves often didn’t see or test the very final products. Sometimes when a ‘dex would finally come out we’d find things changed a bit. Point costs lowered, some wargear altered a bit, that kind of thing. In every single specific case I can remember this wound up making the Codex better.
One thing I will make ABSOLUTELY clear. Almost all our 40k testing was done at 1500 points. Let me say this plain (for this was never any kind of mystery) that is the point value that Warhammer 40,000 is built on and balanced for. Period. That’s how the designers look at it. That’s how the rules are written. That’s how the playtest group tests it. It’s the standard as far as the studio is concerned. This is not opinion, this is what we were told time and again. WFB is 2000.
Also, and just as important, let me be clear that the studio does NOT design for tight tournament play. They barely even consider it. That’s just not the old school British wargaming culture. It’s also why GW doesn’t write super tight rules. They’ve been told they should but just don’t feel it critical. It’s not how they look at the game and most of them wish the rest of the world would come around and enjoy the hobby rather than trying to boil down the exact implications of a single word in a sentence. Folks these people don’t WRITE the rules that tightly for, what is to them, a good reason. I don’t personally agree with that stance, but then no one asked me did they?
There were always elements of GW (almost all of them NOT in the design studio I should mention) who didn’t really like outside playtesting. It’s one reason that GW was always so tense over data leaks – even though virtually all of the leaks were eventually traced to their own employees. There was always a fear that such a leak could kill the program or severely hamper it. Eventually a combination of things altered the program enough that my own involvement was greatly reduced. In many ways this is a good thing for me as I don’t have the time I once did for such things.
So playtest was really very simple and in general fairly effective during the time that the Techpriests were really active. Grab the newest playtest version of the ‘dex, make some lists, go play some games, write some reports. Nothing fancy. Nothing tin-foil hat. Nothing kooky. No strange ‘playtest’ army. Just grab one of your usual forces and try it against the new list someone put up. Write up how the game went and send it in. Sometimes these games would result in changes, sometimes they didn’t and, in fairness, sometimes they couldn’t.
Nowadays, while I hear there is still some playtesting done, it seems a fair bit less formal again. It also doesn’t seem to be as vigorous (Doom of Malantai anyone?) that or GW isn’t listening to its playtesters when they do offer critique. I know that on the couple of projects I have been contacted on in the ‘recent’ past (recent being in the last year or so) the feedback and commentary has been extremely informal. I can’t speak for others and their experiences but I can say that most of the products I’ve seen in playtest have been much better when released. This won’t solely by the folks doing playtest as GW does its own thing internally too. However, as I’ve mentioned, there are times that GW quite simply doesn’t listen – no matter how many times playtesters give them feedback sometimes they simply take the position of ‘This is our job, and you’re just a volunteer and we’re doing it.’ That last can be very disheartening particularly when you KNOW that what GW is going to do isn’t going to be a good thing for the hobby – but they go and do it anyway.
In fairness to GW, this isn’t always because the design team WANTS it that way. GW is a publicly traded company and it is run by suits who have sometimes said and done utterly moronic things. I don’t KNOW how much impact that has on the studio – they were always professional and fairly close-mouthed about such matters but there was this ‘feeling’ you’d get. Little things. Hints, turns of phrase, that would lead you to believe that this or that particular bit of foolishness had come down from on high and those in the studio were being told to shut up and soldier. I work in a corporate environment and the signs were all there.
Remember that GW’s goal is very much to create a good hobby experience. This is, after all, what puts food on their tables and pays their mortgages but, just as important, it’s also a hobby and universe that they themselves love as much as we do. You just don’t write the kind of quality background and fiction we see without their being real passion in the mix. This isn’t just and solely a job for them. That doesn’t mean that they don’t have some yutz suit come down from on – high and create a disaster, but it’s not the goal. The goal is to write cool, fun ‘dexes that everyone can enjoy.
So hopefully that takes a little bit of the mystery out of GW’s playtesting. Keep in mind my experiences are a bit out of date and things may have changed yet again of late – I don’t know. But what I can tell you is that while it occasionally felt a bit Machiavellian, it was never terribly mysterious.
pathtyphon
TastyTaste
Frank Austin
fluger
Brent
Dethtron
newsun
AbusePuppy
nyhil
That Teri Girl
reecerobbins
badtaste
Cool article. It's nice to catch a glimpse into the machine that is GW.
> Remember that GW’s goal is very much to create
> a good hobby experience.
That about says it all right there. GW just wants the rules to be good enough for you to keep playing.
Instead, how about something like, “GW wants the rules and hobby experiment to be the best out there.”
And, nice, Tasty. Way to slip in a pro-1500 article disguised as a behind-the-scenes article. Well my super-secret playtester says it is balanced at 2000!
Yeah, I believe he was trying to find the right nail for the coffin and has done just that.
A very interesting article Tasty. I'm glad you managed to hook up with the fella who wrote this. While I play at 2,000 points and that's just my bag I have had a few games at 1,500 and I can see why G.W tests at that level. It also shows they work to a hobby environment, casual gamers instead of tournament, which again is interesting and along with the points limit goes against some things I have other people say.
Btw, is it ok if I post this on my blog? Full credit to you and link back here
The balanced for 1500 thing is what I find most interesting. Frankly, I always thought that was a rumor!
I'm curious if that has changed though; GW does seem to be promoting larger points values games to push more product.
Brent
Actually that was all me.
I read through the entire “In Defense of 1500″ series here on BoK and decided to just hammer the nail in once and for all on that.
It is quite true btw. What I didn't mention in the article (because it was drifting off topic, it was in one draft) was that the reason for the 1500 balance point is the structure of the Force Org chart. An HQ and 2 Troops as a requirement, and the regular top end cap work best at the 1500 point level. When you go higher, if you were to balance for 2k points say, you'd want to make it a minimum of three troops. Maybe even four, and perhaps require a heavy choice in order to preserve balance.
As soon as you fill your HQ and 2 Troops all the rest of the points left are yours to go where you want. At 2k it's easy to load up on whatever your particular fancy is – be that troops, elites, FA or what have you. At 1500 there is less left so 'loading up' is harder.
This is why the higher the points go in one force org the more the armies look similar.
If it has it's VERY recently.
Keep in mind though that there could be some dichotomy. Developers balancing the game as it's meant to function and sales folks pushing for higher points to generate more revenue. These things happen in companies.
I would strongly doubt that the developer's position on 1500 has changed. If so, they would probably have modified the Force Org chart.
I like it!
“Also, and just as important, let me be clear that the studio does NOT design for tight tournament play. They barely even consider it. That’s just not the old school British wargaming culture. It’s also why GW doesn’t write super tight rules. They’ve been told they should but just don’t feel it critical. It’s not how they look at the game and most of them wish the rest of the world would come around and enjoy the hobby rather than trying to boil down the exact implications of a single word in a sentence. Folks these people don’t WRITE the rules that tightly for, what is to them, a good reason. I don’t personally agree with that stance, but then no one asked me did they?”
Spot on. Most people who disagree with their approach fail to understand the above. Instead they choose to pretend GW's design policies are somehow in line with their own and then redirect their anger over their disagreement at events.
Great article Mike.
This is the best thing I've read about GW in a long while. It's very gratifying to know that the game designers share my philosophy of the game. I've been saying for over ten years that this game was never meant for tournaments, and it was meant to be played for fun. It's disheartening that so many people have forgotten what fun is.
I agree, it's all about fun. That being said, there's no reason to not do tournaments. However, it does go to show how broken a system like 'ard Boyz is. Not only is it 1,000pts over what the game was balanced for, it's also using a force organization chart intended for games at 1,500, not 2,500. You add that all up and it's a mess of an event. However, Europe runs tournaments at the 1,500 mark and they do well with it. Something for us in the US to look into instead of the philosophy of bigger is better.
Well keep in mind that as long as everyone is playing at the same points level the 'imbalance' is the same for everyone, so the playing field is at least level.
Warped, but warped equally if you like.
Astronomi-con has always been 1500 for a bunch of reasons. One was the reality of the game's balance. Another is the amount of time it takes to play a game – the more points the more time. The last was related to balance – at 1500 points armies become a challenge to design because you simply cannot have all the toys. Makes things more of a challenge.
I've played in some 'Ard Boyz events and I'm not a fan. For one thing my model collection just isn't right for competitive play at 2500. For another you just couldn't finish a game in the time alloted (at least we found that). For another a 6×4 table isn't really the right size for a 2500 point army.
Just my opinions mind you but I figure I'm as entitled to them as anyone.
Great article it's nice to see plenty of confirmation in there.
This information totally makes the Throne of Skulls Tournaments make even more sense, GW is basically saying,
“We can't be bothered to write tight rules for tournament play, because it's just not our bag, Baby. This is a hobby, go play with your mates, have some laughs and while you are at it have a pint or two. You'll enjoy the hobby much more that way.”
If playtesting is no longer vigorous, why bother playtesting at all? Seriously this is the only problem I have with GW game design. I can fill the gaps in the main rules easily enough, but fixing all the broke arsed crap in the army lists is jut too much work.
While I am at it let stir thing up a bit. I don't care how many points you play the games at, they not balanced, because GW doesn't care about balance; with game design they only care about having a good beer and pretzels game, which they greatly succeed at. So don't fool your self into thinking GW games are balanced.
'Balance' is an interesting concept. And you're wrong in that GW doesn't care about it.
GW does care but not in the way that, say, Chess is balanced. Think about it a moment. What do you mean when you say 'Balanced?' Balanced to what?
Let's say we have two armies. They're both 1500 points and neither codex is overtly broken. Heck, we'll say one army is Tau and the other one is Orks. Tau in v5 are generally considered to be less powerful than orks. Are those armies 'balanced'?
Most tourney goers and a lot of Tau players would say 'Orks have the advantage'.
Okay, fine. Now let's put those two armies on a completely empty table. Flat. No terrain. Does that change the balance? Of course it does, as now the orks have to run straight into all that pulse rifle fire and die in droves as they try and close.
Now swap that table out for a Jungle, or a City. Suddenly the Orks are supreme as the Tau cannot bring much fire to bear before being smashed in HtH.
The size of the board, the length of the game, the amount of terrain, the skill of the players ALL these things (and many more) affect 'balance'. It is quite literally impossible to write anything in perfect balance with the kind of game we're playing. Add a single tree stand from where the playtest was done and bang – you've changed it.
Now, add into that the reality that GW isn't writing for competitive play as that's only a small, if vocal, part of the hobby. Those codices have to support not only tournaments but leagues, story campaigns, pick up games, narrative campaigns, 'historical' scenarios and much more.
Balance ALL of that? Impossible. Cannot be done.
What GW does strive for is a sort of 'overall balance'. They know it won't be perfect so they do the best that they can on a macro level. And yes, they DO try. Sometimes they fail but they try. But a macro scale balance is all they can do – which means that in many ways you're right, they aren't balanced. Not perfectly. But for most situations they are fairly close and that's the desired effect.
Ok I'll relent doesn't care was a bit harsh, I should have said is not overly concerned.
I do feel after 30+ years of wargaming I have seen enough good and bad systems to spot some of the glaring problems with GW games. I love the worlds that GW has created and with some minor tweaks the game rules are really good.
I must say I have never seen a points system in games that are as inconsistent and random as they are in 40k and to a lesser extent WFB.
The codex/army book system of army lists compounds the problem, especially with GW having no desire to errata the inconsistencies. In 40k alone we have army lists spanning 3 editions of the game, with crap loads of of sweeping points changes over this time. It really does skew the balance of the game.
I'm m not asking for perfectly balanced, but GW needs to be more consistent in their management and updates of the systems.
I wouldn't disagree with even a single word of this post. That pretty much mirrors my own frustrations with GW.
I don't mind the balance approach because it's a business reality, but I sure would like to see a more modern approach of a living rulebook with true errata etc.
Nope, no argument there at all.
I very much agree about 2500 point games, ESPECIALLY the issue of table size.
Never forget they invented organised Tournament play for 40k, and, I believe, for WFB also.
Whether or not that's the way they want to play the game all the time, they let the cat out of the bag.
And anyway, fun is subjective. Some get their kicks from competitive play, y'know.
I disagree – I think that when 5e was created they had no interest in attempting to expand the 'basic' points level above 1500 – but that they have decided to do so since (under corporate pressure, most likely, but it matters not why) – the recent Codexes (IG aside) are far inferior at 1500 to their slightly larger counterparts – and the WFB book is proof indeed of a desire to extend the base level.
GW staff are very much pushing playing at 3k in 8e – this cannot be coincidence, especially given that is main points level mentioned frequently in the book itself.
I believe that things like Planetstrike are experiments in adapting the FO Chart to see how much leeway exists in it – as are the Wolves multiple HQs, and don't forget the BRB specifically states that the FOC may be different in a Codex. If 6e doesn't see a MAJOR change in the way our armies are structured (barring a HUGE shift in policy) I will literally eat something like, but not actually, a hat. (Since a hat would be difficult to digest. More likely I will eat something like a page from my 5e BRB, or printout the front page of my blog and eat that.)
For clarity – they would simply NOT invalidate the SM Codex by editing the FoC without a new BRB to do it with, due to the constant backlash of outdated Codicies and there being “too many Marines” [lol!] already. Rest assured I plan to talk to a playtester I know and see if any indication of this is forthcoming.
I admit to some confusion, as well – though I never heard it from any official source, I have been told several times that the external playtesting was all done in the UK and not North America, especially given the relatively recent explosion of the game there. How were you 'discovered' if you can divulge this?
I had a feeling that would be the case, especially coming from organizer of the fantastically fun themed Astronomi-con Toronto tournaments.
The event sounded so fun, I'd even be willing to slim down my list to 1500 points (yuck!) for the chance to come up there and participate!
All speculation of course. Might be true, I have no way of knowing, but I strongly doubt it. Still, keep that hat handy. We'll find out eventually.
Whoever told you that all external playtest was done in the UK was either repeating something incorrect that they heard or just making stuff up. There were Techpriests from all over. Check some of the names in the Chapter Approved. There were Finns, Italians, US folks, Canadians, Irish as well as folks from the UK and the studio itself and those are only the ones I recall for certain, there may have been others.
Nothing mysterious. I knew Jervis from one of the old Mailing lists that used to dominate in the days before modern blogs and stuff. A lot of the 'priests were recruited from there as it was at that time one of the main concentrations of 'elite' 40k players. It was a small enough community that folks got to know one another well.
What got it started was a commentary I wrote privately to Jervis on how Guardsmen of the day were too cheap for their points value. After that the rest, as they say, is history.
Well we're up to four now, one in Winnipeg (in a couple of days actually), one in Toronto, one in Vancouver (in a couple of weeks) and one in Dallas Texas.
So we're covering a lot of North America. New folks are always welcome and thank you for the nice compliment.
No they really didn't.
There were little local events long before GW started their GTs back in v2 days. I know as I was running some of them and I was not the only one by any means.
GW started the BIG event stuff, but tournaments existed well before they did their first GT.
Hey! A good article on Tasty's blog!
oh come on man, if youve played both levels equally (I rly only play them two the most, rarely on other levels) you find that the game is always more blananced at 1500. It seems small because you cant take your UBER combos and spammed units, isnt that the point to making a game balanced to not have those things?
This is a strawman argument. No one claims they want perfect balance, but I think we can all agree that GW could do much better, and they’re simply unwilling to put in the effort. In fact GW is openly hostile to the tournament scene, which is the place they should be gathering the info to do their balancing.
Could they do better, yes they could.
“Much better?” No, I don’t think ‘we can all agree’ on that. Because GW is NOT writing for competitive play – the codices we game with have to support a wide range of league, story, convention, campaign etc. styles of play well outside the competitive setting. So no, ‘much’ better is probably too strong.
Gathing ‘balance’ information from the tourney scene is kind of pointless. How will that help with a new codex with the tourney scene has never seen? What’s more, who is to say they will receive accurate information. Finally the design of the tournament can (and does) greatly impact who wins and who loses as does the specific army build.
What you describe just isn’t possible. Even if all tourneys were the same, it would be slamming the barn door after the horse has been running free for months.