My experience providing GHB feedback to the design team and my thoughts on GHB2019

[Laststand]

2 hours ago, whispersofblood said:

I think @swarmofseals would be wise to avoid this or if you could more narrowly defined the question?

Faction evaluation is usually about how strong it's best loadouts are. But there are a lot of reasons why say GSG may never fully actualize and with therefore forever look like a weaker 2.0 tome.

So I guess my criticism is maybe what you want evaluated. Competitiveness? Internal consistency? Etc. 

Fair point. Its the consistency that interests me and the points of differential. I.e. unit x and unit Y have 3 attacks, 4, 3, -1, 1 damage. Unit x has a movement bonus, unit Y better bravery. How are the points built up and why do very similar units in different armies have different points. 

[prochuvi]

5 hours ago, Kramer said:

14 hours ago, prochuvi said:

It is imposible as dispossesed player see those changues and think gw know what the meaning of balance is

 

I think you might manage your expectations a bit better to avoid rage quitting the hobby. 

Sorry if i sounded too much pesimist,but i am not one of those old fantasy players that want use their old army in aos.

I am a new player that bougth a new army of dwarfs to start to play(around 200€) 

When i bougth them i knew that they hadnt tome(2 years ago) but then they were very playable vs every army and could win any game,but now with every new tome and gh the gap have increase too much.

My expectations arent too much big,many others non battletome armys have units pretty good but dispossesed havent,every of our units is garbagge.

Per example:swordmasters,phoenyx guard,white lions,black guard per example all are so much better than hammerers and all are elite units of non battletome armys.

I get your point,and i sorry if i sounded too much rude or pesimist.but i was a new player that choosen dispossesed because they are so cool(i hate the aestetical of every new army of aos as water elfs,punk dwarfs etc)and when i bougth them they were at 7\10 but im crying now because after those two years dispossesed are at 3\10 and arent playable vs any tome army.

If gw wont balance them to be playable then why sell them?they get our money of the models but they dont do work to balance them. They could say us that they arent playable at matched play and they wont fix them and we can buy them for use them in other games or for hobby then.

I think that as new player and customer of gw i can demand that they balance the army that i spent so many money on it i guess,i dont have expctations of dispossesed at tome levels but i expect be at swordmasters levels per example

[Ogregut]

6 minutes ago, prochuvi said:

Sorry if i sounded too much pesimist,but i am not one of those old fantasy players that want use their old army in aos.

I am a new player that bougth a new army of dwarfs to start to play(around 200€) 

When i bougth them i knew that they hadnt tome(2 years ago) but then they were very playable vs every army and could win any game,but now with every new tome and gh the gap have increase too much.

My expectations arent too much big,many others non battletome armys have units pretty good but dispossesed havent,every of our units is garbagge.

Per example:swordmasters,phoenyx guard,white lions,black guard per example all are so much better than hammerers and all are elite units of non battletome armys.

I get your point,and i sorry if i sounded too much rude or pesimist.but i was a new player that choosen dispossesed because they are so cool(i hate the aestetical of every new army of aos as water elfs,punk dwarfs etc)and when i bougth them they were at 7\10 but im crying now because after those two years dispossesed are at 3\10 and arent playable vs any tome army.

If gw wont balance them to be playable then why sell them?they get our money of the models but they dont do work to balance them. They could say us that they arent playable at matched play and they wont fix them and we can buy them for use them in other games or for hobby then.

I think that as new player and customer of gw i can demand that they balance the army that i spent so many money on it i guess,i dont have expctations of dispossesed at tome levels but i expect be at swordmasters levels per example

I do get your frustration, I'm an elf player with a load of dark elves waiting for a book but it's all about patience. Waiting 2 years so far isn't that long compared to what we used to have to wait through old warhammer. They have stepped up the release cycle hugely, I've been in the hobby over 25 years and never before has there been 6 army books released in 6 months along with updates to all armies (for good or ill).

There are other ways to play as you said, you can do some cool narrative stuff with dwarves; holding out against a never ending swarm of skaven or goblins, going on a troll hunt to clear out a mine, searching for a lost anvil of power in the realm of death where the spirits won't stay down.

 

[chord]

Great and informative post  Thanks!

I agree that early AoS warscrolls are very out of date and have been left behind, but yet GW does not update them when given the opportunity.  Take the early stormcast units for example.  The newest tome could have updated some of those old warscrolls but instead left them as is.   This applies to any AoS 1.0 army that has had re-released tomes without updated warscrolls. 

[Vextol]

9 minutes ago, chord said:

Great and informative post  Thanks!

I agree that early AoS warscrolls are very out of date and have been left behind, but yet GW does not update them when given the opportunity.  Take the early stormcast units for example.  The newest tome could have updated some of those old warscrolls but instead left them as is.   This applies to any AoS 1.0 army that has had re-released tomes without updated warscrolls. 

I THOUGHT the app was going to be the 'cure-all' to long waits between battletome releases but that didn't happen.  I wish they would release updated scrolls outside battletomes much much more often (NOT like the thunderers!  That was just....shameful). 

Yeah, it would stink that you have all these dumb pamphlets instead of a proper book but heck, we already have to carry around 20 books.  Why not throw in a 3 ring binder?

[swarmofseals]

4 hours ago, Eevika said:

Im interested in whats the WCR of a Troggboss. What points do you think it should be at?

You picked a really annoying warscroll for WDR! The Dankhold Troggboss has two abilities that don't translate cleanly to WDR because they rely on the characteristics of the target (in this case wounds and unit size) and thus we can't count on a specific probability of them working. We can be a bit messy and say that the Crushing Grip attack will do 1 mortal wound on average, which isn't far from the truth. The squig attack though is much less reliable, so I'm just going to calculate two situations: one where the squig attack hits and one where it doesn't. 

• Without the squig attack or the command ability the WDR is .0416
• With the squig attack but without the command ability the WDR is .0488
• Without the squig attack but with the command ability the WDR is .0479
• With both the squig attack and command ability the WDR is .0546

These numbers are a little low, and given that the Troggboss doesn't have any amazing support abilities I'd think it could stand to come down a bit on points cost.

6 hours ago, Laststand said:

Thoughtful and well considered article. Really enjoyed it. I think a crux of GW reasoning is that they just value abilities slightly differently to players. Perhaps many of us dont place enough value on flexibility in roles for a unit. For example Palladors dont have a great offensive ability but their movement is superb and defense decent. What is the value of snatching an objective early from across the board? I bet we all have different perspectives. 

Its a huge ask @swarmofseals  but I'm sure many of us would be very interested to see your faction evaluations if you were willing to share them? 

Unfortunately I'm not really up for sharing the full evaluations. Suffice to say I looked at how each faction was performing in terms of win % (tempering that with a grain of salt) and then evaluated it in terms of internal balance as well. I think the goal should be to make as many factions perform within acceptable parameters of win rate as possible with as diverse internal balance as possible.

I can't comment at all on what GW thinks, but I will add that players themselves don't value warscrolls in a consistent way. When I first started in wargaming, I was fully obsessed with combat efficiency. If you look at most tournament players though they really don't think this way. Combat efficiency is important, but mobility is king. Most players, in my experience, still overly focus on the combat stats. If combat efficiency were everything then this game would be all Plague Monks all the time. When I calculated that warscroll my jaw hit the floor -- it's so messed up. And yet you aren't seeing Plague Monk spam dominating the tournament scene. 

3 hours ago, Kasper said:

Great writeup - Thank you very much for it. I'm curious though, you mention WDR and talk about looking at warscrolls specifically. I'll admit I'm not certain how exactly you calculate WDR, but it sounds like it is purely based on the specific warscroll?

Does this mean that alligiance abilities aren't factored in, in any way, when deeming the value of a specific warscroll? Slaanesh has pretty bonkers alligiance abilities, so comparing a warscroll like Daemonettes to, let's say Gors in BoC really doesn't tell you much and the comparison will be awful compared to reality. This is further complicated by abilities that allows the Daemonettes to fight twice etc.

My question is if WDR is purely based on 1:1 comparions of warscrolls?

WDR can take into account whatever buffs you want it to. It's basically just the expected value of the damage output with a weight assigned to different levels of rend. That weight was determined by evaluating the relative effectiveness of each rend level against the full range of armor saves with each save level given a weight based on its prevalence in the game overall (EG: performance vs 2+ save has a lower weight than vs 4+ save because 4+ saves are MUCH more common than 2+ saves). 

In my analysis I usually focused on first evaluating the individual warscroll in a vacuum and then considering in the allegiance package. Note that the allegiance package in my mind does not consider just the raw buffs that the battle traits give -- it also considers the faction's access to different tools, access to good battalions, etc.

[Kitsumy]

5 hours ago, HollowHills said:

@swarmofseals thanks for such an insightful and detailed post. It's really interesting to hear from someone balancing the divide between rules writers and players.

I have three questions:

1) How do you think the activation system of aos impacts balance? I k ow you've touched a bit on this but I'd like to know further. For example, thralls seem overcosted even at 130. This seems to be primarily on the basis that a 5 up save means that unless the unit goes first, it will probably die. Doing 30 wounds to a unit with a 5 up save is not that hard. However, if they activate first they probably can do a lot of damage. It seems like there are a few units which work like this and end up falling out of favour. 

2) How are base sizes taken into account? For example, bloodreavers and marauders are both horde units that fill a similar role in a chaos army. However, reavers are on 32mm with marauders on 25mm. As such you can get a lot more marauders into combat. 

3) What is the point of units like Lotann and how do you even fix them with points? Support characters that don't contribute a single powerful and reliable buff seem to have no place in AoS. 

 

Exactly that. Glass canon units like thralls are hard to balance since they will mince meat a minimun size unit if can atack. But problem is noone field minimun size units. And thralls are forced to be played like that since i dont know why they only have 1" on his giant weapons. So only the front row will atack always and since even a squeze will kill them in hundreds and they have one of the lowest leadership on the game ( despite being semi elite army and elfs for godsake) u will loose 20+ of them against every confront.

 

I would raise his point to 140 again. Raise his weapon range to 2" and raise all idoneth army leadership in 1 to 7ld.  This would make them so much fun. Since u coukd field them as 20. And would die as fast as now. But at least leadership wouldnt hurt so much and soulrender would start to have a purpose rezzing 2 of them every turn.

 

Till gw dont do those changes thralls will keep being on a weird spot since cant be fielded on big sizes for his base sizes. Low armor and leadership. But arent so good as minimun unit since lack the punch to wipe out the unit the charge and them they will be wiped on the counter. And im sure gw will keep lowering his points on every boock till 110-120 on the future. And no aelve army should be an horde so i highly dislike this.

 

Btw i dont find lothan sooo bad ( despite him and the avatar are the only models i dont have) he isnt so bad dmg and defensewise for a 80p hero. And +1ld to thralls and reroll 1s isnt bad. He isnt great for sure but i think he is as good or even better than soulrender. And fir sure better than the overprized soulscryer for 130p

[Zadolix]

I do hope that GW's choice of holding off on making major points and warscroll changes to the dispossessed is a sign of a battletome coming in the next year or so. I have plenty of other games and models to paint so I don't mind waiting but I'd love to see them get some real love. At least GHB19 shows we haven't been forgotten at least.

[Primes]

@swarmofsealsI agree with everything you say but dont really get one thing: Why are the discounts on armies who obviously wont get a new book soon so conservative. Mainly thinking of Stormcast and Nurgle.

How can a Stardrake be 560 when you compare it to a Verminlord? How can the Prime still be so expensive? Why are Eels 170 and Evos on Dracolines 300? Not speaking of Internal Balance where the Point Changes wont make older units viable but make strong Units weaker. Or are Palladors now worth it?

Whats the WDR of Plaguebearers and Plaguedrones? Does a GUO seem a little expensive? Are Blighlords worth it for 200pts?

I may Sound salty but I would honestly love to be proven wrong from a mathematical standpoint.

Again thanks a lot for your work!

Edited June 18, 2019 by Primes

[Forrix]

My main concern with the GHB is for factions that have had updates recently but are significantly behind the new power curve i.e. Maggotkin of Nurgle, Nighthaunt, and Gloomspite (Gloomspites are a bit of discussion and we haven't seen their GHB changes yet). What happens to them? Poor Rotigus is just over a year old and is pretty much restricted to just praying to the dice gods that he can actually get his spell off and stomp some Nighthaunt heroes.

My other concern is rules like the already infamous terrain rules. I just can't believe that actually got play tested or if it did they even bothered to read feedback. I got the impression from your post that you were providing feedback on the current gaming scene, not that they sent you beta rules or points values to try out. Have you or anyone you know received beta rules before?

 

 

[frostfire]

Don't want to be pessimistic here,  but I really think the version of GHB is kind of awful.

It's been a whole year since the GHB2018 came out, which according to some rumors , is an untested version, terrible enough on that already, and see what changes they make in the new version.

There are so many allegiance remain unchanged, only two factions get spell lore.

Scenery rules that are so weird that get you wonder if the design group do play the game.

18 so-called new battleplans which are mostly another version of the former ones that they even don't change their names.

They do try to make Meeting Engagement interesting, but still no restriction to the summoning. Imagine that FEC or Seraphon who just bring like 1500 to a 1k game, and tell me is it fun or not.

I can't believe that is what they have progressed in a whole year, or if rumors are true then two years.  There are so many contents unchanged in the Matched Game that I can't nearly seen any difference between the GHB2018.

I know people declare that GW offers various ways to enjoy AoS, such as Open play or Narrative Play. But at my local, people are more likely to play matched game. So what's wrong for hoping for better rules for the game mode I prefer?

Still convincing myself that the Faq will help make the matched game better, hoping that GW wouldn'tprove me wrong.

[Forrix]

16 hours ago, Doom & Darkness said:

people feel a real disconnect between how most people enjoy playing Age of Sigmar and how the rules writers play/write Age of Sigmar

This. So much this. I feel this disconnect has actually lessened over time but its still big. I also have a hard time believing that they actually play test. Stacking Command abilities with the launch of 2nd edition was one of the stupidest things I've ever seen in a game. It was so easy to see coming and lo and behold it turned out the way everyone thought it would. Even then they just went through and FAQ'd nearly every command ability in the game instead of just making a core rule of "command abilities don't stack" and then editing the few command abilities they still want to stack to be able to.

I'll note that listening to Stormcast has me convinced that most of their staff only play Open or Narrative games. Which does explain a lot, especially the initial launch of AOS where I swear they were genuinely baffled that people cared so much about points and the implementation of power level for 40k.

Edited June 18, 2019 by Forrix

[Dead Scribe]

I think they playtest.  But they playtest with their small inner circle that enjoys the game a certain way that the wider public does not engage with.

[Zanzou]

18 hours ago, swarmofseals said:

I really don't mean to be rude or combative, but I don't think you read my post very carefully as I spend a huge part of it addressing this exact point. 

The huge part simply says that players will be mad at points going up IF a battletome actually finally comes out, so instead they are completely ignoring that they are useless in the mean time.

I don't mean to be rude or combative either, but it makes sense that the prior poster you are referring to wouldn't be satisfied with that as an option.  Players are ALREADY "livid" that their faction has been dumped on for YEARS, so for those players these continued excuses are, well, inexcusable as GW has failed to meet expectations for those players as of yet (STD PLAYERS, RIP).

[Kitsumy]

57 minutes ago, Primes said:

@swarmofseals

How can a Stardrake be 560 when you compare it to a Verminlord? How can the Prime still be so expensive? Why are Eels 170 and Evos on Dracolines 300? Not speaking of Internal Balance where the Point Changes wont make older units viable but make strong Units weaker. Or are Palladors now worth it?

I may Sound salty but I would honestly love to be proven wrong from a mathematical standpoint

Wont say those stormcast units are op or anything. But i think they are pretty good. Stardrake is great in fact i thought it would get a point increase. Since it can be a behemoth with 2+ save and doing mortals on 4 or 5s etc. But verminlords are absurdly cheap. They cant cost less than 350-400 by any means. Idoneths mage avatar are the same as verminlords and cost 420 per example.

 

Dracholines do 13.64 wounds with -1 rend on the charge if they use the normal weapon.

17.18 -1rend wounds on the charge if they cast his spells that being a 6+ ahould be 100% as median since the median on 2 dices are 7. If the oponnent havent a mage.

2 special weapons:

16.72 roughly 0.75 rend since 2/3 of the atacks have 1 rend and the remaining 0.

21.06 if have his spell.

They have 15 wounds with 4+ save rerroling 1s to shoting. And 8 ld making them pretty inmune to it. And a -1 ld area. And being mages can dispell 1 time.

 

And they do 3 mortals every turn. Not like eels that do the same once per battle only.

So counting the mortals they do:

16.64w or 20,18 with the speel

Special weapons do 19,72 or 23,06 with spell.

 

Eels do 11,44 -1 rend ( half -2  other half 0 so median is 1 in all) on the charge.

And once per game they do roughly 2.4 mortals.

Have 12 w. 4+save. 6ld ( so even with only 1 death they can start losing models despite being the royalty among idoneths). Move 2 more and fly ( pretty useles on aos most of the time)

 

So dmgwise only .maximun dmgper point.

Drach has 0.076 ( on best case) Vs eels 0,067 normal or 0.081 the only turn they use his ability.

 

But tankwise eels have more wounds with sligthy worse save again shoting. A ridiculous leadership etc.

In general MY conclusion ( numbers dont lie but everyone can understand them different) is eels are tankier and do sligthy less to roughly the same dmg per points than dracholines. . There are more differences like eels fliying havind a joke leadership. Stormcast being able to cancel spells aura of -1ld etc. But all in all of course eels are better. But the margin is waaay shorter than i thought it would be since every non idk cry about those eels being so broken and stormcast criying about dracholines being so underpower.

 

Of course this isnt counting external abilitys. Like those eels being fuethan rerolling 1s to wound on mount. Or having volturnos cloose to reroll 1s to hit. Or stormcast having the +1 armor of castellan i think? Or his dhacholine hero buffing them etc. I think eels would win even more than stormcast with this but i dont want to do the numbers now haha.

 

Maybe dracholines should have get a -20 or 30 point decrease on ghb making them great for me. But at least watching those numbers i dont think they be so bad.

 

[Sleboda]

5 hours ago, Ogregut said:

I do get your frustration, I'm an elf player with a load of dark elves waiting for a book but it's all about patience. 

Just a thought for you and @prochuvi

Getting a book is not the miracle cure some folks think. Khorne is on its third (I think) and it's still a joke. 

I think the book vs. non-book gap is overblown. It's really about well designed vs. steaming pile, regardless of where you find the rules.

[Sleboda]

1 hour ago, Dead Scribe said:

I think they playtest.  But they playtest with their small inner circle that enjoys the game a certain way that the wider public does not engage with.

I can say with 100% certainty that this is untrue. I personally know 3 of their testers (and used to be one for old Warhammer myself). All three people have their own focus, but all three very much represent the wider player base.

[Dead Scribe]

Then there is an abject failure happening in the communication between the playtesters and the gw design team.  Or the playtesters they have are not very good at playtesting the game as a whole.

[PrimeElectrid]

1 hour ago, Forrix said:

This. So much this. I feel this disconnect has actually lessened over time but its still big. I also have a hard time believing that they actually play test. Stacking Command abilities with the launch of 2nd edition was one of the stupidest things I've ever seen in a game. It was so easy to see coming and lo and behold it turned out the way everyone thought it would. Even then they just went through and FAQ'd nearly every command ability in the game instead of just making a core rule of "command abilities don't stack" and then editing the few command abilities they still want to stack to be able to.

I'll note that listening to Stormcast has me convinced that most of their staff only play Open or Narrative games. Which does explain a lot, especially the initial launch of AOS where I swear they were genuinely baffled that people cared so much about points and the implementation of power level for 40k.

Well the writers play (and podium) at UK tournaments, so yeah.

[Ogregut]

27 minutes ago, Sleboda said:

Just a thought for you and @prochuvi

Getting a book is not the miracle cure some folks think. Khorne is on its third (I think) and it's still a joke. 

I think the book vs. non-book gap is overblown. It's really about well designed vs. steaming pile, regardless of where you find the rules.

I'm fine without having a book, but look forward to seeing what happens in the future. 

I like the new khorne book, in fact I like all the new books. I'm not one to chase the meta but try different combos and each one allow for different types.

[Primes]

1 hour ago, Kitsumy said:

Wont say those stormcast units are op or anything. But i think they are pretty good. Stardrake is great in fact i thought it would get a point increase. Since it can be a behemoth with 2+ save and doing mortals on 4 or 5s etc. But verminlords are absurdly cheap. They cant cost less than 350-400 by any means. Idoneths mage avatar are the same as verminlords and cost 420 per example.

 

Dracholines do 13.64 wounds with -1 rend on the charge if they use the normal weapon.

17.18 -1rend wounds on the charge if they cast his spells that being a 6+ ahould be 100% as median since the median on 2 dices are 7. If the oponnent havent a mage.

2 special weapons:

16.72 roughly 0.75 rend since 2/3 of the atacks have 1 rend and the remaining 0.

21.06 if have his spell.

They have 15 wounds with 4+ save rerroling 1s to shoting. And 8 ld making them pretty inmune to it. And a -1 ld area. And being mages can dispell 1 time.

 

And they do 3 mortals every turn. Not like eels that do the same once per battle only.

So counting the mortals they do:

16.64w or 20,18 with the speel

Special weapons do 19,72 or 23,06 with spell.

 

Eels do 11,44 -1 rend ( half -2  other half 0 so median is 1 in all) on the charge.

And once per game they do roughly 2.4 mortals.

Have 12 w. 4+save. 6ld ( so even with only 1 death they can start losing models despite being the royalty among idoneths). Move 2 more and fly ( pretty useles on aos most of the time)

 

So dmgwise only .maximun dmgper point.

Drach has 0.076 ( on best case) Vs eels 0,067 normal or 0.081 the only turn they use his ability.

 

But tankwise eels have more wounds with sligthy worse save again shoting. A ridiculous leadership etc.

In general MY conclusion ( numbers dont lie but everyone can understand them different) is eels are tankier and do sligthy less to roughly the same dmg per points than dracholines. . There are more differences like eels fliying havind a joke leadership. Stormcast being able to cancel spells aura of -1ld etc. But all in all of course eels are better. But the margin is waaay shorter than i thought it would be since every non idk cry about those eels being so broken and stormcast criying about dracholines being so underpower.

 

Of course this isnt counting external abilitys. Like those eels being fuethan rerolling 1s to wound on mount. Or having volturnos cloose to reroll 1s to hit. Or stormcast having the +1 armor of castellan i think? Or his dhacholine hero buffing them etc. I think eels would win even more than stormcast with this but i dont want to do the numbers now haha.

 

Maybe dracholines should have get a -20 or 30 point decrease on ghb making them great for me. But at least watching those numbers i dont think they be so bad.

 

Wont derail the General discussion but having fly + high Movement is huuuge. Its the difference between hitting a screen and killing what you want to Kill. On the other hand differences in Leadership and Bravery manipulation matter far less than I would like.

Also I would always prefer one heavy Hit vs. consistent medicore damage. esp. If I can have  three units with this once per game heavy hit. In AOS you seem to want to Kill your target with your first activation.

Getting back on point: For me evaluating those nuances with point costs seems very hard. Would Love Insights!

[gjnoronh]

2 hours ago, Sleboda said:

I can say with 100% certainty that this is untrue. I personally know 3 of their testers (and used to be one for old Warhammer myself). All three people have their own focus, but all three very much represent the wider player base.

I also know several of the past playtesters personally the ones I know were widely regarded as amongst the best competitive players of Warhammer in the US.    At least a few of the current   GW playtesters are members of this forum including Ben Curry who runs this forum.  As noted some of those names with writer/playtest credit podium at big events.    Sleboda was working for GW in the past  possibly when he was a playtester.    GW has talked about their internal playtest process somewhat in a Jervis article in WD from January of this year which I think is well worth reading for those interested.    

I've also been involved personally  as a playtester for companies other then GW.   I can only talk about that experience in relation to my background in medical research.   In terms of data science it's quite hard to get enough playtesting to get truly statistically useful  data.   It tends at best to be 'this feels too weak' 'this feels too strong' - not we tried this unit 100 times each  against these 20 different armies and had enough results to draw a clear statistical trend.    If a game takes 10 minutes and was just two options red vs black you could get the reps in.     In a game like AoS with lots of variables  (big variance in skill from 'average gamer' to top tier player, 20 or so units per book + various internal variants vs 20 or so opposing armies and 18 or so battleplans in circulation and high variance in terrain found on the tabletops in the player world)  and a long  (2-3 hour) game time  it's quite hard to get enough reps in to provide statistically actionable data.   

First weekend after an AoS book drops there's probably thousands or so games played with the book (rough estimate total sales of a book x 10%-25)  - a lot of data gets generated quickly! There just isn't a way to get thousands of games or even hundreds of games with a closed internal playtesting  model in the context of the timeline for release for AoS.   Along those lines I've also in the past worked directly with the head fantasy rules write in various WFB editions  on collating questions for the  GW FAQ.  There were questions that came up during that first month of games that they hadn't thought of (and neither did I when I first read an army book or during my first ten games) that came up because many many many gamers (thousands)  playing a game tend to find the edge cases and suboptimally rules that a smaller group  (even if it's 50-100 or so ) of gamers/rules writers miss.     I bring that experience working with the FAQ team up because it really illustrated for me how a book 'being out in the wild' provided a lot of data that even careful individual readings missed.   

Could the internal  and external balance accuracy of books be better? Absolutely.   But it's hard to get enough playtest data to scientifically guide us to that accuracy.  You can get a feel and  a general sense from the playtests but it will miss things. 

 

Edited June 18, 2019 by gjnoronh

[Sleboda]

17 minutes ago, Dead Scribe said:

Then there is an abject failure happening in the communication between the playtesters and the gw design team.  Or the playtesters they have are not very good at playtesting the game as a whole.

The players in question are all excellent. There very well could be an issue with the design team. 

An anecdote:

When I was a tester, I (and others as well) would get a set of the newest rules, test them (2000 pts, Pitched Battle, 4 x 6 table was the requirement, though we could test other ways as long as we focused on the 'standard' game), and report the results.

Sometimes we would offer suggestions on clarifications to wording. We didn't tell them they stunk at words or anything, just pointed out where things could be more clear. Eventually we were told to stop since they couldn't fix everything "and make rules so long the book would be twice the size." When we then offered reworded options that actually were the same length or shorter, we were yelled at and basically told to stop trying to tell them how to do their jobs, that they were the rules writers, not us.

 

Dunno if that's still the case in the studio, but it was a real thing back then and it was always sad to see a rule that had known issues that could have been fixed make it to the published books.

[tripchimeras]

 

11 minutes ago, gjnoronh said:

I also know several of the past playtesters personally the ones I know were widely regarded as amongst the best competitive players of Warhammer in the US.    At least a few of the  GW playtesters are members of this forum including Ben Curry who runs this forum.   Sleboda was working for GW in the past  possibly when he was a playtester.    GW has talked about their internal playtest process somewhat in a Jervis article in WD from January of this year which I think is well worth reading for those interested.    

I've also been involved personally  as a playtester for companies other then GW.   I can only talk about that experience in relation to my background in medical research.   In terms of data science it's quite hard to get enough playtesting to get truly statistically useful  data.   It tends at best to be 'this feels too weak' 'this feels too strong' - not we tried this unit 100 times each  against these 20 different armies and had enough results to draw a clear statistical trend.    If a game takes 10 minutes and was just two options red vs black you could get the reps in.     In a game like AoS with lots of variables  (big variance in skill from 'average gamer' to top tier player, 20 or so units per book + various internal variants vs 20 or so opposing armies and 18 or so battleplans in circulation and high variance in terrain found on the tabletops in the player world)  and a long  (2-3 hour) game time  it's quite hard to get enough reps in to provide statistically actionable data.   

First weekend after an AoS book drops there's probably thousands or so games played with the book (rough estimate total sales of a book x 10%-25)  - a lot of data gets generated quickly! There just isn't a way to get thousands of games or even hundreds of games with a closed internal playtesting  model in the context of the timeline for release for AoS.   Along those lines I've also in the past worked directly with the head fantasy rules write in various WFB editions  on collating questions for the  GW FAQ.  There were questions that came up during that first month of games that they hadn't thought of (and neither did I when I first read an army book or during my first ten games) that came up because many many many gamers (thousands)  playing a game tend to find the edge cases and suboptimally rules that a smaller group  (even if it's 50-100 or so ) of gamers/rules writers miss.     I bring that experience working with the FAQ team up because it really illustrated for me how a book 'being out in the wild' provided a lot of data that even careful individual readings missed.   

Could the internal  and external balance accuracy of books be better? Absolutely.   But it's hard to get enough playtest data to get us to that accuracy.  

 

I think this is a good point, and very fair.  However, the response to this though is that once that book is released and all that data is come in far too often GW does literally nothing about it.  They rarely use FAQ's the way they should, and even when big updates like GHB do come out, far too often very little actually gets fixed.  This gives the impression GW has no idea what they are doing, doesn't know how to play there own game, and never play tests.   GW needs to be more transparent with their rules development, they need to be far more willing to make sizeable adjustments in the post release faq's with the benefit of those thousands of games they could not get before its release, and they need to either have more frequent GHB style updates OR they need to be much bolder in the pt updates they do have. 

I think GW runs with the attitude of well WE already know what the next 6 books are going to look like and once they are out this will not be an issue and the meta will have passed this by.  Well the problem is that in the meantime, we the players do not know this, and we have to play with it for the 6 months before those books have come out, and in the meantime there is potential for each and every one of those books to introduce something just as if not more problematic then the issue they know will get fixed.  If GW made clear that they were aware of their issues and made good faith fixes once the data existed to do so, there would be much more trust and fewer people would be under the mistaken impression the designers don't know how to play their own game.  I really don't think the meta is in that terrible a place, I remember the end times, in comparison the current state is amazing.  But I still get pissed off everytime GW lets these glaring issues fester when it would be so easy to release a faq 3 months later to fix the issue, especially given that the designers ARE going to these tournaments and know they exist.  It absolutely is possible to release too many updates too quickly, but GW has NEVER been in danger of even coming close to approaching that point.

Edited June 18, 2019 by tripchimeras

[HammerOfSigmar]

13 minutes ago, Sleboda said:

The players in question are all excellent. There very well could be an issue with the design team. 

An anecdote:

When I was a tester, I (and others as well) would get a set of the newest rules, test them (2000 pts, Pitched Battle, 4 x 6 table was the requirement, though we could test other ways as long as we focused on the 'standard' game), and report the results.

Sometimes we would offer suggestions on clarifications to wording. We didn't tell them they stunk at words or anything, just pointed out where things could be more clear. Eventually we were told to stop since they couldn't fix everything "and make rules so long the book would be twice the size." When we then offered reworded options that actually were the same length or shorter, we were yelled at and basically told to stop trying to tell them how to do their jobs, that they were the rules writers, not us.

 

Dunno if that's still the case in the studio, but it was a real thing back then and it was always sad to see a rule that had known issues that could have been fixed make it to the published books.

It makes me think that GW designer are very arrogant......