A data based approach to the "drops" component of list building

[swarmofseals]

For a long time one of the best kept secrets of competitive AoS players was the importance of managing the number of drops in your list in order to increase you chance at having the choice of turn order to start the game. Nobody was trying to keep this a secret, but the information just hadn't trickled down to the low level competitive scene and below.

I think that has changed now, and most competitive or semi-competitive players seem to be aware that limiting drops is very important. I even think there may have been an overcorrection, with some holding the view that lists over 5 drops are generally not viable (with a few exceptions).

What I haven't seen is anyone really attempting to take a more rigorous approach to the question of drop count, so I've decided to take a stab at it.

The ETC 2019 lists give a very nice snapshot of the late GHB2018 metagame. I'm sure things are skewed somewhat due to it being a very competitive team tournament, but a perusal of the lists didn't make me think that it would vary all that much from what you'd see in the top half of any given GT. This dataset is valuable because it gives a complete view of the metagame at a sizable tournament rather than just focusing exclusively on the top lists. There are limitations however, particularly insofar as the dataset represents a somewhat outdated meta. It reflects neither the GHB2019 and FAQ changes nor the new Sylvaneth tome. That said, it's really difficult to say how much this fact skews the dataset, as the new rules push the number of drops in both directions. Reducing the competitiveness of Idoneth and Legions of Nagash and general point increases to many top-tier armies probably nudges the drop counts down some, but the removal of 1-drop Sylvaneth and points reductions among other factions likely nudges the drop counts up. That said, this doesn't account for changes in battalion use. If previous low drop high-tier faction lists solve their points crunch by dropping battalions, that could increase the drop count rather than lowering it. Similarly, if previously overcosted factions use their new points to fit in more battalions, then it could lower their drops instead of raising them.

Basically, I think this data set is good enough to draw some general conclusions from now and it's risky to try to correct it for GHB2019 -- we just need to wait for more data.

Anywho, here are the ETC drop counts:

Spoiler

• 1 drop: 4 lists
• 2 drop: 1 list
• 3 drop: 1 list
• 4 drop: 15 lists
• 5 drop: 7 lists
• 6 drop: 5 lists
• 7 drop: 5 lists
• 8 drop: 8 lists
• 9 drop: 15 lists
• 10 drop: 7 lists
• 11 drop: 2 lists:
• 13 drop: 1 list
• 17 drop: 1 list

I think a lot of people might find this data surprising. The two largest clusters are around 4-5 drops and 8-9 drops, but the high drop cluster is larger. Despite the reputation of the competitive metagame being skewed toward low drop lists, less than 40% of the ETC lists are 5 drops or fewer. And I'd argue that the ETC meta is, if anything, more competitive than the meta as a whole. At ETC fully 48% of lists were top tier factions (defined as Slaanesh, FEC, Fyreslayers, LoN, DoK, Idoneth, and Skaven) vs. 35.7% of the overall meta compiled by The Honest Wargamer.

Here is a chart of your chance to have the choice of turn against a random opponent if you show up to ETC with each of the following numbers of drops:

Spoiler

• 1 Drop: 97.22%
• 2 Drops: 93.8%
• 3 Drops: 92.3%
• 4 Drops: 81.3%
• 5 Drops: 66%
• 6 Drops: 57.6%
• 7 Drops: 50.7%
• 8 Drops: 41.7%
• 9 Drops: 25.7%
• 10 Drops: 10.4%
• 11 Drops: 3.5%
• 12 Drops: 2.8%
• 13 Drops: 2.1%
• 14-16 Drops: 1.4%
• 17 Drops: .6%
• 18+ Drops: 0%

And here is how much turn choice percentage you stand to gain by reducing your list by one drop at each of the following drop counts:

Spoiler

• 1 Drop: NA (if you have 0 drops you left your army at home!)
• 2 Drop: 3.42%
• 3 Drop: 1.5%
• 4 Drop: 11%
• 5 Drop: 15.3%
• 6 Drop: 8.4%
• 7 Drop: 6.9%
• 8 Drop: 9%
• 9 Drop: 16%
• 10 Drop: 15.3%
• 11 Drop: 6.9%
• 12 Drop: .7%
• 13 Drop: .7%

So the numbers of drops that stand to gain the most by going down by 1 are, in descending order: 9, 10/5 (tie), 4, 8, 6, 11/7 (tie). For the rest of the drop counts, the percentage gain is pretty negligible. 

Mathematically, you can determine the expected value of shaving a drop with the following formula:

X = W₁*P - W₂

Where W₁ is the expected increase in winning percentage due to having the turn choice, P is the increased probability of getting the turn choice by reducing to the targeted number of drops, and W₂ is the expected increase in winning percentage due to having whatever units you would be shaving in order to reduce your drops.

This equation is not particularly easy to use primarily because W₁ and particularly W₂ are difficult to determine. Let's look an example:

Spoiler

Bill estimates that his winning percentage when he has the choice of turn is 60%, and his winning percentage when he doesn't have the choice of turn is 40%. He is considering shaving from 5 drops down to 4, which would increase his chance of getting the turn choice by an estimated 15.3%. Thus in this case W₁  would be 20%, P would be .153, and the result would be 3.06%.  So if whatever list changes were needed in order to lower the drops by 1 reduce the winning percentage by more than 3.06%, the change is bad (X ends up negative), and if the changes reduce the winning percentage by less than 3.06% then the change is good (X ends up positive).

Obviously, the bigger W₁ is the more likely reducing your drops will be a good idea. So if you are playing some kind of list that can basically never win if they go last, then you really need to watch your drops. But if you are playing a more normal list then it's a much closer question. 

This inquiry leads to a clear question: assuming you are running an "average" list (ie: not a list that cares about turn order to an unusual degree), what is X₁?

I would love to get some insight from top level players on how they would answer this question.

[tripchimeras]

That is some great data, really interesting.  I do think that the one big issue with using ETC results, is that as competitive as it is, the team format and strategy surrounding that, fundamentally changes the strategic parameters of the game.  Every team is going to include "hold" lists, and lists that are only good against certain builds etc.  IE the lists tend to be a bit more scewy and role focused then you would find in an all comers tourney.  That being said still some great data. 

I do think that also the number of optimal drops, is very much affected by the quality of book batalions.  And I think the number of books with strong batalion rules is pretty small, and also mostly new books.  So I think again, that is why the drop numbers are higher.  The balancing act between drop count and efficiency sacrifice is a super interesting topic, and I think this generalized data, while useful isn't going to get us much closer to answering it.

I think per your last couple of paragraphs the big missing data point in this type of aggregated data, is what is choosing turn priority worth in the first place?  I think given your focus on ETC data, I think to improve the data set above establishing the X1 value by breaking down the ETC19 games  by drop advantage vs game outcome would be the clear companion data point in this instance (I am recommending this, but not volunteering to do it myself, aren't I helpful? haha).  I personally suspect,  that the correlation (like the drop breakdown itself) is not nearly so strong as many of us currently suspect. 

The other variable that makes establising the X1 value so difficult is relative army strength.  This isn't chess we are talking about, so  isolating the win improvement based on turn priority from all other factors is difficult.   For example I think often the strongest armies also have really good battalion options, so likely have less drops.  But are they strong BECAUSE they have good battalions and less drops, or are the good battalions and fewer drops simply a function or consequence of those books general efficiency/rule strength?  This is definitely one of those metrics where given our inability to collect "quality" data we have to work with volume instead as our best hope for removing the influence of a lot of these outside factors we won't have a good way of accounting for.

Regardless bravo for doing this, and its definitely a fascinating look at the drop meta.

[Lord Krungharr]

I have been sorely tempted to make a single-drop Tzeentch army consisting only of Overseer's Fatetwisters, with Burning Chariots as battleline (herald on one as general), and 5 of the heavy hitting endless spells.  It would probably be terrible overall, but to go first and plop that much magic turn 1 would be super fun!  And definitely necessary to go turn 1 to get those endless spells off for maximized mayhem.

[swarmofseals]

@tripchimeras great comments! I'd be happy to try to figure out X₁ for ETC, but unless I can find a complete round by round results summary I don't know how I'll be able to. 

Your criticisms of the team format are reasonable, although honestly when perusing the lists to compile this data I really didn't see that many lists that looked out of place in the usual competitive singles meta. 

[18121812]

Thanks for the write up.

There's a lot more pressure to drop at the 3-4-5 drop count than there is at the higher counts. At that level, you're probably banking a bit on having the choice, even though you've suggested the 9 to 8 sees the greatest improvement.

 By the time you're at 7+ drops, you've accepted that you're probably not going to have choice of turn, and have built a list/strategy that first turn isn't important. Even though going from 9 to 8 at first glance looks like it yields large improvements, it's going from "not likely to have the choice" to, well, still not likely to have the choice. 

Additionally, the math is accurate, but somewhat deceptive. It's somewhat similar to armor saves; depending on how you look at it, the change from a 6+ save to a 5+ save, can be the same, or very different to a 3+ save to a 2+. Sure, in both case, it's a +1/6 (+16.7%) improvement. But in the second case, it fully halves your chance of failure, while in the first the chance of failure is only reduced by one fifth. 

I'd restate the percentage gain as what the decrease in odds of 'failure.' The more useful formula, rather than initial% - new% is (100 - new%/initial%). IE, 20% improvement for 6+ to 5+, 50% for 3+ to 2+ armor save.

Going from 4 to 3 takes you from about 80% to about 90%, meaning your chance of failure has gone from 20% to 10%. So your chance of not having a choice has been cut in half (50% improvement). 

For 9 drops to 8, you go from about 75% chance of no choice, to 60%. That's only a 20% reduction , significantly less than the 50% reduction in failure from 4 to 3.

Edited August 12, 2019 by 18121812

[Smooth criminal]

Seems like 3 drop count is the statistically best cutoff point if you really want to have control about going first. Nice to know.

At 4 the chance of you encountering at least 1 matchup where you get outdropped in the course of 3+ round tournament becomes very real.

Obviously 1 is where you want to be, but not everyone can physically get there. And 2 seems like it's not worth a bother compared to 3.

 

Not sure what X1 is supposed to be. You want to know how much drops you can shave to gain win percentage? I don't think it's possible to quantify in an objective manner. I think it can be simplified to binary choice:

1-3 drops - you really care about going first.

4+ - you don't care or you put your trust in luck.

Edited August 13, 2019 by Smooth criminal

[Agent of Chaos]

This was a really interesting analysis. Thank you very much!

Although its difficult to draw conclusions due to the holes in the data pointed out by others, the breakdown of how many drops people's armies had is really useful. I've got a tournament coming up in a few months and have been practicing with a 2 drop list which I feel really good about right now thanks to you! Mind you its not because my list needs to go first, but its sometimes about denying that opportunity to your opponent, or weighing up whether you can wear the first double turn or not. Its also telling me that I can probably afford to go up to 3 drops and still have a very high chance of having the choice. Thanks again!

[swarmofseals]

11 hours ago, Smooth criminal said:

Not sure what X1 is supposed to be.

X₁ is unknown at the moment, and it varies list by list. Some lists are going to really benefit from having the turn choice while others care much less. X₁ is basically a measure of how much a given list benefits from having the choice of the first turn (specifically the difference in winning percentage when having the turn choice vs. not having it)

17 hours ago, 18121812 said:

 By the time you're at 7+ drops, you've accepted that you're probably not going to have choice of turn, and have built a list/strategy that first turn isn't important. Even though going from 9 to 8 at first glance looks like it yields large improvements, it's going from "not likely to have the choice" to, well, still not likely to have the choice. 

Additionally, the math is accurate, but somewhat deceptive.

I'd restate the percentage gain as what the decrease in odds of 'failure.' 

Couple of things -- for one, you will actually have the turn choice a slight majority of the time at 7 drops. I was really surprised by this, as a lot of people talk about it like you will never get the turn choice unless you are 5 or less. 

Second, I see what you are saying about the math being somewhat deceptive but the way that I framed it is actually very important to the later calculations in the post, and I'd argue that the way you are suggesting framing it is actually substantially more deceptive. For example, consider the gain from going down to 1 drop from 2 drops. In absolute numbers, your odds improve from 93.8% to 97.22%. So going from 1 to 2 increases your failure rate by 115%, or it's a 46.5% drop in failure rate going from 2 to 1. What I'm trying to do, however, is to paint a clearer picture of how this translates into win percentage. Let's imagine a really extreme example -- a list that wins 100% of the time when it has the choice of turn and wins 0% of the time when it does not. In this case, being 1 drop would have a 97.22% win percentage while being 2 drop would have a 93.8% winning percentage. Thus reducing from 2 drops to 1 would increase your winning percentage by 3.42%, or a roughly 3.6% increase in percentage terms. Sure, you could characterize this as a much larger reduction in "losing percentage" but I don't think many players really think that way.

Ideally, my goal is to help people think more clearly about the tradeoffs in list construction between managing drop count and other concerns. Most of the battalions in the game really aren't worth taking just for the bonuses that they provide, but are very much worth considering simply because they reduce drops. But I also think going too far in the other direction and suggesting that you have to be low drop can be a trap as well:

Consider the example above again. If you aren't really thinking about it clearly, you quickly conclude that a list that wins 100% of the time when it has the choice and 0% when it doesn't should do anything possible to get to 1 drop. But what if going from 2 to 1 drop requires diluting the list with battalion and tax unit point investments enough that the win rate when having the choice is reduced to 90%? Then reducing from 2 to 1 drops would actually be a clear downgrade in terms of overall expected value.

I think that framing it in terms of % reduction in the time you don't get the turn choice doesn't lend itself to this kind of calculation nearly as well.