What I meant, which might have been unclear is that replacing a die in a dice pool with one with 4 more faces, is not - as far as my ramblings tried to figure out - as good as adding a die with 6. If you have a dice pool of three d8, add a d6, will basically increase the dice pool to four dice, whereas improving a die does not increase the dice pool size as such.

I believe that having the "dice pool" in mind when discussing this is important.

And I might have used the term "variant results" in a way not specific to an academic field. Statistics is not my strength within my field of study, even if I work with it (granted "work with" means documenting and "cleaning" before online publication of surveys).

So, I decided to do a couple of quick tests of this. I used a random number generator in my little perl script and had it roll each die 1,000,000 times in each test.

The results are

Number of successes, % for Ability, % for Proficiency, % for Ability + Boost

0 49.85% 33.35% 33.33%
1 37.64% 49.92% 41.76%
2 12.51% 16.73% 20.73%
3 0.00% 0.00% 4.17%
 

Upgrading the die results in a significant improvement (33% vs 49% chance of no successes; 50% vs 37% for just one success). Adding a single Boost die to the Ability die also helped, in certain areas. Slightly better chance of 2 successes (20% vs 16% for the proficiency) and the first to have 3 successes as a possibility (4% chance vs 0%).

I could probably extend the script to do lots of test runs of different pools to see how they all fair (success vs failure, advantage vs threat, triumph vs despair).

Kallabecca said:

Great to see that what I'm doing is being reproduced by some other interested parties.  Validation is absolutely key and I welcome people to question my findings or propose alternate methods for comparison.

Based on your results, the dice produce expected successes as follows:

• Ability ONLY: 0(.4985) + 1 (.3764) + 2(.1251) = 0.6266
• Proficiency ONLY:  0(.3335) + 1 (.4992) + 2(.1673) = 0.8338
• Ability + Boost:  0(.3333) + 1 (.4176) + 2(.2073) + 3(.0417)= 0.9573

Again based on these numbers we see the upgrade produces .8338 - .6266 = .2042 additional successes, and the ability + boost die produces .9573 - .6266 = .3307 successes.

Looking at a small set of examples with 1 to 2 dice makes some things become apparent I missed.  The big one is that an ability die + a boost die is just as likely to produce no successes as a proficiency die, and is overall more likely to produce 2 or more successes.  This comparison neglects the advantages produced, but is still an interesting finding.  

Extending that thought, it can be shown that an ability + boost die have a 1/8 * 2/6 = 2/48 = 1/24 (about 4%) probability of producing NO symbols, while a Proficiency die has 1/12 (about 8%) chance of producing no symbols.  Twice as high…

Again, Thanks Kallabecca!  I'd love to see you expand your perl script and see how your results compare for some of the dice pools I've posted.

-WJL

 Speaking of validation I should compare these simulated (based on 1,000,000) results to the initial expected values in the OP.

• Expected value of increased successes produced by upgrade: 5/24 = .2083, compared to simulated by Kallabecca: .2042
• Expected value of increased successes produced by boost + Ability: .3333 compared to simulated by Kallabecca: .3307

Seems pretty close.

-WJL

OK. Advantages were easy to add (just had to remember there is the possibility of 4 for Ability + Boost).

# of successes or advantages, successes from ability, advantages from ability

0 49.94% 50.00%
1 37.51% 37.48%
2 12.54% 12.52%
3 0.00% 0.00%
4 0.00% 0.00%

# of successes or advantages, successes from proficiency, advantages from proficiency
0 33.27% 50.06%
1 50.07% 33.34%
2 16.66% 16.60%
3 0.00% 0.00%
4 0.00% 0.00%

# of successes or advantages, successes from ability + boost, advantages from ability + boost
0 33.30% 24.99%
1 41.73% 35.42%
2 20.77% 27.08%
3 4.20% 10.43%
4 0.00% 2.08%
 

Not sure if FFG forums support tables in posts :(

I don't track the individual results, just the accumulated results. So, the first set of data means that each time you roll the Ability die 50% of the time will have no successes, 50% of the time it will have no advantages (which fits since half the die lacks a success, half the die lacks an advantage). It wouldn't be hard to track the actual possibilities, just didn't bother with this simple script.

OK, and here's the breakdown by the actual roll results (order doesn't matter so ssa and sas are the same result).

my key: a = advantage, s=success, T = triumph, t=threat, f=failure, D=despair

results are in the following order: dice results, ability die %, proficiency die %, ability+boost dice result %

a 25.00% 8.27% 10.43%
aa 12.47% 16.69% 10.41%
aaa 0.00% 0.00% 6.20%
aaaa 0.00% 0.00% 2.08%
aaas 0.00% 0.00% 4.18%
aas 0.00% 0.00% 12.54%
aass 0.00% 0.00% 4.15%
as 12.56% 25.03% 14.61%
ass 0.00% 0.00% 8.30%
asss 0.00% 0.00% 2.09%
s 25.02% 16.66% 10.44%
ss 12.53% 16.69% 8.36%
sss 0.00% 0.00% 2.06%
Ts 0.00% 8.31% 0.00%

A Triumph is also a success, so the Ts is so that future changes to the script can handle things like successes - failures since the success part of a Triumph can be countered by a failure.

LethalDose said:

That is known as the Central Limit Theorem. In a normally distributed population, the more samples you take the closer to the population mean and variance the results get. If I were to up the results another log you'd see the numbers close in on your expected result. Dice will also tend towards the mean as more dice are rolled since each die face has an approximately equal chance of coming up with each roll.

Kallabecca said:

Technically, you're citing the weak law of large numbers: As the sample size increases, the mean converges to the expected value.  While you're correct in that the CLT states that as the sample size increases (i.e. goes to infinity) , the mean becomes normally distributed, I urge you to be very careful about invoking it in situations involving simulated data.

in when simulating results like this, you can (and have) generated an arbitrarily large volume of data, meaning that you can create an arbitrarily small confidence interval around your mean/expected value which, yes, can be assumed to be normal because of CLT.  BUT that represents the distribution of the mean, not the distribution of simulated data.

What Ive found to be much more informative when discussing the source data is to report probability intervals (These are also called "credibility intervals", if you're feeling particularly Bayesian).  Basically, PI's give the values that bound (1-alpha)x100% of the data.  See some of my earlier posts for PI's.

My point in drawing attention to the estimated vs simulated values was just to show that your simulations are consistent with the values I've been presenting since the OP.

-WJL

I think this is an important thread.  It demonstrates statistically that some talents and abilities are not costed or gauged correctly and that's a big problem.  It honestly requires going back through the entire book with the knowledge that "Upgrades" are less valuable, and not more, than "Adding boosts" and correcting all abilities and maneuvers that used them.  I honestly think this is the most significant and important find in the Beta so far.  Bravo Lethaldose.

That depends… how many abilities need Triumph to fire off. The more abilities that need Triumph, the better the upgrades from ability to proficiency are (since they are the only dice with Triumph). This was just a blind test of one small condition. It hardly makes the case for needing to change everything, yet.

Kallabecca said:

This [I assume] is the exact reason that Sam Stewart stated above: 

FFG_Sam Stewart said:

The value of a triumph is, simply put, a f***ing b!tch to quantify.  A previous poster insisted I needed treat a triumph as advantages, to use his words:

koralas said:

While I think "unlimited pool of advantage" is just nonsense, I tried it in a very limited sense as a way of providing SOME kind of quantitative comparison (e.g. avoid apples to oranges) between the two mechanisms.  I found that when you consider the triumph to be about 5.5 advantages, the upgrade to the proficiency die is, on average, as effective as adding a boost die for producing advantages (not successes).

For the record, I am not stating that triumphs should count as 5.5 advantages! I'm merely stating a value to at which equivalency is reached in this comparison!

There are actually very few abilities that require a triumph to activate, the talents disorient and knockdown are the ones that come to mind.  Honestly, Knockdown typically just costs the target a maneuver to get up.  This could be more useful since it's in a melee tree and provides a melee bonus, bu theres also a big chance the prone target will just get up before the marauder has a second chance to attack him.  There are so few ranks of Disorient, I'd be surprised if any character will take more than 2, which means spending a triumph on it is unlikely to cause more than 1 setback die two turns in a row.  

Speaking of ways to spend triumph [non-narratively], i think its kind of funny that, based on what's been shown in this thread, reading the 6-2 on page 133:

• Triumph or two adv can be spent on a boost die to any allies roll.
• A Triumph can be spent to upgrade any allies roll.

If you've read the thread, this seems weird.  Also with only one rank in Disorient (the talent appears once in the gadgeteer tree and twice in the scout tree), a triumph can again be spent for the same benefit as two adv: Targeted enemy gains [Setback] on his next skill check.  Again, based on the math and simulations, these costs just seem borked.  

Now, these weakness of the upgrade vs the boost may be by design.  It may be the intent of the devs that players and GMs should use the triumphs for narrative purposes, not as mechanical boosts, because the potential to "Do something vital to turn the tide of a battle" seems to beat the crap out of an upgrade of ability to proficiency.  I lend very little credibility to this theory, but it bears mention.

Besides, if the triumph is the king of all symbols, and its the reason we need to upgrade… then using a triumph (a factual extant rolled triumph) to buy someone an additional 8% chance to roll a triumph… seems like a pretty raw deal.  I understand that some situations require an individual to roll a triumph, but they're not frequent enough to justify this.

Uh, yeah… Long answer to the first quote.  Hope this clarifies some things though…

-WJL

Taking what I did for the stats tests, I wrote a simple SW:EotE dice roller. You can see a demonstration of it on youtube. Feel free to leave feedback there on ideas that might improve the app.