Play-testing Mathematics: Some Junk Science for Your Approval
by , 01-08-2011 at 04:51 AM (9709 Views)
I play-tested an adventure today for the new Dark Sun living campaign premiering at DDXP later this month, and that inspired me to finally publish this article. Iíve sat through a few play-tests for Living Greyhawk and Living Forgotten Realms (LFR) in my time, but not a lot. I also ran play-tests of my 1st edition throwback delves for synDCon, but those tests were more concerned with how long it would take to complete the delve rather than actual encounter balance. In any case, Iím not a seasoned vet when it comes to play-testing, but Iíve seen wide variation in how adventures play out, and not simply because the DM doesnít play the NPCs well, or because thereís a string of natural 1s or natural 20s. Some adventures seem consistently impossible or consistently painless, and neither of those explanations seems to apply.
Without breaking into some serious mathematical analysis that Iím too lazy to do, I have an idea on how to improve the process. I have no idea whether WotC, Green Ronin, Steve Jackson Games, or any other established game publisher (to whom Iím not giving a shameless plug) uses a modified means of play-testing, but I suspect that at least LFR doesnít. Keep in mind, though, that Iím not trying to dissuade a designer from making an especially hard or especially easy encounter or adventure. Iím simply trying to improve the method for determining whether or not they have successfully done so.
Being that my only significant RPG experience is with D&D, that system (specifically, 4th edition) will be the subject of this analysis. Chris Sims wrote that DnD is balanced so that you should usually hit when you roll a 9 on your d20 (link not available due to Loremaster crash ). Not always, but usually. That means that you should hit 12 out of 20 times, or 60%. Iím assuming that this applies to hitting a skirmisher, lurker, or controller. Brutes and artillery have lower defenses, and soldiers higher. Iím also assuming that this assumes each PC role (leader, striker, controller, and defender) has the same attack bonus, which I know is often wrong. That being said...
Why Not Just Take 9?
To keep things simple and even, just eliminate d20 rolls altogether. It would speed up the otherwise long process of play-testing, allowing more tests. Sadly, that wouldnít work well. In a given encounter, taking 9 could be an automatic hit or automatic miss for any given PC, and you donít want to effectively take a character out of the Ďtest.í No matter how hard or easy the encounter, any character should have the ability to hit or miss. Taking 9 kills that altogether. You still need some random element.
How About a d6 Spread?
My initial thought was to narrow the spread by replacing the d20 roll with 1d6+6. This way, youíre rolling between 7 and 12, and youíre expected to hit 66% of the time. This isnít precisely the same chance provided by the d20, but itís close. Unfortunately, this suggestion screws with the math a bit too much. For example, an ordinary soldier has an AC 2 higher than an ordinary skirmisher. Under the d20 system, an increase of 2 in the NPCís AC should result in a drop from 60% to 50% on your hits. Under the 1d6+6 system, you now hit only 33% of the time. Just by throwing in a soldier (or any NPC given an unusual boost to its AC), you skew the system from being too easy to hit to being unreasonably hard to hit. This wonít result in a good test.
Just Eliminate Crits and Fumbles?
Instead, we can keep the d20, but simply do the math when you roll a 1 or 20, granting neither auto-misses nor critical hits. Of course, the 1s will still miss, and the 20s will still hit, but the critical damage will be ordinary. This keeps the to-hit math 100% on target but eliminates the game-changing effects that are best left to spice up an actual adventure. It also would be easiest for experienced players to implement and probably most mathematically accurate.
Thereís still a huge problem with this, though: The larger the spread, the greater the possibility that youíll get streaks of hits or misses, which I believe is actually the bigger problem. If combat lasts only, letís say, six rounds, missing four times in a row effectively takes you out of the combat. That doesnít properly test the adventure either.
(As I side note, I take complete advantage of these streaks when I play blackjack. As a result, my winnings always pay for my hotel, airfare, and expenses when Iím in Vegas or Atlantic City. Trust me when I say these streaks can change everything.)
Take This with a Grain of Salt: A d10 Spread Is the Answer
The obvious compromise between d6s and d20s is to replace the d20 with d10+4, rolling between a 5 and 14. This will have you hitting 60% of the time. The math is still a bit screwed up; the above example would result in a drop from hitting 60% of the time to hitting 40% of the time. However, the lower the spread, the more of a contribution your PC will give, and thatís best for determining the overall strength and weakness of the encounter.
Perhaps producing a random +1 (or -1 against brutes and artillery) to hit would correct this error while minimizing the larger spread of a d20. This random effect shouldnít be based on the attack die itself. It could be based on a second die roll, or alternating from round to round. If the latter, play-testers would have to avoid meta-gaming their power choices around whether they were going to get the +1 bonus to hit in a given round.
I admit that this would make play-testing a complicated process, but it would (arguably) make the final product better, and youíd get the glory of your name appearing on the final product! Of course, if you're publishing professionally, complaining about the complication isn't justified at all. This is your job, dammit!
Final Note: Whatís So Bad About Natural 1s and 20s?
Let me give a bit more substance to this position. Natural 1s and natural 20s are supposed to represent rare occurrences of heroism and screw ups. They add spice to the game, which in turn makes the game more fun, but play-testing isnít supposed to be fun. Itís supposed to be work. Itís work that hopefully leads to others having fun, but work nonetheless. You shouldnít miss them in a play-test, and in fact they probably interfere with a fair analysis of whether the adventure is properly balanced. I strongly believe they have no place in play-tests. Iíve been through far too many encounters that donít have a single natural 20 to think that they need to be built into the testing process.
Talk to Me, People
So, what do you think? Agree? Disagree? In whole or in part? Iíd be interested in others helping me expand on this idea and forming a stronger battle plan for play-testing. Iíve been play-testing quite a bit lately, and plan to do some more in the near future.