Multiple Choice Questions and Trivial Pursuit OR Sig Figs and 'Educated' 'Guessing'

Over the holidays I happened to be playing Trivial Pursuit with some friends.  I don't play Trivial Pursuit that often, and this was a different version than I've played before.  For reference, it was the Trivial Pursuit: Master Edition.

The edition of Trivial Pursuit I have sitting around somewhere is getting kind of old, and it's definitely what I base my Trivial Pursuit worldview on.  Unlike this old version, this Master Edition had a decent number of multiple choice questions come up.  Some were quite easy, and some were tricky but still offered a good chance at guessing them right.

A few questions came up with numbers in them, and the idea was floated that one way to potentially tell the correct answer (if you knew nothing else) might be to look for the numeric choice that had the greatest number of significant figures.

Some of you might be right there with me, but some of you might have just stalled out, so let's have a quick discussion of significant figures.

Put simply, significant figures are a implicit (or explicit) conveyance of a number's precision.  From a scientific standpoint there are a number of very important rules regarding significant figures and how to treat and convey them.  For our purposes we can deal with the more perceptual nature that you might recognize in your day to day use.

The thing that's consistent across both of these standpoints is that significant figures are all the digits of a number that are not: a) non-notated leading or trailing zeros (e.g. all the zeros in the number 10,000; none of the zeros in 1,000,001) or b) some other number that only came about through a derivation of a bunch of those leading or trailing zeros.  For example, the number 500 has one significant digit - the five.  If we divide that by 2 we would seemingly pick up another significant figure by arriving at 250.  If we divide by 3 we get seemingly infinite significant figures by arriving at 166.66666(repeating).

It doesn't work that way, and if you're so inclined I'd suggest reading up on scientific significant figures and calculations involving significant figures.  There's no need to get that deep into it today, so we'll finish this part with a perceptual example.

Imagine that you're wondering how many people there are in the United States.  You go to a friend and ask them if they know how many people there are, and they say 300,000,000.  What does that mean to you?

Well, my read of that situation is that your friend only really has confidence in the hundred millions digit of population.  They have confidence in the precision of that 3(hundred million), but maybe not in any of the other, smaller digits.

Feeling like this friend doesn't know their stuff you find another friend, and they say that the population of the United States is 311,000,000.  This friend might instill in you a bit more confidence, as they've implicitly given you a better sense of precision by giving you two more digits than your first friend.

You're still not happy with the answer from this friend and find a third.  They're spending more time on their smartphone than they are paying attention to you, and when you ask the question they take a moment of typing before they answer: 311,591,917.  They then go back to not paying attention to you.

This friend has - correct or not - given you much more precision in their number than either of your first two friends.  They have done this, in part, by giving you 9 significant figures instead of only 1 or 3.

The idea as it relates to Trivial Pursuit is the idea that correct answers inherently have more significant figures, as they are actual quantities, and wrong answers may have less.

Well, it's testable, so let's take a look.

I was able to code some data from the cards relating to every multiple choice question asked in Trivial Pursuit: Master Edition.  It turns out that there are 245 of these questions in the game.

245 is a pretty nice number, but not all those questions involve numbers.  In fact, numbers are the minority - the vast majority of questions simply involve word choices.  There are 36 multiple choice questions involving count numbers, and an additional 6 involving percentages.  This unfortunately doesn't give us a whole lot to work with in terms of significant figures, but there's other things we can look at in a bit.

It's also the case that the overwhelming number of cards with numbers on them have numbers which all have the same number of significant numbers.  I mean figures, significant figures.  Overall, there are only 10 questions which have a difference we can examine.  These ten cards have answers in numeric form, and there is a difference between significant figures between the correct answer and the other answers.

These are the cards where we can test out if more significant figures mean better answers.

Well, it doesn't seem to.  Of those 10 questions, 9 of them actually have a correct answer which has less significant figures than at least one of the other options.  In only one case does the correct answer have more significant figures than the other answers.

This might mean that the folks over at Hasbro are on to your clever tricks.  They may have set this up on purpose to trip up exactly the type of logic we've just laid out.  There's other places we can look, though - there are a lot of other multiple choice questions. 

In fact, there are two main types of word questions.  There are questions that ask you to pick the correct answer from a number of choices, and there are questions which ask you to pick the answer that is 'NOT' something, or that fits the idea of all of these 'EXCEPT'.

The most common question is the first type, and those of them with three response options to choose from - there are 93 of these questions.  So is there any information we can use to make better guesses in these situations?

Well, it's pretty straightforward in this case, too.  The numbers are really close - the first option is correct 30 times, the second was correct 29 times, and the third was correct 34 times.  If you have absolutely no idea you do have slightly better odds selecting the third choice, though the gains you get from it aren't very large.  These numbers don't significantly deviate from an even split. 

Fortunately, things get a little more interesting as we move on to some of the other questions.

So far we've covered questions with numeric answers, and word questions looking for the correct answer among three answer choices.  There are also some multiple choice questions with four answer choices.

For those questions, the breakdown seems a bit less random.  In fact, of 22 questions, in an even half of them (11) the second answer choice is the correct answer.  The first and last options are the least likely, with each being correct 3 times, and the third option is only a little better - it's the correct choice 5 times.

While the low number of these questions stops us from saying too much on a statistical side (if we wanted to perhaps generalize to other Trivial Pursuit games?), these numbers do give a bit of an advantage to the player forced to guess on one of these questions.  With no other credible information it makes sense to guess the second option any time four are presented for this version - you'll be right by chance half the time.  If you can eliminate one of the other choices you're going to be above 50/50 odds. 

Let's move on to those questions where you're asked to pick the answer that is 'NOT' or 'EXCEPT' something.  There aren't many of those with only three answer choices - 14 in total.  Your odds increase slightly as you move from the first to last choice - the breakdown of times each are correct is 3/5/6.

'NOT'/'EXCEPT' questions with four answer choices are a bit more plentiful, with 46 in total.  You'll again do a bit better going with later answers if you're randomly guessing.  The breakdown in this case is 7 for the first response, 11 for the second response, 11 for the third response, and 17 for the fourth response.

That's not to say that always choosing based on these numbers is going to always get you a correct answer - in the best situations this knowledge might buy you 50/50 odds (in the case of regular questions with four responses).  There's only 22 of those questions out there, which is only about a tenth of even just the multiple choice questions.  Those questions might be a tenth of the total questions in the game, so you're down in some pretty small territory.  You also have to be playing *this* specific build of Trivial Pursuit, so the help actually seems fairly limited.  Help is still help, though, I suppose.   

That is - of course - unless the people you are playing with *also* read this blog, in which case I would advise them to switch around the answer that you read out the answers to these multiple choice questions.  Then you get into some interesting rock-paper-scissors territory, which is something that I should talk about some other week.