Probability seems simple enough to many people that it can fool them into wrong conclusions. We have had many questions that involve the “Gambler’s Fallacy”, both from people who naively assume it without thinking, and from some who defend it using technical ideas like the Law of Large Numbers.
The Gambler’s Fallacy
Here is a question from 2003 that is a good introduction to the idea:
Gambler's Fallacy A current co-worker and I are in a friendly disagreement about the probability of selecting the winning number in any lottery, say Pick 5. He states that he would rather bet the same set of five numbers every time for x period of time, but I insist that the probability is the same if you randomly select any set five numbers for the same period of time. The only assumption we make here is betting one set of numbers on any given day. Who is correct? I tried explaining to him that the probability of betting on day one is the same for both of us. On day two it is the same. On day three it is the same, etc. Therefore the sum of the cumulative probabilities will be the same for both of us.
Doctor Wallace first examined the appropriate calculation, before moving on to the likely underlying error:
You are correct. If you have the computer randomly select a different set of 5 numbers to bet on every day, and your friend selects the same set of numbers to bet on every day, then you both have exactly the same probability of winning. Tell your friend to think of the lottery as drawing with tickets instead of balls. If the lottery had a choice of, say, 49 numbers, then imagine a very large hat containing 1 ticket for every possible combination of 5 numbers. 1, 2, 3, 4, 5; 1, 2, 3, 4, 6; etc. On the drawing day, ONE ticket is pulled from the hat. It is equally likely to be any of the C(49,5) tickets in the hat. (There would be 1,906,884 tickets in the hat in this case.) Since both you and your friend have only ONE ticket in the hat, you both have the same chance of winning. On the next drawing day for the lottery, ALL the tickets are replaced. Each lottery draw is an event independent of the others. That is to say, the probability of any combination winning today has absolutely NO effect on the probability of that or any other combination winning tomorrow. Each and every draw is totally independent of the others.
That perspective makes it “obvious”: if two people each have one “ticket”, they have the same probability, whether or not it is the same one that was taken last time, since the “ticket” is chosen randomly each time without regard to the past.
So why is the other opinion tempting?
The reason your friend believes that he has a better chance of winning with the same set of numbers is probably due to something called the "gambler's fallacy." This idea is that the longer the lottery goes without your friend's "special" set of numbers coming up, the more likely it is to come up in the future. The same fallacy is believed by a lot of people about slot machines in gambling casinos. They hunt for which slot hasn't paid in a while, thinking that that slot is more likely to pay out. But, as the name says, this is a fallacy; pure nonsense. A pull of the slot machine's handle, like the lottery draw, is completely independent of previous pulls. The slot machine has no memory of what has come before, and neither has the lottery. You might play a slot machine for 2 weeks without hitting the big jackpot, and someone else can walk in and hit it in the first 5 minutes of play. People wrongly attribute that to "it was ready to pay out." In reality, it's just luck. That's why they call it gambling. :)
The same thing comes up in math classes:
This used to be a "trick" question on old math tests: "You flip a fair coin 20 times in a row and it comes up heads every single time. You flip the coin one more time. What is the probability of tails on this last flip?" Most people will respond that the chance of tails is now very high. (Ask your friend and see what he says.) However, the true answer is that the probability is 1/2. It's 1/2 on EVERY flip, no matter what results came before. Like the slot machine and the lottery, the coin has no memory.
That type of question is still valuable. It tests an important idea that students need to think about.
For more about picking the same number in a lottery, see
Lottery Strategy and Odds of Winning
For a very nice refutation of the Gambler’s Fallacy in coin tossing, see
What Makes Events Independent?
The Law of Large Numbers
That question was based on a naive approach to gambling. The next, from 2000, is based on probability theory. (The questions were asked in imperfect English, which I will restate as I understand it, correcting a misinterpretation in the archived version. Doctor TWE got it right.)
Law of Large Numbers and the Gambler's Fallacy If we throw three dice at a time, three times altogether, is the result the same as if we throw nine dice one time? Do we have the same probability to get a given number in the two cases? Where and how different is it?
Doctor TWE answered, covering several possible interpretations of “result”:
The sum, the average, and the probabilities of getting a particular value on one die or more aren't affected by whether the dice are rolled one at a time, in groups of 3, or all 9 at once. These are called "independent events," and the order in which they happen doesn't affect the outcome.
Elvino replied,
So, I have 42.12% probability to get at least one six (or other number), hen I throw three dice at once. If I throw the same dice, in the same way, again and again I will always have the same probability! But, there is a law that says, the more I throw, the more probability I have to obtain a certain number. How does this law of probabilities work?
Checking his calculation, and thereby confirming what he meant, I get the probability of rolling at least one six on three dice to be the complement of rolling no sixes: \(\displaystyle 1 – \frac{5^3}{6^3} = 42.1%\). Next time I will discuss this further.
Doctor TWE confirmed his calculation, then explained what this law means, and does not mean:
There is something called the Law of Large Numbers (or the Law of Averages) which states that if you repeat a random experiment, such as tossing a coin or rolling a die, a very large number of times, your outcomes should on average be equal to (or very close to) the theoretical average. Suppose we roll three dice and get no 6's, then roll them again and still get no 6's, then roll them a third time and STILL get no 6's. (This is the equivalent of rolling nine dice at once and getting no 6's, as we discussed in the last e-mail; there's only a 19.38% chance of this happening.) The Law of Large Numbers says that if we roll them 500 more times, we should get at least one 6 (in the 3 dice) about 212 times out of the 503 rolls (.4213 * 503 = 211.9). This is *not* because the probability increases in later rolls, but rather, over the next 500 rolls, there's a chance that we'll get a "hot streak," where we might roll at least one 6 on three or more consecutive rolls. In the long run (and that's the key - we're talking about a VERY long run), it will average out. There is also something called the Gambler's Fallacy, which is the mistaken belief that the probability on the next roll changes because a particular outcome is "due." In the example above, the probability of rolling at least one 6 in the next roll of the three dice (after three rolls with no 6's) is still 42.13%. A (non-mathematician) gambler might think that the dice are "due," that in order to get the long-term average back up to 42%, the probability of the next roll getting at least one 6 must be higher than 42%. This is wrong, and hence it's called "the Gambler's Fallacy."
The important thing here is that things will “average out” in the long run, so that you get at least one 6 in 42.1% of the rolls, but not because at any one time there is a greater chance, to make up the average — it happens only because there is a very long time to do so. It is not because past events have any effect on future events, pulling them into line with the “averages”.
For more on the Law of Large Numbers, see
The Law of Large Numbers
Digging deeper
In 2008, we had the following detailed question about that from Konstantinos:
Do Prior Outcomes Affect Probabilities of Future Ones? What I want to know is if in matters of luck, such as games of dice, or lotteries, or flipping coins, the future outcomes have any relativity with past results. What I mean in each case is: If I flip a coin and get tails, don't I have bigger or smaller possibilities to get heads on the next roll? I mean I know that the possibility is always 1/2 but since I have already thrown the coin 5 times and rolled 5 tails there aren't possibilities that in the next throws there will be heads? The same question goes for dice rolls, and for lotto numbers. If a number has come more times than others, isn't it possible that for a limited amount of coming times, numbers that haven't come yet, will start showing more? What I find most confusing is that the relation between probabilities and past possibilities, and the outcome in real life. How can a coin come tails 5 or six times in a row when the possibility is always 1/2? Are probabilities only theoretical? I tried noting down results of different trials of luck (dice, past lotteries, coins) but in the end they don't seem to make true to any theory I have heard. I would like to see the magic of numbers in real life and on this subject and how it works as to prove a theory. Do I have to toss the coin 10,000 times? And if I do will I see heads coming in a row after 10 subsequent tails?
I responded to this one:
What you are suggesting is called the Gambler's Fallacy--the WRONG idea that future results of a random process are affected by past results, as if probability deliberately made things balance out. The law of large numbers says that it WILL balance out eventually; but that does not happen by changing the probabilities in the short term. The long-term balance just swamps the short-term imbalance. If you think about it, what could cause a coin to start landing heads up more often after a string of tails? There is no possible physical cause for this; the coin has no memory of what it, or any other coin, previously did. And probability theory does not make things happen without a physical cause; it just describes the results. If I tossed a coin and it landed tails 5 times, I would just recognize that that is a perfectly possible (and not TOO unlikely) thing to happen. If I got tails 100 times, I would NOT expect the next to be heads; I would inspect the coin to make sure it actually has a heads side! An unlikely string of outcomes not only does not mean that the opposite outcome is more likely now; it makes it LESS likely, because it suggests statistically that the basic probability may not be what I was originally assuming.
In response to the question about probabilities being merely theoretical, I explained what probability is and isn’t:
Probabilities are theoretical, but have experimental results, IN THE LONG RUN. The law of large numbers says that, if you repeat an experiment ENOUGH TIMES, the proportion of times an event occurs will PROBABLY be CLOSE to the predicted theoretical probability. Probability can't tell you what will happen the next time, but it does predict what is likely to happen on the average over the next, say, million times. If you started out with ten tails in a row, you will not necessarily get ten heads in a row at any point, or even more heads than tails in the next few tosses; you will just get enough heads in the next million to keep it balanced.
Note all the capitalized words, emphasizing the vagueness of this statement. A formal statement of this theorem will define those more clearly, but they will then all become statements of probability (my “probably”) and limits (my “enough” and “close”). See, for example, the formal statements of the Law of Large Numbers in the Wikipedia page (my emphases in the last paragraph):
The weak law of large numbers (also called Khinchin’s law) states that the sample average converges in probability towards the expected value \(\displaystyle \overline {X}_{n}\overset{P}{\rightarrow} \mu \ {\textrm {when}}\ n\to \infty .\).
That is, for any positive number ε, \(\displaystyle \lim _{n\to \infty }\Pr \!\left(\,|{\overline {X}}_{n}-\mu |>\varepsilon \,\right)=0\).
Interpreting this result, the weak law states that for any nonzero margin specified, no matter how small, with a sufficiently large sample there will be a very high probability that the average of the observations will be close to the expected value; that is, within the margin.
What it does not say is that over, say, 100 trials (or 10,000 trials), we can be sure that the average will be exactly what we expect.
I continued:
In particular, if you were to throw five coins at a time (to make it easier than throwing the same coin five times in a row), and do that 1000 times, you would expect that you would get 5 tails about 1/32 of those times (since the probability of all five being tails is 1/2^5). That's about 31 times! So it's not at all unreasonable to expect that it will occur once in a while. On the other hand, the probability of getting 100 tails in a row is 1/2^100, or 1/1,267,650,600,228,229,401,496,703,205,376, which makes it very unlikely that it has ever happened in the history of the world, though it could!
Again, probability can't tell you what WILL happen, specifically; it is all about unpredictable events. But if you tossed a coin 1000 sets of 10 times, on the average one of those is likely to yield 10 tails. (The probability is 1/2^10 = 1/1024.) The probability of some ten in a row out of 10,000 tosses is a little bigger, but that's harder to calculate.
Regression to the mean
In an unarchived question from 2014, Adam brought in another idea:
If I flip a coin 10 times, the most likely outcome is that I will flip a total of 5 heads and a total of 5 tails. If each round of coin flipping (one round being 10 flips, in my example) is independent of previous rounds, then the probability of flipping a total of 5 heads and a total of 5 tails never changes. However, the concept of "regression to the mean" implies that "rare" events are likely to be followed by less rare events and vice versa. So, if I flip 10 heads in a row in round 1 (a rare event), the odds of flipping a total of 5 heads and 5 tails in round 2 are greater than if I'd flipped 5 heads and 5 tails in round 1. What is the correct way to see this, do the odds change or not? Coin flipping rounds are independent of one another, which implies that the probabilities never change. Regression to the mean states that rare events are likely to be followed by less rare events, implying that the probability of even random events does change.
I replied:
The odds don't change. You are NOT more likely to flip 5 heads if you previously flipped 10. Regression to the mean only says that the most likely event on ANY toss will be about 5 heads, so if you got 10 on one toss, you are more likely to get something closer to 5 on the next toss, simply because something closer to 5 is ALWAYS more likely than 10. If you got 5 on the first toss, you are as likely to get 5 again as ever; but any deviation that does occur will be away from the mean, because there is no direction to go except away! It is an entirely wrong interpretation of the phenomenon to think that, having tossed 10 heads, you are more likely to toss 5 heads than under other circumstances. It's just that the event that is always more likely will be less extreme than that first toss. See this page: http://en.wikipedia.org/wiki/Regression_toward_the_mean
That Wikipedia page says this:
Regression toward the mean simply says that, following an extreme random event, the next random event is likely to be less extreme. In no sense does the future event “compensate for” or “even out” the previous event, though this is assumed in the gambler’s fallacy (and variant law of averages). Similarly, the law of large numbers states that in the long term, the average will tend towards the expected value, but makes no statement about individual trials. For example, following a run of 10 heads on a flip of a fair coin (a rare, extreme event), regression to the mean states that the next run of heads will likely be less than 10, while the law of large numbers states that in the long term, this event will likely average out, and the average fraction of heads will tend to 1/2. By contrast, the gambler’s fallacy incorrectly assumes that the coin is now “due” for a run of tails, to balance out.
So regression to the mean happens merely because, after a rare event, most possible events are less rare — hardly a surprising fact! The result of flipping ten coins is usually less than 10 heads, whether you just threw 10 heads or not.
For some other discussions of aspects of this subject, see
Past Events and Probability Batting Averages