The fifth in a series of seven fables/lessons/meditations on probability.
The teacher had a new mission for the student. “Many monkeys live in the valley below,” she said. “And it is said that in every 10,000 monkeys, there is a single one that possesses all knowledge.”
“All knowledge?” the student said. “What, does it wear robes and talk?”
The teacher ignored her. “You ask it questions, and listen to its replies,” the teacher said. “The Wise Monkey will coo for ‘yes,’ and grunt for ‘no.’”
“And how am I supposed to find it?” the student said.
“Go to the valley,” the teacher said. “Ask questions of the monkeys. If you find a Wise Monkey, it will answer correctly.”
“And what if I find an ordinary monkey?”
“What do you think?” the teacher scoffed. “It will answer randomly.”
The student set out for the valley. She moved from monkey to monkey, asking them questions until they got one wrong. A few monkeys showed promise, getting six, seven, or even (in a few cases) eight questions right. But they all erred sooner or later. As sundown neared, the student began to despair.
“Is purple my favorite color?” she said, approaching yet another monkey.
The monkey grunted. No.
“Right,” the student said. “Is green my favorite color?”
The monkey cooed. Yes.
As the right answers mounted, the student could scarcely believe it. Ten questions later, the monkey was 10 for 10.
“I found one!” the student announced when she’d brought the monkey back with her.
The teacher studied her monkey carefully. “How do you know?”
“I asked it ten questions,” the student reported. “It got them all right.”
“Couldn’t that be a coincidence?”
“I guess,” the student said. “But it seems awfully unlikely. Ten is a lot of questions.”
“But tell me,” the teacher said, “how many monkeys did you speak with today?”
“Hundreds,” the student said. “Maybe a thousand.”
“I see.” The teacher nodded. “Let’s say a thousand. Now, of those, how many got the first question right?”
“Well, by dumb luck, half of them,” the student said. “So 500.”
“Good. And how many got the second question right, too – again by dumb luck?”
“Half of those. So 250.”
“And the third question?”
“Half again. 125.”
“And the fourth?”
“63, more or less.”
“And the fifth?”
“32.” The student got a sudden sinking feeling.
“And the sixth? Seventh? Eighth? Ninth?”
The student tried to keep up. “16. Then 8. Then 4. Then 2.”
“And tell me, my student,” the teacher said, “Of a thousand ordinary monkeys, how many would get all 10 questions right, purely by chance?”
The student slouched. “One would.”
The teacher then turned to the monkey. “Tell me, little one, is the sky blue by day?”
The monkey grunted. No.
“Really, now,” the teacher said. “Is the sky yellow by day?”
The monkey cooed, and the student groaned.
Further Thoughts
This might remind you of the old adage that a thousand monkeys, given infinite time and a steady supply of typewriters, would reproduce the entire works of Shakespeare—minus the dubious “infinite time” part. Here, a thousand monkeys, given ten yes-or-no questions each, will produce one monkey who looks like a genius.
The moral amounts to this: Coincidences happen.
Think of sports—the octopus that accurately picked World Cup winners, or the pattern that the stock market dips when an AFC team wins the Super Bowl, or the countless fans who insist that their private rituals and practices steer their teams’ fortunes. You can chalk it all up to coincidence. If you try enough sea creatures, one is bound to predict the games correctly. If you look at enough meaningless indicators, one is bound to match the stock market’s fate, by sheer luck. And if you do enough random things each time your team plays, one of them is bound to correlate with the days your team wins.
But there are more pernicious examples. Consider money managers who claim they’re excellent at picking stocks. Gather together a thousand of these experts, and watch their performance for ten years. Track whether they perform above or below the market average. Even if their success is totally random, you’ll probably have someone who beats the market average for ten years running. Investors will flock to this wizard’s fund, but he might not be the Wise Monkey he seems to be.
Get the pdf, or check out other stories in the series!
I’d like to thank my father, James Orlin, for providing some foundational ideas for these stories, as well as helpful feedback and conversations. Also for being one fun son of a gun.
So the question is, how many questions does the monkey have to get in order to be convinced he is the wise one?
As for the monkeys typing the entire works of shakespeare… 884421 words, assuming 3 letters per word, ignoring punctuation and spaces. Odds of randomly typing all the letters: 1 in 26^2653263 approximately 8 million digits in that. . . Really would need infinite time!
Yeah, that Shakespeare one is a little misleading, just because it takes SO MANY multiples of the universe’s lifespan before you’re going to get even one scene, much less an entire play (or the complete works).
The “wise monkey” question depends on your Bayesian prior going in. If you think there’s a 1 in a million chance for a wise monkey, then 20 right answers will have you almost persuaded. But if you start by thinking it’s a 1 in a trillion chance, it’ll take more like 40 answers.
A bigger problem is that monkeys don’t type randomly. They bash the keyboards. You might get a lot of dddddddddddd and sddsdsdsd and njmnjjmmm, but you are unlikely to get anything resmbling even one word, unless that word is ass.
My favorite about stock pickers is much less coincidental. Make enough predictions and you are bound to be right some of the time. In fact make a prediction in both directions in slightly different ways and you will be right all the time as long as you only publicize the correct ones. For all those who predicted the crash of 2008 how many are rich from investing the right way during that time? Very few, but it’s amazing how many have claimed they predicted it.
That’s well-put.
A favorite example of our selective memory for predictions: We were making Oscar predictions in college. Everybody knew Slumdog Millionaire would win Best Picture, but one friend picked Milk.
When I asked him why, he said: “If I get it right, I look like a genius. And if I get it wrong, all that happens is I look like a guy who doesn’t know anything about the Oscars. So there’s nothing to lose.”
Oo oo oo, probability and coincidences are so cool. Great post, Ben.
It’s amazing how easy it is to confuse the probability that an event will happen in a particular instance with the probability that an event will happen in *some* instance. I think this may be what makes the birthday “paradox” feel “paradoxical”.
The stock picker thing is really interesting. One thing I often wonder in particular is whether Warren Buffett is *that* smart or if he’s just the lucky bastard who happened to make a bunch of lucky predictions. There are apparently actual cases of people mailing (before the internet) stock predictions to 2^7 * 100 people, then mailing second stock predictions to the 2^6 * 100 who received correct predictions the first time, etc, and then when there were 100 people who had received seven correct responses, they were asked for some huge amount of money for an eighth prediction.
That’s well-put: “the probability that an event will happeni n a particular instance” vs. “that an event will happen in *some* instance” exactly captures the common mistake.
My hunch is that Warren Buffet really does know what he’s doing (especially given that his method is to focus on detailed analysis of companies’ underlying value, rather than on the fluctuations in stock price; although it’s still possible he’s just that one chronically lucky guy). Anyway, I’m also a fan of that particular scam–I actually have a (somewhat poorly worded) question about it in the pdf for this chapter.
Co-incidences, startling co-incidences, like that of the parrot that accurately described the streets of London after listening to Shakespeare-Strange world.
Whoa – I’ve got to go look that up now!
So what you are telling me is the fact that I didn’t wear my lucky Cardinals socks last night and we parked in a different lot for Game 5, was not the reason my team lost the past 2 games. Whew, glad to take the burden of the Cards not winning the World Series off my shoulders! Although, no matter how convincing the math is, I will probably still make sure I wear lucky socks and park in the same lot next year during post season…just in case. 🙂
Great post! Thanks!
I’m saying the reason my Red Sox won is that I DID wear my lucky socks!
(But seriously, a good World Series. I love and fear your guy Craig.)
Interesting story, Ben. I’ve been thinking about this a few days, and perhaps another way of thinking about this problem is the probability of a monkey randomly getting 10 questions right (instead of the expected number of monkeys that will get 10 questions right). In this case, the probability of finding at least one monkey getting 10 questions right is 0.624 (=1-(1-0.5^10)^1000); getting 11 questions right is 0.386; 12 is 0.216; 13 is 0.115; 14 is 0.059; 15 is 0.030. So as Danny writes, another natural problem is how many questions does a monkey need to answer correctly before we believe he possesses all knowledge. From a statistical point of view, I would only believe a monkey possesses all knowledge if there is less than a 5% chance (if we choose alpha=0.05) I would observe so many correct answers if this monkey were randomly guessing (i.e. null hypothesis is there isn’t a monkey possessing all knowledge). Therefore, even if this monkey answered another 4 questions correctly, I still wouldn’t believe he possesses all knowledge; only until he answers 15 questions would I be convinced (under significance level 0.05).
Yeah, good thoughts. I think you’ve got the right framework for analyzing the problem in depth–my frequency-based approach is good for intuition but bad for precision.
Personally, I might set the bar lower than 5%. But it’s interesting that the 5% threshold gives us a pretty reasonable 15-question benchmark.
The stock picker thing is even worse. A money manager may start 100 or 1000 funds, picking stocks at random. After a year, he sees which ones are doing well, by luck. Then those get promoted as proof of his stock picking wisdom! The connection is to survivor bias: We tend to evaluate the thing we’re looking at on its own terms (Wow, this stock portfolio did so well!) rather than recognize it was one of 100 or 1000 trials. You need to know n.
Yeah. This is exactly why I’m scared to invest in mutual funds, and go with index funds instead.
I’m not satisfied with the parable for a number of reasons. Most importantly, the teacher’s reasoning failed to use the highly relevant information given in the problem that 1 in 10,000 monkeys in the valley is wise. It therefore teaches the wrong lesson in my view. Yes, coincidences happen. But making a good decision requires assessment of which of two or more possibly unlikely events is more likely, and by how much.
The student’s problem (implicit I think in the parable) is to decide whether monkey # 1000 is wise after finding 999 monkeys that are demonstrably ordinary. Assuming that we are sampling monkeys randomly from a large population, it is entirely irrelevant to this question that we first found 999 ordinary monkeys. For the same reason, the expected number of ordinary monkeys out of 1,000 that would get 10 answers right is not particularly relevant for making a reasonable decision. The teacher leads the student down a chain of logic that reaches its inevitable end because 1000 is roughly the reciprocal of (1/2)^10. But this information is not helpful for deciding whether the last monkey is wise or just lucky.
To make a wise decision it is not enough to notice that coincidences can happen. The teacher should have focused on the relative probabilities of the possible events that are consistent with the data. In this case there are two relevant events: Either the 1,000th monkey is wise or he is ordinary and lucky. The probability that we selected a wise monkey is .0001 and the probability that we found an ordinary but lucky monkey is .9999*(1/2)^10=.0009765 (approximately). So the odds that we found a lucky monkey rather than a wise one are roughly 10 to 1. Comparing those numbers is important for putting the possibility of coincidence in perspective. Neither one is very helpful by itself. For example, if instead there were 1 wise monkey for every 1,000 population then the odds would be slightly better than even that we had found a wise monkey. (They would be greater still if we were sampling from a finite population without replacement because in that case the elimination of 999 unlucky monkeys raises the probability that the next one will be wise.)
So yes, it is possible for coincidences to occur, and improbable events are likely to happen if we wait long enough. But both of the possibilities in the example are so unlikely that they could fairly be labelled “coincidences.” The right thing to think about is which one is more unlikely, and by how much.
Hey Scott, I think that’s exactly the right way to frame this issue. The original version of this chapter included a very similar discussion, which (for reasons of length) I cut and converted instead into a multi-part question, which is now in the pdf version. I hope that anyone teaching a full lesson on this theme would guide students through an analysis similar to yours.
There’s no denying that the version I present above (without following through on the comparison to the baseline 1/10,000 rate) is simplified and incomplete, though I still hope it may function as a helpful starting point for exploring these issues.
Ah, I remembered two other reasons why I didn’t include this sort of Bayesian analysis in this chapter:
1. It already appears in Ch. 4, The Swindler’s Coin, and I wanted to focus on different ideas here.
2. It’s not clear at all that we can TRUST this base rate of 1/10,000! My own base rate assumption for a monkey understanding and responding accurately to human speech would be far smaller, even if a wise teacher told me otherwise.
The thing with the index funds is actually worse than you think: in many cases, supposed stock market experts actually perform worse than randomness (but still marginally better than supposed psychics, which I guess is good).