I tried explaining this to a friend and ended up in a big argument. It’s hard to explain things. Communication among human and between animals are awesome when seen in this light.
What is Bayesianism?
This article is an attempt to summarize basic material, and thus probably won’t have anything new for the hard core posting crowd. It’d be interesting to know whether you think there’s anything essential I missed, though.
You’ve probably seen the word ‘Bayesian’ used a lot on this site, but may be a bit uncertain of what exactly we mean by that. You may have read the intuitive explanation, but that only seems to explain a certain math formula. There’s a wiki entry about “Bayesian”, but that doesn’t help much. And the LW usage seems different from just the “Bayesian and frequentist statistics” thing, too. As far as I can tell, there’s no article explicitly defining what’s meant by Bayesianism. The core ideas are sprinkled across a large amount of posts, ‘Bayesian’ has its own tag, but there’s not a single post that explicitly comes out to make the connections and say “this is Bayesianism”. So let me try to offer my definition, which boils Bayesianism down to three core tenets.
We’ll start with a brief example, illustrating Bayes’ theorem. Suppose you are a doctor, and a patient comes to you, complaining about a headache. Further suppose that there are two reasons for why people get headaches: they might have a brain tumor, or they might have a cold. A brain tumor always causes a headache, but exceedingly few people have a brain tumor. In contrast, a headache is rarely a symptom for cold, but most people manage to catch a cold every single year. Given no other information, do you think it more likely that the headache is caused by a tumor, or by a cold?
If you thought a cold was more likely, well, that was the answer I was after. Even if a brain tumor caused a headache every time, and a cold caused a headache only one per cent of the time (say), having a cold is so much more common that it’s going to cause a lot more headaches than brain tumors do. Bayes’ theorem, basically, says that if cause A might be the reason for symptom X, then we have to take into account both the probability that A caused X (found, roughly, by multiplying the frequency of A with the chance that A causes X) and the probability that anything else caused X. (For a thorough mathematical treatment of Bayes’ theorem, see Eliezer’s Intuitive Explanation.)
There should be nothing surprising about that, of course. Suppose you’re outside, and you see a person running. They might be running for the sake of exercise, or they might be running because they’re in a hurry somewhere, or they might even be running because it’s cold and they want to stay warm. To figure out which one is the case, you’ll try to consider which of the explanations is true most often, and fits the circumstances best.
via Less Wrong: What is Bayesianism?.