Thursday, April 25, 2013

On Bayesian Statistics


The following quote from Judea Pearl is far more eloquent than I can ever be:

___________________________________________________
"I turned Bayesian in 1971, as soon as I began reading Savage’s monograph The Foundations of Statistical Inference [Savage, 1962]. The arguments were unassailable: (i) It is plain silly to ignore what we know, (ii) It is natural and useful to cast what we know in the language of probabilities, and (iii) If our subjective probabilities are erroneous, their impact will get washed out in due time, as the number of observations increases.

Thirty years later, I am still a devout Bayesian in the sense of (i), but I now doubt the wisdom of (ii) and I know that, in general, (iii) is false. Like most Bayesians, I believe that the knowledge we carry in our skulls, be its origin experience, schooling or hearsay, is an invaluable resource in all human activity, and that combining this knowledge with empirical data is the key to scientific enquiry and intelligent behavior. Thus, in this broad sense, I am a still Bayesian. However, in order to be combined with data, our knowledge must first be cast in some formal language, and what I have come to realize in the past ten years is that the language of probability is not suitable for the task; the bulk of human knowledge is organized around causal, not probabilistic relationships, and the grammar of probability calculus is insufficient for capturing those relationships. Specifically, the building blocks of our scientific and everyday knowledge are elementary facts such as “mud does not cause rain” and “symptoms do not cause disease” and those facts, strangely enough, cannot be expressed in the vocabulary of probability calculus. It is for this reason that I consider myself only a half-Bayesian."

From:  "Bayesianism and Causality, or, Why I am Only a Half-Bayesian"  by Judea Pearl, In D. Corfield and J. Williamson (Eds.) Foundations of Bayesianism,  Applied Logic Series Volume 24, Kluwer Academic Publishers, the Netherlands, 19--36, 2001.
__________________________________________________________

Pearl has developed a coherent theory of causality that integrates Bayesian ideas with what he calls do-Calculus. All his papers can be accessed at http://bayes.cs.ucla.edu/csl_papers.html

2.
Philosophy and the practice of Bayesian statistics,
Andrew Gelman, Cosma Rohilla Shalizi

No comments:

Post a Comment