References

Long, static list of references are a necessity for research articles and advanced texts. However, for elementary textbooks such lists are of no interest to the general reader, and of no help in locating further interesting reading material. So the philosophy of this list is to provide links to stimulating further reading. Nothing more, and nothing less. Neither complete, nor in any way a list of necessary reading, this list is open to requests and suggestions.

The Big and the Beautiful

• James O. Berger: Statistical Decision Theory and Bayesian Analysis. This is a graduate level text building up Bayesian analysis mathematically from the bottom, and is the book to choose if you are interested in pursuing the theory of Bayesian statistics. Temperament: mathematician.
• Bernardo & Smith: Bayesian Theory. This is a graduate level text for those interested in a rather complete book on Bayesian theory. Temperament: statistician.
• E. T. Jaynes: Probability Theory, The Logic of Science. This is a graduate level text by a physicist, building up Bayesian analysis from the depths of philosophy. In terms of technique, it does not take you very far, but it is the book for a scientifically minded persons interested in pursuing the philosophical underpinnings of statistics, and not the least Bayesians statistics. Temperament: philosopher. Warning: Reading this book might make you want to debate statistics online!

For Leisure

• Ian Hacking: The Emergence of Probability. This classic is a very charming read by one of my favourite authors on probability, and will give you a good understanding of how these thing called probability and statistics came into being. Temperament: fanboy. Damn, I'm gonna read everything by that man!
• Peter L. Bernstein: Against the Gods, The Remarkable Story of Risk. A highly entertaining tale of probability and statistics from the standpoint of an investment specialist, seeing it through the lens of that one keyword, risk. Temperament: popcorn. This book is excellent entertainment. Hard to put down.
• David Salsburg: The Lady Tasting Tea, How Statistics Revolutionized Science in the Twentieth Century. Focuses on the modern history of statistics. Temperament: Anxious to start new semester, but not quite ready to start working for real.

The Competition

Books you may want to buy instead of mine. Maybe.

• William M. Bolstad: Introduction to Bayesian Statistics. This is a classical elementary statistics textbook, teaching statistics from a Bayesian angle. Its primary difference from mine in that it does inference directly on conjugate priors rather than via what I consider to be the simpler method of hyperparameters. A well-written book and good read, none the less.
• Brani Vidakovic: Engineering Biostatistics: An Introduction using MATLAB and WinBUGS. A beautifully written book with an emphasis on biostatistics. See also his website.
• Peter D. Hoff: A First Course in Bayesian Statistical Methods. For the advanced or mathematically oriented bachelor or master student.
• Peter M. Lee: Bayesian Statistics: An Introduction. For the advanced or mathematically oriented bachelor or master student.
• John Kruschke: Doing Bayesian Data Analysis. A bachelor-level text with an emphasis on learning through the programming environments R and WinBugs.

Minor References

• Yang & Berger: A Catalog of Noninformative priors. Which (noninfirmative) priors to choose for which processes and applications.
• Kevin Murphy: Conjugate Bayesian analysis of the Gaussian distribution. The theoretical mathematical underpinnings of inference for Gaussian processes.
• Bayarri & Berger: The Interplay of Bayesian and Frequentist Analysis. We can be friends. Even when we know we are right (regardless of who "we" are).
• Walter & Augustin: Bayesian Linear Regression — Different Conjugate Models and Their (In)Sensitivity to Prior-Data Conflict.
• Christian Walck: Hand-book on STATISTICAL DISTRIBUTIONS for experimentalists.
• Sadia & Hossain: Contrast of Bayesian and Classical Sample Size Determination.
• Evan Miller: Formulas for Bayesian A/B Testing. Comparing two Beta distributions. The exact formula for doing this is rather new (2014), so the techniques warrants mention here.