Since the 18th century, there has been a debate between frequentists and Bayesians. The former insisted that statistical procedures only made sense when one uses the relative frequency concept. The Bayesians supported the use of degrees of belief as a basis for statistical practice.

The frequentist position is the one you probably heard at school: perform an experiment lots of times, and measure the proportion where you get a positive result - this proportion, if you perform the experiment enough times, is the probability.

The problem comes in those cases where we haven't performed an experiment yet, or where there's no possible way an experiment could be performed - in these cases, frequentism can't help us.

-- http://en.wikipedia.org/wiki/Frequentist



Bayesianism is the philosophical tenet that the mathematical theory of probability applies to the degree of plausibility of a statement. This also applies to the degree of believability contained within the rational agents of a truth statement. Additionally, when a statement is used with the Bayes theorem, it then becomes a Bayesian inference.

This is in contrast to frequentism, which rejects degree-of-belief interpretations of mathematical probability, and assigns probabilities only to random events according to their relative frequencies of occurrence. The Bayesian interpretation of probability allows probabilities to be assigned to random events, but also allows the assignment of probabilities to any other kind of statement.

-- http://en.wikipedia.org/wiki/Bayesian_probability