Chain rule (probability)

In probability theory, the chain rule (also called the general product rule^[1][2]) permits the calculation of any member of the joint distribution of a set of random variables using only conditional probabilities. The rule is useful in the study of Bayesian networks, which describe a probability distribution in terms of conditional probabilities.

Consider an indexed set of sets $A_{1},\ldots ,A_{n}$. To find the value of this member of the joint distribution, we can apply the definition of conditional probability to obtain:

$\mathrm {P} (A_{n},\ldots ,A_{1})=\mathrm {P} (A_{n}|A_{n-1},\ldots ,A_{1})\cdot \mathrm {P} (A_{n-1},\ldots ,A_{1})$

Repeating this process with each final term creates the product:

$\mathrm {P} \left(\bigcap _{k=1}^{n}A_{k}\right)=\prod _{k=1}^{n}\mathrm {P} \left(A_{k}\,{\Bigg |}\,\bigcap _{j=1}^{k-1}A_{j}\right)$

With four variables, the chain rule produces this product of conditional probabilities:

$\mathrm {P} (A_{4},A_{3},A_{2},A_{1})=\mathrm {P} (A_{4}\mid A_{3},A_{2},A_{1})\cdot \mathrm {P} (A_{3}\mid A_{2},A_{1})\cdot \mathrm {P} (A_{2}\mid A_{1})\cdot \mathrm {P} (A_{1})$

This rule is illustrated in the following example. Urn 1 has 1 black ball and 2 white balls and Urn 2 has 1 black ball and 3 white balls. Suppose we pick an urn at random and then select a ball from that urn. Let event A be choosing the first urn: P(A) = P(~A) = 1/2. Let event B be the chance we choose a white ball. The chance of choosing a white ball, given that we've chosen the first urn, is P(B|A) = 2/3. Event A, B would be their intersection: choosing the first urn and a white ball from it. The probability can be found by the chain rule for probability:

$\mathrm {P} (A,B)=\mathrm {P} (B\mid A)\mathrm {P} (A)=2/3\times 1/2=1/3$.

Footnotes

1. Schum 1994.
2. Klugh 2013.

References

• Schum, David A. (1994). The Evidential Foundations of Probabilistic Reasoning. Northwestern University Press. p. 49. ISBN 978-0-8101-1821-8.
• Klugh, Henry E. (2013). Statistics: The Essentials for Research (3rd ed.). Psychology Press. p. 149. ISBN 1-134-92862-9.
• Russell, Stuart J.; Norvig, Peter (2003), Artificial Intelligence: A Modern Approach (2nd ed.), Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2, p. 496.
• "The Chain Rule of Probability", developerWorks, Nov 3, 2012.