Conditional Probability, Total Probability and Bayes’ Theorem

Conditional probability updates the reference event, total probability combines disjoint cases and Bayes’ theorem reverses a conditional relationship. In brief: the probability that an item came from a machine given that it failed inspection is generally different from the probability of failure given that machine. Bayes’ theorem connects those two directions using the machine’s production share.

These ideas work together in multi-stage models and classification problems. The key is to keep the conditioning event, joint event and overall event distinct. This guide develops the formulas through a hypothetical production example, where every denominator can be checked against counts in a complete sample.

What is conditional probability?

Conditional probability P(A given B), written P(A|B), measures the chance of A within the condition that B has occurred. Its formula is P(A|B) = P(A ∩ B)/P(B), provided P(B) is positive. The denominator rescales the overlap relative to the restricted reference event B.

For example, if 30 of 50 group-B observations also satisfy A, the empirical conditional proportion is 30/50 = 0.6. Observations outside B do not belong in that denominator. Conditioning changes the relevant sample space even though the original complete dataset remains available.

The order matters. P(A|B) and P(B|A) use the same intersection but different denominators, so they usually differ. Reading the vertical bar as given helps preserve the direction: the event after the bar is the information being assumed, not the event being predicted.

How does conditional probability give the multiplication rule?

Rearranging the definition gives P(A ∩ B) = P(B)P(A|B). The same intersection can also be written P(A)P(B|A), when the relevant conditioning probabilities are defined. These two factorizations describe the same joint event in opposite orders.

If A and B are independent, P(A|B) = P(A) for positive P(B), and the product simplifies to P(A)P(B). Without independence, use the conditional factor. Multiplying two unconditional probabilities automatically would impose an assumption that may not match the experiment.

In a probability tree, the first branch supplies the probability of a case and the next branch supplies the probability of an outcome within that case. Their product gives the corresponding path probability. This is why trees remain valid for dependent stages when their branch labels are conditional probabilities.

What is a partition and why is it needed for total probability?

A partition divides the sample space into mutually exclusive cases that cover every possibility. Let B₁, B₂ and so on be those cases. Each outcome belongs to exactly one case, so the event A can be separated into disjoint pieces A ∩ Bᵢ.

Adding those pieces gives the law of total probability: P(A) = ΣP(Bᵢ)P(A|Bᵢ), for cases with positive probability. The weights P(Bᵢ) reflect how common each case is, while P(A|Bᵢ) gives the event chance within it.

The cases must not overlap or omit part of the sample space. If they overlap, the weighted sum can double-count outcomes; if they leave gaps, it can miss probability mass. A complete two-way table or first-stage tree is a useful way to verify the partition before applying the formula.

How do you use total probability in a worked example?

Suppose machine M₁ produces 60% of a factory’s items and machine M₂ produces 40%. In a hypothetical model, 2% of M₁’s items and 5% of M₂’s items fail inspection. Every item comes from exactly one machine, so the machine events form a partition.

The joint probability of M₁ and failure is 0.60 × 0.02 = 0.012. The joint probability of M₂ and failure is 0.40 × 0.05 = 0.020. Adding gives overall failure probability 0.032, or 3.2%.

An unweighted average of 2% and 5% would be 3.5%, which is wrong because the machines produce different shares. Total probability is a weighted average of the conditional rates. The larger production share of the lower-failure machine pulls the overall rate closer to 2% than to 5%.

What does Bayes’ theorem calculate?

Bayes’ theorem expresses P(B|A) as P(B)P(A|B)/P(A), provided P(A) is positive. It reverses a conditional direction by combining the prior probability of B with the likelihood of A under B, then normalising by the overall probability of A.

In the production example, the probability an item came from M₂ given that it failed is P(M₂|failure) = 0.40 × 0.05/0.032 = 0.625. Thus 62.5% of failed items come from M₂ under the model, even though M₂ makes only 40% of all items.

The count table confirms the result: 200 of the 320 failed items came from M₂, so the conditional share is 200/320 = 62.5%. The corresponding probability for M₁ is 120/320 = 37.5%, and the two posterior probabilities sum to 100% because the machines exhaust the possibilities.

What are prior, likelihood, evidence and posterior?

The prior is the probability assigned to a case before incorporating the specified new evidence. Here P(M₂) = 0.40 is the prior machine share. The likelihood P(failure|M₂) = 0.05 describes how probable the observed evidence is if that case holds.

The evidence probability P(failure) = 0.032 is the normalising denominator. The posterior P(M₂|failure) = 0.625 is the updated probability of the case after conditioning on the evidence. These labels refer to roles in the calculation, not to different mathematical kinds of probability.

A high likelihood alone does not determine a high posterior. The prior share matters too, because a rare case may contribute fewer observed events than a common case with a lower conditional rate. Keeping the base rates in the calculation prevents the common mistake of treating P(A|B) as though it directly answered P(B|A).

How do you organise a Bayes calculation step by step?

Start by writing the target conditional probability in words and symbols. Identify the evidence after the conditioning bar and the case whose probability is sought. Then list mutually exclusive cases covering all possibilities, together with their prior probabilities and the likelihood of the evidence within each case.

Multiply each prior by its likelihood to obtain joint probabilities. Add those joints to find the total evidence probability. Finally, divide the joint for the requested case by that total. A count representation using a convenient hypothetical total often makes the same calculation easier to inspect.

For the machine example, the requested quantity is M₂ among failures, so the denominator must include all failures, not all M₂ items or all produced items. Writing that denominator in words is a simple way to keep the reversed conditional direction correct throughout the calculation.

1. Define the cases and the observed evidence.
2. Record prior probabilities and conditional evidence probabilities.
3. Multiply to obtain joint probabilities.
4. Add joints for the total evidence probability.
5. Divide the requested joint by the evidence total.

What mistakes should you avoid with conditional probability and Bayes’ theorem?

The background principle is consistent reweighting of a sample space after information is supplied. Bayes’ theorem follows directly from expressing the same intersection in two ways, so it is not a separate exception to ordinary probability rules. Its usefulness comes from converting a known conditional direction into the one actually needed.

Do not reverse conditionals without adjusting for their denominators, omit prior probabilities or average conditional rates without appropriate weights. Also check that the conditioning event has positive probability. A zero-probability condition is not handled by the elementary ratio formula without a more advanced framework.

Finally, remember that conditioning does not establish causation. Learning that an item failed changes the probability assigned to its source, but does not change which machine produced it. State the model assumptions and interpret the posterior as a conditional probability within that model, not as certainty about a particular individual outcome.