Target Audience: Anyone who can compute simple probabilities and wants the model underneath, stated precisely enough to argue with.
Prerequisites: A Complete Introduction to Combinatorics — every probability on a finite space is a ratio of counts, so the counting techniques are used throughout. Set notation from Sets, Relations and Functions is assumed.
On a finite sample space, every probability statement is a finite sum. Nothing has to be taken on trust, and nothing needs a limit. That is the whole reason to start here rather than with continuous distributions: the model is small enough to write out completely, which means every claim on this page can be checked by enumeration.
The explorer in Section 9 does exactly that. It enumerates the whole space, computes each probability as an exact fraction, and then verifies eight identities by evaluating both sides on the same enumeration — so the witness is independent of the formula being tested.
Two ideas are worth watching for, because both are routinely misremembered. Counting favourable cases is only valid when outcomes are equally likely (Section 3). And linearity of expectation holds for any random variables, dependent or not (Section 9) — while variance does not (Section 10).
The sample space \(\Omega\) is the set of all possible outcomes of an experiment. The outcomes must be mutually exclusive and the list exhaustive.
An event is any subset of \(\Omega\).
Most of the difficulty in a probability problem is choosing \(\Omega\) correctly, and most errors are made here rather than in the arithmetic that follows.
The sample space is the \(36\) ordered pairs \((1,1)\) through \((6,6)\), not the \(11\) possible sums. Taking the sums as outcomes is the classic mistake: it would make \(2\) and \(7\) equally likely, which they are not.
Because events are sets, the set operations mean something immediately: \(A \cap B\) is “both happened”, \(A \cup B\) is “at least one happened”, and the complement is “\(A\) did not happen”. Everything in Sections 4 and 5 is set algebra with weights attached.
An assignment \(P\) of a number to each event such that
On a finite space that is the entire apparatus. Assigning a non-negative weight to each outcome so the weights total \(1\), and defining \(P(E)\) as the total weight of the outcomes in \(E\), satisfies all three automatically.
Any question phrased as “at least one” is usually easier as \(1 - P(\text{none})\). Three tosses giving at least one head: rather than adding the seven favourable outcomes, note that exactly one outcome has no head, so the answer is \(1 - \tfrac{1}{8} = \tfrac{7}{8}\).
This is the formula most people learn as the definition of probability. It is not the definition; it is a consequence of the outcomes carrying equal weight. Applying it when they do not is the most common modelling error in the subject.
Toss a coin twice, with \(P(\text{heads}) = \tfrac{2}{3}\). The sample space is \(\{HH, HT, TH, TT\}\) — four outcomes, three of which contain a head.
Counting would give \(\tfrac{3}{4}\). The correct answer is \(1 - \left(\tfrac{1}{3}\right)^2 = \tfrac{8}{9}\).
The outcomes are not equally likely, so the count is meaningless. The explorer carries this case as a preset and computes both numbers.
On a uniform space the whole of the combinatorics unit becomes available: permutations and combinations count \(|E|\) and \(|\Omega|\), and the probability is their ratio. Most “probability problems” at this level are counting problems wearing a different hat.
Axiom 3 adds probabilities only for disjoint events. When events overlap, adding double-counts the overlap, and the correction is exactly its probability:
For three events the pattern continues:
and in general: add the singles, subtract the pairs, add the triples, alternating through all \(2^n - 1\) non-empty subsets. The Inclusion–Exclusion Principle for Probability proves the two-event case directly and the general case by induction.
An outcome lying in exactly \(m\) of the events is counted \(\binom{m}{1} - \binom{m}{2} + \binom{m}{3} - \cdots\) times, and by the alternating binomial sum that total is exactly \(1\). So every outcome of the union contributes once — which is precisely what a correct formula for \(P(A \cup B \cup \cdots)\) has to achieve. The signs are forced, not a pattern to memorise.
Let \(A\) be “the sum is at least \(9\)”, \(B\) “the first die is even”, \(C\) “a double”. Then \(P(A) = \tfrac{5}{18}\), \(P(B) = \tfrac{1}{2}\) and \(P(C) = \tfrac{1}{6}\).
The seven-term alternating sum gives \(P(A \cup B \cup C) = \tfrac{2}{3}\), which is what counting the union directly gives. The explorer prints every term and compares the two routes.
\(P(A) + P(B)\) for the two dice events above is \(\tfrac{5}{18} + \tfrac{1}{2} = \tfrac{7}{9}\), which exceeds the true \(P(A \cup B)\). If a probability calculation ever produces a number above \(1\), an overlap has been counted twice.
\[ P(A \mid B) = \frac{P(A \cap B)}{P(B)}, \qquad \text{defined only when } P(B) \gt 0. \]
Read it as shrinking the sample space. Conditioning on \(B\) discards every outcome outside \(B\) and rescales what remains so the total is \(1\) again — dividing by \(P(B)\) is precisely that rescaling. Seen that way the formula is obvious rather than arbitrary, and the requirement \(P(B) \gt 0\) is clearly unavoidable: there is nothing left to rescale.
With \(A\) = “sum at least \(9\)” and \(B\) = “first die even” on two dice:
\[ P(A \mid B) = \tfrac{1}{3}, \qquad P(B \mid A) = \tfrac{3}{5}. \]
Different questions, different answers. Swapping the two is the most consequential error in applied probability — it is the same confusion behind misreadings of medical test results, where the probability of a positive test given the disease is mistaken for the probability of the disease given a positive test.
Rearranged, \(P(A \cap B) = P(A \mid B)\,P(B)\). This is the form to use when a problem is described in stages: the probability of a whole sequence is the product of the conditional probabilities of each step given everything before it.
\(A\) and \(B\) are independent when \[ P(A \cap B) = P(A)\,P(B). \]
That product rule is the definition. It is not a statement about causation, and it is not something to decide by intuition — it is checked by computing both sides. Equivalently, when \(P(B) \gt 0\), independence says \(P(A \mid B) = P(A)\): knowing \(B\) changes nothing about \(A\).
On two dice, “first die even” and “second die even” are independent: \(\tfrac{1}{4} = \tfrac{1}{2} \cdot \tfrac{1}{2}\).
But “first die even” and “sum at least \(9\)” are not, even though the dice themselves are: \(P(A \cap B) = \tfrac{1}{6}\) while \(P(A)P(B) = \tfrac{5}{36}\). Independence is a property of events, not of the underlying mechanism.
If \(A\) and \(B\) are disjoint and both have positive probability, then \(P(A \cap B) = 0\) while \(P(A)P(B) \gt 0\), so they are emphatically dependent — knowing \(B\) happened tells you \(A\) certainly did not. The two terms sound similar and mean almost contrary things.
Drawing two cards with replacement gives independent draws; without replacement does not. The explorer's two-card preset has \(P(\text{second is a } 1) = \tfrac{1}{3}\) unconditionally but \(\tfrac{1}{5}\) given the first was a \(1\). A problem that does not state the sampling method is not yet well posed.
A random variable is a function \(X : \Omega \to \mathbb{R}\).
It is neither random nor a variable. It is a rule attaching a number to every outcome; the randomness lives entirely in which outcome occurs. That framing is worth taking seriously, because it is what makes Section 9 easy: two random variables on the same space are two functions, and functions can be added pointwise whatever their relationship.
\(X\) = the first die, \(Y\) = the second die, \(X + Y\) = the sum, and \(\mathbf{1}_{\{X = Y\}}\) = \(1\) on a double and \(0\) otherwise. All four are functions on the same \(36\)-outcome space.
The distribution of \(X\) lists each value it takes with the total probability of the outcomes giving that value. It always sums to \(1\), and it deliberately forgets everything else about \(\Omega\) — which is useful, but is also why two very different experiments can share a distribution.
Two formulas, always equal. The first sums over outcomes and is the definition; the second sums over distinct values. They agree by grouping together the outcomes that share a value — that regrouping is Step 1 of Random Variables and the Linearity of Expectation, and the explorer checks the two against each other on every preset.
The expected number of heads in three tosses is \(\tfrac{3}{2}\), and no outcome gives one and a half heads. “Expected” means long-run average, not likely, and not even necessarily attainable.
\(E[X] = \tfrac{1}{6}(1 + 2 + 3 + 4 + 5 + 6) = \tfrac{7}{2}\), which is not a face of the die. Its variance is \(\tfrac{35}{12}\).
This holds for any random variables on the same sample space, however strongly related. It is the single most useful fact in discrete probability and the most frequently misremembered.
The reason is the framing in Section 7. Since \((X + Y)(\omega) = X(\omega) + Y(\omega)\) by definition, the proof just splits one finite sum into two and reorders them. Independence is never mentioned because it is never used.
\(\mathbf{1}_A(\omega) = 1\) when \(\omega \in A\), and \(0\) otherwise. Its expectation is exactly \(P(A)\) — which turns a counting question into a sum of probabilities.
\(n\) people leave hats at a cloakroom and they are returned at random. How many people expect to get their own hat back?
Let \(X_i = 1\) if person \(i\) gets their own hat. Then \(E[X_i] = \tfrac{1}{n}\), and the total number of matches is \(X = X_1 + \cdots + X_n\). By linearity, \[ E[X] = n \cdot \tfrac{1}{n} = 1 \] for every \(n\).
The matches are strongly dependent — if the first \(n - 1\) people all match, the last one must too — and linearity does not care. The full distribution needs derangement numbers; the expectation needs one line. For \(n = 4\), the explorer confirms \(E[X] = 1\) and \(P(\text{nobody matches}) = \tfrac{9}{24}\), matching \(D(4) = 9\) from the combinatorics unit.
Choose an experiment. The explorer enumerates the entire sample space, computes every probability as an exact fraction, runs inclusion–exclusion over three events both ways, tests independence by the product rule, and reports expectations and variances — then checks eight identities against the same enumeration.
Every value is computed live by enumerating the sample space in exact rational arithmetic — nothing is tabulated, and no probability is rounded.
A measure of spread: how far \(X\) typically falls from its mean. The second form is usually the one to compute with.
\(\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)\) holds only when \(X\) and \(Y\) are independent. So does \(E[XY] = E[X]\,E[Y]\).
The explorer demonstrates both sides of this: for two dice the variances do add, and for the number of heads against the indicator of the first toss — clearly dependent — they do not. That contrast is the point of this section. Expectation is the rule that comes free; everything else has a hypothesis attached.
A single indicator with \(P = p\) has mean \(p\) and variance \(p(1 - p)\). A sum of \(n\) independent copies is binomial: mean \(np\) — by linearity alone, no independence needed — and variance \(np(1 - p)\), where the independence genuinely is required. Noticing which of those two steps needs the hypothesis is a good test of whether this page has landed.
Everything here was finite, so every statement was a finite sum. Continuous probability replaces those sums with integrals and needs the calculus of Tier 3 before it can be done honestly — which is why it waits until later in the plan.
| Result | Statement | Needs independence? |
|---|---|---|
| Complement | \(P(\text{not } E) = 1 - P(E)\) | No |
| Union | \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\) | No |
| Disjoint union | \(P(A \cup B) = P(A) + P(B)\) | Needs disjointness |
| Conditional | \(P(A \mid B) = P(A \cap B)/P(B)\) | No, but needs \(P(B) \gt 0\) |
| Multiplication | \(P(A \cap B) = P(A \mid B)P(B)\) | No |
| Independence | \(P(A \cap B) = P(A)P(B)\) | This is the definition |
| Expectation, two ways | \(\sum_\omega P(\omega)X(\omega) = \sum_v vP(X = v)\) | No |
| Linearity | \(E[aX + bY] = aE[X] + bE[Y]\) | No |
| Indicator | \(E[\mathbf{1}_A] = P(A)\) | No |
| Product of variables | \(E[XY] = E[X]E[Y]\) | Yes |
| Variance of a sum | \(\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)\) | Yes |