The Probability Density Function
Let’s try to figure out what the probability of X = 5 is, in our uniform example.
We know how to calculate the probability of intervals, so let’s try to get it as a
limit of intervals around 5.
Pr(4 ≤ X ≤ 6) = F(6) − F(4) = 0.2
Pr(4.5 ≤ X ≤ 5.5) = F(5.5) − F(4.5) = 0.1
Pr(4.95 ≤ X ≤ 5.05) = F(5.05) − F(4.95) = 0.01
Pr(4.995 ≤ X ≤ 5.005) = F(5.005) − F(4.995) = 0.001
Well, you can see where this is going. As we take smaller and smaller intervals
around the point X = 5, we get a smaller and smaller probability, and clearly
in the limit that probability will be exactly 0. (This matches what we’d get
from just plugging away with the CDF: F(5) − F(5) = 0.) What does this
mean? Remember that probabilities are long-run frequencies: Pr(X = 5) is
the fraction of the time we expect to get the value of exactly 5, in infinitely
many repetitions of our paper-airplane-throwing experiment. But with a really
continuous random variable, we never expect to repeat any particular value —
we could come close, but there are uncountably many alternatives, all just as
likely as X = 5, so hits on that point are vanishingly rare.