Poisson processes §

A counting process is a non-decreasing, integer-valued process that jumps by exactly $1$ at each event; $N_t$ is the number of events that have occurred by time $t$. The two increment axioms are the heart of the definition:

1. Stationary increments say the law of the count in an interval depends only on its length $s$, not on its starting point $t$. A process with this property is “time-homogeneous”: the world looks the same no matter when you start watching.
2. Independent increments say that what happens in one interval tells you nothing about what happens in a disjoint interval. This is the “memoryless in time” property that makes the process tractable.

The rate $\lambda$ is defined by the mean count per unit time: $E[N_t] = \lambda t$. From the two axioms one can show that the count in any interval of length $s$ must be Poisson with mean $\lambda s$ — this is essentially the Poisson limit theorem applied to a fine partition of the interval (split it into $n$ tiny sub-intervals, each contributing a rare Bernoulli event, and let $n \to \infty$). So the increment law is not an extra assumption; it is forced by stationarity, independence, and the mean.

Example. Take $\lambda = 2$ events per hour. The number of events in the first hour is $N_1 \sim \mathrm{Poisson}(2)$, so

Because of stationarity, the same number $0.180$ is the probability of exactly $3$ events in any one-hour window, and because of independence, the count in the first hour is independent of the count in the second.

The staircase is the whole process: flat between events, a unit jump at each arrival. The red dots are the arrival times, which the next section quantifies.

See S. Ross, Introduction to Probability Models, Ch. 4 for the formal definition and the characterization via increments.

The inter-arrival times are where the exponential distribution enters. The key fact is the memoryless property of the exponential: for $s, x \geq 0$,

This is exactly what independent increments demand. Suppose you have already waited $s$ time units with no event. Independent increments say the future is independent of the past, so the additional waiting time must have the same distribution as a fresh wait — it cannot depend on how long you have already waited. The exponential is the only continuous distribution with this property, which is why the inter-arrivals are exponential and not, say, normal or gamma.

Concretely, the survival function of $N_t$ gives the inter-arrival law directly. No event in $(0, x]$ means $N_x = 0$, so

which is precisely the survival function of $\mathrm{Exp}(\lambda)$. Stationarity and independence then make every $X_i$ an independent copy of this.

Example. With $\lambda = 2$ per hour, the mean gap between events is $1/\lambda = 0.5$ hours. The probability that you wait more than $1$ hour for the next event is $P(X_1 > 1) = e^{-2 \cdot 1} \approx 0.135$.

The density starts at $\lambda$ and decays exponentially; the area to the right of any point is the probability of waiting longer than that point.

See R. D. Durrett, Probability: Theory and Examples, Ch. 2 for the memoryless characterization of the exponential.

The arrival times are just the partial sums of the inter-arrival times. Since each $X_i \sim \mathrm{Exp}(\lambda)$ and the $X_i$ are independent, $S_n$ is a sum of $n$ i.i.d. exponentials. Such a sum is called an Erlang random variable (a special case of the Gamma distribution with integer shape). The convolution of $n$ identical exponential densities produces the Erlang density, which is zero until $t = 0$, rises to a peak near $(n-1)/\lambda$, and then decays.

The mean and variance add because the $X_i$ are independent: $E[S_n] = n \cdot (1/\lambda) = n/\lambda$ and $\mathrm{Var}(S_n) = n \cdot (1/\lambda^2) = n/\lambda^2$. Intuitively, the $n$-th event arrives after $n$ independent gaps of average length $1/\lambda$, so on average it lands at $n/\lambda$; the uncertainty grows only like $\sqrt{n}$ (the standard deviation is $\sqrt{n}/\lambda$), so for large $n$ the arrival time is tightly concentrated around its mean.

Example. With $\lambda = 2$ per hour, the $3$-rd event has $S_3 \sim \mathrm{Erlang}(3, 2)$ with mean $3/2 = 1.5$ hours and variance $3/4 = 0.75$ hours$^2$. The density is $f_{S_3}(t) = 2 e^{-2t}(2t)^2/2! = 4t^2 e^{-2t}$.

Unlike the single exponential (which peaks at $t=0$), the Erlang density is zero at $t=0$ and peaks at a positive time: you must wait for all $n$ gaps to elapse.

See I. Karatzas & S. Shreve, Brownian Motion and Stochastic Processes, Ch. 1 for the Gamma/Erlang family and its role as the arrival-time law.

Thinning (or splitting) is the operation of randomly filtering a Poisson stream. Each event is independently tagged “keep” with probability $p$ or “drop” with probability $1-p$. The remarkable conclusion is that the two resulting streams are not merely Poisson — they are independent Poisson processes.

Why independence holds: the original process has independent increments, and the thinning decisions are independent of everything. In any small interval the kept and dropped counts are determined by disjoint, independent coin flips, so the two sub-processes never share randomness. Formally, the joint probability generating function of the kept and dropped counts in an interval of length $t$ factors as

which is the product of two independent Poisson PGFs with means $\lambda p t$ and $\lambda(1-p)t$.

Example. Cars pass a point at rate $\lambda = 10$ per minute, and $30\%$ are trucks. Thinning with $p = 0.3$ gives a truck process of rate $10 \cdot 0.3 = 3$ per minute and a non-truck process of rate $10 \cdot 0.7 = 7$ per minute, independent of each other.

The red sub-stream is itself a Poisson process at rate $\lambda p$, and it is independent of the gray sub-stream.

See S. Ross, Introduction to Probability Models, Ch. 4 for the thinning (decomposition) theorem.

Superposition (or aggregation) is the reverse of thinning: merge several independent Poisson streams into one. The merged count in an interval is the sum of the two independent counts in that interval, and the sum of independent Poisson variables is Poisson with the summed means. So in an interval of length $t$, $N_t \sim \mathrm{Poisson}(\lambda_1 t + \lambda_2 t) = \mathrm{Poisson}((\lambda_1+\lambda_2)t)$, and the independent-increments property is inherited from the two inputs.

The PGF makes the addition of rates transparent: the PGF of a sum of independent variables is the product of their PGFs, and

the PGF of $\mathrm{Poisson}((\lambda_1+\lambda_2)t)$. The result extends to any finite number of independent Poisson processes: the merged rate is the sum of the individual rates.

Example. Two independent phone lines each get calls at rate $4$ per hour. The combined line sees calls at rate $4 + 4 = 8$ per hour, a Poisson process.

The bottom row is the union of the two top rows, ordered in time; it is a single Poisson process at the summed rate.

See R. D. Durrett, Probability: Theory and Examples, Ch. 2 for the infinite divisibility of the Poisson law that underlies superposition.

A plain Poisson process counts events; a compound Poisson process adds a random size to each event. At each arrival the process jumps by an amount $Y_i$ drawn from the jump-size distribution, so $X_t$ is the total accumulated by time $t$. This is the workhorse model for aggregate losses (insurance), queue arrivals with batch sizes, and price jumps.

The moments come from conditioning on $N_t$. Given $N_t = n$, the sum $\sum_{i=1}^{n} Y_i$ has mean $n\,E[Y_1]$ and variance $n\,\mathrm{Var}(Y_1)$. Taking expectations over $N_t \sim \mathrm{Poisson}(\lambda t)$ (using $E[N_t] = \lambda t$ and $E[N_t^2] = \lambda t + (\lambda t)^2$):

Notice the variance uses the second moment $E[Y_1^2]$, not the variance of the jump: the randomness of how many jumps occur adds an extra $E[Y_1]^2$ term, so $\mathrm{Var}(X_t) = \lambda t\,\mathrm{Var}(Y_1) + (\lambda t)^2 E[Y_1]^2 = \lambda t\, E[Y_1^2]$.

Example. Claims arrive at rate $\lambda = 4$ per month, and each claim size $Y$ is uniformly distributed on $\{1, 2, 3\}$ (in thousands). Then $E[Y] = 2$ and $E[Y^2] = (1 + 4 + 9)/3 = 14/3$. The monthly aggregate $X_1$ has mean $4 \cdot 2 = 8$ thousand and variance $4 \cdot 14/3 \approx 18.67$ (thousand)$^2$.

Compare with the plain Poisson staircase: here the vertical jumps have different heights, one per arrival.

See I. Karatzas & S. Shreve, Brownian Motion and Stochastic Processes, Ch. 1 for compound Poisson processes as a component of the Lévy–Itô decomposition.

A hospital’s emergency room receives patients from two independent sources: walk-ins at rate $\lambda_1 = 6$ per hour and ambulance arrivals at rate $\lambda_2 = 2$ per hour.

1. What is the rate of the combined arrival process, and what is the probability that exactly $5$ patients arrive in the next $30$ minutes?
2. Of the combined stream, what is the probability that a given patient is a walk-in?
3. What is the expected waiting time (in minutes) until the next patient arrives?