Definition

A Poisson random variable has a PMF given by \[ p_{X}(k)=e^{-\lambda} \frac{\lambda^{k}}{k !}, \quad k=0,1,2, \ldots, \tag{1} \] where \(\lambda\) is a positive parameter charactering the PMF. This is a legitimate PMF because \[ \sum_{k=0}^{\infty} e^{-\lambda} \frac{\lambda^{k}}{k !}=e^{-\lambda}\left(1+\lambda+\frac{\lambda^{2}}{2 !}+\frac{\lambda^{3}}{3 !}+\cdots\right)=e^{-\lambda} e^{\lambda}=1\ . \]

Number Feature

Expected Value

\[ \begin{aligned} \mathrm{E}(X) &=\sum_{i=0}^{\infty} i P(X=i) \\ &=\sum_{i=1}^{\infty} i \frac{e^{-\lambda} \lambda^{i}}{i !} \\ &=\lambda e^{-\lambda} \sum_{i=1}^{\infty} \frac{\lambda^{i-1}}{(i-1) !} \\ &=\lambda e^{-\lambda} \sum_{i=0}^{\infty} \frac{\lambda^{i}}{i !} \\ &=\lambda e^{-\lambda} e^{\lambda} \\ &=\lambda \,. \end{aligned} \tag{2} \]

Variance

\[ \begin{aligned} \mathrm{E}\left(X^{2}\right) &=\sum_{i=0}^{\infty} i^{2} P(X=i) \\ &=\sum_{i=1}^{\infty} i^{2} \frac{e^{-\lambda} \lambda^{i}}{i !} \\ &=\lambda e^{-\lambda} \sum_{i=1}^{\infty} \frac{i \lambda^{i-1}}{(i-1) !} \\ &=\lambda e^{-\lambda} \sum_{i=1}^{\infty} \frac{1}{(i-1) !} \frac{\mathrm{d}}{\mathrm{d} \lambda}\left(\lambda^{i}\right) \\ &=\lambda e^{-\lambda} \frac{\mathrm{d}}{\mathrm{d} \lambda}\left[\sum_{i=1}^{\infty} \frac{\lambda^{i}}{(i-1) !}\right] \\ &=\lambda e^{-\lambda} \frac{\mathrm{d}}{\mathrm{d} \lambda}\left[\lambda \sum_{i=1}^{\infty} \frac{\lambda^{i-1}}{(i-1) !}\right] \\ &=\lambda e^{-\lambda} \frac{\mathrm{d}}{\mathrm{d} \lambda}\left(\lambda e^{\lambda}\right)=\lambda e^{-\lambda}\left(e^{\lambda}+\lambda e^{\lambda}\right)=\lambda+\lambda^{2}\ , \end{aligned} \tag{3} \]

or \[ \begin{aligned} E\left(X^{2}\right) &=E[X(X-1)+X]=E[X(X-1)]+E(X) \\ &=\sum_{k=0}^{\infty} k(k-1) \frac{\lambda^{k}}{k !} \mathrm{e}^{-\lambda}+\lambda=\lambda^{2} \mathrm{e}^{-\lambda} \sum_{k=2}^{\infty} \frac{\lambda^{k-2}}{(k-2) !}+\lambda=\lambda^{2}+\lambda \ . \end{aligned} \tag{4} \] Then \[ D(X)=E\left(X^{2}\right)-E^{2}(X)=\lambda^{2}+\lambda-\lambda^{2}=\lambda \ . \tag{5} \]

The Moment Generating Function

\[ M_{X}(t)=E\left[e^{t X}\right]=\sum_{x=0}^{\infty} e^{t x} \frac{e^{-\lambda} \lambda^{x}}{x !}=e^{-\lambda} \sum_{x=0}^{\infty} \frac{\left(e^{t} \lambda\right)^{x}}{x !}=e^{\lambda\left(e^{t}-1\right)}. \tag{6} \]

The sum of dependent Poisson random variables still obeys Poisson distribution. It's easy to proof using the MGF.

Source

Poisson distribution has strong relationship with binomial distribution. And it needs to talk with Bernoulli distribution.

Bernoulli Distribution

It is a one-time randomized trial, whose PMF is \[ p_{X}(k)=\left\{\begin{array}{ll} p, & \text { if } k=1 \\ 1-p, & \text { if } k=0 \ . \end{array}\right. \tag{7} \] For all its simplicity, the Bernoulli random variable is very important. In practice, it is used to model generic probabilistic situations with just two outcomes. Furthermore, by combining multiple Bernoulli random variables, one can construct more complicated random variables, such as the binomial random variable.

Binomial Distribution

A coin is tossed \(n\) times. At each toss, the coin comes up a head with probability \(p,\) and a tail with probability \(1-p,\) independent of prior tosses. Let \(X\) be the number of heads in the \(n\)-toss sequence. We refer to \(X\) as a binomial random variable with parameters \(n\) and \(p .\) The PMF of \(X\) is \[ p_{X}(k)=P(X=k)=\left(\begin{array}{l} n \\ k \end{array}\right) p^{k}(1-p)^{n-k}, \quad k=0,1, \ldots, n\ , \tag{8} \] which can be regarded as the PMF of multiple Bernoulli trials.

Poisson Distribution

When the binomial distribution's \(n\) is very large but the \(p\) is very small, it becomes the Poisson distribution: \[ \begin{aligned} \lim _{n \rightarrow \infty} P(X=k) &=\lim _{n \rightarrow \infty}\left(\begin{array}{c} n \\ k \end{array}\right) p^{k}(1-p)^{n-k} \\ &=\lim _{n \rightarrow \infty} \frac{n !}{(n-k) ! k !}\left(\frac{\lambda}{n}\right)^{k}\left(1-\frac{\lambda}{n}\right)^{n-k} \\ &=\lim _{n \rightarrow \infty} \underbrace{\left[\frac{n !}{n^{k}(n-k) !}\right]}_{F}\left(\frac{\lambda^{k}}{k !}\right) \underbrace{\left(1-\frac{\lambda}{n}\right)^{n}}_{\rightarrow \exp (-\lambda)} \underbrace{\left(1-\frac{\lambda}{n}\right)^{-k}}_{\rightarrow 1} \\ &=\lim _{n \rightarrow \infty} \underbrace{\left[\left(1-\frac{1}{n}\right)\left(1-\frac{2}{n}\right) \ldots\left(1-\frac{k-1}{n}\right)\right]}_{\rightarrow 1}\left(\frac{\lambda^{k}}{k !}\right) \underbrace{\left(1-\frac{\lambda}{n}\right)^{n}}_{\rightarrow \exp (-\lambda)} \underbrace{\left(1-\frac{\lambda}{n}\right)^{-k}}_{\rightarrow 1}\\ &=\left(\frac{\lambda^{k}}{k !}\right) \exp (-\lambda)\ , \end{aligned} \] where \(\lambda =np\). Note that \(\displaystyle \lim _{n \rightarrow \infty}\left(1-\frac{\lambda}{n}\right)^{n}=e^{-\lambda}\), so the Poisson distribution is \[ P(X=k)=e^{-\lambda} \frac{\lambda^{k}}{k !}, \quad k=0,1,2, \ldots \] 　As for how to understand Poisson distribution, there is a good explanation. Poisson distribution is the limit of binomial distribution with a very small \(p\), and the normal distribution is the limit of binomial distribution with a vary lager \(n\), making the discrete distribution become the continuous one.

References