A.1 The GAUSSIAN Normal Distribution

A normal distribution in a variate^A.1 $X$ with mean $\mu$ and variance $\sigma^2$ is a statistic distribution with the probability function

$$P(x) = \frac{1}{\sigma\sqrt{2 \pi}} e^{-\frac{(x-\mu)^2}{2 \sigma^2}}$$ (A.1)

on the domain $x \in (-\infty,\infty)$.
DEMOIVRE developed the normal distribution as an approximation to the binomial distribution, and it was subsequently used by LAPLACE in 1783 to study measurement errors and by GAUSS in 1809 in the analysis of astronomical data.
The so-called ``standard normal distribution'' is given by taking $\mu = 0$ and $\sigma^2 = 1$ in a general normal distribution. An arbitrary normal distribution can be converted to a standard normal distribution by changing variables to $Z \equiv \frac{X-\mu}{\sigma}$, so $dz = \frac{dx}{\sigma}$, yielding

$$P(x) dx = \frac{1}{\sqrt{2 \pi}} e^{-\frac{z^2}{2}} dz$$ (A.2)

The normal distribution function $\phi(z)$ gives the probability that a standard normal variate assumes a value in the interval $[0,z]$,

$$\phi(z) \equiv \frac{1}{\sqrt{2 \pi}} \int_0^z e^{-\frac{x^2}{2}} dx = \frac{1}{2} \erf {\left(\frac{z}{\sqrt{2}}\right)}$$ (A.3)

where $\erf (z)$ is the error function. Neither $\phi(z)$ nor $\erf (z)$ can be expressed in terms of finite additions, subtractions, multiplications, and root extractions, and so both must be either computed numerically or otherwise approximated.
The normal distribution is the limiting case of a discrete binomial distribution $P_P(n\vert N)$ as the sample size N becomes large, in which case $P_P(n\vert N)$ is normal with mean and variance

$$\mu = N p$$ (A.4)

$$\sigma^2 = N p q$$ (A.5)

with $q \equiv 1-p$.
The distribution $P(x)$ is properly normalized since

$$\int_{-\infty}^\infty P(x) dx = 1$$ (A.6)

The cumulative distribution function which gives the probability that a variate will assume a value $\le x$, is then the integral of the normal distribution

$$D(x) \equiv \int_{-\infty}^x P(x') dx'$$

$$= \frac{1}{\sigma \sqrt{2 \pi}} \int_{-\infty}^x e^{-\frac{(x'-\mu)^2}{2 \sigma^2}} dx'$$

$$= \frac{1}{2}\left[1+\erf {\left(\frac{x-\mu}{\sigma \sqrt{2}}\right)}\right]$$ (A.7)

The normal distribution function is obviously symmetric about $x=\mu$

$$P(x-\mu)=P(-x-\mu)$$ (A.8)

and its maximum value is situated at $x=\mu$

$$P_{max}=P(x=\mu)=\frac{1}{\sigma \sqrt{2 \pi}}$$ (A.9)

Normal distributions have many convenient properties, so random variates with unknown distributions are often assumed to be normal, especially in physics and astronomy. Although this can be a dangerous assumption, it is often a good approximation due to a surprising result known as the central limit theorem (see next section).
Among the amazing properties of the normal distribution are that the normal sum distribution and normal difference distribution obtained by respectively adding and subtracting variates $X$ and $Y$ from two independent normal distributions with arbitrary means and variances are also normal. The normal ratio distribution obtained from $\frac{X}{Y}$ has a Cauchy distribution.
The unbiased estimator^A.2 for the variance of a normal distribution is given by

$$\sigma^2 = \frac{N}{N-1} s^2$$ (A.10)

where

$$s^2 \equiv \frac{1}{N} \sum^N_{i=1} (x_i - \bar{X})^2.$$ (A.11)

The characteristic function for the normal distribution is

$$\phi(t) = e^{\imath m t - \sigma^2 \frac{t^2}{2}},$$ (A.12)

and the moment-generating function is

$$M(t) = \langle e^{t x} \rangle$$

$$= \int^{\infty}_{-\infty}\frac{e^{t x}}{\sigma \sqrt{2 \pi}} e^{-\frac{(x-\mu)^2}{2 \sigma^2}} dx$$

$$= e^{\mu t + \sigma^2 \frac{t^2}{2}},$$ (A.13)

so

$$M'(t) = (\mu + \sigma^2 t) e^{\mu t + \sigma^2 \frac{t^2}{2}}$$

$$M''(t) = \left[\sigma^2 + (\mu + t \sigma^2)^2\right] e^{\mu t + \sigma^2 \frac{t^2}{2}}$$ (A.14)

Footnotes

... variate^A.1

A variate is a generalization of the concept of a random variable that is defined without reference to a particular type of probabilistic experiment. It is defined as the set of all random variables that obey a given probabilistic law. It is common practice to denote a variate with a capital letter (most commonly X).

... estimator^A.2

An estimator $\hat{\theta}$ is an unbiased estimator of $\theta$ if $\langle \hat{\theta} \rangle = \theta$.