Statistics 3520
Background: Means and Variances of Random Variables

One of the things that we'll talk about in this course from time to time is how our response variable behaves, particularly in terms of its mean and variance. To do this, we need to be familiar with the technical definitions of the mean and variance of random variables. Some of this material may be more familiar to those of you who have taken Stats 2510.

We will define these terms for discrete and continuous random variables (r.v.). A random variable is discrete if it can take on only a finite or countable number of outcomes. A random variable is continuous if its set of possible values is an entire interval of numbers. In other words, for some $a < b$, any number $x$ between $a$ and $b$ is possible. One or both of $a$ and $b$ can be infinity.

If $X$ is a discrete random variable, then its probability distribution or probability mass function (pmf) is defined for every number $k$ by

$$p(k) = P(X = k)$$

If $X$ is a continuous random variable, then its probability distribution or probability density function (pdf) is a function $f(x)$ such that for any two numbers $v \leq w$,

$$P(v \leq X \leq w) = \int_{v}^{w} f(x) dx$$

An important property of either distribution function is, if we either sum of integrate over all possible values of the possible values of the r.v. X:

$$\sum_{all X} p(x) = 1, \qquad \int_{X} f(x) dx = 1$$

Definitions

• The expected value, or mean, of $X$ is
  $$\begin{eqnarray*} \mbox{E}(X) & = & \mu_{X} = \sum_{X} x p(x) \quad \mbox{if $... ... = \int_{X} x f(x) dx \quad \mbox{if $X$\ is a continuous r.v.} \end{eqnarray*}$$
  
  
  
  
  
  The sum or integral is over the range of permitted values for $X$.

• The variance of $X$ is (if $X$ is continuous):
  $$\begin{eqnarray*} \mbox{Var}(X) & = & \sigma^{2}_{X} \\ & = & \mbox{E}[(X - \... ...(x) dx \\ & = & \mbox{E}(X^{2}) - [\mbox{E}(X)]^{2} \mbox{ . } \end{eqnarray*}$$
  
  
  
  
  
  If $X$ is a discrete r.v. , we replace the integrals by summation.

Assorted Rules

We now introduce some rules that can be applied to means and variances of functions of r.v.'s (discrete or continuous). In the rules, $a$ and $b$ are fixed, real numbers and $X$ and $Y$ are random variables.

1. $\mbox{E}(aX + b) = a\mbox{E}(X) + b$.

2. $\mbox{Var}(aX + b) = a^{2}\mbox{Var}(X)$.

3. $\mbox{E}(aX + bY) = a\mbox{E}(X) + b\mbox{E}(Y).$

4. If $X$ and $Y$ are independent, $\mbox{Var}(aX+ bY) = a^{2}\mbox{Var}(X) + b^{2}\mbox{Var}(Y)$.

5. If $X$ and $Y$ are independent, then $\mbox{E}(XY) = \mbox{E}(X)\mbox{E}(Y)$.