Statistics 4540/6571
Mean, Variance and Covariance of Random Variables

This handout summarizes some results on finding the mean, variance and covariance of random variables. We will also state (without proof) some results on working with expected values of sums and products of random variables. I realize that this material will be very familiar to many of you. Our definitions for the mean and variance will assume that our random variable $X$ is continuous:

If $X$ is a continuous random variable, then its probability distribution or probability density function (pdf) is a function $f(x)$ such that for any two numbers $v \leq w$,

$$P(v \leq X \leq w) = \int_{v}^{w} f(x) dx$$

Definitions

• The expected value, or mean, of $X$ is
  
  $$\mbox{E}(X) = \mu_{X} = \int_{X} x f(x) dx$$
  
  
  The integral is over the range of permitted values for $X$.

• The variance of $X$ is:
  $$\begin{eqnarray*} \mbox{Var}(X) & = & \sigma^{2}_{X} \\ & = & \mbox{E}[(X - \... ...(x) dx \\ & = & \mbox{E}(X^{2}) - [\mbox{E}(X)]^{2} \mbox{ . } \end{eqnarray*}$$
  
  
  
  
  

• The covariance of two random variables $X$ and $Y$ is:
  $$\begin{eqnarray*} \mbox{Cov}(X, Y) & = & \mbox{E}[(X - \mu_{X})(Y - \mu_{Y})] \\ & = & \mbox{E}(XY) - \mbox{E}(X)\mbox{E}(Y) \mbox{ . } \end{eqnarray*}$$
  
  
  
  
  

• The correlation of two random variables $X$ and $Y$ is:
  
  $$\mbox{Corr}(X, Y) = \frac{\mbox{Cov}(X, Y)} {\sqrt{\mbox{Var}(X) \mbox{Var}(Y)}}$$
  
  

Assorted Rules

We now introduce some rules that can be applied to means and variances of functions of random variables. In the rules, the $a_{i}$ and $b_{i}$ values are fixed constants, and the $X_{i}$ values represent random variables. All the rules assume:

$$\mbox{E}(X_{i}) = \mu_{i}, \quad \mbox{Var}(X_{i}) = \sigma^{2}_{i}$$

1. $\mbox{E}\left[ \displaystyle \sum_{i = 1}^{n} (a_{i}X_{i} + b_{i}) \right] = \displaystyle \sum_{i = 1}^{n} (a_{i}\mu_{i} + b_{i})$.

2. $\mbox{Cov}[(a_{i} X_{i} + b_{i}), (a_{j} X_{j} + b_{j})] = a_{i}a_{j} \mbox{Cov}(X_{i}, X_{j})$.

3. $\mbox{Var}\left[ \displaystyle \sum_{i = 1}^{n} (a_{i}X_{i} + b_{i}) \right] =... ... a_{i}^{2}\sigma^{2}_{i} + \sum_{i \neq j} a_{i}a_{j}\mbox{Cov}(X_{i}, X_{j})$.

4. If $X_{i}$ and $X_{j}$ are independent, then $\mbox{E}(X_{i}X_{j}) = \mbox{E}(X_{i})\mbox{E}(X_{j})$.

5. If $X_{i}$ and $X_{j}$ are independent, then $\mbox{Cov}(X_{i}, X_{j}) = 0$.

   Question: Based on this result, how can you simplify Rule 3, if all $X_{i}$ are independent?