### [A, SfS] Chapter 2: Probability: 2.5: Linear Combinations of Random Variables

### Linear Combinations of Random Variables

Linear Combinations of Random Variables

In this lesson you will learn how to compute the mean and variance of a linear combination of random variables.

#\text{}#

Linear Combination

Suppose you have #n# random variables (discrete or continuous), denoted #X_1,X_2,\ldots,X_n#, and let #c_0,c_1,c_2,\ldots,c_n# denote #n + 1# constants.

Then #c_0+c_1X_1+c_2X_2+\cdots+c_nX_n# is called a **linear combination** of #X_1,X_2,...,X_n#.

#\text{}#

We would like to be able to compute the mean and variance of the linear combination if we already know the mean and variance of each #X_i#.

Let’s denote the mean of #X_i# by #\mu_i# and the variance of #X_i# by #\sigma^2_i#.

Mean for a Linear Combination

Using the linearity property of both sums and integrals, the mean (also called the expectation) of a linear combination is:

\[E\Big(c_0 + c_1X_1 + c_2X_2 + \cdots + c_nX_n\Big) = c_0 + c_1\mu_1 + c_2\mu_2 + \cdots + c_n\mu_n\]

#\text{}#

The *variance of a linear combination* is more complicated, because it involves squaring. However, we consider a special case for which the formula becomes simple.

Independent Random Variables

Two random variables #X# and #Y# are *independent* if and only if

\[ \mathbb{P}[(X \leq x) \cap (Y \leq y)] = \mathbb{P}(X \leq x) \mathbb{P}(Y \leq y) \]

for every pair of real numbers #x# and #y#. Essentially this means that having information about the value of one of the r.v.s has no impact on the probability distribution of the other r.v.

Similarly, the random variables #X_1, X_2, \ldots, X_n# are *mutually independent* if and only if

\[ \mathbb{P}[(X_1 \leq x_1) \cap (X_2 \leq x_2) \cap \cdots \cap (X_n \leq x_n)] = \mathbb{P}(X_1 \leq x_1) \mathbb{P}(X_2 \leq x_2) \cdots \mathbb{P}(X_n \leq x_n) \]

for every set of real numbers #x_1, x_2, \ldots, x_n#.

Variance of a Linear Combination

If (and only if) the random variables #X_1,X_2,...,X_n# are *independent*, the variance formula becomes:

\[V\Big(c_0 + c_1X_1 + c_2X_2 + \cdots + c_nX_n\Big) = c^2_1\sigma^2_1+c^2_2\sigma^2_2+\cdots +c^2_n\sigma^2_n\]

#\text{}#

To demonstrate the mean and variance of a linear combination of r.v.s, we begin with simple situations and progress to more complex situations.

Linear Combination of a Single Random Variable

Suppose #X# is a r.v. with mean #\mu_X# and variance #\sigma^2_X#.

Let #Y = a + bX# for some constants #a# and #b#. Then:

\[\begin{array}{rcl}

\mu_Y &=& a + b\mu_X\\\\

\sigma^2_Y &=& b^2\sigma^2_X\\\\

\sigma_Y &=& |b|\sigma_X

\end{array}\]

Suppose #X# is a r.v. with mean #10# and variance #4#.

Let #Y = 50 - 5X#. Then:

\[\begin{array}{rcl}

\mu_Y &=& 50 - 5 {\mu_X} = 50 - (5)(10) = 0\\\\

\sigma_Y^2 &=& (-5)^2\sigma^2_X = (25)(4) = 100\\\\

\sigma_Y &=& |-5| \sigma_X = (5)(2) = 10

\end{array}\]

#\text{}#

Linear Combination of Two Random Variables

Next, suppose #X# and #Y# are independent r.v.s with means #\mu_X# and #\mu_Y#, respectively, and with variances #\sigma_X^2# and #\sigma_Y^2#, respectively.

Let #Z = a + bX + Y#. Then:

\[\mu_Z = a + b\mu_X + c\mu_Y\]

and, because of the independence of #X# and #Y#:

\[\sigma_Z^2 = b^2\sigma_X^2 + c^2\sigma_Y^2\]

Suppose #X# and #Y# are independent r.v.s with #\mu_X = 12#, #\sigma^2_X = 9#, #\mu_Y = 8#, and #\sigma_Y^2 = 1#.

Let #Z = 4X - 2Y#. Then:

\[\begin{array}{rcl}

\mu_Z &=& 4\mu_X - 2\mu_y = (4)(12) - (2)(8) = 32\\\\

\sigma_Z^2 &=& (4)^2\sigma^2_X + (-2)^2\sigma_Y^2 = (16)(9) + (4)(1) = 148

\end{array}\]

**Warning:** Many students would miscalculate the variance in this example! In order to implement the variance formula correctly, the linear combination must be in the form #aX + bY#, not #aX - bY#.

That means we should rewrite the combination #aX - bY# as #aX + (-b)Y#.

So in this example we would write #Z = 4X - 2Y# as #Z = 4X + (-2)Y# before using the variance formula. Don’t forget!

#\text{}#

If we plan to select a simple random sample of size #n# from the population and measure some quantitative variable #X# on each element of the sample, we can denote the future measurements on the sample by #X_1,...,X_n#. Because it is going to be a random sample, we consider #X_1,...,X_n# to be independent random variables, since their eventual values are uncertain.

We have already explained that if

\[Y = c_0 + c_1X_1 + \cdot\cdot\cdot + c_nX_n\]

then

\[\mu_Y = c_0 + c_1\mu_1 + \cdot\cdot\cdot + c_n\mu_n\]

and

\[\sigma_Y^2 = c_1^2\sigma_1^2 + \cdot\cdot\cdot + c_n^2\sigma_n^2\]

But if this is a sample from the population, then the means of each #X_i# are the same, and the variances of each #X_i# are the same, that is, the mean of #X_i# for each #i# is the mean of #X#, denoted #\mu_X#, and the variance of #X_i# for each #i# is the variance of #X#, denoted #\sigma^2_X#. Here #\mu_X# is the population mean of #X# if #X# were to be measured on every element of the population.

Consequently, in this situation if

\[Y = c_0 + c_1X_1 + \cdot\cdot\cdot + c_nX_n\]

then

\[\begin{array}{rcl}

\mu_Y &=& c_0 + c_1\mu_1 + \cdots+ c_n\mu_n \\\\

&=& c_0 + \mu_X\Big(c_1 + \cdot\cdot\cdot + c_n\Big)

\end{array}\]

and

\[\begin{array}{rcl}

\sigma_Y^2 &=& c_1^2\sigma_1^2 + \cdot\cdot\cdot + c_n^2\sigma_n^2 \\\\

&=& \sigma_X^2\Big(c^2_1 + \cdot\cdot\cdot + c^2_n\Big)

\end{array}\]

The most common example of this situation is the *sample mean*, denoted #\bar{X}#.

Sample Mean as a Random Variable

Before the sample is selected, the sample mean is a random variable; its eventual value is uncertain. Thus the sample mean has a probability distribution.

The sample mean is a linear combination of the observations of #X# on the sample:

\[\bar{X} = \cfrac{1}{n} \sum_{i=1}^n X_i = \cfrac{1}{n}X_1 + \cdots+ \cfrac{1}{n}X_n\]

So now all the constants have the same value, #\cfrac{1}{n}#.

Thus, **the mean of the probability distribution of the sample mean** is:

\[\begin{array}{rcl}

\mu_{\bar{X}} &=& \mu_X\Big(\cfrac{1}{n} + \cdots + \cfrac{1}{n}\Big) \\\\

&=& \mu_X\Big(\cfrac{n}{n}\Big) = \,\,\mu_X

\end{array}\]

i.e., the mean of the probability distribution of the sample mean is the population mean. We have just tied together 3 different uses of the term “mean”!

Likewise, **the variance of the probability distribution of the sample mean** is:

\[\begin{array}{rcl}

\sigma^2_{\overline{X}} &=& \sigma^2_X\Big(\cfrac{1}{n^2} + \cdot\cdot\cdot + \cfrac{1}{n^2}\Big) \\\\

&=& \sigma^2_X\Big(\cfrac{n}{n^2}\Big) = \,\,\cfrac{\sigma^2_X}{n}

\end{array}\]

Hence, **the standard deviation of the probability distribution of the sample mean** is:

\[\sigma_{\overline{X}} = \cfrac{\sigma_X}{\sqrt{n}}\]

#\text{}#

These three results are of the utmost importance for statistics, so we repeat them again:

Mean, Variance, and Standard Deviation of the Probability Distribution of the Sample Mean

Given future measurements #X_1,...,X_n# of a quantitative variable #X# on a simple random sample of size #n# from some population, where the mean and the variance of #X# measured on the entire population are #\mu_X# and #\sigma^2_X#, respectively, then the mean, variance, and standard deviation of the probability distribution of the sample mean #\bar{X} = \cfrac{1}{n}\sum_{i=1}^n X_i# are:

\[\begin{array}{rcl}

\mu_{\bar{X}} &=& \mu_X \\\\

\sigma^2_{\bar{X}} &=& \cfrac{\sigma_X^2}{n} \\\\

\sigma_{\bar{X}} &=& \cfrac{\sigma_X}{\sqrt{n}}

\end{array}\]

The importance of these results will become clearer as we go.

**Pass Your Math**independent of your university. See pricing and more.

Or visit omptest.org if jou are taking an OMPT exam.