[A, SfS] Chapter 2: Probability: 2.5: Linear Combinations of Random Variables
Linear Combinations of Random Variables
Linear Combinations of Random Variables
In this lesson you will learn how to compute the mean and variance of a linear combination of random variables.
#\text{}#
Linear Combination
Suppose you have #n# random variables (discrete or continuous), denoted #X_1,X_2,\ldots,X_n#, and let #c_0,c_1,c_2,\ldots,c_n# denote #n + 1# constants.
Then #c_0+c_1X_1+c_2X_2+\cdots+c_nX_n# is called a linear combination of #X_1,X_2,...,X_n#.
#\text{}#
We would like to be able to compute the mean and variance of the linear combination if we already know the mean and variance of each #X_i#.
Let’s denote the mean of #X_i# by #\mu_i# and the variance of #X_i# by #\sigma^2_i#.
Mean for a Linear Combination
Using the linearity property of both sums and integrals, the mean (also called the expectation) of a linear combination is:
\[E\Big(c_0 + c_1X_1 + c_2X_2 + \cdots + c_nX_n\Big) = c_0 + c_1\mu_1 + c_2\mu_2 + \cdots + c_n\mu_n\]
#\text{}#
The variance of a linear combination is more complicated, because it involves squaring. However, we consider a special case for which the formula becomes simple.
Independent Random Variables
Two random variables #X# and #Y# are independent if and only if
\[ \mathbb{P}[(X \leq x) \cap (Y \leq y)] = \mathbb{P}(X \leq x) \mathbb{P}(Y \leq y) \]
for every pair of real numbers #x# and #y#. Essentially this means that having information about the value of one of the r.v.s has no impact on the probability distribution of the other r.v.
Similarly, the random variables #X_1, X_2, \ldots, X_n# are mutually independent if and only if
\[ \mathbb{P}[(X_1 \leq x_1) \cap (X_2 \leq x_2) \cap \cdots \cap (X_n \leq x_n)] = \mathbb{P}(X_1 \leq x_1) \mathbb{P}(X_2 \leq x_2) \cdots \mathbb{P}(X_n \leq x_n) \]
for every set of real numbers #x_1, x_2, \ldots, x_n#.
Variance of a Linear Combination
If (and only if) the random variables #X_1,X_2,...,X_n# are independent, the variance formula becomes:
\[V\Big(c_0 + c_1X_1 + c_2X_2 + \cdots + c_nX_n\Big) = c^2_1\sigma^2_1+c^2_2\sigma^2_2+\cdots +c^2_n\sigma^2_n\]
#\text{}#
To demonstrate the mean and variance of a linear combination of r.v.s, we begin with simple situations and progress to more complex situations.
Linear Combination of a Single Random Variable
Suppose #X# is a r.v. with mean #\mu_X# and variance #\sigma^2_X#.
Let #Y = a + bX# for some constants #a# and #b#. Then:
\[\begin{array}{rcl}
\mu_Y &=& a + b\mu_X\\\\
\sigma^2_Y &=& b^2\sigma^2_X\\\\
\sigma_Y &=& |b|\sigma_X
\end{array}\]
Suppose #X# is a r.v. with mean #10# and variance #4#.
Let #Y = 50 - 5X#. Then:
\[\begin{array}{rcl}
\mu_Y &=& 50 - 5 {\mu_X} = 50 - (5)(10) = 0\\\\
\sigma_Y^2 &=& (-5)^2\sigma^2_X = (25)(4) = 100\\\\
\sigma_Y &=& |-5| \sigma_X = (5)(2) = 10
\end{array}\]
#\text{}#
Linear Combination of Two Random Variables
Next, suppose #X# and #Y# are independent r.v.s with means #\mu_X# and #\mu_Y#, respectively, and with variances #\sigma_X^2# and #\sigma_Y^2#, respectively.
Let #Z = a + bX + Y#. Then:
\[\mu_Z = a + b\mu_X + c\mu_Y\]
and, because of the independence of #X# and #Y#:
\[\sigma_Z^2 = b^2\sigma_X^2 + c^2\sigma_Y^2\]
Suppose #X# and #Y# are independent r.v.s with #\mu_X = 12#, #\sigma^2_X = 9#, #\mu_Y = 8#, and #\sigma_Y^2 = 1#.
Let #Z = 4X - 2Y#. Then:
\[\begin{array}{rcl}
\mu_Z &=& 4\mu_X - 2\mu_y = (4)(12) - (2)(8) = 32\\\\
\sigma_Z^2 &=& (4)^2\sigma^2_X + (-2)^2\sigma_Y^2 = (16)(9) + (4)(1) = 148
\end{array}\]
Warning: Many students would miscalculate the variance in this example! In order to implement the variance formula correctly, the linear combination must be in the form #aX + bY#, not #aX - bY#.
That means we should rewrite the combination #aX - bY# as #aX + (-b)Y#.
So in this example we would write #Z = 4X - 2Y# as #Z = 4X + (-2)Y# before using the variance formula. Don’t forget!
#\text{}#
If we plan to select a simple random sample of size #n# from the population and measure some quantitative variable #X# on each element of the sample, we can denote the future measurements on the sample by #X_1,...,X_n#. Because it is going to be a random sample, we consider #X_1,...,X_n# to be independent random variables, since their eventual values are uncertain.
We have already explained that if
\[Y = c_0 + c_1X_1 + \cdot\cdot\cdot + c_nX_n\]
then
\[\mu_Y = c_0 + c_1\mu_1 + \cdot\cdot\cdot + c_n\mu_n\]
and
\[\sigma_Y^2 = c_1^2\sigma_1^2 + \cdot\cdot\cdot + c_n^2\sigma_n^2\]
But if this is a sample from the population, then the means of each #X_i# are the same, and the variances of each #X_i# are the same, that is, the mean of #X_i# for each #i# is the mean of #X#, denoted #\mu_X#, and the variance of #X_i# for each #i# is the variance of #X#, denoted #\sigma^2_X#. Here #\mu_X# is the population mean of #X# if #X# were to be measured on every element of the population.
Consequently, in this situation if
\[Y = c_0 + c_1X_1 + \cdot\cdot\cdot + c_nX_n\]
then
\[\begin{array}{rcl}
\mu_Y &=& c_0 + c_1\mu_1 + \cdots+ c_n\mu_n \\\\
&=& c_0 + \mu_X\Big(c_1 + \cdot\cdot\cdot + c_n\Big)
\end{array}\]
and
\[\begin{array}{rcl}
\sigma_Y^2 &=& c_1^2\sigma_1^2 + \cdot\cdot\cdot + c_n^2\sigma_n^2 \\\\
&=& \sigma_X^2\Big(c^2_1 + \cdot\cdot\cdot + c^2_n\Big)
\end{array}\]
The most common example of this situation is the sample mean, denoted #\bar{X}#.
Sample Mean as a Random Variable
Before the sample is selected, the sample mean is a random variable; its eventual value is uncertain. Thus the sample mean has a probability distribution.
The sample mean is a linear combination of the observations of #X# on the sample:
\[\bar{X} = \cfrac{1}{n} \sum_{i=1}^n X_i = \cfrac{1}{n}X_1 + \cdots+ \cfrac{1}{n}X_n\]
So now all the constants have the same value, #\cfrac{1}{n}#.
Thus, the mean of the probability distribution of the sample mean is:
\[\begin{array}{rcl}
\mu_{\bar{X}} &=& \mu_X\Big(\cfrac{1}{n} + \cdots + \cfrac{1}{n}\Big) \\\\
&=& \mu_X\Big(\cfrac{n}{n}\Big) = \,\,\mu_X
\end{array}\]
i.e., the mean of the probability distribution of the sample mean is the population mean. We have just tied together 3 different uses of the term “mean”!
Likewise, the variance of the probability distribution of the sample mean is:
\[\begin{array}{rcl}
\sigma^2_{\overline{X}} &=& \sigma^2_X\Big(\cfrac{1}{n^2} + \cdot\cdot\cdot + \cfrac{1}{n^2}\Big) \\\\
&=& \sigma^2_X\Big(\cfrac{n}{n^2}\Big) = \,\,\cfrac{\sigma^2_X}{n}
\end{array}\]
Hence, the standard deviation of the probability distribution of the sample mean is:
\[\sigma_{\overline{X}} = \cfrac{\sigma_X}{\sqrt{n}}\]
#\text{}#
These three results are of the utmost importance for statistics, so we repeat them again:
Mean, Variance, and Standard Deviation of the Probability Distribution of the Sample Mean
Given future measurements #X_1,...,X_n# of a quantitative variable #X# on a simple random sample of size #n# from some population, where the mean and the variance of #X# measured on the entire population are #\mu_X# and #\sigma^2_X#, respectively, then the mean, variance, and standard deviation of the probability distribution of the sample mean #\bar{X} = \cfrac{1}{n}\sum_{i=1}^n X_i# are:
\[\begin{array}{rcl}
\mu_{\bar{X}} &=& \mu_X \\\\
\sigma^2_{\bar{X}} &=& \cfrac{\sigma_X^2}{n} \\\\
\sigma_{\bar{X}} &=& \cfrac{\sigma_X}{\sqrt{n}}
\end{array}\]
The importance of these results will become clearer as we go.
Or visit omptest.org if jou are taking an OMPT exam.