1. Review of random variables and summary statistics
Definitions:
• Sample space: A collection of all possible outcomes of a random experiment. Usually denoted by Ω.
• Probability distribution: An assignment of numbers between 0 and 1 to each possible outcomes. Sum of these numbers should be equal to 1. Usually denoted by
P.
• Random variable: A function defined on a sample space. Usually denoted by X.
• Expected value of X: µx = E[X] =

P(ω)X(ω) ω∈Ω (if X is a continuous random variable, then the summation should be replaced by the integral).
• Variance of X:
2
V ar(X) = σx = E[(X − µx )2 ]
• Standard deviation of X:
E[(X − µx )2 ]

σx =
• Covariance of X and Y :

Cov(X, Y ) = E[(X − µx )(Y − µx )]
• Correlation of X and Y :
Cov(X, Y ) σx σx
Consider a sample {(x1 , y1 ), (x2 , y2 ), · · · , (xn , yn )}. Then,
• Sample mean: n 1 x= ¯ xi n i=1 ρxy =

• Sample variance: s2 x

n

1
=
n−1

(xi − x)2
¯
i=1

• Sample standard deviation: sx =

1 n−1 n

(xi − x)2
¯
i=1

• Sample covariance: sxy 1
=
n−1

n

(xi − x)(yi − x)
¯
¯ i=1 1

2

• Sample correlation: rxy =

sxy sx sy

Interpretation of correlation rxy
• −1 ≤ rxy ≤ 1 always.
• If rxy > 0, then large values of X tend to be associated with large values of Y , and small values of X tend to be associated with small values of Y . In this case, X and
Y are positively linearly related.
• If rxy < 0, then large values of X tend to be associated with small values of Y , and small values of X tend to be associated with large values of Y . In this case, X and
Y are negatively linearly related.
• Bigger |rxy | implies stronger linear relation between X and Y .
• If rxy = 0, then X and Y are not linearly related (they may have non-linear relationship).
• If rxy = 1, then all of (xi , yi ) are on a straight line with positive