1/22/07 252corr (Open this document in 'Outline' view!)

L. CORRELATION

1. Simple Correlation

The simple sample correlation coefficient is [pic] or if spare parts [pic], [pic] and [pic] are available, we can say [pic]
Of course, since the coefficient of determination is [pic][pic] , [pic] and it is often easier to compute [pic] and to give the correlation the sign of [pic] . But note that the correlation can range from +1 to -1, while the coefficient of determination can only range from 0 to 1. Also note that since the slope in simple regression is [pic] , [pic] or [pic] or [pic]. The last equation has a counterpart in [pic] , where [pic] is the population correlation coefficient, so that testing [pic] is equivalent to testing [pic]and the simple regression coefficient and the correlation will have the same sign.

2. Correlation when x and y are both independent

If we want to test[pic] against [pic]and [pic]are normally distributed, we use [pic]. But note that if we are testing [pic] against [pic], and [pic], the test is quite different. We need to use Fisher's z-transformation. Let [pic]. This has an approximate mean of [pic] and a standard deviation of [pic] , so that [pic] .

(Note: To get [pic], the natural log, compute the log to the base 10 and divide by .434294482. )

Example: Test [pic] when [pic][pic] and [pic][pic].
To solve this we first compute [pic] and [pic] Compare this with [pic]. Since this is not between these two values of t, reject the null hypothesis. (Note that [pic] so that this is equivalent to an F test on [pic] in a regression.)
Example: Test [pic] when [pic][pic] and [pic][pic].
This time compute Fisher's z-transformation (because [pic] is not zero) [pic] [pic]
[pic]. Finally [pic] Compare this with [pic]. Since –0.591 lies between these two values, do not reject the null hypothesis.
Note: To do the above with logarithms to the base 10, try [pic]. This has an approximate mean of [pic] and a standard deviation of [pic] , so that [pic] .

3. Tests of Association

a. Kendall's Tau. (Omitted)

b. Spearman's Rank Correlation Coefficient.

Take a set of [pic] points [pic]and rank both [pic]from 1 to [pic] to get [pic]. Do not attempt to compute a rank correlation without replacing the original numbers by ranks, A correlation coefficient between [pic]can be computed as in point 2 above, but it is easier to compute [pic] ,and then [pic]. This can be given a t test for [pic] as in point 2 above, but for [pic] between 4 and 30, a special table should be used. For really large [pic], [pic] may be used.
Example: 5 applicants for a job are rated by two officers, with the following results. Note that in this example the ranks are given initially. Usually the data must be replaced by ranks. [pic]
[pic] Test to see how well the ratings agree.
In this case, we have a 1-sided test [pic]. Arrange the data in columns.
[pic]Note that [pic]. Since [pic] [pic]. If we check the table ‘Critical Values of [pic]the Spearman Rank Correlation Coefficient,’ we find that the critical value for [pic] and [pic]is .8000 so we must not reject the null hypothesis and we conclude that we cannot say that the rankings agree.
Example: We find that for [pic][pic]. We want to do the same one-sided test as in the last problem [pic]. [pic] We can do a t-test by computing [pic] and [pic]. This is compared with [pic]. Since the t we computed is above the table value, we reject the null hypothesis. Or we can compute a z-score [pic]. Since this is above [pic] we can reject the null hypothesis.

c. Kendall's Coefficient of Concordance.

Take [pic] columns with [pic] items in each and rank each column from 1 to [pic]. The null hypothesis is that the rankings disagree. Compute a sum of ranks [pic] for each row. Then [pic] , where [pic] is the mean of the [pic]s. If [pic] is disagreement, [pic] can be checked against a table for this test. If [pic] reject [pic]. For [pic] too large for the table use [pic] , where [pic] is the Kendall Coefficient of Concordance and must be between 0 and 1.

Example: [pic] applicants are rated by [pic]officers. The ranks are below. [pic]Note that if we had complete disagreement, every applicant would have a rank sum of 10.5. [pic]. The Kendall Coefficient of Concordance says that the degree of agreement on a zero to one scale is [pic]. To do a test of the null hypothesis of disagreement [pic], look up [pic] in the table giving ‘Critical values of Kendall’s [pic] as a Measure of Concordance’ for [pic] and [pic], [pic]so that we accept the null hypothesis of disagreement.. Example: For [pic] and [pic] we get [pic], and wish to test [pic] Since [pic] is too large for the table, use [pic]. Using a [pic] table, look up [pic] . Since 9 is below the table value, do not reject [pic].

4. Multiple Correlation

If [pic]is the coefficient of determination for a regression [pic], then the square root of [pic] , [pic] is called the multiple correlation coefficient. Note that [pic] where [pic] is the sample variance of [pic], and that for large [pic], [pic].

5. Partial Correlation (Optional)

If [pic], its multiple correlation coefficient can be written as [pic] or [pic]. For example, in the multiple regression problem, we got three multiple correlation coefficients [pic], [pic] and [pic] If [pic] and we compute the partial correlation of [pic]we compute [pic] , the additional explanatory power of the third independent variable after the effects of the first two are considered. If we read [pic] from the computer printout, [pic], where [pic]and [pic]is the number of independent variables.
Example: In the multiple regression problem with which we have been working [pic] is the additional explanatory power of [pic]beyond what was explained by [pic] and [pic]. It can be computed two ways. First [pic]. The partial correlation coefficient is actually [pic]. The sign of the partial correlation is the sign of the corresponding coefficient in the regression. (For the regression equation see below.)
For the second method of computing [pic], recall that the last printout for the regression with which we were working was
Y = 1.51 + 0.595 X - 0.698 W - 0.937 H

Predictor Coef Stdev t-ratio p
Constant 1.5079 0.2709 5.57 0.001
X 0.5952 0.1198 4.97 0.003
W -0.6984 0.4860 -1.44 0.201
H -0.9365 0.5239 -1.79 0.124
Thus the t corresponding to [pic] is [pic] and, since [pic] , [pic].

6. Collinearity

If [pic] , and [pic] are highly correlated, then we have no real variation of [pic] relative to [pic]. This is a condition known as (multi)collinearity. The standard deviations for both [pic] and [pic] will be large and, in extreme cases, the regression process may break down. Recall that in section I, we said that small variation in [pic] can lead to large values of [pic]in simple regression and thus insignificant values of [pic].
Similarly in multiple regression, lack of movement of the independent variables relative to one another leaves the regression process unable to tell what changes in the dependent variable are due to the various independent variables. This will be indicated by large values of [pic] or [pic] which cause us to find the coefficients insignificant when we use a t-test.
A relatively recent method to check for collinearity is to use the Variance Inflation Factor [pic]. Here [pic] is the coefficient of multiple correlation gotten by regressing the independent variable [pic] against all the other independent variables [pic]. The rule of thumb seems to be that we should be suspicious if any [pic] and positively horrified if [pic]. If you get results like this, drop a variable or change your model. Note that, if you use a correlation matrix for your independent variables and see a large correlation between two of them, putting the square of that correlation into the VIF formula gives you a low estimate of the VIF, since the R-squared that you get from a regression against all the independent variables will be higher.

Example: Note that in the printout in section 5, the standard deviations for the coefficients of [pic]and [pic] are quite large, resulting in small t-ratios and p-values which lead us to believe that the coefficients are not even significant when the significance level is 10%. The data from section J4 is repeated at right: [pic]

Computation from these numbers reveals that [pic] [pic]and [pic] Thus [pic] and [pic] so that [pic], [pic] and [pic]. Finally, [pic], a relatively high correlation. This and the relatively small sample size account for the large standard deviations and the generally discouraging results. Though the regression against two independent variables has been shown to be an improvement over the regression against one independent variable, addition of the third independent variable, in spite of the high [pic] was useless. Preliminary use of the Minitab correlation command, as below, might have warned us of the problem.

MTB > Correlation 'X' 'W' 'H'.

Correlations (Pearson)

X W
W -0.068
H -0.145 0.802
Actually for this problem, the largest VIF, for H, is only about 2.86, but it seems to interact with the small sample size.