QTM 3650 – Prof. Rick Cleary Class #9–Mon. 2/24/14

I.) Logistics
- Return and discuss homework problems 12-14; notes below. Collect problems 15-16.
- Final exam time for our class: Wednesday 5/7, 3:30 – 5:30 PM, Gerber 103.
- Review schedule for upcoming two weeks:

10Weds. 2/26 | Classification and Regression TreesProject 1 dueTake home part of Exam #1 assigned | Text Chapter 9 | 11Mon. 3/3 | Finish Chapter 9Catch up/review/special topics/Visualization tools | - | 12Weds. 3/5 |
EXAM #1In class portion and take home due | - |

- A note on ground rules for the in-class exam: Each student may bring one 8.5” by 11” review sheet to the exam with them; OK to write on both sides. The sheet will be turned in with the exam. No electronic devices of any kind except for a calculator that can do simple arithmetic. It’s OK to use a hand-held device (not a laptop) as a calculator as long as it is used only for that purpose. If students don’t have a calculator, Prof. Cleary will have a couple on hand, or students may use the podium computer to do calculations only in Excel. Any connection to the internet or communication to others (texting, etc.) during the exam will be considered an academic integrity violation.
- Today’s topics: Finish Chapter 8 on the naïve Bayes estimator and review the Chi-Squared test of independence.

II.) Comments on homework problems 12-14.

Problem 12.) While we reviewed this last class, enough people missed it on problem 12 that it’s worth saying again. When we have a regression model of the form:

y = b0 + b1x1 + … + bpxp

We predict a value for y (the ‘y-hat’) by plugging in the x’s for the new observation into the equation. Once we have that value, if we want a range where it is likely to fall, an approximate 95% prediction interval for the value of y for this new observation is:

y ± 2s, where s is the standard error of the residuals.

On the homework part (c.) many students used just one standard deviation, or used the standard deviation of all salaries, not the residual standard deviation from the regression model.

Problem 13.) In part (b), some students lost a point for not picking up this important connection between the two-sample t-test and the same data run is a regression with one categorical predictor: The coefficient in the regression equation is exactly the difference in means between the two groups. In this case, the regression coefficient of -2101 has an easy to interpret meaning: On average, faculty with a PhD had salaries $2101 lower than those without PhDs.

Problem 14.) Most people correctly normalized (standardized) the remaining variables and computed the distances correctly, but then there were two common errors:
-Several students ordered the scores from largest to smallest, inadvertently comparing the Celtics to the team most different, rather than most similar to themselves.
-About half the class misinterpreted how to do classification in the k-nn approach. When doing classification, we simply count how many of the nearest neighbors fall in each category. In problem 14, two of the three nearest neighbors for the Celtics did not make the playoffs, thus our 3-nn model incorrectly predicts that the Celtics would miss the playoffs. (It’s even worse with k = 5.) We do NOT need to first do the stop of computing the mean number of wins for the Celtics. That’s an estimation problem.

III.) Today’s topics: Overview of coverage so far, finishing Chapter 8 on Naïve Bayes’ Estimates, reviewing the Chi-Squared test of independence

We begin with a quick look at ‘where we are’ from a modeling perspective. In the models we’ve studies so far we have a single variable that we are interested in as a response, and we have a set of predictor variables. The table below lists choices of tools to use. As the semester goes on we’ll fill this in further and also do some models where we are not just trying to name a single predictor.

| Categorical Response | Numerical Response | Categorical Predictors only | -Pivot tables-Chi-squared test -Exact/Naïve Bayes | -Side by side box plots-Two-sample t (if single cat. pred. with only 2 values.)-ANOVA or regression with dummy variables | Numerical Predictors (with or w/out categorical pred) | -Side by side box plots-k-nn for classification-CART | -Scatter plots-Regression-k-nn for estimation |

Now back to Chapter 8: Last time we looked at the ‘exact Bayes’ approach in which we classified a new observation based on finding exact matches for the given set of categorical predictors. Not surprisingly, this is very difficult to actually accomplish in practice. So we move on to a more broadly applicable generalization, the “Naïve Bayes” approach, which uses the entire data set as described below. We work through this using the data from Homework Problem 15. (Compare to algorithm on text p. 152.)

Suppose our possible response has m different categories (called classes in the text) that a new record could be assigned to, and that there are p categorical predictors in the problem. (For problem 15, what are m and p?)

Now, for each class, we find the probability of being in that class for the data in the new record, one variable at a time. So for problem 15 where we have two classes (0…rejected, 1…accepted), we find the probability that our new record (female (1), no AP(0), not a legacy (0), not being recruited) belongs in each class by each variable. To do this we need the pivot tables in Excel, or we can count in Minitab or R. When we have done this for each variable for a certain class, we multiply all these probabilities together and also times the probability of being in that particular class. When we repeat this every class we have exactly set up Bayes’ rule as discussed last time! To use the Problem 15 notation:

(***) P(A=1|1, 0, 0, 0) =

PA=1*Pgender=1A=1*PAP=0A=1*Pleg=0A=1*P(rec=0|A=1)Sum of term above+similar term for A=0 case

This step includes an assumption of independence between the predictors.

So let’s step through this for Problem 15! Here’s the R-command we need to get, for instance, the recruiting table, with the actual data table following.

Admits <- read.csv("C:\\Users\\Student\\Documents\\QTM3650S14\\HWProb15ExactBayesData.csv", header = T) attach(Admits) Admittable <- table(Admits$Coach,Admits$Admit) # 1st var is rows, 2nd is columns.
Admittable #Typing the name of the table prints it out in console

0 1 0 25 28 1 7 15

Which piece of the equation (***) above can we get from this table? Note that R doesn’t like Gender (M=0), the title of Column 1 on the problem 15 Excel sheet, as a variable name… we have to change that! Gathering every piece, we find that we can update (***) to say:
P(A=1|1,0,0,0) =
(43/75)*( 14/43)*(33/43)*(20/43)*(28/43)4375*1443*3343*2043*2843+3275*532*2532*2632*(2532) = .5675 … a pretty good chance of being accepted! (Note: Compare to exact Bayes’ solution from problem 15?)

We end Chapter 8 with another great Bayesian example that you might know, but probably haven’t thought about in a modeling context.

Now, we review the chi-squared test of independence. This is a statistical test used to determine if the row and column variables in a pivot table are statistically independent of each other. It is not usually a ‘big data’ test, because with very large samples it tends to lead to statistically significant results that may not be practically significant. But it is a very important tool to have for an analyst wanting to make a quick decision about statistical significance in a pivot table.

The data needed to perform the chi-squared (χ2) test is just the ordinary pivot table, also known as a contingency table, in which the entries are the observed frequencies of data that falls into each combination of row and column variable. For each cell, we also compute an expected frequency that gives the value we’d expect if the row and column variables were in fact independent.

The expected value for any cell is computed as follows: ((row total) * (column total))/n, where n is the total sample size.

Important: A chi-squared test should not be used unless the expected value for each cell is at least five. If the table has many categories and some cells have expected values less than five, we can collapse into a small number of rows and/or columns by combining groups.

The calculated chi-squared statistic is a measurement of whether the actual data frequencies we observed were in fact close to what we would expect if the row and column variables were independent, and is computed as follows:

2 calc = ((observed – expected)2 / expected),

where the sum is over every interior cell in the table, If r is the number of categories for the row variable and c is the number of categories for the column variable, we will have to add up this term for r*c cells.

The hypotheses we are testing are:

H0 : The row variable is independent of the column variable. Vs. Ha : The variables are not independent.

Large values of the chi-squared statistic indicate that we should reject the null, and we can use Excel to get a p-value. (Use the command CHIDIST(x,d) where x is the computed value from the table and d is the degrees of freedom.) The degrees of freedom is given by (r-1)*(c-1). We explore the use of the chi-squared test on some of the data from problem 15.

For problem 15, if we want to use R to carry out the chi-squared test to see if being recruited by a coach is independent of being admitted, we simply need to use the command:

> chisq.test (Admittable)

Pearson's Chi-squared test with Yates' continuity correction

data: Admittable
X-squared = 0.936, df = 1, p-value = 0.3333

We compute this step by step in Excel to insure that we have the idea.

IV.) Today’s fun event: Another application of Bayes’ Rule: The famous Monty Hall, “Let’s make a deal’ problem, aka the “Prisioner’s Dilemma.”

V.) Homework due at the START of class #11, Monday 3/3/14 … Note no homework due on Wednesday 2/26 when project is due.

Please bring a hard copy, neatly typed or carefully hand written.

Problem 17.) Naïve Bayes estimation: Problem 8.1, page 162, parts c, d and e only. The data for this problem is on the course Blackboard site. For simplicity, we use only the first 1000 records in the data set, and we consider these the training set. You may compute the pivot tables in any software you like, though of course I always encourage students to keep learning R.

Problem 18.) Continuing with the data set from problem 17, carry out a chi-squared test of independence to decide if the variable “PersonalLoan” is independent of the variable “CreditCard” Carefully state the null and alternative hypotheses, find the chi-squared value and the p-value, and then state the business conclusion carefully in a sentence.

Problem 19.) Much of our class on Monday 3/3 will consist of review for the in-class mid-term. Write a good, detailed question that you would like covered in that review. If possible please reference a particular homework problem, class example or page of the text to help make the question specific.
Not as good: “Could you go over k-nn for classification?”
Better: “Can you explain the voting procedure for k-nn classification like we had on problem 14 for the playoffs?”
I will randomly choose questions to go over in class on 3/3 until we run out of time.