Effects of the California High School Exit Exam on Student Persistence, Achievement, and Graduation Sean F. Reardon Allison Atteberry Nicole Arshan Michal Kurlaender April, 2009 (full paper available at irepp.stanford.edu) Executive Summary Roughly half of the 50 U.S. states now require students to pass a high school exit exam in order to graduate from high school. These exam requirements are predicated on the belief that “raising the bar” for graduation will motivate low‐achieving students to work harder in school, motivate schools to improve their instruction for low‐achieving students, and provide employers with a clearer signal of high school graduates’ skills. Opponents of these requirements, however, argue that creating additional barriers to graduation discourages students—particularly academically and socially disadvantaged students—from persisting in school and hence leads to increased dropout rates and greater inequality of educational outcomes. Prior research has found somewhat mixed evidence regarding the effects of exit exam requirements, though the most rigorous studies suggest that the introduction of exit exam requirements leads to roughly a 2 percentage point decline in high school graduation rates. However, prior research has not examined closely the extent to which this effect results from students leaving school early or from students failing to pass the exam despite staying in school through 12th grade. In addition, prior research has not examined whether the negative graduation effect is offset by improvements in student achievement. Finally, prior research has not examined closely the extent to which the effects of exit exam requirements vary among students of different achievement levels or demographic characteristics. In 2006, California joined the growing list of states requiring a high school exit exam for graduation. Beginning with the high school graduating class of 2006, students in California are required to pass the California High School Exit Exam (CAHSEE) in order to receive a high school diploma. In this study, we use longitudinal student data from four large California school districts to estimate the effects of the CAHSEE requirement on students’ persistence in high school (whether they stay in school through 11th grade and through 12th grade), their academic achievement (as measured by their 11th grade ELA CST scores), and their graduation rates. To do this, we compare the persistence, achievement, and graduation rates of students in a cohort that was not subject to the CAHSEE exit exam requirement (those who were scheduled to graduate in 2005) to similar students in two later cohorts who were subject to the requirement (those who were scheduled to

graduate in 2006 and 2007). In order to determine whether the CAHSEE affects low‐achieving students more strongly than higher‐achieving students, we conduct these comparisons separately for students with different levels of achievement. In order to be sure that between‐cohort differences in outcomes are attributable to the CAHSEE policy rather than to some other factors affecting student outcomes, we take advantage of the fact that students in the cohort scheduled to graduate in 2005 (who were not subject to the CAHSEE requirement) took the CAHSEE exam in 10th grade under the belief that it would be a graduation requirement (the policy was changed shortly after they took the exam in Spring of 2003). This allows us to estimate difference‐in‐difference models that compare changes in outcomes for those who failed the CAHSEE in 10th grade to changes in outcomes for similar students who passed the CASHEE in 10th grade. Because we would expect any secular trend in outcomes to affect both passers and failers similarly, these models provide lower‐bound estimates of the effects of the CAHSEE (they assume that the CAHSEE requirement would only have effects on the outcomes of those students who fail the CAHSEE). For each of the four outcomes of interest, we address the following questions: 1. What is the effect of the CAHSEE graduation requirement on each of these outcomes among students each quartile of 10th grade achievement? 2. Do these effects differ by students’ race/ethnicity, sex, ELL status, and free‐lunch eligibility? 3. If so, what accounts for these differential effects? Main Findings 1. Low­achieving students subject to the CAHSEE requirement had slightly lower rates of persistence in high school than similar students not subject to the requirement. On average, persistence rates (the percentage of students remaining in school in their original district) through 11th grade were 2 percentage points lower among bottom‐quartile students and 1 percentage point lower among 2nd quartile students who were subject to the CAHSEE requirement than among those who were not. Persistence rates through 12th grade were 4 percentage points lower among bottom‐quartile students and 2 percentage points lower among 2nd quartile students who were subject to the CAHSEE requirement than among those who were not. There were no significant differences in persistence rates among students in the upper achievement quartiles. Our difference‐in‐difference models indicate that we cannot be sure that these differences in persistence rates are attributable to the CAHSEE requirement. 2. There is no evidence that students subject to the CAHSEE requirement learned more between 10th and 11th grade than those who were not subject to the requirement. On average, scores on the 11th grade ELA CST test (the state ELA test that all students take for

school accountability purposes) were slightly lower among students subject to the CAHSEE requirement than among similar students not subject to the requirement, though we cannot be certain this decline is a result of the CAHSEE requirement. It is clear, however, that there is no evidence that the CAHSEE requirement improves students’ performance on their 11th grade ELA CST tests. 3. Low­achieving students subject to the CAHSEE requirement have substantially lower graduation rates than similar students not subject to the CAHSEE. On average, graduation rates (the percentage of students receiving a diploma on time from their original district) were 15 percentage points lower among bottom‐quartile students and 3 percentage point lower among 2nd quartile students who were subject to the CAHSEE requirement than among similar students who were not. In the cohort of students not subject to the requirement, a typical student in the bottom quartile of 10th grade achievement had a roughly 45% probability of graduating from his or her original district; in the cohorts subject to the requirement, the same student had a 30% probability of graduation. Our difference‐in‐difference estimates indicate that the CAHSEE requirement caused at least 11 percentage points of the decline in graduation rates (and as much as 15 percentage points). These figures imply that roughly 3.6%‐4.5% of California high school students (roughly 18,000‐22,500 students per year) do not graduate as a result of the high school exit exam policy. 4. The negative effects of the CAHSEE requirement on graduation rates fall disproportionately on minority students and female students. Among students in the lowest quartile of achievement, the CAHSEE requirement has no effect on the graduation rate of white students, but a large negative effect on graduation rates of black, Hispanic, and Asian students. On average, among students in the bottom quartile of achievement, graduation rates were 19 percentage points lower among Black students, 15 points lower among Hispanic students, and 17 points lower among Asian students who were subject to the CAHSEE requirement than among similar students not subject to the requirement. Likewise, graduation rates were 19 percentage points lower among female students, but only 12 points lower among male students who were subject to the CAHSEE requirement than among similar students not subject to the requirement. [SEE FIGURE BELOW] 5. The disproportionate effects of the CAHSEE requirement on graduation rates of minority students are not due to differences among schools attended by minority and white students. The large racial/ethnic differences in the effects of the CAHSEE are evident even when we compare students within the same schools, suggesting that it is not differences in school quality that primarily account for the racial/ethnic differences in the effects of the CASHEE.

6. The disproportionate effects of the CAHSEE requirement on graduation rates appear to be due to large racial and gender differences in CAHSEE passing rates among students with the same level of achievement. Minority students perform less well on the CAHSEE exam than do white students with the same level of prior and current academic achievement (as measured by 8th, 9th, and 10th grade performance on the California Standards Tests). Female students perform less well on the math CAHSEE test than do boys with the same level of academic achievement. This pattern of results is consistent with ‘stereotype threat’ explanations of test performance. Stereotype threat is the phenomenon whereby the fear that if one performs poorly on a high‐stakes test it will confirm a negative societal stereotype about one’s group leads to increased test anxiety among negatively stereotyped student groups—minority students and girls, for example—which in turn leads such students to underperform on such tests relative to similarly skilled non‐stereotyped students. The evidence suggests that the high‐ stakes nature of the CAHSEE test induces minority students and girls to underperform on the CAHSEE, relative to what their prior academic performance (on low‐stakes tests) would predict. Failure to pass the CAHSEE for these groups leads to lower graduation rates. Thus, it is not that the CAHSEE test is biased, but that the high stakes nature of the exit exam testing situation combined with the presence of negative societal stereotypes about the academic skill of minority students and girls (in math) leads to underperformance by minority students and girls. Discussion This study provides evidence regarding three claims commonly made about the effects of high school exit exams: First, some claim that, by “raising the bar” for graduation, exit exams will motivate low‐ achieving students to work harder in school, and schools to improve their instruction for low‐ achieving students. As a result, it is argued, the academic skills of low‐achieving students will improve as a result of the exit exam. This study finds no evidence that the CASHEE has induced improvements in 11th grade academic achievement. Second, some claim that high school exit exams will induce some low‐achieving students (discouraged by the belief that they will not be able to pass the exam) to drop out of school prior to completing high school. Others argue that, by focusing curriculum and instruction on low‐level basic skills, exit exams will discourage even students who pass the exam from remaining in school. Although we do find some evidence suggesting that low‐achieving students in general left school prior to 12th grade at higher rates when the CAHSEE policy was in effect than before it was in effect, we cannot be sure this is attributable to the CAHSEE requirement. Moreover, we find no evidence that low‐achieving students who fail the exam are induced to leave school early as a result of failing. Third, some claim that high school exit exams are important because they make clear (to

employers, for example) that students with a high school diploma have achieved some basic level of academic skills. Therefore it may be acceptable for exit exam requirements to reduce graduation rates, at least in the short term, because this implies that the exam is effective in identifying students with the minimum required set of academic skills. In this study, we find that minority students and girls are less likely to pass the CAHSEE than are white and male students with the same level of academic skills as measured by 8th, 9th, and 10th grade California Standards Tests in math and English Language Arts, a pattern that is consistent with the predications of stereotype threat theory. This implies that the diploma does not convey the same signal to employers for minority students and girls as it does for white students and boys. In effect, given the presence of negative societal stereotypes about the academic skills of minority students and girls, the high stakes test does not provide a neutral signal to employers about the true academic skills of all students. Thus, this study validates none of the three common claims about high school exit exams: the CAHSEE has not improved achievement; nor has it led to early dropout; nor does it provide an unbiased signal to employers about students’ skills.

Percentage of Bottom Quartile Students Graduating, by CAHSEE Requirement
70 Class of 2005 (Pre‐CAHSEE) Classes of 2006 & 2007 (Post‐CAHSEE) 60 53 50 44 43 40 35 46 44 45 37 34 31 30 33 62 56

Percentage Graduating

50

20

10

0

All Students

|

White

Black

Hispanic

Asian

|

Male

Female

Source: Reardon et al (2009) Effects of the California High School Exit Exam on Student Persistence, Achievement, and Graduation

Note: figure shows estimated graduation rates for students who scored in the bottom quartile of the statewide 10th‐grade English Language Arts (ELA) California Standards Test (CST), adjusted for race/ethnicity, gender, free/reduced‐price lunch eligibility, English Learner Status, and 9th‐ and 10th‐grade ELA CST scores. Special education students are excluded from the analyses because they were not subject to the CAHSEE requirement.