Canadian Journal of Behavioural Science 2011, Vol. 43, No. 3, 195–202

© 2011 Canadian Psychological Association 0008-400X/11/$12.00 DOI: 10.1037/a0022697

The Impact of Schooling on Academic Achievement: Evidence From Homeschooled and Traditionally Schooled Students
Sandra Martin-Chang
Concordia University and Mount Allison University

Odette N. Gould and Reanne E. Meuse
Mount Allison University

Although homeschooling is growing in prevalence, its educational outcomes remain unclear. The present study compared the academic achievements of homeschooled children with children attending traditional public school. When the homeschooled group was divided into those who were taught from organized lesson plans (structured homeschoolers) and those who were not (unstructured homeschoolers), the data showed that structured homeschooled children achieved higher standardized scores compared with children attending public school. Exploratory analyses also suggest that the unstructured homeschoolers are achieving the lowest standardized scores across the 3 groups. Keywords: curriculum, education, homeschooling, reading, unschooling

The number of children being homeschooled in North America is growing at an unprecedented rate (Arai, 2000; Barwegen, Falciani, Putman, Reamer, & Stair, 2004; Brady, 2005; Cai, Reeve, & Robinson, 2002; Jones & Gloeckner, 2004a, 2004b; Ray, 2010). In the United States, best estimates place the homeschooling population above 1.5 million children (National Center for Education Statistics, 2008). Similarly, it has been suggested that homeschoolers account for almost 1% of all Canadian children (Davies & Aurini, 2003); however, it is likely that these estimates are too conservative (Basham, Merrifield, & Hepburn, 2007). In addition to religious convictions, dissatisfaction with the public school system is the most prominent factor in the decision to homeschool (Home School Legal Defence Association of Canada, 2006; Mayberry & Knowles, 1989; Van Galen, 1987; Welner, 2002). However, empirical research has not confirmed the pervasive belief that homeschooled elementary children are advancing beyond their public school peers (Basham, 2001; see also Cogan, 2010, for college students). Recent attempts have been made to understand both the demographic characteristics and the ideological underpinnings of the homeschooling community (Collum & Mitchell, 2005; Klein & Poplin, 2008; Merry & Howell, 2009; Van Galen, 1987); yet, at present, very few independent (i.e., nonpartisan) studies have focused on the academic achievements associated with home education. The aim of the current investigation was to

This article was published Online First May 30, 2011. Sandra Martin-Chang, Department of Education, Concordia University, Montreal, Quebec, Canada, and Psychology Department, Mount Allison University, Sackville, New Brunswick, Canada; Odette N. Gould and Reanne E. Meuse, Psychology Department, Mount Allison University. We thank Amanda Barrister and Jessica Chapman for their help in data collection. We gratefully acknowledge financial support from the McCain Fellowship Foundation to Sandra Martin-Chang and from Natural Sciences and Engineering Research Council of Canada to Odette N. Gould. Correspondence concerning this article should be addressed to Sandra Martin-Chang, Department of Education LB-579, 1455, de Maisonneuve Boulevard W., Montreal, QC, Canada H3G 1M8. E-mail: smartinc@ education.concordia.ca 195

evaluate the efficacy of home-based education as measured by standardized achievement tests in a small Canadian sample of homeschooled and public school children. The notion that homeschooling is superior to traditional methods of education can be traced back to a small number of highly touted reports funded by the Home School Legal Defence Association (see Ray & Wartes, 1991, for review). Rudner (1999) carried out one of the most influential reports. Rudner’s data originated from an educational testing company that provided standardized testing to homeschooling families. Prior to receiving the final results, parents of over 20,000 children agreed to release the test scores for evaluation. From this large sample, Rudner reported that the homeschooled children were functioning at a higher level than traditionally schooled children in every grade and over all the curricular areas tested (reading, language arts, mathematics, social studies, science, and information services). On first blush, the data reported by Rudner (1999) seem convincing; however, key methodological flaws make interpreting the results from this study problematic. First, only families who enlisted the services of a privatized educational testing company were approached to participate. In the year the data were collected, it was estimated that between 70,000 and 120,000 children were being homeschooled in the United States, yet only 39,607 used this particular service (Rudner, 1999). Therefore, the families pursuing standardized testing may have differed from the larger homeschooling population in terms of either educational priorities or financial status. The fact that the mean income of the families in this study was higher than the national norms supports the latter interpretation. In addition, only 52% of the families who used the testing service agreed to participate in the study; thus, the parents who were most confident in their children’s abilities may have made up the majority of the sample. In any study of this nature, it is difficult to rebut many of the issues surrounding self-selection. Many homeschooling families are not registered with the local governments or school boards (Arai, 2000; Lines, 1991), and this effectively rules out the possibility of randomized sampling. Nonetheless, the effects of selfselection in Rudner’s investigation (1999) may have been lessened

196

MARTIN-CHANG, GOULD, AND MEUSE

if a group of comparable children in public schools had been tested to act as a comparison group. Unfortunately, this was not the case; Rudner compared the scores of a specially selected group of homeschooled children to test norms established with a general population of public school children. A further concern with Rudner’s (1999) study involves the standardization of the testing situation. Rudner compared the achievement scores of the homeschooled children with the norms obtained from large representative samples of American children. However, the method of administration may have differed systematically between the groups. In the homeschooled population, the child’s parent often administered the tests (see also Kunzman, 2009). In contrast, when the norms were established, the children were tested by professionals. To equate adherence to the standard testing procedures and the level of comfort and support given to the children, it would have been optimal if all students had worked with trained test administrators. Indeed, other research has shown that when the tests are given by a trained assistant, the scores of homeschooled children and public school students do not differ. For example, as part of an unpublished doctoral dissertation, Delahooke (1986, as cited in Ray & Wartes, 1991) administered the Reading, Spelling, and Arithmetic subtests of the Wide Range Achievement Test—Revised to children enrolled in a private school and children receiving homeschooling. Children from both groups were tested in their homes by a trained experimenter. Under these circumstances, no differences were reported between the performance of the children who were attending private school and those who were educated at home. Despite the limitations of Rudner’s (1999) study, one finding in particular suggested that homeschooling may offer advantages above and beyond those experienced in traditional public school. Rudner compared the academic achievement of children who had been exclusively homeschooled with children who had first attended traditional school before switching to a homeschooling program. Here, the homeschooled children who had started their education in public school could act as a comparison group for the children who had been exclusively homeschooled. Although it is possible that the decision to homeschool may have resulted from academic difficulties encountered in school, the use of a comparison group nevertheless minimised several of the methodological issues discussed above. Therefore, it is interesting that the data showed that children who had been exclusively homeschooled had higher overall achievement scores than children who had first attended traditional public school. A large-scale study has recently replicated and extended Rudner’s (1999) original investigation. Ray (2010) collected data from 11,729 participating homeschoolers across America, Guam, and Puerto Rico. His findings fit nicely with previous reports showing that the scores of homeschooled children were higher than the standardized norms across all subtests. In addition, Ray’s data set revealed many interesting correlations between academic achievement and the students’ home environment. Homeschoolers who obtained the highest scores came from high-income families with university-educated parents, who invested at least $600 each year (per child) on educational materials. Student success was also associated with higher amounts of overall “structure” in the homeschooling program and greater amounts of time engaged in formal instruction (e.g., lessons).

However, Ray’s (2010) study was subject to many of the same limitations as Rudner’s (1999). Specifically, the population comprised only those homeschoolers who used the services of academic testing companies. In addition, many of these parents earned higher incomes than the general population. The sample was also exclusively self-selected; it was estimated that the questionnaire return rate ranged between 11% and 19%, and the participating parents may have differed systematically from those who chose not to participate in the study. Finally, the homeschooling parents proctored many of the tests, and Ray failed to include a public school group for comparison. (The results of the homeschooling sample were compared with the 50th percentile from the standardized norms.) Barwegen et al. (2004) have recently narrowed the focus to ask why the scores of homeschooled children might differ from those in public school. Following the recent trend of examining the positive impact parental involvement plays on children’s educational success (Feuerstein, 2001; Heymann & Earle, 2000; Hill & Craft, 2003; Hill & Taylor, 2004; Lee & Bowen, 2006), Barwegen et al. (2004) proposed that the elevated test scores of homeschooled children in previous research may have reflected greater parental involvement rather than general educational superiority. To examine this possibility, they circulated questionnaires measuring perceived parental involvement to 127 public high school seniors. Results showed that students with high perceived parental involvement (e.g., having high expectations, input into course selection, etc.) had significantly higher standardized scores than students with low perceived parental involvement. In addition, the scores of traditionally schooled teenagers with highly involved parents did not differ significantly from those reported from homeschooled students. The conclusions drawn by Barwegen et al. (2004) are intriguing. However, these authors were unable to compare the amount of perceived parental support between the homeschooled and public school groups because they did not administer any questionnaires to children who were homeschooled. Therefore, it is not possible to make direct comparisons between the two groups of students. Furthermore, they did not administer the tests of academic achievement themselves. Like Rudner (1999) and Ray (2010), Barwegen and colleagues used data obtained from private companies. Thus, the self-selective nature of the homeschooled sample and the uniformity of the testing situations remain problematic in the Barwegen et al. work. In light of the paucity of empirical investigation into the effects of home education, the purpose of the current study was to compare the academic achievements of homeschooled children with a similar group of children attending public school. This study is unique for several reasons. First, unlike previous work in this area (e.g., Cogan, 2010; Ray, 2002, 2010; Rudner, 1999), we did not rely on self-reported measures or data gathered by a third party; rather, each child in the present study was administered standardized tests under controlled conditions by a trained experimenter. Second, we included a carefully selected comparison group. Unlike our predecessors who focused mainly on either homeschooled students (Collum & Mitchell, 2005; Ray, 2002, 2010; Rudner, 1999) or traditionally schooled students (Barwegen et al., 2004), we worked with both children who were homeschooled and children attending public school to allow for direct comparisons between the two groups. Finally, the present study was conducted by

HOMESCHOOLING

197

an independent research body that has no ties to homeschooling organisations.

Table 1 Sample Descriptors as a Function of Schooling Group
Public school Homeschooled children children (n 37) (n 37) 4 2 20 6 5 2 0 3 12 4 7 6 3 5 4 24 4 0 0 0 4 15 11 3 3 1

Method
Demographic variable

Participants
Families were invited to participate in the study through announcements posted in the community, sent by e-mail, and broadcast on the radio. Interested parties contacted the researchers to schedule home appointments. Potential candidates were screened during telephone interviews to ensure that the children in the homeschooled group had not attended public school (from Grade 1 onward) and that children in the comparison group had not been homeschooled. Five families were excluded from the study because they were combining elements of both types of education (e.g., unschooling every Tuesday and Thursday, but sending their children to school during the rest of the week). We recruited children from two Atlantic provinces (Nova Scotia and New Brunswick). In total, 74 children (37 homeschooled and 37 public school) between the ages of 5 and 10 years participated in this study. A matching procedure was used so that each homeschooled child was paired with a similar-age public school child living in the same geographical area. Like the children receiving homeschooling, the children attending public school came from a very heterogeneous group; they were selected from a number of different school boards that adhered to widely different curricula. The homeschooled group consisted of 20 boys and 17 girls with a mean age of 7 years 11 months (range 5 years 5 months to 10 years 8 months). Likewise, the public school group contained 21 boys and 16 girls with a mean age of 7 years 11 months (range 5 years 7 months to 10 years 6 months). The mean difference in age between the two groups was 2 months (range 0 to 5 months ). As would be expected, a t test showed that the groups did not differ in terms of age, t(72) 0.012, p .99. All of the mothers in the homeschooled group and all but one of the mothers in the public school group were married or living in committed relationships. Children from the homeschooled group came from families with an average of 2.8 children (SD 1.1, range 1– 6) and children from the public school group came from families with an average of 2.25 children (SD 0.7, range 1– 4). Data regarding maternal education and family income are listed in Table 1. As shown in Table 1, the majority of children in both groups came from homes where the mother had obtained a college diploma or university degree (homeschooled 65%, public school 54%). However, children in the public school sample were more likely to have mothers who had completed postgraduate training (homeschooled 11%, public school 30%). Mothers were also asked to indicate their family’s yearly income by selecting the appropriate category from a list of income ranges. All but two of the mothers reported family income, and these data are listed in Table 1. The mode income for both groups was between $20,000 and $40,000 (for the homeschooled group, 41% chose this category; for the public school group, 32% chose this category). However, the median income for the two groups was slightly different. For children from the public school group, the median category was $40,000 to $60,000; for the homeschooled group, the median income category was $20,000 to

Mother’s highest educational attainment High school Some university or college University degree or college diploma Professional or master’s degree PhD Family income ($) Unreported 10,000 10,000–20,000 20,000–40,000 40,000–60,000 60,000–80,000 80,000–100,000 100,000

$40,000. The difference in income may have reflected the fact that 62.2% of the homeschooled children had mothers who reported leaving the paid workforce after the birth of their children, compared with only 16.2% of the children in the public school group.

Homeschooling Subgroups
Although we had originally planned to carry out a simple comparison of the academic achievements of homeschooled and traditionally schooled children, speaking with the participating families made it clear that our homeschooled sample comprised two distinct subgroups. Although the reasons to homeschool and the methods used by individual families are highly variable (Davies & Aurini, 2003; Kunzman, 2009; Winstanley, 2009), the parents could be divided by how much they identified with the role of “teacher.” The majority of the homeschooling parents reported that they “often” or “always” used premade curricula or structured lesson plans to teach their children. This group adhered loosely to a “school-at-home” methodology (Taylor-Hough, 2010), where the parents/teachers set out clear educational goals for their children and offered structured lessons in the form of either purchased curricula or self-made lesson plans (often some combination of both). The main defining characteristic of this subgroup was that the parents viewed themselves as important contributors to their children’s education. The parents of a sizable minority of children in the homeschooled sample answered “rarely” or “never” to using premade curricula and structured lesson plans. These parents identified more with the pedagogical view that education is gained via the natural consequences of the child’s day-to-day activities (TaylorHough, 2010). For example, “. . . having classical CDs playing in the background gets listed as ‘fine arts,’ watching an episode of Little House on the Prairie counts as history, and figuring out how much they can buy with $2.00 at the gift shop qualifies as the day’s math lesson” (Kunzman, 2009, p. 320). To preserve this division within the homeschooling population, we used the mothers’ responses to the questions regarding curriculum and lesson plans to create “structured” and “unstructured”

198

MARTIN-CHANG, GOULD, AND MEUSE

homeschooling subgroups (see also Ray, 2010). The resulting groups contained 25 children who were taught in a structured environment and 12 children who were not. Although the unstructured homeschooled group was small, we report the group characteristics below for descriptive purposes. The structured homeschooled group contained 13 boys and 12 girls, with a mean age of 7 years 10 months (SD 1 year 8 months; range 5 years 5 months to 10 years 6 months). Overall, 84% of these children had a mother who had attended an institution of higher education (college or university) and came from families with a mean of 3.12 children (SD 1.39). The mode and median income for the families of children in the structured homeschooled group were in the $40,000 to $60,000 range. The unstructured group contained five girls and seven boys, with a mean age of 8 years 1 month (SD 1 year 6 months, range 5 years 6 months to 10 years 8 months). Overall, 91.7% of these children had a mother who had attended an institution of higher education and came from families with a mean of 2.75 children (SD 0.75). The mode and median income for families in the unstructured homeschooled group were in the $20,000 to $40,000 range.

passages. The Word Attack test is a measure of pure decoding. It contains 30 nonwords that children are asked to read aloud according to conventional spelling–sound conversion rules. The Science test taps into scientific knowledge by asking 49 questions pertaining to biology, physics, and chemistry. Likewise, the Social Science test contains 49 questions regarding vocations, geography, history, and politics; and the Humanities test contains 45 items regarding literature, music, art, and popular culture. Finally, the Calculation test contains 58 items in ascending order from simple number identification to complex algebra.

Procedure
The majority of testing took place in the children’s homes (n 71), although in three cases, an alternative location was requested by the participants (home of a friend n 2; mother’s private office n 1). All of the tasks were administered during the last half of the academic year (February to June) and took place during one 45-min session. The testing sessions for homeschooled children and public school children were distributed equally across this time period. The mother– child dyads were seated in two adjoining rooms of the home (e.g., living room and dining room, or kitchen and living room). In families with multiple young children, two researchers administered the tasks and a third research assistant occupied the siblings with quiet games (e.g., colouring, cards). Precautions were taken to ensure that the child was near enough to the other members of the family to establish a high level of security, but separated enough to create a private, distraction-free work area. The children received the subtests in the following order: LetterWord Identification, Passage Comprehension, Calculation, Science, Social Science, Humanities, and Word Attack. The mothers filled out the demographic questionnaire while their children were being tested.

Materials
Demographic questionnaire. Mothers participating in the study were asked to fill out a demographic questionnaire pertaining to family structure (marital status, number of children in the family, etc.), parental education, parental employment, and household income. Two additional questions were also given to the homeschooling parents regarding curriculum use. The mothers were asked, “How often do you use a premade curriculum?” and “How often do you use some form of structured lesson plans?” The mothers responded to the last two questions by selecting one of four choices on a scale ranging from never to very often. The groups formed from the ratings on this variable had strong face validity in terms of both the teaching environment in the home and the attitudes expressed by the parents. Academic achievement. As in previous studies (e.g., Ray, 2002; Rudner, 1999), “academic achievement is considered to be the formal demonstration of learning (including knowledge, understanding, and thinking skills) attained by a student as measured by standardized academic achievement tests” (Ray, 2010, p. 5). In the present investigation, all of the subtests were taken from the Woodcock–Johnson Test of Achievement A Revised (Woodcock & Johnson, 1989) and were administered by the principal investigator. The Woodcock–Johnson is a frequently used standardized test of educational achievement (Baker, Mackler, & Sonnenschein, 2001). It is an untimed test that requires children to answer questions of increasing difficulty until six consecutive items are missed. The seven subtests from the Woodcock–Johnson were selected to measure aptitudes from a wide breadth of areas. The Letter– Word Identification test is a measure of real word reading. It contains 57 questions that progress from naming easy to very difficult items. The Passage Comprehension test uses a cloze procedure to measure comprehension. Children are presented with 43 passages and asked to provide an appropriate word that is missing from the text. In accordance with the standardized instructions, the children were not given any assistance when reading the

Results
Ideally, we would have obtained large enough samples to carry out three-way comparisons among structured homeschooled children, unstructured homeschooled children, and children attending traditional public school. However, we were able to gain access to only 12 participants who fell into the unstructured category; consequently, our main analyses contain only the children in the structured homeschooled (n 25) and public school (n 37) groups. Nevertheless, because the current investigation marks the first attempt at testing even a small sample of children receiving unstructured homeschooling, we have included the mean scores for the unstructured group along with some exploratory analyses at the conclusion of the Results section. Our primary goal was to determine whether the standardized scores of children who attend public school differ from children who are taught with a structured homeschool program. However, the age spread of our participants sometimes rendered the raw scores difficult to interpret. For example, if it was discovered that a child was reading at a Grade 3 level, the first question that was asked was whether the child was enrolled in Grade 1, 3, or 5; in other words, was the child achieving above, equal to, or below grade level? To rectify this problem, we transformed the raw scores in each of the seven subtests into difference scores by

HOMESCHOOLING

199

subtracting the child’s predicted grade level from his or her actual grade level (both predicted and actual grade levels were derived from the Woodcock–Johnson scales). Therefore, a positive score indicated the number of years a child was functioning above grade level, whereas a negative difference score indicated that the child was not meeting grade level standards. As shown in Table 2, the children who received structured homeschooling were superior to the children enrolled in public school across all seven subtests. To gain a broad perspective of the level of standardized achievement in each group, we conducted a multivariate analysis of variance (MANOVA) that included the scores from all seven Woodcock–Johnson subtests. The MANOVA was deemed appropriate because multiple dependent measures were providing indicators of educational achievement (Grimm & Yarnold, 2000). Thus, all seven subtests were used as dependent variables, and schooling group (public school and structured homeschool) was the independent variable. The MANOVA confirmed that the general level of standardized achievement between the two groups differed significantly in favour of homeschooling, Hotelling’s Trace .362, F(7, 54) 2.79, p .015, partial 2 .27. When follow-up t tests were carried out using p .05 as the criterion, all variables except the comprehension measure showed a significant superiority for the structured homeschooled group. When the Bonferonni correction was applied (i.e., using p .007 as the criterion), the Letter–Word, Word Attack, and Social Science variables were significantly different between the two groups despite the relatively small samples involved. The effect sizes are presented in Table 2. Using Cohen’s convention (Cohen, 1988) that a medium effect size is approximately 2 .06 and a large effect is 2 .14, all the variables showed a medium or strong effect. The MANOVA described above was recalculated to account for the slight variations in income level across the groups. The income level covariate was not significant at either the multivariate level or with any of the seven dependent variables (F 1 for all but one variable). The adjusted means and the follow-up t tests showed the same pattern of results as described above (effect sizes differed by .02 at most). Thus, even when the groups were equated on income, the structured homeschooled group had overall superior standardized scores compared with the public school children. Second, the

MANOVA was recalculated using mother’s educational attainment as a covariate. Again, the pattern of findings was not affected. In conclusion, when comparing the test scores of the children attending public school and children receiving structured homeschooling, it becomes clear that the latter group has higher scores across a variety of academic areas. Moreover, there is no evidence that this difference is simply due to the family’s income or the mother’s educational attainment.

Exploratory Analyses
The existence of a distinct group of children whose parents did not use lesson plans or prepared curricula (unstructured homeschooled children) led to a series of exploratory analyses. Owing to the small number of individuals in the unstructured homeschooled group, we conducted simple t tests comparisons on each of the Woodcock–Johnson subtests separately. In the first set of analyses, the unstructured homeschoolers were compared (n 12) with the children attending public school (n 37). As depicted in Figure 1, the children in public school had a higher mean grade level for all seven measures compared with the unstructured homeschoolers (mean differences ranged from 0.64 for the Calculation test to 1.67 for the Reading Comprehension test). Given the small sample size, none of the comparisons were statistically significant using the Bonferonni correction on criterion (p .007). However, it should be noted that the effect sizes for four of the seven variables (range .06 to .13) are considered medium to large using the Cohen (1988) convention: Letter–Word, t(47) 2.26, p .03, 2 .10; Comprehension, t(47) 2.62, p .01, 2 .13; Social Science, t(47) 1.91, p .06, 2 .07; Science, t(47) 1.80, p .08, 2 .06. Neither family income nor mother’s educational attainment was a significant covariate, and the presence of these covariates did not affect the pattern of results. In the second set of analyses, the unstructured homeschooled children (n 12) were compared with the homeschooled children taught from a structured curriculum (n 25). As shown in Figure 1, children in the unstructured group had lower scores on all seven academic measures compared with the structured homeschooled group. The mean differences between the two groups were pronounced, ranging from 1.32 grade levels for the Calculation test to 4.20 grade levels for the Word Attack test. The t tests comparing the two groups were significant (using p .007) for five of the seven measures, and effect sizes were large in all cases: Letter–Word, t(35) 4.13, p .001, 2 .33; Comprehension, t(35) 4.15 p .001, 2 .33; Word Attack, t(35) 2.56 p .015, 2 .16; Science, t(35) 3.07, p .004, 2 .21; Social Science t(35) 4.22, p .001, 2 .34; Humanities, t(35) 2.43, p .02, 2 .15; Calculation, t(35) 3.88, p .001, 2 .30. When these analyses were repeated with family income as a covariate, it was found that income was significant for two of the measures (Reading Comprehension and Social Science). However, the independent variable remained significant when the covariate was applied, with structured homeschooled children having higher grade levels than the unstructured group for all seven dependent variables. Mother’s education was not a significant covariate and did not affect the impact of schooling style on children’s grade level. In conclusion, our exploratory analyses suggest that the unstructured homeschooled children generally score below their expected

Table 2 Follow-Up t Tests Comparing Children’s Difference Scores (as Shown in Grade Levels) in the Public School and the Structured Homeschooled Groups
Public school Test Letter–Word Comprehension Word Attack Science Social Science Humanities Calculation M 1.38 1.58 1.61 1.37 0.59 0.005 0.27 SD 1.89 2.04 3.80 1.75 1.32 1.58 1.27 M 3.11 2.56 4.89 2.61 1.59 0.99 0.94 Structured homeschooled SD 2.36 1.95 4.91 2.31 1.24 2.14 0.92 t(60)a 3.20 1.86 2.90a 2.41 2.97 2.11 2.29 p .002 .068 .006 .019 .004 .039 .026
2

.15 .06 .13 .09 .13 .07 .08

a t test for equal variances not assumed is reported. Levene’s test for equality of variances was nonsignificant for all other variables.

200
6

MARTIN-CHANG, GOULD, AND MEUSE

Structured Homeschool, n = 25

5 Difference Scores in Grade Levels

Public School, n = 37 Unstructured Homeschool, n = 12

4

3

2

1

Comprehension

Word Attack

Social Science

Humanities

-1

Letter Word

Task

Figure 1.

Academic achievement as a function of group and Woodcock–Johnson subtest.

grade level on the standardized test, and that even with this small sample, performance differences are relatively substantial. What is more, our exploratory analyses strongly suggest that the children who are being taught at home in a structured environment score significantly higher than the children receiving unstructured homeschooling. Furthermore, it does not appear that the differences between groups are simply due to either the family’s income or the mother’s educational attainment.

Discussion
It has been stated that “if parents choose to homeschool because they are looking for increased academic achievement as measured by standardized tests, the research shows that any method of homeschooling will most likely raise their child’s test scores above those of their traditionally schooled counterparts” (Taylor-Hough, 2010, p. 6). The data presented here provide evidence that both support and modify this claim. Our results suggest that structured homeschooling may offer opportunities for academic performance beyond those typically experienced in public school. Moreover, the design used in the current study suggests that the benefits of structured homeschooling cannot be accounted for by differences in yearly family income or maternal education. Although we made efforts to ensure that the two groups were drawn from similar populations (as evidenced by the fact that the mode was the same for both variables), mothers’ education and median income were slightly higher for the public school group. It should be noted, however, that this would have been expected to bias the study against finding a homeschool advantage. This was clearly not the case. The results showed that the structured group outscored the traditionally schooled children on the composite of the Woodcock–Johnson subtests.

The findings presented here focus on the patterns of relative differences found between the groups. The fact that the homeschooling community is relatively ill defined renders random sampling of this population a near impossibility. Thus, it is likely that the parents who felt the most passionately about either the importance of education or about their child’s abilities were the ones most comfortable in volunteering to participate. If this were the case, the scores of the children in the structured homeschooled group may be somewhat inflated. And yet, there is every reason to believe that the relative differences between the groups might be accurate because, in the present study, all three groups were self-selected. Indeed, the fact that the students in the public school group achieved above grade level performance on many of the Woodcock–Johnson subtests supports the notion that they also might have been made of an elite group of children; nevertheless, the children who were homeschooled with a structured curriculum outperformed the public school children on the test. This finding also underscores the importance of including carefully selected comparison groups in educational studies because while the disparity between the structured homeschooled group and the public school group was large, it would have appeared much larger if the structured homeschooled children had been compared with national norms as has been standard practice (e.g., Ray, 2010; Rudner, 1999). The evidence presented here is in line with the assumption that homeschooling offers benefits over and above those experienced in public school. This advantage may be explained by several factors including smaller class sizes, more individualized instruction, or more academic time spent on core subjects such as reading and writing (Duvall, Delquadri, & Ward, 2004; Duvall, Ward, Delquadri, & Greenwood, 1997). Ongoing research is currently

Calculation

Science

0

HOMESCHOOLING

201

exploring these possibilities. However, our results also show that the homeschooling community comprises subpopulations and suggest that the clear advantage of homeschooling may be limited to situations where parents create structured environments, at least in terms of performance on academic tests. With regard to the unstructured homeschooled sample, all of the mothers indicated that they “rarely” or “never” used structured curricula or premade lesson plans and the mothers of nine children also described themselves as “unschoolers” when discussing their approach to education. The term unschooling embodies the notion of self-directed learning on the part of the child—free from teachers, textbooks, and formal assessment (Holt, 1964). Although the decision to homeschool and the decision to unschool involve parents exercising their rights to assume the primary responsibility of educating their children, the two groups deviate radically in their views of the parent-as-teacher and in the use of preset curricula. It has been estimated that approximately 150,000 children in the United States were being unschooled in 2005 (Tamura & Gutierrez, 2006). However, the academic effects of this practice are largely unknown. Our data suggest that this group is being outperformed on academic tests both by the traditionally schooled and the structured homeschooled groups. This pattern of results fits nicely with Ray’s (2010) report, where three variables of interest were positively associated with student achievement on academic tests: greater structure in the program, more funds spent on educational materials (e.g., textbooks, tutoring), and more time spent in “structured learning time” (defined as “time during which the child is engaged in learning activities planned by the parent; it is a time during which the child is not free to do whatever he or she chooses,” Ray, 2010, p. 19). It is important to note that Ray also found that students enrolled in a full-service curriculum did not perform any differently from those who were not. This concurs with the view that it is not critical whether the materials are purchased or self-made, the pivotal factor seems to be whether the child is mentored by a knowledgeable “teacher” in tasks that specifically target culturally important skills (including activities such as reading and arithmetic). Schools play several important roles, including socializing future citizens and fostering peer relations between children (Barakett & Cleghorn, 2008). However, the main focus of the current investigation was to compare the scores of children schooled at home versus those children attending public school on a standardized test of academic achievement. Although the evidence provided here is preliminary, it suggests that structured homeschooling may advance the development of academic skills (as measured by standardized tests) beyond what is experienced by attending traditional public school. The fact that the public school children were achieving above grade level expectations on many of the Woodcock–Johnson subtests suggests that this discrepancy did not stem from the poor performance of the public school children but rather resulted from accelerated progress in the children receiving structured home-based education. The same cannot be said for the children whose mothers “rarely” or “never” used structured curricula or lesson plans; the unstructured homeschoolers in this sample achieved the lowest scores throughout testing and fell below grade level in four of seven subject areas. This raises the question of whether a similar pattern would be observed with larger sample sizes, and if so, whether the children receiving

unstructured homeschooling would eventually catch up, or even surpass, their peers given ample time. This highlights two limitations of the current investigation that should be addressed in future work. First, obtaining access to an adequately sized sample is an obstacle when studying homeschooling. The homeschooling community in general, and the unstructured community in particular, tend to be relatively self-contained. As one self-identified unschooling mother explained, “I think unschoolers by definition will be less inclined to want to participate in an education study.”1 On a related issue, when discussing the degree of structure implemented in homeschooling, other reports (e.g., Ray, 2010) used a 7-point Likert scale ranging from very unstructured to very structured. In future studies, it would be interesting to investigate the relationship between level of structure and academic performance, using, for example, a correlational approach. However, the small sample used here did not allow this more in-depth approach. If at any point in time homeschooling becomes more regulated, or if more homeschooling families choose to register with the local public authority, randomly inviting a larger sample of families to participate and obtaining a more sensitive measure of “structure” would be optimal. In summary, the increasing popularity of homeschooling is at odds with the dearth of scientific research being conducted in this area. As argued by Isenberg (2007), “Despite its size, scarce data on homeschooling have impaired our understanding of even the most basic questions” (p. 387). Practical restraints such as the heterogeneity of the population and difficulties in obtaining adequate sample sizes make homeschooling a challenging field of study. Nevertheless, further inquiry is required if parents are to make informed decisions regarding the education of their children. Moreover, identifying the best practices associated with different types of education may facilitate teaching in both traditional and homeschool settings. As such, we hope that our findings act as a catalyst for further investigation into the benefits and limitations associated with different types of home-based education.

1

Quoted with permission from parent.

Resume ´ ´
Quoique l’enseignement a domicile par les parents soit de plus en ` plus repandu, les resultats educationnels de cette approche restent ´ ´ ´ incertains. Cette etude compare l’acquisition des connaissances ´ scolaires d’eleves dans des programmes d’enseignement a domi´` ` cile avec celle d’eleves inscrits a l’ecole publique conventionnelle. ´` ` ´ Quand on a divise les eleves recevant l’enseignement a domicile ´ ´` ` en deux groupes : ceux dont l’enseignement etait base sur des ´ ´ plans de lecon methodiques (enseignement a domicile method¸ ´ ` ´ ique) et ceux qui ne suivaient pas de plans methodiques (ensei´ gnement a domicile non methodique), les donnees ont revele que ` ´ ´ ´ ´´ les eleves recevant l’enseignement a domicile methodique avaient ´` ` ´ des notes standardisees plus elevees que les eleves de l’ecole ´ ´ ´ ´` ´ publique. Des analyses exploratoires donnent a penser que les ` eleves recevant un enseignement a domicile non methodique ont ´` ` ´ les notes standardisees les moins elevees des 3 groupes. ´ ´ ´ Mots-cles : le programme d’etudes, l’education, homeschooling, la ´ ´ ´ lecture, le non enseignement

202 References

MARTIN-CHANG, GOULD, AND MEUSE Isenberg, E. (2007). What have we learned about homeschooling? Peabody Journal of Education, 82, 327– 409. Jones, P., & Gloeckner, G. (2004a). A study of admission officers’ perceptions and attitudes toward homeschool students. Journal of College Admission, 185, 12–21. Jones, P., & Gloeckner, G. (2004b). A study of home school graduates and traditional school graduates. Journal of College Admission, 183, 17–20. Klein, C., & Poplin, M. (2008). Families home schooling in a virtual charter schooling system. Marriage and Family Review, 43, 369 –395. Kunzman, R. (2009). Understanding homeschooling: A better approach to regularization. Theory and Research in Education, 7, 311–330. Lee, J., & Bowen, N. (2006). Parent involvement, cultural capital, and the achievement gap among elementary school children. American Educational Research Journal, 43, 193–218. Lines, P. (1991). Home instruction: The size and growth of the movement. In J. Van Galen & M. Pitman (Eds.), Home schooling: Political, historical, and pedagogical perspectives (pp. 9 – 41). Norwood, NJ: Ablex Publishing. Mayberry, M., & Knowles, G. (1989). Family unity objectives of parents who teach their children: Ideological and pedagogical orientations to home schooling. The Urban Review, 21, 209 –225. Merry, M. S., & Howell, C. (2009). Can intimacy justify home education? Theory and Research in Education, 7, 363–381. National Center for Education Statistics. (2008). 1.5 million homeschooled students in the United States in 2007. Washington, DC: U.S. Department of Education. Ray, B. D. (2002). Customization through homeschooling. Educational Leadership, 59, 50 –54. Ray, B. D. (2010). Academic achievement and demographic traits of homeschool students: A nationwide study. Academic Leadership: The Online Journal, 8, 1–31. Retrieved from http://www.academicleadership .org/emprical_research/ Ray, B., & Wartes, J. (1991). The academic achievement and affective development of homeschooled children. In J. Van Galen & M. Pitman (Eds.), Home schooling: Political, historical, and pedagogical perspectives (pp. 43– 61). Norwood, NJ: Ablex Publishing. Rudner, L. (1999). Scholastic achievement and demographic characteristics of home school students in 1998. Education Policy Analysis Archives, 7, 1–38. Tamura, T., & Gutierrez, T. (2006). No school, no books, no teacher’s dirty looks. CNN. Retrieved from http://ed.cnn.com/2006/US/01/27/ gutierrez .unschooing/index.html?iref story search Taylor-Hough, D. (2010). Are all homeschooling methods created equal? Retrieved from ERIC database. (ED510702) Van Galen, J. (1987). Explaining home education: Parents’ accounts of their decisions to teach their own children. The Urban Review, 19, 161–177. Welner, K. (2002). Exploring the democratic tensions within parents’ decisions to homeschool. New York: National Center for the Study of Privatization in Education, Columbia University. Winstanley, C. (2009). Too cool for school? Gifted children and homeschooling. Theory and Research in Education, 7, 347–362. Woodcock, R., & Johnson, B. (1989). Woodcock–Johnson tests of achievement: Form A. Chicago: Riverside Publishing.

Arai, B. (2000). Reasons for home schooling in Canada. Canadian Journal of Education, 25, 204 –217. Baker, L., Mackler, K., & Sonnenschein, S. (2001). Parents’ interactions with their first grade children during storybook reading and relations with subsequent home reading activity and reading achievement. Journal of School Psychology, 39, 415– 438. Barakett, J., & Cleghorn, A. (2008). Sociology of education. Toronto. Pearson. Barwegen, L., Falciani, N., Putnam, J., Reamer, M., & Stair, E. (2004). Academic achievement of homeschool and public school students and student perception of parent involvement. The School Community Journal, 14, 39 –58. Basham, P. (2001). Home schooling: From the extreme to the mainstream? Public Policy Sources, 51, 3–18. Basham, P., Merrifield, J., & Hepburn, C. (2007). Home schooling: From the extreme to the mainstream (2nd ed). In Studies in education policy (pp. 3–24). Vancouver, BC: The Fraser Institute. Brady, J. (2005). What does “back to school” mean for home schoolers? ABC News. Retrieved from abcnews.go.com/GMA/print?id 1059369 Cai, Y., Reeve, J., & Robinson, D. (2002). Home schooling and teaching style: Comparing the motivating styles of home schooling and public school teachers. Journal of Educational Psychology, 94, 372–380. Cogan, M. (2010). Exploring academic outcomes of homeschooled students. Journal of College Admission, 208, 18 –25. Cohen, J. (1988). Statistical power analyses for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Collum, E., & Mitchell, E. (2005). Home schooling as a social movement: Identifying the determinants of homeschoolers’ perceptions. Sociological Spectrum, 25, 273–305. Davies, S., & Aurini, J. (2003). Homeschooling and Canadian educational politics: Rights, pluralism and pedagogical individualism. Evaluation and Research in Education, 17, 63–73. Duvall, S., Delquadri, J., & Ward, D. (2004). A preliminary investigation of the effectiveness of homeschool instructional environments for students with attention-deficit/hyperactivity disorder. School Psychology Review, 33, 140 –158. Duvall, S., Ward, D., Delquadri, J., & Greenwood, C. (1997). An exploratory study of home school instructional environments and their effects on the basic skills of students with learning disabilities. Education and Treatment of Children, 20, 150 –172. Feuerstein, A. (2001). School characteristics and parent involvement: Influences on participation in children’s schools. The Journal of Educational Research, 94, 29 –39. Grimm, L., & Yarnold, P. (2000). Reading and understanding multivariate statistics. Washington, DC: American Psychological Association. Heymann, S. J., & Earle, A. (2000). Low-income parents: How do working conditions affect their opportunity to help school-age children at risk? American Educational Research Journal, 37, 833– 848. Hill, N., & Craft, S. (2003). Parent–school involvement and school performance: Mediated pathways among socioeconomically comparable African American and Euro-American families. Journal of Educational Psychology, 95, 74 – 83. Hill, N., & Taylor, L. (2004). Parental school involvement and children’s academic achievement. Current Directions in Psychological Science, 13, 161–164. Holt, J. (1964). How children fail. New York: Dell Publishing. Home School Legal Defence Association of Canada. (2006). Questions and answers. Retrieved from http://www.hslda.ca/qu.asp

Received July 9, 2010 Revision received December 20, 2010 Accepted December 21, 2010