A Highly Selective Sample (Ray 2003)
Click here to view this document as a pdf.
In 2003, the Home School Legal Defense Association (HSLDA) commissioned Dr. Brian Ray to conduct a study on the social, educational, and economic successes of homeschool graduates. The two publications resulting from this study—a 2003 summary of the results, entitled “Homeschooling Grows Up”, which was produced by HSLDA and is posted on their website; and a book entitled Home Educated and Now Adults, which was published in 2004 by Ray’s National Home Education Research Institute (NHERI)—are widely cited to support the claim that graduates of homeschooling are well-socialized and go on to lead successful lives.
The debate over exactly what ‘socialization’ entails has led some scholars to distinguish “between two general categories of socialization. The first involves learning how to interact effectively in group settings and broader society, understanding its rules of behavior and social customs. The second category involves navigating a range of social influences—parents, peers, local communities, broader society—in the formation of personal values, beliefs, and commitments” (Kunzman & Gaither 2013:19).
Ray’s study primarily addresses the former definition, finding that his participants—mostly white Protestant female college students whose families were highly involved in the homeschooling movement—had positive attitudes about homeschooling and life in general. These homeschooling graduates were fairly religious, involved in local community organizations, knowledgeable about current events, and active in politics. However, Ray’s survey does not address the latter definition of socialization, so we do not know if his participants were capable of making friends or were secure in their own identities. In addition, the participants’ youth and lack of exposure to life experiences makes it impossible to know how likely they were to have successful careers and families. Furthermore, Ray did not collect data on the homeschool graduates’ backgrounds (such as their parents’ income and education levels), so we cannot determine whether homeschooling played a role in their high life satisfaction or community involvement. Finally, because of the way participants were recruited, we cannot generalize from this study to the homeschool population as a whole.
We can thus only conclude from Ray’s study that some young graduates of homeschooling report high levels of life satisfaction and community involvement. We cannot rule out the possibility that this group has problems with socialization, or that they may have such problems in the future. We cannot assume that this group will go on to have successful careers and families. This study does not prove that all homeschooling graduates have high levels of life satisfaction and community involvement, nor does it prove that homeschool graduates are well-socialized and go on to lead successful lives.
In the following sections, I will give some background of Ray’s study and outline the major points made in “Homeschooling Grows Up” and Home Educated and Now Adults. Next, I will provide a critical analysis of Ray’s study. Finally, I will summarize the actual conclusions we can draw from his study.
Background of the study
The study was commissioned in 2003 by HSLDA (who also provided the primary funding) with the purpose of refuting opponents’ claims that homeschoolers were not well socialized. The back cover of Home Educated and Now Adults cites these concerns: “For decades, critics of homeschooling, and homeschoolers themselves, have been asking: ‘But how will [the home-educated] do in the real world of adulthood?’ and ‘What about socialization?’” (Ray 2004) The focus of the study was homeschooling graduates’ “demographics, educational and family backgrounds, attitudes toward their own home-education experiences, and successes in life. Success was evaluated with respect to civic, social, educational, employment, and worldview traits” (Ray 2004:1).
In order to determine the life success of homeschooling graduates, Ray conducted a study with both quantitative and qualitative portions. The quantitative portion consisted of a survey, which was posted online from February to March 2003 and was also available in a hard-copy format. It contained approximately 120 questions and generated 205 variables. In the qualitative portion, “[p]urposive stratified sampling was used to select 30 respondents for telephone interviews,” (Ray 2004:17) which each lasted about an hour. Ray quotes only sparingly from the qualitative interviews.
Participants in the survey were recruited through “[h]omeschool support organizations’ print and online publications and e-mail lists, contacts via word-of-mouth, and personal networks” (Ray 2004:17). Although Ray does not enumerate which homeschooling organizations he used to get the word out, it is likely that HSLDA, the National Home Education Network (NHEN), the National Homeschool Association (NHA), Utah Home Education Association, and NY Loving Education at Home were involved, as Ray’s survey asks whether respondents were members of these organizations. At least 58,000 emails were sent to known homeschooling families and 500 paper copies of the survey were distributed by mail to anyone who requested them. Ray acknowledges that this method of recruiting participants “[does not] necessarily result in a representative sample of home-educated adults” (Ray 2004:17).
The introduction to the survey exhorted participants to speak up about their homeschooling experiences. “You can ‘make your voice heard.’…How are the homeschooled doing in the real world? Is homeschooling a credible educational form, or not? Whether you like homeschooling or not, think you are a successful adult or not, we would really like you to help us answer these questions” (Ray 2004:18). The framing of this entreaty, though explicitly neutral, is implicitly intended to provoke homeschool activists to respond to an attack.
The back cover of Home Educated and Now Adults includes a quote from Bruce S. Cooper of the Fordham University Graduate School of Education which further supports the idea that the study was intended as a retaliation against homeschooling’s critics: “Grown-up homeschoolers understand politics, vote in larger percentages than the rest of us, and are contributing to the communal life in our society. Ray’s study disconfirms, then, what critics have long said: that without the institutionalized socialization of attending an organized school, kids would be limited in their futures, their education, their well-being, their civic engagement, and their happiness. Not true, for education begins, grows, and continues throughout life, at home.” (Ray 2004)
A total of 7,306 surveys were returned; 2,052 were excluded from Ray’s main analysis because the respondents had been homeschooled for six years or fewer (he only reports their responses to one question about their attitude toward homeschooling). Instead, he focused on the 5,254 surveys of people aged 16 to 69 years who had been homeschooled for seven or more years.
'Homeschooling Grows Up' (2002) major points
“Homeschooling Grows Up”, a 2003 document described by its anonymous authors as “HSLDA’s synopsis of a new research study on adults who were homeschooled, conducted by Dr. Brian D. Ray,” is available on HSLDA’s website in both web and pdf formats. According to the authors of the synopsis, the purpose of the study was to answer homeschooling critics’ concerns about the socialization of homeschooled children with evidence that these concerns were unfounded. With the results already planned before the study was conducted, it is no wonder that the findings appear to support such a conclusion.
The findings that the authors discuss in the pamphlet are as follows:
- About half (49%) of the participants were students enrolled in college courses at the time of the study. Three quarters of the participants (74%, including the students still in college) had taken some college courses. This figure is compared with the 46% of the general population that has taken college courses.
- After “Student,” the most common occupations of the study participants were “Other” (7.9%), “Homemaker, home educator” (7.3%), “Professional 1” (6.8%, including accountants, nurses, and artists), and “Office worker” (6%). 61.4% of study participants reported that they were “very satisfied” with their jobs, as compared with only 39.7% of the general US population age 16 to 29. 48.9% of participants were “pretty well satisfied” with their financial situation, compared with only 22.9% of the US population age 16 to 29. A greater percentage of the study participants (85.3%) than in the general US population age 16 to 29 (68.2%) believed that “hard work” was more important than luck or assistance in getting ahead in the world.
- In terms of civic involvement, study participants were more likely than members of the general public to report that they regularly read books (98.5% of homeschoolers vs. 69% of the US population) and magazines (100% vs. 89%), participate in organizations (88.3% vs. 50%) and community service (71.1% vs. 37%), attend frequent religious services (93.3% vs. 41%), and use a public library (90.3% vs. 56%). They were more likely than the general population to support freedom of speech (91.5% vs. 88%) as well as book-banning (40.8% vs. 36%), and professed greater knowledge of the internet (99.6% vs. 37%) and politics—only 6.2% of participants agreed that “Politics and government are too complicated to understand” as compared to 44% of the general population. The study participants were less likely than the general public to read a newspaper (60.6% vs. 82%) or watch televised news (42.1% vs. 64%). Ray does not attempt to explain these figures. However, it is likely that the high number of young, college-age participants—a much higher proportion of young people than in the general population—was responsible for the distinction in attitudes concerning print vs. online media.
- The study participants reported that they were more involved in politics than the average US citizen. They were more likely than the general population to have contributed money to a political cause, worked for a candidate or political party, attended a public meeting, contacted a public official, participated in a protest or boycott, and voted in the past five years. (Note that the percentages cited for these comparisons are broken down by age group and are not available as overall percentages.)
- The study participants described themselves as having greater life satisfaction than the general US population. 58.9% of the participants described themselves as “very happy”, as opposed to only 27.6% of the US population aged 16 to 29. Similarly, 73.2% of participants found their lives “exciting”, compared to only 47.3% of the US population aged 16 to 29.
- In terms of their opinions on homeschooling, 95.2% of participants were glad they had been homeschooled, 92.4% believed being homeschooled had benefited them as an adult, and 82.1% would homeschool their own children. Only 5.2% of participants believed their economic opportunities had been limited by their educations, and only 2.1% believed their career choices had been limited.
- Of the 807 adult participants who had school-age children, approximately 74% had homeschooled at least one child for one year.
Home Educated and Now Adults (2004) major points
Ray’s 2004 book Home Educated and Now Adults contains 126 pages of text, plus introductory material and an index, and is available for purchase from his website for $9.95. Ray describes his research as “a cross-sectional descriptive and exploratory study” (Ray 2004:15); that is, he does not conduct statistical comparisons between his sample of homeschoolers and any larger population, and he does not claim that his sample is representative of any larger group.
In addition to those results discussed above in “Homeschooling Grows Up,” Ray’s findings about the people he surveyed are as follows.
- Of the total 7306 surveys that were returned, around 35% of respondents had been homeschooled for their entire lives, and 5254 respondents had been homeschooled for seven or more years.
- Those who had been homeschooled for less than seven years had a slightly less positive attitude towards homeschooling than those who had been homeschooled for seven years or more. Ray did not perform any statistical tests to determine the significance of this difference. (Note that the vertical axis in Figure 1 starts at 70%.)
Figure 1: Number of participants who Strongly Agree or Agree
- Ray focused the majority of his analysis on respondents who had been homeschooled for seven or more years. These respondents were most likely to have been homeschooled in middle school. Homeschooling became less frequent at the extreme ends of the age spectrum.
- The overwhelming majority of Ray’s participants who had been homeschooled for seven or more years—85.3%—were raised in a Protestant religious environment. Only 4.5% were raised in a Catholic home, and no other religious group exceeded 1% of the sample. Those who reported that the religious environment of their upbringing did not fit in the given categories composed 9.3% of the sample. The participants’ religious environments were also fairly monolithic—92.8% of them ‘strongly agreed’ or ‘agreed’ with the statement ‘My parents wanted me as a grown adult to hold basically the same religious beliefs as they held.’ Only 3.1% of the sample grew up in dual-religion households.
- The majority of Ray’s participants who had been homeschooled for seven years or more were female—there were approximately 3401 female (65%) and 1852 male participants. The median number of siblings for these participants was three. They were primarily white (93.1%), with 4.4% being members of two or more races. Native Americans, Asians, African Americans, Hispanics, and Pacific Islanders each composed less than 1% of Ray’s sample.
- Participants who had been homeschooled for seven years or more reported that their parents had been motivated by a variety of reasons: 76.7% reported that they had been homeschooled for religious reasons; 61.7% had parents who objected to what the school taught; 48.6% had parents who wanted to individualize their curriculum; 31.3% were homeschooled due to their parents’ safety concerns; 2.4% were homeschooled because of special needs or a disability. Ray lists figures for a number of other motivations as well.
- Participants who had been homeschooled for seven years or more were asked to describe their primary method of instruction. 1303 had been educated with “traditional textbooks and assignments”; 1163 described their education as “eclectic, directed by parent”; 1776 were educated using multiple methods. The most common proprietary curriculum used by Ray’s respondents was that of the Christian fundamentalist Advanced Training Institute (ATI), with 289 participants; and 86 respondents described themselves as unschoolers.
- At the time of the survey in 2003, Ray’s participants who had been homeschooled for seven or more years ranged in age from 16 years to 69 years. However, they were strongly skewed toward younger ages: 78.6% of them were between ages 18 and 24; the mean and median ages were both 21 years; the mode was 19 years; and 342 of these participants (7%) were under age 18. Only 783 participants (15%) were over age 24. Because the plurality (49%) of Ray’s participants who had been homeschooled for seven years or more were full-time students, he chose the subset of 18- to 24-year-old respondents who had been homeschooled for seven or more years to compare with the subsection of the US population ages 18 to 24. He found that 15.2% of his subsample had a Bachelor’s degree or higher, as compared with 7.95% of the general population in this age range.
- Of Ray’s participants who a) had been homeschooled for seven or more years; b) were over age 18; and c) were not full-time students; 66% had completed some college courses or higher. Though this group of participants could have been compared with the general population in terms of their employment and income status, Ray chose not to do so for unexplained reasons.
- Participants who had been homeschooled for seven years or more hailed from all US states. However, homeschoolers in the West and Mountain West were overrepresented, while those in New England were underrepresented. Note that these maps represent the 2003 home addresses of the homeschool graduates, not the locations where they were homeschooled.
- Participants who had been homeschooled for seven or more years reported high satisfaction with their work (95.9% were ‘very satisfied’ or ‘moderately satisfied’, as opposed to 86.8% of the US population age 16 to 29) and high satisfaction with their financial situation (92.2% were ‘very satisfied’ or ‘moderately satisfied’, as opposed to 72% of the US population age 16 to 29).
- Participants who had been homeschooled for seven or more years were slightly less likely to be Protestant or Catholic than their parents; they were slightly more likely to be Jewish, adherents of another religion, or nonreligious. However, 93% of them ‘strongly agreed’ or ‘agreed’ with the statement ‘My religious beliefs are basically the same as my parents’.’
- Participants who had been homeschooled for seven years or more had fairly low incomes. The 2003 poverty threshold for single people under age 65 was $9,573; approximately 63% of Ray’s sample had an income under $10,000. This is not particularly meaningful since many participants were full-time students (although it is unclear how much overlap there was between full-time students and those in the lowest income bracket). However, 4.2% of those who had been homeschooled for seven or more years had used a federal low-income assistance program in the past year (WIC, food stamps, or welfare).
- One quarter (25.5%) of all participants who had been homeschooled for seven or more years reported that they had consumed alcohol in the past 30 days; 3.3% were binge drinkers, and 3.2% were smokers. 1.9% had been convicted of a crime.
- Of those homeschooled for seven or more years, 812 (15% of respondents) had children age 5 or older. 73.9% of these had homeschooled at least one child for one year; 9.9% had sent their children to private school; and 8.6% had sent their children to public school.
- In his study, Ray made an effort to control for subsample biases. In order to do so, he conducted tests of statistical significance on the results of five survey questions which he analyzed with respect to five variables. He found that a) the 4075 participants who responded to a post-survey email from him had significantly more positive attitudes towards homeschooling than those who did not reply; b) the 3868 participants who provided their zip codes had significantly more positive attitudes towards homeschooling than those who did not provide zip codes; c) the 3425 participants whose families had been HSLDA members had significantly more positive attitudes towards homeschooling than those whose families had not been HSLDA members; d) the format of the survey had no significant correlation with participants’ attitudes towards homeschooling; and e) the 3401 females who participated were significantly more likely than males to have positive attitudes towards homeschooling. These were the only statistical tests used in Ray’s study.
- Though Ray does not attempt to explain these statistical findings, it seems most likely that variables a) and b) were measuring engagement with the study; respondents who were more positive about homeschooling would naturally care more about the results of the study and would thus participate more fully. Variable e) is likely explained by the fact that, in general, girls have more positive attitudes towards education than boys.
- Ray summarizes only two of his qualitative interviews—in particular, interviews where the participants reported that they were not glad to have been homeschooled. Here, Ray quotes their reasons for being dissatisfied (mostly social and psychological) but concludes that they actually benefited from homeschooling academically. Interviewee David Koto reports that “Everything is very controlled by your parents when you’re homeschooled and you can’t just hang out with who you want to.” Koto also states that homeschooling has helped him academically, and that he does not believe the disadvantages will have a negative effect on him for life. Interviewee Anne Smithson (a pseudonym) states, “I think I was lonely a little bit. I needed people but people were too much for me…[Homeschooling] just made me different from people my age. I had different experiences…Good or bad, I just wasn’t exposed to it…[It] was a bit traumatic for my sisters and me. To be a little awkward, socially.” However, according to Ray, Smithson also reported that her maturity, strong work ethic, and intelligence were due to being homeschooled.
Ray concludes that “the home-educated adults in this study were very positive about having been homeschooled and toward homeschooling in general, actively engaged in their local communities, keeping abreast of current affairs, highly civically involved, tolerant of others expressing their viewpoints, attaining relatively high levels of formal education, religiously active and wide-ranging in their worldview beliefs, holding worldview beliefs similar to those of their parents, and largely home educating their own children” (Ray 2004:85).
Home Educated and Now Adults is among Ray’s more effective publications, as he was comparatively thorough in his descriptions of his methodology, statistical analyses, and results. Nonetheless, the work has several methodological flaws, including: self-selection and response biases on the part of the participants; a lack of discussion of the findings from several of the survey questions; the fact that the sample of homeschool graduates was not representative either of all homeschool graduates or of all US adults (focusing as it did primarily on the 18 to 24 age bracket); and the low survey response rate.
1) Self-selection bias
The most significant methodological problem with this study is that the participants self-selected for the variables being tested. That is, homeschool graduates who were proud of their achievements and for whom homeschooling was a major aspect of their identity were most likely to participate in the study. Because the main avenues used to recruit participants were “[h]omeschool support organizations’ print and online publications and e-mail lists, contacts via word-of-mouth, and personal networks” (Ray 2004: 17), many participants were likely older siblings of children currently being homeschooled, friends and relatives of homeschoolers, or active and activist members of homeschooling organizations. This is the very population that would be most likely to have strongly positive views about homeschooling, to want to present homeschooling in a positive light, and to be highly involved in their communities.
Only a small percentage of all homeschoolers fall into the category of ‘movement’ homeschoolers, or those strongly involved and invested in homeschooling activism. In 2003, there were around 1,096,000 homeschoolers in the US; there were only about 62,000 HSLDA member families in 2000. This means that HSLDA member families composed somewhere between 6% and 10% of the national homeschooling population when Ray’s study was conducted. However, a full 65% of Ray’s participants’ families were or had been HSLDA members, and Ray found that these participants had a significantly more positive view of homeschooling than those participants who had not been members of HSLDA. Furthermore, HSLDA members were likely to respond to the survey as it was largely funded and promoted by HSLDA. Thus, the study was biased by the artificially inflated homeschooling approval rates of the disproportionately high number of survey participants whose families had been members of HSLDA.
Furthermore, around 35% of Ray’s 7,306 respondents had been homeschooled for their entire lives, and 72% had been homeschooled for seven years or more. However, nationwide only 15% of secular homeschoolers and 48% of religious homeschoolers continue to homeschool beyond 6 years (Isenberg 2007).
The participants in Ray’s study were thus likely to be long-term ideological homeschoolers, for whom homeschooling was not merely an educational method but rather an identity and a way of life. As a result, this population is not only non-representative of all homeschoolers, it may also be prone to social desirability bias—the tendency to give the answers one thinks would be viewed favorably by others. And since Ray recruited some of his participants through his personal networks, these respondents may have been even more reluctant to give answers they thought he would view unfavorably—as the results were not anonymized in any way, Ray may have been able to identify the responses of individuals he knew personally.
2) Response bias
Response bias, which occurs when participants can easily guess the researcher’s preferred answer, was also a major problem with Ray’s study. The survey was framed as a call to action for homeschooling activists and, as discussed above, implicitly prompted respondents to reply in a positive way. For instance, the introduction asks “Is homeschooling a credible educational form, or not?”, apparently inviting a defense of homeschooling.
Though a portion of Ray’s survey questions were drawn from thoroughly tested national surveys (for example, the US census), he wrote many of the other questions himself. Some of Ray’s original questions are strongly ‘leading’; that is, they prompt the reader to respond a certain way. For instance, “Are you a ‘born-again Christian’?” has only “Yes” and “No” as possible answers. Christian respondents who are not ‘born-again’, as well as non-Christian respondents, might have difficulty answering this question. Footnote 10 above discusses another question some respondents might find difficult to answer. Even survey respondents may have recognized the bias—one CRHE Board member who participated in the survey reports feeling at the time that the wording of the questions made the desirable answers obvious.
3) Unreported data
Ray does not report the results of a number of his survey questions. His survey instrument contained questions on participants’ marital status and marital happiness; whether their parents were certified teachers; whether participants had taken standardized tests; whether their families had belonged to a variety of local, state, and national homeschooling organizations; and the US state they had lived in at age 16. The survey also asked participants’ opinion of the Bible; whether they believed in an afterlife; whether they were born-again Christians; whether they had health insurance; and whether they owned their own home.
It is unknown why Ray chose not to present this data.
4) Non-representative sample
Ray’s participants were not representative of homeschoolers as a whole and were not comparable to national statistics due to discrepancies in age, race, gender, and educational attainment. Further, Ray’s survey did not ask for family background information like parents’ income or educational attainment—even though research has consistently shown that these factors affect children’s lifetime academic and economic success, so they must be controlled for in order to isolate the effect of homeschooling.
Ray’s participants were extremely young: 7% were under age 18; 78% were between ages 18 and 24; and only 15% were over age 24. The mean and median ages were both 21 years.
This means that they were, by and large, too young to have experienced the variety of life events we might expect for the average American. They likely had little experience with family life—in 2003, the average age at first marriage was 27.1 years for men and 25.3 years for women; and a 2006 study found that the average age of first-time mothers was 25 years. They had not completed their educations—49% were full-time students—and were too young to know whether they eventually would: approximately one-third (32.7%) of BA recipients are over age 24, according to a 2008 survey; in 2007, 81% of graduate students were over age 24, and in 2003 the median age of doctorate recipients was 33.3 years. Lifetime earnings tend to peak around age 50. The religious beliefs of Ray’s participants also may not remain stable: a 2009 study reported that more than one-quarter (28.4%) of religious conversions occur at age 24 or older.
Thus, it is highly unlikely that Ray’s recipients’ self-reports are representative of the entire spectrum of homeschool graduates’ experiences. They were simply too young at the time of this study to determine whether they would successfully form families, complete higher education, obtain jobs, and become integral members of their communities.
In general, homeschooling tends to be more common for white children than for children who belong to racial or ethnic minorities. While 62% of public school students were white in 2003, the national population of homeschoolers was 77% white, 9% black, 5% Hispanic, and 8% other races—in other words, white children in America are somewhat disproportionately concentrated in homeschools. However, Ray’s (2003) sample is overwhelmingly white at 93%, making it extremely non-representative of the nation. Of the 5246 participants who reported their race, only 9 identified themselves as black.
Due to structural inequalities in education and other aspects of society, black and Hispanic Americans have lower lifetime educational attainment than white Americans—in 2003, 30% of white Americans age 25 or older had attained at least a Bachelor’s degree while only 17% of black Americans and 11% of Hispanic Americans had done so. These structural inequalities also contribute to huge racial wealth gaps—in 2005, the median net worth of white Americans was approximately $130,000, while it was $11,000 for African-Americans and $17,000 for Hispanics. A 2011 study which measured a number of indicators of general well-being, including housing and neighborhood conditions as well as the expectation of help if the need arose, found that while 82% of white Americans had satisfactory well-being, only 71% of black and 72% of Hispanic Americans did.
In effect, Ray’s study tells us virtually nothing about the lifetime success of the nearly one-quarter of homeschool graduates who belong to racial or ethnic minorities. Because of this oversight, his study is not representative of the homeschool population as a whole, and his results cannot be fairly compared to measures of the general population’s lifetime success.
In 2003, 51% of the nation’s homeschoolers were female, and statistics from previous years tend to show a similar age distribution. In Ray’s study, 65% of the participants were female, an imbalance which further indicates that his sample was not representative of the population of all homeschoolers.
Gender is relevant here because females earn higher grades than males—both in primary and in secondary school—and women outnumber men in college. In 2003, 12% of females ages 18-24 nationwide had earned at least a Bachelor’s degree, while only 8% of males had done so. Thus, the educational success of Ray’s homeschooling graduates may be artificially inflated since the sample contains an excess of women.
Furthermore, girls have been shown to have more positive attitudes than boys towards education in general. This may have contributed to the artificially high approval rate of homeschooling.
If the sample were more representative in terms of age, the excess of female participants might artificially deflate the success of Ray’s homeschooling graduates, since women compose a smaller percentage of the workforce nationwide than men (around 10 to 20 million fewer women than men worked in 2003) and earn lower wages—the pay gap between men and women, which was approximately $10,000 in 2003, is well-documented. However, the extreme youth of Ray’s participants makes this irrelevant.
According to NCES data, in 2003 homeschooling parents were more likely than the average American parent to have at least a Bachelor’s degree (45% of homeschooling parents had at least a BA as compared with 37% of US parents).
On the surface, Ray’s (2003) data appears to show that graduates of homeschooling have much lower levels of educational attainment than average—only 19% of his participants had a Bachelor’s degree or higher. This conclusion is deceptive, however, since Ray’s participants were largely too young to have yet obtained Bachelor’s degrees—note that 342 respondents (7% of those represented in this graph) were under age 18.
Ray attempts to remedy his disproportionately young sample of homeschool graduates by comparing his participants aged 18-24 with the general population of Americans age 18-24. He finds that 15% of his participants in this age bracket had at least a Bachelor’s degree, as compared with 8% of the national population. However, this is deceptive as well, because Ray did not collect information about the educational attainment of his participants’ parents. Ray’s other work consistently over-represents homeschooling parents who are wealthier and better educated than the general population; it seems likely that this study did so as well. Without controlling for parents’ income or educational attainment—which have both been shown to affect children’s lifetime academic and economic success—Ray’s findings on his participants’ educational attainment are essentially meaningless.
5) Low response rate
The response rate for the study was approximately 12%. This means that out of the 58,500 people Ray contacted about participating in the study, only about one in eight (7,306 people) actually completed and submitted the survey. Furthermore, when participants who had been homeschooled for less than 7 years were eliminated, fewer than one in ten (9%) of the people Ray originally contacted had their opinions represented in the survey.
In social science research of this type, “a response rate between 70 and 80% is the goal, with 70% being minimally acceptable and a response rate above 80% being the ideal, although highly unlikely. If the response rate falls below 60% it becomes very difficult to claim that the survey is representative of the entire sample because responses are missing from more than one third of surveyed individuals” (Mangione 1995 and Bachman & Schutt 2007; cited in Henninger & Sung 2012:306). Needless to say, Ray’s (2003) response rates are abysmal, indicating that this study’s findings are not applicable to anyone other than the people who participated.
What the Study Actually Says
The participants in this study were primarily white Protestant female college students between the ages of 18-24 who considered having been homeschooled to be an important part of their identity. Few participants had children or an income. They had positive attitudes about homeschooling and were fairly religious, attending religious services regularly and reporting that they had the same religious beliefs as their parents. Participants reported high levels of life satisfaction, job satisfaction, and financial satisfaction. They were involved in local community organizations, knowledgeable about current events, active in politics, and believed in hard work.
These participants were not in any way representative of all homeschoolers, let alone comparable to the general US population, so it is impossible to use this study to claim that all homeschoolers are well socialized, or that socialization is not a problem in the homeschooling community, or that homeschool graduates are successful in life. We can only state that, in this limited and biased sample, most participants had learned at least some aspects of “how to interact effectively in group settings and broader society, understanding its rules of behavior and social customs,” according to Kunzman and Gaither’s (2013:19) first definition of socialization. However, this survey does little to address these homeschool graduates’ ability to navigate “a range of social influences—parents, peers, local communities, broader society—in the formation of personal values, beliefs, and commitments” (Kunzman & Gaither 2013:19). We do not know if these homeschooling graduates are capable of making friends, identifying their own likes and dislikes, working as part of a team, resisting peer pressure, conversing with someone who does not share their religious or political beliefs, standing up to a bully, etc. In addition, the participants’ extreme youth and lack of exposure to life experiences makes it impossible to know how likely they are to have successful careers and families.
We can thus only conclude from this study that it is possible for young graduates of homeschooling to have high levels of life satisfaction and community involvement. We cannot conclude that all homeschooling graduates are well socialized. We likewise cannot conclude that even this tiny subset of homeschool graduates does not have problems with socialization, according to some definitions, or that they will never have such problems in the future as they pursue careers and families.
Ray’s 2003 study on adult graduates of homeschooling is in fact primarily a study of college-age white Protestants for whom homeschooling was part of their identity. The participants’ responses to the survey showed that they were religious, involved in their local communities, interested in politics, and satisfied with their lives. The survey did not show whether they were capable of earning degrees, forming families, or achieving career success because they were too young. Nor did the survey show whether they were able to form close relationships or a sense of security in their own identities.
The study did not show that all homeschoolers are well socialized. It showed that some homeschoolers were well socialized, according to some definitions, at a certain point in their lives.
Advanced Training Institute International. 2011. “About ATI: raising up the foundations of Godly generations.” http://ati.iblp.org/ati/about/
Bachman, Ronet and Russell K. Schutt. 2007. The practice of research in criminology and criminal justice (3rd ed.). Thousand Oaks, CA: Sage.
Barker, Eric. 2011. “Why do girls get better grades even though boys score higher on IQ tests?” Barking Up the Wrong Tree. Business Insider. http://www.businessinsider.com/why-do-girls-get-better-grades-even-though-boys-score-higher-on-iq-tests-2011-8
Bell, Nathan E. 2009. “Data Sources: Non-traditional students in graduate education.” Council of Graduate Schools Research Report. http://www.cgsnet.org/ckfinder/userfiles/files/DataSources_2009_12.pdf
Brightbill, Kathryn. 2013. “How bad homeschool research and statistics hurt homeschoolers.” The Life and Opinions of Kathryn Elizabeth, Person. http://kathrynbrightbill.com/post/56313343522/how-bad-homeschool-research-and-statistics-hurt
Coalition for Responsible Home Education. 2013. “Homeschooling Numbers.” http://www.responsiblehomeschooling.org/homeschooling-101/homeschooling-numbers/
Coleman, Rachel. 2013. “A brief history of homeschooling.” The Politics of Childhood. http://politicsofchildhood.org/2013/05/16/a-brief-history-of-homeschooling/
Davis-Kean, Pamela E. 2005. “The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment.” Journal of Family Psychology 19:2. 294-304. http://www.mikemcmahon.info/ParentEducationIncome.pdf
DeNavas-Walt, Carmen, Bernadette D. Proctor, and Jessica C. Smith. 2010. “Income, poverty, and health insurance coverage in the United States: 2009.” US Census Bureau. US Department of Commerce. http://www.census.gov/prod/2010pubs/p60-238.pdf
Dubow, Eric F., Paul Boxer, and L. Rowell Huesmann. 2009. “Long-term effects of parents’ education on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations.” Merrill Palmer Quarterly 55:3. 224–249. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2853053/
Farenga, Pat. 2003. “What is unschooling?” John Holt and Growing Without Schooling. http://www.holtgws.com/whatisunschoolin.html
Francis, David R. n.d. “Why do women outnumber men in college?” Cambridge, MA: The National Bureau of Economic Research. http://www.nber.org/digest/jan07/w12139.html
Gaither, Milton. 2008. “Brian D. Ray and NHERI, part 2.” Homeschooling Research Notes. http://gaither.wordpress.com/2008/10/07/brian-d-ray-and-nheri-part-2/
Hall, Shawn and Annette Hall. 2010. “Who does HSLDA represent?” Homeschooling is Legal. http://hsislegal.com/who_does_hslda_represent/
Henninger, Alana and Hung-En Sung. 2012. “Mail survey in social research.” Handbook of Survey Methodology for the Social Sciences, ed. by Lior Gideon, 297-311. New York: Springer.
Hoffer, Thomas B. and Vincent Welch, Jr. 2006. “Time to Degree of US Research Doctorate Recipients (NSF-06-312).” Arlington, VA: National Science Foundation. http://www.nsf.gov/statistics/infbrief/nsf06312/
Home School Legal Defense Association. 2003. “Homeschooling Grows Up.” Purcellville, VA: HSLDA. http://www.hslda.org/research/ray2003/
Isenberg, Eric J. 2007. “What Have We Learned About Homeschooling?” Peabody Journal of Education 82. 387-409.
Ironman. 2009. “Average lifetime earnings trajectories by education.” Political Calculations. http://politicalcalculations.blogspot.com/2009/07/average-lifetime-earnings-trajectories.html#.UvgYzbSad9Y
Kunzman, Robert and Milton Gaither. 2013. “Homeschooling: A comprehensive survey of the research.” Other Education: The Journal of Educational Alternatives 2:1. 4-59. http://www.othereducation.stir.ac.uk/index.php/OE/article/view/10/55
Lethbridge-Çejku M, Vickerie J. 2005. Summary health statistics for U.S. adults: National Health Interview Survey, 2003. National Center for Health Statistics. Vital Health Stat 10:225. http://www.cdc.gov/nchs/data/series/sr_10/sr10_225.pdf
Lugo, Luis et al. 2009. “Faith in Flux: Changes in religious affiliation in the US.” Pew Forum on Religion and Public Life. Pew Research Center. http://www.pewforum.org/files/2009/04/fullreport.pdf
Mangione, Thomas W. 1995. Mail surveys: Improving the quality. Newbury Park, CA: Sage.
Mathews, T.J. and Brady E. Hamilton. 2009. “Delayed Childbearing: More women are having their first child later in life (NCHS Data Brief No. 21).” National Center for Health Statistics. Centers for Disease Control and Prevention. US Department of Health and Human Services. http://www.cdc.gov/nchs/data/databriefs/db21.pdf
McCracken, Chelsea. 2014. “Homeschooling academics and demographics (Ray 2010).” Coalition for Responsible Home Education. http://www.responsiblehomeschooling.org/research-analysis/homeschooling-academics-and-demographics-ray-2010/
Princiotta, Daniel, Stacey Bielick, and Christopher Chapman. 2006. “Homeschooling in the United States: 2003. Statistical Report (NCES 2006-042).” National Household Education Surveys Program. National Center for Education Statistics. US Department of Education. http://nces.ed.gov/pubs2006/2006042.pdf
Ratcliffe, Caroline and Signe-Mary McKernan. 2012. “Child poverty and its lasting consequence.” Low-Income Working Families, Paper 21. The Urban Institute. http://www.urban.org/UploadedPDF/412659-Child-Poverty-and-Its-Lasting-Consequence-Paper.pdf
Ray, Brian D. 2003. “Homeschooling Grows Up.” Salem, OR: National Home Education Research Institute. http://www.hslda.org/research/ray2003/HomeschoolingGrowsUp.pdf
Ray, Brian D. 2004. Home Educated and Now Adults: Their community and civic involvement, views about homeschooling, and other traits. Salem, OR: NHERI Publications.
Siebens, Julie. 2013. “Extended measures of well-being: Living conditions in the United States: 2011.” Household Economic Studies. US Census Bureau. US Department of Commerce. http://www.census.gov/prod/2013pubs/p70-136.pdf
United States Census Bureau. 2004. “Table 1. Educational attainment of the population 15 years and over, by age, sex, race, and Hispanic origin: 2003.” US Department of Commerce. http://www.census.gov/hhes/socdemo/education/data/cps/2003/tab01-01.pdf
United States Census Bureau. 2010. “Educational Attainment in the United States: 2003 – Detailed Tables.” US Department of Commerce. http://www.census.gov/hhes/socdemo/education/data/cps/2003/tables.html
United States Census Bureau. 2011. “Table MS-2. Estimated Median Age at First Marriage, by Sex: 1890 to the Present.” Current Population Survey, March and Annual Social and Economic Supplements, 2011 and earlier. US Department of Commerce. http://www.census.gov/population/socdemo/hh-fam/ms2.xls
United States Census Bureau. 2012. “Poverty Thresholds 2003.” US Department of Commerce. http://www.census.gov/hhes/www/poverty/data/threshld/thresh03.html
United States Census Bureau. 2012. “Table 306. Crimes and crime rates by type of offense: 1980 to 2009.” Statistical Abstract of the United States: 2012. http://www.census.gov/compendia/statab/2012/tables/12s0306.pdf
United States Census Bureau. 2013. “Table 1. Median Value of Assets for Households, by Type of Asset Owned and Selected Characteristics: 2005.” Survey of Income and Program Participation, 2004 Panel, Wave 6. US Department of Commerce. http://www.census.gov/people/wealth/files/Wealth Tables 2005.xls
Van Houtte, Mieke. 2004. “Why boys achieve less at school than girls: The difference between boys’ and girls’ academic culture.” Educational Studies 30:2. http://www.researchgate.net/publication/232974467_Why_boys_achieve_less_at_school_than_girls_the_difference_between_boys%27_and_girls%27_academic_culture/file/3deec5250943eb4471.pdf
Woo, Jennie, Caitlin Green, Morgan Matthews, and Ted Socha. 2012. “Profile of 2007-08 First-time Bachelor’s Degree Recipients in 2009 (NCES 2013-150).” Web Tables. National Center for Education Statistics. US Department of Education. http://nces.ed.gov/pubs2013/2013150.pdf.
 Ray cites the 2003 census for this figure.
 This category contrasts with “Professional 2” (minister, doctor, college teacher, etc.).
 Other sources Ray cites for his general population figures include the National Opinion Research Center (2003) and the National Household Education Survey (1997).
 .9% of the participants reported being nonreligious and .1% were Jewish. There was one Muslim participant.
 79.5% “Can give child better education at home”; 73.5% “Teach child particular values, beliefs, worldview”; 69.2% “To develop character/morality”; 56.1% “Poor learning environment at school”; 56% “Desire more parent-child contact”; 46.7% “Individualize learning environment”; 41.8% “Believe parents should educate children”; 37.9% “Desire more sibling-sibling contact”; 33.6% “School does not challenge child”; 31.8% “Closely guide their social interactions”; 19.5% “Other problem with available public/private schools”; 13.4% “Family reasons”; 9.5% “Want private school but cannot afford it”; 7.4% “Other reasons”; 7% “Transportation/distance/convenience”; 4.3% “Parent’s career”; 1.9% “Child not old enough for grade/to enter school”; 1.8% “Student behavioral problems”; .8% “Child has temporary illness”; .8% “Could not get into a desired school”.
 The origin of this 69-year-old homeschool graduate is not explained. While this individual would have been educated in the 1940s, the modern homeschooling movement did not begin until the 1970s. Prior to the beginning of the modern homeschooling movement, children might have been educated at home if they were chronically ill, or by tutors if they were wealthy. See Coleman (2013) for more information.
 Ray does not conduct statistics on these differences, so it is not possible to tell whether they are statistically significant. Since most of the differences are very small, it is unlikely that they are meaningful.
 This figure refers to personal, rather than family, income, and is not broken down by family structure or number of children. Recall also that 7.3% of participants were homemakers.
 In the general population in 2003, 47% of adults were regular drinkers and 22% of adults were smokers.
 There were around 4 crimes for every 100 Americans in 2003.
 It is unclear whether respondents who had been both homeschooled and public schooled, or both private and public schooled, would have replied affirmatively to this question. Respondents were encouraged to check multiple answers, but the public school option appeared to preclude other choices. The survey question read: “If any of your children are age 5 or older, for your child(ren)’s elementary or high school education, did any of your child(ren) ever attend a private school or were they homeschooled for at least one year? (Check all that apply.)” Answer choices were: “No, neither; all attend/attended public school (only)”, “Private school (at least one child for one year)”, “Homeschooled (at least one child for one year)”, “Private school and homeschooled (#2 and #3)”, “Other”, and “Have no children age 5 or older”.
 Ray used the statistical software SPSS; by his choice, p-values of .01 or less indicated significance.
 These questions were rated on a five-point Likert scale of “Strongly Agree” to “Strongly Disagree”. The five questions were: “I am glad I was homeschooled”, “Having been homeschooled is an advantage to me as an adult”, “Having been homeschooled has limited my educational opportunities”, “Having been homeschooled has limited my career choices”, and “I would homeschool my own children.”
 The five variables were: a) whether participants responded to an email from Ray to confirm their completion of the survey; b) whether participants provided a contact zip code; c) whether participants’ families had ever been members of HSLDA; d) whether participants completed the online or paper version of the survey; and e) participants’ gender.
 It is also possible that HSLDA influences its members to have more positive views of homeschooling, and that families for whom homeschooling is an important part of their identity are more likely to join HSLDA.