This post summarizes our recently added research review, which provides a critical analysis of Ray (2003, 2004). Click HERE to read a more in-depth version of the arguments presented.
In 2003, the Home School Legal Defense Association (HSLDA) commissioned Dr. Brian Ray to conduct a study on the lifetime success of homeschool graduates. The results of the study were written up in two documents: a 2003 summary of the results, entitled “Homeschooling Grows Up”, which was produced by HSLDA and posted on their website; and a book entitled Home Educated and Now Adults, which was published in 2004 by Ray’s National Home Education Research Institute (NHERI).
Ray’s study, which took the form of a survey, is widely cited to support the claim that graduates of homeschooling are well-socialized and go on to lead successful lives: Ray reports that his participants had positive attitudes about homeschooling and life in general; that they were fairly religious, attending religious services regularly and reporting that they had the same religious beliefs as their parents; that they were involved in local community organizations, knowledgeable about current events, and active in politics; and that they reported high levels of life satisfaction, job satisfaction, and financial satisfaction.
Ray’s study relies on a fairly limited definition of successful socialization—he asks his participants about their religious beliefs, political activism, and occupations, but he does not ask broader questions about the quality of their relationships, their ability to navigate US culture, and their problem-solving strategies. Nonetheless, it is one of the few studies which have been conducted on homeschool graduates.
Unfortunately, Ray’s study has so many methodological problems that we can draw few conclusions from it. The most glaring methodological problems are:
1) Bias
Ray did not survey a random sample of homeschool graduates. The main method Ray used to recruit homeschool grads to take his survey was to publicize it in homeschooling organizations’ newsletters, as well as to use “word-of-mouth, and personal networks” (Ray 2004: 17). He may have known several of the participants personally, and many others may have been older siblings of children currently being homeschooled, friends and relatives of homeschoolers, or active and activist members of homeschooling organizations. This is the very population that would be most likely to have strongly positive views about homeschooling, to want to present homeschooling in a positive light, and to be highly involved in their communities.
In other words, Ray’s sampling method is like attending a meeting of the chess club and surveying the members to find out who likes chess. Is it really a surprise if 99% of them say yes?
Ray’s questions themselves were also biased, leading respondents to give the answer prompted by the question rather than the honest answer. Our own Kathryn Brightbill, who was one of Ray’s participants, noted that she could easily discern the intended answer for each question.
2) Non-representative sample
Ray’s participants were primarily white Protestant female college students between the ages of 18-24 who considered having been homeschooled to be an important part of their identity. Few participants had children or an income. These participants were not representative of all homeschoolers in their age, race, and gender, nor were they comparable to the general US population. Their youth (7% were under 18) and lack of exposure to life experiences makes it impossible to know how likely they were to have successful careers and families. Furthermore, Ray did not collect data on the homeschool graduates’ backgrounds (such as their parents’ income and education levels), so we cannot determine whether homeschooling played a role in their high life satisfaction or community involvement.
It is impossible to use the responses of this select group to claim that all homeschoolers are well socialized, or that socialization is not a problem in the homeschooling community, or that homeschool graduates are successful in life. We can only state that, in this limited and biased sample, most participants had been socialized to some degree.
3) Unanswered questions
Ray’s report of his findings provokes more questions than it answers. For instance, though 7,306 people answered Ray’s survey, he only analyzed the responses of 5,254 homeschool grads—he excluded people homeschooled for less than seven years from consideration. He tells us virtually nothing about these 2,000 people’s responses—why? Also, his participants answered survey questions on their marital status and marital happiness, as well on other topics like whether they had taken standardized tests while being homeschooled. Why did Ray choose not to report his participants’ responses to these questions? And finally, though most of his participants were too young to accurately judge their lifetime successes, 15% of them were over age 24. Yet Ray does not perform a separate analysis of this group—why? Though Ray does not explain these choices, we might hypothesize that to include this information would be to present homeschooling in a less positive light.
Because of these problems, we can only conclude from this study that it is possible for young graduates of homeschooling to have high levels of life satisfaction and community involvement. We cannot conclude that all homeschooling graduates are well socialized—the study did not show whether the participants would be capable of earning degrees, forming families, or achieving career success. It does not tell us whether they were capable of making friends, identifying their own likes and dislikes, working as part of a team, resisting peer pressure, conversing with someone who does not share their religious or political beliefs, standing up to a bully, etc.
Instead, it shows Ray’s skill at manipulating data to serve his own agenda. This is stated most baldly in a quote from Bruce S. Cooper on the back cover of Ray’s Home Educated and Now Adults (2004): “Ray’s study disconfirms, then, what critics have long said: that without the institutionalized socialization of attending an organized school, kids would be limited in their futures, their education, their well-being, their civic engagement, and their happiness. Not true, for education begins, grows, and continues throughout life, at home.”
Homeschooling Outcomes or Sampling Problems? A Look at Ray 2003
This post summarizes our recently added research review, which provides a critical analysis of Ray (2003, 2004). Click HERE to read a more in-depth version of the arguments presented.
In 2003, the Home School Legal Defense Association (HSLDA) commissioned Dr. Brian Ray to conduct a study on the lifetime success of homeschool graduates. The results of the study were written up in two documents: a 2003 summary of the results, entitled “Homeschooling Grows Up”, which was produced by HSLDA and posted on their website; and a book entitled Home Educated and Now Adults, which was published in 2004 by Ray’s National Home Education Research Institute (NHERI).
Ray’s study, which took the form of a survey, is widely cited to support the claim that graduates of homeschooling are well-socialized and go on to lead successful lives: Ray reports that his participants had positive attitudes about homeschooling and life in general; that they were fairly religious, attending religious services regularly and reporting that they had the same religious beliefs as their parents; that they were involved in local community organizations, knowledgeable about current events, and active in politics; and that they reported high levels of life satisfaction, job satisfaction, and financial satisfaction.
Ray’s study relies on a fairly limited definition of successful socialization—he asks his participants about their religious beliefs, political activism, and occupations, but he does not ask broader questions about the quality of their relationships, their ability to navigate US culture, and their problem-solving strategies. Nonetheless, it is one of the few studies which have been conducted on homeschool graduates.
Unfortunately, Ray’s study has so many methodological problems that we can draw few conclusions from it. The most glaring methodological problems are:
1) Bias
Ray did not survey a random sample of homeschool graduates. The main method Ray used to recruit homeschool grads to take his survey was to publicize it in homeschooling organizations’ newsletters, as well as to use “word-of-mouth, and personal networks” (Ray 2004: 17). He may have known several of the participants personally, and many others may have been older siblings of children currently being homeschooled, friends and relatives of homeschoolers, or active and activist members of homeschooling organizations. This is the very population that would be most likely to have strongly positive views about homeschooling, to want to present homeschooling in a positive light, and to be highly involved in their communities.
In other words, Ray’s sampling method is like attending a meeting of the chess club and surveying the members to find out who likes chess. Is it really a surprise if 99% of them say yes?
Ray’s questions themselves were also biased, leading respondents to give the answer prompted by the question rather than the honest answer. Our own Kathryn Brightbill, who was one of Ray’s participants, noted that she could easily discern the intended answer for each question.
2) Non-representative sample
Ray’s participants were primarily white Protestant female college students between the ages of 18-24 who considered having been homeschooled to be an important part of their identity. Few participants had children or an income. These participants were not representative of all homeschoolers in their age, race, and gender, nor were they comparable to the general US population. Their youth (7% were under 18) and lack of exposure to life experiences makes it impossible to know how likely they were to have successful careers and families. Furthermore, Ray did not collect data on the homeschool graduates’ backgrounds (such as their parents’ income and education levels), so we cannot determine whether homeschooling played a role in their high life satisfaction or community involvement.
It is impossible to use the responses of this select group to claim that all homeschoolers are well socialized, or that socialization is not a problem in the homeschooling community, or that homeschool graduates are successful in life. We can only state that, in this limited and biased sample, most participants had been socialized to some degree.
3) Unanswered questions
Ray’s report of his findings provokes more questions than it answers. For instance, though 7,306 people answered Ray’s survey, he only analyzed the responses of 5,254 homeschool grads—he excluded people homeschooled for less than seven years from consideration. He tells us virtually nothing about these 2,000 people’s responses—why? Also, his participants answered survey questions on their marital status and marital happiness, as well on other topics like whether they had taken standardized tests while being homeschooled. Why did Ray choose not to report his participants’ responses to these questions? And finally, though most of his participants were too young to accurately judge their lifetime successes, 15% of them were over age 24. Yet Ray does not perform a separate analysis of this group—why? Though Ray does not explain these choices, we might hypothesize that to include this information would be to present homeschooling in a less positive light.
Because of these problems, we can only conclude from this study that it is possible for young graduates of homeschooling to have high levels of life satisfaction and community involvement. We cannot conclude that all homeschooling graduates are well socialized—the study did not show whether the participants would be capable of earning degrees, forming families, or achieving career success. It does not tell us whether they were capable of making friends, identifying their own likes and dislikes, working as part of a team, resisting peer pressure, conversing with someone who does not share their religious or political beliefs, standing up to a bully, etc.
Instead, it shows Ray’s skill at manipulating data to serve his own agenda. This is stated most baldly in a quote from Bruce S. Cooper on the back cover of Ray’s Home Educated and Now Adults (2004): “Ray’s study disconfirms, then, what critics have long said: that without the institutionalized socialization of attending an organized school, kids would be limited in their futures, their education, their well-being, their civic engagement, and their happiness. Not true, for education begins, grows, and continues throughout life, at home.”