Be Careful While Reading Research Reports

7/29/2019 Be Careful While Reading Research Reports

1/3

Be Careful While Reading Research ResultsDr. Brian D. RayCaveat emptorLet the buyer beware. The reader should keep in mind that this applies to

receiving and consuming products of research as well as buying a piece of land or a usedcar.Context

It appears good on the face of it. Researchers Pennings, Seel, Van Pelt, Sikkink, and Wiensexplain in their report that Christian schools . . . have served a vital role in the educationallandscape of North America for over 400 years . . . and that from about 1600 . . . until

late in the 18th century, the purpose of education in the U.S. was centered on religion . . .

(p. 9). These authors also rightly note the following: . . . All schools are religious in nature,and therefore parents desire schools in which congruence can exist for their children

between home, religious institution, and school (p. 11).1The researchers then explain why they are undertaking this project: The Cardus EducationSurvey has just this purposeto determine the alignment between the motivations andoutcomes of Christian education, setting a benchmark for further study of Christian

schooling. The reader quickly finds out that Christian education, in this study, includes

homeschooling by Christian parents but is not focused on home-based education.Their survey of graduates (i.e., from secondary school, high school). . . which included

participants from Catholic, Protestant, non-religious private, public, and homeschoolgraduatesfocused on educational and occupational attainment, civic and political

engagement, spiritual formation, marriage and family as well as social psychological

outcomes in the young adult years (p. 44).

FindingsSo . . . what did they find? They found some things that might negatively surprise

advocates of homeschooling. Home-educated graduates are the most likely to get divorced?Most likely to feel helpless in dealing with the problems of life? Most likely to lack any cleargoals or sense of direction? Most likely to feel prepared for a vibrant religious and spiritual

life?Homeschool graduates are the least likely to be involved in political campaigns? Least likelyto spend much time volunteering or going on mission trips? Protestant Christian school andhomeschool graduates hold more strongly to the belief that morality is unchanging and

absolute?But before your columnist provides any more of these researchers findings, clouds theresearch waters, or is quoted out of context, several important points must be made. First,

researchers must operationally define important terms in their studies. In this study,homeschooled was defined as a participant reporting he was homeschooled in high school.

That is, the researchers only reported that the adults were homeschooled, Catholic

schooled, public schooled, and so forth for some of their high school years; no informationwas given on whether they attended that type of schooling for two, eight, or twelve years.

Further, the home educated were categorized as either religious or nonreligious,


2/3

simply depending on whether their mothers attended religious services regularly. That is all

that is known about these adults educational background.Second, about two thousand randomly selected Americans, aged 23 to 49, were sampled

and studied. This sample appears to have been planned and executed well. However, of

those, fewer than ninety were home educated at all. And only some of these fewer-than-

ninety had a religious homeschool mother during high school. This is a small sample sizefrom which very few dependable conclusions can be drawn.

Finally, this studys findings directly contradict the findings of several other studies about

adults who were home educated. For example, Ray found adults who were home educatedto be more engaged in direct civic involvement than the general population of the United

States. They were more involved in activities such as working for a candidate/political

party/political cause, voting in national/state elections, and participating in a protest orboycott.2 Contrariwise, Pennings et al. state that adults from religious homeschooling werenoticeably less civically involved on several measures than adults from several other

categories of schooling.Conclusions

Pennings and his colleagues have developed a valuable research concept. They are trying to

understand to what degree the outcomes of Christian education match the motivations of

adults who design and conduct the Christian education. It should be clearly pointed out,however, that this study has not. . . allowed for the establishment of quality benchmarks .

. . (p. 36) regarding the thinking, religious beliefs, and actual behaviors of adults who werehome educated. They might have quality benchmarks regarding those who attended state

(public), Roman Catholic, and Christian schools but not for those who were homeschooledby Christian parents.One might wonder: Do reports like this matter in the world in which we live? For better orworse, yes. Research is supposed to get at truth and accurate representations of reality.

God says the following: By wisdom a house is built, and by understanding it is established;by knowledge the rooms are filled with all precious and pleasant riches. A wise man is full ofstrength, and a man of knowledge enhances his might, for by wise guidance you can wage

your war, and in abundance of counselors there is victory.3 One can get wiser by reading

and heeding true things but not from integrating skewed or too-limited representations ofreality into his thinking.Unfortunately, research like this often shows up in widely read publications. For example,

the magazine Christianity Todayreported on the findings of Pennings et al. and presented

the culturally perceived positive and negative findings about the religiouslyhomeschooled as if the data were reliable and properly comparable to data from other

adults.The authors of this study should have been very careful to explain the notable limitations oftheir study regarding the home educated. Their caveats should have been clear enough thatothers reporting on their study would find it difficult to misrepresent reality about any of the

statistics in the study.Caveat emptorLet the buyer bewarewhen it comes to reading and using researchreports. Remember, the buyer or digester of researchers findings and conclusions must


3/3

remember the . . . warning that . . . the goods he or she is buying are as is, or subject to

all defects.4Authors Note: Please feel free to send your questions about research related to home-

based education and raising children to [email protected].

Endnotes:1. Pennings, Ray; Seel, John; Van Pelt, Deani A. Neven; Sikkink, David; & Wiens, Kathryn L.(2011). Cardus Education Survey: Do the Motivations for Private Religious Catholic and

Protestant Schooling in North America Align with Graduate Outcomes?Hamilton, Ontario,

Canada: Cardus.www.cardus.ca,www.wnd.com/files/2011/07/110826cardus.pdf2. Ray, Brian D. (2004). Home Educated and Now Adults: Their Community and Civic

Involvement, Views About Homeschooling, and Other Traits. Salem, OR: National Home

Education Research Institute,www.nheri.org.3. Proverbs 24:3-6, ESV.4. Retrieved December 12, 2011, fromlegal-

dictionary.thefreedictionary.com/Buyer+beware.Brian D. Ray, Ph.D., is president of the National Home Education Research Institute, anonprofit research and education organization. Dr. Ray often serves as an expert witness in

courts, testifies to legislatures, and is interviewed by the media. Brian is married to Betsy

and they have eight children and four grandchildren.

Copyright 2012, used with permission. All rights reserved by author. Originally appeared in

the February 2012 issue ofThe Old Schoolhouse Magazine, the family educationmagazine. Read the magazine free atwww.TOSMagazine.comor read it on the go and

download the free apps atwww.TOSApps.comto read the magazine on your Kindle Fire orApple or Android devices.
http://www.cardus.ca/http://www.cardus.ca/http://www.cardus.ca/http://www.wnd.com/files/2011/07/110826cardus.pdfhttp://www.wnd.com/files/2011/07/110826cardus.pdfhttp://www.wnd.com/files/2011/07/110826cardus.pdfhttp://www.nheri.org/http://www.nheri.org/http://www.nheri.org/http://www.legal-dictionary.thefreedictionary.com/Buyer+bewarehttp://www.legal-dictionary.thefreedictionary.com/Buyer+bewarehttp://www.legal-dictionary.thefreedictionary.com/Buyer+bewarehttp://www.legal-dictionary.thefreedictionary.com/Buyer+bewarehttp://www.tosmagazine.com/http://www.tosmagazine.com/http://www.tosmagazine.com/http://www.tosapps.com/http://www.tosapps.com/http://www.tosapps.com/http://www.tosapps.com/http://www.tosmagazine.com/http://www.legal-dictionary.thefreedictionary.com/Buyer+bewarehttp://www.legal-dictionary.thefreedictionary.com/Buyer+bewarehttp://www.nheri.org/http://www.wnd.com/files/2011/07/110826cardus.pdfhttp://www.cardus.ca/

Documents

Be Careful While Reading Research Reports