Interpreting Research in Nephrology Nursing

Evaluating Quantitative Research ReportsCynthia L. Russell

Editors Note: This is the third of a series of columns con-tributed by the ANNA Research Committee to assistnephrology nurses in understanding research approachesand methodologies and evaluating research,

.uantitative research is evaluated for several reasons.You may be trying to decide if the findings are wor-ihy of incorporation into your practice or you may

^ attempting to determine the current state of thereseaK:h in a particular area. In the evaluation process,you will objectively review the strengths and the weak-nesses of a report. Ultimately, you will determine whetherthe strengths of the work are greater than its weaknesses,whether the results can be incorporated into practice, andwhere the fmdings direct the next study.

Evaluating a quantitative research report may initiallyseem like a daunting task, even if you regularly readresearch. However, by using a systematic approach, youcan become more comfortable and proficient in evaluatingquantitative research. Numerous articles have been pub-lished on evaluating research reports (Beck, 1990; Pieper,1993; Ryan-Wengar, 1992; Soeken, 1985; Summers, 1991).In addition, multimedia CD-ROMs have become avail-able to assist with critiquing research for clinical practiceuse (Alderman, 199H; Beyea, 1998}. The purpose of thisarticle is to provide the knowledge and tools needed to suc-cessfully evaluate quantitative research reports.

Components of a Qualitative Research ReportihoLigh journal criteria vary, most qualitative research

reports contain five sections: research problem, review ofthe literature, methods (design, sampling plan, instrumen-tation, procedure, human subjects protection), data analy-sis and results, and discussion.

Research ProblemThe introduction of the report .should thoroughly

describe the background of the research problem so tbatthe need for the study is apparent. The author must builda case from the existing literature that the problem is ofsufficient merit to justify further research. For example, ifthe study's puipose is to measure the effect of infonnationand support on hope and uncertainty in individuals await-ing deceased donor renal tiansplantation, the introductionwould describe the number of people waiting for renaltransplant, the potential impact that interventions couldmake in this area, and the problems with extent research.

The statement of the problem should flow directlyfrom the introduction and should conclude this section.The statement of the problem broadly identifies what

Cynthia L. Russell, PhD, RN, AfSCNS, is Assistant i\ofessor. Universityof Missouri- Columbia, Sinclair School of Nursing, Columbia, MO; and amember of ANNA's Central Missouri Chapter.

needs to be studied, including both who and what will bestudied. A problem statement example follows: Becauseof the long waiting time, there is a need for nursing inter-ventions to assist individuals waiting for deceased donorrenal transplantiition. After you read the problem state-ment, you should have an idea of what completion of thestudy included. You should detennine whether the prob-lem makes a significant contribution to the science andwhether it is relevant to your practice.

Review of the Literature iThis section provides the foundation for helping the

reader understand what the current state of the evidenceis in the selected area of study. The review of the literatureshould present pertinent findings from selected researchreports in an organized and clear fashion. Frequently,reviews are organized by headings thett correspond to keystudy concepts. The review must also present an evalua-tion of the quality of the pertinent literature notingstrengths and weaknesses. As you read the review of liter-ature, it should move from the broad to the specific withthe last section of the review clearly delineating the needfor the study. For example, a review of literature on inter-ventions for those awaiting deceased donor renal trans-plantation would begin with a section summarizing litera-ture on the experience of waiting for a tnmsplant imd thenmove to a section on nursing interventions used to assistthose waiting. The review of the literature would then con-clude with the statement of the gaps in existing literatureand how the current study will address those gaps.

The review of the literature may contain a section onthe theoretical or conceptual framework. If presented, youshould assess whether the theoretical or conceptual frame-work is clearly described including concepts and relation-ships. The problem statement should flow directly fromthe theoretical or conceptual framework.

I IMethods

The methods section describes the steps used by thereseaicher to carry out the study. This section includes thedesign, sampling plan, instrumentation, procedure, andthe protection of human subjects.

Design. The design delineates the plan or blueprint ofthe study. Non experimental designs, which includedescriptive and correlational designs, examine phenome-na as they naturally occur, so no manipulation is involved.A descriptive design allows the researcher to describe the

This column is compiled by the ANNA Research Committee to assistnephrology nurses in understanding research approaches and methodoiogiesand evaiuating research. For additional information, contact Patricia A. Cowan,PhD. RN; ANNA Research Committee through the ANNA National Office; EastHolly Avenue/Box 56; Pitman. NJ 08071-0056; nephrologynursing@ajj.com.

NEPHROLOGY NURSING JOURNAL • January-February 2005 • VoL 32, No. 1 61

Evaluating Quantitative Research Reports

Table 1Criteria for Research Report Evaluation

Research Problem1. is the problem clearly stated?2. Is the problem significant?

Review of the Literature1. Is the literature summarized?2. Is the literature critically evaluated?3. Are gaps and inconsistencies in the literature

described?4. Is the literature current and the review complete?5. If presented, is the theoretical or oonceptual frame-

work clearly described including concepts and rela-tionships?

6. Does the problem clearly link to and flow from thetheoretical or conceptual framework?

Design1. Is the design clearly stated?

Sample1. 1. Is the sample clearly identified?2. Is it clear how the sample will be obtained?3. Is the relationship between the sample and the tar-

get population clearly delineated?4. Is the rationale for the sample size provided?

Instrumentationt. Is it clear which instruments will measure which con-

cepts?2. Is the rationale for instrument selection acceptable?3. Is the reliability for each instrument described and

acceptable?4. Is the validity for each instrument described and

acceptable?

Procedure1. Are sufficient details provided in the procedure?2. Is the procedure written clearly?3. Does the procedure flow logically?4. Are al! steps of the procedure clearly stated?

Protection of Human Subjects1. Has the researcher provided sufficient protection of

human subjects?

Data Analysis and Results1. Is the data analysis section well organized?2. Is the statistical method used for analysis appropri-

ate for the research question{s) and/or hypothesisand level of measurement?

3. Are tables and graphs clearly labeled?4. Do the tables and graphs complement the text?

Discussion1. Does the discussion clearly flow from the data?2. Does the discussion place the study's findings in

context with what is already known?3. If a theoretical or conceptual framework is present-

ed, are the nature of the findings discussed in thecontext of the framework?

4. If the author presents interpretations of the findingsare these clearly distinguished as such?

5. Are justifications offered for the author's conclusions?6. Are study limitations provided?7. Are implications for practice and future research

delineated?

characteristics of the sample, while a correlational designassists the researcher in examining relationships betweenvariables (Polit & Beck, 2004). On the other hand, exper-imental designs involve three key components: (a) manip-ulation of the independent variable, (b) use of a controlgioup, and (c) randomization into groups (Polit & Beck,2004). Experimental studies use the most powerful designs.These designs allow the researcher to control for extra vari-ables that may interfere with the researcher's ability to tellif the measured effect was due to the manipulatioii of Iheindependent variable or due to interference from tlu- unde-sired extra variables. Quasi-experimental designs lack oneof the three key experimentaJ components.

The reader can anticipate the data analysis plan oncethe design is known. For example, if an experimentalstudy is planned, the reader can anticipate use of inferen-tial statistics such as t-tests for data analysis. If a descriptivestudy is planned, then descriptive statistics are anticipatede.g. means, modes, medians.

All studies have research questions. Research ques-tions goiide and direct the study. A well-developedresearch question includes the population and variables tobe studied. At the completion of the study the researchquestions should be answered. Correlational, quasi-exper-imental, and experimental studies also have hypotheses. Ahypothesis is a statement of the relationship between vari-ables predicted by the researcher.

Sampling plan. Thi.s section must clearly describewho was asked to participate in the study and how theywere identified, the characteristics of the target population(tlie population to which the findings are generalized), thesampling procedure, and the size uf the sample. An exam-ple of a well-developed sampling plan follows:

77ie sample included the first 50 participants agreeing to par-ticipate who were on the deceased donor renal transplantationwaiting list at a university-affiliated hospital in the Midwest(Russell & Brown, 2002, p. 202).

Instrumentation. The instruments used to gather datafor the study must be clearly and thoroughly described.The researcher should delineate which concepts eachinstrument will measure. Instrumentation may involveinterviews, questionnaires, scales, observation, and/or bio-physiological measures. Reliability and validity datashould be reported for each instrument. Reliability is theinstilment's ability to accurately and consistently measurethe concept (Brink & Wood, 2001). Validity is the instniment's ability to measure what it is supposed to measure(Brink & Wood, 2001). The following is an example of awell-developed description of an instrument:

Depression was measured using the Beck Depression Inventory(BDI) (Beck et al., l!)(il). This 21 item self-administered, self-report scale addresses mood, pessimimi, sense of failure, lack ofsatisfaction, guilty feeling, sense of punishment, self-hate, self-accusation, self- punitive wishes, crying spells, irritability, socialwithdrawal, indedsiveness, body image, work inhibition, sleepdisturbance, fatigability, loss of appetite, iveight, loss, somaticpreoccupation, and loss of libido. The BDI has high internal con-

62 NEPHROLOGY NURSING JOURNALm January-February 2005 32, No. 1

sistency with ranges from .73 to .92 with a mean of .86Steer, & Garbin, 1988). The BDI has a split- half reliabilityco-efficient of.93 (Beck et ai., 1988) (Russell & Brown, 2002).

Procedure. The procedure should be the "recipe" forthe reseaich process with sufficient details provided so thatyou can easily follow the process. The procedure should bewritten very clearly and flow logically. All steps of the pro-cedure should be described fially. An example follows.

Participants were randomly assigned to either the controlgroup or the treatment group. Those placed in the control groupreceived no intervention phone calls or mailings, which was thecurrent .standard of care. Those randomized into the treatmentgroup received support, which included phone calls and mailings,once every month for six months. Because the current averagewaiting time at the institution was 8 months for blood ̂ oup A,and longer for other blood groups, a six month intervention wasselected. During the phone calls, patients were asked if they hadany questions or concerns that they woutd like to ask about wait-ing on the transplant list. TTie researchers documented key wordsand phrases stated by the subjects in response to the questions. Themailings were spomored by Signature Pharmaceuticals. This pro-gram involved sending cm initial welcoming letter and subsequentnewsletters which provided information on pertinent transplanta-tion issues such as medications, diet, exercise, organ allocation,waiting times, and current media topics. A web site, which couldbe accessed for information, was also provided by Signature. Boththe control and treatment groups completed the Herth Hope Index(HHI) and the MisheVs Uncertainy in Illness for Adults Scale(MUIS- A) at the beginning of the study and six months later(Russell & Brown, 2002, p. 2()H).

Protection of human subjects. The researcher shouldstate how protection of human subjects was assured. Thestudy should have been reviewed and approved by anInstitutional Review Boaid and that should be stated in thereport. Most often the approval is from the institution wherethe researcher is employed. However, if the study is con-ducted in multiple settings., each setting should provideapproval and this should be noted by the investigator.

Data Analysis and ResultsThis section is often the most intimidating for begin-

ning reviewers. However, several steps can make theprocess manageable. Confirm that the researcher has pre-sented results that clearly answer the proposed researchquestion(s). Researchers frequently organize this sectionby research question to facilitate readability. Since thedesign can assist you in determining the appropriate sta-tistics to anticipate, review the design again. If a descrip-tive design is used, you should find descriptive statisticssuch as mean, mode, and median (which measure how thedata tend to be similar or grouped together) and variance,standard deviation, and range (which measure how thedata tend to be spread out). If the design is correlational,then you should anticipate a correlation coefficient, suchas Person r or Spearman rho. A correlation coefficientidentifies the strength and the direction of the relationshipbetween two variables (Holcomb, 2002). An example of

descriptive statistics follows:The sample consisted of 35 males (70%) and 15 females

(30%). The mean age was 48.5 years (SD = 12.6, range 20-70). Participants had an average of 12.5 years of education (SD= 2.52y range 5-20). Eighty-four percent were Caucasian and70% were currently married. The average months since diagnosisof end-stage renal disease was 52.3 (SD = 74.18, range 1-300).ITie average number of days waiting for transplant was 450.4(SD = 1084, range 1-4752) (Russell & Brown, 2002, p. 20H).

If the design is experimental or quasi-experimental,you should anticipate the use of inferential statistics.Inferential statistics answer questions about relationshipsbetween variables and differences between groups(Holcomb, 2002). An outstanding quick reference guidefor assessing an author's appropriate use of statistics basedon the research question and level of measurement isavailable (Ryan-Wengar, 1992). Many studies set the levelof statistical significance as p<.05, the chance of making aType I error in 5 or fewer tests out of 100 tests. Severaltexts provide further discussions of type 1 and type Uerrors (Polit & Beck, 2004). An example of a results sec-tion on inferential statistics follows:

The first research question addressed was: What effect does thenursing intervention of providing information and support haveon the levels of hope and uncertainty in individuals awaitingrenal transplantation between, the treatment and control groups?No statistically significant effect of the nursing intervention wasfound on hope and uncertainty in this sample using Hotel/ing's 7^statistic- (F= 0.5322;p = 0.81) (Russell & Brown, 2(K)2, p. 203).

Tables and graphs are frequently used to summarizeresearch results. The tables and graphs should be clearlylabeled and should complement the article text.

DiscussionThe di.scussion section should clearly flow from the

data and place the study's findings in context with what isalready known. If a theoretical or conceptual frameworkis presented, the nature of the findings should be discussedin the context of the framework. The author may ofleri nteip re ta tions of the findings but these should be clearlyidentified as such. Based upon the logical How of this sec-tion, a determination must be made regarding the justifi-cation of the author's conclusions. The author should pre-sent the limitations of the study. Implications for practiceand future research must be delineated.

SummaryAs a novice reviewer, it is often difficult to trust your

evaluation of a research report. You may feel uncertain inyour interpretations. These are common concerns andcan be remedied by reading and discussing researchreports on research listservs, through journal clubs, or withother nephrology nurses. Practice using the criteria forresearch report evaluation and you too can perfect cri-tiquing a research report! '

NEPHROLOGY NURSING JOURNAL « January-February 2005 « VoL 32. No. 1

ReferencesAlderman, S. (t;)9H). Critiquing research for use in ciiuica! nurs-

ing practice: A CD-ROM review. Nurse Educator, 23{'2), 8.Beck, A.T., Steer, R.A., & Garbin, M.G. (1!)H8). Psychometric

properties of the Beck Depression Inventory: Twenty-fiveyears of evaluation. Clinical Psychology Review, 6^1), 77-100.

Beck, A.T., Ward, C.H., Mendelson. M., Mock,J.. & Erbaugh,J.(liHii). An inventoiy for measuring depression. Archives ofGeneral Psychiatry, < rii\\<>7\.

lieck, C.T. (!!)f)()). The research critique: General criteria forevaluating a research report. fOGNN, 75(1), 18-22.

Beyea. S.C. (I!)*)H). Critiquing research for use in clinical nursingpractice (CD-ROM). Computers in Nursing, ?6{1), lfi-IZ

Brink, PJ., & Wood, MJ. (2001). Basic steps in planning nursingresearch (5ih ed.). Boston: Jones and BarlleU l\iblishers.

Hoicomb, Z.C. (2002). Interpreting basic .statistics (3rd ed.). LosAngeles: Pyrczak Publishing.

Pieper, B. (l̂ O^ )̂. Basics of critiquing a research article. Joumai ofET Nursing, 20, 24.V 2S0.

Polit, D., & Beck, C.T. (2004). Nursing research: Principles and meth-ods (7th cd.). Philadelphia: Lippincott, Williams, & Wilkins.

Russell, C.L, & Brown. K (2(K)2). The efFecls of information and sup-port on individuals awaiting transplant. Process in Transptaniaiimi,72(3), 201-207

Ryan-Weng-ar, N.M. (1J)!)2). Guidelines for critique of a researchreport. Heart & Lung, 21(4), 35H-4O1.

Soeken, K.L. (U)85). Critiquing research: Steps for completeevaluation on an article. AORNJoumal, 41[5\ 882-893.

Stimmers, S. (li)SM). Defining components of the research processneeded to conduct and critique ^\ixdie%. Joumai of Post AnesthesiaNursing, 6(1), 50-55.

DIALYSIS JOBS...Your Lift.".

Your Career...Dedicated e.xdu.siveh lo ilw rccruiimeni and placement of(Ualysis. transplantation and reimbursenwnt professionals

throu^houl (he global nephrology and Inin.'iplantationindustries.

We also specialize in relocation.For more inforttutiion contact:

Michael Maurizit) ('esiir f'lizoit

Aiiiic Ciiacovi'lli LJ\.r\.\—i\ OlO Vicu>r Gertice

JamesMiiieo R E C R U I T I N G •-''''""/'''" ^'"Jeff Kozma Kelly Winder

SPECIALISTS.Amly Lawrence jt>h,i faon,

Nurses!^'Dialysis & Transplantation is our only busiDcss."

(315)768-3322 * Fax: (315)768-4349 * Toll Free: (888) 38-RENAL

v̂ 'W'w, dialysis recni iting.com

Make your job work for you.Join us and remember why you gotinto nursing in the first place.

What does it mean to be a Magnet Hospital? To be considered one of

the finest hospitals in New Jersey? We like to think it's a lot of things.

An environment, culture and a philosophy committed to the

advancement of the profession and the individual and one that fosters

continual growth, learning and development.The constant challenge

that comes with working with a diverse patient load. Respect.

Flexibility.To be honest, there are as many reasons as there are nurses,

and we invite you to find out for yourself.

Clinical Coordinator, HemodialysisOur Hemodjalysis program is one of the finest in New Jersey and in

ouf newly constructed units we treat both the adult and pediatric

populations affected by acute and chronic conditions.

We are seeking a coordinator to manage the overall activities of the

Hemodialysis unit. Must have 3-5 years of hemodialysis experience

and 2-3 years of supervisory experience, and BCLS/ACLS and CNN

certifications.

To find out what's been attracting some of the state's best nurses to

the magnet hospital known ds Morristown Memorial, contact us

today. Apply online at www.atlanticKealth.org or send your resume

to linda.yovanovich@ahsys.org. Equal Opportunity Employer.

^ ^ Morristown Memorial Hospitaly \ r i A M T ( c : H E A L T H S Y S T E F V I

www.atLantichealth.org

REGISTERED NURSES & HEALTH CARE PROFESSIONALS:

ACUTE ANDCHRONICDIALYSISFACILITIESNEED YOU.

WE'RE HERETO MAKE ITHAPPEN. -

Quik Travel Staffing

The f lexible, fr iendly, in-your-corner healthcaretravel employment company, offering temporaryand permanent assignments with chronic andacute Dialysis facil i t ies.You're serious about your career— and we'reserious about gett ing the best si tuat ion for you .

COMMIHED TO TOP SALARIES & GREAT BENEFITSCOMPLETION & REFERRAL BONUSESFREE HOUSING, TRAVEL & LOCAL TRANSPORTATIONPERSONALIZED, ROUND THE CLOCK SERVICE

810-454-2230

NEPHROLOGYNURSINGJOURNAL January-February2005 Vol. 32. No. 1