7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 1/122
Prepared by:
Lucky P. Roaquin, RN, MAN
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 2/122
At the end of the course and given actual situation, agroup of student will be able to:
(COGNITIVE DOMAIN) To describe the meaning of research and its
importance to nursing practice
To discuss the different techniques in data analysis of
the different research approaches To differentiate the different ways of interpreting data
gathered during the implementation process
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 3/122
At the end of the course and given actual situation, agroup of student will be able to:
(AFFECTIVE DOMAIN) To initiate appreciation on the topics provided
To enhance self-esteem through rigorous training onnursing research
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 4/122
At the end of the course and given actual situation, agroup of student will be able to:
(PSYCHOMOTOR DOMAIN) To create a competitive research project
To generate an effective evidenced-based researchimportant in the nursing profession
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 5/122
Preliminary Period
A. The Empirical Phase
- Measurement and the Assessment of QuantitativeData
- Reliability Test of Instruments
- Validity Test of Instruments
- Specificity & Sensitivity
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 6/122
The assignment of numbers to objects according tospecified rules to characterize quantities of someattribute
The purpose of assigning numbers is to differentiatebetween people with varying degrees of attribute
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 7/122
Advantages:
1. Removes subjectivity and guesswork
2. Obtains reasonably precise information
3. Language of communication
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 8/122
Errors of Measurement The difference between true and obtained scores
= ± where:
is the observed score is true component
is error component
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 9/122
Errors of Measurement
Common Influences:
a) Situational contaminants – conditions under whichthey are produced.
b) Transitory personal factors – by such personal statesor people’s motivation to cooperate, act naturally or
do their bestc) Response-set biases – enduring characteristics of
people
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 10/122
Errors of Measurement
Common Influences:
d) Administration variations – alterations in themethods of collecting data
e) Instrument clarity – poorly constructed directions
f) Item sampling – results from sampling of items
g) Instrument format – technical characteristics of aninstrument
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 11/122
Reliability of a quantitative instrument is a majorcriterion for assessing its quality
Reliability is the consistency with which it measuresthe target attribute and concerns accuracy that reflectstrue scores
Reliable measures maximize the true scores andminimize the error
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 12/122
Aspects of Reliability Testing
1. Stability
2. Internal consistency
3. Equivalence
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 13/122
Stability Extent to which the similar scores are obtained on
separate occasions.
It is how reproducible the findings would be if the samemeasurement were repeatedly made on the same object
Assessment of stability involve procedures that evaluatetest-retest reliability (researchers administer the same
measure to a sample twice and then compare thescores).
Reliability coefficient is an index of the magnitude of
the test’s reliability
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 14/122
Stability Correlation coefficient seeks to detect and to explain
relationship among phenomena
Perfect relationship if two variables are obviously related (r=1.00)
Positive relationship if relationship exists but notperfect (between r=0.00 to 1.00)
Negative relationship if two variables are inversely related (between r=0.00 to -1.00)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 15/122
Stability The higher the coefficient, the more stable the measure
Reliability coefficients above .80 usually are consideredgood.
Test-retest measure can be influenced by experiencesbetween tests. And Stability estimates can be influenced
by memory of initial responses.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 16/122
Internal Consistency The degree to which the subparts of a composite scale
are all measuring the same attribute or dimension
It requires one administration
Best means of assessing an especially important sourceof measurement error in sampling of items
The most widely used method for evaluating internalconsistency is coefficient alpha (Cronbach’s alpha)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 17/122
Internal Consistency Cronbach’s alpha estimates the internal consistency of
a composite measure composed of several subparts
Cronbach’s alpha can be interpreted like other reliability coefficients: 0.00 and +1.00, higher values reflect higherinternal consistency
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 18/122
Equivalence Primarily concerns the degree to which two or more
independent observers or coders agree about scoring
If there is a high level of agreement, then theassumption is that measurement errors have beenminimized
Number of agreement
Number of agreement + disagreements
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 19/122
Interpretation of Reliability Coefficients Important indicators of an instrument’s quality
Coefficients in the vicinity of .70 may be adequate butcoefficients of .80 or greater are highly desirable
Reliability is the proportion of true variability to thetotal obtained variability
=
It is clearly why instruments with reliability lower than.70 is risky to use.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 20/122
Factors Affecting ReliabilityHomogenous of sample
Items with no discriminating power
Number of items
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 21/122
Tips: More items tapping the same concept should be added
Greater precision in defining categories or greater clarity inexplaining the underlying construct for rating scales
Heterogeneity of the sample
Know the characteristics of the group with which it was
developed
In selecting an instruments, researchers need to determine which aspect of reliability is relevant
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 22/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 23/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 24/122
Types of Validity Tests
1. Face
2. Content
3. Criterion-Related
4. Construct
5. Convergent & Discriminant
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 25/122
Face Validity• Refers to whether the instrument looks like it is
measuring the target construct
• Not strong but helpful for a measure to have face validity if other types of validity have also been demonstrated
• It is easier to persuade people to participate in the study
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 26/122
Content Validity Concerns the degree to which an instrument has an
appropriate sample of items for the construct beingmeasured and adequately covers the construct domain
Relevant for both affective and cognitive measures
Based on judgment but it is common to use a panel of
experts to evaluate the content validity of newinstruments
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 27/122
Content Validity Content validity index (CVI) is an index of the degree
to which an instrument is content valid, based onaggregated ratings of a panel of experts
Experts are asked to evaluated individual items oroverall instrument
Two issues: (1) individual items are relevant andappropriate; and (2) items taken together adequately measure all dimensions
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 28/122
Content Validity Item CVI (I-CVI) is computed as the number of experts
giving a rating then divided by the number of experts.0.80 is considered an acceptable value
2 approaches in determining the Scale CVI (S-CVI): (a)universal agreement and (b) averaging approach
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 29/122
Criterion-Related Validity• The degree to which scores on an instrument are
correlated with some external criterion
• Key issue: whether the instrument is a useful predictorof other behaviors, experiences or conditions
• Requirement: availability of a reliable and valid
criterion with which measures on the instrument can becompared
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 30/122
Criterion-Related Validity• Criterion-related validity coefficient is computed by
correlating scores on the instrument and the criterion
•
Types: (1) predictive validity and (2) concurrent validity
• Predictive validity refers to the adequacy of aninstrument in differentiating between people’sperformance on a future basis
• Concurrent validity reflects an instrument’s ability todistinguish individuals who differ on a present criterion
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 31/122
Construct Validity • Key criterion for assessing the quality of a study
• Concerns inferences from study particulars to higher-
order constructs
• Hypothesis-testing that is linked to a theoreticalperspective about the construct
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 32/122
Construct Validity 4 Approaches in Construct Validity
1. Known Groups
2. Hypothesized Relationships
3. Convergent and Discriminant Validity
4. Factor Analysis
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 33/122
1. Known-Groups Technique Yields evidence of contrast validity
The instrument is administered to groups hypothesizedto differ on the critical attribute because of a known
characteristic
E.g.
Measurement of fear between primiparas andmultiparas
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 34/122
2. Hypothesized Relationships Involves testing hypothesized relationships, often on the
basis of theory or prior research
Involves hypotheses about the relationship between the
measure of the construct and a variable representinggroup membership
E.g.
According to theory, construct X is positively related to construct Y
Instrument A is a measure of construct X; instrument B is a measure of construct Y
Scores on A & B are correlated positively, as predicted
Therefore, it is inferred that A & B are valid measures of X & Y
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 35/122
3. Convergent and Discriminant Validity Convergence – is evidence that different methods of
measuring a construct yield similar results.
Discriminability – the ability to differentiate the
construct from other similar constructs. Multitrait-multimethod matrix method – significant
validation tool
The point of including two concepts in a single validation study is to gather evidence that the twoconcepts are distinct, rather than two different labels forthe same underlying attribute
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 36/122
4. Factor Analysis A method for identifying clusters of related variables
that is, dimensions underlying a broad construct
It is used to identify and group together different itemsmeasuring an underlying attribute
Constitutes another means of testing hypotheses about
the interrelationships among variables, and for lookingat the convergent and discriminant validity of a large setof items
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 37/122
Interpretation of Validity An instrument’s validity is not proved, established or
verified but rather is supported to a greater or lesserextent by evidence
Researchers do not validate an instrument but rather anapplication of it.
The more evidence that can be gathered that aninstrument is measuring what it is supposed to bemeasuring, the more confidence will have its validity.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 38/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 39/122
Specificity and sensitivity are criteria that areimportant in evaluating instruments used as screeningor diagnostic tools (e.g. a scale to measure risk of osteoporosis)
Screening or diagnostic instruments can be self-report,observational, or biophysiologic measures
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 40/122
SensitivityThe ability of a measure to identify a “case” correctly to
screen in or diagnosis a condition correctly
Results yield to “true positives”
SpecificityMeasure’s ability to identify noncases correctly to screen
out those without the condition
Results yield to “true negatives”
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 41/122
Predictive values – the probability of an outcome afterthe results are known
Positive predictive value (PPV) – proportion of people with a positive result who have the target outcome ordisease.
Negative predictive value (NPV) – proportion of people
who have a negative test result who do not have thetarget outcome or disease.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 42/122
Likelihood Ratio It summarizes between specificity and sensitivity in a
single number
Addresses the question, “How much more likely are weto find that an indicator is positive among those with theoutcome of concern compared to those for whom theindicator is negative?”
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 43/122
URINARY COTININE LEVEL
SELF-REPORTED
SMOKING
Positive(Cotinine >200ng/ml
Negative(Cotinine ≤200ng/ml
Total
Yes, smoked A (true positive)20
B (false positive)10
A + B30
No, did notsmoked C (false positive)20 D (true negative)50 C + D70
Total A + C40
B + D60
A+B+C+D100
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 44/122
Sensitivity = A/(A+C) = .50Specificity = D/(B+D) = .83
PPV = A/(A+B) = .67
NPV = D/(C+D) = .71LR+ = sensitivity/(1-specificity) = 2.99
LR- = (1-sensitivity)/specificity = .60
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 45/122
Receiver Operating Characteristic Curve (ROC curve) A method used in developing and refining a screening
instrument to determine the best cut-off point for
“caseness” The sensitivity of an instrument is plotted against the
false-positive rate over a range of different scores
Screening instrument is excellent if the points crowd to
the upper left corner. Area under the curve – the overall accuracy of an
instrument that can be calculated as the proportion of the area under the ROC curve
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 46/122
Receiver Operating Characteristic Curve (ROC curve)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 47/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 48/122
Efficiencyo Reduces participant burden
o Diminishes number of items in a scale
o Spearman-Brown formula – estimate how reliable thescale would be with fewer items
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 49/122
Efficiency
E.g. Consider a 40-item scale to measure social supportthat has internal consistency reliability 0f 0.94to be
shorten in 30-item scale
=
1 + [ − 1 ]
Where k is the factor by which the instrument is beingdecreased; r is the reliability for the full scale; and isthe reliability for the shorter scale
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 50/122
Efficiency
Activity:
Solve the
for an 80-item scale to measureanxiety among patients undergoing dialysis that has aninternal consistency of .96 to be shorten in 65-item scale
= 0.95
Is it possible to shorten the full scale to a 65-item? Why? Why not?
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 51/122
Efficiency
Activity:
Solve the
for an 95-item scale to measure thehealth index of elderly that has an internal consistency of .79 to be shorten in 70-item scale
= 0.73
Is it possible to shorten the full scale to a 65-item? Why? Why not?
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 52/122
Comprehensibility – participants and researchersshould be able to comprehend the behaviors requiredto secure accurate and valid measures
Precision – an instrument should discriminatebetween people with different amounts of an attributeas precisely as possible
Range – the instrument should be capable of achievinga meaningful measure from the smallest expected value of the variable to the largest
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 53/122
Linearity – a researcher normally strives to constructmeasures that are equally accurate and sensitive overthe entire range of values
Reactivity – the instrument should, avoid affecting theattribute being measured
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 54/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 55/122
Let’s do some
brain damaging!
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 56/122
Enumerate the advantages of measurement and
explain why it is important in doing research.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 57/122
Give at least 5 common influences on
measurement error and provide a concreteexample from each influence.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 58/122
What is reliability? Cite an example wherein
reliability is of significant in doing research.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 59/122
Enumerate the different levels of measurement.
Give actual examples from each levels.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 60/122
Discuss stability.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 61/122
How is validity done in evaluating instrument?
Enumerate the 4 types of validity testing.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 62/122
Discuss equivalence. Explain the formula used
and cite an example.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 63/122
What is coefficient alpha? Discuss the use of
coefficient alpha in determining the reliability ofan instrument.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 64/122
Differentiate face validity from content validity.
Discuss the two approaches to calculate contentvalidity.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 65/122
Explain specificity and sensitivity in evaluating
quantitative instruments. Illustrate an example indetermining specificity and sensitivity of an
instrument.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 66/122
Discuss construct validity and the different ways
to gather evidence about construct validity
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 67/122
Enumerate and explain the other criteria for
assessing quantitative measures.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 68/122
Enumerate the factors that affect reliability and
ways to minimize these factors.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 69/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 70/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 71/122
The Steps in developing high-quality self-report scales:1. Focus on structured self-reported measures rather
than the observational ones.
2. Describe methods of developing multi-item scales
3. Exclude infrequently used scale types, such as
semantic differentials
4. Focus on classical measurement theory rather thanitem response theory
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 72/122
1. Conceptualizing the Construct
Latent variable – an unmeasured variable thatrepresents an underlying, abstract construct which is the
cause of the scores in measurement theory
The strength of the latent variable is presumed to triggera certain numeric value on the scale
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 73/122
1. Conceptualizing the Construct
Be an expert on the construct – first step in scaledevelopment
Identify and understand the different dimensions of theconstruct
Take account of the related constructs from the targetconstruct
Explicit conceptualization and understanding of thepopulation for whom the scale is intended
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 74/122
2. Deciding on the Type of Scale
Two categories of scales: Likert Scales and latent traitscales
Likert scales – declarative statements with a bipolarresponse scale that is often on an agree/disagreecontinuum
Latent trait scales – a scale developed within an item
response theory framework.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 75/122
2. Deciding on the Type of Scale
E.g. Developing a scale to measure risk-taking behaviorin adolescents.
Rasch model – the model that considers item difficulty in assessing items for the scale
In a CMT scale, theitems might include
statements about risk-taking of similar
intensity, with which
the respondents wouldagree or disagree.
In IRT scale, the itemsthemselves would be
chosen to reflectdifferent levels of risk-
taking such as smoking,
using drugs, driving while texting.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 76/122
3. Developing an Item Pool: Getting Started
Items need to be carefully crafted to reflect the latent variable they are designed to measure
Look for possible sources
Existing instruments – adaptation of an existinginstrument which require adding or deleting items
or may involve rewording of items.
The literature – this point, your would already be an“expert” on the construct, this is an obvious source
of ideas for items
Concept Analysis – clarification of conceptualdefinition that will facilitate conceptual clarity
among nurse researchers
In-depth qualitative approach – qualitative research will help understand the dimensions of a
phenomenon, and can give actual words for items
Clinical Observations – patients in clinical settingsmay be an excellent source of items
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 77/122
3. Developing an Item Pool: Getting Started
Devellis (2003) urged scale developers to get started writing scale items without a lot of editing and critical
review in the early stage.
The best way to begin is to develop a simple statement with the key construct mentioned in it
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 78/122
4. Making Decisions about Item Features
Make decisions about such as issues as the number of items to develop, the number and form of the response
options, whether to include positively and negatively worded items, and how to deal with time.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 79/122
4. Making Decisions about Item Features
Number of Items
•
Domain sampling model – the model used indeveloping a scale in the classical measurement theory framework, which involves the random sampling of ahomogenous set of items from a hypothetical universeof items relating to the construct
• The idea is to generate a fairly exhaustive set of itempossibilities, given the construct’s theoretical demands
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 80/122
4. Making Decisions about Item Features
Number of Items
•
Redundancy is good in order to measure the constructof interest with a set of items that capture the centraltheme in slightly different ways so that irrelevantidiosyncrasies of individual items will cancel eachother out
• Generate a very large pool of items
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 81/122
4. Making Decisions about Item Features
Response Options
•
The goal is to array people on a continuum where variability is essential through offering numerousresponse options
• Numbers are used with verbal descriptors to facilitate
coding
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 82/122
4. Making Decisions about Item Features
Positive & Negative Stems
•
The objective was to include items that wouldminimize the possibility of an acquiescence responseset – the tendency to agree with statements
• Acquiescence can be minimized by putting the most
positive response options at the end of the list ratherthan at the beginning
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 83/122
4. Making Decisions about Item Features
Item Intensity
•
The intensity of the statements should be similar andfairly strongly worded
• There is no point in including items with which almosteveryone would either agree or disagree
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 84/122
4. Making Decisions about Item Features
Item Frames
•
Items should be worded in such a manner that every respondent is answering the same question
Clarity – scale developers should strive
for clear, unambiguous items. Wordsshould be carefully chosen with theeducational and reading of the target
population.
Jargon – be cautious about using terms
that might be well-known in healthcarecircles but not familiar to the average
person.
Length – avoid long sentences or phrases.Simple sentences are easiest tocomprehend
Double negatives – it is often preferableto word things affirmatively, but doublenegatives should be avoided.
Double barreled – avoid putting two ormore ideas in a single item.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 85/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 86/122
1. Internal Review
Individual items should capture the construct,and are grammatical and well worded
Assess the scale’s readability (the ease with which the materials can be read by people with varying reading skills, often evaluated throughreadability formulas such as Flesch-Kincaidgrade level score)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 87/122
1. Internal Review
A general principle is to avoid long sentences and words with four or more syllabus
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 88/122
2. Input from the Target Population
Initial pretest the set of items from the targetpopulation with a 10-20 sample
Cognitive questioning is an excellent techniquefor discovering how others process the wordsand ideas presented to them in structuredquestions.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 89/122
2. Input from the Target Population
It is a good idea to peruse the pretest answers tosee if response patterns suggest the need for
item revisions
Focus groups can also be used to determine theclarity of the items, linguistically and culturally appropriateness, inoffensive and relevant toconstruct.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 90/122
3. External Review by Experts
External review by a panel of experts should beundertaken to assess the scale’s content validity
Selecting and Recruiting the Experts• People with strong credentials such as
experts on clinical or personal experience,
published papers, or ongoing researchprogram on the topic.
• Should be knowledgeable about the key construct and the target population
Content Validation of Items• The panel is asked to rate the dimensions of
each items that includes clarity of wording,
relevance of the items and appropriatenessfor the target population
• Relevance is most often used
Content Validation of Items• Concerns an overall recommendation• To consider whether the items taken as a
whole adequately cover the constructdomain
• After agreement, ask for their opinions
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 91/122
3. External Review by Experts
External review by a panel of experts should beundertaken to assess the scale’s content validity
Content validation of the Scale• 2nd round of content validation that is
composed of 3-5 experts from new set of experts or from the subset of the 1st panel• S-CVI is then computed
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 92/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 93/122
Undertake a quantitative assessment of the items, which requires that they be administered to a fairly large development sample.
Important steps include the development of samplingplan and data collection strategy.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 94/122
1. Developing a Sampling Plan
The sample should be a representative of the population
Recruit a sample from multiple sites to enhancerepresentativeness and to assess geographic variation ininterpreting the items
Having a sufficiently large sample is essential to ensurestability in the covariation among the items
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 95/122
2. Developing a Data Collection Plan
Choose an approach that best approximates how thescale typically would be administered after it is finalized
Include measures to assess response biases, especially social desirability.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 96/122
3. Preparing for Data Collection
Care should be taken to make the instrument attractive,professional looking, and easy to understand.
SOs may be asked to evaluate the appearance of theinstrument before it is reproduced
Avoid proximity effect in order minimize artificially inflate estimates of internal consistency
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 97/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 98/122
1. Basic Item Analysis
The performance of each item on the preliminary scaleneeds to be evaluated empirically to determine the
correlation of items
The degree of inter-item correlation can be assessedby inspecting the correlation matrix of all the items.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 99/122
1. Basic Item Analysis
Next is to compute for the item-scale correlations(computation of preliminary total scale scores and then
calculate the correlations between individual items andtotal scores on the scales they’re intended to represent)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 100/122
2. Exploratory Factor Analysis
Factor analysis disentangles complexinterrelationships among items and identifies items that
“go together” as unified concepts
Factor analysis offers an objective, empirical methodof clarifying the underlying dimensionality of a large set
of measures
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 101/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 102/122
A. Factor Extraction
Condenses items into a smaller number of factors andused to identify the number of underlying dimensions
Estimates of initial factors are obtained using principal components analysis (the most commonly usedextraction method)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 103/122
A. Factor Extraction
In principal components analysis, linear combinationsof the observed variables are formed.
The 1st principal component is the combination thataccounts for the largest amount of variance in thesample (1st extracted factor).
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 104/122
A. Factor Extraction
The 2nd principle component accounts for the nextlargest amount of variance and is uncorrelated with
the first (2nd extracted factor).
Successive components explain progressively smallerportions of the total sample variance, and all are
uncorrelated with each other.
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 105/122
A. Factor Extraction
To decide on how many factors we need to representthe data, we use 2 statistical criteria:
Eigenvalues (sum of the squared item weights for thefactor)
The Scree Test ( a sharp drop in the percentage of explained variance that indicates the appropriatetermination point)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 106/122
The determination of the number of factors isusually done by considering only factors
with Eigen valuesgreater than 1.
Factors with a varianceless than 1 are no betterthan a single variable,since each variable isexpected to have a
variance of 1.
Total Variance Explained
Comp
onent
Initial Eigenvalues Extraction Sums of Squared
Loadings
Total % of
Variance Cumulativ
e % Total % of
Variance Cumulativ
e % 1 3.046 30.465 30.465 3.046 30.465 30.465 2 1.801 18.011 48.476 1.801 18.011 48.476 3 1.009 10.091 58.566 1.009 10.091 58.566 4 .934 9.336 67.902 5 .840 8.404 76.307 6 .711 7.107 83.414 7 .574 5.737 89.151 8 .440 4.396 93.547 9 .337 3.368 96.915 10 .308 3.085 100.000 Extraction Method: Principal Component Analysis.
Eigenvalues
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 107/122
The examination of the Scree plot provides a visual of the total variance associated witheach factor.
The steep slope shows the large factors.
The gradual trailing off (scree) shows the restof the factors usually lower than an Eigen value of 1.
In choosing the number of factors, inaddition to the statistical criteria, one should
make initial decisions based on conceptualand theoretical grounds.
At this stage, the decision about the numberof factors is not final.
Scree Test
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 108/122
B. Factor Rotation
In this step, factors are rotated
Un-rotated factors are typically not very interpretable(most factors are correlated with may variables)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 109/122
B. Factor Rotation
Factors are rotated to make them more meaningful andeasier to interpret (each variable is associated with a
minimal number of factors)
Different rotation methods may result in theidentification of somewhat different factors
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 110/122
B. Factor Rotation
The most popular rotational method is Varimaxrotations
Varimax use orthogonal rotations yielding uncorrelatedfactors/components
Varimax attempts to minimize the number of variablesthat have high loadings on a factor. This enhances theinterpretability of the factors
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 111/122
B. Factor Rotation
Other common rotational method used include obliquerotations which yield correlated factors
Oblique rotations are less frequently used because theirresults are more difficult to summarize
Other rotational methods include:Quartimax (Orthogonal)
Equamax (Orthogonal)
Promax (oblique)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 112/122
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 113/122
3. Making Final Decision
The final decision about the number of factors to chooseis the number of factors for the rotated solution that is
most interpretable
To identify factors, group variables that have largeloadings for the same factor
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 114/122
3. Making Final Decision
Plots of loadings provide a visual for variable clusters
Interpret factors according to the meaning of the variables
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 115/122
3. Making Final Decision
Plots of loadings provide a visual for variable clusters
Interpret factors according to the meaning of the variables
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 116/122
Assumption underlying factor analysis include:
The measured variables are linearly related to thefactors + errors This assumption is likely to be violated if items limited
response scales (two-point response scale likeTrue/False, Right/Wrong items)
The data should have a bi-variate normaldistribution for each pair of variables
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 117/122
Assumption underlying factor analysis include:
Observations are independent
The factor analysis model assumes that variables aredetermined by common factors and unique factors. All unique factors are assumed to be uncorrelated with each other and with the common factors
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 118/122
3. Scale Refinement & Validation
Revising the Scale
The analyses undertaken in the development study oftensuggest the need to revise or add items
Examine items with high factoring loading when addingnew items
Examine the content of the items remaining in the scale
Scoring and Transforming the ScaleItem scores are typically just added together to form asubscales scoring of items or to form a total scores
Adding together of items implies equally importantindicators of latent variable.
Development of construct scales that have equalnumber of items per subscale
Conducting a Validation Study Include the worth of instrument in a validation study
Confirmatory factor analysis (CFA) is a hypothesistesting approach to determine where the items belongsfrom a specific factors
Establishing Cutoff Points
Produce dichotomous of scale construct along thecontinuum
There must be a score that distinguishes those who passand those who fail
Highly intended for diagnostic and screening scales
Establishing Norms
This occurs if the expectation is that (a) the scale will be widely used by the population and help evaluate theirscores, and (b ) average scale scores vary markedly by members of well-defined subpopulations
Norms are most commonly established for key demographic characteristics such as age and gender
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 119/122
4. Translating Scales into Other Languages
Well-constructed scales with good psychometricproperties are increasingly likely to be translated for use
of other culture
Translations are often centered (original language) ordecentered approach (allow modification to the
wording of items in the original scale)
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 120/122
4. Translating Scales into Other Languages
Both conceptual equivalence and semanticequivalence are critical to the success of a translated
effort.
The “gold standard” for semantic equivalence involvesback-translation, in which the scale is first translated
from the source language into the target population, andthen translated back to the source language
7/28/2019 Nursing Research II-Prelims
http://slidepdf.com/reader/full/nursing-research-ii-prelims 121/122
4. Translating Scales into Other Languages
The translated version is then tested in a manner of similar to the original scale.
Evidence for semantic equivalence and psychometricsoundness comes from pre-tests of both original andtranslated scale with a sample of bilingual people, and
comparison of reliabilities, factor structures, and other validity estimates between the two scales