View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Qualitative Statistics
Observation
• Can be used to infer correlation, but not causation– Therefore use different statistical tests• Experiments-look to see how different two things might
be• Observational Studies-look to show how closely two
variables relate to one another
• Clarity of what is being observed is essential
Types of Observation• Laboratory Observation
– Conducted in lab setting for control– Allow for ease in conducting, lower validity
• Ethological Observation– In natural environment– High validity, hard to conduct (observation can effect behavior)
• Participant Observation– Joining the group you are studying– Ethically questionable/Hard to keep records
• Epidemiology– Used most often in medical and environmental health research– A form of indirect observation using complicated statistical analysis to
find correlations
Collecting Data in Observational Research
• 2 difficulties in collecting this data– Selective Perception
• We focus on things we are interested in and lose a great deal of info in the process
– Anticipatory Schema• We also must deal with our tendency to look for that which we
already expect to see
• Precautions to avoid these pitfalls– Structure form of observation carefully
• Behavioral Coding• Time-Sampling
Behavioural Coding
• A system that uses a series of categoried for behavior, and fitting all observations that are made into one of these categories
• Usually worked out before starting through similar studies or pilot tests
• Categories must be reasonably comprehensive, but also limited for managing purposes
• Video recording can be done to allow coding to be done later, but camera does not pick up all that the eye can
Time-Sampling
• Often used with behavioural coding to record what is happening with a group over a period of time
• Set intervals are assigned and observations are only made during these intervals
• Can give a reasonably accurate picture of typical activities
Multiple Observers
• Uses independent observers• 2 advantages
– One observer may notice something another has missed– Allows researcher to measure the amount of agreement
between observers• Any data both agree on can be seen as reasonably valid
• Can also measure inter-observer reliability• Correlation between reports of different observers
– Above .8 reliable and acceptably accurat– If lower could be for 2 reasons
» Too much going on for one person to take in» Observers influenced by personal bias or expectations
Ethics of Observational Research
• Should always have full consent of person being observed• Feedback about observations purpose and findings needs
to be provided• Covert Studies need to follow ethical principles
– Utilitarian, cost-benefit criteria– The effects of the participants– Issues of individual integrity
• Passive Observations must be understood to have some effect on behavior– Active research can help to alleviate some of the behavioral
changes
Questionnaire Studies
• Allow researcher to collect info from large numbers of people
• Used as a starter for many studies to help generate Hypotheses
• Can be used to determine how widespread a phenomena is
• Vulnerable to response bias-people say what they feel is the right answer
Uses of Questionaires
• Info on– Facts and Knowledge– Past Behavior– Likely Future Behavior– Motives– Opinions and Attitudes
Design
• A good questionnaire must – Account for differences in population– Make itself crystal clear to every member
• Need to expect the unexpected response and design with that in mind
• Avoiding questionnaire fallacy is also important– Assumption you have found answer to behavior
when it may be limited scope of options to choose
Stages of Design• Working out the aims
– What is main point and hypothesis• Selecting the appropriate question styles
– How free is the response?– Does it fit analysis needs?
• Designing the questions– Need to be structured and phrased with aims in mind
• Piloting – Check for lack of clarity
• Revising– Go for perfection and check to make sure you can score appropriately
• Administering– Choose most suitable technique for topic you are researching
• Analysis– Report all valid statistical data, both Quantitative and Qualitative
• Reporting– Standard format
Common Mistakes
• Begin by writing out questions• Making the scope to broad– Creates indifference in respondents
• Not understanding purpose for analysis before administering
Designing Questions• Sequencing
– Arranging common questions together is important• Priming helps respondents think more clearly
– Funneling-Broad to Specific– Reverse Funneling-Specific to Broad
• Writing– Issues
• Leading Questions• Ambiguous Questions• Multiple-Content Questions• Implicit Questions• Over-Complex Vocab• Over-elaborate phrasing• Patronising Tone
• Complexity– Open Questions
• Pros-High Ecological Validity and Open up New Possibilities• Cons-Hard to analyze
– Require Content Analysis- Looking for common trends or qualitative analysis
– Closed Questions• Pros- Answers pre-specified, makes for easy analysis• Cons- Low validity
Administering• Sampling Technique
– Want Representative Samples• Best ways to do this
– Random Sampling-Every member has an equally likely chance of being selected
• Other Ways– Quota sampling-Selecting people who fit into certain categories and setting quotas that match criteria– Stratified Sampling- Subgroups identified and then random sampling applied to get members from
each subgroup– Opportunity Sampling-Whoever is available, most likely to become biased– RDD-Random Digit Dialing, privacy issues
• Distribution Method– 4 main ways
• Face to face interviewing• Handout Questionnaires• Postal Questionnaires• Telephone Questionnaires
Improving Response Rates
• Prior Contact• Ease in Responding• Small Rewards
Ethical Issues
• How are they administered– No Coercing – Maintain Privacy (phone interviews)
• What is done with the findings– Confidentiality Key
Psychometrics
• 2 types – Attitude scales– Psychometric Tests
Attitude Scales
• Construction more tightly specified and much more rigorous than questionnaires
• Constructed to avoid response bias– Can set up a lie scale-questions scattered
throughout to show if someone is trying to portray themselves in a certain fashion
– Scale should be balanced between yes and no responses
Examples of Attitude Scales• Direct-Likert Scale
– Steps to using• Decide on topic of investigation• Get several people together and generate broad set of statements about topic• Discard repetitive or badly worded statements• Present list to separate group to determine if statements are positive or negative with
regards to topic• Discard ambiguous statements• Select equal number of positive and negative statements and arrange in random order• Allocate 5 point scale• Combine into full Likert Questionaire
• Indirect- Semantic Differential– Asks the respondent to express how the target of the attitude measure would
rate on a number of dimensions– Tries to capture the depth of someone’s attitude more than can be seen in
conventional ways using bipolar adjectives
Assumptions of Attitude Scales
• Attitudes can be expressed in verbal statements
• Same statement has the same meaning for all participants
• Attitudes expressed as verbal statements can be measured and quantified
Psychometric Tests• Used to gain insight into aspects of human
psychology that are normally hidden, or not immediately apparent
• Information can be used to help individual make more informed choices
• 2 types– Idiographic-help therapist gain insight into person’s
ideas or problems– Nomothetic-used to compare people with one
another
Test Construction• Very rigorous including several pilot stages• Construction based on:
– Reliability• Test-retest-same test administered twice to same participants on different occasions
– -1 to 1– .8 acceptable– Disadvantage-people may remember responses from first time
• Split-half-test split in half and given to participants at two times– Spilt is either arbitrary or cumbersome
• Alternative forms-2 equivalent tests generated and given on 2 occasions– Disadvantage- transfer of training and familiarity
– Validity• Face validity- based on appearance
– Not good for psychometric tests
• Criterion validity- comparing to some other standard which already exists– 2 types used in psychology
» Concurrent validity-comparing scores to those on an existing test» Predictive validity—using scores to predict future success
• Construct validity-does test reflect theoretical constructs on which it was based– Questions in subsets scrutinized to see if they correlate to one another more than to questions from other subsets
• Ecological Validity-does test reflect real life equivalents
– Standardization• Procedures must be followed• Population norms must be established• Test outcomes need to fit criteria of other tests like it
Ethical Issues of Psychometric Measurement
• Deception used to keep people from fully understanding intent
• Technical Instrument– Lie scales
• Uses can be unethical socially• Feedback can be falsified• Test must be administered by a professional
Interviews
• Six Types– Hostile-interrogation– Limited Survey-Market Research interview– Rapport-personal experience interview– Asymmetrical Trust-Doctor-Patient interview– Depth-Anthropological Interview– Phenomenological-Specialist Qualitative Research
Interviews
Interviewer Effects
• Non-verbal signals- can greatly skew an interview
• People adjust answers for what they feel are appropriate responses for the interviewer based on social knowledge
• These are called interviewer effects and must be considered in any study
Interview Skills• Requires Sensitivity• Must develop a rapport• Requires Verbal and Nonverbal skill sets
– Verbal• May require standardization of responses to keep verbal influences to a minimum (can be seen as artificial)• Speech registers must be noted, how something is being said before responding with a standardized response
– 5 Types of Registers» Frozen» Formal» Consultative» Casual» Intimate
• Reflection, sounding back, is an important skill• Amplifying, broadening the scope is a must• Non-commital agreement, pulling out more without giving your views is essential
– Non-Verbal• Must pay attention to details (All Important)
– Appropriate Eye contact– Posture– Gestures– Tones of voice
Conducting Interviews
• Establish Range of Information being sought• Determine what type of information needs to
be obtained and which type of method will best help you determine your answer
• Consider the needs of your interview schedule
Structured Interviews
• Very similar to a questionnaire, but with interviewer present
• Can use probe questions to redirect statements to where interviewer is looking, but are highly standardized
• Require same level of planning as developing a questionnaire
• Negative is that its rigidity and lack of spontaneous interaction cause changes in interviewee
Semi-Structured Interviews
• Allow more freedom to ask questions and to answer them
• Contains a number of closed questions which require specific answers
• Clinical Interviews-semi-structured interviews used in a specific setting for research purposes
Open-Structured Interviews
• Seem more like a natural conversation• Allows interviewee to answer questions as
they feel while being directed towards answers
• Primer questions often used to warm people up
• Requires Tape Recorder (Can raise ethical issues)
Analysing Interview Data• Structured- very easy to generate statistics for since choices are limited• Semi-Structures- Requires content analysis
– Info broken into themes– Indications are made of every them mentioned (frequency) and descriptive
statistics can be generated• Number of mentions does not always show importance of a theme
– Can also look for emotively loaded words• Requires answers collected word for word• Can give clues to attitudes underlying what people say
– Can also use salient quotes• Again requires word for word collection• Look for quotations relevant to topic being discussed
• Open-Structure Interviews– Can best utilize emotively loaded words and salient quotes– Very time consuming and complicated
Stages of Interview Research
• Define Aims• Decide form interview will take• Plan content of the interview• Interview Schedule Planned– Same as questionnaire– Requires pilot testing
• Conduct the interviews• Analysing and Summarizing the data
Ethical Concerns
• Putting self outside of professional competence
• Gain informed consent– Recording– Reporting– Using Quotes
• Insure confidentiality of interviewee and anyone mentioned
Case Studies
• Focus on single cases and explore them using more than one research method
• Allow far more detail in exploration• Can be used to provide theoretical insights to
stimulate other research• Regarded throughout history as “unscientific”– Completely false• Much had to do with behaviourist influence• Also feared the dangers of anecdotal evidence
Uses of Case Study
• Used to explore aspects of human experience• Insight into how psychological processes may
be operating• Not used to develop general laws about
behavior
Advantages of Case Studies
• Stimulating New Research• Contradicting Established Theory• Giving new insight into phenomena or
experience• Permitting investigation of otherwise
inaccessible situations
What is a case?
• Does not have to mean a single person• 5 Types– Person– Group– Location– Organization– Event
Methods used in a Case Study
• Multiple methods can be used besides interviewing– Tasks– Repertory grid– Diary
Triangulation
• Using several different research methods to explore the same phenomenon
Systems Analysis Approach
• Looks at 4 major dimensions of the system– Elements-The separate parts which make up the
system– Order-coherence between elements ex. patterned
interactions, mutual understandings and expectations
– Process-Changes over time, or transactions or exchanges
– Functions-the goals or outcomes of activity within the system
Psychological Field
• A way of expressing the complexity of social experience by organizing it into different dimensions– Most important to psychologists is psychological dimension
• Involves aspects of individual experience and identity
– Other important dimensions• Spatial Dimension-relates to the places or locations within which a
particular experience is set• Cultural Dimension-symbols and social rituals involved in its
occurrence• Historical Dimension-how do previous or related events influence
how this one is perceived or dealt with• Social Dimension-involves relationships, lifestyles and social
networks
Vignettes
• Ways of summarizing observations or experiences succinctly
• Can be used to draw comparisons and see what others might not see– Requires Thematic Qualitative Analysis
Repertory Grids
• Allow for the research of idiographic measures– Exploring the distinctive qualities of the individual
• Based on the idea of personal constructs– We all have our own subjective view of the world
based on our own theories about how the world works
• The Grid enables a researcher to explore the personal constructs of an individual– Allows for distinguishing of core constructs and
subordinate constructs
Laddering
• Interviewing technique which uses questioning to move from subordinate constructs to their core constructs
• Style needs to be kept light• Can raise ethical concerns
Disadvantages of the Case Study
• Uniqueness leads to low reliability ratings• Subjective feelings of researcher can influence
case study• Selectivity of reporting• Retrospective case studies can be incorrect in
assumptions• Can not generalize results
Ethical Issues of Case Study
• Confidentiality• Labeling and attribution• Stepping out of professional competence
Ethnography
• Concerned with life as it is lived from day to day
• Key Aspects– Data collected through range of sources– Behavior studied in every day contexts– Early stages of data gathering unstructured by the
researcher– Exploration conducted in depth
Analyzing Documents
• Uses evidence provided by documents to understand human experience
• Can also be used to analyze what psychologists have learned so far
The Diary Method
• Respondents required to keep notes about a particular type of experience or event
• Can be structured or open-ended• Ultimately must make repeated recordings
over time to give clues regarding:• Issues of development• Change• Recurrent experieince
Advantages of Diary Method
• Familiarity• Cost-Effective• Data Sequencing• Intimacy• Exploration• Spontaneity• Historicity
Disadvantages of the Diary Method
• Control of Content• Dropout• Poor Recruitment– Artefacts-distortions in data relating to
uncontrolled variables or artificial situations• Veracity and verification-authenticity• Reactance-change in behaviors due to diary
Designing Diary Studies
• Usually pen and pencil, but can take other forms• Are there set questions? (Good for Quantitative Analysis)
– Makes it easier to compare data• Is there room for personal reflection? (Better With
Qualitative Analysis) – Provides more genuine answers
• Many times diary studies use open-ended format– Tend to be more idiographic (focus on uniqueness) than
nomothetic (focus on comparisons)• Instructions must be clear and unambiguous
– Provide examples/may contaminate responses
Reminders in Diary Studies
• Keeping in contact with participants can reduce dropout numbers in diary studies
• Some form of incentive program like in questionnaire studies has been used by some researchers
Diaries and Documents
• Using diaries and historical documents that were not developed for psychological studies is called document analysis– Goal: to see what they reveal about underlying or
recurrent psychological processes and mechanisms
• Various forms of media can be considered documents for analysis
Problems with pre-existing material
• May not provide full info the researcher needs• Material may have been re-worked keeping
out full subjective experience of the individual• Actual documents from past with intentions of
being personal may include shorthand or codes that make it difficult for a researcher to fully understand
Meta-Analysis
• Documents consist of published research findings
• Looks to make sense of multiple replications of the same study and various findings that are not exactly the same
• Helps to summarize overall implications of research findings to date
• Takes narrative analysis a step further to eliminate subjectivity
Meta-Analysis vs. Narrative reviewNarrative Review Meta-Analysis
Selective, not comprehensive Incorporates every known study on topic
Focus only on statistical significance of studies
Takes into account effect size as well as statistical significance
Difficulty dealing with conflicting results Deals with conflicting results by identifying sources of error and artefact
Conclusions reflect aims and concerns of individual researcher (difficult to replicate)
Conclusions are replicated by other researchers
Valuable in developing new theoretical insights
Used only for summarizing existing research findings
Conducting a Meta-Analysis
• Select relevant independent and dependent variables• Identify and obtain all relevant studies• Identify characteristics which may be pertinent to the
outcome of the study• Calculate effect size estimates for relevant independent
and dependent variables• Calculate mean and standard deviation of the effect
size across the studies• Compare the effect size summaries with characteristics
identified, identify connections or correlations
Effect Size and Statistical Significance
• ES=Mean of Experimental- Mean of Control Standard Deviation of Control Group
Problems of Meta-Analysis
• What are boundaries of a research topic?• What about poorly designed studies?• File Drawer Problem for failed studies– Can be addressed by Rosenthal’s formula which
shows how many failed studies would need to exist to cancel out the effect
Descriptive Statistics• Allow us to describe our data• Can only be used if we covert Qualitative data into a Quantifiable Medium• Can explain via:
– Numbers• Measures of Central Tendency (All should line up if data is not skewed)
– Mean-average – Median- Score which falls in the center of the data set– Mode-most commonly occurring score in the data set
• Measures of Dispersion (Tell us about how scores are distributed)– Standard Deviation- Lets us calculate how different a score is from the mean
– Graphical Images• Graphs- demonstrate how a particular measure varies in response to some other variable• Bar Charts-Used with data that belongs to different categories to compare how often each
category is represented in a set of scores• Pie Chart- shows us proportions for an entire set of scores• Scattergrams- enable us to see correlations and their direction