Graphic Note TakingBy: Marigold Holmes and Kayleen

Salchenberg

World views influence our research/assessment topic = EpistemologiesWorld views influence our research/assessment topic = Epistemologies

Qualitative = Meaning & UnderstandingQualitative = Meaning & Understanding

Chapter 2 – Literature ReviewWritten summary of journal articles, books, past and current state of info.

Chapter 2 – Literature ReviewWritten summary of journal articles, books, past and current state of info.

Defining Research- What is Research?- What is Assessment?- Why are Research/Assessment

important?- Similarities/Differences of

Research / Assessment?

Defining Research- What is Research?- What is Assessment?- Why are Research/Assessment

important?- Similarities/Differences of

Research / Assessment?

Chapter 1 – Introduction1. Research Topic (General area of

study)2. Research Problem (Particular

aspect of scholarly inquiry)3. Research Question (Succinct

statement or two; what do you want to know?)

Chapter 1 – Introduction1. Research Topic (General area of

study)2. Research Problem (Particular

aspect of scholarly inquiry)3. Research Question (Succinct

statement or two; what do you want to know?)

Chapter 3 – Methodological ProcessDescribes general way research/ assessment question will be approached.Methodological process is the way in which you are going to be conducting your research.

Chapter 3 – Methodological ProcessDescribes general way research/ assessment question will be approached.Methodological process is the way in which you are going to be conducting your research.

Ste

ps to

Researc

h S

tep

s to

Researc

h S

tep

s to

R

esearc

h

1st step in defining YOUR research/assessment = find your topic

so it can guide the process

Often lit. reviews help determine the research question

Lit. Review informs research topicObservations

*Simple = simply observing*Contrived = create a specific scenario

Observations*Simple = simply observing*Contrived = create a specific scenario

*Write Descriptively*Write w/ discipline

*Be validated*Make familiar

NEW!

How is

dat

a

colle

cted?

Document AnalysisDocument Analysis

Is the research credible (congruent with reality)?

Threats to trustworthiness:Credibility= unreliable

Transferability= short & specificDependability= bias unstable

Qualitative is

good if it is

trustworth

yQualitative is

good if it is

trustworth

y

How to determine participants

Interviews*Structured = universal questions* Semi structured = structured + free flow *Entirely Emergent = completely free-flow

Interviews*Structured = universal questions* Semi structured = structured + free flow *Entirely Emergent = completely free-flow

SamplingTypical

ExtremeHomogeneous

Maximum VariationCriterionSnowball

SamplingTypical

ExtremeHomogeneous

Maximum VariationCriterionSnowball

Types of DesignGrounded Theory - creation of theoryAction - gather research to improve problemCase Study - specific and bound (time, place, etc.)Ethnography - becomes part of community, Narrative - based on spoken/written word-retell Phenomenology -experience w/in a phenomenon

Types of DesignGrounded Theory - creation of theoryAction - gather research to improve problemCase Study - specific and bound (time, place, etc.)Ethnography - becomes part of community, Narrative - based on spoken/written word-retell Phenomenology -experience w/in a phenomenon

Focus GroupsOrganized group discussion with interaction among key

members*Structured*Semi-Structured*Entirely Emergent

When to use interviewsCannot observe a particular behavior or phenomenonA more controlled inquiry is desiredRich in depth data needed

When to use interviewsCannot observe a particular behavior or phenomenonA more controlled inquiry is desiredRich in depth data needed

When to use focus groupsMultiple perspectives = more or robust dataGroup interaction increases trustworthinessData collection time is limited

Successful Observations

Positivists (THE truth), Interpretive 　 (many interpretations=truth), Postmodernism (individual truth), Critical Social Science (what we know is marred)

Positivists (THE truth), Interpretive 　 (many interpretations=truth), Postmodernism (individual truth), Critical Social Science (what we know is marred)

Variables characteristics that are

measured

Independent - impacts dependent variable Dependent - depends on action of independent variable

Types of DesignExperimental - cause and effect using random samplingQuasi Experimental - cause and effect but can’t randomize sampleNon-Experimental - surveys and correlations

SurveyStrength• can be locally designed & made to

meet specific needs• easy to study large populations• can be updated, replicated• can be used as-is, with permission

from the authorLimitations• Non-response bias – those who don’t

respond may have a different opinion, those who do may be similar

Surveys that are not good• asking people to remember the

past• double barreled questions –

asking two questions at once• leading the respondents to a

particular answer• inaccessible language (ie. use

of jargon, slang, or highly technical terms)

• answers which are not mutually exclusive

• answers which are not exhaustive

Other Considerations• Survey organizations• Introduction / transitions• Time• Location

Quantitative = Statistical Relationship, causation, correlation

Quantitative = Statistical Relationship, causation, correlation

Sampling Simple

StratifiedSystematicClustering

ConvenienceSnowball

How is Data Collected?

What is measured

How to determine

participants

How

is D

ata

Colle

cted

?

Non-Experimental

Designing the survey 　　　　　　　　　　　　　　　

Validity relies on reliability Is research valid? Is it reliable?

Threats to Validity:(Internal)=*history 　 *maturation 　 *selection 　 *mortality

(Treatment)= *diffusion 　 *compensatory equalization/rivalry 　 *demoralization

(Procedures)= *testing familiarity 　 *demoralization

Extantexisting data, information &

observations previously collectedStrength • don’t have to do data collection• expanding, building, repurposingLimitations• risk of data being outdated• population may not fit your target (ie.

only subset)• since you don’t know the purpose of

the data collection previously conducted, there may be external forces you don’t know about

• don’t know the rigor of the data collection that was conducted

Ethical Considerations

Key Events1. Nuremberg Code

(1940)- Post World War II, running unethical experiments on humans, consent. Result was consent from people to have experiment performed.

2. Thaimaldohide- (1950s-1962)- Medicine/drug in Europe= deformities is babies. Informed consent violated/ FDA needed to regulate

3. Tuskegee Syphilis Study (1932-1972)- Violated African American participants by withholding cure of syphilis in order to understand progression

4. Declaration of Helsinki (1964)- With mounting pressure, this was like a “blown up Nuremberg Code”, research needs to be based on lab/animal experimentation

5. National Research Act & Belmont Report (1974)- Response to Tuskegee, protection of human subjects

6. Common Rule (1981-1991)- Put into law and is where we are now!

Belmont Report:Respect for persons: autonomous agents, diminished autonomy entitled to protectionBeneficence- Human subjects should not be harmed, research benefits maximizedJustice- benefits of research must be distributed fairlyCommon Rule:

Institutional compliance required, informed consent, requirement for IRB membership, record keeping, stipulate additional protectuoibs for certain vulnerable research subjects

Chapter 4 – Analyzing & Interpreting Data

Making meaning of the data that is collected

Chapter 4 – Analyzing & Interpreting Data

Making meaning of the data that is collected

Chapter 5 – Discussions

Summary of key finds, explanation of results, suggest limitations in the research and make recommendations for future inquiries.

Chapter 5 – Discussions

Summary of key finds, explanation of results, suggest limitations in the research and make recommendations for future inquiries.

Ste

ps to

Researc

h

Data Preparation1. Organize data 2. Transcribe data from audio recording/field notes to

data *by hand or computer using qualitative computer programs.

Data Preparation1. Organize data 2. Transcribe data from audio recording/field notes to

data *by hand or computer using qualitative computer programs.

Data Analysis - Coding1. Get a general sense of data (preliminary

exploratory) 2. Code the data (make sense out of text data using

labels of segments with codes) 3. Pick one document 　4. code the document with brackets and text segment

　5. Make a list of all code words 6. Take list and go back to data 7. Reduce lists finding themes and categories

The 3 C’s of AnalysisCodes = “storage bins” / general headingsCategories = consolidation of codesConcepts/Themes = categories as meaningful

statements (5-7 themes)

*Be sure to check work against original text

Effective Narrative Data Analysis• Revisit research question frequently• Develop a system, test it, then stick with it• Review all data again & again• Allow concepts to emerge organically• Synthesize information without loosing sight of main

focus• Honor participant voices• Use technology as appropriate

Data Analysis - Coding1. Get a general sense of data (preliminary

exploratory) 2. Code the data (make sense out of text data using

labels of segments with codes) 3. Pick one document 　4. code the document with brackets and text segment

　5. Make a list of all code words 6. Take list and go back to data 7. Reduce lists finding themes and categories

The 3 C’s of AnalysisCodes = “storage bins” / general headingsCategories = consolidation of codesConcepts/Themes = categories as meaningful

statements (5-7 themes)

*Be sure to check work against original text

Effective Narrative Data Analysis• Revisit research question frequently• Develop a system, test it, then stick with it• Review all data again & again• Allow concepts to emerge organically• Synthesize information without loosing sight of main

focus• Honor participant voices• Use technology as appropriate

Results1. Summarize findings 2. Convey personal reflections 3. Compare to literature 4. Limitations and suggestions

Results1. Summarize findings 2. Convey personal reflections 3. Compare to literature 4. Limitations and suggestions

Use Computer Software

for

• Text identification &

retrieval

• Text analysis

Nud*istNVivoAtlas.ti

Ethnograph

Data Preparation• Score the Data • Determine Types of Scores to Analyze • Select Statistical Program • Input Data• Clean & Account for Missing Data

Data Preparation• Score the Data • Determine Types of Scores to Analyze • Select Statistical Program • Input Data• Clean & Account for Missing Data

Data AnalysisDescriptive statistics describe response to

questions an determine overall trends and distribution of data.

Inferential statistics – draw conclusions, inferences, or generalizations from a sample to a population of participants.

Data AnalysisDescriptive statistics describe response to

questions an determine overall trends and distribution of data.

Inferential statistics – draw conclusions, inferences, or generalizations from a sample to a population of participants.

ResultsTables, Figures, Presentations – wordsResultsTables, Figures, Presentations – words

We still don’t quite understand