A Taste of a Method’s Own Medicine: A Content Analysis of Content Analyses

A Taste of a Method’s Own Medicine:A Taste of a Method’s Own Medicine:A Content Analysis of Content AnalysesA Content Analysis of Content Analyses

A Presentation by:A Presentation by:Caroline Graham AustinCaroline Graham Austin

Angeline Grace CloseAngeline Grace CloseSunil ContractorSunil Contractor

Ji Hee SongJi Hee SongQiyu (Jason) ZhangQiyu (Jason) Zhang

INTRODUCTIONINTRODUCTION

Research QuestionsResearch Questions RQ1 RQ1 From the marketing perspective, what is the From the marketing perspective, what is the

overall contribution of content analyses published in overall contribution of content analyses published in the marketing literature from 1977 to 2002?the marketing literature from 1977 to 2002?

RQ2 RQ2 What is the developmental history of CA as a What is the developmental history of CA as a research/analytical method, in the selected marketing research/analytical method, in the selected marketing literature?literature?

RQ3 RQ3 As a method, has CA been adequately applied As a method, has CA been adequately applied in the marketing literature?in the marketing literature?

STRUCTURESTRUCTURE

Literature ReviewLiterature Review MethodMethod Findings 1-3Findings 1-3 ConclusionConclusion

A LOOK INTO THE LITERATUREA LOOK INTO THE LITERATURE

““Given the potential methodological problems associated with content analysis, it is Given the potential methodological problems associated with content analysis, it is

useful to assess the nature of past applications.” (Kolbe and Burnett, 1994 p. 244)useful to assess the nature of past applications.” (Kolbe and Burnett, 1994 p. 244) Kassarjian’s (1977) methodological benchmark Kassarjian’s (1977) methodological benchmark Yale and Gilly (1988), 6 advertising journals- 1976-Yale and Gilly (1988), 6 advertising journals- 1976-

1985: relationship b/w journal & topics 1985: relationship b/w journal & topics Kolbe and Burnett (1994) operationalized Kolbe and Burnett (1994) operationalized

Kassarjian’s directivesKassarjian’s directives Riffe and Freitag (1997) CA of CA: Riffe and Freitag (1997) CA of CA: JQ: JQ: found found

reliability & sophisticated statistical analysis trends reliability & sophisticated statistical analysis trends

METHODMETHOD

SamplingSampling full-length CA articles 1977-2002full-length CA articles 1977-2002 Journal of Marketing Journal of Marketing Journal of Marketing ResearchJournal of Marketing Research Journal of Consumer ResearchJournal of Consumer Research Journal of Business ResearchJournal of Business Research article selection: read abstract, method section, article selection: read abstract, method section,

computerized database search & article index searchcomputerized database search & article index search

Table: Sample Distribution

2.86120012.8611988

2.86119988.5731987

5.712199611.4341986

2.86119958.5731985

5.71219942.8611984

5.71219932.8611983

2.86119922.8611980

5.71219912.8611979

5.71219902.8611978

11.43419892.8611977

Percentage of Sample

No. of Articles

YEARPercentage of Sample

No. of Articles

YEAR

METHODMETHODCoding IssuesCoding Issues conceptual & operational definitionsconceptual & operational definitions coder training: agreement on 29/ 31 itemscoder training: agreement on 29/ 31 items major coders and checkers coded all articlesmajor coders and checkers coded all articles checking and re-codingchecking and re-coding PRAM calculates reliability coefficients (e.g., % PRAM calculates reliability coefficients (e.g., %

agreement and Holsti’s method)agreement and Holsti’s method) global Reliability= 94.1% & Scott’s pi, Cohen’s kappa global Reliability= 94.1% & Scott’s pi, Cohen’s kappa

(86%-100%)(86%-100%) data analysis (e.g., Time series to find trends, cross-data analysis (e.g., Time series to find trends, cross-

tabulation to find correlations) tabulation to find correlations)

RQ 1: Overall Contribution RQ 1: Overall Contribution

RQ1 RQ1 From the marketing perspective, what is From the marketing perspective, what is the overall contribution of content analyses the overall contribution of content analyses published in the marketing literature from published in the marketing literature from 1977 to 2002?1977 to 2002?

RQ1: Overall ContributionRQ1: Overall Contribution

CA of variable: “CA of variable: “Medium Analyzed”Medium Analyzed” magazine advertisements (31%) magazine advertisements (31%) television (28.6%) television (28.6%) academic journals (14.3%)academic journals (14.3%) newspaper advertisements (8.6%)newspaper advertisements (8.6%) comic books (5.7%)comic books (5.7%) outdoor advertising, novels, experimental data outdoor advertising, novels, experimental data

(each 2.9%)(each 2.9%)

RQ1 : Overall ContributionRQ1 : Overall Contribution

CA of variable: “CA of variable: “ContentContent”” Advertising Appeals:Advertising Appeals:(51.4%) (51.4%)

Social Values: Social Values: Social ValuesSocial Values ( (14.3%) (e.g., 14.3%) (e.g., Materialism and the Good Life)Materialism and the Good Life)

International content (20%) International content (20%)

RQ 2: Developmental History RQ 2: Developmental History

RQ2 RQ2 What is the developmental history of CA What is the developmental history of CA as a research/analytical method, in the selected as a research/analytical method, in the selected marketing literature?marketing literature?

Paradigm Shift (e.g.,):Paradigm Shift (e.g.,): Latent variable Latent variable Link to theoretical framework Link to theoretical framework Hypothesis posedHypothesis posed Statistical methods usedStatistical methods used

RQ 2: Developmental HistoryRQ 2: Developmental History

Positivistic analysis to interpretive analysisPositivistic analysis to interpretive analysis # of articles that examined latent variables has # of articles that examined latent variables has

been increasing. been increasing. # of articles that had theoretical framework has # of articles that had theoretical framework has

been increasing.been increasing. # of articles that had advanced statistical # of articles that had advanced statistical

analysis has been increasinganalysis has been increasing

RQ 3: Adequate ApplicationRQ 3: Adequate Application

RQ3 RQ3 As a method, has CA been adequately As a method, has CA been adequately applied in the marketing literature?applied in the marketing literature?

Number of Papers Citing Kassarjian and Following His Guidelines (N=16)

Paper Includes Total # (%)

Coder training (Yes) 6 (37.5%)

Pretest/Pilot (Yes) 6 (37.5%)

Coding rules and procedure (Yes) 16 (100%)

Independent coding (Yes) 13 (81.3%)

Inter-coder reliability(Yes-specific reporting)

13 (81.3%)

Intra-coder reliability (Yes) 1 (6.3%)

Validation (Yes) 1 (6.3%) 

80.60%

36.10%

11.10%19.40%

2.80%

13.90% 16.70%

2.80% 2.80%11.10%

2.80% 2.80%

0.00%10.00%20.00%

30.00%40.00%50.00%60.00%

70.00%80.00%90.00%

Statistical Analysis in CA (1977-2002)Statistical Analysis in CA (1977-2002)

Most give frequencies (80.6%)Most give frequencies (80.6%)Chi^2 (36.1%)Chi^2 (36.1%)

8.30%

50%

63.90%

27.80%

5.60% 8.30%

38.90%

11.10%8.30%2.80%

8.30%

19.40%19.40%

11.10%

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

Anthr

opolog

y

Sociolo

gy

Adver

tisin

g

Mas

s Com

m

Speech

Com

m

Man

agem

ent

Psych

olog

y

Econom

ics

Lingu

istic

sArt

History

Public

Policy

Gender

studies

Ethni

c stu

dies

Fields Used in CA (1977-2002)Fields Used in CA (1977-2002)

CONCLUSIONCONCLUSION

Overall contributionOverall contribution

Developmental history of CADevelopmental history of CA

Methodological rigorMethodological rigor