Upload
noah-ferguson
View
215
Download
2
Tags:
Embed Size (px)
Citation preview
Lecture 2What is a Theoretical Contribution?
RESEARCH METHODSDOCTORAL PROGRAM, NATIONAL RESEARCH
UNIVERSITYHIGHER SCHOOL OF ECONOMICS
DR C S LEONARD
JUNE 2011
RESEARCH METHODS
2 OUTLINE
What is a theoretical contribution?
What is a concept?
A variable is a concept
Validity
External, Internal8/6/2011
RESEARCH METHODS 3
THEORY
Publishing
(Evaluative)
Criteria
8/6/2011
RESEARCH METHODS
4PRACTICAL
GUIDELINES
Choose a hypothesis important in literature but for which no systematic study exists—decide in favour or against the hypothesis
Choose a theory you suspect is false and investigate if it is, indeed, and what alternate might replace it
Design research to illuminate unquestioned assumptions in the literature
Argue that a topic of importance has been overlooked
Show relation of theory from one literature to separate problem from another
8/6/2011
RESEARCH METHODS
5REVISION OF FORMAL
THEORIES?
More than just adding a variable
Showing how it changes perception of the problem
Producing data that are inconsistent with a theory
8/6/2011
RESEARCH METHODS
6BORROW FROM
OTHER DISCIPLINES?
Shed new light on old theories
Investigate qualitative boundaries of a theory, not just quantitative ones
Show why a theory will not work in a new application, not just that it does not
Focus on multiple elements of a theory, not just one inconsistency
8/6/2011
RESEARCH METHODS
7EVIDENCE REQUIRED
FOR REVISING A THEORY
Marshal compelling evidence.
This evidence can be logical (e.g., the theory is not internally consistent),
empirical (its predictions are inconsistent with the data accumulated from several studies), or
epistemological (its assumptions are invalid-given information from another field).
8/6/2011
RESEARCH METHODS
8PROVIDE REMEDIES,
ALTERNATIVES
Is the original actually inferior,
Or simply the best we can do?
8/6/2011
RESEARCH METHODS
9IS IT
PUBLISHABLE?
How radically new is the idea?
Is it linked to evidence, will it change practice?
Why so? Are the underlying logic and sup-porting evidence compelling? Are the author's assumptions explicit? Are the author's views be-lievable?
Well rounded, broad, deep understanding?
At professional standards?
Who cares? Why now?
8/6/2011
RESEARCH METHODS
10WHAT THEORY IS NOT:
(1) REFERENCES
For example:
"This pattern is consistent with findings that industrialization produces growth (Marx 1880, Solow 1956)
This sentence lists publications that contain
conceptual arguments (and some findings).
But there is no theory because no logic is presented
to explain why industrialization produces growth
8/6/2011
RESEARCH METHODS
11WHAT THEORY IS NOT:
(2) DATA
Introduced to show what patterns are there
Not why they are there
This approach relies on brute empiricism,
where hypotheses are motivated by prior data
And not the relevance of theory
8/6/2011
RESEARCH METHODS
12(3) LISTS OF VARIABLES
AND DEFINITIONS
Lists of concepts, similarly, are not theory
Need connections between variables explained
8/6/2011
RESEARCH METHODS
13(4) DIAGRAMS
ALONE
What is needed is causal inference
Time horizon and change over time
Diagrams as Stage props, not the performance
Logic needs to be spelled out, verbally
8/6/2011
RESEARCH METHODS
14(5) HYPOTHESES: WHAT IS EXPECTED TO OCCUR
Well-crafted conceptual argument includes hypotheses
They serve as crucial bridges between theory and data
Making explicit how the variables and relationships that follow from a logical argument will be operationalized.
These are statements about what is expected to occur, not why it is expected to
8/6/2011
RESEARCH METHODS
15IDEAS AND CONCEPTS
A theory generally stems from a small set of research ideas, not a list of testable hypotheses
8/6/2011
RESEARCH METHODS
16QUALITATIVE/QUANTITATIVE
DISTINCTIONS
Qualitative: Research Questions
Quantitative: Set of hypotheses (nul or directional)
8/6/2011
RESEARCH METHODS
17EVIDENCE FOR YOUR
HYPOTHESES/TESTING THEORY
When theories are particularly interesting or important, empirical support can be partial: a small set of interviews, a demonstration, experiment, a pilot survey, archival data
8/6/2011
RESEARCH METHODS
18EVIDENCE FOR YOUR
HYPOTHESES
May point to why a particular process
might be true.
Subsequent research indicated
May show whether the theoretical
statements hold up, or whether they
can repeatedly be falsified 8/6/2011
RESEARCH METHODS 19
EXTERNAL VALIDITY
Economics
8/6/2011
RESEARCH METHODS
20
TWO CONCEPTS OF EXTERNAL VALIDITY,
AS APPLIED TO THEORY
1: Replicability, that the study would hold for other persons, settings, times, or places.
Only one form of validity when the objective of research is to test theory
8/6/2011
RESEARCH METHODS
21EXTERNAL
VALIDITY
2: Applicability
It arises primarily through severe and rigorous tests of theory
rather than by attempts to incorporate "real world' variables into individual studies designed to test theory.
Such variables only become important in the context of evaluating interventions based on theory.
8/6/2011
RESEARCH METHODS
22 CONSTRUCT VALIDITY
One needs to make a distinction between
(1) the construct validity of a concept, as
reflected in the convergence (and
discrimination) of some particular set of
operationalizations of it, and
(2) the construct validity of a relation between
two concepts, as reflected in the "fit" of that
relation within some nomological network.
The fit is linked to considerations of external
validity.
8/6/2011
RESEARCH METHODS
23SAMPLING IS CRITICAL
FOR VALIDITY
Four major sampling strategies that might be
adopted vis a vis any one aspect or facet of the
events under study
1. Sampling homogeneously over the entire study
2. Sampling several subsets, each homogeneous
within subset on the facet but differing on it
between subsets, so that all the subsets
together span the whole range
8/6/2011
RESEARCH METHODS
24 SAMPLING
3. Sampling heterogeneously. but in a
way that yields an overall distribution of
the facet among the cases within the
study that reflects (is representative of)
the distribution of the facet among cases
"in nature”
4. Sampling heterogeneously on the
facet but without regard to
representativeness.
8/6/2011
RESEARCH METHODS
25SAMPLING THREATS
TO VALIDITY
These four strategies offer different opportunities for—and pose different threats to—the exploration of the external validity of any given set of findings with respect to the facet in question.
Define external validity:
the deliberate and systematic search, on a number of facets, for both the scope and the limits over which that given set of findings does and does not hold.
8/6/2011
RESEARCH METHODS
26 REPORTING
Validity as robustnesss
Where are the boundaries?
Can findings be replicated?
8/6/2011
RESEARCH METHODS
27 TESTS FOR VALIDITY
STAGE 1 STAGE 2
Design:
Instrument validity instrument use validity
Comparison validity execution validity
Hypothesis:
Experiment:
Construct validity Operational
Nomalogical validity Predictive
8/6/2011
RESEARCH METHODS
28CHECKING
VALIDITY
Observation:
State validity Attribute
Pattern validity Process
8/6/2011
RESEARCH METHODS 29
THEORY
What is a good
theory?
8/6/2011
RESEARCH METHODS
30 A THEORY DOES NOT COME FIRST IN RESEARCH
Theory:
A reasoned and precise speculation about the answer to a research question, including a statement about why the proposed answer is correct
Implies several specific descriptive or causal hypotheses
Is consistent with prior evidence about a research question
8/6/2011
RESEARCH METHODS
31CHOOSING A
THEORY
Possibly, it will be wrong
It must be falsifiable-what evidence would convince us?
Requires observable implications (many)
Observe the principle of parsimony (a judgment about the nature of the world, which is assumed to be simple)
These rules assume you have not yet collected the data, which can be used afterwards, to modify your theory
8/6/2011
RESEARCH METHODS
32AFTER YOUR
RESEARCH
What to do with a theory
Expand it, dropping a condition or variable, to, say, all countries, all regions
Make it embrace a larger range of phenomena
Do not do the opposite If the theory does not work for some of your
observations, do not squeeze new theory out of a revised and qualified research base
Ie, make it less restrictive, but not more restrictive—unless you go back and collect more data and observations beginning with the new hypothesis
8/6/2011
RESEARCH METHODS
33 DATA
Ensure reliability
Maximize leverage
Explain as much as possible with as little as possible Increase numbers of observable
implications with confirmation
Improve the theory, improve the data
From the beginning, list all the possible implications of your hypotheses—outcomes, responses, interviews, at all levels
8/6/2011
RESEARCH METHODS
34 SCEPTICISM
Report negative findings
Report alternative hypotheses
8/6/2011
RESEARCH METHODS 35
THEORY CONCEPT
How to build a
concept
8/6/2011
RESEARCH METHODS
36CONCEPT
FORMATION
Rigorous approach to concept formation (Osigweh) Defining meaning
boundaries (what the concepts do not include)
Minimizing concept misuse, confusion
8/6/2011
RESEARCH METHODS
37CONCEPT
PRECISION
General, yet precise Moving a concept from low
to high levels of abstraction
Economic oncepts must span several contexts
Yet they must provide a precise guide to what is not included
8/6/2011
RESEARCH METHODS
38CONCEPT
IMPRECISION
Useless: new contexts too easily fit
New topics can be suspect, vulnerable to herding by scholars (CR impact on performance)
8/6/2011
RESEARCH METHODS
39 CONCEPTS
Can be decomposed taxonomically
8/6/2011
RESEARCH METHODS
40 VARIABLES
A variable is a special
kind of concept.
It is a classification into two or more mutually exclusive and totally inclusive
categories (e.g., Hage, 1972; Smith, 1975).
8/6/2011
RESEARCH METHODS 41
CONCEPTS
Stretching and
Travelling
8/6/2011
RESEARCH METHODS
42CONCEPT TRAVELLING:
POSITIVE CONTRIBUTION
Concept travelling, in this sense, means that the concept is precise enough to allow researchers to define it in the same way, and so to test it in a wide range of situations—that is, that the concept is a universal.
8/6/2011
RESEARCH METHODS
43 EXAMPLE
Puzzle -- Each use creates essentially the same image anywhere:
(a) a maze of activities that cannot be correctly solved outside settings or mental frameworks that prescribe following certain paths or specific combination of paths;(jigsaw?Rubik’s cube)?
(b) an exercise of figures, numbers, or behaviours that cannot be engaged in or successfully disengaged in, except through a specific order or steps, procedures, or systems of thought processes;
(c) activities that when completed cannot fail to yield exact answers, known solutions, or ultimates that could be exactly predicted beforehand.
8/6/2011
RESEARCH METHODS
44CONCEPT STRETCHING
(TO BE AVOIDED)
Multi level governance
when refracted through the lens of
lived political experiences-- notions of
policy transfer and of the complex
politics of scale of interventions in
time, place and space--needed
8/6/2011
RESEARCH METHODS
45TOO VAGUE
(STRETCHING)
Decentralization
Would be misapplied in different societies
Too broad
8/6/2011
RESEARCH METHODS
46
TOO NARROW, AS WELL AS VAGUE: OBSERVATIONAL
CONCEPTS
Can be moved up or down the abstract ladder
Communication puzzle, group, decision, problem, conflict, and participation.
8/6/2011
RESEARCH METHODS
47
8/6/2011
Connotation (Depth, Intension) I HAL Low Middle High
DENOTATION
(EXTENSION,
BREADTH)
High abstract level • Traveling concept
•Stretched concept
domain domain
— Broad coverage, of — Broad coverage
distinct classes of — Too many classes
things of things lumped
— Few, but determinate together (with little
attributes of attention to the
distinct classes precision of their
— Precise at a many attributes)
universal level — Packed or too allembracing
RESEARCH METHODS
48
8/6/2011
Connotation (Depth, Intension)
I I I I I I I I I I I I
Low Middle High
DENOTATION (EXTENSION,
BREADTH)
MAL
Mid abstract level • Generalizable
nonuniversal concept domain
— Breadth balanced
— Medium range concepts
— Similarities
RESEARCH METHODS
49
Connotation (Depth, Intension) I I I I I I I I I I I I HAL Low Middle High
DENOTATION (EXTENSION, BREADTH)
MAL LAL • Configurative • Taxonomic domain
Low abstract level
situational concept doman
— Narrow coverage
— Narrow coverage
— Many attributes
— Too few attributes for each class
— Configurative/ — Distinctive typologies
Imprecise taxonomies or precise taxonomies
specific generalities precise at a specific level
taxonomies — Precise at a specific level
8/6/2011
RESEARCH METHODS
50MOVING TO THE
UPPER LEFT CORNER
Move up the ladder by negating
Concepts with negation are precise
Those without are indeterminant
8/6/2011
RESEARCH METHODS
51EXAMPLE: STUDIES OF
“PROBLEM”
The concept of problem is studies without its meaning boundaries being distinctively delineated.
Generated by concrete, intuitive, ad hoc conceptualizations.
Studies of structured problems fall widely apart from a focus on conflict resolution to one on negotiation.
8/6/2011
RESEARCH METHODS
52THEORETICAL
CONCEPTS
Theoretical or universal concepts are defined by their systemic meaning in the sense that each meaning derives from the part that the concept plays in the theory.
Open concepts reflect the availability of different operational criteria for application to different contexts.
As a result, their meanings are not fully defined by reference to observable things and their characteristics.
8/6/2011
RESEARCH METHODS
53 EXAMPLES
Efficacy (Jones, 1986; Osigweh, 1983)
Synergy, feedback, adaptation (Osigweh 1985b, pp. 153- 157),
Homeostasis and isomorphism
8/6/2011
RESEARCH METHODS
54EMPIRICAL CONCEPTS
Observational
Can be moved up or down
8/6/2011
RESEARCH METHODS
55 EXAMPLES
communication,
puzzle,
group,
decision,
conflict
8/6/2011
RESEARCH METHODS
56MOVING UP THE
LADDER
Requires that you keep it clear and not just extend meaning to other kinds of observations
8/6/2011
RESEARCH METHODS
57 ADVICE
Define a concept by saying what it is not;
Give it precise boundaries
Make it clear
8/6/2011
RESEARCH METHODS 58
THEORETICAL CONTRIBUTION
Definitions and
Approaches
8/6/2011
RESEARCH METHODS
59 COMPONENTS
Which factors (variables, constructs, concepts) logically should be considered as part of the explanation
Two criteria exist for judging the extent to which we have included the "right" factors: comprehensiveness (i. e., are all relevant
factors included?) and
parsimony (i.e., should some factors be deleted because they add little additional value to our under-standing?).
8/6/2011
RESEARCH METHODS
60 RELATIONSHIPS
Operationally this involves using "arrows" to connect the "boxes."
Such a step adds order to the conceptualization by explicitly delineating patterns.
It typically introduces causality.
8/6/2011
RESEARCH METHODS
61 WHY IMPORTANT?
What justifies the selection of factors and the proposed causal relationships?
This rationale constitutes the theory's assumptions
It welds the model together.
Why should colleagues give credence to this particular representation of the phenomena?
The answer lies in the logic underlying the model.
The soundness of fundamental views of economics or processes
8/6/2011
RESEARCH METHODS
62FROM THE
EDITOR:
“The mission of a theory-development journal is to challenge and extend existing knowledge, not simply to rewrite it. Therefore, authors should push back the boundaries of our knowledge by providing compelling and logical justifications for altered views.”
8/6/2011
RESEARCH METHODS
63 SET CONTEXTUAL
LIMITS
These temporal and contextual factors set the boundaries of generalizability, and as such constitute the range of the theory.
Do theoretical effects vary over time, either because other time-dependent variables are theoretically important or because the theoretical effect is unstable for some reason.
8/6/2011
RESEARCH METHODS
64 SUMMARY
For purposes of publishing:
Clarity in main ideas
Currency in quality
8/6/2011
RESEARCH METHODS
65PUBLISHING IN
ECONOMICS
Papers are assumed to vary along two quality dimensions
The former is interpreted as the importance of a paper’s main ideas and
the latter as other aspects of quality. Observed trends are thought of as reflecting increases in this
8/6/2011
THE END