Question Development

Module 3

November 2015

Learning Objectives

• Understand the goals of question development1. Tracks laws through a question and answer format

2. Create questions that measure the law, not interpret the law

• Learn how to draft coding questions 1. Review law and secondary sources

2. Identify constructs (features of the law)• Set standard constructs

3. Develop response sets

4. Convert constructs into questions

5. Capture unexpected responses through iterative coding

Goals of question development

1. Show the state of the law through a question and answer format– This generates empirical data that can be

used for evaluation– Asking the right questions can yield the

information you are interested in obtaining for your study

Goals of question development

2. Create questions that call for measurement, not interpretation– The structure and wording of your questions

yields clear responses, which can be collected consistently by different researchers who are unfamiliar with the topic

– Well-crafted questions make coding easy and efficient

How to draft coding questions

1• Review law and secondary sources

2

• Identify constructs in the law• Identify standard constructs for your project

3• Develop response set

4• Convert constructs into questions

5• Add unexpected responses through iterative coding

Review law and secondary sources

• Reviewing the research you did in Module 2 will provide you with the context needed to draft coding questions– Revisit your five state memorandum– Review your sample of laws– Refer to secondary sources

2

Set standard constructs

Standard Constructs Examples

JurisdictionsCountries, states, cities, counties, organizations, etc.

United States, Maryland, Baltimore, Montgomery

Effective DateGenerally month, day, year

January 1, 2015October 1, 2009

FIPS CodeFederal Information Processing Standard

Alabama-01Alaska-02

Coder NameResearcher who coded the current project

John DoeJane Doe

2

Identify constructs in the law

Initial constructs observed in the law

Constructs after narrowing the scope

Device Prohibited

Prohibited Behavior

Age for Prohibition

Emergency Use

Driving Record

Type of Driver

Use exceptions

Device Prohibited

Prohibited Behavior

Emergency Use

Type of Driver

2

Developing response sets

Prohibited Behavior

Texting

Calling

Hands-free Texting

Hands-free calling

3

CONSTRUCT

RESPONSES

Converting constructs into questions

Prohibited behavior What behaviors are restricted?

Type of driver, all drivers Does the state have a law restricting all drivers from using cellphones while driving?

4

CONSTRUCTS QUESTIONS

Parent

Child

Converting constructs into questions

Definition Example

ObservationThings we measure (facts)

Is texting while driving illegal in this state?

InterpretationConclusions we derive from those observations (opinions)

Does this state have a strict texting while driving ban?

4

Add unexpected responses while coding

Prohibited behavior

Texting

Phone calls

Other

5

Prohibited behavior

Texting

Phone calls

Hands-free texting

Hands-free phone calls

Summary

• Drafting questions achieves several goals, including– Converting a project into a question and answer format– Generating robust data which can be replicated by researchers

• Steps in question development:

1. Review law and secondary sources

2. Set standard constructs• Jurisdictions to be covered, effective date for that iteration, FIPS code, and the name of the coder

3. Identify constructs• Each project should develop an initial list of constructs, which must then be narrowed or broadened to

meet the goals of the project

4. Develop response sets • Researchers should identify responses to the constructs in the law

5. Convert constructs into questions• The selected constructs should be converted and organized into questions• The organization of the questions can have a large impact on the information which can be yielded

from the project• Craft questions that call for observation rather than interpretation

6. Capture unexpected responses• As researchers find new responses in the law, they can flag them so that the supervisor can add them

to the potential answers for the questions being studied

7. Modify questions as necessary• As researchers identify new states with a unique legal structure, or additional layers of detail in the

law that can be valuable to a study, questions can be modified