Reading Notes on “Basics of Social Research” II

Chapters 6, 7, 8, 9, 10 (11/12 not required)

Chapter 6 – Survey Research

What is a Survey?

• Surveys used to answer questions about beliefs/behaviours• Surveys sample may respondents who answer the same questions• They measure many variables, test multiple hypos within a single survey• Measures association among variables statistically – coorelational• Positivist, deductive approach

Research questions appropriate for a survey

• types of things a survey can ask and expect valid answers○ behaviour, attitudes/beliefs/opinions, basic characteristics (generally agreed upon facts

such as marital status), expectations, self-classification, knowledge○ basically subjective things, agreed upon factual things as individual knowledge is limited

• Must recognize that how people actually act may differ from what they say, many factors influence responses

• should not ask respondents ‘why’ as not fully aware of causal factors• researcher must build their own causal theory

Steps in Conducting a Survey

• flow chart summarizes on page 137 of text• Step 1

○ Develop hypothesis○ Decide on type○ Design questionnaire *

The questionnaire is the instrument used ie) the sheet of paper with the questions that the respondent marks on

Alternatively an interview schedule can be used – the difference being this is read to the respondent by an interviewer

• Step 2○ Plan data recording

• Step 3○ Identify respondents – decide on sampling frame, sample size

• Step 4○ Do it

• Step 5○ Data entry○ Statistical analysis

• Step 6○ Research report published

Constructing The Questionnaire

• Writing Good Questions○ Give valid measures – difficult when diverse background of respondents is considered

○ 12 main principles to avoid bias and create clear questions○ ** Chart on page 142 provides example of all of these questions

Avoid slang and abbreviations• Questions might be incomprehensible or misinterpreted• Acceptable if targeting a specific population

Avoid vagueness Avoid emotional language

• Could lead respondent Avoid prestige bias

• can colour a question and lead respondent Avoid double barrelled questions

• These are questions that join two questions together• Can lead to inaccurate responses – how do you answer if you do not have the

same opinions on both topics? Don’t confuse beliefs with reality Avoid leading (aka “loaded”) questions

• Many of the above fit under this category Avoid false premises

• If you begin a question with a premise that your respondent does not agree with they will be lost

• Ie) William Shatner is a McGill alumni and one of the most esteemed actors of the modern era. Do you agree or disagree that the administration should rename the “University” building the “Shatner” building as decided in a student referendum? This statement is based on a false presence as Shatner’s greatness is debateable, and actually dropped out of McGill.

No distant future questions• Respondent recall is limited• It can take longer, and is inaccurate over certain periods (ie a year)• to help this the question can be framed with aids to jog respondents’

memories ie) location, time frame Avoid double negatives Avoid unbalanced response categories

• Must be mutually exclusive/exhaustive• Types of Questions

○ Threatening Questions About sensitive issues or may damage self image Include qs about drugs, criminal activity, mental health – deviance People wish to present a positive image and protect their egos and may lie to

project a certain image or may be too embarrassed to admit certain behaviours Underreport negative and overreport positive – based upon societal norms To get truthful answers can frame in context of more serious infractions Can establish rapport with respondent and make them feel at ease

○ Socially Desirable Questions Distort answers to coincide with norms This is the overreporting of the positive May avoid bias by making norm violation less objectionable

○ Knowledge Questions Everybody wants to look smart Phrase questions in a way that acknowledges not everyone will have knowledge

○ Skip/ Contingency Questions

Two parted questions Ie) Are you a student? If you answer no proceed to question 2. If yes do you receive

financial assistance?

Open VS. Closed Ended Questions

• Open Ended○ Response is unstructured and free response○ Valuable in early stages of research○ Respondents can answer in detail and provide unanticipated findings○ Comparision and statistical analysis difficult○ Greater amount of time and effort

• Close Ended○ Responses are structured and respondents have to choose a given response○ Quicker, easier answer to compare, code and analyse○ More likely to elicit responses on sensitive topics○ Not as accurate responses - May suggest ideas that the respondent wouldn’t otherwise

• Choice depends on purpose of study and limitations ($$)• Disadvantages can be reduced by including both types• Probes (not the alien kind) can be used with closed ended questions to reveal reasoning• No opinions

○ Quasi-filter – offers a no opinion option○ Full filter – contingency question, first asks if respondent has an opinion then asks for

opinion if they do○ Best to offer alternatives as respondents with no opinion will be frustrated○ Response set bias – tendency of people to agree without thinking○ Solved by providing detailed alternatives

Question Design Issues

• Length○ Depends of survey format and respondent○ Nobody wants to answer long surveys

• Question Sequence○ Structured, clear introduction that grabs readers attention, middle deals with tough stuff,

then conclude on a positive note○ Order is important as questions create context that the next one will be answered in

Resolved with “funnel sequence” – working from general to specific Or by using two order sets and dividing your population

○ Format Should be very clear Organize questions vertically

Types of surveys

• Chart on page 156• Mail and Self-Adminned Questionnaires

○ Cheap, no personnel costs, Slow○ Can cover a wide area○ Maintain anonymity and interview bias○ Low response rates

○ Respondent on their own – no clarification○ Serious errors can go undetected – liars

• Web Survey○ Fastest and cheapest○ Just like above○ Biased as old people and rural people do not necessarily have access

• Telephone Interviews○ Ability to reach a wide range of people – most have access to a phone○ More expensive, but still cheaper than face to face○ Limited interview length○ Reduces anonymity and interviewer bias possible

• Face-to-Face Interviews○ Expensive, slow○ Highest response rates○ Interviewers can observe nonverbal cues and probe○ Allows for complex questions○ Interviewer bias a possibility

• Interviewer Bias○ When the interviewer expectations, non-verbal cues or voice leads the respondent○ Interviewers trained to reduce this (neutral voice, read exact questions)○ Race and gender may also influence response (much harder to admit you’re racist if your

interviewer is black• Computer Assisted Telephone Interviewing

○ High tech. As you interview you type up your answers into the computer and it saves them

Ethics in Surveys

• Invasion of privacy• Respondents must volunteer – informed consent• Exploitation of surveys/psedosurveys – using survey format to coerce people• Or the very serious issue of “suggle” when charlatans use the guise of a survey to sell something

Chapter 7– Experimental Research

Basic Principles

• Positivist approach• Best method for testing causal relationships• Limited by practical and ethical constraints• Best used for issues with a narrow scope/scale at the micro level• Limits ability to geralize

Random Assignment

• Randomly assign sampled cased to groups • Done so researcher can be sure tested groups are not different and are comparable• Similar to random selection groups are divided so that each subject has an equal chance of being

placed in each group• Ideally both random selection and assignment would be used, but not always the case• Another method for assigning groups is matching

○ Involves choosing a group then trying to directly match each person to create another group

○ The issue is how far do we go, as a perfect match for each individual does not exist and deciding which characteristics need to be matched is tricky

Parts of the Experiment

• Treatment ○ What the researcher changes○ Measure this variable by creating a situation○ This is the independant variable/s

• Dependent variable○ The outcome of the situation/independent variable○ The physical conditions, beliefs, attitudes – whatever is changed in response to the

treatment○ Must be measurable, various methods employed

• Pretest○ Test of the dependent variable given before the treatment to see if there are problems and

to serve as a comparison• Posttest

○ Test of the dependent variable given after the treatment, compared to the pretest• Control Group

○ No treatment• Experimental Group

○ Receives treatment○ If there are many variables being tested there will be multiple groups

Steps

• See page 174 for a diagram of steps• Control is vital in experiments as the effects of the treatment must be isolated to come to any

conclusions• Note deception often used to control subjects and setting, makes the debriefing stage very

important

Design types

• See text for examples, see page 181 for diagram• Classic Experimental design

○ All designs variations of this○ Random assignement, pretest, posttest, experiemental group, control group

• Preexperimental design○ Same as above, no random assignement○ All designs that follow are forms of preexperimental design – they are modified for

constraints, but make identifying a causal relation less certain • One-Shot case study

○ Aka. One group posttest only design○ Only one group, no random assignment, no pretest

• One group pretest posttest○ Only one group, no random assignment

• Static group comparison

○ Aka the posttest only noequivalent group desigm○ No random assignment, no pretest

• Quasi-experimental designs make identifying a causal relation more certain, used where classical design is impossible, the designs that follow fall under this category

• Two group posttest only○ Identical to static except with random assignement○ Only modification from classic no pretest

• Interrupted Time Series○ Uses one group to take multiple pretest measures before and after treatments

• Equivalent Time Series○ Like classical caught in a loop, pretest treatment posttest repeat...

• Latin Square Designs○ Used when want to see how multiple treatments given in different sequences affect a

dependent variable• Solomon Four Group Design

○ The pretest can potentially impact the treatment/dependent variable ex- by exposing the group to skills or ideas they would not previously have had

○ To eliminate this modified design – four groups, two go through classical experiment design (one as experimental one as control), the latter two go through posttest only design (again one is control) to see if there is a difference

• Factorial design○ For studies looking at the simultaneous effects of more than one independent variable○ Uses two or more independent variables in combination○ Effects divided into main effects, and those created by the interaction of the two variables

– the interaction effects

Design Notation

• Shorthand for parts of experimental design○ O = dependent variable, R= random assignment, X= treatment○ O1 = pretest O2= posttest○ When multiple treatments used also subscripted

• See page 181 – good diagram

Internal Vailidity

• The ability to eliminate alternative explanations for the dependent variable• Anything that affects this variable except the indepent variable is a compromise to internal

validity• Following are some examples• Selection Bias

○ If groups not equivalent, a characteristic in one of them could effect dependent variable○ Problem in all designs w/o random assignment

• History○ You never know what’s going to happen – news reports and events may occur in the

middle of study and vary opinions in an unexpected/particular way○ More of an issue in long term studies

• Maturation○ Must be considered that natural processes will affect responses○ Long term study issue

• Testing○ Mentioned in the Solomon design, pretests can affect the experiment

• Instrumentational○ Faulty tools

• Mortality○ Not necessarily death, but discontinuation of study participation

• Statistical Regression○ Problem of extreme values – random mistakes move the group towards average as there is

nowhere to go but down/up○ IE) If I showed a violent video to especially violent inmates and then measured their

responses, expecting an increase in violent tendencies I might not find any as they are already at a “10” for violence, if I retested them I might find a lower response as there is nowhere to go but down

○ Page 183 clearer examples• Diffusion of treatment/Contamination

○ Participants might inform control group• Experimenter Expectancy

○ If experimenters expect a certain response they may give cues and lead respondents○ Can be rectified by using a ‘double blind’ study – the researcher knows the details of the

experiment but the person who is in contact with the research subjects doesn’t○ Both assistant and subjects are blind

External Validity

• Being able to generalize findings to the real world• Reactivity

○ Subjects might react differently in experiments because they know they are being watched○ Ex) Hawthorne effect – people responded to the attention by the researchers

• Field Experiments○ Experiments conducted in real scenarios○ Researcher has less control○ Tend to have greater external validity but lower internal validity. Reverse is true of lab

experiments○

• Chapter 8 – Analysis of Quantitative Data

Coding Data

• Coding means systematically reorganizing raw data so it can be analysed• Also condenses information• When data is not originally in the form of numbers (ex. Gender) assign numbers to variable• Keep all these assigned variables in a codebook – the key to deciphering your data• Coding procedure and codebook often created before data collection• Can place the codes directly on the questionnaire (precode) if not then coding and creating a

codebook is the first stage after data collection

Data Entry

• To enter into computers 4 options○ Code sheet – transfer data by hand onto a code sheet, then type into comp○ Direct-Entry Format – data coded at time of collection if entering directly into computer

○ Optical scan – use a scantron○ Bar code

Cleaning Data

• Must be accurate as errors will ruin research otherwise• Cleaning is when the researcher makes sure the data is accurate• May some part of the data a second time to check for errors, if any found the entry is redone• Can also verify by

○ Possible code cleaning – check categories for impossible codes (ie if gender is 1 or 2 and a 4 is entered)

○ Contingency cleaning- check cross categories for impossible combinations

Descriptive stats

• Describe numerical data, organised into univariate, bivariate and multivariate

Univariate (One variable)

• Frequency distribution easiest way to describe data• Can be used with nominal, ordinal, interval or ratio data• Represented in bar charts, pie charts, frequency polygons (scatter plots)• Central tendency

○ Single numbers that summarize characteristics of the data○ Mean, median, mode – averages○ See chart page 198 to see which measures are appropriate with what kinds of data○ If the frequency distribution forms a bell curve the three measures of central tendency

equal each other, if its skewed they do not○ See page 199 chart

• Measures of Variation○ Measures the variability of the distribution○ Measured by range, percentile and standard deviation○ Standard deviation requires interval or ratio○ Gives an average distance b/w all scores and the mean○ The standard deviation and mean are used to create “Z” scores – ‘standardized score”○ Z-scores let you compare groups by their placement within the broader context.

Ie) If I am a manager and make 50,000 and a manger at another company makes 30,000 z-scores would allow for comparisions by demonstrating our relative positions within our companies. I make 1/3 more than my employees and so does my fellow manager, so our positions are relatively similar despite the salary difference

Bivariate relationships

• Allow for descriptions of relationships• Based on the principles of covariation (coorelation - things vary along with each other) or

Independence (one variable does not affect the other)• To tell if there is a relationship 3 techniques are used

○ Scattergram/graph Can tell by form (is it linear, curvilinear (U), or no pattern) direction of line (positive/negative) Precision (how the points are distributed)

○ Bivariate table/ Percentaged table Start with a Contingency table (compares data of two variables in comparision also

includes a total) but as this is hard to interpret the data is converted into percentages

The result is the percentaged table If there is no relationship the percentages will be about equal across the board See page 206 & 207 for an example Can alternately use a bivariate table with central tendency instead of percentages if

one variable is not measured at the interval/ratio level○ Measures of association (stats)

Single number that expresses the strength of the relationship and direction of the relationship

Expressed as greek letters, each representing a different measure and used for diff forms of data

Probably not necessary, but detailed descriptions found on page 209

Multivariate

• Statistical Control○ Used to eliminate alternative explanations and spurious results○ In non experimental research this is the primary way to control alt exps○ They measure possible alternatives with control variables then place them in relation to

their other variables to help demonstrate if the relation is spurious○ This is usually multivariate, as one or more control variables are introduced○ If the previously noted relationships weaken or disappear after the control variables are

considered suggests spurious or no relation• Trivariate tables

○ Third variables the operationalized alternative explainations○ Trivariate tables are made of multiple bivariate comparisons/tables, they have bivariate

tables of the independent and dependent variable in comparison to each control variable○ These mini tables are called partials○ Difficult to interpret as they get larger

• Elaboration paradigm ○ – system for reading trivariate tables, describes the patterns that emerge○ Replication pattern – pattern the same as original, control variable has no effect○ Specification pattern – one partial replicates the original, but the others don’t – only a

partial relationship○ Interpretation pattern – control variable intervenes between dependent and independent

variables ( ie. Religious upbringing political ideology (control variable) stance on abortion)

○ Explanation pattern -- same as above but control variable comes before the independent variable ( ie. Gender (control variable) religious upbringing stance on gay marriage)

○ Suppressor variable pattern – when control variable suppresses true relationship• Multiple Regression Analysis

○ Controls for many alternative explanations and variables simultaneously○ Measure (R2) Shows how well a set of variables explains a dependent variable

By explains they mean reduced errors when predicting, so this measure is really telling us the accuracy of the variables’ prediction of the dependent variable

○ Also measure the size and effect of each variable on the dependent variable

Inferential statistics

• Use probability theory to test a hypothesis• Statistical significance

○ Means results probably not the result of chance○ Indicates the probability of finding a relationship in the sample when there is none in the

population (checking to see what the odds are that the sample is an accurate reflection of the population and not some random error)

• Levels of Significance○ Can express statistical significance in terms of levels○ Ie) .05, .01 ect...○ These numbers are measurement of the amount of times the results are due to chance○ 0.05 = 5% of the time these results are based on chance

• 0.05 is the accepted sort of ‘top’ level – this is because if you get too specific there is a greater chance of being wrong – two types of logical errors

• Type I errors (when a researcher wrongly says a relationship exists) • Type II errors (when a researcher wrongly says a relationship does not exist)• Want to avoid both errors, but as the odds of making one kind of error decline the odds of making

the other increase

Chapter 9 – ‘Field Research’

• Used when looking to understand, discover or describe a group of interacting people• Study a group of interconnected ‘members’ in a fixed setting• Ethnography

○ Method used in anthropology○ Describes another culture and (in modern applications) seeks to understand it from the

viewpoint of it’s members○ Implicit assumption that members are interpreting life, behaviour is not meaning –

meaning is inferred and interpreted• Ethnomethodology

○ Combines theory, philosophy and method in the study of commonsense knowledge○ Form of epistemology – observes the creation and use of this knowledge○ Argues that method determines research findings just as much as the social reality studies○ Social meaning not stable but flexible, constantly in the process of being recreated

Field Research

○ Goal is examine social meaning and understand multiple perspectives in a natural setting○ Want to get inside the meaning system of others and then return to an outside viewpoint

to disseminate findings○ Very flexible study – steps more of a suggestion

Read lit, gain access to site, learn the ropes, collect data, analyze data and begin to develop a hypothesis, disengage, report

○ Qualitative – begins with a general topic and discovers new ideas leading to hypothesis• Selecting a site

○ Must provide rich data○ Should be unfamiliar

Easier to see cultural relations when you’re outside of the familiar

○ Must be practical Is it easy to get to? Will the ascriptive characteristics of the researcher limit access?

(ie. Gendered domains are often limited to that particular gender)○ Gatekeeps

Individuals with informal or formal authority who control access Often going through this person essential, and helps build rapport Can however be counterintuitive in certain situations ex) warden

• Learning the ropes○ Presentation of self

Important in gaining acceptance Must consider style of dress, appearance Doesn’t have to match those studied, but should be informal where appropriate and

formal where appropriate• Building Rapport & Relationships

○ Major step towards understanding members and feeling empathy○ Build trust and be charming○ Helps to be hot○ Entering into an exchange relationship helps maintain the relationships created○ Relationships usually temporary, may have to be broken to secure others and at end of

study when researcher disengages• Roles

○ Researcher can choose to create a new role for themselves or occupy an existing role○ Ex) can act as a john to study street workers○ Note that the roles available are limited by characteristics (race, sex, physical appearance)○ Also, if research is overt the researcher should attempt to normalize the role of researcher

so it is better understood

Data Collection

• Observation of details and careful listening to what and how it is said • Note taking takes more time than observation• Jotted notes

○ written quickly when observing○ just words and diagrams on whatevers available to jog memory later○ may have to be covert – a person furiously writing arouses suspicion

• Direct Observation notes○ Written with the aid of jotted notes but based mostly on memory○ Written immediately after leaving the field, very detailed, includes verbatim accounts of

conversations○ Contains only direct observations

• Research Inference Notes○ This is where inferred meaning is added○ Kept separate as the meaning of actions is not always evident and open to interpretation

• Analytic Notes○ Here observations interpreted theoretically – includes tactics, theory, ideas, decisions

• Personal Notes○ Private diary kept alongside, as personal state may colour recording or interpretation of

data• Maps and Diagrams

○ Recorded to help organize events in the field and convey the site to others○ Spatial, social and temporal○ see the chart on page 238 – I cannot be the only one who finds this hilarious

Data Quality

• Reliability○ Are research observations internally and externally consistent?○ Internally consistent – is data plausible given what is known about person ○ Externally consistent- cross checking with external sources○ Credibility of informers paramount

• Ethnographic Fallacy○ Occurs when larger social forces ignored, and focuses on immediate concrete situation○ If you take everything informants say at face value you ignore the larger forces they are

unaware of, or do not mention in casual conversation• Vaildity

○ Four kinds of validity○ Ecological Validity

events and interactions occur the same way if the researcher was not present○ Natural history

Describes how research conducted○ Member validation

Asks a member to validate account○ Competent insider performance

The researchers ability to pass as an insider

The Interview

• Unstructured, non directive in depth interviews – informal• Different from formal survey research interviews (very structured)• Field Interviews involve sharing experiences, more like a casual conversation• See page 242 for a direct comparison of formal and informal survey techniques• Types of questions

○ Descriptive, structural, contrast (analytical)○ As time progresses move from asking primarily descriptive questions to contrast questions

Leaving the Field

• Withdraw can take multiple forms, quick lengthy• Must tell members and fulfill obligations• Must consider the members of the group – relationships are usually broken but it is possible to

retain them

Focus Groups

• Qualitative technique where small groups of homogenous people are informally interviewed• Observe and facilitate conversation among members of group• Non directive

The Ethics of Field Research

• Deception

○ Covert research sometimes necessary to gain full knowledge, but it undermines trust○ Covert research is never preferable or easier than overt

• Confidentiality○ Members confidentiality must be upheld from other members of the studied group as well○ Fake names used in field reports

• Deviance○ May be observe or be involved in illegal activity○ Must uphold confidentiality and maintain trust

Chapter 10—‘Writing the Research Report’

• Report is not only a summary of findings but a record or the process and methodology• Must think about writing the report while conducting research and keep careful record of all

decisions• Usually intended for an academic or professional audience

The Writing Process

• Audience○ Tailor writing to a specific audience○ If writing a student report, for a prof

Should demonstrate an understanding of concepts Use technical terms when appropriate

○ For Scholars Do not need definitions of terms What they want is an extensive section on data analysis and results Must be linked strongly to theory

○ For Professionals Short and simple summary of methods and results Like to see Practical outcomes

○ General public Concrete examples Accessible language Practical implications

• Style and Tone○ Formal and succinct○ Professional○ Field reports are an exception – informal tone as their experiences interwoven with data○ Avoid elaborate language or moralizing – goal is to inform not entertain

• Organizing thoughts○ Suggests writing an outline before writing to sequence and group ideas

• Literary Review○ Rarely completed before research project○ Return to literature after finishing project as

New material may have been published Rereading old material will be more fruitful now that the project is focused

○ Page 253 summary of ASA style guide

Steps in Writing

• Prewriting

○ Make notes, prepare outlines, citations○ Think about what the structure will ultimately be

• Composing○ Freewrite to get ideas flowing, write bibliography

• Rewriting○ Revise and edit report○ Suggests rewriting at least 3 or 4 times○ Box of suggestions to clarify writing – page 256—probably not necessary, but could be

good supplemental info○

The Quantitative Research Report

• See chapter 13 page 318-319 for a simplified chart• Abstract/Executive Summary

○ At the beginning of article a short summary to orient readers○ Contains topic, research problem, basic findings and unusual methodology○ Executive summaries are found in applied research reports for practitioners○ They are longer – several pages compared to about 500-700 words

• Introduction○ Several subheadings can be used to present the problem

Intro, lit review, hypotheses ect...○ Includes a statement of research problem and what is being examined○ Provide background to question○ Link problem to theory

• Methods○ Describes how the researcher deigned the study & collected data○ Methodology○ Informs reader what type of study, how data was collected, how variables were measured

and controlled for validity, what is the sample and ethical issues• Results

○ Presents data, but does not analyze or interpret○ Uses minimum number of charts and tables to effectively convey data

• Discussion○ Interpretation of results○ Separated so results can be interpreted independently and readers can arrive at different

conclusions○ Relates data to hypotheses, discusses unanticipated findings, alternative explanations and

weaknesses• Conclusions

○ Summarize findings○ References and appendixes follow

The Qualitative research report

• Much less structured• Tend to be much longer – book length

○ Data hard to condense○ Try to create sense of empathy – detailed descriptions

○ Must explain methodology in depth as use less standard data collection techniques and theirs are likely to be unique

○ Explorative, often develop new concepts which require explanation○ Use more varied writing styles – more literary

• Field research reports○ No fixed format○ Theoretical generalizations and data not separated○ Try to balance data and analysis to avoid error of segregation—when separation is so stark

readers cannot see connection○ Less formal tone—experience of author part of study○ Attempt to provide detailed examples to defend validity—these reports often subject to

scepticism○ Organization

Varies widely Organised chronologically, by natural history (events as researcher discovered

them), zoom lens (thematically)○ Uphold confidentiality and protects subjects

• Historical Comparative Research reports○ No single way to compose○ Frequently structured as a narrative or by analytic categories○ Footnotes and endnotes describe sources/evidence○ Organized chronologically or topic○ Some reports mimic quantitative report style

Research Proposal

• Documents that presents a plan for a project to a review board for evaluation• Ex) dissertation• Purpose is to convince board you are capable of completing project• Proposal describes research problem and describes proposed methodology• Mimics a research report, but obviously excludes results, discussion and conclusion• Grant proposals necessary for most research—funds provide salaries, supplies, travel costs ect.• Researcher should investigate organization funding so as to apply to appropriate sources• How to be successful

○ Have strong C.V. demonstrating past successes○ Neat, professional proposals, with a detailed schedule○ Follow guidelines exactly○ Address an important research question, advancing knowledge or with direct practical

applications○ High standards of methodology○ Include plans for disseminating knowledge○ Realistic financial planning and time budgeting

• Most proposals rejected 1st and 2nd times, reapplications common• Higher success rates after institutions give feedback proposals can be strengthened

No Notes on Chapter 13, as really need diagrams to understand it and most text repeat of organization of report (chapter 10)