Interpreting and Documenting

8/19/2019 Interpreting and Documenting

1/30

Interpreting & Documenting Research &Findings

Published by the Universities of Edinburgh, Glasgo and!trathclyde

W.L. Wilson

"c#noledgements The material from this booklet has been developed from discussion groupsand interviews with the research sta of Glasgow and StrathclydeUniversities

The advice and contributions of r !vril avidson" #r $eri avies" %rofGeorge Gordon" #rs &anice 'eid" r !lan Taylor and #rs Sheila Thompsonare acknowledged.

The advice of the pro(ect Steering Group) %rof #ichael !nderson"University of *dinburgh+ r ,uala -ooth" University of !berdeen+ r an/arter" University of Glasgow+ #s &ean /handler" University of Glasgow+ r

!vril avidson" University of Glasgow+ %rof George Gordon" University ofStrathclyde+ %rof /aroline #aconald" University of %aisley+ %rof &ames#cGoldrick" University of undee+ r !lan 'uncie" University ofStrathclyde+ %rof Susan Shaw" University of Strathclyde+ r !lan Taylor"University of *dinburgh+ %rof 'ick Trainor" University of Glasgow is alsoacknowledged.

The pro(ect was funded by the Scottish 0igher *ducation 1unding /ouncil.

$ther titles in !eriesGaining 1unding for 'esearch

Gathering and *valuating nformation from Secondary Sources

%reparing the 'esearch -rief

2 Universities of *dinburgh" Glasgow and Strathclyde 3444/artoons . -rown 5 W. L. WilsonS-, 6 789:3 :77 6 Printed by Universities Design and Print


2/30


3/30

Introduction

This booklet is one of a series of four aimed at researchers in the earlystages of their career life cycle. The comments within the booklet arebased upon information collected at a series of discussion groups andinterviews at Strathclyde and Glasgow Universities. The ;uestions put tothe discussion groups were based broadly upon the performance criteriaand knowledge re;uirements identiccupationalStandards in 'esearch= ?Gealy et al, 344@A.

The booklet is in two sections. The


4/30


5/30

%ontents

I'ERPRE'IG RE!E"R%( RE!U)'! "D FIDIG!**********************+

0>W > E>U />,1'# T0* '*L!-LTE >1 E>U' '*SULTSF..................................Introduction..........................................................................................4Points to Consider................................................................................4

0>W > E>U !H> G*TT,G ,T> ! 'UT WT0 E>U' !,!LET/!L #*T0>SF...........:Introduction..........................................................................................6Points to consider.................................................................................6

0>W W>UL E>U *1,* ,T*'%'*T!TH* #*T0>SF..........................................7Introduction..........................................................................................8Defnitions o "Interpretative".............................................................. 8

0>W > E>U '*/>G,S* !, !H> -!S , E>U' ,T*'%'*T!T>, >1 E>U'

'*SULTSF.................................................................................................... 4Introduction..........................................................................................9Points to Consider................................................................................9

0>W > E>U *H!LU!T* E>U' '*SULTS , T0* LG0T >1 T0* >-&*/TH*S >1 E>U' >'G,!L %'>%>S!LF..................................................................................33

Introduction........................................................................................11Points to Consider..............................................................................11

W0*, > E>U T0,$ U,/*'T!,TE #!E !'S* >H*' '*SULTS !, T0*' ,T*'%'*T!T>, !," 0>W > E>U *,SU'* T0!T E>U' />,/LUS>,S !'* 1ULLE &UST1* -E T0* '*SULTSF...........................................................................39


0>W > E>U *,T1E %>T*,T!L !'*!S >1 1U'T0*' '*S*!'/0 1'># T0* '*SULTSF...............................................................................................................3


D$%UE'IG RE!E"R%( RE!U)'! "D FIDIG!********************-.

W0!T T*/0,IU*S > E>U US* T> %'*S*,T E>U' 1,,GS" !, %>SS-L* !'*!S >11UTU'* '*S*!'/0 T> >T0*' ,T*'*ST* ->*SF...........................................3@


0>W > E>U '*/>' E>U' '*S*!'/0 !, 1,,GSF !'* T0*'* #*T0>S >1

'*/>',G T0!T E>U W>UL !H>F............................................................37Introduction........................................................................................18Points to Consider..............................................................................18

W0!T *T!LS > E>U %UT , E>U' '*S*!'/0 '*/>'SF W0!T *T!LS S0>UL ,*H*' -* #SS* >UT >1 '*/>'S" !, W0EF...............................................96


0>W > E>U />,1'# T0!T E>U' '*/>'S #**T !LL '*L*H!,T L*G!L !, *T0/!L '*IU'*#*,TSF.........................................................................................99



6/30


7/30

Interpreting Research Results and Findings

How do you confirm the reliability of your results?

Introduction The eCact nature of what is reliable will vary from


8/30

trials. 0is recent study eCamining the rearing of halibut ne engineer suggests the following summary for his own speciality)

aA erive from


9/30

variety of di9erent responses* $ne method of chec#ing theaccuracy of responses is to rephrase the 0uestion and thencompare the ne response ith the anser to the earlier0uestion* It is important to ensure that the analysis of the data isas inclusive of the varied responses as possible* $ne techni0ue byhich this can be done is the inductive procedure of deviant caseanalysis*

E2ample1 eviant case analysis proceeds through eCamination of theuniverse of responses provided to a certain topic. f eCploring the ;uestionof condom use" a basic hypothesis may be that prostitutes wouldencourage their clients to use condoms to ensure their own protectionagainst 0H. 0owever" there might be women who do not articulate theiruse in these terms at all" but refer to other reasons ?e.g. they form ameans of distinguishing between the seC they have with their privatepartners and the seC they provide to clientsA. The overall eCplanation forcondom use as a barrier would still


10/30

E2ample1 >ne research team studying the prostitute population of the redlight area in a large city decided to adopt the biological techni;ue ofMcaptureOmarkOrecaptureN. The study re;uired identi


11/30

There can be problems in securing funding for completely newapproaches.. 0owever attractive a new methodology may appear" it isimportant to ensure that the methodology will not discourage fundingbodies if it is included within your grant application. This can be a =catch99=. Eou want to be adventurous" but cannot move forward becausefunding bodies or collaborators will be cautious of the MeCcessiveN noveltyof your new idea. >n the other hand" in the highly competitive world ofresearch funding" you may need that bit of novelty as an added attraction.f in doubt it is well worth contacting your prospective funding bodies inadvance. Some funding bodies run schemes to promote =blue skies=research such as the 'esearch /ouncil" 'ealising our %otential !wards?'>%!NsA. Though original and novel are not one and the same" a pilot runwill move =novel to original= and help you convince the more scepticalreviewer.

• $eep up with the literature.

• ,etworking is essential.

• Look to other


12/30

used to (udge the model and" in the absence of physical proof" howaccurately does the model reDect what would happen in reality. ThediKculty in our ne researcher remarked that he wanted a particular solution because hewas sure that it was the correct solution" when in fact it was the wrongsolution. t was the interpretation of a brief for a housing developmentscheme. 0e tried to test what was actually meant by the relationship of

the dierent functional elements within the scheme. n this instance hethought he knew the answer because he had worked on similar schemespreviously. Thus" when told by a colleague that there was a mistake in hisinterpretation he failed to check it. The housing scheme was developed tothe full" and then collapsed because of that mistake. t was" he remarked"a painful eCercise often remembered" never to be repeatedP

1ew of us can claim to be completely free of such bias" the followingsection attempts to identify some of the areas where bias commonlyarises" and outline some techni;ues for its recognition.

Points to Consider /ias can arise in the construction of the e2periment rather than inthe interpretation* It is important to ensure that the e2perimental


13/30

design, or the behaviour of the researcher, does not introducebias long before the interpretative stages are reached*

E2ample1 ! team of ethologists were attempting to breed a moreintelligent strain of rat" intelligence being measured by maQe learning

abilities. !s the pro(ect proceeded it appeared that a superior strain ofintellect had been bred. !t least that was the conclusion until thetechni;ues of the workers involved were more closely eCamined. Theresearchers stroked the more MintelligentN rats before introducing them tothe maQe" but did not stroke the supposedly less intelligent rats. mprovedlearning was not a function of superior breeding" but rather of morepleasant handling conditions.

Discuss your results and interpretation of these results ithcolleagues* It is especially helpful to see# out colleagues fromdi9erent bac#grounds and e2perience* It is important to ensurethat the revie of your conclusions ill be genuinely critical,

there is little value to be gained from see#ing e2cessively politeor friendly colleagues*

E2ample1 >ne researcher remarked that a recently retired professortended to think in a dierent way to most of his colleagues. There were afew colleagues who" when a document was put in front of them" wouldreact in a predictable manner. -ut this professor tended to throw up ;uitedierent points from the document. 0e had a very dierent backgroundfrom the rest of the group" as well as having a wider range of eCperience.0e had worked in industry for a number of years" and had had a lot ofeCperience in vaccine development and trials" and marine biology. The

areas in which the researcher was interested" but from a ;uite a dierentperspective.

In intervie4based research, the perception of the intervieerand his;her e2perience of life, has considerable potential toradically colour his;her interpretation of events around them*

E2ample1 The researcher who had been conducting a research pro(ectstudying prostitutes on the streets of Glasgow initially assumed that theywould be afraid of the police" when many did not care about the police atall. The researcherNs interpretation of how they would react to beingapprehended by the police was inDuenced by her own background. She

stood to lose a lot by being prosecuted for anything" the prostitutes" onthe other hand" felt that they had little to lose.

Re


14/30

researcher. rug users re;uire" in their search for drugs" goodmanipulatory skills" and become very skilled social actors. They will oftenonly give an interviewer as much information as they estimate heOshealready possesses. !s the researcher entered more deeply into the


15/30

How do you evaluate your results in the light of the objectivesof your original proposal?

Introduction

!daptation of new methodological techni;ues was identi


16/30

possible in their publications. The reason for the failure was probably thatthe research team had inadvertently eCceeded the capacity of the systemand eCtrapolation of the techni;ue to larger systems was impossible.

• 0ow certain are you that the claims made by others are accurateF

•*nsure an ade;uate allowance of time for retraining and developinguntested techni;ues.

• /onsider that you may not have failed" perhaps the results were not

positive" but that in itself need not constitute failure.

• UneCpected results may result in the research progressing along a

dierent route.

When do you think uncertainty may arise over results andtheir interpretation and, how do you ensure that your

conclusions are fully justified by the results?

Introduction There are a variety of reasons for which interpretation and results maybecome confused or overBinterpreted. These range from decisions maderegarding modine method of avoiding confusion between interpretation and results is to

leave the work alone for siC months. !fter that period when you haveforgotten some of the background" and are free of any unhelpful habits ofthinking you may have inadvertently fallen into" then you should be betterable to spot any confusion between interpretation and results" or for thatmatter bias in interpretation.

When repeated trials provide conDicting results any decisions takenregarding the relative reliability or accuracy of the various results is aninterpretation of the results" and should be noted as such.

/learly dierentiate between the results per se and your

eCtrapolationOinterpretation of them.


17/30

! common error is to confuse a correlation between two variables and anactual cause and eect. The magnitude" signi


18/30

How do you identify potential areas of further research fromthe results?

Introduction>pportunism is one watch word in the identipportunism takes many forms" e.g. capitalising on currenttrends and fashions" and spotting weakness in present methodologies. Thesecond key is to keep an eye on the long term ob(ective. The ability toavoid becoming lost in the woods can be improved by a regularinterchange of ideas with colleagues.

Points to Consider Going away to a ma(or conference can be a good way of focusing on theresults of the past year. t both allows you escape the distractions of yourusual routine and" in conversation with others" identify the direction inwhich your cientsystem, ill often provide ne avenues of research*

E2ample1 The ma(ority of currently available vaccines are based on either

inactivated virus or bacteria" or comprise a synthetic or =recombinant=protein which has been produced in bacteria. The production costs for


19/30

these types of vaccine are high" and some improperly inactivated vaccineshave been responsible for outbreaks of disease. These problems haveencouraged researchers to evaluate the potential of nucleic acid or ,!vaccination as an alternative. Using this techni;ue the ,! is in(ecteddirectly into the animal or person" there is no risk of infection since thewhole virus or bacteria is not used" and there is no costly production andpuri


20/30

Documenting Research Results and Findings

What techniques do you use to present your findings, and possible areas of future research to other interestedbodies?

Introduction>ther interested bodies are a varied group" both in their understanding ofyour sub(ect" and their speci


21/30

the length of ne solution may be to present aneCecutive summary of the research. This increases the likelihood of itsbeing read by focusing all of the ideas into a short and concise section" butof course it leaves out all the proof" evidence" arguments and counterarguments. #ultiBmedia productions oer considerable potential in this

area. !lthough more compleC and eCpensive to produce" they allowreaders to look through your research and pick out what interests them" by

(umping from one point to the another.

When seeking funding consider emphasising the Mbene


22/30

'esearch records should be kept in a form which ensures that theirauthenticity can be appropriately defended. /laims of originality andscienti


23/30

discarded the other branches over the years" the remainder can bediscarded. >n the other hand" archiving old material" even if you do notbelieve you will return to it" may allow you to refer back to a solution to aproblem which you have had to deal with previously.

#ake sure that your record system is accessible" it is of little use if youhave to walk through half the building to access it" or if you have one typeof computer system at home and another in the oKce. Similarly" try not toput it on some obscure computing system that is likely to vanish within theneCt few years.

'a#e care of here and ho you record your list of ?things to do=*%onsider mentioning your ob6ectives to others at co9ee time 4 insi2 months time they may remind you*

E2ample1 >ne researcher admitted that on moving oKce recently shefound a list of Mthings to doN dated four years earlier B none of which she

had done.

• 'esults and methods should be recorded in a manner which can

leave no doubt as to their authenticity.

• !ll records should be signed" dated" and witnessed.

• $eep duplicates.

• 'ecord labelling must withstand the test.

• 'ecords must possess physical longevity.

• That list of Mthings to doN must be high proS) Technology Hentures B ntellectual %ropertyGuidelines ?3447A. >n the whole it is better to err on the side of cautionwhen deciding what to leave in and what to leave out.

Points to Consider US and U$ patent laws are not identical. Thus in the US evidence of thedate of conception of an invention" and proof of diligence in its reductionto practice is re;uired for patenting.


24/30

=*rrors and mistakes should not be erased or obliterated beyondrecognition. ,either should li;uid paper be used. Simply crossing out anerror so that it is apparent what the error was should be ade;uate. *Cplainall errors and mistakes as they occur and initial them. ,ever removepages from the notebook.= ?-TG plcA

'ecord novel concepts and ideas relating to the work though avoid theeCpression of opinions ?-TG plcA.

#any of the researchers in our discussion group kept a day book in whichthey recorded everything they had done that day. The book might includethe chemicals used and in what ;uantity" and anything that had gonewrong. !ny results which come from a printer should be put into that daybook ?if these are perishable printouts" copies must be madeA. Tables ofresults would go into a separate folder.

When recording data it is selfBevident which data set is which. 1ive years

down the line and it is highly likely that you will have forgotten which dataset is which. t is vital to sort and clearly label computer held data fromday one" especially if the computer records will include earlier and laterversions of the same data set. 'ecords should be in a form which canreadily be understood by everybody. This is necessary partly because indebate over patent rights it is vital that the records should be easilyunderstood" and partly for the reasons above B you will feel silly if atsometime in the future you admit you cannot understand your ownrecords.

Log the incidental. There may be conteCtual events or activities which

aect the data" climatic conditions" onBgoing politicalOnewspapercampaign" delay in sampling ?-rown et al." 3448A.

"lthough the comments made in the e2ample belo ere valid,the potential for the records to serve more than one purposeshould be borne in mind* 'he main aims ere to ensure that theor# as repeatable, and that the details ere ade0uate forprotecting intellectual property rights 3hoever, see @Diligence@5*In this light the criticism of research assistants includingirrelevant details as valid* 'he records ould allo theresearcher to point out that hile they accepted that a particularob6ective as not met, nonetheless, it as not their

responsibility*

E2ample1 nclude anything which is remotely likely to be re;uired oruseful. Eou come to recognise through eCperience ?which does not takelong to ac;uireA when you have failed to record points that are going to beneeded in as much detail as possible. 0owever" it is interesting to see theirrelevant details that some people have recorded. 0uge amounts ofirrelevant information" for eCample" was away on holiday" or somethinghas not yet arrived. The critical issues are the date" and a couple of lineson the ob(ective and methods used. n terms of the details" theeCperimental method ?especially if it deviated in any way from the

standard protocolA" and the results" should go straight into the book whichis the dayBtoBday record B in the book not (ust on any piece of paper.


25/30

0owever" -TG would tend to support the researcherNs method ofrecording. =iligence in the reduction to practice of an invention meansthat" as far as possible" generally steady" uninterrupted and constant workoccurred following the conception of an invention. n an interferenceaction ?where %' is challengedA periods of inactivity could lose the case"especially in a situation where each day is critical. !ll activities must belogged" even if it is only to note that you were waiting for" say" sampleanalysis that resulted in delay in the proceedings"= ?-TG plcA.

When recording the results of pilot studies be careful not to be more laCthan is normal in terms of the ;uality of the information recorded.

Data must be recorded carefully* Resist the temptation to recordthe data in rough form and transpose it to your noteboo# at alater date 4 this provides an e2tra opportunity for the introductionof errors 3assuming you get round to it in the 7rst place5* 8henrecording data from instruments, note the settings on the

instrument panel*

'hree e2amples1

3i5! Duorimeter value of M36. unitsN is meaningless and cannot berechecked without notes on scale widths" scale eCpansion factors"wavelengths and all other machine settings.

3ii5 uring electrophoresis note the current and voltage+ this will allow youto calculate the resistance of the gel" and spot a buer of incorrectconductivity.

3iii5 n a chromatography run" note Dow rate" column back pressure"detector settings" column type and" if there is more than one column ofthat type in the laboratory" the serial number of the column ?-eynon"344JA.

• o not eCclude data which you only think may be signi


26/30

Points to Considern ethical terms make certain that your volunteers understand the ethicalpromises you have made them. !lthough your eCplanation may seemclear to you" ask the sub(ects some ;uestions on what you have promisedthem" they may not have understood after all.

E2ample1 >ne researcher working on prostitution and 0H eCplained to allher sub(ects that a Mdouble blindN system was being used. This meant thatthe results of the 0H test could not be identince begun the animals?even during non eCperimental periodsA re;uire constant supervision. This

must be taken into account when planning the pro(ect. !ppropriateeCperimental records to meet the re;uirements for the annual 0ome>Kce returns must be kept. >rganisations such as the #inistry for!griculture and 1isheries ?#!11A will want to know whether you haveconformed to standards such as S> 9666 or Good Laboratory %ractice?GL%A" S> being a standard for eCperimental procedure and the recordingof data. GL% again has particular re;uirements. These often tend to be inthe form of standard checks. Thus results may have to be conKce re;uirements thenthere is a very clear line of responsibility and there will generally besomeone in your epartment responsible for 0ome >Kce re;uirements.

Eour institution will have sta who are responsible for such issues. -e sureto seek out their advice at an early stage of pro(ect planning.

t is not unheard of for data to be destroyed when they have not providedthe eCpected results" or in order to avoid a closer scrutiny of conclusions.

This is unethical. Eour institution will have a policy on scienti


27/30

E2ample1 >ne researcher recently published her dissertation B a Germandissertation B but when she went to the eCamination in Germanydiscovered that her work had been published by a member of sta.

Eou must obtain written permission from respondents to cite eCtracts from

interviews in publications ?even if they have been anonymisedA.

If you are concerned ith legal aspects of animal research yourinstitution ill have an o>ce designated to anser suchen0uiries* If they cannot help, you must contact the (ome $>ce*'hey are very helpful and ill usually give you an anserimmediately*

E2ample1 When the researcher was working on a Kce to clarify the situationregarding the import and eCport of these bacteria and whether or not alicence is re;uired" because it is also the provider of such licences.

For information on safety re0uirements there ill alays besomeone ithin your Department ho ill be able to provide youith the necessary information* Aour institution may have acentral (ealth and !afety $>ce

E2ample1 n microbiology the pathogenic category of the material you areusing should be identiculties ofcarrying out research on one=s on colleagues* "lthough thee2ample is predominately one of gathering the basic data,nonetheless, there are points to be learnt regarding hathappens to that data after they become secondary publisheddata*

E2ample1 Stephen Waters was a teacher who decided" as part of an >penUniversity course" to investigate the role of his own 0ead of epartment.

0e went through a fairly prolonged negotiation period to reassurecolleagues as to his trustworthiness before embarking on his programme


28/30

of research. >f interest here are some of the comments he later maderegarding the guarantees he had given prior to undertaking the research.0e had promised all participants an opportunity to verify statements priorto production of the


29/30


30/30

References

/ell, * ?344JA oing Eour 'esearch %ro(ect) ! Guide for 1irstBTime'esearchers in *ducation and Social Science. >pen University %ress"

-uckingham. 3@: pp.

/eynon, R** ?344JA %ostgraduate study in the biological sciences) !researcherNs companion. %ortland %ress" London 383 pp.

/ron, !*, cDoell, E*, and Race, P* ?3448A 866 Tips for 'esearchStudents. $ogan %age Ltd" London. 39@ pp.

/'G plc $eeping a Laboratory ,otebook. Gulph #ills" US!. 39pp.

Gealy, * and %lar#e, D* ?3447A evelopment of an nterim Workplan for

the 'esearcherNs Lead -ody. #aloney and Gealy" 9B9: #ossbury 'd.London. J6 pp.

Gealy, *, 8estla#e, D*, & %lar#e, D* ?344@A raft >ccupationalStandards n 'esearch. #aloney and Gealy" 9B9: #ossbury 'd. London.84 pp.

!#elton, F* and 8al#er, )* ?3448A %ilot Study to !ssess the -ene

Documents

Interpreting and Documenting