INSTRUMENTATION AND DATA COLLECTION

INSTRUMENTATIONADIBAH ABDUL LATIF, PhDFaculty of EducationINSTRUMENT & INSTRUMENTATIONInstrument is the generic term that researchers use for a measurement device (checklist, test, questionnaire).

Instrumentation is the course of action (the process of developing, testing, and using the device).

INSTRUMENTInstruments fall into two broad categories, researcher-completed and subject-completed, distinguished by those instruments that researchers administer versus those that are completed by participants. INSTRUMENTResearcher-completed InstrumentsSubject-completed InstrumentsRating scalesQuestionnairesInterview schedules/guidesSelf-checklistsTally sheetsAttitude scalesFlowchartsPersonality inventoriesPerformance checklistsAchievement/aptitude testsTime-and-motion logsProjective devicesObservation formsSociometric devicesINSTRUMENTINSTRUMENTOpen ended questions :- when there are a great number of possible answers or researchers cannot predict all the possible answersFor example a question about students reason for selecting particular university would probably be open ended or a question about a college major would be open ended because researchers dont want to the long possible majors.INSTRUMENTThings to remember when writing open-ended questions: Write questions clearly and concisely.Write questions by thinking about the reading level of the target population.Avoid double negatives.Avoid double-barreled questions.Example: Are you satisfied with the place and time of the program?INSTRUMENTOpen-ended questions are useful to explore a topic in depth however, open-ended questions are difficult to (i) Respond and (ii) Analyze.

Therefore, limit the number of open-ended questions to the needed minimum.

INSTRUMENTClose ended questions:- all the possible, relevant responses to a question can be specified and the number of possible responses is limitedExample- gender, ethnic, educational levelINSTRUMENTClose ended questionsMake sure that your answer categories are mutually exclusive.Example: What is your age group?Less than 20 years20-30 years31-40 years41-50 yearsAbove 50 years

INSTRUMENTClose ended questions

Are you a male or female?(fixed alternative)Are you a smoker?(yes or no answer)Please indicate your annual income level for the previous year: RM 0.00 to RM10,000.00 RM10,000.01 to RM 35,000.00 RM 35,000.01 and above(requires the researcher to include all possible responses and to make sure that the responses are mutually exclusive, i.e. only one answer is possible and there is no overlapping categories of income)INSTRUMENTAdvantages of Close-ended Questions:Limiting the answer options makes data analysis easierEnsure desired information is obtainedCan increase the reliability of the studyDisadvantagesOther equally important information may not be retrievedSome respondents may become frustrated with limited yes and no responsesDifficult to know whether the answers were guessed or actually knownIf a question not answered difficult to know whether the respondent purposely did not answer or it was inadvertently missed

INSTRUMENTBoth formats can be used in the same questions. 1. What type of writing assignment do you typically require in your course? a. Reports b. Themes or essays c. Research papers d. Take home test e. Other (Please specify) ____________________ DEVELOPING INSTRUMENTUse research questions, theoretical framework and literature as a guide in constructing instrumentResearchers are communicating with respondents using questions from the instrument. So, respondents must really understand what is the heart of the instrument. This called communication validityDEVELOPING INSTRUMENTReview literature in the domain which you wish to measure (i.e., generic skills"). Conceptual Definition - CD is an element of the scientific research process, in which a specific concept is defined as a measurable occurrence. It basically gives you the meaning of the concept.

DEVELOPING INSTRUMENTConstruct formation - Develop a list of subscales also known as constructs that you wish to sample from the domain. e.g. : Conceptual definition = generic skillsConstruct Formation = communication skills, problem solving, ethics, adaptability and team working

DEVELOPING INSTRUMENTOperational Definitions An OD is a process by which the characteristics of a concept can be defined, including identification and classification. Also known as itemExample for OD (i) I prefer to work in group compared by individually.It is advisable to write at least 5 items/statements for each construct for the first phase.

DEVELOPING INSTRUMENTDraw blueprint / meta data analysis and give sources for each item to guarantee the content validityGive the instrument to at least 3 experts to validate (content and language experts)Calculate the level of agreement among the experts

DEVELOPING INSTRUMENTDo pilot test on the instrument after validationYou can run many statistical analysis to check the validity and reliability of the itemE.g. : Exploratory factor analysis, Confirmatory factor analysis, Principal Component Analysis, Unidimensionality etc.

DEVELOPING INSTRUMENTFactor analysis can help to check construct validity. Review whether each item conceptually belongs with its factor (subscale) and remove those which do not.Check the item fit or factor loading for each item either its belong to the given construct or not.

DEVELOPING INSTRUMENTRun reliability test such as item and person reliability or Croanbach's Alpha for each factor/category (subscale) to investigate internal consistencyreliability.Modify and retest the instrument if necessary (alpha