Creating an Effective Test or Survey Instrument

Creating an Effective Test or Survey InstrumentAssessing learning (Horton, 2012)

1

FIGURE 1 Types of Assessment

Creating an Effective Test or Survey InstrumentAssessing learning (Horton, 2012)

1Assessment & Evaluation2

FIGURE 2 Picture of typical classroom assessment settingWhen are formal test or survey needed?

2Assessment & EvaluationPre-design:3

FIGURE 3 Picture of typical predesign assessment considerationsWhy or what do we want to accomplish from the survey results 2. collecting feedback on a ongoing basis allows one to see possible trends

"What do we need to learn from this survey?"

"What do we want to do with this data?

(My Market Research Methods 2011)

3Effective design begins with a planWhy the need for a test or survey?4

FIGURE 5 Picture of assessment considerationsFIGURE 6 Picture of survey design considerationsThe first step in designing a survey or test, is what information one is attempting to secure and for what purpose.In an educational setting, one may need to design a test to determine students level of accomplishment or areas needing additional attention.In both the business and educational setting, it is necessary to be certain of the target population, allowing for appropriate and relevant questioning.The following slides outline specific design steps, which can be applied to both the business and educational setting, with some modifications.

4Assessment & EvaluationSelecting Your Sample:

5

FIGURE 7 Picture of survey sample group selection processThe first step in effective survey design is to have a plan:

1. What business decisions am I trying to inform?2. If I knew __?__, I would be more prepared to make these important business decisions.3. What am I trying to measure? Attitudes? Behavior? Intentions? Perceptions?4. Who is my audience?5. What kind of statistics do I want to come out of this project (e.g. descriptive vs. inferential statistics)? What will my analysis look like?(Creative Research System ,2013)

deciding what kind of people to interview. (target population)

to decide how many people you need to interview. decision about your sample size based on factors such as: time available, budget and necessary degree of precision.

(Creative Research System ,2013)

If the correct people or population are not correctly selected, then the outcome of the survey test will not be accurate. Sample size must be large enough to ensure that one have a representative sample of the target population.5Survey/test question types6Assessment & Evaluationwhat kind of information are you trying to gather and what are you willing to do in order to get this information(Horton, 2012)

The types of questions included in a survey or test is driven by both the desired result such as the opinion, taste, of a companys customers or identifying students accomplishments or need for remediation, and the manner in which results are recorded.Questions can be subjective, seeking an individual response to a set of questions and scored manually with feed back.Questions can also be objective, seeking responses from a clear standard of correctness. These questions are easily scored by computers for efficiency and correctness.

Pre-test the QuestionnaireIdeally, you should test the survey on the same kinds of people you will include in the main study

(Creative Research System ,2013)

6Questionnaire Design /General Considerations

7

Assessment & EvaluationFIGURE 8 Picture of survey sample questionnaire designThe first rule is to design the questionnaire to fit the mediumKISS - keep it short and simpleAsk yourself what you will do with the information from each question. If you cannot give yourself a satisfactory answer, leave it out.place your questions into three groups: must know, useful to know and nice to knowStart with an introduction or welcome message(Creative Research System ,2013)

KISS - keep it short and simpleA good introduction or welcome message will encourage people to complete your questionnaire.

Allow a Don't Know or Not Applicable response to all questions, except to those in which you are certain that all respondents will have a clear answer(Creative Research System ,2013)

Researchers use three basic types of questions: multiple choice, numeric open end and text open end (sometimes called "verbatims")

Educators make use of similar types of questions to have students make specific judgments, or exhibit critical thinking skills or open ended questions requiring individual narratives.(Creative Research System ,2013)

7Question sequencingmeasure learners ability to put items into meaningful order[by] making judgment about relationships among items in the list(Horton, 2012)8

FIGURE 9 Picture of survey sample sequencing processSequence type questions tests students ability to recognize the relationships, between items with the same or different composition, function, order etc. These sequence - type tests, tests student critical thinking skills, memory and associations skills.In the field of business, the order of survey questions, makes it certain that customers would be more inclined to respond. Easy questions first then more difficult questions at the end and the grouping of similar or related questions ensures ease and completion of survey questions.

Answer choice order can make individual questions easier or more difficult to answer. Whenever there is a logical or natural order to answer choices, use it.Always present agree-disagree choices in that order.For the same reason, positive to negative and excellent to poor scales should be presented in those orders. When using numeric rating scales higher numbers should mean a more positive or more agreeing answer.(Creative Research System ,2013)

Other General TipsKeep the questionnaire as short as possible.Start with a Title (e.g., Leisure Activities Survey). Always include a short introduction who you are and why you are doing the surveyThe overriding consideration in questionnaire design is to make sure your questions can accurately tell you what you want to learn. The way you phrase a question can change the answers you get. Try to make sure the wording does not favor one answer choice over another.Make sure your questions accept all the possible answers(Creative Research System ,2013)

8Feedback9

FIGURE 10 Picture of students receipt of survey or assessment resultstell learners how well they did on the testrelieve the anxiety and let the celebration or remediation begin.(Horton, 2012)

Effective feedback should be timely and detailed. Business depend on timely and detailed customer feedbacks to manage, advertisement, introduction of new line of product, pricing etc.Students require feedback to relieve anxiety, celebrate their accomplishments or clarify misunderstanding. Immediate feed back also contributes to students motivation and willingness to learn.

Results processing or the analysis of survey or tests results, are driven by the purpose of the survey and the effects the results will have on the companys business decisions or students short and long term academic or work performance.Therefore, the processing and analysis of survey results or student tests scores should be objective and systematic. The bench mark for success should be consistent with the purpose and consequence of survey and tests results.The following slides outline a few considerations.

9Program Outcome Learning 7:Hyperlink will direct you to a completed sample survey utilizing Survey Monkey!

http://www.surveymonkey.com/s/C66QQ8S

10This Survey Monkey software was very easy to learn and uneventful! Especially, since I did review a number of online tutorials before beginningMany of tools resemble similar tools found on Moodle, platform that I currently use to create quizzes, communicate with students, list assignments etc.My survey included a number of the different features available. I included an image, multiple choice questions, completion or Text questions, and survey questions.

10ReferencesEpstein Educational Enterprises (nd). Retrieved on 6-14-13. from http://www.epsteineducation.com/home/about/Horton, W. (2012). E-Learning by design (2nd ed.). San Francisco, CA: Wiley.IT. (nd). Retrieved on 6-14-13. from http://www.cod.edu/it/blackboard/testFeedback.htmMy Brainshark. (2012). Retrieved on 6-14-13. from http://my.brainshark.com/PowerPoint-Slide-Makeover-Presenting-survey-results-when-multiple-answers-are-possible-56654656Mymarketresearchmethods (nd). Retrieved on 6-14-13. from http://www.mymarketresearchmethods.com/survey-design-best-practices/Slideshare (nd). Retrieved on 6-14-13. from http://www.slideshare.net/SurveyGizmo/best-practices-in-survey-designSurveygizmo (nd). Retrieved on 6-14-13. from http://www.surveygizmo.com/survey-software-support/tutorials/online-survey-best-practices/

11ReferencesFigure 1: http://www.scholastic.com/teachers/top-teaching/2012/02/interim-assessments-predict-student-performance-state-tests

12FIGURE 2 : http://www.1sqbox.com/FIGURE 3 http://www.ctb.com/ctb.com/control/productCategoryViewAction?productCategoryId=327&p=productsFIGURE 4 http://www.partners4learning.edu.au/news/dsp-default.cfm?loadref=29FIGURE 5 https://www.qualtrics.com/blog/determining-sample-size/FIGURE 6 http://help.verticalresponse.com/how to/tutorial/add_questions_and_other_content_to_a_survey/FIGURE 9 http://speechtimefun.blogspot.com/2012/11/winter-word-fun.htmlFIGURE 10 http://www.dreamstime.com/stock-photography-successful-university-students-over-white-background-full-length-portrait-arms-raised-jumping-image4019508212