1 ICTCM Pre session, Oct 2004 Assessment - overview G. Donald Allen Department of Mathematics Texas A&M University ICTCM – October 28, 2004

11ICTCM Pre session, Oct 2004ICTCM Pre session, Oct 2004

Assessment - overviewAssessment - overview

G. Donald AllenG. Donald Allen

Department of MathematicsDepartment of Mathematics

Texas A&M UniversityTexas A&M University

ICTCM – October 28, 2004

ICTCM Pre session, Oct 2004ICTCM Pre session, Oct 2004 22

SpeakersSpeakers

• Don Allen

• Mike Speed

• Dave Thomas

• Mike Pilant

• Diem Nguyen

• Greg Tobin


What is a “Student Learning What is a “Student Learning Outcome?Outcome?

Student learning outcomes are properly defined in terms of the knowledge,skills, and abilities that a student has attained at the end (or as a result) of his or her engagement in a particular set of education experiences.

Council for Higher Education Accreditation


The Purpose of an Assessment The Purpose of an Assessment Is: Is:

• to provide information about skills in a content area

• to guide instruction for the individual and for groups

• to provide feedback or information • to plan further instruction • to improve instructional practices • to focus on what the student can do • to guide decisions about instruction (Formative)


The Purpose of a Test/Exam Is:The Purpose of a Test/Exam Is:

• to provide feedback about skills learned or obtained via instruction

• to determine the areas in which an individual needs re-teaching

• to provide grades

• to focus on what the student cannot do

• to indicate attainment of skills (Summative)


Characteristics of Educational Characteristics of Educational MeasuresMeasures

• Educational measures are– Indirect– Incomplete – Relative– Used as classification

• Educational measures have errors


Desirable test attributesDesirable test attributes

• Reliability

• Validity

• Objectivity

• Balance

• Fairness

• Practicality

Test length should be

1. Long enough to be reliable

2. Short enough to be administered


Goals for making testsGoals for making tests

• Be concise

• Be singular – focus on one aspect

• Be realistic

• Use definite terms

• Delineate expected behavior


Blooms taxonomyBlooms taxonomy

1. Knowledge (basic)

2. Comprehension

3. Applications

4. Analysis

5. Synthesis

6. Evaluation (advanced)


Simplified Bloom’s for testingSimplified Bloom’s for testing

I. Knowledge (basic)

II. Understanding – ability to restate, interpret

III. Applications – beyond restatement to include analysis, synthesis, and evaluation


Function of a testFunction of a test

• Advancement (criterion referenced)

• Achievement of objectives (norm referenced, items should discriminate well)


Cognitive testing – another viewCognitive testing – another view

• Recall

• Recognition

• Differentiation

• Constructed response

• Problem solving


Capacity to assessCapacity to assess

• Ability

• Achievement

• Personality

• Skill

• Opinion

• Attitude


Assessment Assessment What can it do? What can it do?

• Provide– to students a chance to reflect on what they

have learned and need to learn– to teachers a chance to reflect on what they

have taught well and what needs further attention



• Provide feedback– to the teacher on the clarity of given

assessments– to teachers about the progress of learning



• Providing– clarity to students on the type of mathematics

knowledge valued– clarity to students on the type of mathematics

proficiency valued



• Become a routine part of ongoing classroom activity

• Promote regular class attendance

• Promote multiple solutions and approaches, giving a well rounded picture that allows each student to show his/her best strengths


What Counts as Evidence What Counts as Evidence for Student Learning for Student Learning

Outcomes?Outcomes?

“Evidence” refers to the kinds of information about student learning outcomes that is most appropriate to accreditation settings.

Council for Higher Education Accreditation


Evidence vs. AnecdotesEvidence vs. Anecdotes

However… the plural of anecdote is not evidence!

“Evidence” can simultaneously embrace the results of quantitative and qualitative approaches to gathering information


Types of evidenceTypes of evidence

•Comprehensive examinations and assignments.

• Performance on licensing examinations.

• Professionally judged performances.

• Portfolios of student work.

• Samples of representative student work.


What is a “Standard” of Student What is a “Standard” of Student Learning?Learning?

“Standard" refers to a specific expectation or level of performance that an institution or program establishes for student learning.

Standards are the point of comparison against which to judge the actual evidence of student learning once it is collected.


AssessmentAssessmentCapacity to AssessCapacity to Assess

• Achievement

• Personality

• Skill

• Opinion

• Attitude


Testing for what?Testing for what?

• PREDICTION• EVALUATION • DIAGNOSIS• SCREENING


Assessment WeaknessessAssessment Weaknessess

• Lack of content validity

• Lack of scoring economy

• Scorer unreliability (leniency, severity)

• Halo effect

• Informed judgement

• Extraneous factors (quality of hand-writing, pen vs. pencil, spelling errors)


Essay questionsEssay questions

• Item types– Short answer– Extended answer

• Item Construction– Focus the direction of the response


Guidelines for essay questionsGuidelines for essay questions

• Allow adequate time

• Should have a problem to be solved– Problem should be defined explicitly– Problem should be limited

• Directions should be explicit

• Don’t use optional questions

• Construct detailed key for each question


Assessment – It is…Assessment – It is…

• Broadly defined and used• Complex• Detailed• Wrought with unfairness• Essential• Different in different contexts


EnginesEngines

• ALTEC – online tools for teachers http://altec.org/index.php– QuizStar– Rubistar– Profiler Pro

• Survey Monkey - http://www.surveymonkey.com/

• Blackboard, WebCT


Forms of AssessmentForms of Assessment

• Collaborative/group projects • Direct observation Essays • Examinations (unseen), Examinations (seen/open book) • Multiple-choice tests, Oral questioning after observation • Performance projects, Portfolios • Practical projects, Presentations • Problem sheets, Projects • Self-assessment, Case-studies• Short-answer questions


Two simple assessmentsTwo simple assessments


The Muddiest PointThe Muddiest Point

• What point was least clear during this session?

• What point needs further clarification?

• What point prevented me from learning fully the contents of this session?


The one minute paperThe one minute paper

1. What was the most useful or meaningful thing you learned during this session?

2. What questions remain uppermost in your mind as we end this session?

Answer each question in one or two sentences.

Documents

1 ICTCM Pre session, Oct 2004 Assessment - overview G. Donald Allen Department of Mathematics Texas A&M University ICTCM – October 28, 2004