Providing Students with Effective Feedback

AssessmentPD Cadre WorkshopJune 26, 2014Not everything that counts can be counted. And not everything that can be counted, counts.- Albert Einstein

2Plan For The DayCurriculum, Instruction and AssessmentWhat is Assessment?Instructional Design Model (UBD)Target-Method MatchFeedbackData Conversations

RubricsRigor and Relevance

ResourcesInformation you may already knowResources to support you in your PD rolesWikiSpace: http://harlemassessment.wikispaces.com/Take a minute now to join. Then review the site.

Change

Reminder!!Helping teachers move toward a different way of assessing student achievement and growth can be very challenging for them. It is a process.5Harlem Learning ProcessNot separate, discreet events, but work together to create a strong instructional process.6CurriculumInstructionAssessmentBased on New Illinois State Learning StandardsDelivery of Curriculum ContentEngaged Instructional Strategies Time on TaskBalanced SystemOf and For learningInforms instructional practice

Curriculum Guides: Skills or content that will be taughtCurriculum Aligned with Standards and State Accountability AssessmentsData-Informed Decision Making ProcessCurriculum, Instruction and AssessmentStudentAchievement

7What Role Does Assessment Play in the Instructional Process?What distinctions do you make between "testing" and "assessment"?

Turn and TalkTesting is a small part of assessment. It needs to be part of the picture. Many people who are anti-testing end up sounding anti-evaluation and anti-measurement. A good test has a role to play. The language that we like to use is, it's an audit. It's a snapshot.. You want more than a snapshot, you want lots of snapshots, you want a whole family album, a movie.

9Why is it important that teachers consider assessment before they begin planning lessons or projects?

Turn and TalkBefore you decide exactly what you're going to do with them, if you achieve your objective, what does it look like? What's the evidence that they got it? What's the evidence that they can now do it, whatever the "it" is? So you have to think about how it's going to end up, what it's going to look like. And then that ripples back into your design, what activities will get you there. What teaching moves will get you there?10Assessment inspire us to ask these hard questions: Are we teaching what we think we are teaching?Are students learning what they are supposed to be learning? Is there a way to teach the subject better, thereby promoting better learning?When assessment works best, it does the following:Provides diagnostic feedbackWhat is the student's knowledge base?What is the student's performance base?What are the student's needs?What has to be taught?

When assessment works best, it does the following:Helps educators set standardsWhat performance demonstrates understanding?What performance demonstrates knowledge?What performance demonstrates mastery?

When assessment works best, it does the following:Evaluates progressHow is the student doing?What teaching methods or approaches are most effective?What changes or modifications to a lesson are needed to help the student?

When assessment works best, it does the following:Relates to a student's progressWhat has the student learned?Can the student talk about the new knowledge?Can the student demonstrate and use the new skills in other projects?

When assessment works best, it does the following:Motivates performanceFor student self-evaluation:Now that I'm in charge of my learning, how am I doing?Now that I know how I'm doing, how can I do better?What else would I like to learn?For teacher self-evaluation:What is working for the students?What can I do to help the students more?In what direction should we go next?

Think about your own assessment practice.Turn and TalkHow do you approach assessment in your classroom?Share how you approach assessment in your classroomConsider bullet points just discussed17

Measuring Mastery

Comprehensive Balanced AssessmentA comprehensive balanced assessment system includes:State (Accountability) AssessmentsInterim/Benchmark AssessmentsClassroom AssessmentsFormativeSummative

Each should be aligned to standardsState AssessmentsAccording to The US Department of Education, The purpose of state assessments required under No Child Left Behind is to provide an independent insight into each child's progress, as well as each school's. This information is essential for parents, schools, districts and states in their efforts to ensure that no child--regardless of race, ethnic group, gender or family income--is trapped in a consistently low-performing school.

Benchmark/Interim AssessmentsBenchmark assessments are assessments that are administered periodically throughout the school year, at specified times during a curriculum sequence, to evaluate students knowledge and skills relative to an explicit set of longer-term learning goals (generally a semester or school year).

Universal ScreenerIn the context of an RTI prevention model, universal screening is the first step in identifying the students who are at risk for learning difficulties. Universal screening is typically conducted three times per school year, in the fall, winter, and spring. Universal screening measures consist of brief assessments focused on target skills (e.g., phonological awareness) that are highly predictive of future outcomes .

Assists in identifying grade-wide deficits in curriculum and instruction.Provides a baseline for grade-wide goal setting.Identifies students at risk of academic or behavioral difficulties.Can generate local norms and benchmarks.Screening data, while brief, is authentic, timely, and your first indicator of difficulty with your school, class, or student

Progress MonitorProvide on-going indication of instructional effectiveness Inform decisions regarding changes in instructional programs/interventionsProvide data for level of responsiveness to interventionA General Outcome Measure (GOM), application of skills learned

Summative Assessment Summative assessments are a measure of achievement to provide evidence of student competence or program effectiveness. Summative assessments are found at the classroom, district and state level and can be graded and used in accountability systems. The information gathered from summative assessments is evaluative and is used to categorize students so performance among students can be compared.

Formative AssessmentA process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to help students improve their achievement of intended instructional outcomes. Formative assessment is found at the classroom level and happens minute-to-minute or in short cycles. Formative assessment is not graded or used in accountability systems. The feedback involved in formative assessment is descriptive in nature so that students know what they need to do next to improve learning.

District Assessment FrameworkLevel Of AssessmentElementaryMiddle SchoolHigh SchoolStatewidePARCCPARCCPARCC/PSAEInterim/BenchmarkSTARSTARACT AspireClassroom (Summative)Writing Benchmarks*Common AssessmentsTeacher Developed/From textbooksChapter/Unit assessments*Common AssessmentsTeacher Developed/from textbooks

Chapter/Unit AssessmentsCommon Finals*Common AssessmentsTeacher Developed/from textbooksClassroom (Formative)Determined by teacherDetermined by teacherCommon Pre-AssessmentsDetermined by teacherUniversal ScreenerK: STAR Early Literacy1st: STAR Early Literacy and STAR Math2nd-6th: STAR Reading and STAR MathSTAR Reading and STAR MathLocally developed reading and mathProgress MonitorReading:K-6:, STAR Reading, Running Records, AIMSWebMath:K-6: STAR, AIMSWebReading and Math7-8: STAR, AIMSWeb Running RecordsRunning RecordsAIMSWebThe Backwards Design ModelStage 1: Identify Desired ResultsLinked to Content StandardsStage 2: Determine Acceptable EvidenceOngoing AssessmentPerformance TasksStage 3: Plan Learning Experiences and InstructionSequence of experiencesScaffoldedDifferentiatedBefore InstructionDetermine what you want students to know and be able to doEssential learning, power standards/priority standardsIdentify learning progressions What if students dont know foundational or pre-requisite skillsReview current data to determine students current knowledgeCollect additional data as neededGroup studentsDevelop differentiated classroom instruction based on dataWork with Special Education Teachers and Consultants to determine how instruction will be supported for students with IEPsDuring InstructionWhat formative assessments (not just tests) will I use to determine if students are learning?How will I modify instruction based on that data?Examples of formative assessment (Checking for Understanding)http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessment.html ObservationsQuestioningDiscussionExit/Admit SlipsLearning/Response LogsGraphic OrganizersPeer/Self Assessments Practice PresentationsVisual RepresentationsKinesthetic Assessments Individual WhiteboardsLaundry Day Four CornersConstructive Quizzes Think Pair Share Appointment Clock As I See It

After InstructionDetermine effective and less effective instruction and modify for futureDetermine next steps

ExploreSummative example linksShare your suggestions/recommendationsKeys to Quality Classroom AssessmentClear PurposesWhy Assess?Whats the purpose?Who will use results?Clear TargetsAssess What?What are the learning targets?Are they clear?Are they good?Good DesignAssess How?What method?Sampled how?Avoid bias how?Sound CommunicationCommunicate How?How manage information?How report?Accurate AssessmentEffectively UsedStudent InvolvementStudents are users, too.Students need to understand learning targets, too.Students can participate in the assessment process, too.Students can track progress and communicate, too.34Target -Method MatchSelecting The Right Type of AssessmentClear Targets: Benefits to StudentsStudents who could identify their learning scored 27 percentile points higher than those who could not.~Marzano, 2005

Students can hit any target they can see that holds still for them.

2007 Educational Testing ServiceAssessment Training Institutewww.ets.org/ati 503.228.306036Making targets clear to students increases their achievement.

Students can hit any target they can see that holds still for them. This is common sense and many teachers do it already. The goal here is for all to fit this practice into their teaching and to understand why it increases student learning.Handout page 7A Math Example37GeometrySubjectPythagorean TheoremTopicChapter 10ResourceUse Geometric Tiles for proofActivityThe lengths of the three sides of a right triangle are relatedLearning TargetKinds of TargetsMaster content knowledgeUse knowledge to reason and solve problemsDemonstrate performance skillsCreate quality products 2007 Educational Testing ServiceAssessment Training Institutewww.ets.org/ati 503.228.306038We can think of achievement targets as falling into one of four categories:

Knowledge targets involve things like math facts, important dates in history, and grammar rulesanything we want student to know outright.

Reasoning targets are learning expectations that call for students to use the knowledge in some wayto create an hypothesis, or to analyze a political argument, for example.

Performance skill targets call for students to do things like give an oral presentation, read aloud with fluency, or use equipment correctly. A performance skill target is something we have to watch or listen to in order to evaluate.

Product targets specify that students will create a producta fitness plan in health, a physical model in science, or a research report in English. We evaluate the characteristics of the product to give evidence of achievement of these kinds of learning targets.Handout page 6KnowledgeReasonSkillProduct ListPredictMeasureConstructDefineInferDemonstrateDevelopUnderstandClassifyUse CreateRecognizeEvaluateOperateProduce ExplainSummarizeCalculateLearning Targets with Associated VerbsConverting Learning Targets to Student-Friendly LanguageIdentify important or difficult learning goal.Identify word(s) needing clarification.Define the word(s). Rewrite the definition as an I can statement, in terms that students will understand.Try it out and refine as needed.Have students try this process.

Student-Friendly LanguageWord to be defined: PREDICTIONA statement saying something will happen in the futureStudent-friendly language:I can make predictions.This means I can use information from what I read to guess at what will happen next.

4 types Learning TargetsKnowledgeReasoningPerformance/SkillProduct

Why Its Important to Determine Target TypeKnow if the assessment adequately covers what we taughtCorrectly identify what students know and dont knowKeep track of student learning target by target or standard by standardHelps determine HOW to assess (method)

Target -Method Match: What is it?A way to design assessments that cover our targetsAnswers ASSESS HOW?Types of Assessment MethodsSelected response & short answerExtended written responsePerformance assessmentPersonal communicationSelected Response (SR)Students select correct or best response from a list providedStudents scores are figured as the number or proportion of questions answered correctlyFormats include:Multiple choiceTrue/falseMatchingShort answerFill-in questionsExtended Written Response (EWR)Requires students to construct a written answer in response to a question or task (not select one from a list)Extended = several sentences in lengthExamples:Compare pieces of literatureSolve a math problem, show & explain workInterpret music, scientific info. or polling dataAnalyze artworkDescribe in detail an economics processExtended Written Response Correctness judged by:Giving points for specific info. present ORUse of a rubricScores can be:Percentage of points attained ORRubric scores

Performance Assessment (PA)Based on observation & judgmentRubricJudgment made on qualityExamples:Playing instrument; speaking in foreign language; working in a group (the doing/process is important)Creating products like a lab report, term paper, work of art (quality of product is important)

Performance Assessment 2 parts:Performance task or exerciseScoring guide/RubricScoring guide:Can award points for specific features of performance or productCan take form of rubric: levels of quality describedScores could be number or percent of points earned or a rubric score

Personal Communication (PC)Find out what students have learned through interacting with themOften an informal assessment, but if clear & accurate info. is gathered, can be used for feedback to students, self-reflection for students, goal setting

Examples:Oral examinationsInterviewing students in conferencesLooking at & responding to students comments in journals and logs

Personal Communication Student responses evaluated in 2 ways:Correct/incorrect (for short, simple answers; parallels scoring of written selected response questions)Evaluate quality (for longer, more complex; parallels to extended written response)Could use a rubric to score or scoring guideMatching Target and Assessment Methods Accuracy in the classroom assessment revolves around matching the different target TYPES with the appropriate form of assessment METHODSSelected Response Written Response PerformanceAssessment Personal CommunicationKnowledgeGoodCan assess isolated elements of knowledge and some relationships among themStrong Can assess elements of knowledge and relationships among themPartial Can assess elements of knowledge and relationships among them in certain contextsStrongCan assess elements of knowledge and relationships among them ReasoningGood Can assess many but not all reasoning targetsStrongCan assess all reasoning targets PartialCan assess reasoning targets in the context of certain tasks in certain contextsStrong Can assess all reasoning targetsSkillPartialGood match for some measurement skill targets; not a good match otherwisePoor Cannot assess skill level; can only assess prerequisite knowledge and reasoningStrongCan observe and assess skills as they are being performed PartialStrong match for some oral communication proficiencies; not a good match otherwiseProductPoor Cannot assess the quality of a product; can only assess prerequisite knowledge and reasoningPoorCannot assess the quality of a product; can only assess prerequisite knowledge and reasoningStrong Can directly assess the attributes of quality of productsPoor Cannot assess the quality of a product; can only assess prerequisite knowledge and reasoningTarget-Method Match

Effective Design Select a proper assessment methodSufficient sampling to demonstrate masterySelect or create quality items, tasks, and rubricsSamplegather enough evidence Control for biasDesign assessments so students can self-assess and set goals

55Which METHOD would you use to assess the following?Ability to write clearly and coherentlyPerformance assessmentGroup discussion proficiencyPerformance assessmentReading comprehensionSR, EWR or PC (not PA b/c its a reasoning target)Proficiency using specified math proceduresCould be knowledge or reasoning: SR, EWR, PCProficiency conducting investigations in sciencePerformance assessmentAuthentic assessment can include many of the following:ObservationEssaysInterviewsPerformance tasksExhibitions and demonstrationsPortfoliosJournalsTeacher-created testsRubricsSelf- and peer-evaluation

Lets try it.but first.Google DocsSet up your Harlem Gmail accountOnce you are logged in to your computer as yourself, please visit: http://goo.gl/idcffu. Setting your password through this link will sync your Harlem and Google account so you can activate your account.Your log in will be Firstname.Lastname@h122.org. Example: Albert.Einstein@h122.org. Your password will be the same password you set in the password reset portal.

How to guide is on the Resources page of our Wiki so you can share with teachers in your building.

Lets Try It!Go to the standards page on the WikiSpaceIn groups of 2 or 3, select one standardIdentify the target and write in student friendly language, I-Can statementsDetermine the type of target it isWhat /are the best method(s) to assess this standard?How would you assess mastery?

ShareGoogle DocsSet up your Harlem Gmail accountOnce you are logged in to your computer as yourself, please visit: http://goo.gl/idcffu. Setting your password through this link will sync your Harlem and Google account so you can activate your account.Your log in will be Firstname.Lastname@h122.org. Example: Albert.Einstein@h122.org. Your password will be the same password you set in the password reset portal.

How to guide is on the Resources page of our Wiki so you can share with teachers in your building.

Virtual Gallery WalkDebriefProviding Students with Effective Feedback

What is Feedback? Feedback is an objective description of a students performance intended to guide future performance. Unlike evaluation, which judges performance, feedback is the process of helping our students assess their performance, identify areas where they are right on target and provide them tips on what they can do in the future to improve in areas that need correcting.~ W. Fred MiserWhat Does the Research Say?Feedback seems to work well in so many situations that it led researcher John Hattie (1992) to make the following comment after analyzing almost 8,000 studies: The most powerful single modification that enhances achievement is feedback. The simplest prescription for improving education must be dollops of feedback.

~ Robert MarzanoWhat is Feedback?Research has shown that effective feedback is not a discrete practice, but an integral part of an instructional dialogue between teacher and student, (or between students, or between the student and him/herself).

From Providing Students with Effective Feedback

65What is Feedback?Feedback is not about praise or blame, approval or disapproval. Thats what evaluation is placing value. Feedback is value-neutral. It describes what you did and did not do.

~ Grant WigginsWhat is Feedback?Effective feedback, however, shows where we are in relationship to the objectives and what we need to do to get there. It helps our students see the assignments and tasks we give them as opportunities to learn and grow rather than as assaults on their self-concept. And, effective feedback allows us to tap into a powerful means of not only helping students learn, but helping them get better at learning.~ Robyn R. JacksonWhat is Feedback?Effective feedback not only tells students how they performed, but how to improve the next time they engage the task. Effective feedback is provided in such a timely manner that the next opportunity to perform the task is measured in seconds, not weeks or months.

~ Douglas Reeves, p. 227

Feedback FocusAcademicBehavioral

Primary Purposes of FeedbackTo keep students on course so they arrive successfully at their predetermined destination.~ W. Fred MiserIt is one thing to collect feedback about students progress, but if you simply collect this feedback and never use it to adjust your instruction, then you are collecting it in vain. The data you receive from grading your assignments and assessments will give you feedback about the effectiveness of your own instruction.~ Robyn R. Jackson

What Does the Research Say?grade, socioeconomic status, race, or school setting.When feedback and corrective procedures are used, most students can attain the same level of achievement as the top 20% of students.

~ Bellon, Bellon & BlankAcademic feedback is more strongly and consistently related to achievement than any other teaching behavior.This relationship is consistent regardless of What Does the Research Say?In a major review of the research on assessment, Paul Black and Dylan Wiliam (1998) notedThe research reported here shows conclusively that formative assessment does improve learning. The gains in achievement appear to be quite considerable, and as noted earlier, amongst the largest ever reported for educational interventions. As an illustration of just how big these gain are, an effect size of 0.7, if it could be achieved on a nationwide scale, would be equivalent to raising the mathematics achievement score of an average country like England, New Zealand or the United States into the top five after thee Pacific rim countries of Singapore, Korea, Japan and Hong Kong.~ What Works in Schools, p. 38Power of Accurate FeedbackImmediate impact on resultsLower failuresBetter attendanceFewer suspensionsFailure here undermines EVERY OTHER EFFORT in curriculum, assessment, and teaching~ Douglas Reeves

Characteristics of FeedbackTimelyThe more delay that occurs in giving feedback, the less improvement there is in achievement. (Marzano)As often as possible, for all major assignmentsConstructive/CorrectiveWhat students are doing that is correctWhat students are doing that is not correctChoose areas of feedback based on those that relate to major learning goals and essential elements of the assignmentShould be encouraging and help students realize that effort on their part results in more learning (Marzano)Characteristics of FeedbackSpecific to a CriterionPrecise language on what to do to improveReference where a student stands in relation to a specific learning target/goalAlso specific to the learning at handBased on personal observationsFocused on the product/behavior not on the studentVerified Did the student understand the feedback? Opportunities are provided to modify assignments, products, etc. based on the feedbackWhat is my follow up plan to monitor and assist the student in these areas?)Essential Elements of FeedbackRecognition of the Desired GoalEvidence about Present Position (current work)Some Understanding of a Way to Close the Gap Between the Two

~ Black & WilliamFull article on wikispace76Recognition of the Desired Goal Includes:Clarity of the Learning Goal Clarity about Content AreaClarity of Curricular IndicatorsClarity of Mastery ObjectivesClearly communicating the desired learning goal to students through instruction.A Vision of Excellence

Methods to Ensure Student Understanding of Learning GoalsHave students define what successful achievement of the goals looks or sounds like. (Developing a criteria for success)Provide several samples, models, exemplars, etc. of products that achieve the learning goal in exemplary fashion.Lead students through an analysis of the criteria of successful achievement in terms of the samples provided. Could be through the use of rubrics or descriptions of the practice/product.Compare students product to the criteria for success Have students continue working on a task until they succeed.The Language of AssessmentAs a result of understanding the learning destination and appreciating what quality work and success look like, students:Begin to learn the language of assessment. This means students learn to talk about and reflect on their own work using the language of criteria and learning destinations.Gain the knowledge they need to make decisions that help close the gap between where they are in their learning and where they need to be.~ Anne Davies, p. 38Evidence About Present PositionWhat student work/assignments/projects look like what isI statements-students tell what they know I can explain the difference between fact and opinion.Current work samples

Ways to Close the Gap between Goals & Current StateProvide guidance on how to improve (strategies, tips, suggestions, reflective questioning, etc.)Provide student-friendly version of learning targets along with actual samples of student workuse exemplars!Provide help to improve performance Provide time to work on the improvement, apply the feedbackSharing FeedbackOral, interactive (one-on-one) feedback is best whenever possibleUse descriptive, not evaluative languageFocus on what went well and what can be improved in language students understandSeek consensus with the student(s) do you agree on the assessment of this product?Focus on the performance and/or behavior not the studentFocus on those behaviors that the student can do something about.Provide a demonstration if how to do something is an issue or if the student needs an example.Group/class feedback works when most students missed the same concept, providing an opportunity for reteaching.

Feedback TimingGood TimingBad TimingReturning a test or assignment the next dayGiving immediate oral responses to questions of factGiving immediate oral responses to student misconceptionsProviding flash cards (which give immediate right/wrong feedback) for studying facts

Returning a test or assignment two weeks after it is completedIgnoring errors or misconceptions (thereby implying acceptance)Going over a test or assignment when the unit is over and there is no opportunity to show improvement ~ Susan Brookhart

Amount of FeedbackFor students to get enough feedback so that they understand what to do but not so much that the work has been done for them (differs case by case)For students to get feedback on teachable moment points but not an overwhelming number

~ Susan Brookhart

Throughout instructional processbuild in benchmarks.Not only at summative.84Amounts of FeedbackGood AmountsToo MuchSelecting 2-3 main points about a paper for commentGiving feedback on important learning targetsCommenting on at least as many strengths as weaknessesReturning a students paper with every error in mechanics editedWriting comments on a paper that are more voluminous that the paper itselfWriting voluminous comments on poor-quality papers and almost nothing on good-quality papers~ Susan Brookhart

Strategies to Help Students Learn to Use FeedbackModel giving and using feedback yourself.Teach students self- and peer assessment skills to:Teach students where feedback comes from.Increase students interest in feedback because its theirs.Answer students own questions.Develop self-regulation skills, necessary for using any feedback.~ Susan Brookhart

Strategies to Help Students Learn to Use FeedbackBe clear about the learning target and the criteria for good work.Use assignments with obvious value and interest.Explain to the student why an assignment is given what the work is for.Make directions clear.Use clear rubrics.Have students develop their own rubrics or translate yours into kid-friendly language.Design lessons that incorporate using the rubrics as students work.~ Susan Brookhart

Strategies to Help Students Learn to Use FeedbackDesign lessons in which students use feedback on previous work to produce better work.Provide opportunities to redo assignments. (Comparing a rough draft to the rubric/criteria/exemplar.)Give new but similar assignments for the same learning targets.Give opportunities for students to make the connection between the feedback they received and the improvement in their work.~ Susan Brookhart

Attaining ExcellenceStudents must have routine access to the criteria and standards for the task they need to master; they must have feedback in their attempts to master those tasks; and they must have opportunities to use the feedback to revise work and resubmit it for evaluation against the standard. Excellence is attained by such cycles of model-practice-perform-feedback-perform.~ Grant WigginsFeedback LevelsFeedback may be directed at one of four levels:The task The best task-level feedback corrects flawed interpretations rather than a lack of knowledge and helps students focus on using strategies to achieve their learning goals. ~ Center on InstructionThe processing of the task ~ facilitating depth in learning (encouraging students use of strategies to check their work, recognize errors, and self-correct)Self-regulation ~ helping students internalize the practice of self-monitoring their learning and work.The student as an individual ~ least effective feedback

Video Examplehttps://www.youtube.com/watch?v=0DAeiBB6zT0

To reduce discrepancies between current understandings / performance and a desired goalThe Discrepancy Can Be Reduced ByTeachersProviding appropriate challenging and specific goalsORAssisting students to reach them through affective strategiesStudentsIncreased effort and employment of more effective strategiesORAbandoning, blurring or lowering the goalsEFFECTIVE FEEDBACK ANSWERS THREE QUESTIONSFeed UpWhere am I going?(The Goals)

Feed Back

How am I going?

Feed Forward

Where to next?

PURPOSEHattie & Timperleys Feedback ModelLets PracticeExemplarsPairs1 student, 1 teacherFocus on Development of DetailsSwitchFacilitating Data ConversationsYou dont need an advanced degree in statistics and a room full of computers to start asking data-based questions about your school, and using what you learn to guide reform. - Victoria Bernhardt96Effective data conversations takes place whenThere is mutual trust among group membersData is not used to place blame or point fingersThe data, not Your dataEveryone needs to understand what the data being presented represents and how it was derivedEveryone must acknowledge that they play a role

Adapted from The Data Dialogue, Laurie Olsen9697Focus, Organization, Structure Build data inquiry around a school-wide focus or established goal for improvementIdentify and focus on factors that the school can controlTeams need to meet regularly to engage in dialog, monitor outcomes, and prescribe timely interventionsUse a structured protocol for examining and analyzing data

Propose Norms for Looking at Data Why is This Important ?i.e. Describe only what you see.Just describe the data in front of youResist the urge to immediately work on solutionsSeek to understand differences

Ask questions when you dont understandSurface the lenses and experiences you bring to the dataSurface assumptions and use data to challenge and support themMaybe we can give them the question and ask them to come up with data norms, protocols theyve . SURFACE IS THE CORRECT WORD98Nancy Love (2002). Using Data/Getting Results. Norwood, MA: Christopher-Gordon.Eisenhower National Clearinghouse for Mathematics and Science Education - http://www.enc.org

1. Begin with a question2. Examine the data3. Raise questions about the data4. Interpret the dataPropose actions/interventions: Set goals99100Data Driven DialogPhase 1: Predictions - Surfacing of perspectives, beliefs, assumptions, predictions, possibilities, questions, and expectations.

Phase 2: Observations Examining and analyzing the data for patterns, trends, surprises, and new questions that jump out.

Phase 3: Inferences Generating hypotheses, inferring, explaining, and drawing conclusions. Defining new actions and interactions and the data that is needed to guide implementation (monitor) and build ownership for decisions.

Based on the work presented by Nancy Love, author of Using Data/Getting Results, (2002).101Phase 1: Predictions and AssumptionsPredictions: Informed by your knowledge of what work your school has been engaged in for closing the achievement gap, as well as your own critical findings, make predictions. Assumptions: What thinking, beliefs, or expectations drive your predictions?Hear and honor all assumptions and ideasActive Listening---not a conversationEach person shares their own ideas

101Facilitator Notes: There are three basic types of data: Demographics describe the students, the staff, or the community. In addition to characteristics like ethnicity or gender, we can describe students by their participation in a particular program; for example, in athletics or special education. Teaching staff can be described in terms of years of teaching experience or type of credential held.When were talking about student achievement, we generally think of outcomes as what or how well students did; for example, the grades they received in their courses or the scores they received on assessments. Outcomes may also refer to what or how well students do once they leave school, for example, their post-secondary school attendance or employment experiences.The third type of data is Process data. Generally, this refers to what actions or events have occurred that may contribute to an observed outcome. For example, completion of an intervention program may contribute to improved student achievement in the next course.102Purpose: Predictions & AssumptionsActivate prior knowledge and create readiness to examine and discuss the data Surface diverse assumptions, beliefs, and expectationsPhase 2: Observations Examining and analyzing the data for patterns, trends, surprises, and new questions that jump out.Phase 3: Inferences Generating hypotheses, inferring, explaining, and drawing conclusions. Defining new actions and interactions and the data that is needed to guide implementation (monitor) and build ownership for decisions.

Data ReviewThe Source106PracticeIllinois Interactive Report CardLook at District DataFollow the three stepsDebriefProcess of Transforming Data Into KnowledgeAdapted from Keeping Teachers in the Center : A Framework of Data Driven Decision MakingDaniel Ligh.t Education Development Center ,Inc. Center for Children and Technology USA,2004The literature identifies 6 broad steps a person goes through to transform data into knowledge. 108Using Data To Inform Your InstructionUsing Data To Inform InstructionThinking SheetInstructional FocusData SourceStudents AboveProficiencyStudents AtProficiencyStudents NotProficientStudents Far BelowProficiencyOn Resources page in WikiIs this a tier 1 issue? How do I differentiate?110CRITICAL StepWhat Proficient Work Looks LikeObstacles/Misconceptions/ChallengesWHY? How do you determine where the misconceptions are? Scaffolding111

112Rigor and RelevanceWhat is the framework? What do you know?Read the articleWith 1-2 colleagues, discuss connection to problem-based learningBe prepared to share your thinking113

114Tips For Effective Rubric DesignHow to:design a rubric that does its jobwrite precise criteria and descriptorsmake your rubric student-friendly Expert Input Experts agree:Rubrics are hard to design.Rubrics are time-consuming to design.A rubric is only as useful as it is good. Using a bad rubric is a waste of time--Michael Simkins in Designing Great RubricsExperts disagree: how to design a good rubric

Bottom line: Is it working for you and for your students?

116Checklists do not reflect developmentalindicates only presence or lack of a traitThe CookieTask: Make a chocolate chip cookie that I would want to eat.Criteria: Texture, Taste, Number of Chocolate Chips, RichnessRange of performance:Delicious(14-16 pts)Tasty(11-13 pts)Edible(8-10 pts)Not yet edible(0-7 pts)The RubricDelicious4Tasty3Edible 2Not yet edible1# chipsChips in every bite75% chips50% chipsLess than 50% chipstextureConsistentlychewyChewy middle, crispy edgesCrunchy

Like a dog biscuitcolorEven golden brownBrown with pale centerAll brownOr all paleBurnedrichnessButtery, high fatMedium fatLow-fat flavorNonfat flavorAssess The CookieOverall scoreDeliciousTastyEdibleNot yet edibleBy criteriaNumber of chipsTextureTasteRichness

Oops, What Went Wrong?Did the product match expectations?Effective rubrics dont exist in a vacuum.The good newsHolistic Or AnalyticWhich To Use?HOLISTICviews product or performance as a whole; describes characteristics of different levels of performance. Criteria are summarized for each score level. (level=degree of successe.g., 4,3,2,1 or Tasty)(criteria= what counts, facets of performancee.g., research or number of chips or presentation)

Holistic Or Analytic?HOLISTICpros and cons+Takes less time to create. Well +Effectively determines a not fully developed performance as a whole+Efficient for large group scoring; less time to assess- Not diagnostic- Student may exhibit traits at two or more levels at the same time.122An overall judgment. Generally speaking, not recommended for classroom use because of diagnostic limitations. If our goal is to give students feedback on performance, the more specific, the better.Holistic ExampleCookieDelicious level (4)Chips in every biteConsistently chewyEven golden brownButtery, high fatHolistic Or Analytic?Analytic=Separate facets of performance are defined, independently valued, and scored.Example: Musicskill=string improvisation developmentFacets scored separately: melody; harmonics; rhythm; bowing & backup; confidenceHolistic Or Analytic?Analyticpros and cons+Sharper focus on target+Specific feedback (matrix)+Instructional emphasis-Time consuming to articulate components and to find language clear enough to define performance levels effectively125Analytic and holistic can be combinedsum of analytical scores =integration or holistic score.Or add scores and take average for holistic representationThe DebateIs the whole the sum of its parts?Wiggle room or valid criterionOverall DevelopmentOverall Impression Overall impactWeightingNumber range 126See Slide #30Tip #1Dont use generic or canned rubrics without careful consideration of their quality and appropriateness for your project.These are your students, not someone elses.Your students have received your instruction.127This is the other extreme of too many task-specific rubrics.A project rubric should not be used to assess everything from a digital montage to a PowerPoint presentation on market economics. Yet, there are excellent resources available for you to adapt.Evaluate the resources available on the Webdont just use one because it is free and dont think because its in a textbook that it is good!Find the middle grounda template that you can adjust and tweak according to the specifications of a given task.Tip #2Avoid dysfunctional detail.in most instances, lengthy rubrics probably can be reduced to succinctmore useful versions for classroom instruction. Such abbreviated rubrics can still capture the key evaluative criteria needed to judge students responses. Lengthy rubrics, in contrast, will gather dust (Benjamin 23).--Includes wordiness, jargon, negativity128This includes educational jargon! Avoid sole adjective descriptors such as inadequate and avoid adjectives of averagenessbelow, above. The lowest score should describe what a novice, not bad performance looks like.Wordinessoften happens when groups deviseincludes a little something for everyoneTip #3Limit the number of criteriaDont combine independent criteria. very clear and very organized (may be clear but not organized or vice versa).129Whats important? Tip #4Use key, teachable criteria (What counts)Dont vaguely define levels of quality. Concrete versus abstractpoorly organized (Organization: sharply focused thesis, topic sentences clearly connected to thesis, logical ordering of paragraphs, conclusion ends with clincher)inventive creative imaginative UNLESS

Key Question to ask yourself: What does it look like?130Not so much an issue of diction as describing the concrete behaviors and evidence of critical thinkingCreativity= uses ideas from others (Developing), modifies ideas implemented by others (Basic), composition is self-generated (Proficient), composition is unique and imaginative(Advanced)--MyraTip #5 Use measurable criteria.--Specify what quality or absence looks likevs. comparatives (not as thorough as) or value language (excellent content)---Highlight the impact of the performance --Was the paper persuasive or problem solved? (Note importance of PURPOSE)--What are the traits of effective persuasion?131Actual traits that constitute good or poor persuasion, problem-solving.Be careful not to bury criteriahere is where some people find that their rubrics do not match their expectationsbe sure that the descriptor is not a criterion and vice versaTip #6Aim for an even number of levelsCreate continuum between least and mostDefine poles and work inwardList skills and traits consistently across levels

1324 or 6 recommendedEven recommended for delineating proficiency---Unless you want an equivocal position.Even number requires a decision between almost there and barebones.No implied levels.Tip #7Include students in creating or adapting rubricsConsider using I in the descriptorsI followed preciselyconsistentlyinconsistentlyMLA documentation format.I did not follow MLA documentation format.133You may also want students to self-assess and even use highlighters to document their claims.Tip #8Do they understand the criteria and descriptors? How do you know?When do you give the rubric to your students?134In their hands at beginning.Use as revision tool.Give a quiz (Veronika!)Add self-assess column and defense piece or use as revision activityhighlight where it isTip #9Provide models of the different performance levels.135Have students list criteria for What Counts. Prompt them to think about any criteria they have missed and add them yourself. After class, combine criteriacreate categories, making sure not to bury criteria that you want to emphasize.Dont Forget the Check-in StageUse your rubric as a formative assessment to give students feedback about how they are doing.Isolate a particularly challenging aspectHave student isolate an area of difficultyCenter revision instruction around rubricSteps in Developing a RubricDecide on the criteria for the product or performance to be assessed. Write a definition or make a list of concrete descriptorsidentifiable-- for each criterion.Develop a continuum for describing the range of performance for each criterion.Keep track of strengths and weaknesses of rubric as you use it to assess student work.Revise accordingly.Step back; ask yourself, What didnt I make clear instructionally? The weakness may not be the rubric.137See cookieSteps in Modifying a Canned RubricFind a rubric that most closely matches your performance task.Evaluate and adjust to reflect your instruction, language, expectations, content, studentsCriteriaDescriptorsPerformance levels

138Eliminate criteria that do not reflect your assignment. Its hard workExpect to reviseand reviseOne problem is that the rubric must cover all potential performances; each should fit somewhere on the rubric.

There are no final versions, only drafts and deadlines.

When youve got a good one, SHARE IT!The Mini-RubricThese are the quick ones.Fewer criteria and shorter descriptions of qualityYes/no checklistsDescribe proficient level of quality and leave other boxes for commentary during grading.Use for small products or processes:PosterOutlineJournal entryClass activityMini-rubric ExampleVocabulary Poster Purpose: to informContent criterion (50%) 4321____written explanation of denotationaccuracy/thoroughness____examples in actionaccuracy/variety____visual symbol or cartoon conveys word meaningaccuracy/clarity____wordplay---weighs synonyms for subtleties of meaning--accuracy/thoroughnessPresentation criterion (50%)4,3,2,1--neat 4,3,2,1--clear organizational pattern4,3,2,1--no error in Conventions4,3,2,1--uses visual space to catch and hold attention Score= Content__+Presentation___divided by 2=______GRADEComments:141Content criteria repeats phrasing on assignment sheet.Comments allows teacher to justify any score not a 4. CautionDont let the rubric stand aloneALWAYS, ALWAYS provide specific feedback on your rubric and/or on the student product itself.

Sentence StemsTo establish 4 levels of performance, try sentence stems.Example:Yes, I used surface texture and deep carvings effectively to create individualizing detail.Yes, I used surface texture and deep carvings, but I needed to include more for individualizing detail.No, I did not use surface texture, but I did use deep carvings or vice, versato create some individualizing detail.No, I did not use surface texture or deep carvings.Rubric Criterion Across The CurriculumContent (substance, support, proof, details)RelevantSpecificThoroughSynthesizedBalancedConvincingAccurate144The content is the meat of the projectwhat are the requirements for successful content for a lab report in science, a pen and ink portrait, a character sketch, a PowerPoint presentation, a math problem?Brainstorm other adjectives that specifically describe what you expect of ANY content. Now use these as a bank from which to draw and insert them in your template.ReferencesBellon, Jerry, Bellon, Elner, & Blank, Mary Ann. Teaching from a Research Knowledge Base: A Development and Renewal Process, New York: Macmillan Publishing Company, 1992.Black & William, Inside the Black Box: Raising Standards through Classroom Assessment Phi Delta Kappan, October 1998.Brookhart, Susan M. How to Give Effective Feedback to Your Students. ASCD, 2008.Davies, Anne. Involving Students in the Classroom Assessment Process Ahead of the Curve: The Power of Assessment to Transform Teaching and Learning. Douglas Reeves, Editor. Solution Tree, 2007.Jackson, Robyn R. Never Work Harder Than Your Students & Other Principles of Great Teaching. ASCD, 2009.Marzano(1), Robert. Classroom Instruction that Works. ASCD, 2001.Marzano(2), Robert. Designing a Comprehensive Approach to Classroom Assessment. Ahead of the Curve: The Power of Assessment to Transform Teaching and Learning. Douglas Reeves, Editor. Solution Tree, 2007.References, page 2Marzano(3), Robert. What Works in Schools: Translating Research into Action. ASCD, 2003.Miser, W. Fred. Giving Effective FeedbackProviding Students with Effective Feedback Academic Leadership LIVE: The Online Journal; Volume 4, Issue 4, February 12, 2007.Reeves, Douglas. Challenges and Choices: The Role of Educational Leaders in Effective Assessment. Ahead of the Curve: The Power of Assessment to Transform Teaching and Learning. Douglas Reeves, Editor. Solution Tree, 2007.Stiggins, Rick. Assessment for Learning: An Essential Foundation of Productive Instruction. Ahead of the Curve: The Power of Assessment to Transform Teaching and Learning. Douglas Reeves, Editor. Solution Tree, 2007.Synopsis of The Power of Feedback by Center on Instruction, 2008. [Hattie & Timperleys research]Wiggins, Grant. Educative Assessment: Designing Assessments to Inform and Improve Student Performance. San Francisco: Jossey-Bass Inc., 1998.