Ruth CurreyARE 6905: Research Trends in Art Education

Creating a Standard Arts Assessment:Written Evaluation vs. Project Based

An Informal Case Study

Purpose

• Conduct an informal study that evaluates students on technological processes in both a written format as well as a skills test format• Evaluate and consider the data collected from each form of assessment• Make preliminary conclusions concerning modes of assessment• Create a potential options for Technology-Based Art Classrooms based on Industry Standard Software

• With Animation I classrooms packed to the brim, written exams may be the only feasible option due to time constraints• Although National Exams such as the NAEP are available for a portion of the arts, these exams do not cover any technology based skills or processes

• Relevant competency exams are needed for technology based art classrooms• No data exists currently that accurately demonstrated technology based learning

Determine and evaluate the possibility of creating a National Assessment for computer-based visual arts classrooms

Goals and Processes

Personal Interest & Current Status

Questions to Consider• Will creating a written standard assessment truly demonstrate the artistic knowledge of the students?• Can multiple choice tests identify a student’s artistic potential?

• Students are currently allowed to enroll in an AP Drawing class based on PSAT, FCAT, and Stanine scores in Reading and Math

• Can a project based assessment be created?• What would it look like?• Would separate standards have to be created by artistic content area?• Would standards be set based on grade level?

• Does the assessment tool measure what you need it to measure?• Realistically, can a set of National Standards be designed based on the art making process?

Possible Limitations to ConsiderAssumption – Students will have the same sequence of classes to ensure receipt of all necessary information

• Depending on the school and guidance department, students are given the opportunity to skip classes in a sequence due to scheduling

Assumption – Students will have access to the necessary supplies needed to meet the National Standards for project based assessment art supplies and media availability change from school to school

• Funding is not available to all schools in order to provide the exact same material at every single school

Student AComputer Animation I – Written Skill Exams

Preliminary Case Study Examples

Fill in the Blank Process Quiz Score: 11/12 (92%)

Process Sequential Order Quiz Score: 12/12 (100%)

Student BComputer Animation I – Written Skill Exams

Preliminary Case Study Examples

Fill in the Blank Process Quiz Score: 2/12 (17%)

Process Sequential Order Quiz Score: 3/12 (25%)

Ball Bounce using Motion GuidesScore: 47/50 (94%)

Ball Bounce using Motion GuidesScore: 49/50 (98%)

Oral Skills Demonstration QuizScore: 23/25 (92%)

Oral Skills Demonstration QuizScore: 25/25 (100%)

Project Based Assessment

Animation I Student B

Animation I Student A

Review of Literature

The Teacher as Stakeholder in Student Art Assessment and Art Program Evaluation

By: Charles DornArt Education, 55(4), pp. 40-45

“What cannot be tested cannot be taught, the arts in the near future may face being left out of the curriculum.” (Dorn, 2002, p. 40)

Article Discusses• National Education Goals and National Art Standards• NAEP Assessment• Authentic Assessment

• What teachers teach and assessment based on information taught• What needs to be taught

• Models for assessing arts performances

Recommendations• Teachers develop their own authentic assessment instruments• Teachers develop school and district assessment plans using a peer review process• Teachers develop ways to document student progress and establish sensible and appropriate record keeping systems that will meet the agreed upon goals of the district and state

“In the end, we must realize that US school children are not equal in their aesthetic abilities and US schools are also not equal in the aesthetic opportunities they provide. But if we can at least entertainthe possibility that either one or both of these conditions are reversible, our best hope lies in deciding

what it is that kids need to know and be able to do and make that the primary focus in reforming schools and schooling.” (Dorn, 2002, pp. 44-45)

Performance Assessment in the ArtsBy: Robin E. Clark

Kappa Delta Pi Record, Fall 2002

“Despite the difficulty of assessing a non quantifiable field like art, teachers in the artsand other areas should creatively embrace different methods of performance assessment

to evaluate student work more accurately.” (Clark, 2002, p. 32)

What is Performance Assessment?• In a standards-based curriculum, student performance generally is expected to be the most important indicator that learning is taking place

• Teachers are required to provide evidence of learning based on multiple assessments• According to Clark, art should be leading the way in performance based assessment

Process• The entire range of activities required to produce a work of visual or performing art from

concept to creationProduct

• The aesthetic quality of the student’s finished result of work of art

Art teachers typically recognize the need for both forms of assessment• A finished product or performance does not necessarily reveal skills, knowledge, creative thought, and personal meaning of the process that created it

“A balanced approach to grading considers both student engagement in the process ofcreation and the quality of the resulting product or performance” (Clark 2002, p. 30).

Performance Assessment in the ArtsBy: Robin E. Clark

Kappa Delta Pi Record, Fall 2002

Assessment Methods• Portfolios: self-assessment and collections of student work over a period of time• Critique: causes students to reflect on his or her work and hypothesize a means for improvement• Demonstrations: makes student knowledge level visible based on teaching their peers• Working Exhibitions: opportunities to actively discuss their all works on display• Annotated Sketchbooks: documents essential points along their creative process• Other forms of assessments can be modified to accommodate performance based assessment

• Written research papers, presentations, class discussions, tests quizzes and exams

Time-Efficient Assessment• Teachers are open to suggestions and tips regarding time management particularly with assessment• Well-constructed rubrics

• Provide a way for teachers to evaluate performance easily without compromising divergent qualities of the individual creative processes as well as the final product performances• Needs to be a balance between subjective and objective elements• Essential to have a clear statement regarding what students should know by the end of the project• Students should receive copies of the rubric prior to the start of the lesson

“Students’ performance can offer solid proof that they know and are able to do whatyou are trying to teach them and what the standards require” (Clark, 2002, p. 32).

Measuring Student Learning in Art EducationBy: Donald D. Gruber

Art Education, September 2008

Difference between Evaluation and AssessmentEvaluation

• Encompasses the global aspects of the curriculum and measures program efficacyAssessment

• More tightly focused measurements at the level of the individual student and their interactions within the art program

Developing Assessment CriteriaProcess Criteria

• Considering the development of learning and growth within a program• Teachers report attendance, effort, class participation, homework, etc.• Reliable measures of student growth and establish a baseline to gauge growth

Product Criteria• Significance of where students are is more important than how far they have come• Overall assessments, final exams scores, and final products• Provide a valid estimate of student learning as an indication of current ability and knowledge

“No single aspect of assessment can provide a representative and accurate measure ofstudent learning in art” (Gruber, 2008, p. 42).

Measuring Student Learning in Art EducationBy: Donald D. Gruber

Art Education, September 2008

Methods of AssessmentWritten Tests

• Most widely used source of assessment information• Researchers warn about relying on test validity due to their inability to measure aesthetic response

Observations• Consistent surveillance during all phases of the lesson including

• Performance• Attitude (level of engagement / sharing discoveries / attention to task)• Work habits and behavior• Other observational strategies

• Checklists: lists of attributes of projects that can be compared• Rubrics: detailed guides for scoring student products, performance, and portfolios

Portfolios• Formative Portfolios: contain all aspects of a particular lesson from initial conceptualizations through research and revisions to final products• Summative Portfolios: contain collections of finished products that give indications of abilities at a given point in the instruction (usually at the end)

Assessment Strategies• Must consider objectives• Unproductive to assess without regard to stated objectives• Measure how well the objectives of each lesson are being met• Based assessment on a set of established standards

Research Results & Data

Data Charts• Students we separated based on educational status

• Gifted=G• Mainstream=M• ESE=E

• Numbers reflect points earned for each assessment method

• A total of 95 students are included in this study with the breakdown being:

• Gifted = 10 students (10%)• Mainstream = 70 students (74%)• ESE = 15 students (16%)

*Final analysis will include charts and graphs for each subdivision of students

Overview of Results

Overview for Gifted Students

Gifted BreakdownSequential Order Average Score: 12/12 (100%)

Fill in the Blank Average Score: 11/12 (91%)

Overall Written Assessment Average: 96%

Ball Project in Flash Average Score: 48/50 (96%)Oral Skills Test Average Score: 24/25 (96%)

Overall Project Assessment Average: 96%

0% Differential

• No student scored below an 84% on any written assessment

• All scores for project based assessment ranged between 92% to 100%

• Gifted students demonstrated a level consistency across assessment modes

Overview for ESE Students

ESE BreakdownSequential Order Average Score: 5/12 (41%)Fill in the Blank Average Score: 4/12 (33%)

Overall Written Assessment Average: 38%

Ball Project in Flash Average Score: 40/50 (80%)Oral Skills Test Average Score: 21/25 (83%)

Overall Project Assessment Average: 82%

44% Differential

• Scores ranged from 8% to 66% on written assessments

• Student scores increase significantly for project based assessment ranging from 72% to 100%

• Students successfully demonstrated learning gains primarily in animation processes through project based assessment

Mainstream BreakdownSequential Order Average Score: 8/12 (66%)Fill in the Blank Average Score: 8.6/12 (72%)

Overall Written Assessment Average: 69%

Ball Project in Flash Average Score: 41.5/50 (83%)Oral Skills Test Average Score: 22.25/25 (89%)

Overall Project Assessment Average: 85%

16% Differential• Scores for written assessment modes exhibited fairly large learning gaps between the two poles ranging from 16% to 100%

• Project based scores had a significantly lower learning gap ranging from 75% to 100%

• Scores demonstrated a statistically significant increase from written assessment to project based assessment modes

• Data exhibits a consistent incline as the mode of assessment shifts from written to project

Mainstream Student Data

Preliminary conclusions• Preliminary research indicated that the most effective means of assessment was to maintain a balanced approach

• Based on the data collected, I would be inclined to agree • Gifted students seemed to excel across the board (96% for both written & project based)• Mainstream students maintained a steady incline as the assessments moved towards project based (69% written / 85% project based) • ESE students responded significantly higher when assessed through project based (38% written / 82% project based)• Data also indicated multiple occurrences where ESE students out performed mainstream students on project based tasks• Written assessment seems to yield lower scores for mainstream and ESE students• Preliminary data indicates failing scores for written assessments, however the point values are lower in comparison to the project based values

• For this reason, the data was informally compared to the final grade received• When comparing results to the final grades, most students from each subgroup were able to successfully complete the course with a “C” or higher• However…

• Does adding the written assessments automatically decrease their final grade potential?• Should that matter?

• Consideration also needs to be give to other potential factors such as lack of reading or writing skills

Possible SolutionsWritten Assessment

• Lends itself well to overcrowded introductory level courses• Approximately two-thirds of the subgroups struggled when faced with written assessments

Project Based Assessment• Does not lend itself to individualized work due to overcrowding and lack of available technology• Success rates are higher for all subgroups

Based on the data collected, the best option would be to create a balanced curriculum that requires multiple modes of assessment in order to provide the most opportunities for success for the students

A Balanced Combination• Students will be assessed in a manner that reflects the principles of differentiated instruction• Students will be exposed to written as well as project based assignments in order to fully demonstrate where their strengths and weaknesses lie• Students will have the opportunity to work with partners to compensate for overcrowding, However, individual oral exams as well as project accountability must be built into the rubric and the curriculum

Possible Limitations• Provides the best opportunity for students to succeed and demonstrate their learning gains and individual skills if the value attached to the assessment is balanced over the entire curriculum• Does not lend itself to overcrowded classroom due to the time needed to log in each individual student for individual assessment. • Exam periods prove difficult

Personal Learning Gains & Reflection• Statistics and I have decided to maintain our love hate relationship

• Teaching and assessing students effectively are trial and error processes

• When working in a relatively new aspect of a content area there are no absolutes

• More research needs to be conducted in the field of technology based art education before any definitive recommendations can be made

• Just from my own personal research, I have learned such a great deal about how my students function in my classroom and how they respond to the assessments and activities I present• If more teachers took the time to consider how their assessments are actually evaluating their students, more effective assessments may be possible

• Even with all of the data that was collected, I am still undecided on how to proceed with the assessments for the Animation I course

• The written assessments demonstrate a task oriented understanding based on the representation and interpretation of language• The project assessments demonstrate a literal grasp of the information and exhibit direct learning gains based on a tangible product

References

Clark, R. (2002). Performance Assessment in the Arts. Kappa Delta Pi Record, 39(1), pp. 29-32.

Dorn, C. (2003). Models for Assessing Art Performance (MAAP): A K-12 Project. Studiesin Art Education, 44(4), pp. 350-370.

Dorn, C. (2002). The Teacher as Stakeholder in Student Art Assessment and Art ProgramEvaluation. Art Education, 55(4), pp. 40-45.

Gruber, D. (2008). Measuring Student Learning in Art Education. Art Education, 61(5),pp. 40-45.