National Assessment of Educational Progress

National Assessment of Educational ProgressAshley SingerUniversity of Central FloridaARE 6905April 16, 2013PurposeTeachers demand for creating applications from results

Add more value and clarity to existing NAEP testing by addressing the lack of teacher questionnaire in the visual arts assessment that is present in nearly all other NAEP subjects.

Why Its ImportantExists as a result of lack of lucidity that accompanies NAEP visual arts assessment dataNothing offers possible explanations for resultsTeachers are left to interpret the numbers without guidanceAdd some context to the dataBe able to see what is current or trending in the classroomWhat could be lacking in their curriculumWhat is proving successful in their practiceUniversities and schools could use to see what educator training programs have been successful in equipping teachers for their fieldsWhat teachers may be lacking in their classrooms and how to give it to themResearch QuestionsHow can NAEP clarify the results of the visual arts assessment by adding a teacher questionnaire with common practice and teacher background similar to existing teacher questionnaires of other subjects?How can the demographic and background information be applied to understanding knowledge and experience as well as hiring trends?Do the findings suggest certain training and specialties lead to classroom achievement?What areas of art education are being concentrated on and what areas are being neglected?How could we take the results to further develop a NAEP curriculum and understand best practices?What Ive LearnedCreating a well-done questionnaire is difficult

Basic structure and style is simpleScientific approach is tedious and thought-provokingVarious steps to developing a questionnaireNot just writing whatever questions you want answers to and expecting reliable results from themOverwhelming to develop questions that would yield best applications for educators while answering my questions

What Ive LearnedCritical to review who is writing tests, papers, and surveys

What do you want to know?Likely based on what they know, their experience, or what they want to knowMay not be true representation of the informationBoards and panels are importantCan also be influenced by central philosophy or philanthropistReduce bias based on multiple experiences and perspectivesWhat Ive LearnedObjectivity and Adjustments

Analyzing previous and test-specific dataResearchs ultimate progressMinor and major changes made to improve testsDiscussion of limitations shows what could be betterChanges are not personal just progress

Adjustments to create another testDesigned around teachers training and preparednessAreas of focus, certification process, work history, etcWhat is making teachers ready for the classroomWhat Ive LearnedNAEP

Obvious need for more clarityIf complaints are lack of application, they have to find ways to make it relevantEducators must be a part of the processEither in test development, research, or advocacyNAEP could find more ways to reach out to teachers

Whether you are a researcher or a teacher, you cannot continue doing things the same way and expect different or better resultsReview of LiteratureFinally, the arts assessment reminds us once again that arts education is for all students, not just for the talented. No one has suggested that math or science should be taught only to students with talent in those disciplines. The arts, similarly, provide long-term benefits that are important for every student. Experience has demonstrated to arts educators that all children can learn basic arts skills and knowledge, provided that they begin instruction early enough. (Lehman, 1999)Most NAEP assessments have teacher questionnaire (NAEP, 2012)The common education practitioner often has difficulty gleaning consequence and meaning from the scores must ask what we know about these teachers (Eisner, 1999)Test performance, like paintings, needs to be read, not only seen. Information needed to give test scores a deep reading is very limited (Eisner, 1999Recent study - revealed that untrained people do not simply walk into classrooms and become successful prepared and certified teachers are more successful than the untrained ones (Hatfield, 2007)Test results only leave readers with value without clarity (Diket & Brewer, 2011)

Review of LiteratureWhile teachers completion of the questionnaire is voluntary, NAEP encourages their participation since their responses make the NAEP assessment more accurate and complete (Teacher Questionnaire, 2011)Covers: teaching experience, certifications, degrees, major and minor fields of study, coursework in education, course work in specific subject areas, the amount of in-service training, the extent of control over instructional issues, and the availability of resources for the classroom (Teacher Questionnaire, 2011)pre- and in-service training, the ability level of the students in the class, the length of homework assignments, use of particular resources, and how students are assigned to particular classes (Teacher Questionnaire, 2011)MethodologyPopulation

Similar to NAEP sample selectionNeed to be directly related to the test resultsTeacher questionnaires must match up with NAEP participants classrooms, schools, districts, etc.NAEP participation is entirely voluntaryTeachers survey would also be voluntaryNo way to accurately forecast who will be undergoing the research and how they represent the actual population of the United States visual arts classroomNAEP visual arts exam only covers eighth grade studentsOnly be administered to corresponding eighth grade teachers of the visual arts programMethodologyProcedures

Similarly follow NAEP testing to adhere to procedural protocolTeachers will be given a general background questionnaire and a subject-area specific questionnaireConsists of a series of select-response questionsTeachers will mark their answers in their booklet or record answers online as accurately as possibleOnce the survey is finished the online answers will be saved or the booklet can be given to the NAEP school coordinatorMethodology Descriptive/QuantitativeUsed to look for trends and graph opinions, facts and demographic data Used to make recommendations for classroom applicationCould prove to be effective information for correlation testsInstrumentationDevelopment based on:Other teacher questionnairesReading and writing teacher questionnaire. (2011). National Assessment for Educational Progress.Writing teacher questionnaire. (2010). National Assessment for Educational Progress. Teacher data in NAEP Data ExplorerNAEP 1997 national theatre results. (2002). National Assessment of Educational Progress.Considered questionnaire development resourcesGillham, B. (2000). Developing a questionnaire. New York, NY: Continuum.NAEP teacher questionnaire overviewsTeacher questionnaire. (2011). National Assessment of Educational Progress.

Data AnalysisBest done by professional statisticianPer advice for collaboration within quantitative research (Brewer, 2013)

DescriptiveAnalysis will show trends, demographic data, etc.

CorrelationCorrelation testing to note potential relationships between student results and teacher questionnairesResults and ImplicationsSpeculative in natureDescriptive and correlation researchWhatever results are reported, they will be limited to:Making recommendations, not judgmentsSeeing relationships, not causesAdd transparency to resultsShow that specific subjects are highly promoted or often neglected in classroomsSee what practices (i.e. writing, production, assessment, presentation, critical analysis) are being done in classrooms and which are notEducational background and current practice and training in the field of the teachers

With that information, we can compare the educators with the ideal practices and see how their classrooms performed on NAEP testing and determine possible explanations for success or failure by looking for patterns.Results and ImplicationsMore background information = results will likely be more generalizable and reliable (Brewer, 2013)Results from NAEP follow principle with teacher backgroundGeneralizability usefulness

Step towards examining school structure and culture that Eisner calls for in order to make improvements in student achievement (1999)

Could likely affect the qualifications for hiring and successful preparation programs if Hatfield is correct

Relationships between student success and certain visual arts subjects and practices individual classroom structures may progress and a possibility for curriculum improvementsRationale for teacher adjustments

The call for direct applications may finally be heard and answered.LimitationsNot having a board or a panel creating the surveySolely developed by meBased on what I want to know no hidden agendasNo other perspectives or experiencesBased on my experience or lack thereofCause assumptions because of what I think I know about the issues (Gillham, 2000)Quick developmentNo pre-pilot or pilot stageAffects wording and understanding (Gillham, 2000)Assumed done from other questionnairesSample population variableInstrumentation

Instrumentation

Instrumentation

Instrumentation

Instrumentation

Instrumentation

Instrumentation

ReferencesBrewer, T. (Forthcoming, 2013). A primer for todays quantitative research in art education. In K. Miraglia & C. Similian (Eds), Inquiry in Action: Research Methodologies in Art Education. Reston, VA: National Art Education Association.Diket, R. M., & Brewer, T. M. (2011). NAEP and policy: Chasing the tail of the assessment tiger. Arts Education Policy Review, 112(1), 35-47. Retrieved from http://www.informaworld.com.ezproxy.lib.ucf.edu/openurl?genre=article&id=doi:10.1080/10632913.2011.518126Eisner, E. W. (1999). The national assessment in the visual arts. Arts Education Policy Review, 100(6), 16-20. Retrieved from http://ezproxy.net.ucf.edu/login?url=http://search.ebscohost.com.ezproxy.net.ucf.edu/login.aspx?direct=true&db=eric&AN=EJ624037&site=ehost-liveGillham, B. (2000). Developing a questionnaire. New York, NY: Continuum.Hatfield, T. A. (2007). The unevenness of arts education policies. Arts Education Policy Review, 108(5), 9-13. Retrieved from http://ezproxy.net.ucf.edu/login?url=http://search.ebscohost.com.ezproxy.net.ucf.edu/login.aspx?direct=true&db=eric&AN=EJ771257&site=ehost-liveLehman, P. R. (1999). Introduction to the symposium on the "NAEP 1997 arts report card.". Arts Education Policy Review, 100(6), 12-15. Retrieved from http://ezproxy.net.ucf.edu/login?url=http://search.ebscohost.com.ezproxy.net.ucf.edu/login.aspx?direct=true&db=eric&AN=EJ624036&site=ehost-liveMathematics teacher questionnaire. (2013). National Assessment for Educational Progress. Retrieved from http://nces.ed.gov/nationsreportcard/bgquest.aspNAEP 1997 national theatre results. (2002). National Assessment of Educational Progress. Retrieved from http://nces.ed.gov/nationsreportcard/tables/art1997/sdt02.aspNational Assessment for Educational Progress (NAEP). (2012). Questionnaires for Students, Teachers, and Schools. Retrieved from http://nces.ed.gov/nationsreportcard/bgquest.aspReading and writing teacher questionnaire. (2011). National Assessment for Educational Progress. Retrieved from http://nces.ed.gov/nationsreportcard/bgquest.aspTeacher questionnaire. (2011). National Assessment of Educational Progress. Retrieved from http://nces.ed.gov/nationsreportcard/tdw/instruments/noncog_teach.aspWriting teacher questionnaire. (2010). National Assessment for Educational Progress. Retrieved from http://nces.ed.gov/nationsreportcard/bgquest.asp