Beyond Penguins Evaluation Report 2008-2009

Miami University 408 McGuffey Hall Oxford, OH 45056

Phone: 513-529-1686 Fax: 513-529-2110 Website: http://ohioeval.muohio.edu

Evaluation of Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom

Year 2 Report 2008-2009

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom ii

Please cite as follows: Woodruff, S. B., Morio, K. L., & Li, Y., (2009). Evaluation of beyond penguins and polar bears: Integrating literacy and IPY in the K-5 classroom. Oxford, OH: Miami University, Ohio’s Evaluation & Assessment Center for Mathematics and Science Education. Distributed by Ohio’s Evaluation & Assessment Center for Mathematics and Science Education 408 McGuffey Hall Miami University Oxford, Ohio 45056

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom iii

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom

Ohio’s Evaluation & Assessment Center for Mathematics and Science Education

Miami University Oxford, Ohio

Sarah B. Woodruff Principal Investigator Yue Li Senior Researcher and Statistician Kristen Morio Research Associate Jennifer Sutton Research Associate

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom iv

Table of Contents Tables of Contents................................................................................................................ iv

List of Tables........................................................................................................................ vi

List of Figures ......................................................................................................................vii

Introduction ..........................................................................................................................1

Background.....................................................................................................................1

Evaluation.............................................................................................................................2

Evaluation of Website ......................................................................................................2

Expert Reviews..........................................................................................................2

Article Reviews ....................................................................................................2 Website Reviews..................................................................................................2 Website Rating Results.........................................................................................3 Overall Website Evaluation Results........................................................................3

Strengths ......................................................................................................3 Minor Concerns..............................................................................................4 Suggestions...................................................................................................4 Summary ......................................................................................................4

Webmetrics .........................................................................................................5 Traffice Sources.............................................................................................6 Search Keywords ...........................................................................................6 Time on Website............................................................................................7

User Feedback.....................................................................................................7 Online Questionnaires ..............................................................................7 Article-Specific Questionnaire....................................................................7 General Website Questionnaire .................................................................8

General Website Questionnaire Respondent Demographics .....................................8 Respondents’ Reasons for Visiting Website ............................................................9 Respondents’ Comfort with Technology ............................................................... 10 Homepage Review by Focus Group ..................................................................... 11

Evaluation of Impact...................................................................................................... 12

Instruments .................................................................................................................. 12

Teacher Questionnaire ....................................................................................... 12 Student Questionnaire........................................................................................ 13

Demographics and Data Analysis: Teacher Questionnaire ................................................. 14

Teacher Demographics....................................................................................... 14 Findings ............................................................................................................ 15

Teaching and Learning................................................................................. 15 Knowledge of the Polar Regions.................................................................... 18 Integration of BPPB Materials ....................................................................... 19 Evidence of Impact on Students.................................................................... 20 Classroom Field Testing................................................................................ 21

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom v

Demographics and Data Analysis: Student Questionnaire.................................................. 23

Student Demographics ....................................................................................... 23 Findings ............................................................................................................ 23

Dissemination................................................................................................................ 26

Presentations at Local and National Conferences and Meetings ............................. 26 Distributed Promotional Materials at Conferences and Meetings ............................ 26 National Science Teachers Association Conference Presentation Evaluations .......... 27 National Science Teachers Association Web Seminar Evaluations........................... 28

Polar Geography: May 27, 2008 .............................................................. 28 Physical Science from the Poles: October 29, 2008................................... 28

Energy and the Polar Envrironment: Novemeber 13, 2008 ........................ 29 National Science Digital Library Brown Bag Series: Web Seminar Evaluations ......... 29

States of Matter: January 13, 2008 ......................................................... 29 Rocks and Minerals: January 22, 2009..................................................... 29 Life in the Polar Extremes: April 7, 2009 .................................................. 30

Polar Adventure Weekend at COSI ...................................................................... 30

Continuing Activities ............................................................................................................ 31

Summary ............................................................................................................................ 32

Website ............................................................................................................ 32 Impact.............................................................................................................. 32 Dissemination.................................................................................................... 32

Recommendations ............................................................................................................... 34

Website ............................................................................................................ 34 Impact.............................................................................................................. 34 Dissemination.................................................................................................... 34

References.......................................................................................................................... 35

Appendices ......................................................................................................................... 36

Appendix A. Expert Review Summaries ........................................................................... 37 Appendix B. Website Evaluation Form ............................................................................. 38 Appendix C. Full Evaluation Document Summary ............................................................. 39 Appendix D. Teacher Pre-Questionnaire .......................................................................... 49 Appendix E. Teacher Post-Questionnaire ......................................................................... 54 Appendix F. Student Questionnaire ................................................................................. 60 Appendix G. Teacher Questionnaire Frequencies (pre-questionnaire)................................. 62 Appendix H. Teacher Questionnaire Frequencies (post-questionnaire) ............................... 72 Appendix I. OCEPT - Teaching Observation Protocol (OTOP)............................................. 79 Appendix J. NSTA Web Seminar Evaluation: Polar Geography ........................................... 81 Appendix K. NSTA Web Seminar Evaluation: Physical Science ........................................... 87 Appendix L. NSTA Web Seminar Evaluation: Energy and The Polar Environment ................ 93 Appendix M. NSDL Web Seminar Evaluation: States of Matter ........................................... 99 Appendix N. NSDL Web Seminar Evaluation: Rocks and Minerals..................................... 101 Appendix O. NSDL Web Seminar Evaluation: Life in the Polar Extremes ........................... 103 Appendix P. COSI Evaluation Report ............................................................................. 105 Appendix Q. COSI Entry/Exit Questionnaires.................................................................. 111

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom vi

List of Tables

Table 1. Website Rating Results..............................................................................................3

Table 2. Survey Respondents’ Positions...................................................................................8

Table 3. Grades Taught by Classroom Teacher Respondents.....................................................8

Table 4. Subjects Taught by Respondents ...............................................................................9

Table 5. Respondents’ Years of Teaching Experience................................................................9

Table 6. Respondents’ School Setting......................................................................................9

Table 7. Respondents’ School Type.........................................................................................9

Table 8. Reliability for the Student Science Views Questionnaire, Grade 3................................ 14

Table 9. Respondent’s Years of Teaching Experience.............................................................. 14

Table 10. Respondent’s Highest Degree Earned ..................................................................... 14

Table 11. Respondents’ Instructional Activities....................................................................... 15

Table 12. Students’ Learning Activities .................................................................................. 17

Table 13. Respondents’ Content Knowledge of Polar Regions.................................................. 18

Table 14. Respondents’ Integration of BPPB Materials ............................................................ 19

Table 15. Teachers’ Evidence of Impact on Students.............................................................. 20

Table 16. Respondent Student Gender .................................................................................. 23

Table 17. Respondent Student Age ....................................................................................... 23

Table 18. Respondent Student Ethnic Background ................................................................. 23

Table 19. Pre-Post Comparisons for Grade 3 Student Respondents.......................................... 24

Table 20. Pre-Post Comparisons for Kindergarten Student Respondents .................................. 25

Table 21. NSTA Conference Presentation Evaluation Results ................................................... 27

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom vii

List of Figures

Figure 1. Unique visitors to the BPPB website, March 2008 – March 2009 ..................................5 Figure 2. Percent of visits from search engines ........................................................................6 Figure 3. Frequency of visits resulting from teaching and learning keyword searches..................7 Figure 4. Top resources searched by respondents .................................................................. 10 Figure 5. Percent of teacher and non-teacher respondents familiar with digital tools................. 10 Figure 6. Percent of teacher and non-teacher respondents comfortable with digital tools .......... 11

Beyond Penguins and Polar Bears: Integrating Literacy and IPY in the K-5 Classroom 1

Introduction

Ohio’s Evaluation & Assessment Center for Mathematics and Science Education (E & A Center) is the external evaluator for the NSF-funded Beyond Penguins and Polar Bears (BPPB) project. Dr. Sarah B. Woodruff, Miami University, is the Principal Investigator for the evaluation. Kristen Morio and Jennifer Sutton are the Project Directors and Yue Li is the Senior Researcher and Statistician for the project. This report is divided into four sections. Section one provides background information about the BPPB project and goals. Section two provides information and findings from Year 2 evaluation and project activities. Section three provides future evaluation plans and ongoing evaluation activities. Section four summarizes the evaluation for Year 2 and provides recommendations for further project years.

Background

Beyond Penguins and Polar Bears (BPPB) is a collaboration between the Ohio State University (OSU) College of Education and Human Ecology, the Ohio Resource Center for Mathematics, Science, and Reading (ORC), the Byrd Polar Research Center, the Center of Science and Industry (COSI) Columbus, and the National Science Digital Library (NSDL). The project focuses on integrating science and literacy to educate Grades K-5 teachers on polar concepts using an electronic magazine interface. The project has four goals:

1. Providing context to online resources by creating, identifying, selecting, and adapting quality learning resources from the National Science Digital Library, the Ohio Resource Center, other IPY-funded projects, and additional high-quality content providers.

2. Modifying and building communication, production, and cyberinfrastructure tools to amplify resource discovery and access to resources, increase the ease of reuse and repurposing of content, decrease production times, and increase automated dissemination of IPY materials to various audiences.

3. Disseminating deliverables through presentations, publications, digital libraries, and push technologies.

4. Evaluating the impact of the project deliverables on Grades K-5 teachers and students.


Evaluation

The overarching evaluation effort focuses on assessing the progress toward project goals and monitoring project implementation. During Year 1 of the evaluation, the E & A Center staff worked with the Project Team on the development and refinement of the evaluation matrix. The E & A Center and the Project Team have communicated via email, conference calls and face-to-face meetings to discuss the progress of the evaluation and project. Year 2 evaluation activities included monitoring website statistics, conducting expert review of project deliverables, and analyzing online surveys to determine the effectiveness of the Beyond Penguins and Polar Bears website (see Evaluation of Website). Additionally, in the Evaluation of Impact section pre- and post-questionnaire data as well as content from teacher journals and classroom observations were analyzed to evaluate impact on the target audience. Dissemination data, including a list of dissemination activities as well as conference and web seminar evaluation data, were analyzed to determine the effectiveness of dissemination efforts (see Dissemination).

Evaluation of Website

The Beyond Penguins and Polar Bears website is an electronic magazine that was launched on March 1, 2008. Three sources of data were used to evaluate the website. The first source was content reviews by experts in the fields of literacy, science, and education. The second source of data was monthly webmetrics reports provided by the BPPB Project Director. Third was user feedback gathered via a focus group evaluation of the website homepage and two online questionnaires completed by visitors to the website.

Expert Reviews

Article Reviews

Three experts in literacy, science, and science education reviewed articles from Issues 1 through 5 of the electronic magazine (e-magazine) on the BPPB website. These expert reviews were summarized in terms of strengths and areas of concern. Content experts noted many strengths of the e-magazine, including easy accessibility for the intended audience; accurate, high-quality content; and a relevant and appealing range of themes and activities. Particularly noteworthy is the effective approach of the magazine to integrating literacy and content with a focus on increasing student learning. Some recommendations for improvement included adding inquiry lessons that promote active student investigation and making modifications to the writing style and tone to ensure appropriateness for the Grades K-5 teacher audience. Complete summaries of strengths and areas of concern for Issues 1 through 5 can be found in Appendix A. Website Review

Four experts in the fields of science, literacy and elementary education reviewed the website as a whole. The lead reviewer, an expert in the fields of literacy and technology, recruited three anonymous reviewers based upon their (a) interest in science and literacy, (b) their classroom


teaching experience, and (c) their potential to be BPPB site users. The reviewers evaluated the website based upon appearance, organization, usability, and appeal to potential users. A Website Evaluation Form (Appendix B), developed by Oklahoma State University (n.d.) and based upon the work of Lynna Ausburn (2001) was used by reviewers to rate several features of the website. After two hours of independent review, reviewers were asked to assign ratings based upon the evaluation form, as well as provide comments and suggestions. A summary of their reviews is provided in Website Rating Results and Overall Website Evaluation Results. The full website evaluation summary is provided in Appendix C.

Website Rating Results

The Website Evaluation Form used to evaluate the BPPB website consisted of 22 items rated on a scale using no (1), somewhat (2), and yes (3). The instrument was divided into three subscales. “Part 1: Quality and Source of Information” contained 6 items for a maximum score of 18. Part 1 items evaluated the accuracy, objectivity and currency of the website. Evaluating the technical function of the website, “Part 2: Technical Quality” consists of 9 items for a maximum score of 27. “Part 3: Multimedia Components and Features” is a 7-item subscale evaluating the use, appearance, appropriateness and purpose of the tools and features included on the website. Part 3 had a maximum score of 21. A total score was obtained by summing item ratings across all subscales. Scores were categorized as good (52-66 total points), okay (37-51 total points), or not good (22-36 total points). Table 1 displays the mean subscale and total scores of all three reviewers. Table 1. Website Rating Results

Website Evaluation Criteria Reviewer Mean Score

Possible Score

Part 1: Quality and Source of Information 17.3 18

Part 2: Technical Quality 23 27

Part 3: Multimedia Components and Features 19 21

Total 59.3 66

Overall Website Evaluation Results

Strengths

In general, the qualitative feedback obtained from all reviewers was positive and complimentary with respect to the overall quality of the website. The following comments were synthesized from the remarks of the three reviewers.

• The site is “very pleasing to the eye and fairly simple to navigate.”

• “I really liked this section [the ‘Browse by Column’ section]. It was really easy to use and was organized well. The lessons were nice to see. I also liked the sections on books to use for the classroom.”

• “The blog feature was fun.”

• “I am extremely impressed with the electronic books, especially for the primary learners.”


• The website is “beautifully designed, contains an enormous amount of helpful, practical information and is very well written. Teachers will enjoy using this site and will find it to be a great resource.”

• “I love the feature story each month, especially the eBook version.”

• “I was never overwhelmed or ‘lost’ on Beyond Penguins.”

• “Overall, I think that the site has a lot to offer both teachers and students …. Thank you

for all of your work! I will definitely be passing on the name of the site to the teachers that I work with here.”

Minor Concerns

Reviewers noted that they experienced some difficulty with two of the site features. First, they reported difficulty using the search engine. Second, they reported being somewhat confused about (a) the intent of the “Change Language” feature of the website and (b) the nature of the BPPB website, as an online magazine rather than simply a regular website. Specific comments pertaining to each of these concerns are found in the full review (Appendix C). Suggestions

While reviewers gave most features of the website high marks, they offered some suggestions, designed to enhance the website appearance, organization, appeal, and usability.

1. Place the “Contributors” section at the end of the site, not at the beginning.

2. Link the Adobe files and electronic books when opening to a new window.

3. Provide black line copies of illustrated books for those schools that may not have color printers available.

4. Consider placing a word label on the paw prints where the kids should click to keep

reading or go back.

5. Specify what subscribers will get and also provide assurances that email addresses will not be shared or sold.

6. Clarify the role or purpose of the “Change Language” feature.

7. Highlight the fact that BPPB is an “Online Magazine” not just a regular website.

Summary

Reviewers relayed that it was apparent that the planning and development of the BPPB website was thoughtful, purposeful and time-consuming and agreed that the website has a great deal to offer its intended audience. Collectively, reviewers believed the website was “beautifully designed, contains an enormous amount of helpful, practical information and is very well written. Teachers will enjoy using this site and will find it to be a great resource.”


Webmetrics

Web statistics were collected for the Beyond Penguins and Polar Bears website immediately after the launch date of March 1, 2008. Data available in the webmetrics reports include how visitors are finding the website (traffic sources), keywords used to find the site (search keywords), how long each visitor remained on the website (time on website), and how many visits to the website were from unique visitors. Figure 1 shows an increase in unique visitors to the website between March 2008 and March 2009.

Figure 1. Unique visitors to the BPPB website, March 2008 – March 2009.


Traffic Sources

The website http://beyondpenguins.nsdl.org was indexed in Google and started appearing in search results in the middle of May 2008. Since that time, search engines have been the primary traffic source for the website, increasing to over 80% of all visits by September 2008 and remaining at that rate through March 2008 (Figure 2). Referring websites made up the second largest source of traffic followed by direct visits. The top three referring sources were http://images.google.com (3,418 visits), http://expertvoices.nsdl.org (1,540 visits) and http://nsdl.org (754 visits).

Figure 2. Percentage of visits from search engines. Search Keywords

Webmetrics also report top search engine keywords used by visitors to find the Beyond Penguins and Polar Bears website. “Penguins”, “polar bears” and “beyond penguins and polar bears” made up the majority of keywords used in March and April 2008. Starting in May 2008, thousands of keywords, resulting in visits to the BPPB website, were collected each month. To assess resource discovery among the target audience, teaching and learning terms, including “teaching”, “lessons”, “activities”, “misconceptions”, “assessment”, “classroom”, and “projects”, or indicating a grade level were counted. The number of keyword searches using these terms increased substantially in September 2008 and remained at high levels through March 2009 (Figure 3). Other common keywords included “discrepant events”, “expository articles”, and “rocks and minerals”.


Figure 3. Frequency of visits resulting from teaching and learning keyword searches. Time on Website Time spent on the BPPB website is collected through webmetrics to compare traffic source usage. Using all traffic sources, the average amount of time visitors spent on the website between March 2008 and March 2009 was 2 minutes 38 seconds. Although a much larger number of visitors were referred from http://images.google.com, visitors referred from the NSDL websites spent a longer, average amount of time on the website (32 seconds versus 4 minutes 27 seconds). The large number of short duration visits to the site from the referring sites (such as Google) produce an average visit time that may not be representative of a visit by the typical site user.

User Feedback

Online Questionnaires

BPPB project staff created two online questionnaires to integrate into the website to gather immediate feedback from website visitors. One questionnaire was a General Website Questionnaire and the other was specific to articles posted on the website. The General Website Questionnaire consisted of eight items and five questions asking for demographic information. Six items were multiple-choice and two items items were on a Likert-type scale. The Article-Specific Questionnaire consisted of nine items; six multiple-choice items, two Likert-type items, and one open-response item. A total of five website visitors completed the Article-Specific Questionnaire and 36 completed the General Website Questionnaire. Article-Specific Questionnaire

Website visitors completing this questionnaire included two classroom teachers, a district K-12 teacher, a parent, and a preservice teacher. Based on their experiences with an article in the e-magazine, three of the five respondents selected strongly agree to returning to the article, sharing the article, and being motivated to explore more of the magazine. Three of the five respondents chose very good when commenting on the quality of the featured lessons, the appropriateness of the article for the grade level, and the correlation of the article to the National Science Education Standards (NSES) or National Council of Teachers of English (NCTE)


standards. All five respondents chose good or very good when responding to items regarding integration of science and literacy and the article’s use of images. No respondents chose a negative response for any item on the questionnaire. General Website Questionnaire

This instrument gathered user-related information from website visitors. Items included on this questionnaire collected demographic information, information regarding visitors’ use of technology, and website feedback.

General Website Questionnaire Respondent Demographics

Nearly half of the respondents (47.2%) were Grades P-12 classroom teachers. Other respondents included an academic librarian, a college administrator (K-12 outreach), a district K-12 teacher, a journalist, a literacy tutor, a manager, and a radiology technician. Table 2 displays respondents’ position descriptions. Table 2. Survey Respondents’ Positions

Position n Percentage Classroom teacher (P–12) 17 47.2 College faculty 1 2.8 Informal educator 3 8.3 Other 8 22.2 Preservice teacher 4 11.1 Researcher 2 5.6

Table 3 through Table 7 illustrate respondent demographics only for those that identified themselves as classroom teachers. A majority of classroom teacher respondents (76.5%) taught the targeted K-5 grade band. The typical website user was a Grades K-5 science teacher with fewer than ten years of teaching experience (Tables 3, 4, and 5). Most respondent teachers taught in an urban, public school setting (Tables 6 and 7). Table 3. Grades Taught by Classroom Teacher Respondents

Grade Band n Percentage Preschool 2 11.8 Primary (Grades K-2) 8 47.1 Elementary (Grades 3-5) 5 29.4 Middle (Grades 6-8) 3 17.6 High (Grades 9-12) 1 5.9

Note. Respondents chose all that apply.


Table 4. Subjects Taught by Respondents

Subject n Percentage Elementary self-contained (all subjects) 8 47.1 Science 7 41.2 Reading 2 11.8 Language Arts (writing and grammar) 2 11.8 Math 4 23.5 Social Studies 1 5.9 Other 3 17.6

Note. Respondents chose all that apply. Table 5. Respondents’ Years of Teaching Experience

Years of Experience n Percentage 0-10 years 7 41.2 11-20 years 5 29.4 20+ 5 29.4

Table 6. Respondents’ School Setting

School Setting n Percentage Urban 8 47.1 Suburban 5 29.4 Rural 4 23.5

Table 7. Respondents’ School Type

School Type n Percentage Public 14 82.4 Private 3 17.6 Charter 0 0.0 Other 0 0.0

Respondents’ Reasons for Visiting Website

Ten of the 36 respondents reported their primary reason for visiting the BPPB website was to find teaching resources. Specific types of teaching resources searched are displayed in Figure 2. Other reasons for visiting the website included wanting to learn about climate change and website curiosity.


Figure 4. Top resources searched by respondents. (*Respondents chose all that apply.)

Respondents’ Comfort and Familiarity with Technology

The General Website Questionnaire measured respondent familiarity (Figure 5) and comfort (Figure 6) with digital tools. Data indicated that compared to those that identified themselves as P-12 teachers (n = 17), non-teachers reported more comfort and familiarity with all digital tools except YouTube/TeacherTube. Teachers had a markedly lower level of comfort and familiarity with RSS feeds and a lower level of familiarity with social networking than did non-teachers. In general, respondents (n = 36) were least familiar and least comfortable with virtual worlds.

Figure 5. Percentage of teacher and non-teacher respondents familiar with digital tools.


Figure 6. Percent of teacher and non-teacher respondents comfortable with digital tools. Homepage Review by Focus Group

On September 9, 2008 a focus group of eight elementary teachers, under the guidance of a facilitator, visited the BPPB homepage and individually responded to a series of open-response questions and Likert-type items rating different website components. According to 7 of the 8 participants, the most prominent item on the homepage was the image. This group’s first impression of the homepage was positive (4.25 on a scale of 1-5, 5 = most positive). Generally, participants reported that the layout of the homepage was clear, organized, attractive, and aesthetically pleasing. One participant commented that the layout of text on one side and image on the other made the text appear cluttered, while another commented that the image was too small. Most participants (6 of the 8) correctly identified the intended audience as Grades K-5 teachers, claiming they were able to deduce this from the website header or homepage text. All participants were able to identify the source of funding for the BPPB project as the National Science Foundation from the information provided on the homepage and were able to easily identify who to contact from the “About” page or the “Contacts” link. All eight participants also were able to identify magazine issue titles by finding the “Archives” page. Participants stated that the task of identifying these titles was very easy (5 on a scale of 1-5, 5 = most positive).

Overall, participants indicated that the layout and general appearance of the homepage would compel them to further explore the website. Participants stated that they would have been deterred by unattractive images or by the lack of immediate information regarding the rest of the website. One teacher commented on the lack of information on the homepage regarding teaching resources. Participants made suggestions for improvement of the homepage, including adding a way to preview what is included on the rest of the website, adding an explicit reference to the website goal of integrating science and literacy, and choosing a more universally appealing title for the homepage heading.


Evaluation of Impact

Impact data was collected in Year 2 using three sources: pre- and post-questionnaires administered to teachers and students, classroom observation, and teacher field journals. Two instruments were used to collect impact data for project evaluation. Each instrument is described below. Both instruments originally were developed for the 1993 National Survey of Science and Mathematics Education (Weiss, Matti, & Smith, 1994). These instruments were used for the evaluation of Ohio’s Statewide Systemic Initiative, including use in the Ohio Middle Level Mathematics and Science Education Bridging Study (Kahle & Rogg, 1996). Currently, multiple E & A Center projects, including the University of Pennsylvania Mathematics and Science Partnership project (Kahle & Scantlebury, 2006) and City University of New York Mathematics Science Partnership in New York City project (Kahle, 2006) use modified versions of these instruments. Additionally, field-test teachers (n = 2) were asked to keep a field journal during the implementation of one or more BPPB-inspired lessons. Observations of field-test teacher classrooms were conducted by BPPB project personnel.

Instruments

Teacher Questionnaire

The Teacher Questionnaire is an adaptation of a valid and reliable instrument used in the Ohio Middle Level Mathematics and Science Education Bridging Study (Kahle & Rogg, 1996). Two of the five original subscales were used for the BPPB project, “How I Teach” and “What My Students Do.” Items were added to both subscales to reflect the project focus on literacy in the science classroom. Seven items were added to the “How I Teach” subscale and four items were added to the “What My Students Do” subscale. These additional items were developed by project staff and modified by the E & A Center evaluation team. An additional subscale was added to this questionnaire to assess teacher science content knowledge specific to the project content focus. Project staff developed items for this new “Knowledge of Polar Regions” subscale. The evaluation team modified the items and formatted the subscale. The adapted pre-questionnaire can be found in Appendix D. Three subscales were included on pre- and post-questionnaires. The teaching and learning subscales (“How I Teach” and “What My Students Do”) each consisted of 19 items on a 5-point Likert-type scale with responses ranging from almost never (1) to very often (5). The “Knowledge of Polar Regions” subscale consists of 11 items. Responses for this subscale were on a 3-point Likert-type scale with responses ranging from disagree (1) to agree (3). The pre-questionnaire also collected demographic information. The post-questionnaire consisted of the subscales previously noted with two additional subscales that measured impact of project materials on teacher participants and their students. These two additional subscales are: “Integration of BPPB Materials” and “Evidence of Impact on Students.” Both of these subscales consisted of 11 items and were on a 5-point Likert-type scale with responses ranging from strongly disagree (1) to strongly agree (5). The “Evidence of Impact on Students” subscale had three parallel items to the Student Questionnaire developed by the E & A Center. These items are (a) my students like science, (b) my students believe they are good at science, and (c) my students understand more of what goes on in science. The remaining items


on the “Evidence of Impact on Students” subscale were created by the project staff and modified by the E & A Center evaluation team. The “Integration of BPPB Materials” subscale was modeled after an instrument retrieved from the Online Evaluation Resource Library (OERL) and adapted with permission to reflect the goals of this project. This 1994-1995 Evaluation Survey (n.d.) was originally created for a similar NSF project to determine the impact of an integrated program on participating teachers. The post-questionnaire can be found in Appendix E. Student Questionnaire

The Student Science Views Questionnaire is an 11-item instrument originating from a subscale of the Student Questionnaire developed for the Ohio Middle Level Mathematics and Science Education Landscape Study (Kahle & Rogg, 1996). The original instrument consisted of eight items measuring the views of students regarding science. Three additional items were added to the instrument to assess student perceptions of the importance of literacy integration in science. These items are (a) reading is important in science, (b) writing is important in science, and (c) discussion is important in science. These three new items were developed by project staff and modified by the E & A Center evaluation team. All 11 items were modified to be accessible to kindergarten and elementary-aged students by reducing the Likert-type scale from 5 to 3 points and by changing the responses from numbers to emoticons (i.e., = disagree, = I don’t know, = agree). The adapted student questionnaire can be found in Appendix F. Reliability coefficients were calculated for the student instrument for Grade 3 and kindergarten respondents based on pre-questionnaire data. The instrument was found to have low reliability when used with both samples (α = 0.56 for Grade 3; α = 0.27 for Grade K). Reliability coefficients were recalculated for Grade 3 data for scenarios in which each item was removed (see Table 8). Removing or modifying item 5, science is more for boys than girls, would increase the instrument reliability coefficient substantially (to α = 0.65). The results of the instrument reliability analysis for Grade K indicate a need for further research into methods of obtaining reliable results from children of this age. Items 3 (if I had a choice I would not study more science), 5 (science is more for boys than girls) and 6 (learning science is mostly memorizing facts) were problematic. The wording of these items may have confused younger students. See the Recommendations section at the end of the report for further discussion of this issue.


Table 8. Reliability for the Student Science Views Questionnaire, Grade 3

Demographics and Data Analysis: Teacher Questionnaire

Teacher Demographics

Two field-test teachers and four non-field-test teachers completed the Teacher Questionnaire. All respondents were female, most were White (83%), and none had a degree in a science-related discipline. All respondents reported having had professional development experience, 67% with some and 33% with much. Tables 9 and 10 display characteristics for all teacher questionnaire respondents. Table 9. Respondent’s Years of Teaching Experience

n Percent 0-1 year 0 0% 2-5 years 1 17% 6-10 years 1 17% 11-15 years 1 17% Over 15 years 3 50% Total 6 100%

Table 10. Respondent’s Highest Degree Earned

n Percent Bachelor’s 1 17% Master’s 3 50% Master’s +30 2 33% Doctorate 0 0% Total 6 100%

Cronbach Alpha Grade 3 (n = 21) 0.557 Item-Total Statistics Cronbach Alpha if item Deleted Question 1 0.503 Question 2 0.448 Question 3 0.550 Question 4 0.517 Question 5 0.652 Question 6 0.541 Question 7 0.536 Question 8 0.505 Question 9 0.509 Question 10 0.493 Question 11 0.552


Findings

Teaching and Learning

Table 11 through Table 15 provide frequency data for field-test (n = 2) and non-field-test (n = 4) teachers. The responses often and very often were combined, and almost never and seldom were combined for the teaching and learning subscales (“How I Teach” and “What My Students Do”). The responses strongly disagree and disagree were combined, and strongly agree and agree were combined for the “Integration of BPPB Materials” and “Evidence of Impact on Students” subscales. Due to small sample sizes, no statistical tests were performed. These baseline data provide information against which progress of the project may be calibrated. For complete frequency tables see Appendices G (pre-questionnaire) and H (post-questionnaire). Table 11 provides information regarding the first subscale of the teacher questionnaire, “How I Teach.” Seven items from this subscale showed no change in the aggregate level of agreement after the use of BPPB materials. Although all respondents indicated that they often or very often integrate science instruction with literacy by providing opportunities for discussion, 50% or fewer indicated often or very often to providing time for students to discuss subject-specific ideas among themselves or having them discuss experiments from the history of science. There was a decrease in the number of respondents who chose often and very often to describe opportunities provided to their students to discuss the work of scientists. Few respondents chose often or very often to indicate how frequently they give lectures. Table 11. Respondents’ Instructional Activities (n = 6)

Item Almost Never & Seldom

Sometimes Often & Very Often

In my classroom, I… (%) (%) (%) arrange seating to facilitate student discussion. Pre 0 0 100

Post 0 0 100

use open-ended questions. Pre 0 0 100

Post 0 17 83 require that my students supply evidence to support their claims. Pre 0 0 100

Post 0 17 83

encourage questions from my students. Pre 0 17 83

Post 0 0 100 allow my students to work at their own pace. Pre 0 0 100

Post 0 17 83 encourage my students to explain concepts to one another. Pre 0 17 83

Post 0 17 83 encourage my students to consider alternative explanations. Pre 0 17 83


Post 0 17 83

discuss the work of scientists. Pre 0 33 67

Post 0 83 17 provide time for my students to discuss subject-specific ideas among themselves. Pre 17 50 33

Post 17 33 50

discuss experiments from the history of science. Pre 33 67 0

Post 0 67 33

use non-traditional/authentic assessments. Pre 0 17 83

Post 0 17 83

give lectures. Pre 83 0 17

Post 67 33 0

integrate science instruction with literacy by providing opportunities for reading. Pre 0 0 100

Post 0 0 100 integrate science instruction with literacy by providing opportunities for writing. Pre 0 17 83

Post 0 17 83 integrate science instruction with literacy by providing opportunities for discussion. Pre 0 0 100

Post 0 0 100

choose nonfiction, science-themed articles and books for my classroom. Pre 0 17 83

Post 0 0 100 choose nonfiction articles and books that are not science-themed for my classroom.

Pre 0 17 83

Post 0 33 67

regularly include science texts in my reading instruction.* Pre 0 17 50

Post 17 17 67 frame science concepts using grade-appropriate real world applications. Pre 0 33 67

Post 0 17 83 Note. Responses of 60% or more are indicated in bold. *n = 4, two respondents did not answer this item on the pre-questionnaire.

Concerning student learning activities (Table 12), four items showed no change in the aggregate level of agreement after exposure to BPPB materials. There was an increase in the number of respondents that chose often and very often to describe the amount of time their students talk with one another to promote learning. There was a decrease in the number of respondents that


chose often and very often to describe the amount of time their students write to communicate and explain scientific results. Table 12. Students’ Learning Activities (n = 6)

Item Almost Never & Seldom

Sometimes Often & Very Often

In my classroom, my students… (%) (%) (%) use data to justify responses to questions. Pre 0 50 50

Post 0 33 67 argue or debate with one another about the interpretation of data. Pre 33 33 33

Post 33 50 17

repeat experiments to confirm results. Pre 17 50 33

Post 17 33 50 use multiple sources of information to learn. Pre 17 0 83

Post 0 17 83 consider alternative explanations to accepted theories. Pre 50 0 50

Post 17 50 33

design activities to test their own ideas. Pre 17 33 50

Post 17 33 50

consult one another as sources for learning. Pre 0 50 50

Post 17 17 67 talk with one another to promote learning. Pre 0 33 67

Post 0 0 100

use educational technology in class. Pre 0 33 67

Post 0 33 67

develop scientific literacy skills. Pre 0 17 83

Post 0 17 83 learn about real world applications of science.* Pre 0 0 83

Post 0 17 83

take notes and listen to lectures.* Pre 50 50 0

Post 50 33 0

do worksheets.* Pre 17 50 17

Post 33 67 0

learn scientific facts by using worksheets.

Pre 67 33 0


Post 67 33 0

memorize scientific facts. Pre 67 33 0

Post 67 33 0

use science as the basis for authentic reading. Pre 17 17 67

Post 0 50 50 use science as the basis for authentic writing. Pre 17 33 50

Post 0 50 50

write to communicate and explain scientific results. Pre 0 17 83

Post 0 50 50

read science-themed, non-fiction in class. Pre 17 17 67

Post 0 17 83 Note. Responses of 60% or more are indicated in bold. *n = 5, one respondent did not answer these items on either the pre- or post-questionnaire.

Knowledge of the Polar Regions

Responses to the “Knowledge of the Polar Regions” subscale are displayed in Table 13. Four items showed no change in the aggregate level of agreement after use of BPPB materials, including the item penguins and polar bears live together in the polar regions. All teachers agreed with the statement that the Arctic and Antarctica play an important role in regulating global climate. All respondents disagreed with the statements there is no human life in the polar regions, and there has been little physical change to the polar regions in our natural history. There was an increase in the number of respondents that chose disagree for the item stating there is little difference in climate between the Arctic and Antarctica after accessing the BPPB materials. Insufficient data have been collected to statistically analyze significant impact of the materials on teachers’ content knowledge by pre-post analysis. Table 13. Respondents’ Content Knowledge of Polar Regions (n = 6)

Item Disagree Undecided Agree (%) (%) (%) There is little difference in climate between the Arctic and Antarctica. Pre 50 17 33

Post 83 0 17

The Arctic and Antarctica play an important role in regulating global climate. Pre 0 0 100

Post 0 0 100

There is no human life in the polar regions. Pre 100 0 0

Post 100 0 0


The polar regions are cold and snowy year-round. Pre 50 17 33

Post 67 0 33

There are few living organisms in the polar regions.

Pre 83 0 17

Post 100 0 0

It is always dark in the polar regions. Pre 100 0 0

Post 83 0 17

There is little difference in geography between the Arctic and Antarctica.

Pre 83 0 17

Post 100 0 0

There has been little physical change to the polar regions in our natural history. Pre 100 0 0

Post 100 0 0

Penguins and polar bears live together in the polar regions. Pre 67 0 33

Post 67 0 33

There is little difference in flora and fauna between the Arctic and Antarctica. Pre 67 17 17

Post 67 33 0

There is high humidity in the polar regions. Pre 33 33 33

Post 50 33 17 Note. Responses of 60% or more are indicated in bold.

Integration of BPPB Materials

Table 14 displays response frequencies for the “Integration of BPPB Materials” post-questionnaire subscale. All participants agreed that as a result of the BPPB project they have designed activities for my students that use the objectives of the BPPB project. All participants agreed that their science content knowledge has increased (though this cannot be substantiated by findings of the content knowledge subscale), they have learned that the Arctic and Antarctica are important geographic regions, and that they have become more aware of changes taking place in the polar regions as a result of the BPPB project. Half of the respondents disagree that they have changed the ways I assess student understanding after use of the BPPB materials.

Table 14. Respondents’ Integration of BPPB Materials (n = 6)

Item Strongly Disagree & Disagree

Undecided Agree & Strongly

Agree As a result of the “Beyond Penguins and Polar Bears” Project, I have… (%) (%) (%)

incorporated the BPPB content into a specific unit of study. 17 0 83


designed activities for my students that use the objectives of the BPPB project. 0 0 100

changed the instructional strategies that I use in my classroom. 17 0 83

changed the ways I assess student understanding. 50 17 33

changed the ways I use educational technology. 33 0 67

changed the science curriculum in my classroom.

17 0 83

increased my own science content knowledge. 0 0 100

gained confidence in teaching science to my students. 17 17 67

learned that the Arctic and Antarctica are important geographic regions. 0 0 100

become more aware of changes taking place in the polar regions. 0 0 100

become an advocate for protection of the polar regions. 0 17 83

Evidence of Impact on Students

Post-questionnaire frequencies for the “Evidence of Impact on Students” subscale are displayed in Table 15. The majority of teacher participants (from 66% to 83%) indicated that they agree or strongly agree with all items on the “Evidence of Impact on Students” subscale. Table 15. Respondents’ Evidence of Impact on Students

Item Strongly Disagree & Disagree

Undecided Agree & Strongly

Agree As a result of the “Beyond Penguins and Polar Bears” Project… (%) (%) (%)

my students like science more. 0 33 66

my students believe they are good at science.

0 33 66

my students understand more of what goes on in science. 0 33 66

my students have increased their knowledge of specific science concepts through the BPPB materials.

0 17 83


my students have learned the BPPB content. 0 17 83

my students’ scientific literacy has increased. 0 17 83

my students’ achievement/performance I science has improved. 0 17 83

my students’ effort in science has improved. 17 17 66

my students’ reading skills in science have improved. 0 33 66

my students’ writing skills in science have improved.

0 33 66

my students’ discussion skills in science have improved. 0 33 66

Classroom Field Testing

Two teachers were recruited to field test the BPPB project deliverables. One field-test teacher taught Grade 3 and the other taught kindergarten. Each teacher was asked to keep a journal about the use of one or more topics selected for lesson development from the BPPB website. Each teacher also was observed by a BPPB project team member using the Oregon Collaborative for Excellence in the Preparation of Teachers (OCEPT) - Teaching Observation Protocol (O-TOP), (Wainright, Morrell, Flick, & Schepige, 2004) included in Appendix I. The O-TOP instrument provided quantitative scoring to assess the teaching practices used as the field-test teachers implemented BPPB deliverables. Ten categories were scored on a scale ranging from Not Observed (N/O) to Characterizes Lesson (4). Two inter-related lessons were observed for the Grade 3 teacher participant. One lesson was observed for the Grade K teacher participant. Grade 3: Teacher Journal

The Grade 3 field-test teacher kept a field journal on the planning and implementation of a lesson associated with the BPPB website. This participant chose Issue 6: Rocks and Minerals (September 2008) due to topic alignment with the Grade 3 curriculum. Resources chosen from the issue of BPPB included a cover story about, and a guest speaking appearance by, geologist Julie Codispoti from the United States Polar Rock Repository, located at the Ohio State University Byrd Polar Research Center. The teacher also used the “Virtual Bookshelf” from the website and a rock kit obtained from Ohio State University. The participant described activities of the lesson, including student groups sharing and discussing rocks that they brought from home. The participant indicated that she acted as a facilitator for the lesson and that the rock kit, as well as the “Virtual Bookshelf”, proved to be very helpful and appropriate teaching tools. The participant was positive regarding the effects of the activities and resources on students. The participant suggested that the website include a space for students to post ideas and thoughts regarding concepts. She also indicated that through her work with BPPB project personnel, she learned the importance of hands-on activities in the classroom.


Grade 3: Classroom Observation

Both Grade 3 lessons observed included challenging of ideas (

€

x = 3.5), probing of students existing knowledge (

€

x = 3.5), and encouragement for students to respond to each other (

€

x = 3). Observations suggest that neither lesson allowed for the students to be reflective about their learning (

€

x = 2). One of the lessons encouraged students to use various methods of problem solving, while the other lesson lacked opportunities for such activities. The observer noted that the teacher did not have a deep understanding of the concept for the first observed lesson on rock collecting and sorting, however, she had a much stronger understanding of the concept for the second, follow-up lesson. Observation notes indicated that the teacher utilized literature effectively for both lessons. These notes also indicated that the students were engaged in both lessons as evidenced by thoughtful questions and participation, as well as collaborative discussion. The observer also noted that the teacher reached out for support after both lessons to determine strategies for how to proceed with the concept.

Grade K: Teacher Journal

The Grade K field-test teacher kept a field journal on the planning and implementation of a lesson associated with the BPPB website. This participant chose three issues: Issue 1: A Sense of Place (March 2008); Issue 4: Climate and Weather (June/July 2008); and Issue 5: Water, Ice and Snow (August 2008). The kindergarten teacher chose these issues as they align with the kindergarten curriculum. Resources used included books from the “Virtual Bookshelf” and printable books from the “Science and Literacy” pages. The teacher reported modifying a printable book to make it more accessible to her students. For these topics, the teacher used many books that connected geography concepts with weather and climate. The students were focused with a “Know, Want to know, Learned” (KWL) chart following the progress of information attainment of the class. The students also worked through stations that each contained an activity related to the concepts. At these stations, students looked at icebergs, snowflakes, and properties of whale blubber. The teacher indicated that she was happy with the discussion that was elicited among her students. She stated that all of the books, especially the printable books, were appropriate and engaging for her students, though she suggested that the printable books be larger for students of lower grade levels. Grade K: Classroom Observation

A few of the categories and indicators from the O-TOP instrument were marked by the observer as inappropriate for the age group. This resulted in a lower score for some categories and other categories that were not observed at all. The lesson and activities involved collaborative work and challenging of ideas. The activities were shown to assess prior knowledge appropriately. Students were encouraged to think divergently and make connections with other subject areas. The teacher showed a firm understanding and used a variety of methods to convey the concept. Observation notes indicated that although the students had a difficult time transitioning from the hands-on activity to listening to a story, they were engaged and active in the activities. The observer noted that the teacher used an effective combination of BPPB deliverables and her own activities and materials. The observer noted that the students demonstrated a deep understanding of polar concepts for such a young age group.


Demographics and Data Analysis: Student Questionnaire

Student Demographics

Table 16 through Table 19 display characteristics for the Beyond Penguins and Polar Bears (BPPB) student respondents. Student field-test respondent groups included 23 Grade 3 students and 27 kindergarten students. A slight majority of the students were male and the majority of the students in both grades were White. Most students were either 5 or 6 years old, or 8 or 9 years old.

Table 16. Respondent Student Gender

n Percent Female 23 46% Male 27 54% Total 50 100%

Table 17. Respondent Student Age

n Percent 5 17 34% 6 9 18% 7 2 4% 8 14 28% 9 8 16% Total 50 100%

Table 18. Respondent Student Ethnic Background

n Percent African American 9 19% American Indian/Alaskan Native 1 2% Asian or Pacific Islander 4 8% Latino/a 0 0% White 29 60% Other 5 10% Total 50 100%

Findings

Grade 3

The E & A Center analyzed pre- and post-data for Grade 3 respondents. For analysis, items were coded as follows: disagree (1), I don’t know (2), and agree (3). Grade 3 student pre-post comparisons showed significant differences for four items. These items were if I had a choice, I would not study any more science, science is more for boys than girls, learning science is mostly memorizing facts and writing is important in science. Pre-post comparisons are shown in Table 19.


Table 19. Pre-Post Comparisons for Grade 3 Student Respondents

Item Pre/Post n Mean SD t-value df p-value Q1: I like science. Pre 23 2.78 0.42 0.57 22 0.575 Post 23 2.74 0.45 Q2: I am good at science. Pre 23 2.30 0.56 0.00 22 1.000 Post 23 2.30 0.47 Q3: If I had a choice, I would not study any more science.

Pre 22 1.32 0.57 -2.63 21 0.016

Post 22 1.64 0.85 Q4: I understand most of what goes on in science.

Pre 22 2.77 0.53 1.67 21 0.110

Post 22 2.45 0.74 Q5: Science is more for boys than girls.

Pre 20 1.40 0.50 2.18 19 0.042

Post 20 1.20 0.41 Q6: Learning science is mostly memorizing facts.

Pre 23 2.22 0.74 2.61 22 0.016

Post 23 1.91 0.73 Q7: Almost all people use science in their jobs.

Pre 22 2.23 0.75 1.31 21 0.204

Post 22 2.00 0.69 Q8: Science is useful for solving everyday problems.

Pre 22 2.50 0.60 0.49 21 0.628

Post 22 2.41 0.67 Q9: Reading is important in science.

Pre 23 2.61 0.66 -0.77 22 0.451

Post 23 2.74 0.54 Q10: Writing is important in science.

Pre 23 2.43 0.73 -2.15 22 0.043

Post 23 2.78 0.42 Q11: Discussion is important in science.

Pre 23 2.61 0.50 -0.62 22 0.539

Post 23 2.70 0.47 Note. Significant findings (p < .05) are indicated in bold. Analysis of student responses shows that after exposure to teacher instruction, utilizing the BPPB resources, students indicate less agreement with the statements that science was mostly memorizing facts and that science is more for boys than girls. Another important post-treatment finding showed that Grade 3 students tended to agree more with the item indicating that writing is important in science.


Kindergarten

The E & A Center analyzed pre- and post-data for kindergarten respondents. For analysis, items were coded as follows: disagree (1), I don’t know (2), and agree (3). Kindergarten student pre-post comparisons showed significant differences for two items. These items were almost all people use science in their jobs, and reading is important in science. The post-mean scores for the two significant items were lower than the pre-treatment mean scores. It should be noted that teacher comments included with pre-questionnaire data indicated that four of the 22 respondents answered yes to all of the pre-questionnaire items. Pre-post comparisons are shown in Table 20. Table 20. Pre-Post Comparisons for Kindergarten Student Respondents

Item Pre/Post n Mean SD t‐value df p‐valueQ1: I like science. Pre 18 2.89 0.47 0.57 17 0.579 Post 18 2.78 0.65 Q2: I am good at science. Pre 18 2.67 0.77 1.72 17 0.104 Post 18 2.22 1.00 Q3: If I had a choice, I would not study any more science.

Pre 18 2.11 1.02 1.00 17 0.331

Post 18 1.78 1.00 Q4: I understand most of what goes on in science.

Pre 18 2.17 0.99 1.59 17 0.130

Post 18 1.78 0.94 Q5: Science is more for boys than girls.

Pre 17 2.06 1.03 1.00 16 0.332

Post 17 1.82 1.01 Q6: Learning science is mostly memorizing facts. Pre 18 2.22 0.94 -0.53 17 0.604

Post 18 2.39 0.85 Q7: Almost all people use science in their jobs. Pre 18 2.89 0.32 2.37 17 0.030

Post 18 2.28 0.96 Q8: Science is useful for solving everyday problems. Pre 18 2.17 0.99 0.00 17 1.000

Post 18 2.17 0.99 Q9: Reading is important in science. Pre 18 2.89 0.32 2.49 17 0.024

Post 18 2.22 1.00 Q10: Writing is important in science. Pre 18 2.61 0.78 0.52 17 0.607

Post 18 2.50 0.86 Q11: Discussion is important in science. Pre 18 2.61 0.78 0.64 17 0.528

Post 18 2.44 0.92 Note. Significant findings (p < .05) are indicated in bold.


Dissemination

Project team members disseminated project information at conferences, meetings and electronic events, including the National Science Teachers Association (NSTA) and National Science Digital Library (NSDL) collaborative web seminar series. BPPB also partnered with the Center of Science and Industry (COSI) to create a Polar Adventure weekend event. The following is a list of dissemination activities: Presentations at Local and National Conferences and Meetings • Polar Geography Web Seminar (NSDL/NSTA Seminar Series) (5/27/08) • Monthly Tapped in Events (7/9/08; 8/8/08; 9/4/08; 10/9/08; 11/6/08; 12/11/08; 1/8/09;

2/5/09) • National Science Digital Library Annual Meeting (9/31-10/2/08) • Physical Science From the Poles Web Seminar (NSDL/NSTA Seminar Series) (10/29/08) • NSTA Regional Conference (Charlotte, NC) (10/30 – 11/1/08) • Elementary Science Focus Group (NSTA Regional Conference, Charlotte, NC)

(10/30-11/1/08) • California Science Teachers Association Conference (11/1/08) • National Middle School Association Conference (10/30 – 11/1/08) • Energy and the Polar Environment Web Seminar (NSDL/NSTA Seminar Series) (11/13/08) • DR K-12 Meeting (11/12-11/13/08) • NSTA Regional Conference (Portland, OR) (11/20 – 11/22/08) • Regional Reception (NSTA Regional Conference, Portland, OR) (11/21/08) • NSTA Regional Conference (Cincinnati, OH) (12/4 – 12/6/08) • IRA Regional Conference (Nashville, TN) (12/8 – 12/10/08) • States and Changes of Matter Web Seminar (NSDL Brown Bag Series) (1/13/09) • Rocks and Minerals Web Seminar (NSDL Brown Bag Series) (1/22/09) • McMillan McGraw-Hill Meeting (1/31/09) • Etech Conference (Columbus, OH) (2/2 – 2/4/09) • AAAS Conference (Chicago) (2/14/09) • NSTA National Conference (New Orleans) (3/19 – 3/22/09) • Ecosystems: Life in the Polar Extremes Web Seminar (NSDL Brown Bag Series) (4/7/09) • Arctic and Antarctic Birds Web Seminar (NSDL/NSTA Seminar Series) (4/21/09) Distributed Promotional Materials at Conferences and Meetings • Ohio State College of Education and Human Ecology Info Rally (Columbus, OH) (9/6/08) • User Testing with Group of Elementary Teachers (Columbus, OH) (9/9/08) • National Science Digital Library Annual Meeting (9/31-10/2/08) • NSTA Regional Conference (Charlotte, NC) (10/30 – 11/1/08) • Elementary Science Focus Group (NSTA Regional Conference, Charlotte, NC)

(10/30-11/1/08) • National Middle School Association Conference (10/30 – 11/1/08) • DR K-12 Meeting (11/12-11/13/08) • NSTA Regional Conference (Portland, OR) (11/20 – 11/22/08) • Regional Reception (NSTA Regional Conference, Portland, OR) (11/21/08) • NSTA Regional Conference (Cincinnati, OH) (12/4 – 12/6/08) • IRA Regional Conference (Nashville, TN) (12/8 – 12/10/08) • Etech Conference (Columbus, OH) (2/2 – 2/4/09) • AAAS Conference (2/14/09)


• COSI: Polar Adventure Exhibit (2/27 – 3/1/09) • NSTA National Conference (New Orleans, LA) (3/19 – 3/22/09) • OCTELA Conference (Columbus, OH) (3/27 – 3/28/09)

National Science Teachers Association Conference Presentation Evaluations Beyond Penguins and Polar Bears project team members presented at four NSTA regional conferences in Cincinnati, OH (12/5/08), Charlotte, NC (11/1/08), Boston (3/30/08), and New Orleans, LA (two presentations, 3/21 & 3/22/09). Participants at these conferences answered a 9-item questionnaire regarding the presentation, the speaker and the topic. This questionnaire was provided by NSTA conference organizers and administered to conference attendees after each conference session. Responses were on a 5-point Likert-type scale with responses ranging from strongly disagree (1) to strongly agree (5). Evaluations for presentations by BPPB project team members at each regional conference are displayed in Table 21. Table 21. NSTA Conference Presentation Evaluation Results

Evaluation Questions Mean

Boston (n = 11)

Mean Cincinnati

(n = 4)

Mean Charlotte (n = 9)

Mean New Orleans

Session 1 (n = 5)

Mean New Orleans

Session 2 (n = 4)

Overall Mean

(n = 33)

I selected this session for immediate use in the classroom.

4.6 4.7 4.9 4.5 4.5 4.64

I selected this session based on the reputation of the speaker.

3.2 3.5 3 4.2 2.2 3.22

I selected this session to improve my personal pedagogical knowledge/skill.

4.3 4.7 4.8 5 5 4.76

I selected this session to improve my science content knowledge.

4.4 5 5 5 4 4.68

This session met my needs. 4 4.7 5 5 4.7 4.68

The information presented was clear and well-organized.

4.4 5 5 5 4.5 4.78

Safe practices were employed. 4.8 5 5 5 4.5 4.86

The session avoided commercial solicitation. 4.7 5 4.8 5 4.3 4.76

The session should be repeated at another NSTA conference.

4.2 4.7 5 5 5 4.78

Overall Average 4.3 4.7 4.7 4.9 4.3 4.58


National Science Teachers Association Web Seminar Evaluations Beyond Penguins and Polar Bears project staff members, Jessica Fries-Gaither (Ohio State Univeristy) and Dr. Carol Landis (Byrd Polar Research Center) presented three web seminars (webinars) through the NSTA website. The NSTA invited members to complete a common evaluation form for all of the web seminars. The evaluation included 4 items on the structure and content of the seminar, four items rating each presenter and four open-response items. The scaled items ranged from poor (1) to excellent (5). Participants received a one-year subscription to an NSTA SciGuide for completing the evaluation. NSTA staff summarized the evaluations and provided presenters with the results. The NSTA-reported results for the three webinars (5/27/08, 10/29/08, 11/13/08) are described below. Full summaries can be found in Appendices J, K, and L respectively. Polar Geography: May 27, 2008

Forty-seven participants attended the Polar Geography webinar presented by Jessica Fries-Gaither and Dr. Carol Landis. Thirty-nine of these participants responded to a survey and rated aspects of the webinar on a scale ranging from poor (1) to excellent (5). Overall, respondents rated the webinar as valuable (4.64), relevant (4.59) and interactive (4.69). The respondents agreed that the web seminar time fit their schedules (4.33). All respondents agreed they would like to see similar webinars offered in the future. The respondents found the content relevant for clarifying background knowledge on the polar regions, preparing weather/climate and food chain units, or using polar geography topics with a group of gifted students. Overall comments included an appreciation of the content and presentation format. The presenters were rated on the same scale as the webinar (poor = 1, excellent = 5) on subject knowledge, clarity, responsiveness, and pace. Neither presenter received an average score lower than 4.69. Respondents commented that the presenters were easy to listen to and knowledgeable. In addition, the respondents expressed appreciation for the presenters’ ability to recommend resources and for continuing the presentation past the allotted time in order to respond to additional questions. Respondents recommended dedicating time to discussing implementation strategies in future webinars. They also suggested adding a slide listing discussed resources for easy reference. For a full summary of the Polar Geography webinar evaluation report, see Appendix J. Physical Science from the Poles: October 29, 2008

Sixty participants attended the Physical Science webinar presented by Jessica Fries-Gaither and Dr. Carol Landis. Forty-five participants responded to the webinar evaluation. Overall, respondents rated this webinar as valuable (4.58), relevant (4.44) and interactive (4.62). Respondents agreed that the webinar time fit their schedules (4.29). All respondents agreed they would like to see similar webinars offered in the future. The respondents commented that the webinar content was valuable and relevant to them, because it covered common misconceptions, clarified global warming and climate change issues, and provided suggestions for hands-on activities that were appropriate for all grade levels. According to open-response questions, respondents appreciated (a) the opportunity to chat with other teachers, (b) the newly discovered webinar format, (c) the dispelling of common misconceptions, and (d) the coverage of science topics for primary grades. Both presenters were rated on knowledge, clarity, responsiveness and pace, and neither received an average score lower than 4.67. Additional comments indicated respondents’ plans to visit the BPPB website and an appreciation for having two presenters. For a full summary of the Physical Science webinar evaluation report, see Appendix K.


Energy and the Polar Environment: November 13, 2008

Forty-one participants attended the Energy and Polar Environment webinar presented by Jessica Fries-Gaither and Dr. Carol Landis. Thirty-four participants completed the webinar evaluation. Overall, respondents rated this webinar as valuable (4.65), relevant (4.59) and interactive (4.62). The respondents agreed that the webinar time fit their schedules (4.38). All respondents agreed that they would like to see similar webinars offered in the future. Comments on the content of the webinar included an appreciation for content suited for differentiated learning, polar regions content that could be used for enrichment, and material that addresses state assessment content. In addition, one participant was thankful for the use of literacy resources for a range of reading abilities. Overall, webinar comments reflected on the ability to use the content across many disciplines, including chemistry, physics, geography, mathematics and literacy, and an appreciation for the presentation of student misconceptions. Both presenters were rated on knowledge, clarity, responsiveness and pace, and neither received an average score lower than 4.82. Additional comments from the respondents included an appreciation for the blend of content and resources used, the ease of following along, and the knowledge and engagement of the presenters. For a full summary of the Energy and the Polar Environment webinar evaluation report, see Appendix L. National Science Digital Library Brown Bag Series: Web Seminar Evaluations Project director, Jessica Fries-Gaither, presented three webinar sessions (1/13/08, 1/22/09, and 4/7/09) as part of the NSDL Brown Bag Webinar Series. NSDL provided a common online feedback questionnaire to evaluate the session and the presenter. The questionnaire included one item on a scale ranging from poor (1) to excellent (5) and four open-response items. The three evaluations were summarized by NSDL staff and provided to the presenter. Below is a synopsis of the NSDL summaries with full versions included in Appendices M, N, and O. States of Matter: January 13, 2008

Nine participants attended the States of Matter webinar presented by Jessica Fries-Gaither. Six of the nine completed the online questionnaire. Overall, session content was rated by respondents as good or excellent (4.4). Reasons given for attending the session included knowledge attainment, material acquisition, and lesson improvement. Respondents cited they liked the format and presentation of the session, the amount of information provided (“not too much”), and the technology-based lesson ideas that were given. Respondents expressed concern about too much time on the phone and the lack of a “chat feature.” Respondents appreciated being introduced to background material, content clips, the BPPB website, and the cited resources and activities. One respondent appreciated the chance to see that other teachers struggled with similar issues. For a full summary of the States of Matter webinar evaluation report, see Appendix M. Rocks and Minerals: January 22, 2009

Six participants attended the Rocks and Minerals webinar presented by Ms. Fries-Gaither. Five of the six completed the online questionnaire. Overall, respondents rated the session content as excellent (5.0). Reasons provided for attending the session included resource acquisition, science and literacy integration help, and finding new ways to teach the topic. One respondent was attending an NSDL webinar for a second time to gain more knowledge about the BPPB website. Respondents stated they appreciated the opportunity to share ideas and talk with other teachers. Respondents also commented positively on the format of and opportunity to learn about the


BPPB website/electronic magazine. Respondents appreciated being introduced to multiple resources. For a full summary of the Rocks and Minerals webinar evaluation report, see Appendix N. Life in the Polar Extremes: April 7, 2009

Ten participants attended the Life in the Polar Extremes webinar presented by Ms. Fries-Gaither. Seven of the ten completed the online questionnaire. Overall, respondents rated the session content as good or excellent (4.3). Reasons provided for attending the session included resource acquisition for a unit currently being taught, and the quality of the presenter. Respondents stated they appreciated the resources and ideas provided. One respondent expressed confusion stating, “I really didn’t know what I was getting into and I feel like a lot of teachers did not sign up for this because no one really understood what this was about or what you had to do while listening.” Additional comments included three suggestions for future topics (e.g., weather, plants). For a full summary of the Life in the Polar Extremes webinar evaluation report, see Appendix O. Polar Adventure Weekend at the Center of Science and Industry (COSI) The Polar Adventure weekend was held at COSI February 27 through March 1, 2009. BPPB team members collaborated with COSI to organize the weekend events, which included information and exhibits about polar science. Participating in the event were scientists and researchers from the Byrd Polar Research Institute, Miami University, the Center for Automotive Research, and Valley Road Outfitters. COSI evaluated the weekend based upon attendance, revenue, and operational impact. Audience members were given Entry and Exit Interview Questionnaires to evaluate their experience. The COSI evaluation report is summarized here and included in Appendix P. The Entrance and Exit Questionnaires can be found in Appendix Q. Exhibits included an Arctic tent, polar gear, polar rocks, an ice core, ice core drill bits, and ice core transportation boxes. Miami University provided specimens of flora and fauna from Antarctica and Valley Road Outfitters displayed a team of sled dogs. Professors and graduate students from the Byrd Polar Research Institute presented elements of their research pertinent to the Polar Adventure Weekend theme and appropriate for the target audience. During the weekend event, COSI attendance increased 18% compared to the same weekend in 2008. COSI member attendance increased almost 40% on this weekend compared to 2008. The Polar Adventure weekend was advertised in the Columbus Dispatch and in the COSI Member Newsletter. Questionnaires were completed by 352 event attendees. Almost 39% of entry questionnaire respondents and 29% of exit questionnaire respondents indicated they were COSI members. More than half of entry (54%) and exit (61%) respondents attended with children in the target age range for the event (ages 5-12). Respondents indicated quality time spent with family/friends as their reason for visiting COSI on this weekend. Fun learning outside of the classroom also was noted as a reason for attending.


Continuing Activities

Year 3 evaluation activities will combine qualitative and quantitative data collection to assess further progress of the BPPB project toward goals. These activities will focus primarily on the collection and analysis of impact data. Pre- and post-questionnaire data will be collected from teachers recruited to field-test the BPPB deliverables. Pre- and post-questionnaire data also will be collected from students that are taught by field-test teachers. Protocol-based classroom observations and articifacts of teacher and student learning also will be collected and analyzed. Focus groups will be interviewed to assess usefulness of dissemination efforts. Webinar and conference evaluation data and webmetrics reports will continue to be collected and analyzed.


Summary

Upon review of current project data, formative findings for the Beyond Penguins and Polar Bears (BPPB) project are as follows: Website

• The website is both aesthetic and easy to use for the intended audience. • The website provides accurate and audience appropriate deliverables. • The website has increased traffic through dissemination activities and search engine

optimization.

Impact

• Grade 3 students have changed their opinion of science as 1) a subject that is both for girls and boys, 2) a subject that they would continue to pursue and 3) more than just memorizing facts. After exposure to the program deliverables, students were more likely to disagree with all of these statements.

• Grade 3 students, after exposure to the program deliverables, were more likely to agree with the statement that writing is important in science.

• Grade K students, after exposure to the program deliverables, were more likely to disagree with the statements that almost all people use science in their jobs and reading is important in science.

Dissemination

• BPPB staff members continue to disseminate program materials and information that have been found to be useful to the target audience.

• Webinars are appreciated for content coverage, use of misconceptions as a focal point, and resource information.

• Webinars are successfully piquing the interest of participants in the BPPB website. The following observations are offered regarding progress toward BPPB project goals: Goal 1: Providing context to online resources by creating, identifying, selecting, and adapting quality learning resources from the National Science Digital Library, the Ohio Resource Center, other IPY-funded projects, and additional high-quality content providers.

According to expert reviewers and website visitors, quality online resources are accessible to the target audience. As of May 8, 2009, 14 issues of the online magazine have been published to the BPPB website. Goal 2: Modifying and building communication, production, and cyberinfrastructure tools to amplify resource discovery and access to resources, increase the ease of reuse and repurposing of content, decrease production times, and increase automated dissemination of IPY materials to various audiences.


Webmetrics data and expert reviews indicate that BPPB website tools and resources are accessible to and appropriate for the target audience. Webmetric trend data suggest that visitors return to the site repeatedly and the number of new visitors is increasing monthly. Webmetric data indicate that a great majority of visitors are finding Beyond Penguins and Polar Bears through search engines indicating that visitors are previously unfamiliar with the website. Keyword searches include terms associated with teaching and learning indicating that teaching resources are a noteworthy purpose for the largest source of visits. Goal 3: Disseminating deliverables through presentations, publications, digital libraries, and push technologies. Dissemination activities are ongoing and continue to reach members of the target audience. As of May 2009, BPPB staff members have presented at 15 web seminar events and 14 conferences or meetings. BPPB promotional material was distributed at 16 educational or science events. Questionnaire data indicates that dissemination is effective at communicating information about the project and its goals. Goal 4: Evaluating the impact of the project deliverables on K-5 teachers and students.

Impact cannot be measured accurately until a sufficient number of field-test teachers can be recruited that are representative of a broader sample. Year 2 pre- and post-questionnaire data show little change in content knowledge and teaching practices for both field-test and non-field-test teachers. It should be noted, however, that classroom observations and pre-questionnaire data suggest that a ceiling effect may explain the lack of change seen on post-questionnaire data. Field-test observations and journal data also indicated a high level of reformed-teaching strategies were part of the teacher repertoires prior to exposure to project materials. Improvement was noted in the Grade 3 teacher content knowledge and comfort teaching the concepts. Low instrument reliabilities for the student questionnaire must be addressed in order to produce any robust findings regarding changes in students’ views of science and literacy. Findings of statistical significance in Year 2 are tentative based upon these instrument reliability issues, but some evidence suggests that Grade 3 students’ attitudes toward science and literacy were improved after exposure to BPPB materials.


Recommendations

Evaluation team members offer the following recommendations for the Beyond Penguins and Polar Bears (BPPB) project team: Website

• Focus group comments suggested redesigning the website homepage to facilitate resource discovery by making the content of the entire website more visible from the homepage. The project team has completed this task. The project team should consider assessing the new homepage with a follow-up focus group.

• Continue to follow webmetric trends and pursue opportunities to increase traffic via direct and referring sites.

Impact

• Modify problematic questionnaire items (item 3 and item 5) of the Student Questionnaire to increase reliability of the instrument.

• Consider the use of cognitive labs1 to design and test items for future student instruments.

• Develop a more precise and valid measure of teacher content to better reflect the diversity of content available on the website.

Dissemination

• Use dissemination efforts to target and recruit more participant teachers to improve the likelihood of measuring the impact of the deliverables on K-5 educators and their students.

• Continue and, if feasible, increase the frequency of webinar sessions that have been found useful to the target audience.

1 “A cognitive lab is a method of studying the mental processes one uses when completing a task such as solving a mathematics problem or interpreting a passage of text.” (Zucker, Sassman & Case, 2004, p.2)


References

Ausburn, L. J. (2001) Learning on the Internet: Balancing access and quality. Oklahoma Association for Supervision and Curriculum Development Journal.

Oklahoma State University (n.d.) Website evaluation form. Retrieved May 5, 2009, from:

http://fp.okstate.edu/efolio/kasi/web_eval_form.htm Online Evaluation Resource Library (OERL). (n.d.). 1994-1995 Evaluation survey. Retrieved

September 1, 2008, from: http://oerl.sri.com/instruments/te/teachsurv/instr50.html Kahle, J. B. (2006). The mathematics science partnership in New York City (MSPinNYC), City

University of New York – years 1 & 2. Oxford, OH: Miami University, Ohio’s Evaluation & Assessment Center for Mathematics and Science Education.

Kahle, J. B., & Rogg, S. R. (1996). Bridging the gap: Equity in systemic reform. (NSF Grant No.

REC 9602137), Washington, DC. Kahle, J. B. & Scantlebury, K. C. (2006). Evaluation of University of Pennsylvania Science Teacher

Institute – 2005-06. Oxford, OH: Miami University, Ohio’s Evaluation & Assessment Center for Mathematics and Science Education.

Wainright, C., Morrell, P.D., Flick, L., & Schepige, A. (2004) Observation of reform teaching in

undergraduate level mathematics and science courses. Social Science and Mathematics. 104(7), 322-335.

Wiess, I. R., Matti, M. C., & Smith, P. S. (1994). Report of the 1993 national survey of science

and mathematics education. Chapel Hill, NC: Horizon Research, Inc. Zucker, S., Sassman, C., & Case, B.J. (2004). Cognitive labs. Retrieved March, 2009, from

http://www.pearsonassess.com


Appendices

Appendix A. Expert Review Summaries ................................................................................. 37

Appendix B. Website Evaluation Form ................................................................................... 38

Appendix C. Full Evaluation Document Summary ................................................................... 39

Appendix D. Teacher Pre-Questionnaire ................................................................................ 49

Appendix E. Teacher Post-Questionnaire ............................................................................... 54

Appendix F. Student Questionnaire ....................................................................................... 60

Appendix G. Teacher Questionnaire Frequencies (pre-questionnaire)....................................... 62

Appendix H. Teacher Questionnaire Frequencies (post-questionnaire) ..................................... 72

Appendix I. OCEPT – Teaching Observation Protocol (O-TOP)................................................. 79

Appendix J. NSTA Web Seminar Evaluation: Polar Geography ................................................. 81

Appendix K. NSTA Web Seminar Evaluation: Physical Science ................................................. 87

Appendix L. NSTA Web Seminar Evaluation: Energy and The Polar Environment ...................... 93

Appendix M. NSDL Web Seminar Evaluation: States of Matter................................................. 99

Appendix N. NSDL Web Seminar Evaluation: Rocks and Minerals .......................................... 101

Appendix O. NSDL Web Seminar Evaluation: Life in the Polar Extremes ................................. 103

Appendix P. COSI Evaluation Report ................................................................................... 105

Appendix Q. COSI Entry/Exit Questionnaires........................................................................ 111

Beyond Penguins and Polar Bears: Integrating Literacy and IPY into K-5 Classrooms

Summary of Review of Online Magazine

Strengths: 1. The magazine provides a theme related to polar science that is different

between issues. 2. The materials and activities are easily accessible for teachers via the

Internet. 3. The materials are consistent with theory and research in regards to

integrating literacy and content within a teacher’s curriculum and instruction.

4. The online magazine provides a great dialogue in how integrating science-literacy instruction can enhance teaching and learning.

5. Issues 1-3 provided not only the background of each science topic but also how science concepts are connected to other concepts.

6. The science concepts are explained in simple term allowing a reader with little content knowledge to understand the concept.

7. The materials presented in the articles exceed expectations relative to the goals of the magazine. More specifically, the articles: • address vocabulary and its role in learning about polar science. • address issues of equity and differentiation of instruction for all learners,

including English language learners. • offer a variety of ancillary resources (e.g., books, websites, etc.) aimed

at supporting teachers in integrating the teaching of science and literacy.

8. The magazine articles are short and concise—providing an easy read for online use.

9. Each article provides supplemental links to educational information.

Areas of concern: 1. The inquiry-based lessons and student activities are didactic and require

minimal student investigation. 2. The teacher’s likelihood of using the materials depends on their own

comfort level related to the concept, pedagogical knowledge, and experience.

3. Consider tweaking the vocabulary so that the writing style and tone of the articles speak more directly to the targeted K-5 teacher audience.


WebsiteEvaluationFormInstructions:Selectthemostappropriatenumberforeachdescriptionofthewebsite.1=No;2=somewhat;3=Yes.Addallthenumberstogetherforatotalscore.Scoreinterpretation:52‐66isgood;37‐51isokay;22‐36isnotgood.

Part1:QualityandSourceofInformation 1 2 3

Accuracy:Theinformationpresentedatthewebsiteiscorrectanderrorfree Authority:Thesitewriter/presenterisqualifiedandcredible. Theinformationsourceisreputableandbelievable. Objectivity:Theinformationisfairandunbiased(freeofadvertisingand“soapboxing”). Currency:Theinformationistimelyandup‐to‐date. Coverage:Thenumberoftopicsanddepthofinformationdetailsarerightformeandmyneedforthisinformation.

Part2:TechnicalQuality 1 2 3Suitability:Theinformationanditstechnicalpresentationareappropriateforme. Browsability:Thesitehasgoodnavigationtoolsanddesign.IcaneasilyfindwhereIamandgetto

Thereasitemapavailable. Searchability:Thereisabuilt‐insearchengineorutilitythatworkseasilyandaccurately. Interactivity:Icandomoreatthissitethanjustreadinformation. Theactivitiesareuseful,interesting,andpurposeful. Theactivitiesworkeasilyandcorrectly. Accessibility:Thissitecanbeaccessedwithoutspecialoradvancedhardware. Thissitecanbeaccessedwithoutdownloadingsoftwareorbrowserplug‐ins.

Part3:MultimediaComponentsandFeatures Purpose:Themediaismorethan“clickandtwitch”or“blendermedia”(turnitonandwatchitspin).

Appropriateness:Themediacomplementsorcontributestothecontentratherthandistractingfromitorcompetingwithit.

Eyeappeal:Themediademonstrategooduseofcolor,layout,anddesign. Userfriendliness:Themediaiseasytouseorplay. ThemedialoadsandplaysquicklyenoughthatIdidnotloseinterest. Balanceandharmony:Themediahasaunifiedstylethatfitsthegenerallookandfeelofthewebsite.

Thedesignerhasusedjusttherightamountofmedia‐nottoomuchandnottoolittle. TotalScore=_______

Ausburn,LynnaJ.LearningontheInternet:Balancingaccessandquality.OklahomaAssociationforSupervisionand

CurriculumDevelopmentJournal,Fall2001.


EVALUATION OF HTTP://BEYONDPENGUINS.NSDL.ORG/

Lead Evaluator: Kouider Mokhtari, Ph.D. With three anonymous BPPB web site reviewers

Iowa State University [email protected]

(515) 294‐9138

In this report, I provide a summary evaluation based on a review of the BPPB website, which was completed between March 27 and April 10, 2009. In completing this evaluation, I focused almost exclusively on the quality of the website: its look and feel, its usability, and appeal to potential users, which include classroom teachers and other education professionals.

In this summary evaluation, I relied a great deal on input obtained from three

classroom teachers whom I recruited specifically to assist me in completing the website review. All three reviewers (a) have a strong interest and experience in both literacy and science education, (b) have classroom teaching experience, and (c) are potential users of the BPPB website. The following is a brief biographical sketch for each of the three anonymous reviewers:

1. Reviewer #1 is currently a Program Assistant at Iowa State University for a federally

funded science and literacy research project. She holds a Bachelor of Arts degree in elementary education with a reading endorsement. She has five years of classroom teaching experience in kindergarten and first grade.

1. Reviewer #2 is currently a graduate assistant for the Iowa Science Literacy Project

at Iowa State University. Prior to joining ISU to pursue her Doctoral degree in Science and Literacy, she was an elementary school teacher and educator at the Science Center of Iowa.

2. Reviewer # 3 is currently a first grade teacher in Marshalltown, IA. She graduated

from Northwestern College in Orange City, IA, with a bachelor's degree in elementary Education and is presently completing a master's degree in literacy education at Iowa State University.

For purposes of this evaluation, I instructed each of the three reviewers to

conduct independent reviews of the BPPB website focusing on the quality on the website in terms of its look and feel, usability, and appeal to potential teachers and other educators. I asked each reviewer to complete the review and provide me with written comments and suggestions relative to their overall as well as specific reactions to the website. Although I did not provide them with specific guidelines for completing the reviews, I asked them to provide a rating of the website using a Website Evaluation Form (Ausburn, 2001), which is attached.


For purposes of this evaluation, I asked reviewers to complete the reviews within a 2‐week period of time. During that evaluation period, I asked them to spend at least two hours browsing the website; however, I encouraged them to spend as much time as needed in order to provide a thorough review of the website. I further asked them to record the amount of time they spent on their respective evaluations so as to be compensated for their evaluation time and effort. All three reviewers competed their reviews and turned them back in to me, via e‐mail, within the allotted time frame. Following the reviews, I had a chance to visit about each reviewer individually to discuss their findings.

In this report, I will provide a summary of the evaluation highlights pertaining to

the overall quality of the website: its look and feel, its usability, and appeal to potential site users. These highlights are provided in two related sections. In the first section of the report, I will share the reviewers’ ratings of the website with particular emphasis on (a) the quality of information, (b) technical quality, and (c) media components and features. In the second section of the report, I will summarize the written and oral feedback obtained from all reviewers, including myself, relative to qualitative comments and suggestions regarding the website’s overall look and feel, usability, and appeal. These highlights are provided along with reviewers’ verbatim comments and suggestions, which I hope you will find constructive and helpful as you take steps in making the site available to potential users. Website Rating Results

Table 1 presents the ratings of three reviewers relative to the overall quality of the website. As the results indicate, reviewers strongly agree (100% agreement) that the website is first‐rate with respect to the quality of information available, the site’s technical quality, and the media components and features. Reviewers gave high individual ratings to each of the categories (Overall M= 59.3 out of 66). These superior ratings speak very highly about the website, and are quite consistent with reviewers’ oral and written comments about the overall quality of the website.

Table 1: Website Rating Results1 Website Evaluation Criteria Mean Scores

Part 1: Quality & Source of Information

17.3 (18)

Part 2: Technical Quality

23 (27)

Part 3: Multimedia Components & Features

19 (21)

Total 59.3 (66) Website Evaluation Overall Average Score is (62+58+58)/3=59.3 out of 66 Score Interpretation: 52‐66 is Good; 37‐51 is Okay; 22‐36 is Not Good

1 Ausburn, L. J. (2001). Learning on the Internet: Balancing access and quality. OASCD Journal, 11(1), 45‐48. (See form attached).


Overall Website Evaluation Results Reviewers were free to organize their website reviews in any way they saw fit. As you

will note from the written comments below, two of the reviewers organized their comments and suggestions around the menu items provided in the website (e.g., contributors, stories for students, podcasts, etc.). One of the reviewers used the categories provided in the website evaluation rubric as a guide for organizing her comments and suggestions.

Strengths

In general, the qualitative feedback obtained from all reviewers was remarkably positive and complementary with respect to the overall quality of the website. The following written comments, triangulated via individual visits with each of the reviewers following their reviews, provide only a sampling of these highly positive reviews.

1. The site is “very pleasing to the eye and fairly simple to navigate”

2. “I really liked this section [the ‘Browse by Column’ section]. It was really easy to use and was organized well. The lessons were nice to see. I also liked the sections on books to use for the classroom.”

3. “The blog feature was fun.”

4. “I am extremely impressed with the electronic books, especially for the primary

learners.”

5. The website is “beautifully designed, contains an enormous amount of helpful, practical information and is very well written. Teachers will enjoy using this site and will find it to be a great resource.”

6. “I love the feature story each month, especially the eBook version.”

7. “I was never overwhelmed or “lost” on Beyond Penguins.”

8. “Overall, I think that the site has a lot to offer both teachers and students…Thank

you for all of your work! I will definitely be passing on the name of the site to the teachers that I work with here.”

Minor Concerns

Reviewers noted that they experienced some difficulty with two of the site features. First, they reported difficulty using the search engine. Second, they reported being somewhat confused about (a) the intent of the “Change Language” feature of the website and (b) the distinct nature of the BPPB website, which is an “online magazine” rather than simply a regular website. See specific comments pertaining to each of these minor concerns in the attached reviews.


Suggestions While reviewers gave the website high marks on several grounds, they offered the

following suggestions, which are designed to enhance its look and feel, its appeal, and its usability.

1. Place the “contributors” section at the end of the site, not at the beginning. 2. Link the Adobe files and electronic books opening to a new window. 3. Provide black line copies of illustrated books for those schools that may not have

color printers available 4. Consider placing a word label on the paw prints where the kids should click to keep

reading or go back. 5. Specify what subscribers will get and also providing assurances that email

addresses will not be shared or sold. 6. Clarify the role or purpose of the ‘Change Language’ feature. 7. Highlight the fact that BPPB is an ‘Online Magazine’ not just a regular website.

In Summary, it is evident to all reviewers that an enormous amount of time and effort has been devoted to the planning and development of the BPPB website. The website has a great deal to offer to its intended audience. Collectively, we believe it is “beautifully designed, contains an enormous amount of helpful, practical information and is very well written. Teachers will enjoy using this site and will find it to be a great resource.” REVIEWERS’ VERBATIM WRITTEN COMMENTS AND SUGGESTIONS REVIEWER # 1: I found this website to contain a huge amount of valuable information and resources. It’s very pleasing to the eye and fairly simple to navigate. I have organized my evaluation comments by the menu items across the top of the homepage. In some cases I had a lot to say and in others I had just a few comments. Suggestions specific to each section is highlighted in bold. At the end I have given my overall impressions and some suggestions to improve the site overall. Contributors This section is a nice touch. It is good to see the credentials of who is behind the creation of the website. I would, however, put this section at the end of the menu instead of the beginning since it is not the focus of your website. Stories for Students I sampled several of these, but I didn’t read each and every entry. They all seem to be presented in a very consistent, predictable manner. This is helpful to teachers so they know what to expect when they use this section of the website. The descriptions of how the media is available and how to assemble the books are very practical and helpful. I appreciate how these are at the very beginning of the section so you have an idea of what your end result will be when you click on the links. It piqued my interest to start clicking.


One thing I would like to see is the links to the Adobe files and electronic books opening to a new window. That way you don’t have to click on the “back” button to get back to the page and you could feasibly open several Adobe files at once and compare them side by side to make your decision as to which file to use (the text, the illustrated book or the electronic book). The descriptions leading into the stories are well written and succinct. They give a good, quick summary of what’s ahead. The illustrated books are eye catching and appear to be easy to assemble. I wonder if black line copy could be provided for those schools that won’t have color printers available. Sometimes colored originals don’t reproduce as well as a backline copy when using just a black and white copier or printer. The text of the stories is very engaging and the vocabulary is rich. It does not appear that the writers have tried to water down the content, yet the text appears to be on grade level as far as decoding/word identification and comprehension expectations are concerned. I don’t have a lot of science education background, but the content appears to be accurate and relevant to the grade levels. I’ll defer to the science educator evaluators for their evaluation of the science content accuracy. I am extremely impressed the electronic books, especially for the primary learners. My only suggestion is to put a word label on the paw prints where the kids should click to keep reading or go back. The electronic books are very engaging and give a child a chance to really soak in the content if they want or move quickly through the text. The use of the play button also gives them a chance to read the text first if they choose. Browse Columns

Professional Learning This section contains a large quantity and wide variety of good quality information and resources to help teachers feel more comfortable in teaching in the topic areas. I especially like how the national standards are referenced. There were several times when I clicked on something from a menu expecting either another menu or to go straight to the item named on the link, but instead I got a “menu” of just that one item. If there is only one item in a menu, then a user should only have to click there once to get to it. One example is the Professional Bookshelf in the Browse Columns/Professional Learning section. On the main page of the Browse Columns section, there is a list of items under Professional Learning. Most of them take me to another menu, but the Professional Bookshelf item opens to another “menu” of just one item, also called the Professional Bookshelf. The same is true in this section for Planning a Polar Festival. Internet users want to get to the information they are seeking in as few clicks as possible, so I would take out that “middleman” menus. Of course, this suggestion is null and void if the web developers plan on adding items to those menus that currently only have one item.

Science and Literacy I thought this section had a very useful variety of literature and activities to add to the teachers’ repertoires. The Virtual Bookshelf will serve as a time saver for


teachers looking for recommended books on the subjects. I’m assuming that the Feature Stories are placed here for convenience since they are already available through the Stories for Students section of the website. This is a good use of repetition so teachers don’t have to click back and forth to find all the resources they want. Lastly, I think the Student and Teacher Work section has a lot of potential. As the site grows, I’m assuming teachers will submit items for this section which will be a great resource for ideas.

In the Field: Scientists at Work I’m not sure who the audience is for this section. I’m assuming it’s for teachers, who will in turn share it with students. The information is interesting, but if it’s intended for classroom use, some clarifying text and/or ideas about how to use it in the classroom would probably be helpful. If it’s simply for the education of the teachers, I would include it in the Professional Learning Section instead.

Across the Curriculum This section has a lot of good information, resources and ideas as well.

Polar News and Notes This section is a great resource to use to make the most current information available to teachers and students alike.

Archive This section is self‐explanatory and well designed. Podcasts The use of podcasts is very current and useful. I am wrestling with whether this should be it’s own section of the website, or whether it be part of the other resources in the Browse Columns section. Change Language This is an interesting option to include on the site. Since it simply translates the words on the site and doesn’t actually translate the Stories for Children or offer books in the Virtual Bookshelves actually written in that language, I’m not sure how valuable it is. Teachers who use this website are likely English speaking. The translator function would be helpful to ELL parents and students, but for teachers it would only be helpful if the resources were offered in different languages. I am not, however, suggesting that these items should be offered in multiple languages. My point is that this function may be unnecessary. Blog I think the blog is a very good idea to keep teachers up‐to‐date and further expand their knowledge on specific topics. It, like the podcasts, is a very current way to reach teachers. Email Updates This appears to be a great resource for teachers as well. However, it’s not clear what the email updates will contain. Will they simply be a message that another issue has been


added to the website or will it be a separate resource complementing the website? I suggest specifying what subscribers will get and also providing assurances that email addresses will not be shared or sold. Book Club This appears to be a great resource, but I didn’t sign up to be a member so I’m not sure what the benefits would be. Perhaps a line or two of text explaining this to potential members would be beneficial. Overall Comments I accessed this site using a satellite Internet connection which is slower than a regular high‐speed connection. The site isn’t heavy‐laden with bells and whistles, which really helps with slower connection. The larger files did take a while to download, but that’s unavoidable when you have graphics and illustrations. The only software needed was Adobe Reader, which is a very basic piece of software that most users have. If a user doesn’t have Adobe Reader, they can get it for free with very little technological knowledge. The videos in the professional learning section did take a while to load on my connection. But I could look at other resources while it downloaded, especially since I opened it in a new window myself to allow myself the ability to browse through the rest of the resources. All of these technical factors left me with a favorable impression of the site. I personally prefer links to open in a new window, especially links taking me away from the original website or opening files like Adobe or a video. That way I don’t have to remember how to get back to where I was before I clicked the link and I can toggle back and forth between resources easily. I think this would make it easier for busy teachers to use the valuable resources this site provides. The use of a wide variety of media is very beneficial and up‐to‐date with current technology. Teachers and students won’t have a chance to get bored because of the variety of ways information is presented and made available. My biggest “complaint” in navigating the site was this: if someone were to come upon your website and not realize it was set up as a magazine they may not realize all the treasure available here. I know it says “online magazine” on the home page, but people surfing the Internet aren’t necessarily going to notice this. For instance, I was busy looking through your Browse Columns section and wondered why it was called that. I was thinking how great it would be if all these resources were made available in a one‐stop‐shop type of page where I could get information just on the topic I was teaching. That, in my mind, would put this website over the top as far as excellence goes. Then I revisited the front page and found that you had done just that. The following suggestion is based on the experience I just described and assuming that I am an internet user of average intelligence who is in a hurry to find information, bookmark a good page and get on to the task of planning a unit. My suggestion is to retool the main menu to have an item named “View Resources by Topic” and another one called “View Resources by Type” or something along those lines. For example, if I’m a teacher looking


Browse By Column I really liked this section. It was really easy to use and was organized well. The lessons were nice to see. I also liked the sections on books to use for the classroom. Archive The archive was also well organized and easy to use. Change Language A very important tool for the site! It was nice to see that it was included. Consider moving the word language up next to the word change. It could be that because my screen is smaller it was moved down. Blog This feature was fun. It was really nice to see that it is being updated consistently. That will help keep people coming back! Book Club This looks really interesting. I could see a class really engaging with it. Podcasts I think that these are great tools for teachers and students. It is nice that the information is offered in multiple formats. Overall Impression Overall, I think that the site has a lot to offer both teachers and students. There are a few things that I would change (as mentioned above) but was pleased with the information and organization. The key will be getting the name of the site out there to increase your traffic. Thank you for all of your work! I will definitely be passing on the name of the site to the teachers that I work with here. REVIEWER # 3: Part 1: Quality and Source of Information I’m not a science expert, but all the information on the website seems well researched and fact based. I also think the science content knowledge piece is great at giving information on the monthly topic. The related information is also very useful to go deeper into the topic and help teachers understand the sometimes‐complex topic. I would feel ready to teach this to children because I was given lots of information and places to go if I need more information to answer questions my students might have. Part 2: Technical Quality At first, I thought this website wouldn’t be useful to me. I teach 1st grade and we don’t have a science time during the day. I just teach science through my basal stories and


for information on Rocks and Minerals, I would go to “View Resources by Topic” and select “Rocks and Minerals,” which would bring me to the section you’ve already beautifully designed as your Rocks and Minerals issue and start collecting information for my unit. On the other hand, if I’m a teacher looking for ideas about how to teach my science units engaging in cross‐curriculum instruction, I would go to “View Resources by Type,” which would bring me to your page currently named Browse Columns. From there I would select “Across the Curriculum” and read through the resources there to get ideas. In my opinion, this is the only liability of the site. It’s beautifully designed, contains an enormous amount of helpful, practical information and is very well written. Teachers will enjoy using this site and will find it to be a great resource. REVIEWER # 2: Initial Comments The layout of homepage looks good. It appears to be easy to find items. The pictures are wonderful! However, it might be nice to have a title below each picture to help users get to specific items faster. The colors are catchy. Searching Comments I typed in polar bears because I know many students are interested in them. The search feature was a little tricky to navigate. It took me to Issue 13 but then I clicked on Issue 13 and nothing came up. I had a hard time getting to information about polar bears. To be honest, I actually got a little frustrated and gave up. I will try another search. This time I typed in literacy. This search went much better. It was nice to see books from the NSTA highlighted. The page that came up was easy to read and informative. Contributors It was nice to see the variety of professions involved in the site. I was little surprised to see that there were not very many scientists. Stories for Students The books at different levels are a great idea. I think they will be much appreciated by teachers and students. Is there a way to make the books so that they print out ready to go though? By this I mean, I printed one of the books and assembled it. However, as mentioned in your assembling directions, the pages needed to be glued together to make it into a “book”. I then went back and saw that I could try to print the story by copying single pages into double. This got a little tricky and I was not able to get my printer to do it (it is old). The books are very interesting. Please consider making that change so that teachers would be able to print and use, and use less paper for them. Thanks!


fit it in whenever possible. However, the more I looked at different articles I realized any teacher can benefit from the articles and material on this website. Help with teaching writing research reports to how to get girls more interested in science are subjects any teacher could utilize. Part 3: Multimedia Components and Features I love the feature story each month, especially the eBook version. On the home page, one thing I was wondering that could be changed was the words “change language” and “join our book club”. They didn’t fit on the same line and the misalignment didn’t look very good. I looked at other online magazines to compare multimedia features and Beyond Penguins definitely isn’t as fancy or have as many features as other online magazines. That was okay with me, though, because sometimes if you get too much stuff it can be overwhelming and I was never overwhelmed or “lost” on Beyond Penguins.


Beyond Penguins and Polar Bears Teacher Questionnaire Pre

0 3 6 0 1 8

PARTICIPANT INFORMATION

Instructions: Please use a dark pen or pencil. Please provide answers that best represent your situation. We request the following information so that we can match this questionnaire with one you may be asked to complete in the future. Your responses will be completely confidential. No identifying information will be used in any report or paper.

1. The first letter of my FIRST name is: Example: My first name is Chris Answer here:

2. The first letter of my LAST name is: Example: My last name is Smith Answer here:

3. My date of birth is:

Example: Answer here:

Month Day Year Month Day Year

4. During this school year, I am teaching grade(s): (Please circle all that apply.)

K 1 2 3 4 5 6 7 8 9 10 11 12

5. I am certified/licensed to teach grade(s): (Please circle all that apply.)

K 1 2 3 4 5 6 7 8 9 10 11 12

6. I have been employed as a teacher for: (Please select one.)

0-1 year 6-10 years Over 15 years

2-5 years 11-15 years

7. My highest degree earned is: (Please select one.)

Bachelor’s Masters + 30 credit hours or more

Masters Doctorate

S

C



2

8. I have a degree in a science-related discipline (e.g. chemistry, biology.)

Yes No

9. Which best describes your professional development experience in science (e.g., workshops, online courses, interactions with science organizations) since receiving your certificate/license? (Please select one.)

I have participated in no science professional development.

I have participated in some science professional development.

I have participated in much science professional development.

10. My gender is: (Please select one.)

Female Male

11. My background is best described as: (Please select one.)

African American Latino/a

American Indian/Alaskan Native White

Asian or Pacific Islander Other



HOW I TEACH Please circle the response that best reflects how often Almost Never each practice occurs in your classroom. Seldom Sometimes Please circle “Not Applicable” if the teaching practice is not Often appropriate for your students’ grade-level. Very Often In my classroom, I ...

Not Applicable

1. arrange seating to facilitate student discussion. AN Se So O VO NA

2. use open-ended questions. AN Se So O VO NA

3. require that my students supply evidence to support their claims. AN Se So O VO NA

4. encourage questions from my students. AN Se So O VO NA

5. allow my students to work at their own pace. AN Se So O VO NA

6. encourage my students to explain concepts to one another. AN Se So O VO NA

7. encourage my students to consider alternative explanations. AN Se So O VO NA

8. discuss the work of scientists. AN Se So O VO NA

9. provide time for my students to discuss subject-specific ideas among themselves.

AN Se So O VO NA

10. discuss experiments from the history of science. AN Se So O VO NA

11. use non-traditional/authentic assessments. AN Se So O VO NA

12. give lectures. AN Se So O VO NA

13. integrate science instruction with literacy by providing opportunities for reading.

AN Se So O VO NA

14. integrate science instruction with literacy by providing opportunities for writing.

AN Se So O VO NA

15. integrate science instruction with literacy by providing opportunities for discussion.

AN Se So O VO NA

16. choose nonfiction, science-themed articles and books for my classroom.

AN Se So O VO NA

17. choose nonfiction articles and books that are not science-themed for my classroom.

AN Se So O VO NA

18. regularly include science texts in my reading instruction. AN Se So O VO NA

19. frame science concepts using grade-appropriate real world applications.

AN Se So O VO NA

Source: Adapted from E&A, CUNY, 2008-09 Instruments



4

WHAT MY STUDENTS DO Please circle the response that best reflects how often Almost Never each practice occurs in your classroom. Seldom Sometimes Please circle “Not Applicable” if the teaching practice is not Often appropriate for your students’ grade-level. Very Often In my classroom, my students ... Not Applicable

1. use data to justify responses to questions. AN Se So O VO NA

2. argue or debate with one another about the interpretation of data.

AN Se So O VO NA

3. repeat experiments to confirm results. AN Se So O VO NA

4. use multiple sources of information to learn. AN Se So O VO NA

5. consider alternative explanations to accepted theories. AN Se So O VO NA

6. design activities to test their own ideas. AN Se So O VO NA

7. consult one another as sources for learning. AN Se So O VO NA

8. talk with one another to promote learning. AN Se So O VO NA

9. use educational technology in class. AN Se So O VO NA

10. develop scientific literacy skills. AN Se So O VO NA

11. learn about real world applications of science. AN Se So O VO NA

12. take notes and listen to lectures. AN Se So O VO NA

13. do worksheets. AN Se So O VO NA

14. learn scientific facts by using worksheets. AN Se So O VO NA

15. memorize scientific facts. AN Se So O VO NA

16. use science as the basis for authentic reading. AN Se So O VO NA

17. use science as the basis for authentic writing. AN Se So O VO NA

18. write to communicate and explain scientific results. AN Se So O VO NA

19. read science-themed, non-fiction in class. AN Se So O VO NA




5

KNOWLEDGE OF POLAR REGIONS Please circle the response that best reflects your agreement with each of the following statements. Disagree

Undecided Agree

1. There is little difference in climate between the Arctic and Antarctica. D U A

2. The Arctic and Antarctica play an important role in regulating global climate. D U A

3. There is no human life in the polar regions. D U A

4. The polar regions are cold and snowy year-round. D U A

5. There are few living organisms in the polar regions. D U A

6. It is always dark in the polar regions. D U A

7. There is little difference in geography between the Arctic and Antarctica.

D U A

8. There has been little physical change to the polar regions in our natural history.

D

U

A

9. Penguins and polar bears live together in the polar regions.

D U A

10. There is little difference in flora and fauna between the Arctic and Antarctica. D U A

11. There is high humidity in the polar regions D U A


Beyond Penguins and Polar Bears Teacher Questionnaire Post

0 3 6 0 1 8

PARTICIPANT INFORMATION

Instructions: Please use a dark pen or pencil. Please provide answers that best represent your situation. We request the following information so that we can match this questionnaire with one you may be asked to complete in the future. Your responses will be completely confidential. No identifying information will be used in any report or paper. 1. The first letter of my FIRST name is: Example: My first name is Chris Answer here:





S

C



2

HOW I TEACH Please circle the response that best reflects how often Almost Never each practice occurs in your classroom. Seldom Sometimes Please circle “Not Applicable” if the teaching practice is not Often appropriate for your students’ grade-level. Very Often In my classroom, I ...

Not Applicable

1. arrange seating to facilitate student discussion. AN Se So O VO NA

2. use open-ended questions. AN Se So O VO NA

3. require that my students supply evidence to support their claims. AN Se So O VO NA

4. encourage questions from my students. AN Se So O VO NA

5. allow my students to work at their own pace. AN Se So O VO NA

6. encourage my students to explain concepts to one another. AN Se So O VO NA

7. encourage my students to consider alternative explanations. AN Se So O VO NA

8. discuss the work of scientists. AN Se So O VO NA

9. provide time for my students to discuss subject-specific ideas among themselves. AN Se So O VO NA

10. discuss experiments from the history of science. AN Se So O VO NA

11. use non-traditional/authentic assessments. AN Se So O VO NA

12. give lectures. AN Se So O VO NA

13. integrate science instruction with literacy by providing opportunities for reading.

AN Se So O VO NA

14. integrate science instruction with literacy by providing opportunities for writing.

AN Se So O VO NA

15. integrate science instruction with literacy by providing opportunities for discussion.

AN Se So O VO NA

16. choose nonfiction, science-themed articles and books for my classroom.

AN Se So O VO NA

17. choose nonfiction articles and books that are not science-themed for my classroom.

AN Se So O VO NA

18. regularly include science texts in my reading instruction. AN Se So O VO NA

19. frame science concepts using grade-appropriate real world applications.

AN Se So O VO NA




WHAT MY STUDENTS DO Please circle the response that best reflects how often Almost Never each practice occurs in your classroom. Seldom Sometimes Please circle “Not Applicable” if the teaching practice is not Often appropriate for your students’ grade-level. Very Often In my classroom, my students ...

Not Applicable

1. use data to justify responses to questions. AN Se So O VO NA

2. argue or debate with one another about the interpretation of data.

AN Se So O VO NA

3. repeat experiments to confirm results. AN Se So O VO NA

4. use multiple sources of information to learn. AN Se So O VO NA

5. consider alternative explanations to accepted theories. AN Se So O VO NA

6. design activities to test their own ideas. AN Se So O VO NA

7. consult one another as sources for learning. AN Se So O VO NA

8. talk with one another to promote learning. AN Se So O VO NA

9. use educational technology in class. AN Se So O VO NA

10. develop scientific literacy skills. AN Se So O VO NA

11. learn about real world applications of science. AN Se So O VO NA

12. take notes and listen to lectures. AN Se So O VO NA

13. do worksheets. AN Se So O VO NA

14. learn scientific facts by using worksheets. AN Se So O VO NA

15. memorize scientific facts. AN Se So O VO NA

16. use science as the basis for authentic reading. AN Se So O VO NA

17. use science as the basis for authentic writing. AN Se So O VO NA

18. write to communicate and explain scientific results. AN Se So O VO NA

19. read science-themed, non-fiction in class. AN Se So O VO NA




4

KNOWLEDGE OF POLAR REGIONS Please circle the response that best reflects your agreement with each of the following statements. Disagree

Undecided Agree

1. There is little difference in climate between the Arctic and Antarctica. D U A

2. The Arctic and Antarctica play an important role in regulating global climate. D U A

3. There is no human life in the polar regions. D U A

4. The polar regions are cold and snowy year-round. D U A

5. There are few living organisms in the polar regions. D U A

6. It is always dark in the polar regions. D U A

7. There is little difference in geography between the Arctic and Antarctica.

D U A

8. There has been little physical change to the polar regions in our natural history.

D

U

A

9. Penguins and polar bears live together in the polar regions.

D U A

10. There is little difference in flora and fauna between the Arctic and Antarctica. D U A

11. There is high humidity in the polar regions D U A



5

INTEGRATION OF BPPB MATERIALS

Please circle the response that best reflects your experience with the Beyond Penguins and Polar Bears Project.

Strongly Disagree

Disagree Undecided Agree As a result of the “Beyond Penguins and Polar Bears” Project, I have…

Strongly Agree

1. incorporated the BPPB content into a specific unit of study. SD D U A SA

2. designed activities for my students that use the objectives of the BPPB project.

SD D U A SA

3. changed the instructional strategies that I use in my classroom. SD D U A SA

4. changed the ways I assess student understanding. SD D U A SA

5. changed the ways I use educational technology. SD D U A SA

6. changed the science curriculum in my classroom. SD D U A SA

7. increased my own science content knowledge. SD D U A SA

8. gained confidence in teaching science to my students.

SD D U A SA

9. learned that the Arctic and Antarctica are important geographic regions.

SD D U A SA

10. become more aware of changes taking place in the polar regions.

SD D U A SA

11. become an advocate for protection of the polar regions. SD D U A SA

Source: Adapted from The Online Evaluation Resource Library (OERL) at http://oerl.sri.com/instruments/te/teachsurv/instr50.html



6

EVIDENCE OF IMPACT ON STUDENTS

Please circle the response that best reflects your observations of your students.

Strongly disagree

Disagree Undecided As a result of the “Beyond Penguins and Polar Bears” Project, . . . Agree Strongly agree

1. my students like science more. SD D U A SA

2. my students believe they are good at science. SD D U A SA

3. my students understand more of what goes on in science. SD D U A SA

4. my students have increased their knowledge of specific science concepts through the BPPB materials.

SD D U A SA

5. my students have learned the BPPB content. SD D U A SA

6. my students’ scientific literacy has increased. SD D U A SA

7. my students’ achievement/performance in science has improved. SD D U A SA

8. my students’ effort in science has improved. SD D U A SA

9. my students’ reading skills in science have improved. SD D U A SA

10. my students’ writing skills in science have improved. SD D U A SA

11. my students’ discussion skills in science have improved. SD D U A SA

Source: Adapted from E&A, Penn MSP, Student Instrument


1

Beyond Penguins and Polar Bears Student Questionnaire

0 3 6 0 7 9

Instructions

DO NOT WRITE YOUR NAME ON THIS TEST. Please use either a blue/black pen or pencil to complete this test. When you answer the questions, please place an X in only ONE box per item. For Example: 1. The first letter of my FIRST name is: Example: My first name is Chris Answer here:





4. What grade are you in? (Please select only one.)

K 1st 2nd 3rd 4th 5th

5. My gender is: (Please select only one.)

Girl Boy

6. What is your age? (Please select only one.)

5 6 7 8 9 10 11 7. My background is: (Please select only one.)

African American Latino/a

American Indian/Alaskan Native White

Asian or Pacific Islander Other

S

C


Beyond Penguins and Polar Bears Student Questionnaire

Science Views Disagree I don’t know Agree

Circle the response that best reflects your views.

1. I like science. 2. I am good at science. 3. If I had a choice, I would not study any more science. 4. I understand most of what goes on in science. 5. Science is more for boys than girls. 6. Learning science is mostly memorizing facts. 7. Almost all people use science in their jobs. 8. Science is useful for solving everyday problems. 9. Reading is important in science. 10. Writing is important in science. 11. Discussion is important in science.

Source: PENN MSP, Instruments, Student_Quest_Science101


Teacher Questionnaire Frequencies (pre-questionnaire)

In my classroom, I… Almost Never Seldom Sometimes Often Very

Often Not

Applicable arrange seating to facilitate student discussion.

6

use open-ended questions.

2 4

require that my students supply evidence to support their claims. 4 2

encourage questions from my students.

1 1 4

allow my students to work at their own pace.

4 2

encourage my students to explain concepts to one another. 1 3 2

encourage my students to consider alternative explanations. 1 3 2

discuss the work of scientists.

2 3 1

provide time for my students to discuss subject-specific ideas among themselves. 1 3 2

discuss experiments from the history of science. 2 4

use non-traditional/authentic assessments.

1 3 2

give lectures.

4 1 1

integrate science instruction with literacy by providing opportunities for reading. 3 3



integrate science instruction with literacy by providing opportunities for writing. 1 2 3

integrate science instruction with literacy by providing opportunities for discussion. 3 3

choose nonfiction, science-themed articles and books for my classroom. 1 1 4

choose nonfiction articles and books that are not science-themed for my classroom. 1 3 2

regularly include science texts in my reading instruction. 1 2 1 2

frame science concepts using grade-appropriate real world applications. 2 2 2



In my classroom, my students… Almost

Never Seldom Sometimes Often Very Often

Not Applicable

use data to justify responses to questions.

3 1 2

argue or debate with one another about the interpretation of data. 1 1 2 2

repeat experiments to confirm results.

1 3 1 1

use multiple sources of information to learn.

1 3 2

consider alternative explanations to accepted theories. 1 2 3

design activities to test their own ideas.

1 2 3

consult one another as sources for learning.

3 2 1

talk with one another to promote learning.

2 1 3

use educational technology in class.

2 3 1

develop scientific literacy skills.

1 4 1

learn about real world applications of science.

3 2 1

take notes and listen to lectures.

1 2 3

do worksheets.

1 3 1




1 3 2

memorize scientific facts.

1 3 2

use science as the basis for authentic reading.

1 1 4

use science as the basis for authentic writing.

1 2 3

write to communicate and explain scientific results. 1 3 2

read science-themed, non-fiction in class.

1 1 3 1



Disagree Undecided Agree

There is little difference in climate between the Arctic and Antarctica.

3 1 2

The Arctic and Antarctica play an important role in regulating global climate. 0 0 6

There is no human life in the polar regions.

6 0 0

The polar regions are cold and snowy year-round.

3 1 2


5 0 1

It is always dark in the polar regions.

6 0 0

There is little difference in geography between the Arctic and Antarctica. 5 0 1

There has been little physical change to the polar regions in our natural history. 6 0 0

Penguins and polar bears live together in the polar regions.

4 0 2

There is little difference in flora and fauna between the Arctic and Antarctica. 4 1 1

There is high humidity in the polar regions

2 2 2



Item Almost

Never/Seldom Sometimes Often/Very Often

Not Applicable

In my classroom, I… (%) (%) (%) (%) n arrange seating to facilitate student

discussion. 100 6


100 6

require that my students supply evidence to support their claims. 100 6


17 83 6

allow my students to work at their own pace. 100 6

encourage my students to explain concepts to one another. 17 83 6

encourage my students to consider alternative explanations. 17 83 6


33 67 6

provide time for my students to discuss subject-specific ideas among

themselves. 17 50 33 6

discuss experiments from the history of science. 33 67 6

use non-traditional/authentic assessments. 17 83 6

give lectures.

67 17 17 6



integrate science instruction with literacy by providing opportunities for

reading. 100 6


writing. 17 83 6


discussion. 100 6

choose nonfiction, science-themed articles and books for my classroom. 17 83 6

choose nonfiction articles and books that are not science-themed for my

classroom. 17 83 6

regularly include science texts in my reading instruction. 17 50 33 6

frame science concepts using grade-appropriate real world applications. 33 67 6



Item Almost

Never/Seldom Sometimes Often/Very Often

Not Applicable

In my classroom, my students… (%) (%) (%) (%) n

use data to justify responses to questions. 50 50 6

argue or debate with one another about the interpretation of data. 33 33 33 6


17 50 33 6

use multiple sources of information to learn. 1 5 6

consider alternative explanations to accepted theories. 50 50 6


17 33 50 6

consult one another as sources for learning. 50 50 6

talk with one another to promote learning. 33 67 6


33 67 6


17 83 6

learn about real world applications of science. 83 17 6


50 50 6



do worksheets.

20 60 20 5

learn scientific facts by using worksheets. 67 33 6


67 33 6

use science as the basis for authentic reading. 17 17 67 6

use science as the basis for authentic writing. 17 33 50 6

write to communicate and explain scientific results. 17 83 6

read science-themed, non-fiction in class. 17 17 67 6



Item Disagree Undecided Agree (%) (%) (%) n

There is little difference in climate between the Arctic and Antarctica. 50 17 33 6

The Arctic and Antarctica play an important role in regulating global climate. 0 0 100 6


100 0 0 6

The polar regions are cold and snowy year-round.

50 17 33 6


83 0 17 6


100 0 0 6

There is little difference in geography between the Arctic and Antarctica. 83 0 17 6

There has been little physical change to the polar regions in our natural history. 100 0 0 6

Penguins and polar bears live together in the polar regions.

67 0 33 6

There is little difference in flora and fauna between the Arctic and Antarctica. 67 17 17 6


33 33 33 6


Teacher Questionnaire Frequencies (post-questionnaire)

In my classroom, I… Almost Never Seldom Sometimes Often Very

Often Not

Applicable arrange seating to facilitate student

discussion. 0 0 0 0 6 0


0 0 1 1 4 0

require that my students supply evidence to support their claims. 0 0 1 4 1 0


0 0 0 2 4 0

allow my students to work at their own pace. 0 0 1 4 1 0

encourage my students to explain concepts to one another. 0 0 1 2 3 0

encourage my students to consider alternative explanations. 0 0 1 3 2 0


0 0 5 1 0 0

provide time for my students to discuss subject specific ideas among themselves. 0 1 2 1 2 0

discuss experiments from the history of science. 1 3 2 0 0 0

use non-traditional/authentic assessments.

0 0 1 4 1 0

give lectures.

2 2 2 0 0 0

integrate science instruction with literacy by providing opportunities for reading. 0 0 0 2 4 0



integrate science instruction with literacy by providing opportunities for writing. 0 0 1 3 2 0

integrate science instruction with literacy by providing opportunities for discussion. 0 0 0 2 4 0

choose nonfiction, science-themed articles and books for my classroom. 0 0 0 2 4 0

choose nonfiction articles and books that are not science-themed for my classroom. 0 0 2 1 3 0

regularly include science texts in my reading instruction. 0 1 1 2 2 0

frame science concepts using grade-appropriate real world applications. 0 0 1 3 2 0



In my classroom, my students… Almost Never Seldom Sometimes Often Very

Often Not

Applicable use data to justify responses to questions.

0 0 2 3 1 0

argue or debate with one another about the interpretation of data. 1 1 3 0 1 0


0 1 2 3 0 0

use multiple sources of information to learn.

0 0 1 4 1 0

consider alternative explanations to accepted theories. 0 1 3 2 0 0


0 1 2 2 1 0

consult one another as sources for learning.

0 1 1 2 2 0

talk with one another to promote learning.

0 0 0 3 3 0


0 0 2 3 1 0


0 0 1 3 2 0

learn about real world applications of science. 0 0 1 4 1 0


2 1 2 0 0 0

do worksheets.

1 1 4 0 0 0




1 3 2 0 0 0


2 2 2 0 0 0

use science as the basis for authentic reading. 0 0 3 3 0 0

use science as the basis for authentic writing. 0 0 3 3 0 0

write to communicate and explain scientific results. 0 0 3 1 2 0

read science‐themed, non‐fiction in class.

0 0 1 4 1 0



Disagree Undecided Agree

There is little difference in climate between the Arctic and Antarctica. 5 0 1

The Arctic and Antarctica play an important role in regulating global climate. 0 0 6


6 0 0

The polar regions are cold and snowy year‐round.

4 0 2


6 0 0


5 0 1

There is little difference in geography between the Arctic and Antarctica. 6 0 0

There has been little physical change to the polar regions in our natural history. 6 0 0

Penguins and polar bears live together in the polar regions. 4 0 2

There is little difference in flora and fauna between the Arctic and Antarctica. 4 2 0


3 2 1



As a result of the “Beyond Penguins and Polar Bears” Project, I have…

Strongly Disagree Disagree Undecided Agree Strongly

Agree incorporated the BPPB content into a specific unit

of study. 0 1 0 3 2

designed activities for my students that use the objectives of the BPPB project. 0 0 0 4 2

changed the instructional strategies that I use in my classroom. 0 1 0 4 1

changed the ways I assess student understanding.

0 3 1 1 1

changed the ways I use educational technology.

0 2 0 3 1

changed the science curriculum in my classroom.

0 1 0 4 1

increased my own science content knowledge.

0 0 0 4 2

gained confidence in teaching science to my students. 0 1 1 2 2

learned that the Arctic and Antarctica are important geographic regions. 0 0 0 4 2

become more aware of changes taking place in the polar regions. 0 0 0 4 2

become an advocate for protection of the polar regions. 0 0 1 3 2



As a result of the “Beyond Penguins and Polar Bears” Project, . . .

Strongly Disagree Disagree Undecided Agree Strongly

Agree my students like science more.

0 0 2 2 2

my students believe they are good at science.

0 0 2 3 1

my students understand more of what goes on in science. 0 0 2 3 1

my students have increased their knowledge of specific science concepts through the BPPB

materials. 0 0 1 3 2

my students have learned the BPPB content.

0 0 1 4 1

my students’ scientific literacy has increased.

0 0 1 4 1

my students’ achievement/performance in science has improved. 0 0 1 5 0

my students’ effort in science has improved.

0 1 1 3 1

my students’ reading skills in science have improved. 0 0 2 4 0

my students’ writing skills in science have improved. 0 0 2 4 0

my students’ discussion skills in science have improved. 0 0 2 3 1




Web Seminar Participant Feedback Results

National Science Digital Library Beyond Penguins and Polar Bears: Integrating Science and

Literacy for the K-5 Classroom, Seminar 1: Polar Geography

May 27, 2008


2

Web Seminar Participant Feedback Results NSDL: Beyond Penguins and Polar Bears: Integrating Science and Literacy for the K-5

Classroom, Seminar 1: Polar Geography May 27, 2008

This report provides the participant feedback results for the Web Seminar titled NSDL: Beyond Penguins and Polar Bears: Integrating Science and Literacy for the K-5 Classroom, Seminar 1: Polar Geography. Recommendations for future iterations of Web Seminar events are provided at the conclusion of the report. WEB SEMINAR SUMMARY....................................................................................................................................3 WEB SEMINAR CONTENT OVERVIEW..............................................................................................................3 WEB SEMINAR PARTICIPANT FEEDBACK RESULTS ...................................................................................4 PRESENTER’S EVALUATION: JESSICA FRIES-GAITHER ............................................................................5 PRESENTER’S EVALUATION: DR. CAROL LANDIS .......................................................................................5 RECOMMENDATIONS ............................................................................................................................................5 APPENDIX A: SCREEN SNAPSHOTS FROM THE WEB SEMINAR ...............................................................6


NSTA Web Seminar NSDL: Beyond Penguins and Polar Bears: Integrating Science and Literacy…

May 27, 2008

3

Web Seminar Evaluation Results NSDL: Beyond Penguins and Polar Bears: Integrating Science and Literacy for the K-5

Classroom, Seminar 1: Polar Geography May 27, 2008

Web Seminar Summary This Web Seminar, developed in collaboration with the National Science Digital Library (NSDL) took place on Tuesday, May 27, 2008, from 6:30 p.m. to 8:00 p.m. Eastern time. In this program Jessica Fries-Gaither, Project Director and Elementary Resource Specialist for the Ohio Resource Center, and Dr. Carol Landis, Education Coordinator at the Byrd Polar Research Center, featured the online magazine titled: Beyond Penguins and Polar Bears. This magazine provides resources and strategies for teachers interested in integrating science and literacy. Forty-seven (47) participants were in attendance at the Web Seminar and thirty-nine (39) of them completed the online feedback survey at the end of the session. This web seminar’s archive can be found at: http://learningcenter.nsta.org/products/symposia_seminars/NSDL2/webseminar11.aspx Overall, the participants rated the Web Seminar content as valuable, interactive, and relevant. Open response comments complimented the organization of the seminar and confirmed the ability of participants to engage with the facilitator and content experts via this medium. One hundred percent (100%) of the participants would like to see more of these types of synchronous online learning experiences offered. Web Seminar Content Overview The Beyond Penguins and Polar Bears online magazine specializes in providing educators with resources and strategies for integrating science and literacy with a focus on the Earth’s Polar Regions. Fries-Gaither and Landis started out by presenting a brief overview of characteristics of the Arctic and Antarctic—temperature, geography, wildlife, and other comparative factors between the poles. Using polar geography as a context, Fries-Gaither then demonstrated literacy related activities from the magazine such as content area reading strategies, graphic organizers, and reading and writing exercises for use in classrooms of grades K-2 and 3-5.

Participating educators represented the states of Alabama, Alaska, California, Colorado, Connecticut, Florida, Idaho, Illinois, Indiana, Kansas, Maine, Maryland, Massachusetts, Michigan, Minnesota, Nevada, New Jersey, New York, Ohio, Texas, Washington, and Wyoming. One participant attended the program from Australia. Participants received a one-year subscription to one of NSTA’s SciGuides for completing the evaluation form at the end of the program.



May 27, 2008

4

Web Seminar Participant Feedback Results Session Structure and Content: Please evaluate the session using the following scale: 5 - Excellent, 4 - Good, 3 - Average, 2 - Fair, 1 - Poor Overall, the Web Seminar was valuable to me. 4.64/5.0 The content of this Web Seminar was relevant to me. 4.59/5.0 The interactive nature of the Web Seminar was valuable. 4.69/5.0 The time the Web Seminar was held fit my schedule. 4.33/5.0 Please describe how this content was valuable or relevant to you.

• This content provided good background information for me to help students develop clear ideas of what our Polar Regions are and are not.

• I don't directly teach about the Polar Regions but I can incorporate some of the information into weather/climate in Earth Science and food chains in Life Science.

• We do a lot of work with environments - how they are similar and how they are different. • I loved the online resources - especially the magazine! It is always nice to be able to

access these on our own time. • I'll be teaching MST and literacy later this summer to K-6 teachers. This will be a

marvelous resource for them. • I teach gifted students who are very worried about global warming and losing the polar

animals. Would you like to see more Web Seminars like this one offered in the future?

Yes 100% No 0%

Please share your overall comments about this Web Seminar, for example, what you enjoyed the most, what you found most interesting. Tell us one thing you learned by attending this Web Seminar.

• Not only does this share excellent scientific information in a straight-forward, easy to understand format, but also ongoing resources to help us better teach literacy.

• The graphic showing the size of Antarctic to the US brought home the size relation. • I really liked the way it was organized, the interactive capabilities of the white board for

the presenters and the animations. • Again, the content designed to enhance background knowledge of elementary teachers

was excellent. I was able to flesh out my ideas about how to define and explain Arctic and Antarctic and got great resource sites. I love the magazine format!

• This was my first Web Seminar and I will definitely come back for more! This topic doesn't directly relate to my subject, but it was a truly great experience. I enjoyed the academic dialogue throughout the presentation.

• I learned that Antarctica is 1.5 times the size of the continental U.S. (something new!) I liked the maps of Antarctica especially.



May 27, 2008

5

Presenter’s Evaluation: Jessica Fries-Gaither Please evaluate the presenter using the following scale: 5 - Excellent, 4 - Good, 3 - Average, 2 - Fair, 1 - Poor Knowledge of subject 4.87/5.0 Clarity of explanations 4.82/5.0 Responsiveness to questions 4.79/5.0 Pace of delivery 4.67/5.0 Presenter’s Evaluation: Dr. Carol Landis Please evaluate the presenter using the following scale: 5 - Excellent, 4 - Good, 3 - Average, 2 - Fair, 1 - Poor Knowledge of subject 4.90/5.0 Clarity of explanations 4.82/5.0 Responsiveness to questions 4.82/5.0 Pace of delivery 4.69/5.0 Additional comments from participants:

• Very knowledgeable and easy to listen to as they spoke. • Super presentation. Thanks so much. • I especially appreciated that the presenters told us where to go for more information when

they didn't have totally all the answers....like who expects them to?! Finding information for ourselves makes us more invested in the subject.

• This was a great experience. It makes me want to go with the People to People Organization next time they invite me to go to the Antarctic. It sounds like a truly amazing place to see. Thanks so much!

• Continuing beyond the presentation time is a wonderful way for others to ask more questions.

Recommendations

• One of the presenters had audio problems right before the start of the recording. The moderator will do a final audio check a bit earlier before starting the recording to make sure the presenters still have their audio before the recording begins.

• If this web seminar is repeated participants suggested that time is dedicated to discuss

ways of implementing (strategies) the content in the classroom.

• If this web seminar is repeated, the presenters might consider including a slide with a list of websites and their URLs. Participants suggested that such list can be printed as a resource for teachers.



May 27, 2008

6

Appendix A: Screen Snapshots from the Web Seminar

Figure 1: The presenters compared the size of Antarctica to the United States.

Figure 2: What are the mean summer and winter temperatures in the Arctic?

Figure 3: Where does the Arctic begin? Several definitions exist.



National Science Digital Library Beyond Penguins and Polar Bears:

Physical Science from the Poles

October 29, 2008


2

Web Seminar Participant Feedback Results NSDL: Beyond Penguins and Polar Bears: Physical Science from the Poles

October 29, 2008

This report provides the participant feedback results for the Web Seminar titled NSDL: Beyond Penguins and Polar Bears: Physical Science from the Poles. Recommendations for future iterations of Web Seminar events are provided at the conclusion of the report. WEB SEMINAR SUMMARY....................................................................................................................................3 WEB SEMINAR CONTENT OVERVIEW..............................................................................................................3 WEB SEMINAR PARTICIPANT FEEDBACK RESULTS ...................................................................................4 PRESENTER’S EVALUATION: JESSICA FRIES-GAITHER ............................................................................5 PRESENTER’S EVALUATION: DR. CAROL LANDIS .......................................................................................5 RECOMMENDATIONS ............................................................................................................................................5 APPENDIX A: SCREEN SNAPSHOTS FROM THE WEB SEMINAR ...............................................................6


NSTA Web Seminar NSDL: Beyond Penguins and Polar Bears: Physical Science from the Poles

October 29, 2008

3

Web Seminar Evaluation Results NSDL: Beyond Penguins and Polar Bears: Physical Science from the Poles

October 29, 2008

Web Seminar Summary This Web Seminar, developed in collaboration with the National Science Digital Library (NSDL) took place on October 29, 2008, from 6:30 p.m. to 8:00 p.m. Eastern time. Jessica Fries-Gaither, Project Director and Elementary Resource Specialist for the Ohio Resource Center and Dr. Carol Landis, Education Coordinator at the Byrd Polar Research Center, featured resources from the online professional development magazine, Beyond Penguins and Polar Bears for K-5 educators Sixty (60) participants were in attendance at the Web Seminar and forty-five (45) of them completed the online feedback survey at the end of the session. This web seminar’s archive can be found at: http://learningcenter.nsta.org/products/symposia_seminars/NSDL3/Webseminar2.aspx Overall, the participants rated the Web Seminar content as valuable, interactive, and relevant. Open response comments complimented the organization of the seminar and confirmed the ability of participants to engage with the facilitator and content experts via this medium. One hundred percent (100%) of the participants would like to see more of these types of synchronous online learning experiences offered. Web Seminar Content Overview Dr. Carol Landis provided background information on the physical properties of water and concepts in changes in states of matter. Using the formation of glaciers as a context to understand these ideas, Dr. Landis also discussed the different types of ice formations from ice sheets to ice floes and glaciers to sea ice. Jessica Fries-Gaither addressed misconceptions that students often have related to these concepts and teaching strategies through science content and literacy approaches that are a major focus of the Beyond Penguins and Polar Bears magazine, available free online.

Participating educators represented the states of Alaska, Arizona, California, Florida, Louisiana, Maine, Massachusetts, New Hampshire, New Jersey, New York, Ohio, Oklahoma, Pennsylvania, Tennessee, Texas, Virginia, West Virginia, Wisconsin, and Wyoming. One participant attended the program from Sweden. Participants received a one-year subscription to one of NSTA’s SciGuides for completing the evaluation form at the end of the program.



October 29, 2008

4


• I teach density with weather and touch on global warming. • I am very interested in climate change issues. • I liked the simple demonstrations and breaking of misconceptions. • I can use the information to teach my elementary gifted students. • Excellent with the general misconceptions and great clarifications. Wonderful

suggestions for hands-on activities with your students. Activities mentioned were appropriate to all age levels.

• I have questions here in West Virginia about snow and ice that I never had in Texas! I have gained valuable insight that I never had before; maybe I needed to move to the cold environment way before now.

Would you like to see more Web Seminars like this one offered in the future?

Yes 100% No 0%


• I enjoy the conversation with other teachers. The presenters are well prepared and have great resources.

• The experiment with ice cubes in water and alcohol sounds like something I might try. • This was my first web seminar. I really enjoyed it and will be sure to register for more. • I enjoyed hearing answers to questions that I know my students are going to ask. I also

enjoyed getting tips and ideas from all those present in the chat window. • I thought it was very informative and beneficial. I learned the common misconceptions

students have pertaining to the properties of ice and water. • Excellent resources and knowledge of subject. Appreciate efforts to include primary

grades in science emphasis.



October 29, 2008

5


• By far, the best presentation overall for the seminars that I have attended. All the information presented was relevant to what I do and the suggestions for activities are directly applicable to our programs and easily adapted for a large age range of students.

• Professional development with people from all over the world while eating chocolate in my pajamas! Can life get any better?

• Thank you and I will be back for more! I plan to visit your web site just as soon as I finish this evaluation!

• This was my first online seminar and I enjoyed it! • Having two presenters was great!

Recommendations

• Some of the participants had trouble accessing the evaluation survey at the end of the program. The moderator will be sure to copy the link to the survey as a back up if the link needs to be posted multiple times on the chat.

• The moderator posted the timer for presenters. It helped to keep the program on time

without requiring the presenters to look at prompts from the moderator on the chat to tell them how much time was left. The moderator will continue to do this for each seminar.



October 29, 2008

6


Figure 1: The presenters defined the states of matter as solid, liquid, and gas.

Figure 2: Participants use their clip art to answer one of the presenters’ questions.

Figure 3: The presenters talked about the physical properties of water.



National Science Digital Library Beyond Penguins and Polar Bears: Energy and the Polar Environment

November 13, 2008


2

Web Seminar Participant Feedback Results NSDL: Beyond Penguins and Polar Bears: Energy and the Polar Environment

November 13 2008

This report provides the participant feedback results for the Web Seminar titled NSDL: Beyond Penguins and Polar Bears: Energy and the Polar Environment. Recommendations for future iterations of Web Seminar events are provided at the conclusion of the report. WEB SEMINAR SUMMARY....................................................................................................................................3 WEB SEMINAR CONTENT OVERVIEW..............................................................................................................3 WEB SEMINAR PARTICIPANT FEEDBACK RESULTS ...................................................................................4 PRESENTER’S EVALUATION: JESSICA FRIES-GAITHER ............................................................................5 PRESENTER’S EVALUATION: DR. CAROL LANDIS .......................................................................................5 RECOMMENDATION...............................................................................................................................................5 APPENDIX A: SCREEN SNAPSHOTS FROM THE WEB SEMINAR ...............................................................6


NSTA Web Seminar NSDL: Beyond Penguins and Polar Bears: Energy and the Polar Environment

November 13, 2008

3

Web Seminar Evaluation Results NSDL: Beyond Penguins and Polar Bears: Energy and the Polar Environment

November 13, 2008

Web Seminar Summary This Web Seminar, developed in collaboration with the National Science Digital Library (NSDL) took place on November 13, 2008, from 6:30 p.m. to 8:00 p.m. Eastern time. Jessica Fries-Gaither, Project Director and Elementary Resource Specialist for the Ohio Resource Center and Dr. Carol Landis, Education Coordinator at the Byrd Polar Research Center, provided content and resources on the topic of energy and how they relate to the poles. Teaching strategies for integrating science and literacy were also featured. Forty-one (41) participants were in attendance at the Web Seminar and thirty-four (34) of them completed the online feedback survey at the end of the session. This web seminar’s archive can be found at: http://learningcenter.nsta.org/products/symposia_seminars/NSDL3/Webseminar3.aspx Overall, the participants rated the Web Seminar content as valuable, interactive, and relevant. Open response comments complimented the organization of the seminar and confirmed the ability of participants to engage with the facilitator and content experts via this medium. One hundred percent (100%) of the participants would like to see more of these types of synchronous online learning experiences offered. Web Seminar Content Overview Dr. Landis introduced concepts of albedo, energy balance, solar radiation, and how these factor into scientists creating models for studying climate change and its effects on the poles. While it is a complex system of variables, studying changes at the poles provides a way of teaching and understanding these concepts with real-world application. Ms. Fries-Gaither talked about teaching energy in elementary grades by discussing student misconceptions about the Sun’s energy and reflectivity. She also provided literacy strategies to integrate in teaching science.

Participating educators represented the states of Arizona, Florida, Illinois, Maryland, Massachusetts, Minnesota, Mississippi, Missouri, New Hampshire, New Jersey, New York, North Carolina, Ohio, Pennsylvania, Tennessee, Texas, Wisconsin, and Wyoming. One participant attended the program from Romania. Participants received a one-year subscription to one of NSTA’s SciGuides for completing the evaluation form at the end of the program.



November 13, 2008

4


! I am teaching about this subject as well as taking a college course relative to this information.

! Good information for my lower level students but can also be adjusted for the higher level students.

! Still working on IPY with the kids and looking to keep an enrichment program going on global climate change.

! Introduced and reinforced content material I need to properly teach these areas to my students. These are content areas addressed on the ISAT.

! I am director of an elementary school with a focus on sustainability. This topic is directly aligned with our current unit on Natural Resources and energy from the sun.

! Explained the importance of albedo to our global climate situation. Showed clear and practical resources. Terrific literacy resources that cover a range of reading levels.

Would you like to see more Web Seminars like this one offered in the future?

Yes 100% No 0%


! The student misconceptions presented were very enlightening. ! This (Web Seminar) as well as the others that I have attended are excellent. I enjoy the

overall atmosphere and the knowledge of the presenters. ! I learned that there are several different disciplines of science that can be applied to this

topic to create a thematic unit, including chemistry, physics, geography, geology, and of course, math as well as literacy.

! Great web seminar! Provided content knowledge and lesson plan ideas! What a great resource for science teachers!

! I loved this seminar! It clarified some topics for me and gave me fantastic ideas about avoiding misconceptions with my youngest students.



November 13, 2008

5

! I learned about albedo. Not sure I will teach it to my 5th graders, but I am always adding to my store of knowledge. The more I can hear this sort of information, the stronger I become at teaching general science.


! Thank you for all of your hard work in getting this info to us. ! Very well done. All materials presented are extremely useful and applicable to our style

of teaching. ! Great presentation! Thank you! I will be on board in the spring for the next one. ! Everyone did a great job presenting information! ! Nice blend of content and resources. Easy to follow. ! Both presenters were knowledgeable and engaging.

Recommendation

! The presenters did a great job of responding to participants questions during the presentation and in the post-program chat. The moderator will test out video with presenters prior to the seminar since there was one video we tried to run at the end of the program that did not load up well on computers in time to show.



November 13, 2008

6


Figure 1: Participants use their clip art to answer one of the presenters’ questions.

Figure 2: Participants learn about albedo.

Figure 3: Participants answer a poll question about climate change.


NSDL Brown Bag Report

Date of Report: 1.15.08

Brown Bag Title: Beyond Penguins and Polar Bears: Integrating Science and Literacy in the Elementary Classroom---States of Matter

Date of Program: 1.13.08, 6:30pm-7:45pm Eastern Moderator: Robert Payo Presenter: Jessica Fries-Gaither, Project Director Beyond Penguins and Polar Bears, Ohio State University

This webinar was developed in collaboration with Ohio State University and Stafford County Public Schools, VA.

9 participants attended during the session and 6 of them completed the online feedback survey at the end of the session. 7 of 9 participants stayed on the session for the full presentation with one participant on for 30 minutes and another for 20 minutes. (One additional participant logged on for 2 minutes and is not counted in these numbers.) Participants learned about the seminar through Rita Truelove, science coordinator for Stafford County Public Schools. Brown Bag Overview Jessica Fries-Gaither provided an overview of resources related to Beyond Penguins and Polar Bears with a focus on science concepts, misconceptions and literacy strategies related to teaching states of matter and changes in matter. Survey Feedback Results Rate the quality of session content (1-poor to 5-excellent) 4.4 avg

Sample Participant comments What was the reason you attended this session?

• To increase background knowledge • Gathering more materials to use with my teachers • Personal interest in the topic was a primary reason. I am also striving to find new

ideas and resources to make my classroom a better learning environment for the students. The mixing of hands-on and literacy engages and reaches more of my students than just informative talk.

• I have a personal interest and want to improve science lessons within the classroom.

NSDL Web Seminar Evaluation: States of Matter


What did you like most about the session? • The format and presentation • It was all fantastic, not too much information, just enough to get teachers salivating to

get into the cyberzine • I liked most the science experiment ideas that were given and the fact that technology

was integrated quite a bit into the lessons. Most of the time we think lectures/vocab but the students takeaway the most when it's personal to them. This is a technology generation.

What did you like least about the session? • It feels like a long time on the phone • Would have liked to use the chat feature to write in questions.

What was the biggest takeaway for you from this session?

• Background material • Content Clips. Can’t wait to use it! • Using the website to help connect my content areas (science and reading) • The discussion on matter. I want to use some of the books and experiments • To see that other educators struggle with similar problems that I have

Additional comments

• This was definitely worth my time! Thanks! • I am really looking forward to the 2nd webinar and am so happy this one will be

archived. I only wish that all teachers who signed up were able to be here. That is why the archive is a good idea.

NSDL Web Seminar Evaluation: States of Matter




Brown Bag Title: Beyond Penguins and Polar Bears: Integrating Science and Literacy in the Elementary Classroom---Rocks and Minerals



6 participants attended during the session and 5 of them completed the online feedback survey at the end of the session. 5 of the participants stayed on for the full duration of the presentation. Participants learned about the seminar through Rita Truelove, science coordinator for Stafford County Public Schools. Brown Bag Overview Jessica Fries-Gaither provided an overview of resources related to Beyond Penguins and Polar Bears with a focus on science concepts, misconceptions and literacy strategies related to teaching rocks and minerals. Survey Feedback Results Rate the quality of session content (1-poor to 5-excellent) 5.0 avg Sample Participant comments What was the reason you attended this session? • To gain more resources for my teachers. • I enjoyed the last presentation and wanted to learn more about the website. • Integrating science and literacy (a specific science topic wasn't necessary) is one of

my personal/professional goals. • Rocks and Minerals is always a topic my 5th grade teachers ask me for support with.

They are always looking for engaging ways to teach the concept. I wanted something new to offer them.

• I thought I would hear about how to extend and build upon my penguins and polar bears unit.

NSDL Web Seminar Evaluation: Rocks and Minerals


What did you like most about the session? • I like the time given for teacehrs to respond and share their great ideas. • I like the format of the slide and the voice interaction. • The "I've been there" aspect. • Very well done...I am excited to get off this and get on the website to explore the

features myself. • I liked learning about the website and the Beyond Penguins magazine.

What did you like least about the session? No comments. What was the biggest takeaway for you from this session? • I am going to download those podcast immediately. • The resources available through the website. I like the Arctic rock box idea. • Screened web resources. • Definitely the Antarctic Rock Box - how cool is that??? • I was surprised that even though it was about geology, I found many tools and

resources I can use to extend any of my science unit with scientific expertise!

Additional comments • Once again, these have been two of the most informatiove useful webinars I have

participated in. • I really enjoyed this! It is well worth my time! • I am wondering if you have a suggested outline to use as a year long overarching

theme. We teach several of the topics you have mentioned and can certainly supplement with additional suggestions while exploring other science topics - such as electricity or the moon phases. I would love to use this as an overall hook next year. Any thoughts?

• Thank you to those who made the web seminar available to Stafford County teachers.

NSDL Web Seminar Evaluation: Rocks and Minerals




Brown Bag Title: Beyond Penguins and Polar Bears: Integrating Science and Literacy in the Elementary Classroom---Life in the Polar Extremes



Number of Participants: 10 (8 stayed for the duration with 2 staying for 30 minutes) Number of Registrants: 23 Number of completed surveys: 7 Participants learned about the seminar through Rita Truelove, science coordinator for Stafford County Public Schools, NSDL channels and Beyond Penguins. Brown Bag Overview Jessica Fries-Gaither provided an overview of resources related to Beyond Penguins and Polar Bears with a focus on science concepts, misconceptions and literacy strategies related to teaching ecosystems and life in extreme climates. Survey Feedback Results Rate the quality of session content (1-poor to 5-excellent) 4.3 avg

Participant comments What was the reason you attended this session?

• Currently teaching ecosystems • Personal interest • The presenter • I'm teaching habitats and wanted some tips or resources to use. • Support my teachers.

What did you like most about the session?

• The online resources • The wonderful ideas that were given as well as the resources! • Good presentation, trusted content - good 'tips for use'

NSDL Web Seminar Evaluation: Life in the Polar Extremes


What did you like least about the session?

• The topic overview a little too much. • I wasn't sure of how this thing worked. I didn't know if I was supposed to take

notes and write everything down...is there a list of all these resources somewhere? I really didn't know what I was getting into and I feel like a lot of teachers did not sign up for this because no one really understood what this was about or what you had to do while listening. [Note: This was from a participant outside of Stafford County teachers]

What was the biggest takeaway for you from this session? • Resources • The web resources and the fact that I can use it tomorrow • The diigo list

Additional comments

• I hope we can continue this relationship next year as well. I think we have really reached some folks. I had some great comments from some middle school folks.

• These brown bags are a great add-on for the site - as you repurpose things, consider offering the highlights distributed at conferences as pdfs for downloading/archiving. what next? since you are versed in climate change... move from the poles and look at what polar science predicts for temperate/tropical regions, rainforests, etc with the same 'eye' to education?

• I liked [the] presentation, but I guess it would be neat to have a broader topic. We don't even "need" to teach the tundra in 3rd grade anymore (I still do) but, for me, I think if I were going to do this again, I would want it on an entire SOL rather than just a tiny aspect/detail of it. Or if it wasn't on an entire topic, then at least give the presentation on the detail and give us more resources on the broader topic.

• I enjoy all presentations given by Robert and Jessica! Very professional and well organized! The BEST part is that I can go to school tomorrow and use things I learned about tonight! YIPEE!!

• I would participate in another if Ms Truelove is able get NSDL to do another one or two especially if I could see something on weather and plants.

• I really enjoyed this. Thank you.

NSDL Web Seminar Evaluation: Life in the Polar Extremes


Polar Adventure was held February 27th‐March 1st at COSI. This was a special event weekend around the theme of polar science and polar exploration. This was an additional outreach component of a grant with Beyond Polar Bears and Penguins. This was an opportunity for real researchers to showcase their work for the general public and to connect the public to real scientists. Programming components for the weekend were: an Arctic tent, polar gear, polar rocks, an ice core, ice core drill bits, and ice core transportation boxes. The Ohio State Center for Automotive Research brought a car for display. Miami University brought flora and fauna from Antarctica for our guests to examine. Shackleton’s Adventure was also shown on our Extreme Screen. A special demonstration that showed the properties of water was presented before each speaker. Speakers for the program were 6 professors and graduate students from Byrd Polar Research Institute, 1 professor from Miami University, and 1 dogsled team owner. There were three goals set forth by the planning team. 1) Drive attendance 2) Generate revenue 3) Hold the event with minimal operational and team impact. The first two goals were accomplished. We are still determining if Polar Adventure met the goal of minimal operational impact on the COSI team. Attendance We can see by looking at the graph below that Polar Adventure did drive attendance:

Polar Adventure Weekend at COSI February 27 - March 1, 2009


Looking at these numbers we can see 6844 people attended Polar Adventure Weekend and that almost 1500 people attended over the same weekend last year and ahead of plan. There could be a number of determining factors to dive attendance such as weather, the economy, or lack of other options but we can still see there was a general positive result when looking at attendance. We see a slight increase with the general public with little over 100 people but a large increase in our members, with over 900 additional people. This was almost a 40% increase over last year. We did get several mentions in the Columbus Dispatch about Polar Adventure and it was mentioned in our Member Newsletter. Key points:

• Increase in attendance over plan and FY09 • The large increase was mainly attributed to our members

Revenue We can see by looking at the chart below that Polar Adventure Weekend was a success in bringing in additional revenue to COSI when compared to FY08 and the goal for FY09.


We see that for the general public for both the exhibits and the Extreme Screen we exceeded both FY08 and the goal for FY09. The biggest surprise is that, for members, their movie ticket purchase was up 158% over FY08 and 61% over plan for FY09. This is significant because, while we do not receive any additional revenue from member exhibit attendance, we do from movie attendance. It is important to note that a polar themed movie was shown this weekend but there was no distinction in tickets purchased between Shackleton’s Adventure and the other movies being shown in this data. Key Points:

• Increase in revenue over plan and FY08 • Members purchased Extreme Screen tickets significantly over plan and FY08

Operational Impact: This concerned team time dedicated to the event and logistical challenges that presented themselves leading up to and during the event. Below is the team time contributed to Polar Adventure in hours.

Looking at the hours we can see this event took 184.5 hours from the COSI team. The hours were split evenly with 92.5 hours for planning and 92 hours for running of the event. The only operational hiccup was that it was not communicated with enough lead time that the WOSU dock would need to be cleared in order to bring in the car from the Center for Automotive Research. Thanks to quick moving by John Shaw and his team, the car was brought in without incident. Key points:

• This event took up significant staff time • Communication of special needs are very important to the success of an event


Program and Partnerships This weekend was the result of a grant in partnership with Beyond Polar Bears and Penguins. It offered COSI the opportunity to put on a large themed event with dedicated resources. We partnered with Byrd Polar, the Center for Automotive Research, and Miami University in addition to Tom Roig of Valley Road Outfitters. It also allowed COSI to test having real researchers present to a general public audience and to gauge the audience reaction. For the research presentations, in order to draw a crowd, COSI did a short demonstration before the researcher took the stage. The COSI team member exploded a cast iron container showing how when water turns into ice, it expands causing the container to burst. This demonstration was extremely popular with the visitors and would not have been possible without the grant’s assistance due to costs. It also gave the presenter that followed a built in audience. This format was very successful in meeting both the expectation of fun and real science for our guests. The researchers did comment how they were worried to follow such a dynamic demonstration. The guests did enjoy the presenters though, one commented via twitter, "the presenters/researchers were really ready and excited to see the kids. My two year old and my inner two year old had fun." All partners involved were very happy with the event. Rick Lee from Miami University said “Anytime you are going to do something like this, give a call. We had a great time.” Tom Roig from Valley Road Outfitters was very happy with the event as well saying, “I had no idea that there would be so many people. The dogs loved it.” It also gave the opportunity for David to meet Barry Lyons the Director of Byrd Polar just before Dr Lyons presented on his work in Antarctica. Key points:

• The partners felt Polar Adventure was worth their while and met expectations • The public enjoyed the researchers’ presentation

Audience Evaluation: An evaluation of Polar Adventure Weekend took place. Entry and exit interviews were taken of visitors with a total of 352 usable questionnaires completed. Of those surveyed 38.5% at entry and 28.9% were members. The target age for this weekend was children ages 5‐12. For entry 53.8% and 61.4% at exit had children within those age ranges. While this event garnered good media with coverage in the Columbus Dispatch and the COSI Member Newsletter 75% of surveyed visitors did not know that Polar Adventure was happening before attendance. For members, 41.1% knew the event was taking place; for non member the number dropped to 26%. This correlates with the attendance numbers, with the most significant jump in attendance coming from our members. On entry and exit the visitors’ values and motivations were measured. The visitors showed a strong preference to viewing COSI as an opportunity to spend quality time with family/friends. They also showed a strong preference for viewing COSI as a fun learning institution outside of the classroom. Scoring low on the preference was recognizing COSI as a place to learn about science careers. The visitors entered COSI with the dominant value of quality time, and were highly satisfied with the experience, but on exit were even more satisfied with the perceived valued of education and learning during Polar Adventure Weekend at COSI. Overall our visitors revealed a strong satisfaction with their visit to COSI during the Polar Adventure Weekend.


Key points:

• Many members were aware of Polar Adventure Weekend • The general public was aware of Polar Adventure Weekend but at a much lower rate than

members • 50%‐60% of our surveyed guests were in the target age group • Overall, our guests enjoyed Polar Adventure Weekend

Other Issues: Being that this weekend was about polar science there was discussion among the leadership team of what COSI’s response should be for the issue of global climate change as an institution. This spirited debate was carried out over email in just one day. Leadership should be commended for reaching consensus quickly. The final statement says, “Welcome to COSI. We're so happy you joined us this weekend. Please know COSI welcomes all people and we respect the diversity of opinion on this and other issues. The current consensus of the scientific research community including the National Academy of Sciences, American Meteorological Society, and American Association for the Advancement of Science is that global climate change is real and that human activity is the main contributor to increased greenhouse gases in our atmosphere.” We encountered no backlash from the team or public over this statement. David posted a blog on March 20th about how COSI came to this statement on global climate change. Bringing the sled dogs into COSI on Sunday March 1st showed a significant jump in attendance. There was, however, a reaction from the internet to COSI promoting the sport of dog sled racing. Kelli Nowinsky did receive numerous emails from individuals around the world protesting COSI promotion of the sport. Upon further investigation, Kelli’s email was added to a form letter petition and this was not the results of individuals directly contacting COSI. Fortunately, Kate Storm had emailed the Capital Area Humane Society to ask if they would have objections to sled dogs at COSI. The Humane Society said that if presentation was informational and not promotional they would have no issue. There were no incidents on site from individuals surrounding the issue of sled dogs. This incident underscores how when dealing with animal it is important to research all possible implications before they are brought to COSI. Guests’ response to the sled dogs was very positive. One stated via twitter, "Frogs were great. Loved the virtual dissection. Survey on the way out was really confusing though. Oh yeah... LOVED getting to meet the sled dogs!!" The team reacted well to both the issues of global climate change and opposition to sled dogs in a quick and efficient way. Kelli Nowinsky and Leadership Team rose to the challenge that was presented to them. These two issues do highlight that, as COSI moves to bringing real and relevant science to the public, we will need to move beyond operational thinking. We need to think big picture and to think of how the public will react to the issues that are presented. While these issues could not have been avoided, more time to debate COSI’s response could have been allowed if these issues were identified in the planning process. Key points:

• Working with divisive issues requires institutional planning • Animals while popular, can carry unintended consequences

Lessons Learned: When planning a special event it is important to think past operations and look at the event in the eyes of the public. What issues might people have with the event? If a contentious point such as climate change is discussed, COSI must be prepared to have a talking point for the team.


Anything with animals must be thoroughly thought out. Our audience likes animals and they are a big draw, but we must research what issues might exist out in the public. Kate Storm, having contacted the Capital Area Humane Society, helped to gauge the local community’s possible reaction to the sled dogs. The internet opposition to sled dogs was a surprise. Fortunately no incident on site resulted. The presentation format of having a demonstration followed by a presenter works well and is able to hold an audience’s attention. This format should be used in the future. Always have a dedicated point person with the partner for an event. Carol Landis was key in the success of this event. In having a person dedicated at Byrd Polar, she was able to handle the schedule of presenters and the acquisition of display items that helped to make this event special for our guests. Funding for this event allowed it to be much larger and more dynamic that it otherwise would have been. The ice bomb demonstration and sled dog team would not have been possible without the grant assisting Polar Adventure. Conclusions: Polar adventure was a success in meeting two of its three goals. Polar Adventure met its goals for driving attendance and revenue. We are still in the process of evaluating operational impact. We had a higher than normal adult attendance rate. The main attendance increase was from members and they bought movies at a much higher rate than anticipated. The visitor overall, were very satisfied with Polar Adventure and the partners were very happy with the event. Polar Adventure could serve as a model for how special events could run in the future at COSI.


Why are you here today? Read the following statements then check the 5 that best reflect why you are here today. Then, for those 5 statements only, indicate the importance of the reason.

If a statement represents a very important reason you are here today, circle 7. If a statement represents a less important reason you are here today, circle 1.

Check 5

Less Importa

nt Reason

More

Important

Reason

I like the types of things I can learn here. 1 2 3 4 5 6 7

I came a long time ago and want to come back. 1 2 3 4 5 6 7

I actively support science education in the community. 1 2 3 4 5 6 7

It is one of the best places to visit around here. 1 2 3 4 5 6 7

I love science. 1 2 3 4 5 6 7

Science fills me with wonder. 1 2 3 4 5 6 7

My wife/partner/husband made me come. 1 2 3 4 5 6 7

I discover things about myself when I come here. 1 2 3 4 5 6 7

I frequently visit science centers when I go on trips. 1 2 3 4 5 6 7

I get more here than going to the mall or a movie. 1 2 3 4 5 6 7

It was my choice for how to spend the day. 1 2 3 4 5 6 7

I support the mission to better understand science. 1 2 3 4 5 6 7

My family/friends have good experiences here. 1 2 3 4 5 6 7

This is a good way for my family/friends to share quality time.

1 2 3 4 5 6 7

I feel energized in these surroundings. 1 2 3 4 5 6 7

My family/friends enjoy themselves here. 1 2 3 4 5 6 7

Coming here helps me appreciate the complexity of the world.

1 2 3 4 5 6 7

I like to see everyday phenomena explained. 1 2 3 4 5 6 7

I like to experience different ways things are taught. 1 2 3 4 5 6 7

This is an important institution in this community. 1 2 3 4 5 6 7

Please turn the page, there are some more questions on the next page


Topic Quality Time

Careers Educational Place to be Read through the above statements. Then use the chart below to help you compare EACH of these statements against all the others. On each line, put a checkmark in the blank that shows how strongly you prefer one statement over the other. If they’re fairly close, you’d put a check in one of the middle blanks. If you really prefer one, you’d put a check in a blank closer to that statement.

Topic _____ _____ _____ _____ _____ _____ Quality time

Educational _____ _____ _____ _____ _____ _____ Place to be

Place to be _____ _____ _____ _____ _____ _____ Careers

Educational _____ _____ _____ _____ _____ _____ Quality time

Careers _____ _____ _____ _____ _____ _____ Topic

Quality time _____ _____ _____ _____ _____ _____ Place to be

Place to be _____ _____ _____ _____ _____ _____ Topic

Careers _____ _____ _____ _____ _____ _____ Educational

Topic _____ _____ _____ _____ _____ _____ Educational

Quality time _____ _____ _____ _____ _____ _____ Careers

Please turn the page, there are four more questions on the last page

I like to come here with my family and friends. It is important to have a nice place to spend time with people I am close to in order to have a quality experience together. I also enjoy meeting people in places like this because they share the same values.

COSI is a top place to visit in Columbus and when we plan activities, COSI is on the list. This is a fun place to spend the day and I recommend COSI to friends and people from out of town as an example of what Columbus has to offer.

I am interested in having my child know more about science and I want them exposed to careers in science and technology. I think COSI is a good place to learn about careers both in and outside of science.

I think it is important for people to know that learning takes place outside the classroom. We visit COSI because it is educational and I think science in a fun environment helps people learn more about science in their lives.

I love science and the topic of polar exploration really excites me. I know a fair amount about this subject and always want to learn more. It is important to have a place where I can continue to learn things and challenge myself.


Are you a COSI Member? Yes No

Please describe the family/group with whom you are visiting COSI today:

Number of adults:

Number of children:

Ages of children:

Was there anything special that made you think of visiting COSI today? Please tell us what prompted your visit.

Did you know about COSI’s “Polar Adventure” activities before deciding to visit today?

Yes No

Thank you very much for taking the time to complete this survey – your responses will help COSI develop

and present the best experiences for you and your family. Thank you!!


Topic Quality Time

Careers Educational Place to be Read through the above statements. Then use the chart below to help you compare EACH of these statements against all the others. On each line, put a checkmark in the blank that shows how strongly you prefer one statement over the other. If they’re fairly close, you’d put a check in one of the middle blanks. If you really prefer one, you’d put a check in a blank closer to that statement.

Topic _____ _____ _____ _____ _____ _____ Quality time

Educational _____ _____ _____ _____ _____ _____ Place to be

Place to be _____ _____ _____ _____ _____ _____ Careers

Educational _____ _____ _____ _____ _____ _____ Quality time

Careers _____ _____ _____ _____ _____ _____ Topic

Quality time _____ _____ _____ _____ _____ _____ Place to be

Place to be _____ _____ _____ _____ _____ _____ Topic

Careers _____ _____ _____ _____ _____ _____ Educational

Topic _____ _____ _____ _____ _____ _____ Educational

Quality time _____ _____ _____ _____ _____ _____ Careers

Please turn the page, there are some more questions on the next page

I like to come here with my family and friends. It is important to have a nice place to spend time with people I am close to in order to have a quality experience together. I also enjoy meeting people in places like this because they share the same values.

COSI is a top place to visit in Columbus and when we plan activities, COSI is on the list. This is a fun place to spend the day and I recommend COSI to friends and people from out of town as an example of what Columbus has to offer.

I am interested in having my child know more about science and I want them exposed to careers in science and technology. I think COSI is a good place to learn about careers both in and outside of science.

I think it is important for people to know that learning takes place outside the classroom. We visit COSI because it is educational and I think science in a fun environment helps people learn more about science in their lives.

I love science and the topic of polar exploration really excites me. I know a fair amount about this subject and always want to learn more. It is important to have a place where I can continue to learn things and challenge myself.


Reflect on today’s visit to COSI and circle a number to indicate to what degree you:

Not at all

Completely

Had a positive experience today 1 2 3 4 5 6 7 Felt welcomed 1 2 3 4 5 6 7 Had opportunities to expand knowledge 1 2 3 4 5 6 7 Experienced appropriate exhibits 1 2 3 4 5 6 7 Had good information about what you were seeing/experiencing 1 2 3 4 5 6 7 Felt the exhibits were of high quality 1 2 3 4 5 6 7 Felt the shows and demonstrations were of high quality 1 2 3 4 5 6 7 Had positive interactions with the COSI Team 1 2 3 4 5 6 7 Learned new things about polar exploration. 1 2 3 4 5 6 7 Spent quality time with your friends or family. 1 2 3 4 5 6 7 Found out something about careers in science that you didn’t know before. 1 2 3 4 5 6 7

Had fun while learning science. 1 2 3 4 5 6 7 Participated in a community event. 1 2 3 4 5 6 7 Felt like you were at an exciting place. 1 2 3 4 5 6 7 Believe that learning was taking place at COSI today. 1 2 3 4 5 6 7 Saw children learning about science careers. 1 2 3 4 5 6 7 Enjoyed socializing with people today. 1 2 3 4 5 6 7 Were challenged by the science content. 1 2 3 4 5 6 7

Are you a COSI Member? Yes No

Please describe the family/group with whom you are visiting COSI today:

Number of adults:

Number of children:

Ages of children:

Thank you very much for taking the time to complete this survey – your responses will help COSI develop

and present the best experiences for you and your family. Thank you!!


Documents

Beyond Penguins Evaluation Report 2008-2009