UNIVERSITY OF NEVADA LAS VEGAS Awards/Alison_Sl… · University of Nevada, Las Vegas Dear Dr. Pritchard, I hereby nominate Dr. Alison Sloat, Assistant Professor-in-Residence, College

March 6, 2017 Dr. Laurel Pritchard Acting Vice Provost for Undergraduate Education University of Nevada, Las Vegas Dear Dr. Pritchard, I hereby nominate Dr. Alison Sloat, Assistant Professor-in-Residence, College of Sciences, for an Academic Assessment Award for Individual Faculty. Alison’s passion for teaching, commitment to improve the educational experiences of UNLV undergraduate students, and work ethic make her a strong candidate for this award. Alison teaches and coordinates Science 101 – Introduction to the University for Science Majors, the First Year Seminar offered by the College of Sciences. She designed the course to teach students critical thinking skills needed to be successful in their science and mathematics degree programs, and in their professional careers. Alison has successfully structured SCI 101 around the five University Undergraduate Learning Outcomes (UULOs) used to define expectations for all UNLV graduates: 1, intellectual breadth and lifelong learning; 2, inquiry and critical thinking; 3, communication; 4, global/multicultural knowledge and awareness; and 5, citizenship and ethics. Indeed, all the course assignments that Alison has created satisfy one or more UULOs. Two examples illustrate this point. A major assignment in the course is the Scientific Ethics Research Poster Project, which requires students to apply research methods to study a scientific (UULO 1), ethical problem (UULO 5), analyze scientific information (UULO 2), and collaborate with another student (UULO 4) to create and present a scientific poster summarizing their findings (UULO 3). Similarly, the Career Project assignment requires students to take a career self-assessment to identify four suitable careers (UULO 2), research those four careers (UULO 1), and analyze their career path and goals (UULOs 4 and 5) in a written paper (UULO 3). The students in SCI 101 spend the majority of the semester conducting research on a scientific, ethical problem to complete the Scientific Ethics Research Poster Project. Alison assesses the Critical Thinking UULOs in a subset of the student poster projects every semester. In May 2016, results of the assessment indicated that students were at the Beginning to Developing stages of Critical Thinking, an appropriate level for students completing a First Year Seminar. Nevertheless, to increase the level of analytical thinking and student learning and success, Alison made several changes to class assignments and exercises. She included additional activities related to critical thinking during class discussions, analysis of information through written assignments, and campus involvement through guest lectures and/or student participation in campus events outside the classroom. She also revised and updated other assignments to increase their effectiveness and relevance, particularly during the Presidential Debate week held on campus in October 2016. As a result of these modifications, in Fall 2016 the quality of the posters noticeably increased, and students earned higher marks, compared to previous semesters.

UNIVERSITY OF NEVADA LAS VEGAS

UNLV

Recommendation letter for Alison Sloat Page 2 As the Coordinator of SCI 101, Alison assesses student work and uses her findings to revise the course curriculum in a cohesive fashion to enhance student learning, which positively impacts student progression and retention. She supervises between one and six instructors teaching SCI 101 in a particular semester, and therefore she is also responsible for ensuring consistency across all sections of the course. She accomplishes this goal by creating the syllabus, course lectures, activities, assignments, and custom rubrics to evaluate student performance in all sections, and by periodically meeting with each instructor to collectively strategize how to effectively present this information to their students, while respecting the academic freedom of each instructor. The Fall-to-Fall retention of new, full-time freshmen who took SCI 101 in Fall 2015 and re-enrolled in UNLV in Fall 2016 was 83.8%, highest among all Colleges in the university. This finding suggests that the modifications that Alison has implemented in SCI 101 as a result of her ongoing assessment of the course, as well as the skills that students acquire in the course have contributed to a noticeable improvement in learning and achievement of our undergraduate students. In Spring 2017, Alison started using Top Hat, an interactive teaching platform that enables students to answer questions in real time during lectures using their smart phones, tablets, or laptops. The immediate feedback allows her to precisely assess which topics the students find challenging, which prompts specific recitation and further explanations, leading to greater understanding of the concepts and thus increased student learning. Alison plans to incorporate Top Hat into all sections of SCI 101, starting in Fall 2017. The additional documentation attached to this dossier details further academic assessment activities that Alison has conducted in SCI 101 since Fall 2014. In addition to her active and intimate involvement in assessment activities, Alison is an effective instructor. Since she began teaching SCI 101 in Fall 2013, the average of her student teaching evaluations is an impressive 4.64 (median = 4.68) of 5 (29 sections, 599 evaluations). Alison does not attain these high marks by being an “easy teacher,” but by encouraging students to become engaged in the course and perform as well as they can, and by being supportive and accessible during and after class. In conclusion, I enthusiastically and unconditionally recommend Alison for an Academic Assessment Award for Individual Faculty. She is a dedicated teacher who continuously assesses SCI 101 in an effort to enhance student learning and increase retention of information. Her determination and accomplishments guarantee that she will continue to make a positive contribution to UNLV’s educational mission for years to come. Sincerely,

Javier A. Rodríguez, Ph.D. Associate Dean E-mail: [email protected]

College of Sciences 4505 S. Maryland Parkway • Box 454001 • Las Vegas, Nevada 89154–4001

(702) 895-5551 • FAX (702) 895-4159

Assessment in Science 101

Science 101 A. Sloat

1

Assessment in Science 101

My underlying goal as the Course Coordinator of Science (SCI) 101: Introduction to the University for Science Majors is to assess student learning to continuously improve the course and the critical thinking skills of undergraduate students majoring in UNLV’s College of Sciences. SCI 101 is a First Year Seminar (FYS) course designed to develop the analytical skills that students need to successfully progress through and complete their degree program at UNLV in a timely manner.

I have revised SCI 101 in a manner that all course assignments and exercises meet one or more of UNLV’s University Undergraduate Learning Outcomes (UULOs), which encompass: 1, Intellectual Breadth and Lifelong Learning; 2, Inquiry and Critical Thinking; 3, Communication; 4, Global/Multicultural Knowledge and Awareness; and 5, Citizenship and Ethics. The course assignments and the UULOs they satisfy include:

• Scientific Ethics Research Poster Project (UULOs 1–5) • Career Paper (UULOs 1–4) • In-Class Exercises (UULOs 1–5) • Journals (UULOs 1–5) • Outside Event Reflection Paper (UULOs 4–5)

For the Scientific Ethics Research Poster Project, the students spend the majority of the semester actively engaged in the process of conducting research on a scientific (UULO 1), ethical problem (UULO 5). Examples of past topics include whether it is ethical to prescribe opioid medication for pain, use electroconvulsive therapy on humans, allow plastic microbeads in cleaning products, and utilize stem cells to treat diseases. During this intellectual process, students collaborate with another student (UULO 4) to locate, identify, analyze, and cite scientific literature (UULO 2); write an annotated bibliography (UULO 3); write an abstract (UULO 3); design graphs to illustrate data (UULOs 2 and 3); create a scientific poster (UULO 3); and present their poster to their peers in a scientific poster session (UULO 3).

These exercises reflect the learning outcomes of the course, emphasizing the Inquiry and Critical Thinking UULO, which the FYS Coordinators from across campus identified as the most important UULO for academic assessment in Fall 2014. The assignments also reinforce the three main skills I want students to retain from SCI 101, skills that will help them to succeed in their academic and professional careers:


2

Skill 1: Identify problems using critical and analytical thinking.

Skill 2: Analyze and apply scientific evidence to solve problems.

Skill 3: Make conclusions based on reliable scientific evidence.

To assess whether students are achieving these learning objectives in SCI 101, I conduct an annual assessment of 10% of the student’s Scientific Ethics Research Posters using a custom rubric developed to evaluate student success in the Inquiry and Critical Thinking UULO (UULO 2). This UULO is subdivided in seven categories (UULOs 2.1–2.6).

UULO 2.1: Identification of Problem UULO 2.2: Documentation UULO 2.3.1: Analysis UULO 2.3.2: Interpretation UULO 2.4: Perspectives UULO 2.5: Future Work UULO 2.6: Conclusions

Problems Identified As a Result of Assessment

My common concerns when students completed their assignments or posters were that they were not following my verbal or written instructions, which we had spent most of the semester discussing. I attributed these problems to either their lack of understanding, lack of motivation, or to the fact that I needed to adopt a more effective pedagogical approach.

Formally assessing 10% (60 of 602) of the student posters from the Fall 2014 and Spring 2015 semesters enabled me to quantitatively identify students’ achievements in the Inquiry and Critical Thinking UULOs. As a result of this assessment, four main problem areas became the focus of improvement:

Problem 1: Lack of evaluation of the problem from at least two different perspectives (UULO 2.4: Perspectives) (Mean Score: 3.10/5.0). Problem 2: Inadequate or lack of analysis of results, data, and figures (UULO 2.3: Analysis and Interpretation) (Mean Score: 3.06/5.0).


3

Problem 3: Lack of reasonable and defendable conclusions (UULO 2.6: Conclusions) (Mean Score: 3.20/5.0). Problem 4: Lack of, or incorrect citation of scientific evidence (UULO 2.2: Documentation) (Mean Score: 3.15/5.0).

These problems were especially troubling to me, because students were not achieving the three main skills I had identified as necessary for them to progress successfully through their courses in the College of Sciences. In an effort to address these issues and to improve the learning experience for students in SCI 101, I participated in the Transparency in Learning and Teaching (TILT) Project.

Course Changes Made As a Result of Assessment

After identifying the aforementioned problems during assessment and participating in the TILT Project, I modified the SCI 101 assignments and in-class exercises in Fall 2015, as described in Table 1. Furthermore, I added explicit instructions regarding the purpose of the assignment and the task the student was to complete, and provided examples of appropriate student responses to each assignment and exercise.

Table 1. Course content added or modified to address problems identified as a result of assessment.

Problem

Course Content Addition/Modification

Problem 1: Lack of evaluation of the problem from at least two different perspectives (UULO 2.4).

A journal assignment (Appendix A) was added to show how to identify three pieces of evidence supporting two perspectives of an argument related to a scientific ethical question. An example of a sample journal was provided to students. Exercise 3 was modified to include analyses of sample posters to evaluate the two different perspectives discussed.

Problem 2: Inadequate or lack of analysis of results, data, and figures (UULO 2.3).

Exercise 6 was added to show students how to make graphs to illustrate scientific data. Students now prepare graphs for three different sets of data using Microsoft Excel, including explanations and analyses of those data.


4

Table 1. Continued.

Problem 3: Lack of reasonable and defendable conclusions (UULO 2.6).

A journal assignment (Appendix A) was added to illustrate how to derive defendable conclusions based on supporting evidence. An example was provided to students. Exercise 3 was modified. Students now analyze sample posters and evaluate their scientific credibility, evidence, and conclusions.

Problem 4: Lack of, or incorrect citation of scientific evidence (UULO 2.2).

Exercise 2 was modified. Students now practice citing several different types of sources. Exercise 4 was modified. Students now practice paraphrasing journal articles, using in-text citation, and writing references.

Closing the Loop: Additional Assessment and Changes

The changes I made to the course curriculum of SCI 101 had a positive effect on student learning, as shown in the results of the assessment of 10% (53 of 526) of the posters from the Fall 2015 and Spring 2016 cohort of students. Students’ achievement in all six of the Inquiry and Critical Thinking UULOs increased after I implemented the curriculum changes. Students showed the most improvement in UULO 2.1: Identification of Problem (+0.78), UULO 2.3: Analysis and Interpretation (+0.39), UULO 2.4: Perspectives (+0.34), and UULO 2.6: Conclusions (+0.25) (Figure 1). Additionally, students attained higher achievement in the four problem areas that I identified in Spring 2015.


5

Figure 1. Assessment results for 2014–2015 (Before Course Changes) and 2015–2016 (After Course Changes).

To continue to improve student learning, since Spring 2016 I give pre-tests to SCI 101 students at the beginning of each lecture. The pre-tests serve four purposes: 1, they elicit students’ interest in the course material; 2, they give students a preview of the lecture topic of that day; 3, they provide reinforcement of material discussed in previous lectures; and 4, they serve as a quick tool to help the students assess their own current knowledge by showing them what they do not know. The short length, short completion time of fewer than five minutes, and non-graded nature of the pre-test makes implementation easy on my end (Figure 2), but more importantly, the reinforcement of material through practice and analysis helps the students to earn higher grades on their posters (Figure 3).

Assessment of the posters from the four sections of SCI 101 that I taught in Fall 2016 shows that combining the previous course changes with the pre-tests contributed to higher poster scores, compared to previous semesters (Figure 3). The percentage of posters earning top scores dramatically increased, and equally important, the percentage of posters receiving low scores decreased. An example of an excellent poster is provided in Appendix B.


6

Figure 2. Example Pre-Test given at the beginning of class to introduce or reinforce material to students.

Figure 3. SCI 101 student Poster Project scores from Fall 2014 (before course changes) to Fall 2016 (after course changes were implemented). N = 338.

Immediate and Positive Effects

The changes I made to SCI 101 have had an immediate and lasting positive effect in increasing student learning in the course. The assessments and subsequent course changes resulted in increased performance on the UULOs (Figure 1), and higher marks on the posters (Figure 3). After making the assignments more transparent to students (Table 1) and using the pre-tests (Figure 2), I more effectively reached the underserved student populations, and help student performance improve at all ability levels (Figure 3).

Lecture 12 Pre-Test: Scientific Evidence Think of the following question: Is it ethical for the government to restrict sugar consumption? List four pieces of evidence that would convince your reader that this practice is a) unethical; or b) ethical.


7

The modified, improved exercises constitute successful assignments that better serve English-language learning, First-Generation, and under-prepared students. By providing these populations with higher-quality work samples and explicit instructions, SCI 101 students know exactly what is expected of them, have a similar opportunity to complete the assignments successfully, and more significantly, learn more.

To close the loop on assessment and to continuously improve SCI 101, in Spring 2017 I started using Top Hat, an interactive teaching platform that enables students to answer questions in real-time using their smart phones, tablets, or laptops during lectures (Appendix C). Using Top Hat, I embed questions in my lecture slides that students answer in class. The responses allow me to immediately assess students’ understanding of the material, and to explain areas that the students find challenging or unclear in greater detail or in a different manner, which results in greater understanding of the concepts and increased student learning. Additional assessment at the completion of this semester will help me to determine whether combining Top Hat with the course changes I have already implemented further improves student achievement.

The Fall-to-Fall retention of new, full-time freshmen who took SCI 101 in Fall 2015 and re-enrolled at UNLV in Fall 2016 was 83.8%, the highest percentage among all Colleges in the university. This finding suggests that the changes implemented in SCI 101 as a result of ongoing assessment, and the skills that students acquired in the course have contributed to a noticeable improvement in learning and achievement of our undergraduate students.


8

Appendix A. Revised Journal 4 assignment.

Journal 4: Topic for Scientific Ethics Research Poster

Purpose: The purpose of this assignment is to help you begin working on your ethics project by identifying a topic and considering the possible evidence.

Skills: You’ll practice identifying and summarizing an ethical question in science.

Knowledge: You’ll gain preliminary knowledge about the potential sides of an ethical scientific problem and the places where you’ll find more evidence.

Task: Answer the following questions. You will be graded on the completion of your answers.

1. Identify the ethical question that is being asked (Example: Is it ethical to use antibiotics in poultry production?)

2. Why are you investigating this topic? What prompted you to research this? Is there controversy surrounding this issue?

3. List the evidence you anticipate to find in support of and in opposition of your question. In Opposition (Unethical) In Support (Ethical) Example: Example: 1. Creates superbugs (SPR) 1. Prevents infection among animals (SPR) 2. Chickens unhealthy (SPR) 2. Chickens are healthier (SPR) 3. Poor working conditions (Pop) 3. Safer working conditions (Pop) 4. Passed through the food chain (SPR) 4. Low amounts in food not dangerous (SPR) 5. Antibiotics ineffective when needed (SPR) 5. Antibiotics required for industry (Pop) 6. Illegal in European countries (NSPR) 6. Legal in the United States (NSPR)

4. Examine the potential pieces of evidence listed in #3 above. Identify whether they will likely be from popular (Pop), scientific peer-reviewed (SPR), or non-scientific (e.g., law journals) peer-reviewed (NSPR) sources, and note each statement above as (Pop), (SPR), or (NSPR). Do you think there will be enough scientific evidence from peer-reviewed articles to make a valid and convincing scientific conclusion? Why or why not? Example: I think there will be enough scientific evidence to show that the antibiotics are dangerous (unethical) because they are found in the processed foods that humans consume, they contribute to antibiotic resistance, and they are not necessary if the poultry is given adequate living conditions. I think there will be enough evidence to show that antibiotics are also necessary (ethical) because they are helping to keep the poultry healthy when they are in tightly packed living conditions, they help produce more food, and the low amounts that are found in the food are not dangerous to humans.


9

Appendix B. Example of a student poster on scientific ethical research. The poster earned 147 of 150 points.


10

Appendix C. Example of a Top Hat question and student responses.

Match the following characteristics with the appropriate type of article, “Popular” or “Scientific Peer-reviewed.”

Most Popular Matches

Question

Match the Article Characteristic

Type of Article (Scientific or Peer-reviewed)

Number of

Correct Student

Responses

1

Analyzed and accepted by other experts in the field

Scientific Peer-reviewed

19

2

Reviewed by an editor (or by no one)

Popular

16

3

Original research with data and graphs


18

4

Vague description of research and meaning

Popular

18

5

Opinion

Popular

18

6

Facts with scientific evidence


15

Assessment Reports

11

2014-2015 Summary of Critical Thinking UULO Assessment SCI 101: Introduction to the University for Science Majors

The Ethics Poster Project assignment was chosen for the assessment because it included aspects of all of the Critical Thinking UULOs. The following UULOs were assessed:

2.1: Identify Problems 2.2: Documentation of sources 2.3.1: Quantitative and qualitative analysis 2.3.2: Interpretation of data and results 2.4: Analysis from different perspectives 2.5: Further inquiry 2.6: Conclusions The UULOs were assessed in 10% of randomly-selected student posters in Fall

2014 (n = 40) and Spring 2015 (n = 16). The UULOs were assessed using the grading rubric on a scale of 2 points (Beginning), 3 points (Developing), 4 points (Proficient), to 5 points (Mastered).

The overall UULO mean score in Fall was 3.37 (Developing) and 2.93

(Beginning to Developing) in Spring. Students performed best on UULO 2.1, with a Fall mean score of 3.98 and a Spring mean score of 3.19. Students performed lowest on UULO 2.3.2, with a Fall mean score of 3.38 and a Spring mean score of 2.63. For reasons that are unclear, the student performance on the mean UULO scores between Fall and Spring decreased. Assessment will continue to be conducted for Fall 2015 and Spring 2016 semesters using the same methods as above.

12

General Education Assessment Report Template

Academic Year: 2015-2016 Course Name/Catalog Number: SCI 101 General Education Component: First-Year Seminar UULO(s) assessed this year:

☐ Intellectual Breadth/Life-long Learning X☐ Inquiry/Critical Thinking ☐ Communication ☐ Global/Multicultural Knowledge and Awareness ☐ Citizenship & Ethics

Other learning outcomes assessed this year: Clickheretoentertext. Process: Please provide a brief narrative of the assessment process for this course. Include a description of the type of student work assessed (e.g., research papers, exams, etc.), the number and roles of people involved in the process, any tools used for the assessment (e.g., checklists, rubrics, etc.), and how student learning was evaluated. In SCI 101, students work in teams of two researching a scientific question throughout the semester. At the end of their research, they create a scientific poster and present it to the class. The UULOs were assessed in 10% of randomly-selected student posters from each SCI 101 section for Fall 2015 (n = 26) and Spring 2016 (n = 16). The UULOs were assessed using the grading rubric on a scale of 2 points (Beginning), 3 points (Developing), 4 points (Proficient), to 5 points (Mastered). The posters were assessed by Alison Sloat, the SCI 101 coordinator, using the rubric that is used to grade the student posters. The seven Critical Thinking UULOs (2.1: Identifying a Problem; 2.2: Documentation of Sources; 2.3.1: Analysis; 2.3.2: Interpretation of Data and Results; 2.4: Analysis from Perspectives; 2.5: Identification of Future Work; and 2.6: Conclusions) are specified in the rubric and individually assessed. The scores were compared to previous semester’s scores.

13

Results: Please provide a brief summary of the results of your assessment process. Include both what you learned about your students’ achievement of the specified learning outcomes and what you learned about the assessment process itself, if applicable. Mean student achievement on the Critical Thinking UULOs between Fall 2014 to Spring 2016 is shown in Figure 1. Because of the demographics of the students, comparing Fall-Spring student scores is not as effective as comparing Fall-Fall student scores. In Fall 2015, students did not perform as well as the previous Fall 2014, particularly in the data analysis, conclusions, and future work categories. Some additional changes were made in the course in Spring 2016 including the implementation of Pre-Tests across all sections. These Pre-Tests are designed to spur the interest of the student and to help in retention of the information taught during lecture. Student performance in the data analysis, conclusions, and future work improved in Spring 2016 after the implementation of these Pre-Tests in all sections. Students still struggle and could use improvement in identifying future research and work that is needed (UULO 2.5). This is a crucial component of the research process because it shows the students clearly understand the current state of research in that particular field of study, they can think critically about the current state of the research, and they can identify data gaps and/or future research that is needed in the area. This is a component of the course that will be specifically addressed in greater detail starting Fall 2016.

14

Figure 1. SCI 101 performance on Critical Thinking UULOs from Fall 2014 to

Spring 2016. Conclusions: Please describe how the results of this assessment process might be used to revise instruction in this course and/or refine the assessment process in future years. After analyzing the current state of research, students still need improvement in identifying future research needs. Starting in Fall 2016, we will focus on this UULO in greater detail. We will also continue to work on improving UULO 2.2, which is the proper documentation of sources. In certain sections, students in the Spring semester consistently made the same mistakes, which will be addressed for Fall 2016.

3.98

3.18 3.23 3.38 3.333.13

3.403.19 3.13 3.00

2.632.88

2.693.00

4.42

3.08 3.213.00 3.13

2.793.08

4.31

3.44

3.943.63 3.75

3.06

3.81

0

1

2

3

4

5

UULO2.1

UULO2.2

UULO2.3.1

UULO2.3.2

UULO2.4

UULO2.5

UULO2.6

UULO

Score

Cri,calThinkingUULO

Cri,calThinkingUULOAssessment:Fall2014toPresent

Fall2014Mean

Spring2015Mean

Fall2015Mean

Spring2016Mean

Documents

UNIVERSITY OF NEVADA LAS VEGAS Awards/Alison_Sl… · University of Nevada, Las Vegas Dear Dr. Pritchard, I hereby nominate Dr. Alison Sloat, Assistant Professor-in-Residence, College