Each paddler in the canoe has a significant role in the success of the voyage. Presenters Patricia Ishimaru and Kua‘ana Lewis, Assessment Section 1

Each paddler in the canoe has a significant role in the success of the voyage. Presenters Patricia Ishimaru and Kuaana Lewis, Assessment Section 1

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Canoe Paddlers RolesSchool Team Members 1st Paddler: Keeps focused on the front of the canoe and is responsible for setting the pace 2nd Paddler: Calls out changes to the paddling sides 3rd and 4th Paddlers: Powerhouse paddlers who are the strongest members of the team 5th Paddler: Supports the 1st through 4 th paddlers and provides extra strength when a moment of weakness may occur 6th Paddler: Steersperson who is the overall captain and is responsible for keeping the canoe on course toward its destination and to provide the necessary encouragement and support for the team Alternate Paddlers: Support other paddlers who may need assistance Test Coordinator Curriculum Coordinator Test Administrators Technology and Student Services Coordinators Principal or Vice Principal Other school colleagues, Assessment Section staff, and American Institutes for Research (AIR) Help Desk Staff 2

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Good News for Public and Public Charter Schools! Students, test administrators, test coordinators, and technology coordinators will use the same online systems that were provided for the Hawaii State Reading, Mathematic, and Science Assessments during the past four years for the spring 2015 Smarter Balanced ELA/Literacy and Mathematics Assessments with a few upgrades. This means that most of the language and administration procedures used for the Smarter Balanced practice tests, training tests, interim assessments, and summative assessments will be familiar to previous users. New users will be able to consult with other staff members at their schools who are knowledgeable about the online systems while they are learning how each system operates. 3

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Test Administration Manual Pages 6-7 provide Table 1: User Roles in the Online Testing System and descriptions of their key responsibilities. Principal Test Coordinator (TC) Test Administrator (TA) Teacher (TE) Checklists for the first three role groups are provided in Appendices J, K, and L on pages 73, 76, and 81. Use of these checklists is a school decision. The Assessment Section staff is currently updating the links to documents and websites for some customized Hawaii activities that we inserted in the Smarter Balanced Principals, Test Coordinators, and Test Administrators checklists during fall 2014 and also in the Technology Coordinators, Curriculum Coordinators, and Student Services Coordinators checklists that we created for these three role groups. We will post these customized Hawaii checklists for the six role groups on the Smarter Balanced Resources page at alohahsap.org by Friday, January 23, 2015. Use of these customized Hawaii checklists is a school decision. 4

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Pages 7-8 provide Table 2: Personnel Who May Serve as Test Administrators and the related requirements. General Education Teacher Special Education Teacher School Counselor Instructor in content areas where there is a shortage Long-term substitute teacher Identified public charter school employees Test Coordinator 5

Important Online Training Materials for Test Administrators Test Administration Manual Usability, Accessibility, and Accommodations Guidelines Test Administrator User Guide Guide to Navigating Online HSAP Administrations (revised title) Associated Training Modules Complete Online TA Certification Course The first four online training materials listed above can be accessed on the Smarter Balanced Resources page at alohahsap.org. The Online TA Certification Course can be accessed at the same website by clicking on the Training Sites icon on the Test Coordinators/Administrators page. 6 Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments

Pages 9 - 10 provide Table 3: Additional Resources for the Summative Assessments with descriptions. TIDE User Guide Braille Requirements and Testing Manual Secure Browser Installation Manual Technical Specifications Manual for Online Testing Test Administrator User Guide Guide to Navigating Online HSAP Administrations (revised title) Online Reporting System User Guide Usability, Accesssibility, and Accommodations Guidelines 7

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Pages 10 11 provide Table 4: Training Modules and Online TA Certification Course for Different Student Role Groups Online TA Certification Course (TA, TC) Accessibility and Accommodations Module (TA, TE, TC) Embedded Universal Tools and Online Features Module (Students, TA, TE) Performance Task Oveview Module (TC, TA TE) Test Administration Overview Module (TC, TA, TE) Student Interface for Online Testing Module (Students, TC, TA) Technology Requirements for Online Testing Module (Tech Coordinator) Test Administrator (TA) Interface for Online Testing Module (TC, TA) TIDE (Test Information Distribution Engine) Module (TC, TA) Online Reporting System Module (TC, TA, TE) What Is a CAT (Computer Adaptive Test)? Students, TC, TA) 8

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Pages 22 31: General Test Administration Information Page 22: Assessment Participation All students enrolled in grades 38 and 11 are required to participate in the ELA/Literacy and Mathematics Assessments except: Students with the most significant cognitive disabilities who meet the criteria for the state-developed ELA/Literacy and Mathematics Alternate Assessments based on alternate achievement standards (approximately one percent or fewer of the student population). The Hawaii Department of Education's participation rule for the ELA/Literacy Assessment will be provided before the opening of the grades 3-8 testing window on March 10, 2015 for ELL program students who enrolled in a U.S. school for the first time within the last twelve months prior to the beginning of testing in that school or another school. All ELL program students are required to take the Mathematics Assessment. 9

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 22: Attemptedness Rules for Participation These rules will be provided before the opening of the grades 3-8 testing window on March 10, 2015. Page 23: General Rules of Online Testing This section provides a brief overview of the general test administration rules for different portions of each content area assessment as well as information about test tools and accommodations. For more information, refer to the Test Administrator User Guide Guide to Navigating Online HSAP Administrations (revised title) located at alohahsap.org on the Smarter Balanced Resources page. 10

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 23: Basic Online Testing Parameters Computer Adaptive Test (CAT) and Performance Task (PT) will be presented as separate tests. Students may not return to a test once it has been completed and submitted. Within each test there may be segments. For example, the grades 6 - 8 and 11 Mathematics CAT test includes a segment with an embedded calculator available and another segment where the embedded calculator is not allowed. A student may not return to a segment once it has been completed and submitted. Students must enter an answer for all items on a page before going to the next page. Some pages contain multiple items. Students may need to use the vertical scroll bar to view all items on a page. Students may mark items for review and use the Past/Marked Questions drop- down list to return to those items within a segment. 11

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments 5 Minutes for Questions and Answers What questions do you have regarding the information we have just reviewed on selected pages in the Smarter Balanced Test Administration Manual? If we do not have time to answer all of the questions during this 5 minute time frame, you may submit additional questions to Patricia Ishimaru via the Departments Lotus Notes email system or patricia_ishimaru @notes.k12.hi.us for non-Lotus Notes users. 12

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 23: Pause Rules During the Computer Adaptive Test (CAT) portion of the test: If a test is paused for more than twenty minutes the student is: Required to log back in to the student interface; Presented with the page containing the item(s) he or she was working on when the test was paused (if the page contains at least one unanswered item) OR with the next page (if all items on the previous page were answered); and NOT permitted to review or change any previously answered items, even if they are marked for review (with the exception of items on a page that contains at least one item that was not answered yet). 13

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 23: CAT Pause Rules (Continued) Any highlighted text and notes on the digital notepad which is an embedded universal tool will not be saved when a test is paused regardless of how long the test is paused. In the event of a technical issue (e.g., power outage or network failure), students will be logged out and the test will automatically be paused. The students will need to log in again upon resuming the test. See Appendix E: CAT Pause Rules Scenarios to review the rules that govern pausing during the test. 14

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Pages 23-24: Performance Task (PT) Pause Rules There are no pause restrictions. If a test is paused for twenty minutes or more, the student can return to the section and continue entering his or her responses. Any highlighted text and notes on the digital notepad which is an embedded universal tool will not be saved when a test is paused regardless of how long the assessment is paused. In the event of a technical issue (e.g., power outage or network failure), students will be logged out and the test will automatically be paused. The students will need to log in again upon resuming the test. 15

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 24: Test expiration Computer Adaptive Test (CAT) A students CAT remains active until the student completes and submits the test or forty-five calendar days after the student has begun the test, whichever occurs sooner. However, it is recommended that students complete the CAT items portion of the test within five days of starting the designated content area. Performance Task (PT) The PT is a separate test that remains active only for ten calendar days after the student has begun the PT. However, Smarter Balanced recommended that students complete the PT within three days of starting it in each content area. A summary of recommendations for the number of sessions and session durations is provided in section 7.3 Testing Time and Recommended Order of Administration. 16

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 25: Smarter Balanced Summative Assessment Testing Windows Grades 38 Testing shall not begin until at least sixty-six percent (66%) of a schools annual instructional days have been completed, and Testing may continue up to and including the last day of school. Grades 3-8: March 10, 2015 - June 3, 2015 (10 weeks) Grade 11 Testing shall not begin until at least eighty percent (80%) of a schools annual instructional days have been completed, and Testing may continue up to and including the last day of school. Grade 11: April 13, 2015 June 3, 2015 (7 weeks) 17

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments The information provided below is posted on the Smarter Balanced Resources page in the General Information section at alohahsap.org and is not included in this Test Administration Manual A Parent Information Booklet (PIB) will be available in English and in 14 languages during mid-February 2015 on the Smarter Balanced Resources page. The PIB will include information for the grades 3-8 and 11 Smarter Balanced ELA/Literacy and Mathematics Assessments and the grades 4 and 8 Hawaii State Science Assessments. Printed copies of one English PIB per student and one translated PIB for identified ELL students whose parents have indicated their home language is a language other than English will be shipped to each school during late February or early March 2015. Parent Letters will be available in English and 14 languages for 12 student populations on the Smarter Balanced Resources page in February 2015. The Parent Letters will include information for the grades 3-8 and 11 Smarter Balanced ELA/Literacy and Mathematics Assessments and the grades 4 and 8 Hawaii State Science Assessments. 18

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Pages 25 26 Table 9: Estimated Testing Times for Smarter Balanced Assessments Classroom Activities are designed to fit into a thirty-minute window; however, the time within the window will vary due to the complexity of the topic and individual student needs. When developing a testing schedule, use the estimated testing times to calculate the number of days and the amount of time it will take to complete an assessment in each content area and grade level. The number of available computers and TAs also needs to be considered. These estimates do not account for any time needed to start computers, load secure browsers, and log in students. Nor do they account for breaks. TAs should work with TCs to determine precise testing schedules. Recommended order of online administration The assessments are comprised of two components (tests) for ELA and mathematics: a CAT and a PT. All PTs must be preceded by the administration of a Classroom Activity. Smarter Balanced recommends that students take the CAT and PT items on separate days. For each content area, Smarter Balanced also recommends that students begin with the CAT items, followed by the Classroom Activity, and then the PT. Schools may opt to administer in a different order if needed; however, the Classroom Activity, which is designed to introduce the PT, must occur prior to the PT. 19

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 27: Classroom Activity The purpose of the Classroom Activity is to introduce students to the context of a performance task so they are not disadvantaged in demonstrating the skills the task intends to assess. Classroom Activities do not address content information; instead, they focus on vocabulary and key contextual topics. The Classroom Activity is designed to be an introduction and not an assessment. The Classroom Activity is non-secure; however, it should not be supplemented with any other content that the teacher may think is helpful. Supplementing the Classroom Activity may detract from the intended purpose of the Classroom Activity and is not advised. 20

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 28: Duration and timing information for ELA/Literacy and Mathematics The scheduling/rules for the CAT, Classroom Activity, and PT components are included in Table 10 on page 28 and Table 11 on page 29. Note that the duration, timing, break/pause rules, and session recommendations vary for each content area and component. Page 30: Important reminders The number of items will vary on the CAT portion of each students test. The tests are not timed, so all time estimates are approximate. Students should be allowed extra time if they need it, but test administrators need to use their best professional judgment when allowing students extra time. Students should be actively engaged in responding productively to test questions. The test can be spread out over multiple days as needed. The Classroom Activity must be completed prior to administration of the PT. Inadvertently administering the PT before the Classroom Activity is considered a testing irregularity. 21

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 34: Establishing Appropriate Testing Conditions Test Coordinators (TCs) and Test Administrators (TAs) will need to work together to determine the most appropriate testing option(s) and testing environment based on the number of computers available, the number of students in each tested grade, and the estimated time needed to complete each test. Testing students in classroom-sized groups is preferable. Establishing classroom-sized groups reduces test fear and anxiety for the students and facilitates monitoring and control for the TA. 22

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 34: Establishing Appropriate Testing Conditions (Continued) Establish procedures to maintain a quiet testing environment throughout the test session, recognizing that some students will finish more quickly than others. If students are allowed to leave the testing room when they finish, explain the procedures for leaving without disrupting others and where they are expected to report once they leave. If students are expected to remain in the testing room until the end of the session, instruct them on what quiet activities they may engage in after they finish the test that may not include use of their computers to access online instructional programs. 23

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 34: Establishing Appropriate Testing Conditions (Continued) Test Coordinators (TCs) and Test Administrators (TAs) will need to work together to determine the most appropriate testing option(s) and testing environment based on the number of computers available, the number of students in each tested grade, and the estimated time needed to complete each test. Testing students in classroom-sized groups is preferable. Establishing classroom-sized groups reduces test fear and anxiety for the students and facilitates monitoring and control for the TA. 24

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 34: Establishing Appropriate Testing Conditions (Continued) During test administration, students may have access to and use of additional required resources listed in Table 13 specific to each assessment and content area. ELA Headphones for Listening portion and text-to-speech Scratch paper Mathematics Online calculator for grades 6-8 and 11 only - Grade 6: four function - Grades 7-8: scientific - Grade 11: scientific/graphing/regression Scratch Paper Graph Paper 25

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 37: Day of Test Administration Use the following information and script to assist students with the login procedures. Please refer to the Test Administrator User Guide Guide to Navigating Online HSAP Administrations (revised title) to become familiar with the Online Testing System. RECOMMENDATION: Consider printing this section (pages 37-45) to be used on the day of testing for both the computer adaptive test (CAT) and performance task (PT) portions of each content area test because the script (directions) that TAs must read to students are included. 26

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 37: Day of Test Administration (Continued) To ensure that all students are tested under the same conditions, the TA should adhere strictly to the script for administering the test. These instructions can be found in the boxes in bold. When asked, the TA should answer questions raised by students but should never help the class or individual students with specific test items. No test items can be read to any student for any content area, unless specified as an accommodation as listed in the Usability, Accessibility, and Accommodations Guidelines. Please remember that the script must be followed exactly and used each time a test is administered. If the class is resuming a test and the TA is sure that all students are able to log in without hearing the login directions again, the TA may skip the italicized portions of the directions. 27

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 48 51 APPENDIX A: FREQUENTLY USED TERMS Table 14 defines terms that are specific to the Smarter Balanced assessments. Page 52-56 APPENDIX B: ITEM TYPES Item and response types As students engage with the Smarter Balanced assessments, they will be asked test questions that require them to respond in several ways, some of which may be new to students. Smarter Balanced has produced several resources that teachers and students can use to get ready for the test, including a Practice Test and a Training Test. It is highly recommended that ALL students access the Training Test site before taking the test. Doing so will provide students an opportunity to view and practice each of the item types. 28

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 13: ENSURING TEST SECURITY All summative test items and test materials are secure and must be appropriately handled. Secure handling protects the integrity, validity, and confidentiality of assessment items, prompts, and student information. Any deviation in test administration must be reported as a test security incident to ensure the validity of the assessment results. 30

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Pages 13-14: Security of the Test Environment Table 6 describes security requirements for the test environment before, during, and after testing. The test environment refers to all aspects of the testing situation while students are testing and includes what a student can see, hear, or access (including access via technology). [Note: The Classroom Activities are not secure materials.] TAs and TCs or other individuals who have witnessed, been informed of, or suspect the possibility of a test security incident that could potentially affect the integrity of the assessments or the data should follow the steps outlined in section Responding to Testing Improprieties, Irregularities, and Breaches in this manual on pages 16-20. 31

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 16: Definitions for Test Security Incidents Impropriety (Low Severity) An unusual circumstance that has a low impact on the individual or group of students who are testing and has a low risk of potentially affecting student performance on the test, test security, or test validity. These circumstances can be corrected and contained at the state level and do not need to be reported to the Consortium. Page 62: Examples Student(s) making distracting gestures/sounds or talking during the test session that creates a disruption in the test session for other students. Student(s) leave the test room without authorization. Administrator or Coordinator leaving related instructional materials on the walls in the testing room. 32

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 16: Definitions for Test Security Incidents (Continued) Irregularity (Medium Severity) An unusual circumstance that impacts an individual or group of students who are testing and may potentially affect student performance on the test, test security, or test validity. These circumstances can be corrected and contained at the state level and do not need to be reported to the Consortium. Page 62: Examples Student(s) cheating or providing answers to each other, including passing notes, giving help to other students during testing, or using hand-held electronic devices to exchange information. Student(s) accessing the Internet or any unauthorized software or applications during a testing event. Student(s) accessing or using unauthorized electronic equipment (e.g., cell phones, PDAs, iPods, or electronic translators) during testing. 33

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Page 16: Definitions for Test Security Incidents (Continued) Breach (High Severity) An event that poses a threat to the validity of the test. Examples may include such situations as a release of secure materials or a security/system risk. These circumstances have external implications for the Consortium and may result in a Consortium decision to remove the test item(s) from the available secure bank. Page 62: Examples Administrator or Coordinator modifying student responses or records at any time. Adult or student posting items or test materials on social media (Twitter, Facebook, etc.). Adult or student copying, discussing, or otherwise retaining test items, reading passages, writing prompts, or answers for any reason. This includes the use of photocopiers or digital, electronic, or manual devices to record or communicate a test item. 34

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Lets check our understanding of the new Smarter Balanced test security requirements. What is the severity level of actual testing incidents that occurred in some of our schools during the last two school years? Impropriety (Low Severity) Irregularity (Medium Severity) Breach (High Severity) 35

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments A test administrator helped Special Education students by pointing to the correct answers on their computer screens. Severity Level? _________________________________ The teachers decided to post instructional strategies charts in each of their testing rooms on the morning of a test session and instruct students to use the strategies while answering questions. Severity Level? _________________________________ A test administrator directed some students to sit next to smarter students and discuss answers to online questions before selecting responses. Severity Level? _________________________________ A teacher used her own money to pay each student $5 if he/she passed the test. Severity Level? _________________________________ 36

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments A student was singing loudly during a test session about how dumb the test was and would not stop singing when told to do so three times. Severity Level? _________________________________ A student used a camera phone to take screen shots of questions in a paper/pencil test booklet and posted them on Instagram. The students friends at other schools saw the questions when they were at school or at home and informed their teachers or parents. The principals at the schools of the students friends were informed. Severity Level? _________________________________ Three students who were friends used sign language in the testing room to give each other answers to different questions that were displayed on each student's computer screen. By leaning far to the right or left in the computer lab, they could see the questions on each others' screens. They kept their hands in their laps while forming the letter A, B, C, or D to indicate the answer to multiple-choice questions. Severity Level? _________________________________ 37

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments A February 28, 2014 Department of Education Memo titled Consequences for Students and School Staff Who Use Electronic Devices to Breach the Security of Any Test Item Included in a Statewide Student Assessments is still in effect for 2014-15. Due to the importance of test security for all of the statewide assessments, any school that has a student or staff member who uses an electronic device which results in the breach of a test item will be subject to the following actions/consequences: 1.The involved students incomplete test or complete test that has been scored will be invalidated by the Department of Educations Student Assessment Section. 2.The school administration will be responsible for determining the consequences for the involved student per the Hawaii Administrative Rules, Title 8, Chapter 19 and/or the consequences for the involved certificated or classified staff member per the Office of Human Resources procedures. 3.The school administration and test coordinator must retrain the staff regarding the test security requirements for all statewide assessments during a face-to-face meeting. 4.The test administrator who was conducting the test session when an electronic device was used by one or more students or a staff member will be required to retake the Test Administrator Certification Course in order to administer any additional assessments to students. 38

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments List of Consequences Included in the February 28, 2014 Memo Continued 5.The school will be required to have a second adult present in the testing room with the test administrator who was conducting the test session when the electronic device was used to breach the security of one or more test items during each subsequent test session for the remainder of the current school year. 6.The school will be required to pay for the cost of each test item that was posted on a social networking site which required its removal from the current online, adaptive item bank. 7.The school must submit its detailed testing schedule that includes the date, time, and student seating chart for each test session for all statewide assessments administered during the remainder of the current school year and the next school year. 8.Unannounced site monitoring by the Complex Area Superintendents identified staff and the Systems Accountability Offices staff as determined by Superintendent Matayoshi may extend beyond one school year. 39

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments Who can assist me when I have questions throughout the school year? Questions regarding test administration and security procedures Patricia Ishimaru, Assessment Section (808) 733-4100 Lotus Notes: Patricia Ishimaru Other Users: [email protected] Questions regarding the use of online testing systems Hawaii Statewide Assessment Program (HSAP) Help Desk 1-866-648-3712 (Monday Friday, 7:30 a.m. 4:00 p.m. HST) [email protected] 41

Teaming for a Successful Administration of Smarter Balanced Summative ELA/Literacy and Mathematics Assessments 42 When we value each other and work together in harmony and unity, any task, whether big or small, becomes easier and more successful and makes the voyage so much more enjoyable and purposeful. WE DO OUR BEST FOR THE SUCCESS OF OUR STUDENTS AND THEIR FAMILIES!

Documents

Each paddler in the canoe has a significant role in the success of the voyage. Presenters Patricia Ishimaru and Kua‘ana Lewis, Assessment Section 1