MSA Toolkit Sept 2014

LISA PULLMAN, INDEX EXECUTIVE DIRECTOR JONATHAN E. MARTINAND THE INDEX MISSION SKILLS STEERING COMMITTEE

SEPTEMBER 2014

Mission Skills Assessment (MSA)USER’S GUIDE AND TOOLKIT

Welcome

Table of ContentsWELCOMEIntroduction ..................................................................................................................................................4

STRATEGIESCase Study: New Canaan Country School ...................................................................................................710 Steps to a Successful MSA ...................................................................................................................10Case Study: Far Hills Country Day School .................................................................................................12Administering the MSA: Communication Tips ............................................................................................16Administering the MSA: Managing the Testing ..........................................................................................18Case Study: Lexington School ...................................................................................................................20Reading and Sharing Your Report .............................................................................................................24Case Study: Shady Hill School ...................................................................................................................27Using the MSA to Improve Student Learning Outcomes in the Six Skills: A Preliminary Guide ................30

UNPACKING THE SIX SKILLSThe MSA Crosswalk ...................................................................................................................................36MSA Indicators ...........................................................................................................................................37Assessing the Skills ....................................................................................................................................38Mission Skills Research ..............................................................................................................................41

APPENDIXFrequently Asked Questions for Educators ................................................................................................47Frequently Asked Questions for Parents ....................................................................................................51Checklist for School MSA Program Administrators ....................................................................................53Elevator Speech .........................................................................................................................................55Suggested Additional Reading and Resources .........................................................................................56Acknowledgements ....................................................................................................................................58Data Informed Decision Making: A Primer ................................................................................................59

"MM�TUBUFNFOUT�FYQSFTTFE�JO�UIJT�BSUJDMF�BSF�UIF�BVUIPST��BOE�EP�OPU�SFnFDU�UIF�PGmDJBM�PQJOJPOT�PS�QPMJDJFT�of the Educational Testing Service.

MSA is owned and managed by INDEX. www.indexgroups.org

http://www.indexgroups.org

WEL

COM

E

4 ©2014 INDEPENDENT SCHOOL DATA EXCHANGE. ALL RIGHTS RESERVED.

IntroductionWelcome to the Mission Skills Assessment User’s Guide and Toolkit. A commitment to developing character, social and emotional skills, and 21st century competencies can be found in the missions of many schools of all types. In addition to academic learning, schools proclaim their commitment to developing students in becoming MJGF�MPOH�MFBSOFST �TLJMMGVM�DPMMBCPSBUPST �NPSBM�JOEJWJEVBMT �DPOmEFOU�BOE�QFSTJTUFOU�QSPCMFN�TPMWFST �PSHBOJ[FE�and conscientious students, and innovative thinkers. However, far too often, the academic side of their mission takes center stage, at least in part because only intellectual development has been able to be effectively measured -- until now.

5IJT�JT�UIF�PQQPSUVOJUZ�BOE�UIF�QPXFS�PG�UIF�.4"��/PX�TDIPPMT�DBO�MFWFM�UIF�QMBZJOH�mFME �NFBTVSJOH�UIF�GVMM�breadth of what matters, with evidence-based, research-tested, assessments. The MSA is an important and JOOPWBUJWF�QSPKFDU��*U�XJMM�BTTFTT�PVS�JOTUJUVUJPOBM�FGGFDUJWFOFTT�BOE�QSPWJEF�TDIPPMT�XJUI�PCKFDUJWF �TDJFOUJmD�GFFE-back and useful data on how well our programs develop essential skills in our students: Teamwork, Creativity, Ethics, Resilience, Curiosity, and Time Management. Research shows that these skills are integral for success in the 21st century workplace and that these skills can be taught and built over time.

When the RAND Corporation in 2013 combed the nation to identify the very best tools available for “measuring 21st century competencies” for inclusion in a report of that name, they selected only one for a “deeper look”: the MSA. As the report explains, it was chosen because it was especially innovative, cost-effective, and well “safeguarded” against “threats to reliability and validity.”

The MSA originated in conversations among committee members of the Elementary Schools Research Col-laborative (now INDEX.) If the educational excellence of their schools was rooted in both academic growth and character development equally, shouldn’t both be measured with equal emphasis? Unsure about the potential, but eager to explore, the committee selected ETS and its Center for Academic and Workforce Readiness and Success to do the research, and twenty INDEX member schools volunteered to participate in the pilot. To deter-mine the particular skills to be measured, the committee, working with ETS, reviewed the mission statements of BMM�QJMPU�QBSUJDJQBUJOH�TDIPPMT �BOE�BGUFS�B�MFOHUIZ�EJTDVTTJPO�BOE�EFCBUF �BSSJWFE�DPOTFOTVBMMZ�BU�UIF�mSTU�mWF��ETS researchers then suggested adding time management as a sixth, because of their prior work and success with that construct. ETS drew upon its long history of assessment to design a multi-method approach incorporat-ing and “triangulating” student self-assessments, teacher assessments of each student, and a student perfor-mance task.

/PX �JO�� OFBSMZ��TDIPPMT�BDSPTT�UIF�64�BOE�JOUFSOBUJPOBMMZ�BSF�QBSUJDJQBUJOH�JO�UIF�.4"�BOE�BSF�mOEJOH�JU�valuable in many different ways. Some have joined to support the research project the MSA represents: gleaning from the data of thousands of middle school students which skills are most correlated with academic success and life satisfaction and which interventions most effectively improve these outcomes. Others are looking to study their school’s data, longitudinally over years, to inform their program development, identify priorities, and measure the impact of their initiatives. Some want to ensure they have data of equal (or greater) validity BOE�SFMJBCJMJUZ�UP�NBUDI�VQ�UP�UIFJS�BDBEFNJD�UFTUJOH �UP�FWFO�UIF�QMBZJOH�mFME�JO�UIF�XBZ�UIFZ�FWBMVBUF�UIFJS�educational programs in an era when we know “what gets measured gets done.” Some schools have the intent in time to be able to demonstrate publicly the quality of their character education, to be able to tell the world

WELCO

ME

5MISSION SKILLS ASSESSMENT USER’S TOOLKIT

their program is among the best of its kind (though the guidelines of the MSA prohibit using it to disparage or denigrate other schools.)

But the most common use of the MSA is to position it in support, or as the centerpiece, of school-wide character education. Schools show how the MSA is related to other social and emotional learning activities underway, and use the MSA constructs, materials, network, and resources to inform and guide the ongoing work in those other activities. Some embrace this wholeheartedly and use it to drive a full-blown Mission Skills Program. This CFDPNFT�UIFJS�QMBUGPSN��B�DPIFSFOU �UBSHFUFE �mOJUF�TFU�PG�TLJMMT�VQPO�XIJDI�UIFZ�GPDVT�UIFJS�DIBSBDUFS�FEVDBUJPO��

'PS�NBOZ�QBSUJDJQBOUT �UIF�NPTU�WBMVBCMF�CFOFmU�PG�BENJOJTUFSJOH�UIF�.4"�JT�UIF�QPXFS�PG�QBSUJDJQBUJOH�JO�UIF�OFUXPSL�PG�.4"�TDIPPMT �DPNJOH�UPHFUIFS�JO�DPOGFSFODF �TIBSJOH�SFTPVSDFT�BOE�QSBDUJDFT �CFOFmUUJOH�GSPN�UIF�JOTQJSBUJPO�BOE�BGmSNBUJPO�PG�TIBSFE�QVSQPTF�BOE�DPMMFHJBMJUZ��

This publication is intended to be a comprehensive user’s guide and resource kit for schools already using, CFHJOOJOH�UP�VTF �PS�DPOTJEFSJOH�VTF�PG�UIF�.4"��*U�JT�CPUI�UBDUJDBM�BOE�TUSBUFHJD��8F�TIBSF�TQFDJmD�BDUJPOT�GPS�you to consider taking as you communicate about the MSA, administer the assessment at your school, and read and interpret the report you get each year. Because we believe that ultimately the point of administering the MSA is to use it to improve student outcomes, (though it is still quite early in the history of the tool), we offer suggested strategies for how the MSA might inform and empower your character education program development.

Knowing that the most effective educational phrase available in the English language is “for example,” we provide four case studies of schools. As can be seen in the table of contents, this toolkit also includes a variety of other resources, such as FAQs for educators and parents, a set of suggested readings for more information, a detailed overview of the six skills, and a short primer on data-informed decision-making. Thank you for joining, or considering joining, our collaborative effort to better understand, assess, and

develop character. We believe this work to be of the utmost importance in educating future leaders and

citizens, and we know that the more we work together, the more effective we can be in doing so.

Strategies

STRATEGIES


New Canaan Country School“It became increasingly clear to me that we were not doing a good job assessing what we believed we EP�XFMM��*�IBWF�BMXBZT�CFFO�DPODFSOFE�UIBU�XF�UFTU �DIFDL �BOE�DPOmSN�UIBU�XF�EP�XFMM�XIBU�XF�TBZ�we do well, and this was an exciting opportunity to improve this work,” New Canaan Country School Head (2000-2014) Tim Bazemore remembered thinking when raising his hand to volunteer for a committee in 2009. “I did think this might be a fool’s errand: I was not optimistic we could do this well, measuring character reliably.” The committee began scanning various articles and as they did they found particularly valuable Elena Silva’s 2008 paper for EdSector, “Measuring Skills for the 21st Century.”

%JTDVTTJOH�UIBU�QBQFS �UIF�HSPVQ�BTTJHOFE�#B[FNPSF�UP�GPMMPX�VQ�XJUI�POF�PG�UIF�FYQFSUT�4JMWB�DJUFE �3JDIBSE�Roberts, Ph.D., at ETS’s Center for Academic and Workforce Readiness and Success. A conference call fol-MPXFE �BOE�JOJUJBM�TLFQUJDJTN�UIBU�&54�TFFNFE�POMZ�B�DPHOJUJWF�UFTUJOH�PSHBOJ[BUJPO�RVJDLMZ�EJTTJQBUFE�BT�3PCFSUT�FYQMBJOFE�UIF�XJEF�FYQFSJFODF�IJT�HSPVQ�IBE�XJUI�UIJT�XPSL��#B[FNPSF��i8F�EJE�MPPL�BU�PUIFS�DPNQBOJFT�BOE�other research, but ETS seemed most accessible and applicable to what we wanted to do, and Roberts was JODSFEJCMZ�LOPXMFEHFBCMF�BOE�IFMQGVM �BTTJTUJOH�VT�FWFO�XJUI�EFmOJOH�PVS�RVFTUJPOT�BOE�HPBMT�CFGPSF�QSPDFFE-ing to the test methodology.” 'SPN�UIF�TUBSU �#B[FNPSF�DPOTVMUFE�XJUI�IJT�BENJOJTUSBUJWF�UFBN�BOE�FTQFDJBMMZ�IJT�NJEEMF�TDIPPM�EJWJTJPO�IFBE��*U�UPPL�MJUUMF�DPOWJODJOH �IF�SFDBMMFE��UIFZ�KVNQFE�PO�CPBSE��5IFZ�UIFO�TFU�PVU�BT�B�UFBN�UP�DPNNVOJDBUF�WJHPSPVTMZ�UP�UFBDIFST �QBSFOUT �BOE�TUVEFOUT�BCPVU�XIBU�UIFZ�XFSF�EPJOH�BOE�XIZ��#B[FNPSF��iJU�XBT�B�CJH�13�FGGPSU �CVU�JU�XBTO�U�IBSE��JU�SFBMMZ�TFFNFE�UP�DBQUVSF�PVS�DPNNVOJUZ�T�JNBHJOBUJPO�w�5IF�NFTTBHF�FNQIBTJ[FE�the importance of these skills in today’s and tomorrow’s workplace, and cited sources such as the Partnership for 21st Century Skills and the Business Roundtable study, “Are They Ready to Work?” He couched the assessment initiative always as a research project and a tool for continuous improvement, never as a rating or accountability tool. He remembers receiving only a few questions, mostly about individual student results (no) and the possibility of a character report card (not soon.) Teachers were shown sample questions, and time was taken to cultivate their buy-in. Some teachers were concerned about the “one more thing” problem, and sensibly the leadership UFBN�JEFOUJmFE�BO�BOOVBM�UFBDIFS�UBTL�UIFZ�DPVME�UBLF�PGG�UIF�UBCMF�BT�B�TXBQ��i1FPQMF�JOUVJUJWFMZ�VOEFSTUPPE��PG�course we should be doing this. It was not as hard as I expected, securing support.”

8IFO�UIF�UJNF�DBNF�GPS�BENJOJTUFSJOH�UIF�iUFTUw�UP�TUVEFOUT �#B[FNPSF�BOE�PUIFS�TDIPPM�BENJOJTUSBUPST�NBEF�the somewhat unusual decision to personally supervise. They did so partly to cover for teachers, giving them the

CASE STUDY

http://www.educationsector.org/publications/measuring-skills-21st-century

STRA

TEG

IES


break as an offset for their time to do their piece of the MSA, but also because they wanted to demonstrate their QFSTPOBM�JOWFTUNFOU�BT�TDIPPM�MFBEFST��*U�XBT�HPPE�UP�TFF�JU�mSTUIBOE �iJOWBMVBCMFw�#B[FNPSF�TBZT �BOE�UIFZ�were able to better brief students and motivate them to do their best. They recommend this practice strongly—it is so important for administrators to visibly demonstrate their commitment to the success of this program. 4UVEFOUT�XFSF�UPME�JO�BEWBODF�UIBU�UIJT�UFTU�XBT�WFSZ�EJGGFSFOU�GSPN�DPOWFOUJPOBM�TUBOEBSEJ[FE�UFTUJOH �BOE�XFSF�encouraged to select the answers that felt most true to themselves, not what they guessed the test-makers “were MPPLJOH�GPS�w�*O�UIF�mSTU�ZFBS�PS�UXP �UJNF�XBT�UBLFO�UISPVHIPVU�UIF�TDIPPM�ZFBS �BU�TDIPPM�NFFUJOHT �BEWJTPSJFT �and elsewhere, to explain what was meant by each of the six skills, but over time they have become so ingrained in the school that this is no longer necessary. As teachers prepared to do their part of the MSA assessment, they took the time to generate common understanding about the meaning of each rating term, such as what does “usually, often, sometimes, rarely“ mean in practice. (Please refer to the teacher test in the Demo account found in your school’s MSA Portal.) This IBT�SFBMMZ�IFMQFE�UFBDIFST�GFFM�DPNGPSUBCMF�BOE�DPOmEFOU�BT�UIFZ�SBUF �BOE�NPSF�FGmDJFOU�BT�UIFZ�DPNQMFUF�UIFJS�forms: this preparation is a good investment. “The whole experience has generated wonderful conversations about 21st�DFOUVSZ�TLJMMT�BNPOH�FWFSZPOF�IFSF��the work has become much more intentional and explicit.” As he explained in another report about the MSA at his school, “The conversation at our school has changed, because we’ve brought science and research to measure the mission skills of our school. Hallway conversations are no longer only about a storyline from a piece of literature in English class. The conversations are about how the characters in the story could have developed

better teamwork or been more resilient. This is hap-pening only two years into the MSA program. I can POMZ�JNBHJOF�XIBU�JU�MM�CF�MJLF�JO�mWF�ZFBST�w

8BMLJOH�BSPVOE�UIF�TDIPPM�RVJDLMZ�DPOmSNT�#B[FNPSF�T�SFQPSU��.4"�QPTUFST�BSF�WJTJCMF�JO�FWFSZ�classroom, and student work on display makes reference to the mission skills. Teachers display no hesitation when asked about how they use the MSA. It is remarkable the degree of involvement this program has generated for teachers at the school.

Country School division head Kirsten Rosolen and UFBDIFS�MFBEFS�4V[Z�5BZMPS�IBWF�XPSLFE�DBSFGVMMZ�UP�build faculty examination of the six mission skills via a cross-divisional teacher group pulled together for

UIJT�QVSQPTF��"U�mSTU�B�HSPVQ�PG�� JU�IBT�HSPXO�UP��QBSUJDJQBOUT �NFFUJOH�TFWFSBM�UJNFT�B�ZFBS��5IFZ�XPSL�to develop common vocabulary across the faculty and for the students, using monthly assemblies devoted to showcasing each skill, one at a time, with rich examples and inspirational modeling. The committee also presents what they call “pop-ups” at teacher lunches, in which they succinctly share teaching strategies for mission skills, such as identifying them in the actions of characters in the literature they are studying.

“The whole experience has generated wonderful conversations about 21st century skills among everyone here; the work has become much more intentional and explicit.”

STRATEGIES


When the report came, they took the time to educate the faculty on how to read the data, particularly the box BOE�XIJTLFS�QMPUT �BOE�PO�TUBUJTUJDBM�TJHOJmDBODF��3PTPMFO�BOE�5BZMPS�BSF�BMTP�XPSLJOH�IBSE�UP�CVJME�B�TUSPOHFS�teacher network across the MSA participating schools, believing that ultimately this is where the greatest impact of the MSA will be, when teachers can compare different school practices in light of the comparative data. #B[FNPSF�BMTP�UPPL�TQFDJBM�DBSF�UP�FEVDBUF�IJT�CPBSE�BCPVU�UIF�.4" �EFEJDBUJOH�B�EBZ�MPOH�CPBSE�SFUSFBU�UP�it with a panel of experts presenting about the value of noncognitive skills and the new tool for assessing them. "DBEFNJD�BDIJFWFNFOU�EBUB�JO�HFOFSBM�JT�TIBSFE�POMZ�TQBSJOHMZ�XJUI�UIF�#PBSE�BU�$PVOUSZ�4DIPPM �CVU�#B[FNPSF�has presented MSA highlights to his trustees. He began by asking board members to predict the skills at which the school would do best, and that query by itself, even prior to viewing the data, stimulated good, “generative” conversation, he explained.

His trustees enjoyed thinking about and discussing the school from this angle, and they also became engaged considering how the data might be effectively used in the future. He thinks the MSA will assist the school in recruit-ing new board members, particularly those who work in mOBODF�BOE�OVNCFST�BMM�EBZ�BOE�BQQSFDJBUF�UIF�TDIPPM�T�FGGPSUT�UP�CFUUFS�RVBOUJmBCMZ�NFBTVSF�JUT�TVDDFTT�� He sees an impact too in enrollment. Country School discusses this work in every Open House, and “prospective parents really sit up and respond when we discuss the MSA and the seriousness of our commitment to character education.”

“Prospective parents really sit up and respond when we discuss the MSA and the seriousness of our commitment to character education.”

STRA

TEG

IES


Include Widely

The most effective programs thoroughly and meaningfully include administrators and faculty members in the entire process of planning for and using the MSA.

Communicate with Clarity

From the beginning and throughout the process, educational leaders, both administrative and faculty, should clearly communicate to all constituencies what the MSA is, what the purposes of participation are, and how it will be used (and how it won’t.)

Seek Learning and Improvement, not Accountability

The most successful programs will be those which convey and underscore that this is a research and JNQSPWFNFOU�QSPKFDU �BOE�OPU�B�UPPM�GPS�SFXBSET�PS�QVOJTINFOU��4PNF�TDIPPMT�mOE�UIBU�FNQMPZJOH�UIF�.4"�JT�B�QPXFSGVM�WFIJDMF�GPS�PSHBOJ[BUJPOBM�MFBSOJOH�BCPVU�UIFTF�TJY�TLJMMT�BOE�UIFJS�JNQPSUBODF �BCPVU�BTTFTTNFOU�methodology, and about data literacy and interpretation.

Form Communities of Practice

This work is too hard and too important to do alone. Participate actively in the MSA network, attending conferences, sending teachers to network meetings, and form “buddy school” relationships. Consider also BGmOJUZ�HSPVQ�OFUXPSLT�XJUI�TDIPPMT�ZPV�TIBSF�DPNNPO�QSJODJQMFT�PS�QSBDUJDFT�XJUI �BOE�XJUI�UIFN �TUVEZ�EBUB�collaboratively, develop common interventions, and compare practices and results.

Appreciate Different Avenues

5IFSF�JT�OP�POF�SJHIU�XBZ�UP�JNQMFNFOU�BOE�VTF�UIF�.4"��#F�PQFO�NJOEFE��VTF�UIJT�HVJEF�BOE�UIF�OFUXPSL�UP�identify and explore many different approaches.

Commit Resources "TTJHO�SFTQPOTJCJMJUZ�UP�TQFDJmD�BENJOJTUSBUPST�BOE�QSPWJEF�UIFN�EFEJDBUFE�UJNF��TVQQPSU�UFBDIFST�JO�UIF�UJNF�UIFZ�OFFE�UP�QBSUJDJQBUF��QSPWJEF�NFFUJOH�UJNFT�GPS�UIF�GBDVMUZ�UP�JOUFSQSFU�SFTVMUT�BOE�QMBO�BQQSPQSJBUF�BDUJPO��

Connect to Mission and Culture

1VU�ZPVS�PXO�NJTTJPO�JO�UIF�.JTTJPO�4LJMMT�"TTFTTNFOU��)FMQ�FWFSZPOF�SFDPHOJ[F�UIF�SFMBUJPOTIJQT�GSPN�UIF�TJY�.4"�TLJMMT�UP�UIF�QBSUJDVMBS�RVBMJUJFT�ZPVS�TDIPPM�FNQIBTJ[FT�BOE�DFMFCSBUFT��

10 Steps for a Successful MSA Program

STRATEGIES


Make It School-Wide

The MSA only assesses students in grades six to eight, but its most effective users are extending the conversation and promoting its impact across the breadth of their program, reaching out to include and inform their elementary and secondary colleagues.

Practice Action Bias, Data Skepticism

Peter Elbow’s legendary 1983 title on teaching captures it brilliantly: educators must “Embrace Contraries.” In every meeting around the MSA, simultaneously resist making too much of the data while ensuring the conversation is about what can we do with this information to improve teaching and learning.

Integrate and Connect

One rule of thumb for data-informed decision-making is not to take action on a single data point. Expand your data collection about key skills to supplement the MSA with other meaningful internal, authentic, and survey-based tools, and act with this wider set of data.

STRA

TEG

IES


Far Hills Country Day School“Surprising as it seemed, there just weren’t any good tools out there for measuring these areas that are so important in middle school,” Jennifer Phillips, Far Hills Intermediate School Director remembers BCPVU�IFS�mSTU�DPOWFSTBUJPOT�XJUI�UIF�TUFFSJOH�DPNNJUUFF�UIBU�mSTU�CFHBO�UIJT�XPSL�VOEFS�UIF�BVTQJDFT�of the ESRC, the INDEX predecessor. The few tools that were out there relied exclusively on self-assessments, and that just wasn’t good enough for the committee. These conversations were in 2009 and 2010, (as Phillips remembers, “before Paul Tough’s articles and books heightened interest in character”)—and they took place because the committee believed they were such critically important distinguishing elements of educational excellence in the middle school years.

Coming back from her committee meetings, she was pleased to receive full support from her then Head of School Jayne Geiger to proceed with implementing the new tool in her school. “This has been my most FYDJUJOH�BOE�NFBOJOHGVM�XPSL�PG�NZ�DBSFFS w�TIF�TBZT��)FS�mSTU�TUFQ�XBT �BT�TIF�FYQMBJOT �iUP�CVJME�B�DPOUFYUw�for teachers and parents at her school about why these skills were important enough to be measured in this way. As a faculty they read widely from books by Tony Wagner, Paul Tough, and Sir Ken Robinson, and many faculty went to see Tony Wagner’s keynote from the NJAIS biennial conference. The Far Hills’ school leadership reinforced this work in a series of dialogues, publications, and presentations to faculty and parents about the importance of these noncognitive skills and their relation to the school’s mission. An entirely understandable question Phillips received from teachers was “Why?” We already teach these skills, UIFZ�TBJE��iXIZ�UIJT�OFX�UFTU w�i"OE�BSFO�U�PVS�LJET�UFTUFE�FOPVHI�BMSFBEZ w�"GUFS�SFTFBSDI�BOE�SFnFDUJPO �Phillips replied: we need to do more to elevate the importance of these skills, and we need to better demonstrate our commitment to and our effectiveness in intentionally building these skills over time in our students. 'BNJMJBSJ[JOH�UFBDIFST�XJUI�UIF�XBZ�UIF�.4"�XPSLT �BOE�XIBU�JU�SFRVJSFT�UFBDIFST�BOE�TUVEFOU�UP�EP �BMTP�IFMQFE�GBDVMUZ�SFDPHOJ[F�UIJT�XBT�EJGGFSFOU�GSPN�iKVTU�BOPUIFS�TUBOEBSEJ[FE�UFTU�w From parents, the only pushback or concern was the lack of individual reports for their children. It is understandable that parents will want to know how their son or daughter particularly is doing, and might CFMJFWF�UIBU�UIJT�JT�UIF�POMZ�KVTUJmDBUJPO�GPS�UBLJOH�UIF�UJNF�UP�BENJOJTUFS�UIF�BTTFTTNFOU��#VU�1IJMMJQT�SFNFNCFST�overcoming this by helping parents appreciate the value of this tool for improving the school’s work in these JNQPSUBOU�BSFBT��"T�BO�BTJEF �JOUFSOBUJPOBM�PSHBOJ[BUJPOT�BOE�UIF�64�%FQBSUNFOU�PG�&EVDBUJPO�SFHVMBSMZ�administer tests such as the PISA and NAEP which use sampling methods to assess educational effectiveness XJUIPVU�QSPWJEJOH�JOEJWJEVBM�TUVEFOU�SFTVMUT��UIJT�JT�B�NPSF�DPNNPO�QSBDUJDF�UIBO�TPNF�NBZ�SFBMJ[F�

CASE STUDY

STRATEGIES


3KLOOLSV�UHSRUWV�WKDW�PDQ\�OHVVRQV�ZHUH�OHDUQHG�LQ�WKH�FRXUVH�RI�WKH�ÀUVW�IHZ�DGPLQLVWUDWLRQV�DW�)DU�+LOOV�Country Day School.

t� 4DIFEVMF�BENJOJTUSBUJPO�PG�UIF�.4"�PO�UIF�DBMFOEBS�FBSMZ�BOE�mSNMZ �PS�JU�XJMM�CF�IBSE�UP�TRVFF[F�JO�ZPVS�always busy school year.

t� Avoid scheduling student test-taking time during breaks or PE time: students will respond with a negative BUUJUVEF �XIJDI�DPVME�JOnVFODF�UIFJS�SFTVMUT��

t� Study the technical requirements carefully for administration, and follow their guidelines.

t� Have a detail-oriented person manage the testing planning.

t� Think twice about who actually is in the room to supervise students when they are taking the assessment: Phillips has found that if it is not someone who understands deeply and is invested meaningfully in the program, there can be some opportunities lost in conveying its purpose and value.

t� As precious as their time is, it can be a worthwhile commitment by academic administrators to be present, iTIPX�UIF�nBH w�BOE�DPOWFZ�QFSTPOBMMZ�BOE�QFSTVBTJWFMZ�UIF�JNQPSUBODF�PG�UIJT�QSPHSBN�UP�UIF�TUVEFOUT��

Phillips also has found good value in student “debriefs,” usually in advisory, with guided conversations unpack-ing what was interesting or challenging about the assessment. Again, this might be time worth spending by mid-dle school division heads, curriculum coordinators, and perhaps even the Head of School. Finally, it is respectful to students, Phillips believes, to come back and share the results when they arrive, at least the headlines. Teachers at Far Hills are usually provided one or two faculty meeting sessions to complete the teacher assessments, and are asked to do them together so as to be able to talk together as they do, and share perspectives on students. Phillips reminds teachers that it is only the aggregate data that is reviewed, never the individual, and accordingly encourages them not to get hung up on any individual student report. 8IFO�SFDFJWJOH�UIF�SFQPSUT �UIF�GPMLT�BU�'BS�)JMMT�mOE�UIF�&54�BOBMZTJT�BOE�UIF�mOEJOHT�BCPVU�DPSSFMBUJPOT�BNPOH�BMM�.4"�TDIPPMT�NPTU�VTFGVM �NVDI�NPSF�TP�UIBO�BOZ�TQFDJmDT�PG�UIFJS�PXO�TDIPPM�EBUB �BOE�UIFZ�UBLF�DBSF�UP�SFMBZ�UIF�CSPBEFS�mOEJOHT�UP�GBDVMUZ�BOE�QBSFOUT��&TQFDJBMMZ�JNQPSUBOU�JT�UIF�XBZ�JO�XIJDI�QBSUJDVMBS�NJTTJPO�TLJMMT�are found to support academic achievement, which is useful in their communications about their school’s value. That “soft skills lead to hard results” has become a central and compelling message at this school. *O�PSEFS�UP�CSPBEFO�UIF�JNQBDU�PG�UIF�.4" �UIF�UFBN�mSTU�FTUBCMJTIFE�B�GBDVMUZ�BENJOJTUSBUJPO�i$IBSBDUFS�5BTL�Force” to catalog and review their myriad character educational projects, and happily found that the six MSA skills were well embedded already across the curriculum. They also prepared a mapping or “crosswalk” of their school’s 5 Pillars, (Respect, Kindness, Responsibility, Honesty, and Leadership), to the MSA Six, which was helpful for their school community to understand the relationships and buy-in to this new tool.

All this helped build deeper support for the MSA throughout their school community, leading to what Phillips describes as ”not just a Mission Skills Assessment, but truly a Mission Skills Program.”

STRA

TEG

IES


FAR HILLS 5 PILLARS CROSSWALK WITH MSA

MISSION SKILL FAR HILLS 5 PILLARS

TEAMWORK 5HVSHFW��5HVSRQVLELOLW\��.LQGQHVV��/HDGHUVKLS��+RQHVW\

CREATIVITY Leadership, Respect, Responsibility

ETHICS 5HVSHFW��5HVSRQVLELOLW\��.LQGQHVV��/HDGHUVKLS��+RQHVW\

RESILIENCE Responsibility, Leadership

CURIOSITY Responsibility, Leadership

TIME MANAGEMENT Responsibility

"T�UIF�MBOHVBHF�BOE�UIF�TJHOJmDBODF�PG�UIF�.4"�BSF�CFDPNJOH�NPSF�XJEFMZ�FNCSBDFE�BDSPTT�UIF�TDIPPM �its educators are increasingly building them into other school assessments. Rubrics for their project-based curricular units are shifting to align with the MSA Six. Student discipline has also shifted, so that students OFFEJOH�UP�DPNQMFUF�B�iCFIBWJPSBM�QSPCMFN�SFnFDUJPOw�do so through the lens of the Mission Skills, considering how a problem might relate to a growth opportunity in ethics, teamwork, or time management. They are also looking forward to using the Mission Skills as a framework for parent conference conversations and for progress SFQPSUT �TQFDJmDBMMZ�JO�UIFJS�iFGGPSU�HSBEFT�w�5IFZ�XBOU�to do more to study the observable student actions associated with each of the six, and then as teachers, use them to be better able to provide students ongoing, consistent, and clear feedback. “It’s about tying things together, connecting the dots, so that we have a more consistent, continuous approach to developing and reinforcing these critical character skills.” A new Head of School has arrived at Far Hills, Tom Woelper, and he has made a commitment to carry the program forward, which Phillips explains is of the utmost importance: “you have to have strong backing from the school’s chief executive to make this successful.” Knowing the value of focus, Phillips expects that Far Hills will begin preparing targeted interventions on individual skills they deem most important, perhaps beginning with time management and resilience. For the former, they are exploring adapting for their students research-supported practices based on recent initiatives at The Lawrenceville School (see the Suggested Additional Reading and Resources appendix for this information). They are currently reviewing their K-5 math curriculum

“It’s about tying things together, connecting the dots, so that we have a more consistent, continuous approach to developing and reinforcing these critical character skills.”

STRATEGIES


as well, and working to add more problems requiring resilience, and more projects entailing teamwork, in order that the math curriculum better support the broader Mission Skills education. The school’s new MSA program is also helping in another important domain, admission and market-ing, where they have taken care to communicate its implementation clearly to all prospective families. It is highlighted in the admission publications, and two website pages explain further the importance of the MSA to the character education program at the school. “Prospective families tell us that our commit-ment to teaching character skills, so evident in our work with the MSA project, is a differentiator and is

JOnVFOUJBM�JO�UIFJS�TFMFDUJOH�PVS�TDIPPM�PWFS�PUIFST��0OF�QBSFOU�TBJE �A5IJT�TUPPE�PVU�GPS�VT �BOE�JT�XIZ�XF�QJDLFE�your school,’” Phillips reports. Conversion of admission inquiries to applications is improving, and middle school enrollment is robust, not due entirely to MSA, of course, but Phillips believes it has been a contributor.

“Prospective families tell us that ourcommitment to teaching characterskills, so evident in our work with the MSA project, is a differentiator BOE�JT�JOnVFOUJBM�JO�UIFJS�TFMFDUJOH�our school over others.”

STRA

TEG

IES


Administering the MSACOMMUNICATION TIPS

Schools vary widely in regards to communication about the MSA; some deliberately soft-pedal their messaging about the assessment, others proudly and loudly trumpet their involvement. These TVHHFTUJPOT�BSF�GPS�ZPVS�DPOTJEFSBUJPO��DIPPTF�UIF�POFT�UIBU�mU�XJUI�ZPVS�BQQSPBDI�BOE�HPBMT��

t� In launching the program, school-leaders should communicate clearly and consistently to all constituencies--the board, the faculty, students and parents-- the decision to participate in the MSA and explain the goals GPS�EPJOH�TP��*U�JT�QSPCBCMZ�CFTU�UP�FYQMJDJUMZ�EFmOF�ZPVS�TDIPPM�T�PCKFDUJWFT�GPS�HFUUJOH�JOWPMWFE��5IJT�TIPVME�CF�GSBNFE�JO�UXP�QBSUT��XIBU�NBLFT�UIFTF�TLJMMT�JNQPSUBOU�UP�ZPVS�TDIPPM �JUT�NJTTJPO �WBMVFT �BOE�USBEJUJPO��BOE�IPX�XJMM�ZPVS�FEVDBUJPOBM�QSPHSBN�CFOFmU�GSPN�BTTFTTJOH�UIFTF�TLJMMT �

t� The most common pushback from constituents is about the use of time-- is this worthwhile? -- and it recommended that school-leaders address this head-on: explain the time it will take (not much), and why it is a very good use of this precious resource.

t� Take care to explain and underscore to all that the assessment carries little or no stakes and will not be used to evaluate individual students or teachers. This appears to be the second most common question or critique from parents, and it is also good to address this forthrightly, and explain again its value to your school despite not generating individual student scores.

t� Consider use of outside expert speakers to generate support in your community from parents, teachers, and trustees. A speaker can present to various audiences and explain the importance of these skills and the value of measuring them, and help school-leaders build consensus. As is well known, sometimes it takes an “outside” expert to legitimate such an exercise or generate wider support. New Canaan Country School did this with their board, Far Hills and Shady Hill with their parents and teachers.

t� Use language carefully. Be wary of the word “values.” These are skills your school values, to be sure, but the word itself can convey or connote different things to different people, and can lead some to think that you are seeking to impose a particular morality which might disturb some in your community. Also be careful about use of term “trait.” Sometimes a term like “trait” can slip into our vernacular, but the developers of the .4"�BSF�FNQIBUJD�UIBU�UIFTF�BSF�TLJMMT��OPU�mYFE�PS�JOCPSO�CVU�UIJOHT�WFSZ�NVDI�BCMF�UP�CF�EFWFMPQFE�BOE�improved in learning environments.

t� Feature this work on the school website and social media feeds, to reinforce the initial communications and garner greater support and involvement. Some schools have created new website tabs, usually under i"CPVUw�PS�i"DBEFNJDTw�GPS�UIF�.4"�XPSL�UIFZ�BSF�EPJOH��PUIFST�UBH�JUFNT�JO�UIF�'BDFCPPL�GFFE�XIJDI�exemplify the school’s commitment to these six skills. At Lexington School, the Head used his website DPMVNO�UP�QSPmMF �NPOUI�CZ�NPOUI �FBDI�PG�UIF�TJY�TLJMMT�BOE�XIZ�UIFZ�XFSF�JNQPSUBOU�UP�IJT�TUVEFOUT�BOE�

STRATEGIES


their futures. (These are worth checking out: visit the Lexington School site to read them). See the New Canaan Country School website for a video they’ve made about the MSA.

t� If your school already prominently celebrates a certain set of skills or character qualities, (at one MSA school it is “scholarship, integrity, opportunity, community”), it is probably worthwhile to create and distribute a so-called “crosswalk” showing the relationship of the MSA Six to the school’s set-- to help people understand and appreciate the extent to which these are not altogether new. (See crosswalk examples in the Far Hills Country Day School case study and in the appendix).

t� Ensure teachers understand early upon launching the MSA the particular format of the assessment, and share sample questions with them (see Unpacking the Six Skills). Ideally, the project will be rolled out with or via a committee or working group which includes teachers, in such a way that it diminishes the perception that this is coming exclusively from the administration. Be frank about the workload entailed, but consider TVHHFTUJOH�XPSL�PGGTFUT��IPX�DBO�UFBDIFST�CF�TVQQPSUFE�PS�iDPWFSFEw�GPS�XIFO�UIFZ�OFFE�UP�EP�UIF�UFBDIFS�assessment forms. Be sure to explain the value of this initiative to generate greater investment from teachers.

t� It is important not to neglect communication to students: you very much want them to understand the value of these skills and the reason for “another test.” Low student motivation will hurt the quality of the data you DPMMFDU��5BLF�UJNF�UP�FOFSHJ[F�UIF�TUVEFOUT��:PV�DBO�IBWF�GVO�XJUI�UIJT��BU�POF�TDIPPM �%FUSPJU�$PVOUSZ�%BZ�School, it was introduced to students with a security camera-shot video of middle school administrators dressed all in black darting back and forth through a hallway like secret agents accompanied by the musical theme to Mission Impossible. Some schools are posting MSA posters in every classroom and creating bulletin board dedicated to the skills.

t� %PO�U�OFHMFDU�ZPVS�#PBSE��4PNF�TDIPPMT�BSF�mOEJOH�JU�WFSZ�VTFGVM�UP�TIBSF�UIJT�QSPHSBN�XJUI�UIFJS�USVTUFFT �who appreciate the data and results-oriented aspects of the program. Board members report that it is, among other things, a great nugget they can add to their “elevator speech” about the school they serve. *U�NJHIU�CF�CFOFmDJBM �BT�POF�TDIPPM�GPVOE �UP�EFWPUF�B�GVMM�NFFUJOH�PS�B�CPBSE�SFUSFBU�UP�FYQMPSJOH�UIF�.4"�and its uses, perhaps bringing in outside presenters about the skills and tools, as New Canaan Country School did.

t� Some schools are also making their school’s adoption of the MSA prominent in admission communications-- WJFXCPPLT �BENJTTJPO�XFCTJUFT �BENJTTJPO�UPVST �BENJTTJPO�JOUFSWJFX��BOE�mOEJOH�JU�B�WFSZ�FGGFDUJWF�differentiator in crowded markets. No school reports publishing MSA results in these communications. Rather, they are explaining how using MSA is improving their character education program, and also FNQIBTJ[JOH�UIF�CFOFmUT�UIFZ�EFSJWF�GSPN�QBSUJDJQBUJPO�JO�UIJT�OBUJPOBMMZ�SFDPHOJ[FE �DVUUJOH�FEHF�assessment program. “We now have more concrete ways to teach soft skills,” is the way Chuck Baldecchi, Lexington’s Head of School explains it, and he says he gets admiring nods of appreciation from his audiences. Several schools believe this communication has assisted their school’s growth in enrollment.

http://www.thelexingtonschool.org/programs/mission-skills-assessment-overview

http://vimeo.com/99543330

http://vimeo.com/99543330

STRA

TEG

IES


Administering the MSAMANAGING THE TESTING

4FF�.4"�"ENJOJTUSBUJPO�$IFDLMJTU�JO�UIF�"QQFOEJY�GPS�4QFDJmD�*OGPSNBUJPO�BCPVU�.4"�BENJOJTUSBUJPO��See also on the MSA Portal a separate document called MSA 2014 Technical Overview and Best Practices for the technology and technical details on administering the MSA.

Key points for school-leaders to consider as they oversee the administration of the MSA:

t� Have a detail-oriented person manage the testing planning.

t� 4DIFEVMF�BENJOJTUSBUJPO�PG�UIF�.4"�CFGPSF�UIF�TDIPPM�ZFBS�TUBSUT �PS�JU�XJMM�CF�IBSE�UP�TRVFF[F�JO �BOE�schedule it as late in the testing window as possible, so teachers have the most time to get to know their students. Avoid scheduling student test-taking time during breaks or PE time.

t� Study the technical requirements carefully for administration, and follow their guidelines. Ensure technology is up to task, and provide students log-in information on paper for reference. These will be generated by INDEX and available on each school’s MSA portal.

t� Schools report it is especially important and useful to support teachers in doing their assessments of students. Reassure them that they are hard for everyone, have them discuss with each other how to set standards, have them practice using the demo account, and remind them that for technical issues all emails submitted through the MSA Portal and phone calls to the support line number provided on the portal will be answered in a timely fashion. Provide meeting time to allow teachers to discuss in some detail what each rating term (“somewhat well”,”pretty well”,”very well”,”outstanding”,”truly outstanding”, etc.) mean to them, and forge consensus. This is a great topic for MSA teacher-networks: the more consistent raters can be, the better. Use the practice “demo account” that will be provided on each school’s MSA portal. Explain in advance and help teachers understand anchoring items (the three questions at the start of the teacher assessment used to equate different teacher perspectives.) Consider inviting experienced teachers at other schools to share their experience via Skype.

t� &YQMBJO�UP�TUVEFOUT�JU�JT�WFSZ�EJGGFSFOU�GSPN�UZQJDBM�TUBOEBSEJ[FE�UFTUJOH �BOE�UIBU�UIFSF�JT�OP�SJHIU�BOTXFS��Encourage students: “to let go of the feeling that you need to provide just the right answer you think they’re looking for, and instead answer in a way where you are true to yourself.” Remind students that there is no individual scoring, but do so in a way that doesn’t diminish their effort by explaining how the school will use the data to improve their school.

t� Note: The student assessment is not to be viewed by teachers or administrators. ETS asks that schools avoid viewing/reading the student areas, in order to reduce possibility of “coaching,” consciously or unconsciously, students on the “right” answers.

STRATEGIES


t� Consider having administrators proctor the student assessment, rather than teachers, making time available to teachers for their assessment duties. Think twice about who actually is in the room to supervise students when they are taking the assessment. At Far Hills Country Day School, Jennifer Phillips has found that if the proctor is not someone who understands deeply and is invested meaningfully in the program, there can be some opportunities lost in conveying its purpose and value. As precious as their time is, academic BENJOJTUSBUPST�mOE�JU�XPSUIXIJMF�UP�CF�QSFTFOU �iTIPX�UIF�nBH w�BOE�DPOWFZ�QFSTPOBMMZ�BOE�QFSTVBTJWFMZ�UIF�importance of this program to the students.

t� Seek to “de-stress” all participants, especially, about the technology. Teachers and assessment supervisors should be advised that the system uses a constant auto-backup system, so nothing will be lost in a glitch, and know that there is always someone available by phone during the assessment, so if something doesn’t work, it can be corrected promptly. Students can stop their assessment at any time and return to it where they left off, with no loss of data. This feature also lets students see how much of the assessment they have completed.

t� Gather survey feedback at regular intervals from everyone at the school involved with the implementation of the assessment. This might mean a short follow-up survey of students, but most importantly involves TUBOEBSEJ[FE�GFFECBDL�GSPN�UFBDIFST��2VBMJUBUJWF�GFFECBDL�F�H��OPUFT�GSPN�JOGPSNBM�DPOWFSTBUJPOT�NBZ�CF�important as well, in particular at smaller schools with fewer teachers. However, where possible quantitative feedback from follow-up surveys (e.g. 85% of teachers felt X way about Y aspect of the MSA) will provide UIF�TJNQMFTU�XBZ�PG�TVNNBSJ[JOH�BOE�BTTFTTJOH�ZPVS�TUBGG�T�FYQFSJFODF�XJUI�UIF�UPPM��1MFBTF�GPSXBSE�SFMFWBOU�feedback to INDEX to support its work in continuously improving the tool.

STRA

TEG

IES


Lexington SchoolFor the educators at The Lexington School, the “A” is the least important part of the MSA. There, they viewed the assessment tool as a prompt, a lever, and a vehicle by which the entire school, preschool through eighth grade, could activate and emphasize its character education program—and they’ve done so loudly and clearly, in ways that no constituent can overlook. The school’s website has a tab (or sub-tab) for MSA under Programs and it is prominently featured on the Head’s Welcome page, fre-quently written about in the school’s newsletters and magazines, and spoken about at parent, board, and community meetings.

For the MSA’s introduction, Head of School Chuck Baldecchi wrote a series of seven articles, month by month, for his community, introducing the program and then discussing each skill in detail. In his introduction, he wrote i5IF�-FYJOHUPO�4DIPPM�JT�FYDJUFE�BCPVU�UIF�QSPTQFDU�PG�EFTJHOJOH�B�UFTU�UIBU�BDDFTTFT�NBOZ�PG�UIFTF�A��TU��$FO-UVSZ�TLJMMT��*�BN�OPU�DPOWJODFE�UIBU�BOZ�UFTU�XJMM�CF�BCMF�UP�USVMZ�SFnFDU�BOE�BTTFTT�UIF�USVF�BCJMJUJFT�PG�FBDI�DIJME �but we are excited to be a part of a pioneering group of educators who are looking for a more authentic way to determine whether we are doing what we truly say we are doing.” In every one of his communications, he under-TDPSFE�UIJT�JOJUJBUJWF�XBTO�U�B�GBE��JU�XBT�EFFQMZ�DPOOFDUFE�UP�UIF�TDIPPM�T�MPOHTUBOEJOH�WBMVFT�BOE�NJTTJPO� Baldecchi wrote in each of the individual skill essays how the skills are cultivated in students throughout their growth in his school. As an example, from his piece about time management “In Middle School the VOEFSTUBOEJOH�PG�UJNF�NBOBHFNFOU�DPOUJOVFT��.PSF�JOEFQFOEFODF�JT�FYQFDUFE �CVU�QSJPSJUJ[BUJPO�JT�BMTP�introduced. Some things are more important than other things. A test calendar will include daily homework, RVJ[[FT �NBKPS�UFTUT �BOE�QSPKFDUT�PO�BOZ�HJWFO�EBZ��/PU�POMZ�BSF�TUVEFOUT�FYQFDUFE�UP�PSHBOJ[F�UIFJS�EBZT�BOE�TDIFEVMF�UJNF�PWFS�B�UXP�PS�UISFF�XFFL�QFSJPE �CVU�UIFZ�CFHJO�UP�MFBSO�IPX�UP�QSJPSJUJ[F�UIFJS�UJNF�w Ethics is one of the six MSA skills, and listening to Baldecchi, you can hear how for him, the work of assessing character education is itself a matter of integrity. Nearly every school claims they develop student character, CVU�WFSZ�GFX�IBWF�FWJEFODF�UP�CBDL�VQ�UIJT�DMBJN��#BMEFDDIJ��i"HBJO�BOE�BHBJO �XF�UFMM�QFPQMF�XF�EP�Y �Z �PS�[ �BOE�UIFO�UFMM�B�TUPSZ�PG�BO�BMVNOVT�XIP�FYFNQMJmFT�UIBU�DIBSBDUFS�RVBMJUZ��#VU�CPBSET�BOE�GBNJMJFT�EPO�U�XBOU�BOFDEPUFT�BOZ�NPSF��UIFZ�XBOU�EBUB��8F�WF�IBE�B�MPOH�DPNNJUNFOU�UP�EFWFMPQJOH�SFTJMJFODF �CVU�IPX�EP�ZPV�prove it? Telling a story isn’t good enough anymore.” Now he feels he can do something to demonstrate “whether you really are doing what you say you are doing.” This is a theme heard at many MSA user schools: it is about putting your money where your mouth is, about

CASE STUDY

http://www.thelexingtonschool.org/programs/mission-skills-assessment-overview

STRATEGIES


mOBMMZ�CFJOH�BCMF�UP�TBZ�XJUI�BMM�TFSJPVTOFTT�UIBU�ZPV�BSF�FWBMVBUJOH�XIFUIFS�ZPVS�TDIPPM�JT�GVMmMMJOH�JUT�NJTTJPO�JO�its breadth, and using more than academic test scores and GPA’s to demonstrate it.

And it is working, he reports. Lexington trumpets in admissions communications its use of the MSA in order to convey its seriousness of purpose about character education: it is a leading message for them. Trustees report they now have something far more concrete to discuss and highlight in their “elevator speech” when they speak to community members about the school’s value. Admission and enrollment statistics demonstrate strong growth for the school since they’ve begun doing so. But it’s more than integrity and evidence. It’s also about transformation. Proud of its Kentucky heritage, Lexington School is a traditional school, and Baldecchi views his leadership challenge as perpetuating its traditional TUSFOHUIT�XIJMF�BMTP�FOTVSJOH�JU�CFDPNFT�B�USVF��TU�DFOUVSZ�MFBSOJOH�PSHBOJ[BUJPO��8IBU�T�HSFBU�JT�UIBU�UIF�MSA covers this breadth: some of the MSA skills are very traditional, such as ethics and time management (work ethic), whereas others represent more forward-looking educational components, such as creativity and curiosity. Baldecchi explains that his school performed strongly in time management, but less so in creativity, and this offers him an opportunity to elevate the school’s attention to these other skills. “The data don’t matter nearly as much the as conversation it prompts. It’s just a vehicle for great, sustained conversations about what might be done differently to elicit more creativity and curiosity day-to-day in our classrooms.” Baldecchi takes the long view: schools are like giant ocean-tankers, both take great time and distance to change direction. But in time, he hopes and expects to see quantitative growth in IJT�UBSHFU�BSFBT��i*OUFOUJPOBMJUZw�JT�IVHF�GPS�IJN��he believes and observes that having teachers UBLF�TQFDJmD�BDUJPOT�UP�QSPWJEF�DIBSBDUFS�FEVDBUJPO�IBT�NFBOJOHGVM�JNQBDU��5IF�XPSL�IBT�VOGPMEFE�PWFS�TFWFSBM�years, in what he describes as a three-year plan, gradually moving from introduction and program review in UIF�mSTU �UP�FNCFEEJOH�UIF�MBOHVBHF�BOE�DPODFQUT�PG�UIFTF�QBSUJDVMBS�TLJMMT�JOUP�UFBDIFST��OBSSBUJWF�SFQPSUT�BOE�wider sharing of instructional practice, to a fully integrated Preschool to 8th grade program by the third year. Fortuitously, when the MSA began Lexington had recently expanded its commitment to instructional coaching XJUI�B�OFX�QPTJUJPO�mMMFE�CZ�UFBDIFS�+BJNF�-BTTNBO��-BTTNBO�SFDPHOJ[FE�UIBU�IF�DPVME�CFUUFS�DPBDI�UFBDIFST�in using technology by focusing on the goals and outcomes which technology can aid students in meeting, and saw that MSA provides a great framework for these outcomes. As a result, his new position has increasingly become one with a large emphasis on coaching teachers in their work on educating the six Mission Skills, both with and without technology. For Lassman too, it is in part a matter of integrity. If the school was going to proclaim that their use of the MSA was evidence of their commitment to character, they had better be sure they walked that talk. As he remembers about the early conversations in his curriculum committee, the concern was “How can we mean what we say? How can we make the MSA program real, and ensure it has a genuine impact on students’ learning and growth?”

“But boards and families don’t want anecdotes any more; they want data. We’ve had a long commitment to developing resilience, but how do you prove it? Telling a story isn’t good enough anymore.”

STRA

TEG

IES


5IF�DPNNJUUFF�T�mSTU�TUFQ�XBT�UP�BTL�UFBDIFST�TJNQMZ�UP�VTF�TJY�OFX�CPYFT�FNCFEEFE�JO�UIFJS�"UMBT�3VCJDPO�curriculum maps to identify which of the MSA skills their upcoming curriculum units would be addressed. “We just wanted to get these skills on their radar, and it worked: slowly but steadily we starting hearing more and more references to each of these skills across the breadth of the school.” Important to note: though the MSA assessment itself only occurs in grades six through eight, Lexington’s work on Mission Skills curriculum and instruction is happening across the entire breadth of the school. The Preschool teachers step right up into these conversations, sharing the way these character qualities are at the center of their instruction, and seeking to support their colleagues in becoming more intentional about them.

-BTTNBO�SFDBMMT�UIBU�JU�XPVME�IBWF�CFFO�CFUUFS�UP�CF�NPSF�TQFDJmD�XIFO�BTLJOH�UFBDIFST�UP�EPDVNFOU�UIFTF�skills, perhaps by asking them to go beyond just a single checkbox to clarifying if it were an instructional FMFNFOU�UIBU�KVTU�BSJTFT�JO�DPOWFSTBUJPO �PS�JT�FYQMJDJUMZ�UBVHIU �BOE�PS�JT�TQFDJmDBMMZ�BTTFTTFE� In step two, expectations were raised. Every teacher in the school, in every grade and every specialty, was asked to assess each and every one of the six skills in each marking period. Lassman recalls that this did put stress on teachers (“in retrospect, it was probably too big a goal”)—but the school was lucky to have him in place to talk through teacher concerns one by one, sometimes dedicating entire class periods just to explaining

again why they were doing this and how this would be used. It was helpful too to have had a full week of June meetings dedicated to teachers discussing, sharing, BOE�SFnFDUJOH�VQPO�UIFJS�BTTFTTNFOU�BOE�JOTUSVDUJPOBM�strategies for the six skills. The conversations were rich, expansive, and important. Lassman recalls an art teacher sharing recollections of a previous effort to put on report cards grades for creativity, BOE�IFS�QFSDFQUJPO�UIBU�JU�CBDLmSFE��*O�TPNF�DBTFT �HSBEJOH�DSFBUJWJUZ�TFFNFE�UP�QSPEVDF�B�mYFE�NJOE�TFU�UIBU�inhibited or diminished their subsequent creative skills. And grading for resilience? How is a “D” for resilience going to assist a student’s growth? Accordingly, the school is not quantifying assessment of these skills by and large, and is not adding them to report cards. Instead, they are preparing to embed them in narrative reports and parent conferences, using supporting and constructive language to encourage and counsel student development.

An interesting difference has been observed between two sets of teachers: the grade six to eight teachers who annually assess each of their students on the teacher form of the MSA, and those who don’t ever see the UFBDIFS�GPSNT��5IF�GPSNFS�NBZ�IBWF�B�IFBE�TUBSU�BOE�BO�BEWBOUBHF�SFHBSEJOH�IPX�FBDI�TLJMM�JT�EFmOFE�JO�observable practice. It might be useful to encourage teachers in other divisions, outside the middle school to take the time to review the MSA teacher forms, even practice completing a few, in order to better build common understanding about the six skills and how they are observed in action.

“The data don’t matter nearly as much as the conversation theyprompt. The MSA results are a vehicle for great, sustained conversations about what might be done differently to elicit more creativity and curiosity day-to-day in our classrooms.”

STRATEGIES


-BTTNBO�JT�DPOmEFOU�UIBU�IF�BOE�IJT�UFBDIFST�BSF�OPU�iVOEFS�UIF�HVOw�GSPN�)FBE�PG�4DIPPM�#BMEFDDIJ�PS�BOZPOF�FMTF�UP�SBJTF�TDPSFT�UIJT�JT�OP�DSB[Z�IJHI�TUBLFT�DPOUFTU��*OTUFBE �JU�JT�B�UPPM�BOE�WFIJDMF�GPS�1%�BOE�DPOUJOVPVT�improvement. Going forward, he thinks they will be lightening the load a tad for teachers. No longer insisting every teacher assess every student on every skill, now they’ll be asking teams of teachers to ensure that each student is assessed on each skill by at least one teacher. He sees an important shift for teachers and teams in using their Mission Skills assessment formatively, both to better coach students in their development and to shape their teaching practice in a loop of continuous and frequent adjustment and improvement. "T�B�QBSFOU �-BTTNBO�CFMJFWFT�UIF�.4"�QSPHSBN�IBT�JOGPSNFE�BOE�JOnVFODFE�IJT�QBSFOUJOH�QSBDUJDF�BOE�he thinks that gradually this might happen for most or all parents at MSA schools. He understands now better than ever why, for instance, resilience is important, and what it looks like in practice, and how it can better be supported. And beyond children: he is already planning how he can structure PD programs for teachers which help them develop further these same six skills in themselves. The ripple effect of the MSA, he thinks, can be huge. Baldecchi has big plans ahead for Lexington and the MSA. In Fall 2014, his school will receive its seven-year BDDSFEJUBUJPO�WJTJU �BOE�JUT�TFMG�TUVEZ�IBT�DBSFGVMMZ�FNQIBTJ[FE�BOE�SFnFDUFE�UIF�XBZ�JO�XIJDI�UIF�.4"�JT�informing curriculum development. He’s gone so far as to recruit for his school’s visiting team an administrator from another MSA school so that she can, through the process, observe Lexington’s MSA usage and give them DPVOTFM�PO�IPX�UP�EP�TP�CFUUFS�JO�UIF�GVUVSF��)F�T�BMTP�EFUFSNJOFE�UP�XPSL�XJUI�UIF�.4"�TQPOTPS�PSHBOJ[BUJPO�UP�plan events in which high performing schools in each skill area can showcase their practices and hence better HVJEF�UIF�XPSL�PG�FWFSZ�PUIFS�TDIPPM��)F�T�EFUFSNJOFE�BOE�DPOmEFOU�UIBU�UIF�.4"�JT�BOE�XJMM�DPOUJOVF�UP�CF�B�powerful tool to brighten the future of both his school and his students.

STRA

TEG

IES


Reading and Sharing Your Report

Sometime in the spring after the fall MSA administration, your school will receive a comprehensive SFQPSU�BCPVU�UIF�QFSGPSNBODF�PG�ZPVS�TUVEFOUT�BOE�UIF�mOEJOHT�PG�UIBU�ZFBS�T�.4"�BENJOJTUSBUJPO�across all participating schools.

Effective MSA programs will create and execute a thoughtful plan for reading, interpreting, and sharing

WKH�UHVXOWV�RI�WKHLU�VFKRRO�DQG�WKH�JHQHUDO�ÀQGLQJV��7KLV�SODQ�ZLOO�LQHYLWDEO\�DQG�DSSURSULDWHO\�YDU\�IURP�school to school, but below are some suggestions for best practice.

t� Circulate the report among academic administrators, inviting them to read, study, and construct their JOEJWJEVBM�PCTFSWBUJPOT�BOE�RVFTUJPOT �QBSUJDVMBSMZ�BSPVOE�XIBU�XBT�GPVOE�UP�DPOmSN�FYQFDUBUJPOT�PS�surprise them. Commit real time in an agenda-- an hour at least-- to discuss inferences and implications. Identify key things to share with faculty, assign further study by appropriate academic/student life committees, and share with other constituencies.

t� Read next with a joint admin-faculty data-study committee, if your school has one. Follow the same steps as above, and prepare a thoughtful plan for sharing with the full faculty.

t� Facilitate an interactive session with faculty. Many schools have found useful a process of asking teachers to NBLF�QSFEJDUJPOT�mSTU�BCPVU�IPX�UIFJS�TUVEFOUT�XJMM�QFSGPSN�BOE�UIFO�DPNQBSF�UIF�SFTVMUT�UP�UIFJS�QSFEJDUJPOT��You can start with very general predictions: which skills do your students score strongest on, and which MPXFTU �#VU�UIFO�HP�UP�NPSF�EJTBHHSFHBUFE�EBUB��DPNQBSF�CPZT�BOE�HJSMT �PS�mOBODJBM�BJE�SFDJQJFOUT�UP�GVMM�payers, sixth graders to eighth graders. These conversations, even before studying the data, have their own TJHOJmDBOU�WBMVF �BOE�UIFO�NPSF�TP�XIFO�UIF�SFTVMUT�BSF�FYBNJOFE�JO�UIJT�DPOUFYU��$POTJEFS�DPSSFMBUJPOT��which skills will correlate highest with academic achievement? Which with life satisfaction? Be sure to use UIF�.4"�SFQPSU�UP�DBMM�BUUFOUJPO�UP�CPUI�UIF�BMM�TDIPPM�HFOFSBM�mOEJOHT�BOE�UP�ZPVS�TDIPPM�TQFDJmD�SFTVMUT��JOEFFE �UIF�GPSNFS�JT�BDUVBMMZ�QSPCBCMZ�NPSF�VTFGVM �UIPVHI�PGUFO�VOEFS�FNQIBTJ[FE��

t� Take time to use the MSA as an excellent opportunity to build data and assessment literacy among administrators and teachers: data literacy particularly is an essential 21st century skill for students and adults alike. Don’t expect that most people will be able to read the many different tables and charts without practice (though some will). 0OF�QMBDF�UP�CFHJO�JT�UP�DPOTJEFS�UIF�BTTFTTNFOU�T�TUBOEBSE�FSSPS�BOE�PS�DPOmEFODF�JOUFSWBMT �FOTVSJOH�people understand how to apply them and what their implications might be when studying the results. -PPL�BU�UIF�CPY�BOE�XIJTLFS�QMPUT��%PO�U�FYQFDU�FWFSZPOF�XJMM�CF�BCMF�UP�SFBE�UIFTF�BU�mSTU�CMVTI �BOE�EPO�U�make the easy-to-make mistake of asking people if they have any questions, and taking no response as evidence they get it. Few will be brave enough to volunteer they don’t understand what they’re looking at. Ask small groups to develop and share their interpretations and explanations in their own words, divvying

STRATEGIES


them up and having groups teach each other. Spend time in full groups or small groups unpacking these, VOEFSTUBOEJOH�IPX�UIFZ�XPSL �BOE�HPJOH�CFZPOE�UIF�NFDIBOJDT�UP�EJTDVTT�UIF�TJHOJmDBODF��#FZPOE�UIF�comparisons at the median, what does it mean for understanding and practice if the boxes are larger or smaller when compared to each other. What does it mean that there are no high outliers? What do we infer when we compare whiskers? Examine correlations of data about skills and intended outcomes (GPA or life satisfaction), helping people learn to read and make meaning of them. Look at the regression lines. The report says that the steeper the MJOF �UIF�TUSPOHFS�UIF�SFMBUJPOTIJQ��IFMQ�QFPQMF�VOEFSTUBOE�XIZ�UIJT�JT�TP �BOE�EJTDVTT�UIF�TJHOJmDBODF��5IF�table described as “the score percentiles associated with the medians for your school relative to all other participating INDEX schools in the most recent wave” is another one which could be valuably unpacked. Finally, take time to examine and consider longitudinal trends.

t� In the book, Data Wise: A Step by Step Guide to Using Assessment Results to Improve Teaching and Learning, Chapter 3 is devoted to this same topic of close data analysis. They offer several valuable suggestions, including the following: A. Choose a focus area. #�� "OBMZ[F�UIF�%BUB�BOE�'JOE�UIF�4UPSZ�� C. Display the Data. (“Graphic displays are key tools.”) D. Compare the Performances of Groups E. Display Performance Trends. F. Allow Staff Members and Teachers to Work with and Make Sense of the Data For more information, see that excellent book and the data-informed decision-making primer in the appendix.

t� Some schools choose to intentionally soft-pedal the MSA, and often will do no more with the data than the above (if that). Other schools are choosing to spotlight the MSA as a tool to elevate the entire community’s attention to character, whole child, and social/emotional learning, and those schools often do choose to TIBSF�TPNF�.4"�EBUB�QPJOUT�XJUI�TUVEFOUT��5IJT�JT�BO�IPOPSBCMF�QSBDUJDF��JU�JT�PGUFO�WJFXFE�BT�BO�FUIJDBM�principle in survey and assessment that the participants in studies have some opportunity to see results BOE�mOEJOHT��"T�XJUI�BENJOJTUSBUPST�BOE�GBDVMUZ �UIJT�JT�B�XPOEFSGVM�PQQPSUVOJUZ�UP�FEVDBUF�TUVEFOUT�JO�EBUB�literacy. And ask students: what did they expect, how are they surprised, and what learning experiences do they think help them most in each skill area.

t� Many schools report sharing PowerPoint presentations of highlights of their school report. They facilitate general conversation with parents and board members, being careful not to place undue emphasis on the TJHOJmDBODF�PO�UIF�mOEJOHT �BOE�FYQMBJOJOH�UIBU�JU�XPVME�CF�QSFNBUVSF�UP�NBLF�UPP�NVDI�PG�UIF�EBUB�BOE�UIBU�UIF�TQSFBE�BNPOH�TDIPPMT�JT�OPU�HSFBUMZ�TJHOJmDBOU��

t� Although many schools are showcasing in Marketing and Admission their participation in the MSA, no school JOUFSWJFXFE�SFQPSUT�IBWJOH�TIBSFE�EBUB�BCPVU�JUT�QFSGPSNBODF��*U�JT�OPU�GPSCJEEFO��UIF�POMZ�SVMF�JO�UIJT�SFHBSE�is that the MSA data cannot be used to disparage another school. But many schools feel it is premature to go out on this limb at this juncture.

STRA

TEG

IES


t� Several schools report sharing, or looking forward to sharing, their school’s data in their accreditation QSPDFTT��5IFZ�SJHIUGVMMZ�CFMJFWF�JU�JT�B�WBMVBCMF�DPNQPOFOU�PG�TFMG�TUVEZ�BOE�TFMG�SFnFDUJPO �BOE�IPOPST�UIF�ever-rising expectation and/or obligation that schools, when seeking re-accreditation, employ meaningful, mission-appropriate, external measurements of educational effectiveness. One school has gone so far as to request that its visiting committee include an administrator from another MSA school so as to enhance the way the MSA data and program can inform the accreditation process.

STRATEGIES


Shady Hill SchoolWhile administrators at Shady Hill School were enthusiastic about the MSA opportunity, it was also clear from the culture of the school that the MSA could not be implemented smoothly without the sup-port of the faculty. Because of this, a faculty member was given a stipend to introduce and implement UIF�.4"��5IF�mSTU�ZFBS�JU�XBT�PWFSTFFO�CZ�UIF�IFBE�PG�TFSWJDF�MFBSOJOH�GPS�UIF�TDIPPM �BOE�UIF�TFDPOE�year the project was run by a 6th grade teacher. Assistant Head Allison Webster described the teach-er as one who is held in high esteem for her “model classroom culture: an absolutely amazing climate with wonderful student citizens.” The teacher saw the value of the MSA, with its relatively low invest-ment of time but exciting opportunity to support whole child education, and she worked to generate support from other teachers. This was especially important at Shady Hill; because of its long and deep progressive ethos, testing of any kind has been viewed with healthy dose of skepticism about what can be learned in any single assessment. That this teacher, who herself manifested the school’s value and an excellence in social emotional learning, took the lead was critical to the project’s smooth implementation.

“We laid lots of groundwork with parents, faculty, and students,” Webster recalls. They kicked off the MSA to students with a video about character in action, and underscored that these skills were what were most important to them and their school. They also had students speaking to students, explaining to each other the NFBOJOH�BOE�WBMVF�PG�FBDI�TLJMM��"ENJOJTUSBUPST�QMBZFE�WFSZ�MJUUMF�QVCMJD�SPMFT�JO�UIJT��JU�XBT�PXOFE �JO�B�TFOTF �from the beginning by students and teachers. When skepticism was expressed that these competencies could CF�NFBTVSFE�JO�B�TUBOEBSEJ[FE�UFTU �UIF�SFTQPOTF�XBT�OPU�UP�EJTQVUF�CVU�UP�DPOOFDU��TLFQUJDJTN�JT�NFSJUFE �BOE�this is intended as an ongoing research project and experiment to determine if indeed it is possible to do this task. The same message went out to parents: participation was called for not only for the data that would be generated but also because of the national scope and potential long-term value of the study. This was further reinforced with messages about the strong connection of the MSA skills to the school’s mission. The foundation XBT�BMSFBEZ�JO�QMBDF �8FCTUFS�TBZT��QBSFOU�FEVDBUJPO�BSPVOE�QPTJUJWF�QTZDIPMPHZ�BOE�UIF�JNQPSUBODF�PG�noncognitive skills had been underway for several years. When the testing took place, administrators stepped in to cover for teachers and give them time to do their piece of the assessment. Shady Hill students didn’t entirely enjoy the experience: “Our kids felt a bit hemmed in, and hated to have to choose among the options, sometimes,” Webster explains. “It was important for us to build in extra time to debrief the experience and provide students the chance to process the experience, and discuss

CASE STUDY

STRA

TEG

IES


the uncertainties they had about various questions and tasks. But that is OK too: these conversations were educational in themselves.” Shady Hill administrators take only limited time in their academic administration group to review their MSA results, providing the report in advance of the meeting and requesting administrators to bring questions and PCTFSWBUJPOT �CVU�UIFZ�EPO�U�mOE�B�MPU�PG�EFFQ�TJHOJmDBODF�JO�UIFN�BU�UIJT�QPJOU��i0WFS�UJNF �UIF�USFOET�BOE�patterns will be of interest, but for now we only have a few dots on a graph.” The spread among participating schools is not wide, so the differences revealed are only small, but Webster says that they do see some correspondence with their expectations. To date, they haven’t done anything to share their MSA results with anyone beyond their administrative group. She believes that it is still premature to make much of the data, and UIBU�IFS�GBDVMUZ�XJMM�OPU�ZFU�UBLF�TFSJPVTMZ�UIF�TDJFOUJmD�TJHOJmDBODF�PG�UIFJS�TDIPPM�T�SFTVMUT�� Instead, the approach there is to connect the MSA skills to the school’s mission and values, underplaying the .4"�JUTFMG�BOE�TVCTVNJOH�JUT�TLJMMT�VOEFS�NPSF�XJEFMZ�SFDPHOJ[FE�DPOTUSVDUT�BOE�QSPHSBNT��-JUUMF�JT�EPOF�UP�IJHIMJHIU�UIF�.4"�4JY�TQFDJmDBMMZ �CVU�NVDI�JT�EPOF�UP�UBML�BCPVU�UIF�TDIPPM�T�DPNNJUNFOU�UP�UIF�XIPMF�DIJME��only in that context is reference made to the MSA. There are many ways in which this happens at Shady Hill. For example, much is made at the school about the importance of brain-based learning and the role of executive function, and as those discussions are held and instruction is developed, some reference is made to the MSA “time management” skill and the related data. Similarly, the school has long had a program they call TOAD, Talks on Adolescent Development, and it was a natural place for them to add in a discussion of resilience for students. “We know our traction for these kinds of things will be much greater when we attach them to other things we are already doing than if we just pop up and say let’s do this because it’s in the MSA.” Webster appreciates that the constructs are so broad that they lend themselves to this kind of reinforcement of ongoing and more deeply rooted school programs. Shady Hill distinguishes itself for the annual academic program reviews in which visiting teams visit, review documents and observe in order to provide feedback and recommendations, and though it hasn’t happened yet, Webster says she looks forward to the day not too far off when they host such a visit for their program of noncognitive education. After the initial rollout, communications have been low-key, but it does come up occasionally in conversation among parents at Shady Hill, in the context of broader discussions about whole child education. It is not the norm to present detailed student achievement data to the Board, but rather to share broader trends and aggregated data. The Board is aware of the MSA and supports the project. The school has also treaded lightly in Admission communications about the MSA, but they do sometimes make mention of it when discussing character education in general. The school’s commitment to MSA skills began long before the testing, and it is the school’s more foundational mission that parents seek to understand during the Admission process.

“We know our traction for these kinds of things will be much greater when we attach them to other things we are already doing than if we just pop up and say let’s do this because it’s in the MSA.”

STRATEGIES


"MUIPVHI�UIF�.4"�UPPM�JO�BOE�PG�JUTFMG�JT�TPGU�QFEBMFE�BU�4IBEZ�)JMM �8FCTUFS�mOET�HSFBU�QSPNJTF�JO�UIF�QPXFS�PG�the MSA user network. She believes the character side of education to be generally under-attended, and sees the greatest opportunity of the MSA to inform important national conversations among educators about these TLJMMT �UIFJS�JOTUSVDUJPO �BOE�XIBU�UIF�EBUB�DBO�SFWFBM��8FCTUFS�FNQIBTJ[FE�UIF�JNQPSUBODF�PG�FEVDBUPST�BU�.4"�schools attending the annual MSA conference and actively participating in the still emerging MSA teacher networks.

STRA

TEG

IES


Using the MSA to Improve Student Learning Outcomes in the Six Skills: A Preliminary Guide

“First and foremost, academic leaders need to ensure that every educator understands that his or her job is to identify and work to close the inevitable gaps between mission and reality, between desired learning results and actual performances by students on measures that matter.” Grant Wiggins and Jay McTighe, Schooling by Design

Note: This section is labeled as preliminary because the tool is still too new for there to be a track record of demonstrable impact. A more meaningful, evidence-based, program of using the MSA to improve student outcomes would require several more years of data and associated school interventions and a careful study. It is anticipated that this section particularly will be revised and updated in future years after more work has been done in this area. Few schools want to go to the effort of collecting data only to let a report sit on a shelf, and few are participating in the MSA only for the sake of the research and the network, as important and rewarding as they are. Educators are at heart all about improvement: they want to see their students learn and grow more every year, and they want to use every tool at their disposal to enhance their effectiveness and further advance student development. Ultimately the point of assessment is to inform instructional improvement and generate improved student outcomes, and INDEX intends the MSA to be a tool in service to this work. Some schools are choosing for now not to use the MSA assessment and its data results as part of a school improvement strategy. This section is intended primarily for those schools using MSA as an element of ongoing program review and continuous development. Our suggestions are provided in a dozen general principles and three distinct approaches.

GENERAL PRINCIPLES:

t� Promote Data Literacy Using data to inform improvement requires substantial savvy about data interpretation, and educators shouldn’t embark on this work without putting time and attention toward enhancing the data literacy of themselves and their colleagues and studying thoroughly the statistical information MSA provides. One suggested resource is Data Wise in Action, which offers an excellent chapter-length guide to developing EBUB�MJUFSBDZ �JODMVEJOH�B�RVJ[�FEVDBUPST�DBO�UBLF�UP�JEFOUJGZ�BSFBT�JO�XIJDI�UIFZ�DBO�JNQSPWF�UIFJS�TBWWZ��

STRATEGIES


t� Assign Administrative Responsibility Successful schools take steps to ensure data interpretation and instructional improvement doesn’t fall CFUXFFO�UIF�DSBDLT��FGGFDUJWFMZ�VTJOH�UIF�.4"�FOUBJMT�BTTJHOJOH�SFTQPOTJCJMJUZ�BOE�BMMPDBUJOH�UJNF�UP�conscientious and intentional supervision of this work.

t� Engage Faculty

"T�BMM�BENJOJTUSBUPST�SFDPHOJ[F �FGGFDUJWF�JOTUSVDUJPOBM�JNQSPWFNFOU�SFRVJSFT�NFBOJOHGVM�GBDVMUZ�FOHBHFNFOU��Grant Wiggins writes in Schooling by Design “all teachers [should] be actively involved at some level in analysis of student performance data from multiple sources.” Develop a faculty committee for the MSA particularly, or for character education or data analysis generally, and provide them genuine issues and substantial time to do their work. Schools using Professional Learning Communities (PLCs) or Critical Friends Groups (CGFs) in one form or another might consider asking one of those groups to take this work on.

t� Commit to Evidence-Based Inquiry

Use the MSA as a vehicle by which you can change the conversation in your school from subjective perspective (“I think our school does a great job with creativity”) to consideration of objective evidence (“why might it be that our sixth graders perform relatively strongly on the MSA in creativity but then don’t improve as much in subsequent years as at other schools?”). Use MSA data not to answer questions but to form them, and then seek other data, quantitative and qualitative, to form conclusions. But…

t� Exercise Caution Regarding Over-Attribution

Data may be more trustworthy than anecdote, but we all know what Mark Twain said about statistics—“there are three kinds of lies: lies, damned lies, and statistics.” Study the data carefully and draw appropriate JOGFSFODFT �CVU�CPUI�QSJWBUFMZ�BOE�QVCMJDMZ�EFNPOTUSBUF�B�mSN�SFTPMWF�OPU�UP�PWFSTUBUF�UIF�TJHOJmDBODF�PG�your school’s results.

t� Make a “Commitment to Action”

The Harvard Graduate School of Education authors of Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning make this one of their three essential “Habits of Mind.” “Shared commitment to action, assessment, and adjustment means that all the work you do is geared toward doing, not just talking. Every meeting that you have has clear objectives, including coming up with questions, analyses, and decisions that help move the work forward.” They call upon educational leaders to navigate carefully between “analysis paralysis” on the one hand and “forging blindly ahead” on the other.

t� Network, Collaborate, Share

It is far more effective to study and draw meaning from data in collaboration than in isolation. Attend MSA conferences and participate in the various emerging MSA networks of administrators and teachers, but do more than that too: reach out to build one or several “buddy schools” with which you can privately and transparently share data, compare notes, discuss appropriate inferences, and explore possibilities.

t� Make It School-Wide

At least for now, the MSA is administered only at the middle school grades, six through eight. But most MSA schools have more grade levels than those, and the most effective approaches to using the MSA for outcome improvements entail making it a school-wide initiative. Teachers in the grades below sixth have an FOPSNPVT�JOnVFODF�PO�UIF�EFWFMPQNFOU�PG�UIF�.JTTJPO�4LJMMT �BOE�CFOFmU�GSPN�LOPXJOH�IPX�UIFJS�TUVEFOUT�XJMM�CF�BTTFTTFE�JO�UIFTF�SFHBSET��HSBEFT�BCPWF�NJEEMF�TDIPPM�DBO�EP�NVDI�UP�QFSQFUVBUF�BOE�BEWBODF�UIF�

STRA

TEG

IES


TLJMMT�EFWFMPQFE�JO�UIF�FBSMJFS�ZFBST��"MM�DBO�QSPmU�GSPN�CSPBE�CBTFE�GBDVMUZ�DPOWFSTBUJPOT�BCPVU�JNQSPWJOH�practice in MSA skill development. Don’t isolate the MSA in the middle school.

t� Promote a Culture of Risk-Taking and Learning.

Measuring ourselves against national norms can be anxiety-inducing, but in every way this program is intended to be low stakes, not high pressure. School-leaders should convey emphatically that this initiative is BCPVU�FYQFSJNFOUBUJPO �EJTDPWFSZ �SFnFDUJPO �BOE�JUFSBUJPO �OPU�BCPVU�BDDPVOUBCJMJUZ�

t� Consider Other Intended Outcomes Correlated With Skills

Choose which skills to focus upon for improvement not only based on your school’s relative performance (i.e. low creativity scores leading to priority for developing creativity) but also upon the desired outcomes. If you wish to improve student GPA, focus on supporting time management, and then measure success in performance in that skill and in student GPA.

t� Align, Align, Align

Don’t let any assessment become an island unto itself. Align the MSA program and data with other educational initiatives, other data sources, and other assessment tools and techniques. Consider what other programs are already in place which the MSA can reinforce and inform. See how MSA performance seems to correlate with other measurements in your school. If you are committed to student growth as measured with the MSA, consider aligning classroom assessments, report cards, parent-teacher conferences, and/or other assessment methods with the MSA framework.

t� 'RQ·W�2YHUO\�5HO\�RQ�$Q\�6LQJOH�6RXUFH�RI�'DWD� "T�UIF�BVUIPST�PG�%BUB�8JTF�XSJUF �i"�LFZ�QPJOU�XF�FNQIBTJ[F�JT�UIJT��EPO�U�UBLF�BDUJPO�PO�UIF�CBTJT�PG�a single measure. Any time you are tempted to act on one score, ask yourself: What other evidence or information can I gather to paint a more complete picture of what is going on?”

GENERAL APPROACHES

Three general approaches emerge from our studies. (Though they are not entirely independent of each

other: they can in some cases be mixed and matched, or hybridized.)

t� 6TF�UIF�.4"�OPU�KVTU�BT�B�UFTU�CVU�BT�B�QSPHSBNNBUJD�QMBUGPSN�BOE�GSBNFXPSL�UP�JOGPSN �PSHBOJ[F �BOE�HVJEF�

a broad (new or renewed) character education (or social-emotional learning) initiative.

t� &NCFE�.4"�FMFNFOUT �TVDI�BT�EFmOJUJPOT �EBUB �BOE�SFTPVSDFT �BT�BQQSPQSJBUF�JOUP�PUIFS�TDIPPM�JOJUJBUJWFT�

t� Use MSA data to identify what Wiggins calls “mission-reality gaps,” to prompt improvement initiatives, and to evaluate the effectiveness of those interventions.

Approach 1:

As reported in the school magazine, because of the MSA, at Lexington School “character strengths have gone from unwritten to written into the curriculum. Whereas in the past when we have seized upon impromptu teachable moments, we now create opportunities and are intentional in ensuring we are not missing instruction

STRATEGIES


of the six measurable skills. Mr. Lassman likens teachers assessing the character skills to soccer coaches continuously assessing their players’ particular skill levels – coaching in the classroom – seeing which athletes OFFE�TQFDJmD�DPBDIJOH �XIP�DBO�LJDL�XJUI�UIF�SJHIU�GPPU�BOE�XIP�XJUI�UIF�MFGU�w The MSA can be a powerful vehicle to identify and convey what’s most important, to create common vocabulary, and to shift your school’s culture. For many schools, the “A” for assessment is the least important letter in the .4"��UIBU�UIF�TDIPPM�JT�BOOVBMMZ�FWBMVBUJOH�UIFTF�TJY�TLJMMT�JT�POMZ�UIF�TUBSUJOH�QPJOU�PG�XIBU�JT�CFDPNJOH�B�CSPBE�and coherent school conversation about teaching what matters. The MSA, for these schools, is transformation, not just information.

5ISFF�PG�UIF�GPVS�DBTF�TUVEJFT�QSPWJEFE�JO�UIJT�EPDVNFOU�TIPXDBTF�UIJT�mSTU�BQQSPBDI�BT�JU�JT�CFJOH�QSBDUJDFE�BU�New Canaan Country School, Lexington School, and Far Hills Country Day School. These schools have loudly BOE�QSPVEMZ�USVNQFUFE�UIFJS�FNCSBDF�PG�UIF�TJY�.4"�TLJMMT��DBSFGVMMZ�UJFE�UIF�TJY�TLJMMT�UP�UIFJS�PXO�TDIPPM�T�NJTTJPO�BOE�USFBTVSFE�WBMVFT��FEVDBUFE�QBSFOUT�BU�CBDL�UP�TDIPPM�OJHIUT�BOE�QBSFOU�BTTPDJBUJPO�NFFUJOHT �BOE�JO�TDIPPM�QVCMJDBUJPOT��UBVHIU�UIFJS�TUVEFOUT�PO�UIF�.4"�4JY�JO�BTTFNCMJFT �BEWJTPSJFT �BOE�PO�QPTUFST�BOE�XFCTJUFT��BOE�FWFO�GBDJMJUBUFE�CPBSE�SFUSFBUT�BCPVU�UIF�TJY�DSJUJDBM�TLJMMT�BOE�IPX�UIF�TDIPPM�T�FEVDBUJPO�develops them. Teachers are forming committees, studying instructional techniques, and sharing best practices for cultivating Teamwork, Creativity, Ethics, Resiliency, Curiosity, and Time Management. It is entirely possible schools could create entire courses to be taught in one or more of these grade levels devoted to the EFWFMPQNFOU�PG�UIFTF�TLJMMT��B�DPNQBSBCMF�TVDI�DPVSTF �UBVHIU�JO�6UBI�QVCMJD�IJHI�TDIPPMT �IBT�CFFO�GPVOE�UP�be effective in research conducted by the same ETS group which built the MSA. At some schools, the MSA framework is for the adults too: it is being found valuable for underscoring and reminding the adult learners what is most important. The most successful 21st century schools are ones which form and sustain adult learning communities, and these six skills are a great way to highlight what’s most important and generate ongoing conversation with common reference points about how we all, students, teachers and administrators, are growing and needing to improve in our interpersonal and intrapersonal skills. Some of these “all-in” MSA schools are re-tooling other assessments to align with the MSA Six. The six skills can CF�VTFE�UP�HFOFSBUF�SVCSJDT�GPS�QSPKFDU�CBTFE�MFBSOJOH��UP�HVJEF�TUVEFOU�MFE�QBSFOU�DPOGFSFODF�DPOWFSTBUJPOT��to structure student narrative reports, or even to build addenda to report cards. One school is using them for UIFJS�TUVEFOU�EJTDJQMJOF�SFnFDUJPOT� At these schools, there is not a lot of close attention to or anxiety about quantitative improvement. They expect their efforts will lift their student performance, and they will track the data, but they don’t place any undue emphasis or overly high expectations for it. They expect that in time the data will inform their future initiatives, CVU�UIFZ�BSF�OPU�TVSF�FYBDUMZ�IPX��GPS�OPX �JU�JT�UIF�GPVOEBUJPO�GPS�XIBU�UIFJS�FEVDBUJPO�JT�BCPVU �OPU�UIF�XJOEPX�into what’s working and not. Approach 2:

"U�UIF�GPVSUI�TDIPPM�QSPmMFE�XJUI�B�DBTF�TUVEZ�JO�UIJT�HVJEF �4IBEZ�)JMM �UIF�FEVDBUPST�BSF�DIPPTJOH�OPU�UP�highlight MSA in and of itself, but instead view it as a very useful tool to reinforce other school initiatives and inform their program reviews. To do so, educational leaders might undertake an audit of their school’s various social and emotional learning programs—both external and internal, both large-scale and small—and then map them to one or two of the

STRA

TEG

IES


MSA Six. What does the MSA data about that particular skill area suggest about the effectiveness of the related school or classroom endeavor? When looking at the teacher assessment form for that skill, determine the observable and meaningful student actions in that skill, and consider how that informs the work to strengthen the skill area. They might also look at what other MSA schools report about their successful educational programming on that particular skill.

In this approach, MSA is the subordinate approach—the supporting actor rather than the featured star— but as with every such supporter, still very valuable. Approach 3:

None of the studied schools is fully engaged yet in this third approach, but several said they were anticipating doing so in the near future. One school spoke of their plans. “Let’s look at one skill in particular, focus on it, and EFUFSNJOF�IPX�XF�DBO�NBLF�TUVEFOU�EFWFMPQNFOU�NPSF�JOUFOUJPOBM �FYQMJDJU �FNQIBTJ[FE��-FU�T�SFBMMZ�IPOF�JO�PO�POF�QJFDF�PG�JU��EJTBHHSFHBUF�UIF�EBUB�BOE�MPPL�BU�UIF�mOFS�EFUBJMT�PG�HFOEFS�EJGGFSFODFT �HSBEF�MFWFM�EJGGFSFODFT �and related outcomes.”

In this approach, schools seeking to strengthen student growth in a particular skill expect to look to that skill’s top performing schools and study their practices. Taking it further, schools might send small committees to visit and observe high performers, and invite teacher-leaders from those schools to return the visit and provide IPOFTU�BOE�IFMQGVM�GFFECBDL��'PS�UIJT�UP�IBQQFO �UPQ�QFSGPSNFST�XJMM�IBWF�UP�CF�JEFOUJmFE�XJUIJO�UIF�OFUXPSL �which is anticipated will be a voluntary option in the future.)

These schools won’t limit themselves to MSA models, of course. Other experts can be brought to observe, provide feedback, give guidance and train teachers. Educators should seek program changes (aka “interventions”), which reconcile practices rooted in school culture with those established by sound research. It will be important to scan the educational literature to learn more about the skill, and study the available evidence-based best practices for these interventions, resources for which are provided in the section unpacking each construct and in the additional resources section of the appendix. Once selected, undertaking interventions provides ideal opportunities for teachers and potentially even students UP�IPOF�UIFJS�SFTFBSDI�TLJMMT��.VDI�JT�IBQQFOJOH�DVSSFOUMZ�JO�UIF�mFME�PG�UFBDIFS�iBDUJPO�SFTFBSDI w�BOE�TDIPPMT�could work to form control and experimental groups, generate internal assessment methods to evaluate student growth and measure impact. One potential resource for funding school-based research is Character Lab (www.characterlab.org.)

0OF�TDIPPM�FYQMBJOFE�UIBU�XFSF�UIFZ�UP�mOE�UIFJS�TUVEFOUT�VOEFSQFSGPSNJOH�JO�DVSJPTJUZ �UIFZ�XPVME�JNBHJOF�training teachers to put a special emphasis on observing student curiosity, create a baseline for internal observation data, and then conduct interventions changing instructional practice to better cultivate curiosity, do B�TFDPOE�SPVOE�PG�PCTFSWBUJPOT�UP�EFUFSNJOF�QSPHSFTT �BOE�UIFO�WBMJEBUF�UIFJS�mOEJOHT�XJUI�TVCTFRVFOU� MSA data.

http://www.characterlab.org

Unpacking the Six Skills

UN

PAC

KIN

G T

HE

SIX

SKIL

LS


The MSA CrosswalkThe six MSA skills were selected after a thorough review of the mission statements of the pilot participating schools, and are intended to be easily recognizable and widely appreciated character components, as well as key contributors to success and satisfaction in school and life. Each has been vetted by ETS as having sound research foundations for validity and reliability when assessed. The crosswalk provided here connects the MSA Six to the key sub-elements measured, the observable indicators of the skill in action, the Big Five Personality skill each is most closely corresponding to, and other terms in common parlance which the skill most closely aligns with. As explained on Wikipedia, “The Big Five model is able to account for different skills in personality without overlapping. Empirical research has shown that the Big Five personality skills show consistency in interviews, self-EFTDSJQUJPOT�BOE�PCTFSWBUJPOT��.PSFPWFS �UIJT�mWF�GBDUPS�TUSVDUVSF�TFFNT�UP�CF�GPVOE�BDSPTT�B�XJEF�range of participants of different ages and of different cultures.”

MSA SKILLS Big Five Trait Most Closely Associated

Attributes Popular in General Circulation

Teamwork

*OnVFODF�PO�0UIFST $POnJDU�3FTPMVUJPO Guiding OthersCooperation

Agreeableness (and Extraversion)

Collaboration Empathy Social Skills Social and emotional intelligence

Creativity

Originality Imagination Fluency

OpennessCreativity Imagination Broad Retrieval Ability

Ethics

Honesty AltruismConcern for Others

Agreeableness Empathy Integrity

Resilience

Effective Coping Core Self-Evaluation (Positive Self-Concept)

Neuroticism (and Openness)

Grit Resilience

Curiosity Intrinsic Motivation Openness Motivation Love of Learning

Time

Management

Determining One’s NeedsSetting Goals for One’s Needs1SJPSJUJ[JOH�BOE�1MBOOJOH�UP�Meet One’s Goals

ConscientiousnessGrit Self-Control Work Ethic Responsibility

UN

PACKIN

G TH

E SIX SKILLS


MSA IndicatorsEarly in the development of the MSA, the Steering Committee recognized the need to provide users a better picture of what each skill looks like in action. These indicators can be used for educators to know what to look for and how to talk about each of the MSA Six.

TEAMWORK'JOET�TPMVUJPOT�EVSJOH�DPOnJDUTDemonstrates respect for feelings of others and listens wellKnows when and how to include others Adapts effectively to different group situations

CREATIVITYGenerates new ideas4ZOUIFTJ[FT�JOGPSNBUJPO�JO�OFX�XBZTExplores new ideas eagerlyAsks and answers questions to deepen understanding

ETHICSMakes decisions and takes actions based on needs of community and not just selfTreats people with respect, sensitivity, and fairnessDemonstrates honesty and trust

RESILIENCEFinishes what he or she beginsTries very hard even after experiencing failureWorks diligently and independentlySees setbacks as temporary

CURIOSITYPursues opportunities for independent learningShows enthusiasm for trying and learning new thingsAsks and answers questions and knows how to ask the next question

TIME MANAGEMENTComes to class prepared to learnPays attention and resists distractionGets to work rather than procrastinates.BJOUBJOT�B�TZTUFN�UP�DPNQMFUF�XPSL�BOE�QSJPSJUJ[F�UBTLT

UN

PAC

KIN

G T

HE

SIX

SKIL

LS


Situational Judgement Test

How do you feel when you work on a group project for school, and you do more work than any of the others in the group. Your teacher gives the group a good grade, but does not know that you did most of the work.

Self-Report

I like to work with people

Teacher Ratings

This student is a good team member

CREATIVITY: REPRESENTATIVE ITEMS

Fluency

The category is “round.“ Write all the things that are round or that are round more often than any other shape

Self-Report

I generate novel ideas

Assessing the SkillsIn the method used in the MSA, each skill is assessed via three independent approaches: teacher SBUJOHT �TUVEFOU�TFMG�SFQPSU �BOE�B�UIJSE �PUIFS�DBUFHPSZ��#FMPX�ZPV�DBO�mOE�SFQSFTFOUBUJWF�RVFTUJPOT�PS�items from each of the three approaches to better understand how the assessment works.

Teacher Ratings

This student thinks “outside the box”

TEAMWORK: REPRESENTATIVE ITEMS

CREATIVITY

TEAMWORK

UN

PACKIN

G TH

E SIX SKILLS


Biographical Data

How many times in the past year IBWF�ZPV�DIFBUFE�PO�B�RVJ[�PS�UFTU�in school?

Self-Report

I am an honest person

Self-Report

I overcome challenges and set-backs

Situational Judgement Test

Imagine that this situation is happening to you right now. You are feeling stressed about the amount of homework that you have. Below are some ways that you might think, feel, or act in this situation. Rate how often you do each activity when you feel stressed from having too much homework to do. Scale: Never or Rarely; Sometimes; Often; Usually or Always

t� I blame myself for having put off my homework t� *�USZ�UP�HFU�PSHBOJ[FE�UP�HFU�PO�UPQ�PG�NZ�IPNFXPSL��t� I tell myself that this will never happen again. t� I ask myself: “How could this happen to me?” t� I go out and buy myself something. t� *�CMBNF�NZ�UFBDIFST�GPS�IBWJOH�HJWFO�NF�UPP�NVDI�IPNFXPSL�JO�UIF�mSTU�QMBDF��t� I take control and say to myself: “I can do this!”

ETHICS: REPRESENTATIVE ITEMS

Teacher Ratings

This student takes time to help others

RESILIENCE: REPRESENTATIVE ITEMS

Teacher Ratings

This student is easily discouraged

ETHICS

CREATIVITYRESILIENCE

UN

PAC

KIN

G T

HE

SIX

SKIL

LS


SituationalJudgement Test

You have been studying for a DMBTT�RVJ[�BOE�ZPV�BSF�HFUUJOH�tired. Your friends want you to stop studying and go to a movie with them. Although you think it would be fun, you EFDJEF�UP�DPOUJOVF�TUVEZJOH�GPS�UIF�RVJ[�instead of going with them. How likely are the following reasons for YOU doing this in the situation?

1. Studying hard is worth it because it will help me get into college one day.

2. 4UVEZJOH�GPS�RVJ[[FT�JT�FOKPZBCMF�CFDBVTF�*�MFBSO�XIJMF�*�BN�EPJOH�JU�

3. *�XJMM�OPU�HP�PVU�CFDBVTF�NZ�QBSFOUT�FYQFDU�NF�UP�EP�XFMM�PO�UIF�RVJ[�

4. It is more important for to me to get good grades than to go out with my friends.

Self-Report

I love to learn

CURIOSITY: REPRESENTATIVE ITEMS

Teacher Ratings

This student is a curious student

Biographical Data

How many times in the last month have you been late to class?

Self-Report

I take due dates seriously

Teacher Ratings

5IJT�TUVEFOU�JT�PSHBOJ[FE�with his/her schoolwork

TIME MANAGEMENT: REPRESENTATIVE ITEMS

CURIOSITY

TIMEMANAGEMENT

UN

PACKIN

G TH

E SIX SKILLS


Mission Skills ResearchEach of the six skills used in the MSA is built upon a sound evidence basis. Users have asked for more information about the research supporting each skill, and for resources available about other assessment tools and research-supported interventions to strengthen the skills. This information is encapsulated below.

TEAMWORKTeamwork can be seen as a near-exact synonym for collaboration, which is one of the 4 C’s highlighted by the Partnership for 21st Century Skills as being the most essential skills for students to learn and develop in school. Many schools include this skill in their mission statement, such as New Canaan Country School, which commits itself to “challenge students to collaborate.”

A key paper on this topic is entitled “Development and Validity Evidence Supporting a Teamwork and Collaboration Assessment for High School Students” by Wang, MacCann, Zhuang, Liu, & Roberts, published by ETS in 2009. This paper describes a research study performed with high school students using a very similar methodology as is used in the MSA, which found moderate predictive validity with academic achievement and other, later, measurements of student teamwork. In both that research study and in the MSA, teamwork JT�VOEFSTUPPE�UP�CF�SFMBUFE�UP�DPODFQUT�TVDI�BT�DPPQFSBUJPO �BEWPDBDZ�UP�BOE�JOnVFODF�PWFS�PUIFST �DPOnJDU�resolution, and guiding others.

For schools wanting to supplement the MSA scoring of student teamwork with other evidence-based NFBTVSFNFOUT�PG�UIJT�UZQF�PG�TLJMMT�JODMVEF�UIF�4PDJBM�$PNQFUFODF�4DBMF�GPS�5FFOBHFST �UIF�$POnJDU�3FTPMVUJPO�Scale, and the Peer Friendship Scale for Teenagers, all freely available on the STRIVE Together website,

(Reports: Beyond Content, Emotional Competence).

Evidence-based intervention programs to support student social skills and teamwork include Student Success Skills, Mindful Schools, Tribes, and Yale’s RULER. Find out more about each of these in the free online resource

JXLGH�IURP�3XEOLF�3URÀW��´6WUDWHJLHV�WR�3URPRWH�1RQ&RJQLWLYH�6NLOOV� “

Suggested Reading:

t� Boss, Larmer, and Mergendoller. PBL for 21st Century Success: Teaching Critical Thinking, Collaboration, Communication, and Creativity

t� Doorley and Witthoft. Make Space: Setting the Stage for Creative Collaboration

http://www.strivetogether.org/emotional-competence#overlay-context=growth-mindset-or-mastery-orientation


http://www.publicprofit.net/Services/Training/NonCognitiveSkillsGuide/


UN

PAC

KIN

G T

HE

SIX

SKIL

LS


CREATIVITYCreativity is likely the broadest and most complex of the six mission skills, and the one which most overlaps with the cognitive domain. It can be found in many a mission statement, such as in Duke School’s pledge to “inspire learners to boldly and creatively shape their future.” Many researchers have found strong correlations of creativity and

academic achievement, perhaps most notably in the wide-ranging work of Robert Sternberg. Creativity predicts graduate school outcomes beyond GRE verbal and math scores (Bennett & Rock, 1995, 1998) Creativity is NFBTVSFE�JO�UIF�.4"�JO�TFWFSBM�DPNQPOFOUT �JODMVEJOH�PSJHJOBMJUZ �JNBHJOBUJPO �BOE�nVFODZ �XIFSF�TUVEFOUT�BSF�asked to generate many ideas or associations. There are many other creativity assessment tools, including most prominently the Torrance Tests of Creative Thinking.

Suggested Reading:

t� Wagner, Tony. Creating Innovators t� Sternberg, Robert: Teaching for Wisdom, Intelligence, Creativity and Success and Handbook

on Creativityt� Beghetto and Kauffman. Nurturing Creativity in the Classroomt� Robinson, Ken. Out of Our Minds: Learning to be Creative.t� Sawyer, Keith. Explaining Creativity and other titles. t� ,FMMFZ�BOE�,FMMFZ��$SFBUJWF�$POmEFODF��6OMFBTIJOH�UIF�$SFBUJWF�1PUFOUJBM�8JUIJO�6T�"MM�t� Catmull, Ed. Creativity Inc. t� $TJLT[FOUNJIBMZJ �.JIBMZ��$SFBUJWJUZ��t� Seelig, Tina. InGenius: A Crash Course in Creativity and TEDxStanford Talk.

ETHICS

&UIJDT�JT�BOPUIFS�BSFB�PG�HSFBU�DPNQMFYJUZ��UIF�.4"�PSHBOJ[FT�JU�JOUP�UISFF�LFZ�DPNQPOFOUT �honesty, altruism, and concern for others. Integrity is a frequent synonym for ethics, and can be seen in many mission statements, as in this one: “The Lexington School seeks to instill integrity.”

Altruistic Behavior, Concern for Others, and Empathy surveys for teens are all available freely at the STRIVE

Together website (Resources/Reports/Beyond Competent/Emotional Management). Some of the interventions suggested above for teamwork and collaboration also will support student development of concern for others as measured in this ethics construct.

Suggested Reading:

t� Lickona, Thomas. Educating for Charactert� Sternberg, et.al: Teaching for Wisdom, Intelligence, Creativity, and Success



UN

PACKIN

G TH

E SIX SKILLS


RESILIENCEResilience is built from the components of core self-evaluation (i.e. positive self concept and TFMG�FGmDBDZ �XIJDI�SFTFBSDIFST�BSF�mOEJOH�JT�B�WFSZ�JNQPSUBOU�iBDBEFNJD�NJOETFU w�BOE�effective coping in the face of adversity, which can be related to perseverance, a much-touted capacity in recent years. MSA co-constructor Rich Roberts says that resiliency also is closely related to the Big Five personality trait of Emotional Stability (Neuroticism).

4FMG�FGmDBDZ�JT�TUSPOHMZ�DPSSFMBUFE�XJUI�BDBEFNJD�BDIJFWFNFOU �BT�XBT�GPVOE�JO�B�MBOENBSL��NFUB�BOBMZTJT�CZ�.VMUPO �mOEJOH�B��FGGFDU�TJ[F�GPS�BDBEFNJD�QFSGPSNBODF �BMUIPVHI�UIF�EJSFDUJPO�PG�UIJT�DPSSFMBUJPO�JT�particularly complex. The Multon studies have found stronger relationships with low-achieving students, (Rosen FU�BM �Q��CVU�0&$%�1*4"�TUVEJFT�IBWF�GPVOE�UIF�PQQPTJUF��UIBU�TFMG�FGmDBDZ�IBT�NPSF�TJHOJmDBODF�GPS�BDIJFWFNFOU�BU�IJHIFS�MFWFMT�PG�TUVEFOU�QSPmDJFODZ��0&$%�� *** �Q��3FHBSEJOH�TFMG�DPODFQU �SFTFBSDI�“suggests the correlational relationship between self-concept and academic outcomes is overwhelmingly positive.” (Rosen et. al, p. 128)

5IF�453*7&�5PHFUIFS�OFUXPSL�QSPWJEFT�TFWFSBM�BTTFTTNFOU�UPPMT�GPS�TUVEFOU�TFMG�FGmDBDZ �TVDI�BT�UIF�1"-4�4FMG�&GmDBDZ�TVCTDBMF �)BSUFS�T�4FMG�1FSDFQUJPO�QSPmMF �4UVEFOU�4FMG�SFQPSU �BOE�TFWFSBM�PUIFST�JO�JUT�Academic

6HOI�(IÀFDF\�FDWHJRU\.

&WJEFODF�CBTFE�JOUFSWFOUJPOT�GPS�4FMG�&GmDBDZ�BOE�4FMG�$PODFQU�DBO�CF�GPVOE�JO�UIF�FYDFMMFOU�6OJWFSTJUZ�PG�Chicago guide, Teaching Adolescents to Become Learners. Other resources include CASEL and the Yale Center for Emotional Intelligence.

Suggested Reading: t� Farrington, Camille. Teaching Adolescents to Become Learners. t� Rosen, et.al. NonCognitive Skills in the Classroom: New Perspectives on Educational Researcht� Tough, Paul: How Children Succeed. t� OECD PISA. Ready to Learn: Students’ Engagement, Drive, and Self-Beliefs, 2013.t� #BOEVSB �"MCFSU��4FMG�&GmDBDZt� Ginsburg, Kenneth. Building Resilience in Children and Teens: Giving Kids Roots and Wings t� Duckworth, Angela. “Grit: Perseverance and Passion for Long-Term Goals.” (2007)

http://www.strivetogether.org/academic-self-efficacy

http://www.strivetogether.org/academic-self-efficacy

UN

PAC

KIN

G T

HE

SIX

SKIL

LS


CURIOSITY (INTRINSIC MOTIVATION)Curiosity and intrinsic motivation are of enormous importance for our students. Shady Hill School announces in its mission its commitment that students will “become joyful, active learners,” and the cultivation of a life-long love-of-learning can be found in the promises of many schools. In the MSA this is measured by enthusiasm for learning and trying new

things and in independently seeking out learning. Many studies have demonstrated the value of motivation for BDBEFNJD�BDIJFWFNFOU��JO�B�SFDFOU�SFWJFX�PG��TUVEJFT�iUIBU�BOBMZ[FE�UIF�SFMBUJPOTIJQ�CFUXFFO�NPUJWBUJPO�BOE�BDIJFWFNFOU ��SFQPSUFE�QPTJUJWF�BTTPDJBUJPOT�PS�JOnVFODFT�w�3PTFO�FU��BM �� Q��4PNF�TDIPMBST�WJFX�curiosity and motivation as being closely associated and overlapping with Carol Dweck’s growth mindset, in which students believe that their learning and mastery is the result of their own effort, not any innate ability.

A few other motivation assessments are available for schools wanting to assess, such as the Achievement Goal Orientation and the PALS Mastery Orientation Scale, and there are also several growth mindset scales, all available at the STRIVE Together compendium (under Growth Mindset). Regarding interventions, Brainology is a widely used evidence-based tool to support the development of the Growth Mindset.

Suggested Reading:

t� Deci, Edward. Why We Do What We Do. t� Dweck, Carol. Mindset. t� Robinson, Ken. The Element: How Finding Your Passion Changes Everything. t� Tough, Paul: How Children Succeedt� Wagner, Tony. Creating Innovators

http://www.strivetogether.org/growth-mindset-or-mastery-orientation

UN

PACKIN

G TH

E SIX SKILLS


TIME MANAGEMENT5JNF�.BOBHFNFOU�XBT�UIF�MBTU�TLJMM�BEEFE�UP�UIF�PSJHJOBM�TFU�PG�mWF �TVHHFTUFE�CZ�&54�because of recent research conducted which found it highly valid for predicting academic achievement. It is composed of the following components: determining one’s needs, TFUUJOH�HPBMT�UP�NFFU�OFFET �BOE�QSJPSJUJ[JOH�BOE�QMBOOJOH�UP�NFFU�HPBMT��*U�JT�TFFO�BT�

DMPTFMZ�BGmMJBUFE�XJUI�$POTDJFOUJPVTOFTT �UIF�#JH�'JWF�QFSTPOBMJUZ�USBJU�XIJDI�NPTU�TUSPOHMZ�QSFEJDUT�BDBEFNJD�achievement (NRC, Education for Life and Work). A research study conducted at ETS about time management reported that “There is a growing body of research that suggests time management is positively related to academic performance.” (Burrus et.al, 2013, p.1)

Other middle school student assessments for related areas, such as self control, self-regulation, and learning strategies, can be found at 675,9(�7RJHWKHU��XQGHU�WKH�FDWHJRU\�RI�´6HOI�5HJXODWHG�/HDUQLQJ�DQG�6WXG\�6NLOOV,” TVDI�BT�UIF�"DUJWF�BOE�4VQFSmDJBM�-FBSOJOH�4USBUFHZ�6TF�4VSWFZ �2VFTUJPOOBJSF�PO�4FMG�3FHVMBUJPO �BOE�4UVEFOU�Approaches to Learning (SAL) Instrument.

Evidence based interventions for “Learning Strategies” are recommended by 3XEOLF�3URÀW�LQ�LWV�JXLGH, such as the “Student Success Skills” curriculum and the “Self-Authoring” student guided exercise. Farrington’s Chicago guide, “Teaching Adolescents to Become Learners,” also provides helpful information on effective strategies for learning study skills. Especially helpful and pertinent is an intervention implemented at Lawrenceville School, which included “assignments designed to improve the three facets of time management that students received GFFECBDL�PO�w�#VSSVT�FU�BM� �� Q��3FTFBSDIFST�SFQPSUFE�iJOJUJBM�TVQQPSU�GPS�UIF�FGmDBDZ�PG�UIF�UJNF�NBOBHFNFOU�JOUFSWFOUJPO��4QFDJmDBMMZ �FGGFDUT�XFSF�GPVOE�GPS�TFMG�SFQPSUFE�TUSFTT�BOE�BOYJFUZ�4UVEJFT��BOE��and advisor ratings of time-management-related behaviors (Study 2).” (p. 25).

Suggested Reading:

t� US Department of Education: Promoting Grit, Tenacity and Perseverancet� Hoerr, Thomas. Fostering Gritt� National Research Council, Education for Life and Work. 2012t� #VSSVT �FU�BM��i&YBNJOJOH�UIF�&GmDBDZ�PG�B�5JNF�.BOBHFNFOU�*OUFSWFOUJPO�GPS�)JHI�4DIPPM�4UVEFOUTw��

ETS. 2013t� Farrington, Camille. Teaching Adolescents to Become Learners

http://www.strivetogether.org/self-regulated-learning-and-study-skills


Appendix

APPEND

IX


Frequently Asked Questionsfor Educators

:KDW�GR�WKH�WHUPV�6-7��VHOI�UHSRUW��ELR�GDWD��RYHU�FODLPLQJ��DQG�ÁXHQF\�PHDVXUH� refer to?

Situational Judgment Test (SJT):

The Situational Judgment Test technique presents an individual with a description of a problematic or critical situation (generated from observers, surveys, and interviews of specialists to discuss situations and appropriate corresponding behaviors vital for success) followed by a number of possible problem-solving responses to this situation. In an SJT item, a scenario is presented to the participant. This scenario is followed by instructions to the participant to rate the quality of various courses of action associated with this hypothetical situation. The participant is required to make a judgment for each and every course of action using rank-orders, multiple choice, a binary decision (i.e., appropriate or inappropriate), or a Likert-type rating scale (e.g., very unlikely [1] to very likely [4]). Self- report: A term used to describe an assessment/survey that requires an individual to report about his or her behavior, attitudes, or feelings about a particular subject. It should be noted that one of the shortcomings of self-report BTTFTTNFOUT�JT�B�DPODFSO�BCPVU�TPDJBM�EFTJSBCJMJUZ�JOnVFODJOH�JOEJWJEVBM�SFTQPOTFT�F�H� �POF�DIPPTFT�UIF�socially correct response or answers in a way to make him/herself appear better). However, this limitation can easily be reconciled by also collecting data from external raters (e.g., teachers/parents also rate the students’ skills and responses are compared). Self-reports have been applauded for their affordability and ease of administration. Bio data:

Refers to factual questions about an individual’s life experiences. For example, name, age, education, hobbies, etc., are all types of bio data. For this survey, students will also provide information about lateness, planning, PSHBOJ[BUJPOBM�IBCJUT �BOE�DIFBUJOH� Overclaiming:

Refers to excessive assertions of knowledge and familiarity with a topic or event. In this assessment, overclaiming is measured through a series of items (some actual academic terms, works of art, or historic MBOENBSLT��PUIFS�iGPJMTw�IBWF�CFFO�JOWFOUFE�CZ�UIF�SFTFBSDIFST�UIBU�BTLT�UIF�JOEJWJEVBM�UP�SBUF�UIFJS�GBNJMJBSJUZ�with each. Fluency Measure:

Describes one’s ability to quickly produce multiple ideas or solutions to a problem in a limited amount of time. For the purposes of this study, students are given three minutes to produce lists of objects and ideas that fall within a given category (e.g., “blue”).

APPE

ND

IX


+RZ�ZHUH�WKH�WHVW�LWHPV�JHQHUDWHG"

The test items included in this study were created based on psychological and educational theories. Some of these instruments are brand new, and others have been tested numerous times in middle school (and several other) populations and have undergone extensive factor analysis to ensure that each instrument and subsequent sub-scales are both reliable and valid. Also of note, each item included in the assessment has been through an extensive fairness and sensitivity review to ensure the items are developmentally appropriate for middle school students and ethical.

+RZ�ZLOO�WKH�WHVWV�EH�VFRUHG"

Each of the tests in this assessment suite has its own unique scoring rubric. Typically, for most, but not all, of the assessments responses are summed and reported at the scale level. In general, similar to the creation of the items, the scoring rubrics are based on both theory and sophisticated statistical techniques. Preliminary data analysis will be conducted to remove any items that appear to have been problematic for the students. Once poor items have been removed, additional analysis will be performed to give students scores for each measure.

Who developed the assessment and who will do the scoring?

The Center for Academic and Workforce Readiness and Success, at the Educational Testing Service (ETS).

Do we follow standard testing guidelines for students with test accommodations (stu-

dents who are eligible for readers, extended time etc.)?

When necessary, any accommodations can be made for students who require additional time or aid. Further, teachers may use their own discretion if he/she feels a student needs more time to complete the survey. Please OPUF��IPXFWFS �UIBU�POF�NFBTVSF�'MVFODZ�JT�UJNFE��5IFSFGPSF �XF�SFDPNNFOE�UIBU�OP�BEEJUJPOBM�UJNF�CF�provided for any student when completing this part of the assessment.

Do students always feel comfortable answering all of these questions?

4PNF�TUVEFOUT�NBZ�FYQFSJFODF�B�MJUUMF�EJTDPNGPSU�XIJMF�BOTXFSJOH�TPNF�PG�UIF�RVFTUJPOT��4QFDJmDBMMZ �TPNF�students may be uncomfortable answering questions about cheating. It is important to remind the students that their answers are anonymous.

What grades will take the test?

The test is administered to all 6th, 7th, and 8th graders.

APPEND

IX


When is the test administered?

The test is administered once a year in the fall. The test window is set each year for the fall and communicated to schools in summer. Schools can administer the test any time within this window.

What equipment will our school need?

We suggest that schools administer the assessment on wired Mac or PC desktops or laptops. See separate Technical Overview and Best Practices Document for more detail.

+RZ�PXFK�VWXGHQW�WLPH�LV�UHTXLUHG"

It is estimated that the student part of the assessment will take about 35-40 minutes.

+RZ�PXFK�WHDFKHU�WLPH�LV�UHTXLUHG"

Each student must be “assessed” by a teacher (this can be an advisor, homeroom teacher, etc.) It takes approximately 5-8 minutes to “assess” one student. The teacher-rater assessment is also an online assessment.

+RZ�DUH�WKH�VWXGHQW�DQG�WHDFKHU�WHVWV�GHOLYHUHG"

Both the student and teacher-rater assessments are online (so each student and teacher must have access to a computer and internet.)

What outcome data are used?

We relate the results to a variety of outcome data. Some of the outcome data include: Absences/tardies, ERB TDPSFT �(1" �DPVSTF�HSBEFT �HFOEFS �HSBEF �ZFBST�BU�TDIPPM �FUIOJDJUZ �mOBODJBM�BJE�BXBSET �FUD��&BDI�TDIPPM�will be given a list of the outcome data needed for each student.

What does it mean to say that this is a Multi-Trait Multi-Method matrix assessment?

And convergent and divergent construct validity?

The MSA assessment uses a highly regarded methodology for establishing construct validity and reliability DBMMFE�UIF�.VMUJ�5SBJU�.VMUJ�.FUIPE�.5..�NBUSJY �mSTU�EFWFMPQFE�JO��CZ�$BNQCFMM�BOE�'JTLF��"T�UIF�OBNF�suggests, it requires that multiple, (the more the better), “traits” (or skills, etc.) be measured, and that each “trait” be measured in three (or more) entirely separate ways.

http://http://indexgroups.org/msa/docs/MSA2014_Technical%20Overview%20and%20Best%20Practices.pdf

APPE

ND

IX


MTMM models are useful for establishing two types of important construct validity: convergent, that very similar traits in theory when measured in multiple methods correlate closely, and discriminant, that different traits don’t correlate too closely with each other. In other words, MTMM models allow us to see whether individual traits or DPOTUSVDUT�BSF�SPCVTU��FGGFDUJWFMZ�NFBTVSJOH�UIF�TQFDJmD�USBJU�XF�XBOU�UP�NFBTVSF�BOE�BMTP�EJGGFSFOUJBUFE �OPU�measuring something else. MTMM methods establish the ability to compare various methods assessing the same trait. The MTMM design is basically a giant correlation table that will allow you to see how the various methods relate to one another.

Why were only grades six through eight selected?

Grades six through eight seemed especially appropriate for the MSA both because the students are at an age when the multi-method assessment technique can effectively be employed and, more importantly, because it is in these middle school years when these skills are especially malleable and especially important to be devel-oped for success in high school and beyond.

Who will see MSA results?

MSA results are sent in a comprehensive institutional report only to the administrator(s) designated by the school to receive them, and are the property of the school alone to do with as they choose, while abiding by the data-use policy which prohibits schools from using the data to disparage other schools. INDEX, ETS, and Pragmatics respect the privacy of school data and will never release data identifying individual schools without the school’s written approval.

APPEND

IX


Frequently Asked Questionsfor Parents

This is intended for schools to use either to assist in answering questions from parents or as a model PG�B�EPDVNFOU�TDIPPMT�BSF�SFDPNNFOEFE�UP�QPTU�PS�EJTUSJCVUF�UP�QBSFOUT��/PUF��.BOZ�TDIPPMT�XJMM�mOE�JU�NPTU�FGGFDUJWF�UP�XSJUF�UIFJS�PXO�WFSTJPO �CVU�GFFM�GSFF�UP�DPQZ�BOE�BNFOE�UIFTF�JO�BOZ�XBZ�ZPV�mOE�useful (with no need for attribution).

What is the MSA?

5IF�.4"�TUBOET�GPS�.JTTJPO�4LJMMT�"TTFTTNFOU �BOE�JT�B�TUBOEBSEJ[FE�UPPM�GPS�NFBTVSJOH�UIF�QSPmDJFODJFT�PG�groups of students in six non-cognitive areas, or character qualities, which are most commonly found in the mission statements of schools: Teamwork, Creativity, Ethics, Resiliency, Curiosity, and Time Management.

Who developed the MSA?

A group of independent schools with middle grades programs, now known as INDEX, originally set out to measure how their students were performing and improving in each of these skills, and engaged research scientists from ETS, the Educational Testing Service, maker of the TOFEL, GRE, and many other tests, to construct a valid and reliable test.

Can character really be measured?

5IJT�JT�BO�VOEFSTUBOEBCMF�RVFTUJPO �BOE�DFSUBJOMZ�UIF�BOTXFS�EFQFOET�PO�UIF�EFmOJUJPO�PG�UIF�XPSE�DIBSBDUFS��The MSA measures six skills commonly associated with “character,” and does so in a way that has been found to be reliable across multiple administrations and to demonstrate validity when compared to particular intended outcomes. But it makes no claim to effectively predict every potential element of what we consider character generally, so like all measurement tools, it is only measuring partial aspects of the true breadth and depth of students’ character.

What is expert opinion on the MSA and its effectiveness?

ETS is the global leader in the development of educational assessments, and its work is highly regarded. Each skill is assessed with evidence-based approaches and techniques drawn in most cases from peer-review journal published research. Researchers from the RAND Corporation in 2013 published a report entitled “Measuring

21st century Competencies” (freely available online) which singled out the MSA as an especially effective tool, praising it for its innovative and cost-effective approach, and its reliability and validity.

http://asiasociety.org/files/gcen-measuring21cskills.pdf

http://asiasociety.org/files/gcen-measuring21cskills.pdf

APPE

ND

IX


Why is the school administering the MSA?

It is recommended schools provide their own answer to this question.

+RZ�GRHV�LW�PHDVXUH�WKHVH�VL[�VNLOOV"�

ETS has implemented a multi-trait multi-method assessment for each of the six skills. In each area, teachers complete a report on each student’s demonstrated skills, and students complete both a self-report, and complete specially designed tasks that reveal their strengths. This “triangulation” method has been found to be FGGFDUJWF�JO�NJOJNJ[JOH�UIF�XFBLOFTTFT�JOIFSFOU�JO�BOZ�TJOHVMBS�BQQSPBDI�TVDI�BT�UFBDIFS�FWBMVBUJPO��

&DQ·W�LW�EH�IDNHG"�

Students are not individually rated by the MSA, and hence this is a “low stakes” assessment for which there TIPVME�CF�OP�JODFOUJWF�GPS�UFTU�UBLFST�UP�GBLF�UIFJS�XBZ�UP�B�IJHIFS�TDPSF�CZ�JOnBUJOH�UIFJS�BOTXFST�PO�UIF� self-evaluation.

:LOO�,�DV�D�SDUHQW�JHW�P\�FKLOG·V�LQGLYLGXDO��UHVXOWV"�

No. The MSA is a tool for research nationally on critical “noncognitive” skills and their relationship to student academic achievement and life satisfaction, and for schools to use to evaluate their effectiveness, to plan for improvements, and to assess the success of their improvement efforts. But the tool is not built to be used for individual student data, and no individual performance data is provided by ETS to schools.

Is doing the MSA really worth the use of time it takes?

5IF�.4"�JT�WFSZ�FGmDJFOUMZ�BENJOJTUFSFE��JU�UBLFT�MFTT�UIBO�BO�IPVS�PG�FBDI�TUVEFOU�T�UJNF�UP�CF�DPNQMFUFE �and it takes approximately 5-8 minutes for a teacher to assess one student, on average. When comparing UIJT�TNBMM�BNPVOU�PG�UJNF�UP�UIF�HSFBUFS�TJHOJmDBODF�PG�UIFTF�TJY�TLJMM�BSFBT �BOE�UIFJS�SFTFBSDI�EFNPOTUSBUFE�impact on future success, both academically and professionally as well as in life-satisfaction, we think it is a very worthwhile use of time.

+RZ�ZLOO�WKH�VFKRRO�XVH�WKH�UHVXOWV"��

Each school should provide their own answer.

APPEND

IX


Checklist for School MSA Program Administrators

This is the checklist for staff members charged with MSA administration. Please also read the section on communicating and administering the MSA.

BEFORE THE MISSION SKILLS ASSESSMENT 9 Be sure that all constituencies (administrators, faculty, parents, and students) know why we are doing the

MSA and what the outcomes will be. (See elevator speech.)

9 *G�QPTTJCMF �FOHBHF�GBDVMUZ�JO�B�EJTDVTTJPO�BCPVU�UIF�EFmOJUJPOT�PG�UIF�TJY�TLJMMT�CFJOH�BTTFTTFE�BOE�IPX�UIFZ�are manifested in a school setting.

9 Reassure advisors/teachers to offer their best responses to the questions about their advisees/students. 8F�SFDPHOJ[F�UIBU�TPNF�RVFTUJPOT�SFRVJSF�TVCKFDUJWF �BOE�QPTTJCMZ�OPU�XIPMMZ�DPSSFDU �KVEHNFOUT��5IF�assessment is designed to test the skills in numerous ways. Their best opinion on a student’s skill level is all that is required for now. Results will be in the aggregate.

9 Reassure students that this is a fun “test”. All results are anonymous, and students should feel no pressure about the results. Please tell students to simply answer all questions honestly and truthfully.

9 It is expected that schools will test ALL 6TH, 7TH, AND 8TH graders.

9 Remind students that they are engaged in an important research study. This is great work and their DPOUSJCVUJPOT�XJMM�CF�TJHOJmDBOU��,FFQ�B�QPTJUJWF �BQQSFDJBUJWF �MJHIU�UPOF�

9 Students will complete the online assessment at different times, so please have a plan in place for those mOJTIJOH�FBSMZ�BOE�PS�GPS�UIPTF�XIP�OFFE�FYUFOEFE�UJNF�

9 Consider dedicating a meeting time slot for advisors/teachers to complete the MSA for their students. 1SPWJEJOH�UIJT�UJNF�SFnFDUT�PO�IPX�NVDI�ZPV�WBMVF�UIJT�XPSL�

DURING THE MISSION SKILLS ASSESSMENT 9 Circulate around the testing room to address any issue one-on-one. Create a positive, enthusiastic tone.

9 The test is designed to take about 35-40 minutes. This is not a timed test, however, so students can be HJWFO�FYUSB�UJNF�UP�mOJTI�

9 Students must answer every question.

9 Teachers and test supervisors can take student questions and provide assistance before and during the test so that the students fully understand and can answer the assessment questions. Examples of this

APPE

ND

IX


JODMVEF�HJWJOH�EFmOJUJPOT�UP�XPSET�F�H��SFTJMJFODF �QSPDSBTUJOBUF �FUD� �IFMQJOH�UIFN�UP�VOEFSTUBOE�UIF�ethnicity question, etc.

9 Teachers and test supervisors should not, however, view the student test.

9 If students seem anxious or perplexed by a question, reassure them that they only need to provide their best response. Again, they are not being assessed individually on their answers.

9 If you see a student who seems hindered by one question, encourage him/her to make their best response and move on.

9 1SPDUPST�TIPVME�OPUF�BOZ�JTTVFT �RVFTUJPOT�BTLFE �XPSET�OFFEJOH�UP�CF�EFmOFE �FUD��BT�XFMM�BT�TUVEFOU�*%�for any students that run into glitches (technical, etc.) during the test.

9 Be sure to thank all those involved, especially the students.

AFTER THE MISSION SKILLS ASSESSMENT 9 If possible, provide time in advising meetings and/or faculty meetings to gather observations about what

went well with the MSA, what could be better, what questions emerged. Please take notes and submit them to Lisa Pullman at [email protected].

APPEND

IX


Elevator Speech

The MSA is an important and innovative national project. It will provide our schools with objective, TDJFOUJmD�GFFECBDL�BOE�EBUB�PO�IPX�XFMM�PVS�QSPHSBNT�EFWFMPQ�FTTFOUJBM�DIBSBDUFS�PS�OPODPHOJUJWF�skills in our students, skills that we all know are important and which our school promises to deliver in our mission: Teamwork, Creativity, Ethics, Resilience, Curiosity, and Time Management.

Many research reports widely available show that that these skills are integral for success in school, college, and the 21st century workplace, and that these skills can be taught and improved over time.

One comprehensive national study by the RAND research corporation found the MSA the most innovative, cost-effective, and reliable assessment of 21st century skills presently available.

Most educators intentionally and successfully develop and hone these skills. But how can we do an even better, more intentional job of instilling these skills in our students?

Using the MSA helps schools create more clarity, focus, and accountability for character or social/emotional education. Schools can communicate more clearly what skills are most important, hold themselves accountable for delivering them, and be part of a powerful network with other schools that are teaching the same skills and sharing resources about best practices for their instruction.

The MSA will provide key information, direction, and much more for our schools to effect change and to improve the most important lessons of our students’ lives.

APPE

ND

IX


Suggested Additional Reading and Resources

GENERAL BOOKSt� Bandura, Albert. 6HOI�(IÀFDF\

t� Dweck, Carol. Mindset

t� Ginsburg, Kenneth. Building Resilience in Children and Teens: Giving Kids Roots and Wings

t� Hoerr, Thomas. Fostering Grit

t� Lickona, Thomas. Educating for Character

t� Light, Richard. Making the Most of College: Students Speak Their Minds

t� Nettle, Daniel. Personality: What Makes You the Way You Are (An Introduction to the Big FIve

Personality Traits)

t� Peterson and Seligman. Character Strengths and Virtues

t� Robinson, Ken. The Element and Out of our Minds

t� Rosen, et.al. 1RQ&RJQLWLYH�6NLOOV�LQ�WKH�&ODVVURRP��1HZ�3HUVSHFWLYHV�RQ�(GXFDWLRQDO�5HVHDUFK

t� Sedlacek, William. %H\RQG�WKH�%LJ�7HVW��1RQFRJQLWLYH�$VVHVVPHQW�LQ�+LJKHU�(GXFDWLRQ

t� Sternberg, Robert. Wisdom, Intelligence, and Creativity Synthesized; College Admissions in the 21st Century��and many other titles

t� Tough, Paul. +RZ�&KLOGUHQ�6XFFHHG

t� Wagner, Tony. Creating Innovators and The Global Achievement Gap

REPORTS AND RESEARCHt� 0HDVXULQJ��VW�&HQWXU\�&RPSHWHQFLHV��$VLD�6RFLHW\�DQG�5$1'�&RUSRUDWLRQ��

t� Teaching Adolescents to Become Learners: University of Chicago Consortium on Chicago School Research

��

t� 86�'HSDUWPHQW�RI�(GXFDWLRQ��)RVWHULQJ�*ULW��7HQDFLW\��DQG�3HUVHYHUDQFH��&ULWLFDO�)DFWRUV�IRU�6XFFHVV�LQ�WKH��VW��FHQWXU\��

t� 3HOOHJULQR�HW�DO��(GXFDWLRQ�IRU�/LIH�DQG�:RUN��1DWLRQDO�5HVHDUFK�&RXQFLO��

t� ´%XLOGLQJ�D�&DVH�WR�'HYHORS�1RQFRJQLWLYH�$VVHVVPHQW�3URGXFWV�DQG�6HUYLFHV�7DUJHWLQJ�:RUNIRUFH�5HDGLQHVV�DW�(76�µ��(76��

http://www.amazon.com/gp/r.html?R=3OOODHI71592I&C=3T5MUR8F9569O&H=9U3MXPAYO5IMYIF9SXMGPNOJU1WA&T=C&U=http%3A%2F%2Fwww.amazon.com%2Fdp%2F0674004787%2Fref%3Dpe_385040_30332200_TE_item

APPEND

IX


t� ´([DPLQLQJ�WKH�(IÀFDF\�RI�D�7LPH�0DQDJHPHQW�,QWHUYHQWLRQ�IRU�+LJK�6FKRRO�6WXGHQWVµ�>DW�/DZUHQFHYLOOH�6FKRRO@��(76��

t� ´3HUVRQDOLW\��0RWLYDWLRQ��DQG�&ROOHJH�5HDGLQHVV��$�3URVSHFWXV�IRU�$VVHVVPHQW�DQG�'HYHORSPHQW�µ��(76��

t� $UH�7KH\�5HDG\�WR�:RUN"��3DUWQHUVKLS�IRU��VW�&HQWXU\�6NLOOV�DQG�WKH�&RQIHUHQFH�%RDUG��

t� 6SHFLDO�5HSRUWV�IURP�WKH�7KLQN�7DQN�RQ�WKH�)XWXUH�RI�$VVHVVPHQW��DQG��6HFRQGDU\�6FKRRO�Admission Test Board.

t� 2(&'�3,6$��5HDG\�WR�/HDUQ��6WXGHQWV·�(QJDJHPHQW��'ULYH��DQG�6HOI�%HOLHIV��

t� 0HDVXULQJ�6NLOOV�IRU��VW�&HQWXU\µ��(G6HFWRU�5HSRUW��

VIDEOSt� Science of Character, Video by Let it Ripple

t� $QJHOD�'XFNZRUWK�7('�7DONV�RQ�*ULW

t� Introduction to the Mission Skills Assessment (MSA)

DATA INFORMED DECISION MAKINGt� Boudette, City, & Murane. 'DWD�:LVH��8VLQJ�$VVHVVPHQW�5HVXOWV�WR�,PSURYH�/HDUQLQJ�DQG�'DWD�:LVH�

in Action

t� Wiggins and McTighe, 6FKRROLQJ�E\�'HVLJQ

EVIDENCE-BASED ASSESSMENT TOOLS AND INTERVENTIONSt� STRIVE Together: Compendium of assessments (self and teacher, mostly) for key non-cognitive attributes.

(http://www.strivetogether.org/resources/introduction-social-and-emotional-learning-competencies_)

t� Strategies to Promote Noncognitive Skills. Public Purpose. IUUQ��XXX�QVCMJDQSPmU�OFU�4FSWJDFT�5SBJOJOH�NonCognitiveSkillsGuide/

t� What Works Clearinghouse: Institute of Educational Sciences. http://ies.ed.gov/ncee/wwc/

t� CASEL: Collaborative for Academic, Social, and Emotional Learning (CASEL) Guide on Effective SEL Programs http://www.casel.org/guide

t� Yale Center for Emotional Intelligence http://ei.yale.edu/

http://www.letitripple.org/character

http://www.indexgroups.org/video/MSA-Intro.pptx

http://www.strivetogether.org/resources/introduction-social-and-emotional-learning-competencies_

http://www.publicprofit.net/Services/Training/NonCognitiveSkillsGuide

http://www.publicprofit.net/Services/Training/NonCognitiveSkillsGuide

http://ies.ed.gov/ncee/wwc

http://www.casel.org/guide

http://ei.yale.edu

APPE

ND

IX


AcknowledgementsOur thanks go to the following individuals who supported this work by participating in interviews, responding to email inquiries, and reading and reviewing drafts of this guide.

Chuck Baldecchi, Lexington School (KY)

7LP�%D]HPRUH��IRUPHUO\�1HZ�&DQDDQ�&RXQWU\�6FKRRO��&7��QRZ�&DWOLQ�*DEHO�6FKRRO��25�-XOLH�%LDQFKL��'HWURLW�&RXQWU\�'D\�6FKRRO��0,�Wendy Calise, Countryside Montessori School (IL)

-HQHHQ�*UDKDP��6W��0DUJDUHW·V�(SLVFRSDO�6FKRRO��&$�Jaime Lassman, Lexington School (KY)

-HQQLIHU�3KLOOLSV��)DU�+LOOV�&RXQWU\�'D\�6FKRRO��1-�.LUVWHQ�5RVROHQ��1HZ�&DQDDQ�&RXQWU\�6FKRRO��&7�.DWKOHHQ�6KDUS��&DSH�+HQU\�&ROOHJLDWH�6FKRRO��9$�

$OOLVRQ�:HEVWHU��6KDG\�+LOO�6FKRRO��0$�

FROM ETS0HJKDQ�%UHQQHPDQ��(G�'�3DWULFN�.\OORQHQ��3K�'�.HYLQ�3HWZD\��3K�'�6DP�5LNRRQ��3K�'�

OTHERStacy Kruse, Pragmatics

%HFN\�0F*UDWK��,1'(;��*UDSKLF�'HVLJQ5LFKDUG�5REHUWV��3K�'��3UR([DP

Thanks also to the members of the MSA Steering Committee for their work in leading this program:

&KXFN�%DOGHFFKL��+HDG�RI�6FKRRO��/H[LQJWRQ�6FKRRO��.<�7LP�%D]HPRUH��+HDG�RI�6FKRRO��&DWOLQ�*DEHO�6FKRRO��25�'DYH�0LFKHOPDQ��+HDG�RI�6FKRRO��'XNH�6FKRRO��1&�1HG�0XUUD\��+HDG�RI�6FKRRO��(SLVFRSDO�'D\�6FKRRO��*$�-HQQLIHU�3KLOOLSV��'LUHFWRU�RI�,QWHUPHGLDWH�6FKRRO��)DU�+LOOV�&RXQWU\�'D\�6FKRRO��1-�/LVD�3XOOPDQ��([HFXWLYH�'LUHFWRU��,1'(;��0$�&LQG\�6WDGXOLV��'LUHFWRU�RI�)LQDQFH�DQG�2SHUDWLRQV��'ZLJKW�(QJOHZRRG�6FKRRO��1-�$OOLVRQ�:HEVWHU��$VVLVWDQW�+HDG�RI�6FKRRO��6KDG\�+LOO�6FKRRO��0$�

APPEND

IX


Data Informed Decision Making: A Primer

“We found no evidence that good to great companies had more or better information than the com-parison companies. None. Both sets of companies had virtually identical access to good information. The key, then, lies not in better information but in turning information into information that cannot be ignored.” Jim Collins, Good to Great.

Schools have a responsibility to their mission, their students, and their accreditation to not ignore their informa-tion, and school-leaders should take positive steps to ensure they have a process thoroughly established and XJEFMZ�SFDPHOJ[FE�JO�UIFJS�PSHBOJ[BUJPOT�GPS�VTJOH�EBUB�UP�JOGPSN�EFDJTJPO�NBLJOH��

Two excellent resources can serve as foundation and framework for schools as they work to improve and implement their process. One is Grant Wiggins and Jay McTighe’s companion volume to their landmark 8QGHUVWDQGLQJ�E\�'HVLJQ��6FKRROLQJ�E\�'HVLJQ��0LVVLRQ��$FWLRQ��DQG�$FKLHYHPHQW�

THE WIGGINS AND MCTIGHE SBD MODEL CALLS FOR AN ONGOING 3-STEP CYCLE:

In SbD, a chief responsibility (“Job Function 3”) for every school-leader is that of “Gap Analysis,” managing the above step 2 by using multiple “measures that matter” to “identify and close the inevitable gaps” between mis-sion and reality. After generating effective and mission driven goals, administrators must be very careful to “take time to clarify the needed assessment evidence and initial data before they generate a detailed action plan.” To do so, the authors advise to “think photo album, not snapshot” by employing a broad portfolio of different types PG�NFBTVSFNFOUT��FYFSDJTF�DBVUJPO�BCPVU�iBTTFTTJOH�XIBU�T�FBTZ�UP�NFBTVSF�JOTUFBE�PG�XIBU�T�WJUBM�w�VTF�GFFE-CBDL�GSPN�TUVEFOUT�BOE�TUBGG��BOE�iBOUJDJQBUF�DPODFSOT�BOE�NJTVOEFSTUBOEJOHT�w

An increasingly clear vision of where we want to end up, based on our mission and agreed-upon learning principles.

"�DPOTUBOU�BOE�VOnJODIJOH�assessment of where we stand at present against the mission.

Timely adjustments based on regular analysis of the gap between vision and reality, between goals and results.”

1 2 3

APPE

ND

IX


In the third SbD phase, it is time for action. A set of tactics are suggested:

In a chapter for teachers, Wiggins and McTighe call for ongoing, embedded, “action research.” Every department or division of teachers should, every year, as a job requirement, would “issue a written report to all staff describing performance strengths and weaknesses measured by local assessments against program goals, actions taken to improve areas of weaknesses, and areas of concerns to be targeted via future action research.”

THE DATA WISE “IN ACTION” 8-STEP CYCLICAL PROCESSThe other excellent resource for this work is the pair of Data Wise books, the original and the “In Action” volumes, edited by a team at Harvard’s Graduate School of Education, Boudett, City, and Murnane. They’ve developed an eight step cyclical process, which loops back in on itself at phase three and four, for “using BTTFTTNFOU�SFTVMUT�UP�JNQSPWF�UFBDIJOH�BOE�MFBSOJOH�w�5IF�mSTU�UXP�TUFQT �MBCFMFE�i1SFQBSF w�IBT�UFBNT�PG�FEVDBUPST�TUFQ��iPSHBOJ[JOH�GPS�DPMMBCPSBUJWF�XPSLw�JOUP�DPNNJUUFFT�XJUI�DSJUJDBM�SFTPVSDFT�QSPWJEFE �BOE�UIFO�(step 2) “building assessment literacy” among themselves and their colleagues. (The two books provide very helpful primers on assessment literacy to support this step.)

In the second phase, “Inquire,” working groups set out to (step 3) “create a data overview” by providing data WJTVBMJ[BUJPOT �DBSFGVM�DPNQBSJTPOT�PG�TUVEFOU�HSPVQ�QFSGPSNBODFT �BOE�BUUFOUJPO�UP�MPOHJUVEJOBM�USFOET��5IFZ�then help teachers “make their own meaning” of the data, and thus generate better “buy-in” to the problem they will choose to address. Also highly recommended in this step is for teachers, when possible, to experience the UFTU�JUTFMG �iUBLJOH�UIF�TBNF�BTTFTTNFOUT�BT�UIFJS�TUVEFOUT�JO�GVMM�PS�JO�QBSU w�UP�MFBSO�imSTU�IBOE�BCPVU�UIF�DPOUFOU�that is being assessed and the ways it is assessed.” (Not every test can be viewed by teachers, and the policy for MSA is not to have teachers see the student assessment.) In further sessions with teachers, they spend more time (step 4) “digging into the data,” asking essential questions, drawing inferences, identifying needs for additional information, and “triangulating the data” among multiple sources. They also determine what the “learner-centered problem” is in areas of under-performance, and what skills students need to improve. This phase culminates in closely (step 5) “examining instruction” related to the area of focus.

Finally, the loop builds to the third phase, “Act.” Begin by (step 6) “developing an action plan,” which after brainstorming potential solutions and selecting among them, will detail what instructional strategies will be

t� “Diagnose before prescribing:” ensure everyone understands the problem being solved.

t� “Go with talent and interest:” get the right people in the right places.

t� i.JOJNJ[F�EJTSVQUJPOT�w�.BLF�MJGF�MJWBCMF�EVSJOH�UIF�DIBOHFT�

t� “Practice and model what you preach.” Show and highlight student work exemplifying the changes you seek.

t� “Think big, start small, and go for an early win.”

APPEND

IX


implemented to address the problem, what this will look like when implemented in the classroom, and deciding in advance how you know if the plan is working. And don’t forget to “put the plan in writing.” The next step builds on the previous by taking more time for (step 7) “planning to assess progress,” choosing short, medium, and long term sources of data for evaluating your success. These might include homework assignments BOE�DMBTTSPPN�PCTFSWBUJPOT�JO�UIF�TIPSU�UFSN��NBKPS�UFTUT�BOE�QSPKFDUT �BOE�QPUFOUJBMMZ�JOUFSJN�CFODINBSL�BTTFTTNFOUT�JO�UIF�NFEJVN�UFSN��BOE�UIF�BOOVBM�GPSNBMJ[FE�TUBOEBSEJ[FE�UFTUJOH�JO�UIF�MPOH��'JOBMMZ �BGUFS�BMM�this preparation, study, and planning, it is time for (step 8) “acting and assessing.” Communicate clearly, build supportive teams, observe instruction, “promote consistency more than conformity,” align PD, “be honest in evaluating progress,” “celebrate success,” and then “revisit the criteria and raise the bar.”

Data Wise in Action provides a set of eight comprehensive case studies of schools employing this process. In one extended example, an elementary school discerns during the data overview step that though the school’s reading comprehension scores were higher than average, there was a previously overlooked performance gap in student answers on open-response test questions. Teachers “dug into the data” and found a matching pattern in in-class assessments called “reading response letters,” which among other things didn’t seem to improve much over the course of upper elementary grades. When examining instruction, it was observed that teachers varied widely in the way they assigned and assessed these tasks, a so-called “problem of practice,” and togeth-er they developed and implemented a new, stronger, protocol for this type of student work, and mini-lessons to support it. They also determined multiple ways to measure progress, including working in teacher teams to com-monly evaluate sample student work for signs of improvement at regular meetings, and of course, on the annual reading comprehension open-ended responses.

APPE

ND

IX


A MODEL CYCLE FOR USING DATA TO INFORM DECISIONSDrawing upon the excellent frameworks from Schooling by Design and Data Wise, we offer here our own model for school-leaders to use when strengthening their data informed decision-making.

1. Measure Your Mission: Select Carefully What to Assess Far too often it seems, we evaluate our success with only what is easy or conventional to assess, and not what is most important. Start with the mantra that you Measure Mission, and implement it by intentionally selecting what particular goals to assess by referring to what your mission calls upon your school to accomplish.

2. Lead, Communicate, Provide Resources It can’t happen by default or by delegation. Data informed decision making must start at the top, and the school-leader should demonstrate personal investment, concern, and expertise in this work. Communicate clearly and loudly what will be measured and why and how this work will be happening. Assign administra-tive resources-- time, money, space-- ensuring everyone knows where the responsibility lies and how the resources will be used.

Measure YourMission

Lead, Communicate,ProvideResources

InvolveFaculty

BuildCommunities

of PracticeUse Tools to MeasureWhat Matters

Develop & Align InternalAssessments

Study theEvidence,and DevelopData Literacy

Connect Dots and DeterminePriorities

Planand Act

Assess, Adjust,Repeat

Resources

Lead,Communicate,Provide

InvolvFacult

BCoC mm

ofo Pvety

itiesice

UUseMMeasureat tters

Develop &Align Internal

tto MWhaMat

Measure YourMission

AssesAdjus

aat

ss,st,

Evidence,and DevelopData Literacy

nnect ots and etermineDe

Pri

nAct

CoDoDe

Studdy y ththeE id1

23 4 5

6

78910

APPEND

IX


3. Involve Faculty If the change is to happen in the classroom, the planning must include the faculty. Ensure you have a faculty advisory committee or working group, (i.e. PLC, CFG, Action Research) charged with this responsibility and fully involved throughout the process.

4. Build Communities of Practice 5IJT�JT�IBSE�XPSL �BOE�EJGmDVMU�UP�EP�JO�JTPMBUJPO��$VMUJWBUF�B�iDPNNVOJUZ�PG�QSBDUJDFw�BNPOH�ZPVS�OFUXPSL �formally or informally, for mutual support, data sharing, and comparative practice. It is great to have both a wider network of schools studying similar issues (such as the Mission Skills) and a smaller set of “buddy schools,” with which you can have more intense discussion.

5. Use Tools to Measure What Matters ow, in deliberation with your assigned administrators, faculty working group, and colleagues externally, TFMFDU�UIF�mOJUF�TFU�PG�UPPMT�UP�NFBTVSF�ZPVS�TDIPPM�T�TVDDFTT�JO�BSFBT�XIJDI�NPTU�NBUUFS��

6. Develop & Align Internal Assessments Don’t stop with external tools. Carefully identify and develop those internal assessments-- capstone assignments, report cards, parent-teacher-student conferences, student self-evaluations, etc.-- which do or can be aligned with the external tools.

7. Study the Evidence, and develop Data Literacy

$PMMFDU�UIF�EBUB�BWBJMBCMF �BOE�NBLF�JU�FBTZ�UP�WJFX�BOE�TUVEZ��6TF�EBUB�WJTVBMJ[BUJPO�UFDIOJRVFT��B�QJDUVSF�is worth a thousand words. Be sure to use these activities as powerful “teachable moments,” for all partici-pants to strengthen their assessment and data literacy. Ask groups tough and even perhaps trick questions about the data to boost this savvy.

8. Connect Dots & Determine Priorities Go from studying the data to make meaning of it. Form essential questions and draw out meaningful inferences. Find patterns, generate hypotheses, and identify gaps between mission and reality, between dis-aggregated groups of students, between cohorts or grade levels. Expand the conversation to the full faculty, in study groups, and let them come to their own understandings as much as is possible: the buy-in is priceless.

9. Plan and Act Collectively brainstorm and research initiatives-- curricular, instructional, cultural-- which might address the gap you seek to close, trying to reconcile the competing value of actions which are rooted locally and supported on the ground with evidence-based remedies from outside your school. Prepare and distribute a written plan for the changes, the reasoning, the observable impacts, and the measures by which success will be determined.

10. Assess, Adjust, and Repeat Use multiple measures in varied time frames to evaluate success of your improvement initiatives, and review regularly. Adjust in the interim periodically as necessary, but usually only in tweaks, so as to not compromise the longer-term assessment analysis. Upon review of the long-term assessments, determine next steps, and then repeat the cycle.

www.indexgroups.orgLisa Pullman - [email protected]

http://www.indexgroups.org

mailto:lisapullman%40indexgroups.org?subject=

Documents

MSA Toolkit Sept 2014