Using SNAAP Data for Positive Change

Slide 1

Using SNAAP Data for Positive Change3 Million StoriesMarch 8-9, 2013

Sally Gaskill, DirectorAmber D. Lambert, Research AnalystAngie L. Miller, Research Analyst

Center for Postsecondary Research, Indiana UniversityBloomington

[email protected]

Session OutlineSNAAP Basics

Resources

Ways to Use SNAAP Data on campus and in the community

Group Discussion

2Introduction to SNAAP

3SNAAP MissionWe investigate the educational experiences and career paths of arts graduates nationally

Provide the findings to educators, policy makers, and philanthropic organizations

To improve arts training, inform cultural policy, and support artists

4Leadership PartnersIndiana University Center for Postsecondary Research

Curb Center for Art, Enterprise, and Public Policy at Vanderbilt

National Advisory Board

5Generous start-up fundingSurdna Foundation (leadership gift)Houston EndowmentBarr Foundation (Boston)Cleveland FoundationEducational Foundation of America

National Endowment for the Arts (4 consecutive grants)

SNAAP Basics: Who is surveyed?Participants drawn from..Arts high schoolsIndependent arts colleges, arts schools or departments in comprehensive colleges & universities

All arts alumni, all years (since 2011)2008-2010: surveyed selected cohorts

SNAAP began in 2008 and tested the questionnaire and survey operations with three annual field tests that were free to participating institutions. In 2011, we fielded the first national administration, surveying all alumni from each participating school. 7

Increasing Numbers2011 Administration36,000 respondents66 institutions

2012 Administration33,000 respondents70 institutions

Combining 2011 and 2012 respondents to create SNAAP Database with 69,000 respondents

Questionnaire TopicsFormal education and degreesInstitutional experience and satisfactionPostgraduate resources for artistsCareerArts engagementIncome and debtDemographics

The questionnaire is divided into 7 sections.9Participating institutions receive...Institutional Reports: Customized, ConfidentialSeparate reports for undergraduate and graduate alumniBoth quantitative and qualitative dataNew report on Recent GradsComparative data with other schools

Complete data file of responses

Workshops/webinars on how to use data

PASS AROUND COPIES OF SAMPLE IRs10Resources

11Aggregate Reports

Each year since 2009, SNAAP has published an Aggregate Report of all survey responses by level (high school, undergraduate and graduate).

Look under the Findings section of the SNAAP web site (snaap.indiana.edu): http://snaap.indiana.edu/snaapshot/findings_2011.cfm (2011 Aggregate Report)http://snaap.indiana.edu/snaapshot/findings_2010.cfm (2010 Aggregate Report)http://snaap.indiana.edu/snaapshot/findings_2009.cfm (2009 Aggregate Report)12Aggregate Report (detail)

2009, 2010, and 2011 available on website

13Annual Reports

SNAAP has published an Annual Report in both 2012 and 2011.

14Annual Report (detail)

The Annual Report outlines key findings from each years survey.15SNAAP Special Reports

This Special Report presents topical findings from the 2010 data16SNAAP Special Reports (detail)

This Special Report presents topical findings from the 2010 data17SNAAP DataBriefs

DataBriefs are sent via email and present selected findings from the most recent SNAAP data. They also profile SNAAP users and provide examples of how they are using their SNAAP data, and provide information about SNAAP conferences and administration. You can sign up to receive these on our website: http://snaap.indiana.edu/about/databrief_signup.cfm 18

Available at: http://snaap.indiana.edu/snaapshotThe SnaapShot is a compelling interactive graphic display of annual SNAAP findings. SnaapShot of 2011 Data: http://snaap.indiana.edu/snaapshot/SnaapShot of 2010 Data: http://snaap.indiana.edu/snaapshot_2010/19Using the Data

Here we provide examples of how SNAAP institutions have begun using their data.20Using the data ...

Sharing on campusAlumni and donor outreachRecruitmentAssessment and curricular changeStrategic planningAdvocacy and public policy

These are some of the primary ways to begin using your data -- if you are at an institution OR if you are a funder OR policy maker21Sharing on Campus

Sharing your own results on campus with campus leadership as well as faculty and staff22

One of the first institutions to share their data on-campus was Miami University of Ohio. This Assessment Brief was published by Miamis institutional research office after it received its SNAAP report of 2010 data. 23

Purdue University published a four-page report of its 2011 SNAAP data. 24

Purdue 2 of 425

Purdue 3 of 426

Purdue 4 of 4272012 SNAAP ResultsSarah B. CunninghamExecutive Director of Research

Virginia Commonwealth University has adapted a year-long strategy to roll out its SNAAP results. Here is the overview, provided by Sarah Cunningham, Executive Director of Research for VCUArts.

We rely on our academic leaders to shape the institution, most often through curriculum and content. The SNAAP alumni survey supports the decision-making of campus leaders. For VCU, one of the greatest values comes when the SNAAP survey gives a clearer voice to concerns and hunches already discussed within the institution. Alumni serve as a source of new ideas, but sometimes confirm existing goals or new initiatives. Each academic department is a critical part of the process. The survey information complements their leadership in terms of where the discipline is going and where it should be to serve our students.The new Dean had a tangible trajectory of the school starting from the 1950s. I think this was amazing - to pull out comments and quotes from the first women students to the first African-American students to the students who attended during the draft even though they had "no arts talent." This was a reminder that we, as much as anyone else, struggle to meet the times in which our schools are living, and that we are subject to the major shifts which are typical of larger society.I gave special information to departments that were lauded in some way by alumni or in statistics for special projects and efforts conducted (i.e., we have a design center that gives design students professional consulting experience and was noted by alumni). We tried to email existing faculty when they were positively acknowledged in narrative comments. SNAAP became a huge help in replying to press reports on the wages and income of sculpture and painting graduates. SNAAP data allowed us to provide a more accurate and comprehensive data to counter news reporting that favored the starving artist mythology.April-May 2012:2-6 weeks for reading through and selecting out information. Check-ins with the SNAAP national office regarding validity of interpretation.June-July: Presentation to Deans and select internal staff, including alumni affairs, development, and communications director. Discussion and comments led to a revision for a faculty-staff retreat. Mid-August: Powerpoint presentation of 2011 undergraduate results to Department Chairs and Program Directors (18+) at the Administrative Retreat. We used the data used for planning/thinking about the upcoming year. However, in the first year of survey reporting, I suggest you provide time to familiarize your community with the SNAAP Project, what it asks from alumni, and how it can be useful to the school. Mid-September: 4 weeks later, new PPT presentation of graduate results to Department Chairs and graduate study directors. Discussion of how graduate results relate to undergraduate results.Mid-October: Presentation to the Provost and other university administrators, including assessment officers.Mid-October: Apply for funding to develop a space that serves alumni by delivering skills requested in the SNAAP survey. While we had already planned to support this skill development through Creative Entrepreneurship courses, the survey confirmed goals in development. The survey provided a picture of skills relevant to alumni after graduation. In this way, we continue protect and develop the "fine arts approach" core that the alumni love and appreciate, but then provide alumni with opportunities to link back to VCU to gain skills they need as their careers develop in specific ways. Notably, we are making an effort to connect young alumni to alumni mentors. We are in the very beginning stages of this but it was something requested in the narrative comments. Mid-November-Spring: Report back to alumni via communications materials regarding some general results and information.

28Alumni and Donor Outreach

Newsletters29

The Houston School of the Visual and Performing Arts shared some alumni comments in its quarterly newsletter sent to donors, parents, alumni and friends. 30

The Herron School tells the story of one alumna in a post on its web site31

The College of Fine Arts at the University of Texas at Austin began sharing its SNAAP results in 2011 with the Deans letter in its quarterly print publication. In this example, the dean thanks his alumni for participating and shares selected findings: http://www.utexas.edu/finearts/alumni/alumni-snaap-thank-you-letter32

In 2012, UT Austin developed a web page that illustrates some of its findings http://www.utexas.edu/finearts/about/mission-vision/alumni-snapshot33

UT Austin shared portions of its Institutional Report (being careful to not identify any respondents). http://www.utexas.edu/finearts/sites/default/files/attach_download/utaustinfineartssnaap2011report.pdf34

The Herron School of Art and Design at IUPUI in Indianapolis created a web site to share selected findings. 35

Herron 2 of 236Recruitment

37

This web page from Kent State shares aggregate SNAAP findings to make the case for a visual arts education. 38

The Herron School of Art and Design created a recruitment brochure based on its alumni achievements. This page includes comments from the SNAAP survey. 39

Herron (Creativity) 2 of 240Assessment and Curricular Change

41Using SNAAP for Curricular Assessment and Change

The SNAAP survey asks alumni to rate the importance of 16 different skills and competencies to their current profession or work life (whether they work in the arts or in a different occupation). 42Using SNAAP for Curricular Assessment (cont.)

Most important skills:Creative thinking and problem solvingListening and revisingInterpersonal relations and working collaborativelyBroad knowledge and educationCritical thinking and analysis of arguments and informationRecommend that faculty:Incorporate open-ended projects (top skill #1) and group projects (top skill #3)Require analysis of theories or reviews/critiques (top skill #5) and provide opportunities for feedback and revision (top skill #2)Ensure curricula include a firm knowledge foundation in a wide variety of areas (top skill #4)In 2011, the most important skills identified by alumni were those listed above. Based on this information, a school could recommend that faculty build in assignments as outlined above. 43Using SNAAP for Curricular Assessment (cont.)

This question asks how much the institution helped the respondent acquire or develop each of the 16 skills and abilities. 44Using SNAAP for Curricular Assessment (cont.)

Identify strengths: What skills have the highest % of alumni reporting the institution helped them develop very much or quite a bit?Identify areas for improvement: What skills have the highest % of alumni reporting the institution helped them develop very little or not at all?Peer group information provides context: Do other institutions have similar strengths and weaknesses?45Examples From 2011 Aggregate Findings

This chart shows the 2011 overall responses to three of the skills questions. An interpretation of how to analyze these findings is contained on the next slide. 46Using SNAAP for Curricular Assessment (cont.)

Alumni receive strong training in learning artistic techniques

Discrepancies between those who say a skill is important for their work and those who say the institution helped them develop that skill suggest some improvements that could be made, such as:Requiring business and financial classes, or incorporating these elements into existing courses Include classes looking at the nontraditional career paths of arts graduatesUsing SNAAP for Program Assessment

Here, we distinguish program assessment (extra-curricular or non-credit programs) from curricular assessment (courses of study). 48Using SNAAP for Program Assessment (cont.)

Programs and services with low satisfaction may need to be revised

Career advising had 59% report either very dissatisfied or somewhat dissatisfied

Additional resources could be devoted to developing new components of career advising such as:Alumni career panel presentationsRsum or portfolio building sessionsNetworking opportunities for graduating studentsHere are some ideas on how to respond to the data in the previous slide. 49Strategic Planning

50Strategic PlanningProvides an overview to make decisions about the future direction of the school

Reinforces leadership by documenting success

Proof that you are involved in analyzing effectiveness of your institution

All institutions are regularly engaged in strategic planning. SNAAP data can be used to inform many planning topics. 51Advocacy and Policy

52Advocacy and PolicyIn your local community

With state and other public sector funders

With donors

SNAAP data can be used for advocacy and public policy purposes. You can use your own institutions data as well as the national findings that we report. 53Institutional Location & Careers

In the first 5 years after leaving your institution, did you take up residency in the town/city where it is located to pursue you career?This table provides aggregate 2011 data on whether alumni pursued their careers in the city where their institution is located.54

Source: snaap.indiana.edu/pdf/2012/2012_Annual_Report.pdf

This table from the SNAAP 2012 Annual Report provides findings on arts alumni support for the arts, broken down into three categories. 55

This is an example of a Letter to the Editor that uses SNAAP findings to demonstrate the impact of the arts in a community. 56

The dean of the University of Florida College of Fine Arts uses SNAAP data to talk about the value of arts training.57

SNAAP National Advisory Board Members publish op-eds that advocate for arts education based on SNAAP data. Here, Ann Markusen, an economist, writes about important traits of arts graduates. 58

SNAAP findings make it into articles about the value of getting a law degree in Forbes.com

59

SNAAP Research Director Steven Tepper and Board Member Douglas Dempster wrote about the real value of an intensive education in the arts for Inside Higher Ed. 60

The Chronicle of Higher Education published this compelling piece by Steven Tepper and George Kuh about the unique role of arts training in teaching creativity.61

The New York Times and International Herald-Tribune covered the publication of SNAAPs 2012 report. 62

Steven Teppers 2011 piece on the value of an arts degree appeared at the time of spring commencements. 63

This excerpt from the 2012 SNAAP Annual Report demonstrates that the unemployment rate for arts alumni is half of that of the general population. 64Discussion

Allow audience to discuss in small groups how they or their institutions might be able to use SNAAP data65Some discussion questions...What are other ways of using SNAAP data for positive change?

What obstacles might you might face in using SNAAP data?

How might SNAAP data benefit your institution?

Questions to discuss in small groups, then share with larger group66Strategic National Arts Alumni ProjectIndiana University Center for Postsecondary ResearchSally Gaskill, [email protected] snaap.indiana.edu

67Assessment Brief #62October 12, 2011

Using Feedback from Miami Alumni to Improve Educational Effectiveness

Surveying AlumniMiami students are frequently surveyed throughout their college experiences. However, assessing the long-term impact of students Miami education can also require reaching out to students after they graduate. Feedback from alumni, who are now using the skills they developed at Miami, can greatly improve educational effectiveness. The Strategic National Arts Alumni Project (SNAAP) survey gathers information about fine arts alumni to better understand the relationship between arts education and arts-related occupations. The SNAAP participants from Miami University consisted of 220 undergraduate fine arts alumni who graduated in the following years: 1990, 1995, 2000, and 2005-2009.The survey included questions about institutional experiences and career choices. To capture institutional experiences, the survey prompted alumni to report their overall satisfaction with their education as well as their satisfaction with specific areas (e.g., academic advising, freedom to take risks). In the career section, alumni reported their current and previous occupations, their satisfaction with these jobs, and their current level of fine arts engagement.

To explore the intersection between institutional experience and careers, the survey asked alumni about the skills and competencies they developed at Miami University as well as which skills were most important in their current job.

By reviewing these results, faculty and staff can better understand how students experiences at Miami prepare them for their career.Key Findings Fine arts alumni were satisfied with their experiences at Miami University; 94% of undergraduate arts alumni rated their overall experience as good or excellent.

Arts alumni were especially satisfied with their sense of belonging at Miami and with their instructors.

Respondents were least satisfied with opportunities to network with alumni and others, advice about further education, career advising, and work experience. The vast majority of respondents reported developing critical and creative thinking skills while at Miami and found these skills important in their future careers. Fine arts alumni were less likely to report that Miami helped them to develop business and technological skills related to their field.

Student Satisfaction

RecommendationsThe SNAAP survey highlights the importance of gathering alumni feedback. Such feedback is a valuable resource for assessing educational impact and improving educational effectiveness across the university. The SNAAP survey helps faculty and staff in fine arts by identifying the following: Common occupations and post-secondary degrees among graduates

Skills and competencies that students will frequently use in their careers

Levels of student satisfaction with various aspects of their Miami experience These results can help the division improve retention and graduation rates and better prepare students for their future careers.If you have comments or questions, please contact the Center for the Enhancement of Learning, Teaching and University Assessment at [email protected] or 513-529-9266. Previous Briefs are available online at: http://www.units.muohio.edu/celt/assessment/briefs/. Miami University

Assessment

Documents

Using SNAAP Data for Positive Change