Try, Try, Try Again: Learning from Technology Failure in Advising

Farifteh Shahbazian, M.S. & Marina Zarate, M.P.A.October 12, 2017

Introduction & Overview• The institution • Our story• The importance of failure• What we tried & how we failed

– Tools (pros and cons)– Leveraging resources

• Data & research• Lessons learned

Learning Objectives

• The benefits of failure

• Pros and cons of technology tools

• Creating opportunities to be flexible & adaptable

• Focusing on continuous improvement methodology

California State University, Fullerton Institution Snapshot

• 4-year public institution with over 40,000 students

• HSI (Hispanic-Serving Institution)

• Eight Colleges • One in four students on

campus is a business major

Mihaylo College of Business and Economics Snapshot

• Largest business school on the West Coast

• Dual accreditation in AACSB (Accounting and Business Administration)

• Over 9000 students• Approx. 1,500-1,800 new transfer

students admitted each semester • Mandatory new transfer student

advising

52% 48% TransferStudents

First-timeFreshman

Our Story• Modernizing a stagnant system

– The Business Advising Center was first established in 1979

– Assistant Dean of Academic Services appointed to oversee a team of graduate student advisors • Faculty advising was used to supplement a

“professional” advising system– Processes and policies were developed that

remained in place over the course of 32 years

• Online BA in Business Administration introduced in 2012– Opportunity to build in changes that incorporated

technology-based initiatives • There was no precedent for updating processes

– The Business Advising Center was the first college-based full-time professional advising center

– Changing mindsets related to technology-use• This required a shift in paradigm at both the micro level

(other advisors) and macro level (administration)– Understanding our students

• Commuter campus with a growing population of technology savvy interests

The Importance & Benefits of Failure• Deeper analysis of potential problems

– Can draw attention to previously overlooked issues and challenge status quo

– Learn more effectively from failure• Increased search for solutions

– Large failures may elicit protectiveness, but small failures can fuel exploration

• Motivation to make changes– Triggers the need to take corrective action

• Willingness to take risks and decreased complacency– Moderate risk taking is associated with increased innovation

(Madsen & Desai, 2010) (Sitkin, 1992)

What We Tried & How We Failed

• Initiative-based • Tech tools implemented

– Instructional technology– Learning Management

System – Web conferencing

• Pros and cons of each • Things to avoid

The Inception of “Online Advising”

• All transfer students admitted to the Mihaylo College are required to meet with advisor– What happened to those students that could not meet in-

person? (out of area, working professionals, international, etc.)

• Online advising became an option• Existing process for new students lacked interactive

element

New Transfer Student Advisement

Spring 2017• Titanium community• Camtasia tutorial • Provided email advising upfront• Gave option to meet “online”

Fall 2017• Titanium community• Worked with Instructional

Design team to create interactive tutorial with quizzes

• Added several steps (multiple advisor-student exchanges)

• Delayed advising• Required “online” meeting

Instructional Technology

Pros • Interactive, can be entertaining• Used assistance of experts • Can be thorough and

condensed – include all relevant information in 15 minutes

• Accessibility – adaptable to learning styles (visual + auditory, CC, asynchronous, ability to play back)

Cons• Learning new technology and

self-training• Time – on other people’s

schedule• Not easy to edit• Gaining access and frequency

of report – no instant access• Lack of clean data – glitches in

the system and report structure was difficult to analyze

New Transfer Student AdvisementSpring 2017• Advisement offered to 21 students• All 21 students completed

necessary steps • None opted for in-person advising

instead• 100% online advising completion

rate• 18 of 21 advised online enrolled in

classes (86%)

Fall 2017• Advisement offered to 25 students• 17 students completed tutorial

(average quiz score 73%)• 13 students completed all necessary

steps • 9 students opted for in-person

advising instead• 52% online advising completion

rate• 10 of 13 advised online enrolled in

classes (77%)

New Transfer Student Advisement

Lessons learned:• Adding more components decreased the likelihood of advising completion • When provided with option to complete tutorial (more components), more

students opted out of online advising • Getting buy-in from students is difficult • Success = online advising is effective (students who completed online

advising correctly registered, declared, enrolled full-time) – Of those who completed tutorial, fewer came in for in-person advising– Built relationships with students who participated in online advising– Student connection to college

• Failure = not as many students opted in (volume does not match workload)

Building an Online Community • Create a space for incoming students• Help students feel affiliated and welcomed to the college

(without the direct human interaction)• Ease of access to information not readily available

– Central location to obtain information given to other students in-person

Building an Online Community

Pros • Existing resource• Familiarity• Easy for advisors• Add downloadable content• Embed videos

Cons• Students are not familiar

when first entering University

• Students did not see the value

• Not easily accessible by students

• Tedious maintenance for advisors

Learning Management System – Moodle

Online Conferencing• Integrated as a method of one-to-one advising• Campus rolled out Skype for Business in fall

2016 for use amongst faculty/staff• Opportunity/tool for expanded online advising • Training and downloading to use Skype for

Business• After implementation for less than one

semester, campus announced use of Zoom • Preferred method to communicate with

students• Re-training for use and capacity of program • Ever-changing technology

Skype vs. Zoom

Pros & Cons

• Pro: Zoom is easier to access for the student – direct link – No downloading additional

components• Pro: More reliable connection &

fewer bandwidth issues• Pro: Screen sharing

• Con: no option for document sharing

Pros & Cons

• Pro: Most students have access to Skype for personal use

• Pro: Screen sharing • Pro: Document sharing options

• Con: Additional download component for those that do not have access to Skype

• Con: Connectivity issues • Con: Less secure in sharing

information

Online Walk-in Advising

• Two semester pilot• 186 hours total• Number of students

served: 32• Format did not

work• But students want

this option! (conundrum)

Adobe Connect

Pros • Convenient and easy to use• Can chat privately with

multiple students at once• Can present PowerPoint• Can share screen, provide

handouts

Cons• Not the correct tool to use

for instant messaging – Inefficient interface – Have to manually save

transcript of the chat– Students logging in can

use any name: no way to verify identity

• Discontinued on our campus

The “Online Advising” Survey

• Original implementation of the online advising process was based on the advisor perspective– What did we think the students wanted?

• After failing for two semesters, we asked ourselves, what do our students actually want?

• Survey launched to 9000 active students – Approximately 400 responses

• Here’s what we learned…..

Online Advising PreferencesHow would you define “online advising?”

How likely are you to use online advising options?

Extremely Likely

Moderately Likely

Slightly likelySlightly likely

Which of the following online advising methods would you use INSTEAD of meeting with your advisor in-person?(ranked 1-4 in order of most preferred to least preferred)

1) I would ask my advisor questions via email

2) Instant messaging my advisor

3) I would have a phone meeting with my advisor

4) I would meet my advisor online via Skype or another online conferencing program

What are the reasons you might choose to utilize online advising versus in-person advising? (mark all that apply)

I believe it would be more convenient 35.75%

I believe I would receive a faster response/answer to my question 24.89%

In-person advising availability does not fit my schedule 23.83%

None of the above, I would prefer to meet with myadvisor in person 12.52%

Other (please indicate): 3.02%

What are the reasons you might choose to utilize online advising versus in-person advising? “Other”

Why Did This Blow Our Minds?• We were surprised with how students defined online advising

– We wanted to create a multi-tiered, intricate system for delivery of services, but students wanted a simple solution

• Primary concerns of launching an online system– FERPA and privacy– Complicated questions with complicated responses– Record keeping – Becoming too prescriptive– Losing human interaction element – Lack of advising staff to handle volume

• Goal was to create a system to address concerns, however, students reported on the survey that they do in fact value face-to-face connection and will continue to seek that when they run into complex issues

Connecting the Dots: Research in Practice

• Online advising can be used for basic processes to allow the advisor more time to spend on one-on-one advising (Thompson & Prieto, 2013).– Students reported that they will self-select to meet in-person if

questions are more complex– But are likely to use online advising for basic or “one word” answer

questions

• Online students and on-ground students want the same thing: personalized experience and quick response time (Allen, Smith & Muehleck, 2014).– The reason students would use online advising is because they

believe they would receive a faster response

Connecting the Dots: Research in Practice• Even though students report that they are

interested in online advising, they tend to prefer a mix of online advising techniques and face-to-face meetings depending on their needs (Shana & Abdullah, 2014).– Some students reported that they would

prefer to meet their advisor in-person and would only use online advising as a last resort

Things to Avoid• Overly complex/complicated tools or processes requiring

additional training that can be time consuming• Going in blind – put tools to the test, start with soft

launch/beta• Waiting too long to implement

– Technology changes so often and can become outdated quickly

– Buy-in without quick execution can result in decreased motivation and confidence, a need for retraining, and less likely to get buy-in again.

Progressive Attitude• Out with the old, in with the new

– Leveraging existing resources – seek out new and existing tools and think outside of the box in how to use those tools

– You might get told no… keep moving • Create and maintain buy-in

– Regardless of failures, be transparent in your plans and future intentions

– Communicate desire to move forward and highlight successes (no matter how small)

• Continuous improvement– Don’t allow failure or set-backs to cause you to lose

momentum

Continuous Improvement (CI) Methodology

• Management-based methodology usually applied to manufacturing and product development, but can be applied to different types of work environments

• Incremental improvements through innovation

• Search for source of problem, waste, and variation⎻ Find ways to minimize them⎻ On-going cycle of improvement

• Decreases likelihood of repeating mistakes

(Bhuiyan & Baghel 2005)

Continuous Improvement (CI) MethodologyDifferent Typologies

• Organic CI ⎻ Multifunctional work group⎻ Improvement activities not left to experts⎻ Decision-making not left to authorities

• Individual CI⎻ Suggestion system: individuals come up with

ideas which are then presented to specialists who will implement

⎻ Tie it to the mission and strategy of organization = buy-in

CI Cycle

(Bhuiyan & Baghel, 2005)

In Summary

• Dispel fear of failure• Focus on lessons learned & be

open to change• Inspire advisors to see

innovation through failure • Finding Your Process

(handout)

“I have not failed. I've just found 10,000 ways that won't work.”

-Thomas A. Edison

Contact InformationMarina Zarate, MPA

Senior Undergraduate Advisormjzarate@fullerton.edu

657-278-2214

Farifteh Shahbazian, M.S.Senior Advisor

fshahbazian@fullerton.edu657-278-8286

ReferencesAllen, J.M., Smith, C.L., & Muehleck, J.K (2014). Pre- and Post-Transfer Academic Advising: What Students

Say Are the Similarities and Differences. Journal of College Student Development, 55(4), 353-367.

Bhuiyan, N. & Baghel, A. (2005). An overview of continuous improvement: From the past to the present. Management Decision, 43(5), 761-771.

Madsen, P.M., & Desai, V. (2010). Failing to learn? The effects of failure and success on organizational learning in the global orbital launch vehicle industry. The Academy of Management Journal, 53 (3), 451-476.

Shana, Z. & Abdullah, S.A.K. (2014). SAAS: Creation of an e-Advising Tool to Augment Traditional Advising Methods. Computer and Information Science, 7(1), 41-57.

Sitkin, S.B. (1992). Learning through failure: The strategy of small losses. Research in Organizational Behavior, 14. 231-266.

Thompson, L.R. & Prieto, L.C. (2013). Improvising Retention Among College Students: Investigating the Utilization of Virtualized Advising. Academy of Educational Leadership Journal, 17(4), 13-26.