Sharing the Secret Recipe: Deconstructing Key Performance Indicators for

Quality Distance Social Work Education

Social Work Distance Education Conference

Indianapolis, Indiana

April 15, 2015

Debora Rice, Ph.D., MSW., LCSW

Pablo Arriaza, Ph.D., MSW., LICSW

Sally Brocksen, Ph.D., MSW., MBA

Presentation Objectives

• Critically evaluate performance measures used in traditional higher education

• Identify factors for determining quality in online education around teaching, learning, and support

• Present key performance indicators (KPIs) used in Walden University’s Social Work Program

• Discuss how three experienced online social work professors interpret and use KPIs to identify quality and encourage performance

Reluctance to Embrace Online Education

• Only 28% of higher education leadership say their faculty accept the “value and legitimacy of online education”

• 66% of faculty consider online courses as inferior or somewhat inferior to a comparable face-to-face course

• 79% of faculty say they have concerns about the quality of online education at for-profit institutions

• Social work faculty specifically appear to have an embedded hesitation to embrace social work education in an online environment

References:

Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium.

Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium.

Conceptualizing Quality Performance

Learning outcome measures

Student evaluation of faculty

Faculty self-evaluation

Evaluation of technology delivery system

Absent

Absent

Learning outcome measures

Student evaluation of faculty

Faculty self-evaluation

Evaluation of technology delivery system

Automated student accountability

Faculty Key Performance Indicators (KPIs)

Learning outcome measures

Student evaluation of faculty

Faculty self-evaluation

N/A

Absent

Absent

Traditional Walden’s OnlineEarly Distance Education

Today’s online environment requires:– Evidence-based benchmarks specific to the

online environment

Online social work education requires:– A baseline for accreditation and reaffirmation of

hybrid and online programs

Questions for Today

• What factors (beyond the traditional) help determine quality performance in online instruction?

• How can Walden University's KPIs and Lead Faculty Model demonstrate successful evaluation and high quality of teaching in the online environment?

Factors That Help Determine the Quality of Online Instruction

• Responsiveness & availability• Frequency & quality of presence• Facilitation of discussion• Prompt, quality feedback• Developing cooperation among students• Responding to diverse talents & styles• Management of administrative aspects• Setting high expectations

References:

Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet

and Higher Education,15(2), March 2012, 118-126.

Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-14.

Operationalizing Walden’s Model

• KPI data goes to the Program Director weekly• In cases where KPIs are not met, data goes to

the individual professor and Lead Faculty for immediate action

• Lead Faculty serve as:– Subject matter experts (SME)– First line problem solvers– Evaluators– Coaches/Cheerleaders

Walden University’s

Key Performance Indicators

(KPIs)

Walden’s Key Performance Indicators (KPIs)

• Log into class 4 out of 7 days• Participate in discussion posts 4 out of 7 days• Respond to at least 2/3 of student posts to each

discussion question• Timely grade submissions with specific,

substantial feedback• Assignments and grades for discussion questions returned within 7 calendar days

from the due date on the course calendar

Additional Walden KPIs

• Personalize the classroom– Upload picture, CV, and brief narrative

• Review explicit class expectations• Provide substantial feedback to promote/enhance students’

critical thinking• Respond to students’ questions on the Class Café and

Contact the Instructor tabs within 48 hours• Use Announcements to update students, provide relevant

external links/resources, and promote a sense of presence • Create and maintain a respectful and safe environment in

the classroom

Interpreting, Embracing, and Applying Walden’s Model of Excellence in Distance Education

The Trajectory

and Experiences

of Three Core

Faculty Professors

_ _ _ _ ___ _______ __________

Three Stories

Debbie’s Story

Pablo’s Story

Sally’s Story

Questions?

Additional indicators relevant to social work?

References

Allen, I. E. & Seaman, J. (2012). Changing course: Ten years of tracking online education in the United States. United States: The Sloan Consortium.

Allen, I. E. & Seaman, J. (2012). Conflicted: Faculty and online education. United States: The Sloan Consortium.

Allen, I. E. & Seaman, J. (2014). Grade level: Tracking online education in the United States. United States: The Sloan Consortium.

Boling, E.C., Hough, M., Krinsky, H., Saleem, H., Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences, The Internet and Higher Education, 15(2), March 2012, 118-126.

Council on Social Work Education (CSWE) (2012/2013). Annual report. Retrieved 24 October, 2014 from www.cswe.org/File.aspx?id=68989

Crawford-Ferre, H.G. and West, L.R. (2012). Effective online instruction in higher education, The Quarterly Review of Distance Education, 13 (1), 2012,11-

14.

References

Cummings, S.M., Chaffin, K.M., Cockerham, C. (2015). Comparative analysis of an online and a traditional MSW program: Educational outcomes, Journal of Social Work Education, 51:109-120.

Flynn, M. Maiden, P. R. , Smith W., Wiley, J. & Wood. G. (2013). Launching the virtual academic center: Issues and challenges in innovation. Journal of Teaching in Social Work, 33, 331–338.

Lackey, K. (2011). Faculty development: An analysis of current and effective training strategies for preparing faculty to teach online, Online Journal of Distance Learning Administration, 14(4). Retrieved from http://westga.edu/distance/ojdla/wnter144/lackey144.html

Shelton, Kaye. (2011). A review of paradigms for evaluating the quality of online education programs. Online Journal of Distance Learning Administration, IV(I), Retrieved from http://www.westga.edu/~distance/ojdla/spring141/shelton141.html

Thyer, B. A. and Polk, G. (1997). Distance learning in social work education: A preliminary evaluation, Journal of Social Work Education, 33(2), 363-368.