M AY 2 0 1 7

IN THIS DOCUMENT Management Guru Peter Drucker has famously said that “You can’t improve what you don’t measure.” This document explores an application of the TRTW to evaluate changes in team communication during a Business Process Reengineering project.

ABOUT IMPAQ INTERNATIONALWith research studies, program evaluations, implementation assistance, surveys and data collection, technical solutions, and communications strategies, IMPAQ helps governments, businesses, foundations, non-profits, and universities evaluate and enhance their programs and policy.

IMPAQ International evaluates and enhances public programs and policy.

ISSUE BRIEF AUTHORSAndres RomualdoYang Chen

ISSUE BRIEFT E A M R E L AT E D N E S S A N D T E A M W O R K F L O W

E V A L U AT I N G S TA N D A R D I Z AT I O N A N D C O M M U N I C AT I O N

OVERVIEWIMPAQ used Team Relatedness and Team Workflow (TRTW) Matrices before and after the adoption of a new business process at the Tennessee Department of Human Services.

The Team Relatedness Matrix measures team members’ assessments of the extent to which collaboration required for optimal performance. The Team Workflow Matrix reflects staff perception of the way that information flows between team members. The effectiveness of the TRTW in measuring team communication has been supported by research in the field of Industrial Psychology.1

In this Issue Brief, we describe the positive results of using the TRTW to measure communication among teams.

E V A L U AT I N G & E N H A N C I N G P U B L I C P R O G R A M S A N D P O L I C Y I S S U E B R I E F | 2

In 2014, the Tennessee Department of Human Services (TDHS) contracted with IMPAQ, through a federal grant, to develop a sustainable strategy for continuous improvement in the prevention, detection, investigation, and prosecution of Supplemental Nutrition Assistance Program (SNAP) recipient fraud and trafficking. IMPAQ worked collaboratively with the TDHS to provide training and technical assistance to facilitate Business Processing Reengineering (BPR).

BPR requires fundamental changes and redesign of workflow processes. It brings about dramatic improvements through analysis of workflow of tasks, jobs, and organizational structures. BPR techniques allowed TDHS to conduct a holistic assessment of their current system and processes, identify areas for improvement, and develop more efficient and effective workflow processes.

Prior to the start of the project, TDHS did not have standardized processes in place for developing and adjudicating SNAP trafficking claims. Throughout the life of the project, IMPAQ collaboratively developed new processes with TDHS. This iterative process included the development of tools, standards and protocols. TDHS staff vetted new processes and standards to ensure accuracy and consistency, and took the lead in writing protocols.

BACKGROUND

We used the TRTW to evaluate the level of shared understanding among staff before and after the implementation of new standardized processes. The TRTW measures two main components or factors: (a) team relatedness and (b) team workflow. In the past, the TRTW has been successfully used in diverse jobs and industries to identify tasks that require team work for optimal team performance and communication. Below we describe each of the two factors in greater detail.

All factors are rated on a 5-point Likert scale, as shown in Exhibits 1 and 2. Team relatedness reflects the extent to which working with members of the team is required for optimal team performance of specified tasks. In other words, it relates the extent to which the task is necessarily a group activity. Team workflow reflects the way that work and/or information between team members flows for the optimal performance of specified tasks for successful team performance.

IMPAQ administered the TRTW to TDHS Investigative Specialists twice – before and after implementing new standardized processes. Investigative specialists are responsible for screening claims, documenting investigation activities, obtaining employment and household verification; and entering case-related data. We asked investigative specialists to rate each of the 13 tasks that they commonly encounter in their jobs. We measured change by comparing the results of the post-project survey against baseline measures.

THE TRTW MATRICES

EXHIBIT 1: TEAM RELATEDNESS

EXHIBIT 2: TEAM WORKFLOW

1) Not a team task 2) Pooled/AdditiveInterdependence

4) Reciprocal Interdependence

3) Sequential Interdependence

5) IntensiveInterdependence

E V A L U AT I N G & E N H A N C I N G P U B L I C P R O G R A M S A N D P O L I C Y I S S U E B R I E F | 3

One of keys to the successful implementation and standardization of new workflow processes is a shared understanding among staff. In the SNAP Trafficking project, IMPAQ staff used a variety of tools and methods to support TDHS staff in the development of a clear, shared mindset about their work tasks. To complement qualitative evaluation methods on the improvement of communication, such as staff interviews and observations, we also sought to include quantitative methods. In this project, the IMPAQ team adapted the TWTR from the field of Industrial Psychology, and gained a powerful measure of the internal communication among staff. Our work demonstrates that methods from different fields of social science can complement the implementation of BPR and provide useful measurement of its effects on team communication.

DISCUSSION

EXHIBIT 3: TEAM RELATEDNESS: STANDARD DEVIATIONS

EXHIBIT 4: TEAM WORKFLOW: STANDARD DEVIATIONS

When analyzing the survey results, we focused on the changes in the level of staff agreement on the various factors related to workflow and communication. The degree of agreement is measured by the variability across staff in their perceptions of the team relatedness and team workflow of each task. For example, if most staff rate a task as “very much required to work with team members,” then there is low variability in the team relatedness factor of this task. On the other hand, if half of the staff rate the task as “not required to work with team members,” while the other half rate the task as “very much required to work with team members,” then the variability is high, suggesting that staff do not share the same mindset regarding this task. In our study, we calculate the standard deviations across staff’s responses to each task to quantify the variability in staff’s perceptions.

Exhibits 3 and 4 displays the standard deviations across 24 investigative specialists for team relatedness and team workflow, respectively. At the baseline, the average standard deviation across all tasks for team relatedness and team workflow was 1.42 and 1.54, respectively. The relatively high baseline standard deviation indicates that, on average, there was disagreement among Investigative Specialists at the beginning of the project. Not only was there a lack of agreement on what tasks were team-based and those that required independent work, there also seemed to be a lack of agreement on how the work flows between people and across processes.

As shown in Exhibits 3 and 4, standard deviations of most tasks decreased from the baseline to the follow-up. At follow-up, the average standard deviation across all tasks for team relatedness and team workflow dropped to 1.35 for both. The results suggest that the BPR project increased the shared understanding of tasks among the Investigative Specialists. After the project, they consistently agreed more on both team relatedness and team workflow of their tasks. The improvement provides evidence of an effective implementation of process standardization.

RESULTS

1.1

1.3

1.5

1.7

1.9

Baseline Follow-up

Selected Tasks Team Workflow: Standard Deviations

Task 6 Task 8 Task 10

Task 11 Task 12 Task 13

1.11.31.51.71.9

Baseline Follow-up

Selected Tasks Team Relatedness: Standard Deviations

Task 6 Task 8 Task 10 Task 11

Task 12 Task 13 Task 1 Task 4

Task 5 Task 6 Task 10 Task 11

E V A L U AT I N G & E N H A N C I N G P U B L I C P R O G R A M S A N D P O L I C Y I S S U E B R I E F | 4

1 Arthur, Edwards, Bell, Villado, & Bennett (2005). Team task analysis: Identifying tasks and jobs that are team based. Human Factors, 47, 654–669.2 Gupta, Crowne, & Noble (1999). Business Process Reengineering. Encyclopedia of Electrical & Electronics Engineering, 1(2), 645-657.3 Arthur, W. Jr., Glaze, R. M., Bhupatkar, A., Villado, A. J., Bennett, W. Jr., & Rowe, L (2012). Team task analysis: Differentiating between jobs and tasks using team–relatedness and team workflow as metrics of team task interdependence. Human Factors, 54, 277-295.

ANDRÉS ROMUALDO

A Senior Research Analyst at IMPAQ with over six years of experience in government consulting. Andrés is a Certified Lean Six Sigma Green Belt with interests in behavior and process changes. He was a co-presenter at the 2016 31st Annual Society for Industrial and Organizational Psychology Conference in Anaheim, CA for his work on advancing teamwork through value stream mapping. Andrés’ current work is focused on interviewing and site visitation for multiple government agencies.

YANG CHEN

Former Research Associate at IMPAQ, Dr. Chen received a Ph.D. in Economics from Ohio State University with specializations in Labor Economics, Applied Econometrics, Demographic Economics, and Law and Economics.

A special thanks to Alok Bhupatkar for his contribution to this document.

OUR EXPERTS

IMPAQ International evaluates and enhances public programs and policy. Questions and comments can be directed to IMPAQ International. For more information visit www.impaqint.com or call (443) 259-5500

W W W. I M PA Q I N T. C O M

IMPAQ’S MISSION

IS TO CREATE ENDURING

SOLUTIONS TO GLOBAL

PROBLEMS THROUGH

RIGOROUS APPROACHES

AND INNOVATIVE THINKING.