Joining Minds Group Modeling to Link People, Process, Analysis, and Policy Design

G. P. RichardsonJuly 2006

1

Rockefeller College of Public Affairs and PolicyRockefeller College of Public Affairs and PolicyUniversity at AlbanyUniversity at Albany

Joining Minds Group Modeling to Link People, Process, Analysis, and

Policy Design

GP Richardson, DF Andersen, LF Luna-ReyesRockefeller College of Public Affairs and Policy

State University of New York at Albany

(Presented at the annual meeting of the Association for Public Policy Analysis and Management, 2004)


2


Overview

• What is group modeling?• An extended example: Welfare Reform• Other cases• The Albany group modeling approach• Evaluating group model building efforts• Why does it work?


3


Ancestry of GMB

• GDSS• Quinn, Nunamaker, Eden & Ackmann, DeSanctis & Gallupe, …

• Decision conferencing• Milter & Rohrbaugh, Schuman & Rohrbaugh, …

• System dynamics• Forrester, Richardson & Pugh, Sterman, …

• Mental models & systems thinking• Checkland, Senge, …

• For a rich history, see Zagonel


4


What is Group Modeling?

• A form of group decision support, involving a group of stakeholders with a complex problem• Group facilitation

• Model building and refinement in public

• Simulation of scenarios and options

• Extensive facilitated discussion and analysis

• Facilitated policy design and decisions


5


What is Group Modeling?

• Management team (10-20) with a Modeling/Facilitation team (2-4)

• Four full days over 3-to-4 months• Extensive between meeting work• Rapid prototyping of model with finished simulation

product• Facilitation of implementation plans


6


Primary GMB references in the System Dynamics Community

• “Decision modeling”: Reagan-Cirincione et al.

• “Teamwork”: Richardson & Andersen

• “Scripts”: Andersen & Richardson

• “Group model building”: Vennix

• Special issue of the System Dynamics Review on GMB (1997)


7


Why System Dynamics Modeling?

SystemConceptualization

Representation ofModel Structure

Perceptions ofSystem Structure


8




ModelFormulation


Comparison andReconcilation


Diagramming andDescription Tools


9



Empirical Evidence


ModelFormulation




Alternative Models,Experience, Literature



10



Empirical Evidence


ModelFormulation




Alternative Models,Experience, Literature

Empirical andInferred Time

Series

Comparison andReconciliation.

Deduction OfModel Behavior


Computing Aids


11



Empirical Evidence


ModelFormulation




Alternative Models,Experience, Literature Literature


Series




Computing Aids


12



Empirical Evidence


ModelFormulation




Alternative Models,Experience, Literature Literature


Series




Computing Aids

StructureValidatingProcesses

BehaviorValidatingProcesses


13


The Albany Teamwork Approach

• Facilitator / Elicitor• Modeler / Reflector• Process coach• Recorder• Gatekeeper


14


Components of the Process

• Problem definition meeting• Group modeling meeting• Formal model formulation• Reviewing model with model building team• Rolling out model with the community• Working with flight simulator• Making change happen


15


A Typical Room GMB Session


16


An Example: Welfare Reformin New York State Counties

• Initial interest within NYS Department of Social Services

• TANF model in Cortland County• Safety net model in Dutchess County• Joined TANF/SafetyNet model in Dutchess• Calibration in Cortland, Dutchess, & Nassau• Implementations in Cortland & Dutchess


17


First Group Model Building Meeting

• Introductions: Hopes and Fears• Stakeholders• Introduction to simulation: Concept models• Client flow elicitation• Policy resources and clusters• Mapping policy influences• Next steps for client group and modeling team


18


Who Was in the Room?

• DSS Commissioner

• Deputy commissioner

• DSS director of medical services

• DSS director of administrative services

• DSS director of income maintenance

• NYS DSS representatives

• Health commissioner Mental health administrative manager

• Executive director of Catholic Charities

• Representative from the Department of Labor

• Minority leader of the county legislature

• Managed care coordinator


19


Introduction to Simulation

• Concept models• Introduce the stock, flow, and causal link icons used

throughout the workshop

• Demonstrate there are links between feedback structure and dynamic behavior

• Initiate discussion about the structure and behavior of the real system

• Less than 30 minutes


20


Concept Model Progression:“Models are ours to change and improve.”

Onassistance

At riskemployedJob finding

rate

Job loss rate

Avg length of stayon assistance

Fraction losing jobper year

Onassistance


rate

Job loss rate



Onassistance


rate

Job loss rate



Jobs

Employmentratio

Onassistance


rate

Job loss rate



Jobs

Employmentratio

Onassistance


rate

Job loss rate



Jobs

Employmentratio

Unemployedand

unassisted

Loss ofassistance rate

Avg total time onassistance

Fraction losingassistance per year

Onassistance


rate

Job loss rate



Jobs

Employmentratio

Unemployedand

unassisted

Loss ofassistance rate

Avg total time onassistance

Fraction losingassistance per year


21


Concept Model Progression:“Behavior is a Consequence of Structure”

At Risk Populations4,000

3,000

2,000

1,000

0

0 2 4 6 8Time (Year)

On assistance : welf1At risk employed : welf1


3,000

2,000

1,000

0




3,000

2,000

1,000

0




3,000

2,000

1,000

0




3,000

2,000

1,000

0


On assistance : welf3At risk employed : welf3Unemployed and unassisted : welf3


3,000

2,000

1,000

0


On assistance : welf3At risk employed : welf3Unemployed and unassisted : welf3


22


Client Flows in the Resulting TANF Model


23


Client Flows in the “Safety Net”


24


Confidence building processes

• Structure of the model emerging from group process

• Parameters based on administrative data everywhere possible

• Parameter and table function group elicitations

• Group contributions to tests of model behavior


25


Simulated vs Actual Caseload


26


Three Policy Mixes

• Base run (for comparison)• Flat unemployment rate

• Historical client behaviors

• Investments in the “Middle”• Additional services to TANF families

• Increased TANF assessment & monitoring

• Safety net assessment & job services

• Investments on the “Edges” • Prevention

• Child support enforcement

• Self-sufficiency promotion


27


Investing in the “Middle”


28


Investing on the “Edges”


29


Base, “Edges,” and “Middle” Compared:Populations on the Welfare Rolls


30


Total Job-Finding Flows from TANF


31


Program Expenditures


32


Emerging Lessons

• Unemployment dominates system performance

• Loss of eligibility will shift the next economic cycle’s costs and caseloads

• Endogenous management makes a smaller difference

• Employment programs at the middle of the system are low leverage points

• Policies at the edges of the system have high leverage

• Community-wide partnerships are needed to implement “Edge” policies

• Performance measures continue to be problematic


33


Resource allocation: Unpacking the Policy Resources for Implementation

• 43 participants about 30 agencies and organizations in the county

• Three stage process• 9 groups

• 6 larger groups

• 3 final groups

• Ending with five initiatives, costing about $675,000


34


Final proposals implemented in Cortland

• Job center ($150K)• Centralized location for all referrals

• Resource center ($150K)• Coordination of community effort toward diversion

• Program to support employed self-sufficiency ($200K)• Job counselors, case managers, private sector

• Computer-based comprehensive assistance ($150K)• Link all providers and case managers, shared database

• Expansion of child-care services ($75K)


35


Does It Work?

• Categories of evaluation data• Modeling team reflections

• Participant reflections

• Measurable system change

• Results• Methodological problems

• Implementation in about half of GMBs

• Positive measure of success in about half of the implemented interventions


36


Why Does It Work?

• Engagement• Mental models• Complexity• Alignment• Refutability• Empowerment


37


What are we really doing?

• Microworlds?• Data-based representations of a policy reality

• Tools for finding what options really work best to solve a complex dynamic problem

• Boundary objects?• Socially constructed representations of a negotiated world that

may not exist

• Tools for facilitating discussion and agreement in contentious environments

Documents

Joining Minds Group Modeling to Link People, Process, Analysis, and Policy Design