Bell Laboratories1 Organizational Patterns James O. Coplien Bell Laboratories Naperville, Illinois, USA [email protected] JaCC Software Developers

Bell Laboratories 1

Organizational Patterns

James O. Coplien

Bell Laboratories

Naperville, Illinois, USA

[email protected]

JaCC Software Developers

16 September 1999

11:00 - 12:30

Bell Laboratories 2

Organizational Patterns

James O. Coplien

Bell Laboratories

Naperville, Illinois, USA

[email protected]

IIT Colloquium Series

Tuesday, February 8, 2000

M-50 Auditorium

Bell Laboratories 3

Overview

What are organizational patterns? Methodology Our model of organizational change Using patterns for change management The patterns themselves can be found

athttp://www.bell-labs.com/cgi-user/OrgPatterns/OrgPatterns

Bell Laboratories 4

What is a pattern?

A solution to a problem in a context Architectural patterns ideas first published by

Alexander in 1977 Look at issues of structure, not just parts Build on proven practice, not “promising theories” Adopted by the software community starting in 1990;

started gaining critical mass by 1993 Today, widely used as a design tool, especially in the

OO community

Bell Laboratories 5

What are organizational patterns?

Solutions to organizational problems in a context First appeared in the Alexander + software context at

PLoP in 1994 (Coplien, Whitenack); received with some skepticism

Now, a growing body of knowledge

Or, a construct from anthropology, Kroeber:» Universal patterns: transcend cultures» Systemic patterns: have a common root in an ancient

culture» Total culture patterns: give a culture its identity

Patterns define culture

Bell Laboratories 6

Why do we care about organizational patterns?

Because process-based approaches have serious limitations that hark back to the era of the industrial revolution

Because experience with project management, technology transfer, etc. show software to be a primarily social activity

Because of the tie to cultures, and the study of cultures is about social activity

Because we should care about human comfort in addition to product quality

Because they work!

Bell Laboratories 7

The Pasteur Process Research Project at Bell

Labs

Goal: To know how to design highly productive software development organizations

Premise: Organization drives process, which drives productivity

Strategy: Correlate properties of organizations with order-of-magnitude productivity

Technique: Patterns

Bell Laboratories 8

A Theory of Organizational Change

The idea of an organization as an anthropomorphic entity» Some are intelligent: good reactions, learn from mistakes» Some are stupid: react in insane ways, don’t learn

Organizations can learn! Organizations can’t be taught Therefore, learning is experiential

» Experience is a hard teacher

Can be helped with empirical foundations and introspection

Bell Laboratories 9

Empirical foundations

“Pattern Mining” Finding recurring structures and practices in

healthy organization To a degree, seeking the absence of these

structures in dysfunctional organizations A soft science Akin to anthropology and its many branches

Bell Laboratories 10

Organizational Pattern Mining

Based on social network analysis with informal extensions

Organizational role-play to gather social network data

Dyadic and triadic sociometric data Uses CRC cards Light facilitation Half-day exercise


CRC Cards: Classes, Responsibilities, and

Collaborators

Subsystem coord.

Validate MR listsBuild group products

Administer ENVY Resolve logical deps.

Subsystem coord.Change committee

DesignersSystem test


Work-life Role-Play

Identify project roles Study subjects play roles Development scenarios drive role-play Capture interaction & coupling on CRC cards Social Network Analysis Tools

» Organization Structure Visualization» Organizational Metrics

Capture Trends as Generative Patterns


Distilling the Patterns

Sociometric analyses» Pasteur analysis tool» Analytical studies of data

Catalog sociograms, sociomatrices, sociometric data

The pattern must recur The pattern must solve a problem Document in pattern form


QPW as a Subject

Fourth generation product in one generation Remarkable press reviews The most effective organization we have found

» Productivity» Quality» Interval» Market

Used to calibrate our “spectrum” Analysis done at my request I am not a QPW user


Organization Metrics

0123456789

10

2 3 4 5 6 7 8 9

Average Collaborators Per Role

Frequency

QPW


Writers’ Workshops

A forum to review patterns Borrowed from the creative literature community Designed to protect the dignity of the author Several basic steps:

» Moderator introduces author» Author reads the work» Author becomes a fly on the wall» Someone summarizes the pattern» The strengths are emphasized» Suggestions for improvement» Author asks for clarification» Thank the author


Using the Patterns Let dysfunctional organizations become

exposed to the patterns to see what is possible

Use organizational role-play as an introspection tool

Collect and process sociometric data from the role play

Present the data to the organization for a second round of introspection—a “mirror”

Learning and growth build on the insights


The Organizational Role-Play

Same technique as used for pattern mining Largely self-running Role-based

» Useful in most development organizations

Actor-based» Useful for diagnosing specialized team problems» “Ethnological” studies


Facilitation Guidelines Involve everybody Don’t try solving other peoples’ problems

» Other organizations must solve their own problems

» Interface issues can be dealt with in another forum» Interfaces generally involve politics

Don’t look for “the problem”» Problems are systemic» Solutions are generative» Let the patterns cause the “aha”s


More principals of organizational change

Happens bottom-up, perhaps with the top-down support of a sponsor or patron

Patterns are closely tied to value systems—which must be elicited bottom-up

Organizational change means building a new culture: reverse anthropology» New language

» New normative behaviors

» New values

» New symbols

» New stories (patterns)

» New rituals


What makes organizations excel?

No silver bullet Well-known productivity enablers A little luck The right balance of techniques to make

a working social (and economic and technical and...) system


Communication is Key

The “Buffalo Mountain” pattern Decentralized, yet a graceful distribution Almost fully connected Anti-schmisogenetic

» No splinter groups» No time-serial sub-processes» No “subroutines”


Communication Intensity

0102030405060708090

100

0 20 40 60

Number of Roles

% Saturation

QPW


Size

Easy to socialize the architecture Self-directing

» Correlates to Gerry Weinberg’s SEI/CMM parallel

» The true hallmark of a mature organization Divide and conquer Essentially large organizations need

hierarchy


Architect Also Implements

Problem: Preserving the architectural vision through to implementation

Context: An organization of developers that needs strategic technical direction.

Forces: Totalitarian control is viewed by most development teams as a draconian measure. The right information must flow through the right roles. The architect is one of the few people, or may be the only person, with the big picture. Big picture knowledge is important even during implementation, especially when using iterative development. But the architect usually isn’t around during implementation.

Solution: Beyond advising and communicating with developers, architects should also participate in implementation.

Resulting Context: A development organization that perceives buy-in from the guiding architects, and that can directly avail itself of architectural expertise. The architect learns from implementation constraints and may modify the architecture accordingly.

Rationale: Vitruvius notes: “...[A]rchitects who have aimed at acquiring manual skill without scholarship have never been able to reach a position of authority to correspond to their pains, while those who relied only upon theories and scholarship were obviously hunting the shadow, not the substance. But those who have a thorough knowledge of both, like men armed at all points, have the sooner attained their object and carried authority with them.” Larry Constantine emphasizes the value of “street creds” in an architect.


Distribute Work Evenly

Like QPW adjacency graph Sociometric analysis: low graph

centrality Can be remedied by load leveling,

staffing, splitting roles, combining roles


Organization Metrics for Distribute Work Evenly

05

1015202530354045

0 2 4 6 8

Communication Intensity Ratio

Number of Roles

QPW


Personnel Issues

Personal excellence and integrity Highly specialized Balance of teamwork (communication) and

individuation (specialization, personal sign-offs)

How does one assemble such a group?1. Magic: A secret that only Philippe knows

2. Outside interests? (Gabriel)

3. Self-selecting Team (another pattern)


Engage Quality Assurance

QA is tightly coupled in the social network

Large beta program–customer engagement

Developers reserve early verification to themselves

QA first-in, last-out


Reward Excellence

A sense of contribution» Reinforced in project signoffs» Reinforced in reward system

Financial incentives» No first-hand insights into QPW» Controversial, but claims demonstrable results

Two cultures with extravagant rewards:1. The U. S. West Coast

2. Financial trading

Overcomes fear of change


Process?

The ability to introspect well The ability to self-manage Organizational structure over task

sequencing Effective meetings, and lots of them


Conclusion

Organizational patterns are:» Patterns that capture the “architecture” of

organizations» In anthropology, the fabric of social structure

Role-playing data gathering and introspection Organizational change is generative Organizations can use patterns as exemplars

for improvement A rich body of literature exists and is growing

Documents

Bell Laboratories1 Organizational Patterns James O. Coplien Bell Laboratories Naperville, Illinois, USA [email protected] JaCC Software Developers