Complexity

Bruce Kogut

October 2006

We are entering the epoch of the digitalization of knowledge: past, present,

and futureSciences bring to this new epoch:

– More powerful statistical methods (some which date back 50 years in physics)

– Tools to manage and mine large datasets.– Methods, such as imaging, to make

inferences from ‘damaged’ and ‘missing’ data.– Visualization technologies that make our

standard powerpoint slides look very sad

What Complexity Seems

to Mean In Practice

Interdisciplinary sharing of knowledge and creation of a larger community of scholarship.

Appreciation of the ‘non-linear’ view of the world where critical events and self-organization matter.

Analyzing the statistical properties of large datasets (100,000+)

Understanding local interactions by micro-rules whose effects depend on topology (structure)

Identification of common patterns by re-scaling and normalization (e.g. power laws) to seek more general explanations.

Example from Venture Capital deals

• Over 200,000 transactions over 40 years in the US alone.

• Several thousand VC investors, targets

• Let’s start by posing a simple question:– Can we find rules by which VC companies do

deals?– Do these deals explain aggregate patterns?

Examples of Rule-based Formation

• We choose 18 firms that are connected from the actual data. Actual links are green lines.

• We simulate for 60 iterations, with each click representing implementation of the stochastic rule. Rules are analyzed one at a time.

• Red lines show outcome.

• We collect network statistics at the end.

Simulations with Four Rules

PreferentialRandom

Propensity Transitivity

Simulation Strategy: Estimate from Data,Simulate Forward to the Future

The common hypothesis is that Venture Capital Deals Favor the Big Players: Rich get Richer (sometimes called ‘preferential attachment’)

Fre

q7

5

K1 61

1

167.387

Fre

q8

0

K1 140

1

378.742

Fre

qu

en

cy9

0

K1 623

.694947

1527.65

Fre

qu

en

cy

K1 1104

.223825

3416.19

Inference by adduction: the dog did not bark, the graph does not show linear slopes, there are no power laws in degree, hence the culprit of rich get richer is innocent and released. So much for complexity, big science, and robust patterns.

Power Laws in Complex Weighted Graphs: Incumbents like to rely upon trusted partners

• Most Deals are Incumbent to Incumbent

• Hence we find power laws in repeated ties.

• Trusted expertise based on experience,

1

10

100

1000

10000

1 10 100 1000 10000 100000

Strength sF

req

uen

cy

These modest results can be shown to refute the leading hypothesis on venture capital: VC partnerships are not clustered by regions, they span regions even in the early history of the industry.

Complexity is Already Percolating, and We Need to Take Notice

• Organizational studies: claims of self-organizing teams and innovations.

• Finance: clustered volatility, whereby there is ‘memory’ and hence inefficiency in markets.

• Marketing: Data mining is the principal example.• Economics: Re-scaling shows remarkable commonalities

in size and growth distributions.• Macro-sociology: Dynamics result in scale-free networks.• Statistics: Search for new methods applicable to big

networks can sort out whether you smoke because you are born that way, or because you hang out with the (wrong/right?) friends.

• Operations Research: once again, we can do math.

Today’s Panel• Nigel Gilbert: sociologist and founding editor of the

Journal of Artificial Societies and Simulation, shows how ideas from complexity have implications for how we do social science and management research.

• Roberto Serra: Physicist, former manager, academic, now engaged in promoting complexity education.

• Ralph Dum: Physicist, EU Commission, Catalyst for Complexity Research, Wants to See a European Santa Fe Institute built.

• Jeff Johnson, Researcher in Design,Believer that Complexity requires math and math can be learned even by Ph.D. graduates.