Block, Boda, Hollway, Vörös May 2017

Multiple methods for the statistical analysis of networks exist. Mainly: The SAOM/Siena model. The ERGM and various extensions (LERGM, TERGM, StERGM, ##ERGM).

Finding your way through this jungle is tricky, because: There are lots of models. Many modelling consequences are not spelled out explicitly. Everybody claims their model is the best. Some even claim (on questionable grounds) that other’s models are bad.

This presentation shall provide some guidance to model choice (and handling reviewer comments).

General: Actor-oriented vs. tie-oriented modelling (SAOM or ERGM). Longitudinal: Continuous-time vs. discrete-time (SAOM or TERGM). Not today: Continuous-time models (SAOM and LERGM).

Definition of the ERGM:

(See Block, P., Stadtfeld, C., & Snijders, T. A. (2016). Forms of Dependence Comparing SAOMs and ERGMs From Basic Principles. Sociological Methods & Research.)

ERGM: A model for the probability of a observed network, depending on network characteristics

(network statistics). Poss. interpretation: the probability of a tie, given the rest of the network. Poss. interpretation: the outcome of a tie-wise updating process in equilibrium.

SAOM: A model for the probability of network evolution, depending on network characteristics

(network statistics) from the perspective of an actor. Dominant interpretation: the choice of an actor to change one of its outgoing ties, rather

than any other possible option.

Compare how the micro-models of network transitions differ and what that means.

Definition of the ERGM on a transition level:

Definition of the SAOM on a transition level:

ERGM vs SAOM Per Block 8

Only feasible for small networks (<= 5 nodes)

For ERGM: Using the basic ERGM formula.

For SAOM: Use transitions between all network states.

1. How do the transition probabilities between adjacent networks differ for “equivalent” models?

2. How does this relate to different dependence assumptions?3. How does this leads to different meso- and macro-level network

features in their equilibrium state for identical target statistics?4. How do differences in stationary distribution lead to difference in

estimates for “identical” models?

ERGM vs SAOM Per Block 10

The SAOM naturally has a different level of dependence assumptions resulting from its actor-oriented nature.

Consider the following case:

ERGM vs SAOM Per Block 11

These are all possible transitions from Triad 012.

ERGM vs SAOM Per Block 12

The likelihood to reach any state with 2 ties in the next step in the ERGM with only a density parameter:

ERGM vs SAOM Per Block 13

These are all possible transitions from Triad 012.

ERGM vs SAOM Per Block 14

The same probability in the SAOM:

but:

ERGM vs SAOM Per Block 15

The existence of a tie influences the probability to form another tie by the same actor. The outgoing ties of each actor are dependent, even in a model that only contains a

density parameter. We can think of this as sender dependence. On the meso-level, this leads to a different outdegree distribution of the

SAOM compared to the ERGM.

ERGM vs SAOM Per Block 16

The outdegree distribution of the stat. dist. of an ERGM and a SAOM with a density of 0.2, 5 nodes and only a density parameter:

ERGM vs SAOM Per Block 17

The baseline models have a different degree of outdegree centralisation. Estimates of effects that model outdegree centralisation can differ by

model due to this difference in dependence.

ERGM vs SAOM Per Block 18

Dependence between ties differs not only for the density effect, but literally all effects.

The multinomial choice function introduces all sorts of dependence assumption in the model.

ERGM vs SAOM Per Block 19

What dependence assumptions are “better”?

Not only is the level of dependence different, the form dependence can take also differs.

In the SAOM, the case that the tie k → l is dependent on j → h, but not vice versa, is possible.

We call this asymmetric dependence which has serious modelling consequences.

ERGM vs SAOM Per Block 21

Consider the following transitions around 030T:

ERGM vs SAOM Per Block 22

In the SAOM, the change statistic is counted from the perspective of the focal actor, in the ERGM there is no focal actor.

The tie i → j depends on the tie h → j, but not vice versa. Asymmetric Dependence allows to test specific hypotheses that relate to

actor motivations!

ERGM vs SAOM Per Block 23

We calculated the stat. dist. of a series of networks with 5 nodes, a density and a transitivity parameter and fixed target statistics.

We compared the stationary distribution w.r.t. the expected triad census.

ERGM vs SAOM Per Block 24

The expected number of Triads 021D and 021U

ERGM vs SAOM Per Block 25

The expected number of Triads 120D and 120U

ERGM vs SAOM Per Block 26

1. Interpretation of the parameters differs.2. GoF-based model selection can lead to different model specifications. The control effects included can have a strong impact on substantive conclusions drawn. Unbiased model comparison becomes difficult.

3. Indegree and Outdegree centralisation parameter estimates can differ.

ERGM vs SAOM Per Block 27

All the above was studied in isolation for illustration purposes. Does it matter in empirical analysis or all the outlined differences

marginal in consequences? We tested this by doing a forward model selection on the same dataset to

compare the final models and conclusions we can draw (friendship network, n=129).

ERGM vs SAOM Per Block 28

ERGM vs SAOM Per Block 29

Do actors weigh ties against one another or is each tie evaluated on its own (→ actor- or tie-oriented)? E.g. costly vs. costless ties.

Are theories explicit about mechanisms of tie dependence (→ asymmetric dependence)? Do we have strong theories about tie formation and presistence?

Rarely: Which model fits better.

ERGM vs SAOM Per Block 30

Partly in response to the work by Cranmer, Leifeld & Desmarais

(Block, P., Koskinen, J., Hollway, J., Steglich, C., Stadtfeld, C. (2017). Change we can believe in. Somewhere, soon.)

Previous part was based on academic interest. To date, ERGMs are dominant in the analysis of cross-sectional data and cannot (without

adjustments) be applied to longitudinal data. SAOMs are used for longitudinal data and uncommon in the analysis of cross-sectional

data. This part is guided by practical considerations for the analysis of

longitudinal data. Especially some people claim that a specific ERGM extension is the way forward in

longitudinal network models (Cranmer, Leifeld & Desmarais).

(Statistical) models can be compared on the basis of prediction or explanation.

Prediction: Intuitively, a good model should adequately extrapolate to the future. But: for networks

what do we predict (ties or dependence)? Explanation: A model should have realistic and testable assumptions. A model should have a consistent interpretation that allows inference on social

mechanisms. A model should be transparent and understood in its mechanics.

In its most basic form, a TERGM is an ERGM, in which a previous realisation (time-point) of the network is a predictors (dyadic covariate).

Information of a previous time-point can be included in more elaborate ways as dyadic covariates.

In the extreme this leads to a TERGM as proposed by Hanneke et al., in which only information from the past is included, making it essentially a logistic regression (see Lerner et al.).

For the SAOM time is explicitly modelled and dependence unfolds over time.

In the TERGM, dependence is modelled between ties in the last observation, but decoupled from time!

The SAOM can be interpreted on the level of the mini-step, showing what new ties are more/less likely to exist in the next step.

TERGM interpretation parallel to the ERGM: Network level: The probability to observe more or less of a specific pattern than expected

“by chance”, conditioning on the past.▪ But the past might contain many of the same dependencies, so any interpretation is somewhat net off these

dependencies in the past

Tie level: No interpretation similar to the ERGM possible, as equilibrium assumption is unreasonable given it includes interpretation on the past.▪ Not being able to interpret on the tie-level is problematic, as this provides the link to social mechanisms.

Not a model for change!

The network level interpretation of the TERGM is net off the dependencies in the past.

If little change between 2 networks has happened, many dependencies of the current network will be explained by the past network.

This means that, even if we have time-homogeneous process parameter size will depend on time in the TERGM (but not the SAOM).

Create a dataset based on the Glasgow data.

10 periods are simulated into the future with realistic parameter values.

One period 1-10 waves apart is analysed, not using information of the waves in between.

It is possible to test the predictive power of the models with simple out-of-sample tests.

We focus here on tie-level prediction, as impartial comparison of prediction of structures is very difficult, given that different effects model different dependence structures.

First expectation: Both models are bad at predicting. This is because, like statistical models in the social sciences in general, we are not very

good at predicting rare events; creation of ties are rare events, given the number of options.

Modelling dependence structures will equally improve correct predictions, if other ties are predicted correctly, but also predict wrong ties, if the other tie is predicted incorrectly.

See the parallel to Multilevel modelling: E.g. outcome 𝑌𝑌 of individual 𝑖𝑖in group 𝑗𝑗 (𝑌𝑌𝑖𝑖𝑖𝑖) will not be predicted better by inclusion of random group-level intercept, unless we have information about the outcome of other members of group 𝑗𝑗, i.e. we need information about the future to predict the future.

x(t0)

x(t1)

xa(t2)

Dlogit < DTERGM Dlogit ≈ DTERGM Dlogit > DTERGM

xc(t2)xb(t2)

Analysing period 1 from 2 datasets (one ASSIST school and the undirected fisheries treaties).

Simulating wave 3 from the obtained parameters 100 times.

Comparing the precision and recall from the simulations (and a logistic regression) to a trivial persistence model.

How come other articles came to such different conclusions?

In the article by Leifeld and Cranmer on arxiv, two analysis claim that the TERGM has a better out-of-sample predictive performance. These claims have restricted validity: No comparison to a null-model. The first analysis uses simulated data with extremely unrealistic parameters (e.g. negative

tie stability) to show how model performance differs. The second analysis relies on real data (good), but estimates TERGMs using pseudo-

likelihood estimation, known to result in unreliable parameters. For simulation of out-of-sample parameters, information about the future was taken into account!

The TERGM has a murky decoupling of time and dependence, which makes interpretation of parameters difficult on the network level, or impossible on the process-level.

The TERGM is not a model of change, but a model of structure; it does not allow conclusions about mechanisms that drive network evolution.

Both models are really bad at predicting, thus it is not a good criterion for model discrimination.