Chapter 1

MULTI-RELATIONAL CHARACTERIZATION OF

DYNAMIC SOCIAL NETWORK COMMUNITIES

Yu-Ru Lin, Hari Sundaram and Aisling Kelliher School of Arts, Media and Engineering Arizona State University, Tempe AZ, USA

1. Introduction

The emergence of the mediated social web – a distributed network of participants

creating rich media content and engaging in interactive conversations through In-

ternet-based communication technologies – has contributed to the evolution of

powerful social, economic and cultural change. Online social network sites and

blogs, such as Facebook, Twitter, Flickr and LiveJournal, thrive due to their fun-

damental sense of ―community‖. The growth of online communities offers both

opportunities and challenges for researchers and practitioners. Participation in on-

line communities has been observed to influence people's behavior in diverse

ways ranging from financial decision-making to political choices, suggesting the

rich potential for diverse applications. However, although studies on the social

web have been extensive, discovering communities from online social media re-

mains challenging, due to the interdisciplinary nature of this subject. In this ar-

ticle, we present our recent work on characterization of communities in online so-

cial media using computational approaches grounded on the observations from

social science.

Motivation – human community as meaning-making eco-system. A key idea

from situated cognition is that knowledge is fundamentally situated within the ac-

tivity from which it is developed [6]. Brown, Collins and Duguid [6] offers an

analysis of how we make meaning with the lexical and grammatical resources of

language – people can interpret indexical expressions (containing such words as I,

you , here, now, that, etc.) only when they can find what the indexed words might

refer to. The concept of indexicality suggests [6] that ―knowledge, which comes

coded by and connected to the activity and environment in which it is developed,

is spread across its component parts, some of which are in the mind and some in

the world much as the final picture on a jigsaw is spread across its component

pieces.‖ In other words, knowledge does not solely reside in the mind of an indi-

vidual, but is distributed and shared among co-participants in authentic situations.

The meaning-making eco-social systems, denoted by Lemke [25], shape and

create meaning not by individual components (people, media, objects, etc.), but by

their co-participation in an activity situation.

2

Being influenced by this theory, we believe that semantics is an emergent artifact

of human activity that evolves over time. Human activity is mostly social, and the

social networks of human are conceivable loci for the construction of meaning.

Hence, it is crucial to identify real human networks as communities of people inte-

racting with each other through meaningful social activities, and producing stable

associations between concepts and artifacts in a coherent manner.

Motivating applications. The discovery of human communities is not only philo-

sophically interesting, but also has practical implications. As new concepts emerge

and evolve around real human networks, community discovery can result in new

knowledge and provoke advancements in information search and decision-

making. Example applications include:

Context-sensitive information search and recommendation: The discovered

community around an information seeker can provide context (including ob-

jects, activities, time) that help identify most relevant information. For exam-

ple, when a user is looking at a particular photo, the community structure may

be used to identify peers or objects likely co-occurring with the photo.

Content organization, tracking and monitoring: The rapid growth of content

on social media sites creates several challenges. First, the content in a photo

stream (either for a user or a community) is typically organized using a tem-

poral order, making the exploration and browsing of content cumbersome.

Second, sites including Flickr provide frequency based aggregate statistics in-

cluding popular tags, top contributors. These aggregates do not reveal the rich

temporal dynamics of community sharing and interaction. Community struc-

ture may be used to reflect the social sharing practice and facilitate the organ-

ization, tracking and monitoring of user-generated social media content.

Behavioral prediction: Studies have shown that individual behaviors usually

result from mechanisms depending on their social networks, e.g. social ebed-

dedness [17] and influence [13]. Community structure that accounts for inhe-

rent dependencies between individuals embedded in a social network can help

understand and predict the behavioral dynamics of individuals.

Data characteristics and challenges. Large volumes of social media data are be-

ing generated from various social media platforms including blogs, FaceBook,

Twitter, Digg, Flickr. The key characteristics of online social media data include:

Voluminous: Recent technological advances allow hundreds of millions of

users to create social and personal content instantly. The amount of data and

the rate of data production can be enormous.

Dynamic: Users’ online actions are constantly archived with timestamps.

These online activity records enable a fine-grained observation on the dynam-

ics of human interactions and interests.

Context-rich: Most social media platforms allow a wide array of actions for

managing and sharing media objects – e.g. uploading photos, submitting and

3

commenting on news stories, bookmarking and tagging, posting documents,

creating web-links, as well as actions with respect to other users (e.g. sharing

media and links with a friend), or on media objects produced by other users.

The complex social interactions among users result in multi-relational net-

work data.

The large-scale, fine-grained, rich online interaction records pose new challenges

on community discovery:

Lack of well-defined attributes: Traditional studies of human communities are

often based on fixed demographic characteristics [3,38]. Within online social

networks, individuals may shift fluidly and flexibly among communities de-

pending on their online social actions (e.g. who they recently interact with,

what they recently share with each other, etc.) [4].

Limitation in network centric analysis: Classical social network analyses

mostly focus on static interpersonal relationships (e.g. self-reported friend-

ships), with a primary interests on the graph topological properties These stu-

dies range from well-established social network analysis [38] to recent suc-

cessful graph mining algorithms such as HITS [22], PageRank [5] and

spectral analysis [36]. These methods are limited in discovering important as-

pects of online communities since the interpersonal relationships may evolve

with their online interactions and may involve rich media contexts (e.g. tags,

photos, time, and space).

Scalability requirement: The Internet scale social network data requires a

scalable analysis framework to support community discovery based on infor-

mation latent in the multi-relational social network data [23].

Problem overview. We are interested in characterizing human communities that

emerge from online interpersonal social activities. Given the challenges discussed

above, we have focused on the following research problems:

How to identify meaningful interpersonal relationship from online social ac-

tions? In social network literature, community discovery usually refers to de-

tecting cohesive subgroups of individuals within networked data collected

based on well-defined social relationship such as self-reported friendship

[38]. Characterization of communities in online social networks deviates from

traditional social network analysis because the social meaning of the networks

is not definite.

How to identify sustained evolving communities from dynamic networks? On-

line communities are temporal phenomena emerge from sustained human ac-

tions and interests, and the actions and interests may evolve over time. Tradi-

tional analysis of social networks focuses on the properties of a static graph

(aggregation or snapshot of network), which overlooks the temporal characte-

ristics of communities. Discovering communities based sustained activities

and at the same time characterizing their temporal evolution is challenging.

4

How to identify communities with rich interaction context? Online social me-

dia websites (e.g. Flickr, Facebook) enable rich interaction between media

and users, as well as complex social interactions among users – two users

may share similar tags or read the same feeds. Discovery communities from

such complex social interactions pose technical challenges that involve deal-

ing with networked data consisting of multiple co-evolving dimensions, e.g.

users, tags, feeds, comments, etc. Existing high dimensional data mining

techniques are usually computational intensive and not suitable for dealing

with large scale social networked data.

Our approach. Our work concerns approach to the three problems (see Figure 1

for illustrating summarization):

Figure 1: Our work concerns multiple aspects on community analysis: (a) Mu-

tual awareness – a bi-directional relationship indicating how well a pair of

bloggers is aware of each other, as fundamental property of a community. (b)

Mutual awareness expansion – a random walk based distance measure which

estimates the probability that two bloggers are aware of each other on the net-

work. (c) FacetNet – for analyzing communities and their evolutions in a uni-

fied process. (d) MetaFac – the first graph-based multi-tensor factorization

framework for analyzing the dynamics of heterogeneous social networks.

mutual awareness

time

(a) (b)

time

(c)

usersdocuments

tags

projects

(d)

5

Mutual awareness: We propose mutual awareness (ref. Figure 1(a)), a bi-

directional relationship indicating how well a pair of bloggers is aware of

each other, as fundamental property of a community. We provide computa-

tional definition to quantify mutual awareness and use it as a feature for

community discovery. Then, we capture the amount of mutual awareness ex-

panding on the entire network using a random walk based distance measure,

commute time, which estimates the probability that two bloggers are aware of

each other on the network (ref. Figure 1(b)). We propose an efficient iterative

mutual awareness expansion algorithm to extract communities, which parti-

tions the network by maximizing the commute time distance between two sets

of bloggers. The experimental results for community extraction in terms of

standard evaluation metrics are promising.

FacetNet: We introduce the FacetNet framework to analyze communities and

their evolutions in a unified process. In our framework (ref. Figure 1(c)), the

community structure at a given timestep is determined both by the observed

networked data and by the prior distribution given by historic community

structures. Algorithmically, we propose the first probabilistic generative mod-

el for analyzing communities and their evolution. The experimental results

suggest that our technique is scalable and is able to extract meaningful com-

munities based on social media context. (e.g., dramatic change in a short time

is unlikely).

MetaFac: We propose MetaFac, the first graph-based tensor factorization

framework for analyzing the dynamics of heterogeneous social networks (ref.

Figure 1(d)). In this framework, we introduce metagraph, a novel relational

hypergraph representation for modeling multi-relational and multi-

dimensional social data. Then we propose an efficient multi-relational factori-

zation algorithm for latent community extraction on a given metagraph. Ex-

tensive experiments on large-scale real-world social media data and from the

enterprise data suggest that our technique is able to extract meaningful com-

munities that are adaptive to social media context.

Organization. The rest of this article is organized as follows. Section 2 presents

community discovery based on mutual awareness. Section 3 presents method for

extracting sustained evolving communities. Section 4 presents method for extract-

ing communities with rich interaction context. Section 5 concludes with future di-

rections.

6

2. Actions, Networking and Community Formation

In this section we study the online blog network and propose computational ap-

proach for discovering communities in the blogosphere. Blogs (or weblogs) have

become popular self-publishing social media on the Web. Although they are a

type of websites, the analysis of blog communities is different from traditional

Web analysis literature. The differences lie in the different semantics and struc-

tures of the hyperlinks in the context of blogosphere. A blog is typically used as a

tool for communication. Driven by an event (such as a real-world news), bloggers

publish entries that refer to each other. Thus links among blog entries are consi-

dered to be interactions between two bloggers and have significant temporal local-

ity. On the Web, it is common that a new page refers to a relevant page that exists

for a long time, such as an authoritative page [5,22]. A ―community of web pages‖

due to the links of relevance is thus different from a ―community of bloggers‖

formed due to the links of interactions. The analysis of blog network also deviates

from traditional social network analysis [38] because the social meaning of the

blog network is not as well-defined as in traditional social networks (e.g. links

represent friendship). Hence, community discovery in the blogosphere requires a

new analytical framework grounded in the unique characteristics of the blog me-

dia.

2.1 Mutual Awareness and Community Discovery

The notion of virtual community, or online community, has been discussed exten-

sively in prior research. Rheingold [34] defined virtual communities to be ―social

aggregations that emerge from the Net when enough people carry on those public

discussions long enough, with sufficient human feeling, to form webs of personal

relationship in cyberspace.‖ Jones [20] considered four characteristics as the ne-

cessary conditions for the formation of a virtual community: interactivity, com-

municators, virtual common-public-place where the computer-mediated commu-

nication takes place, and sustained membership. These conditions echo

Garfinkel’s observation on the necessity of mutually observable actions [14]. The

same idea that interactivity forms a social reality has also been discussed by Dou-

rish [12]. According to Dourish, interaction involves presence (some way of mak-

ing the actors present in the locale) and awareness (some way of being aware of

the other’s presence). In what Dourish called an action community, members share

the common sense understandings through the reciprocal actions. The common

aspect of the prior work is the emphasis on the significance of action and interac-

tion in online communities. However, little work has studied the counter perspec-

tive – how to discover communities due to actions.

We introduce mutual awareness that is fundamental to blog community formation.

By mutual awareness of action we mean that individual blogger actions must lead

to bloggers becoming aware of each other’s presence. The idea is in the light of

Locale theory [12] that discusses how social organization of activity is supported

7

in different spaces. While the domains of activity must provide means for the

community members to act, the space must also accord members’ presence and

facilitate mutual awareness.

Note that mutual awareness may be related to, but is different from, link reciproci-

ty, which refers to the tendency of vertex pairs to form mutual connections be-

tween each other [15]. It is also related to tie strength discussed in the social net-

work literature [16] – ―the strength of a tie is a (probably linear) combination of

the amount of time, the emotional intensity, the intimacy (mutual confiding), and

the reciprocal services which characterize the tie.‖ However, quantifying tie

strength based on these elements remains challenging. In fact, mutual awareness

can be considered as a mechanism for tie strength situated in particular communi-

cation media.

2.2 Extracting Communities based on Mutual Awareness Structure

We propose a computational approach for community discovery in the blogos-

phere. Grounded on the actions of individual bloggers, we propose to discover

community based on the idea of mutual awareness. We propose a computational

definition for determining mutual awareness in the blog network. Then, using mu-

tual awareness as features, we propose methods for extracting blog communities.

2.2.1 Computable Definition for Mutual Awareness

Let us examine the actions of individual bloggers – how bloggers read and com-

municate ideas with other bloggers. The bloggers can act in the blogosphere, in

several ways: surf / read, create entries (containing entry-to-entry links, entry to

blog / web, or no link), comment or change blogroll. Some actions (e.g. surf/read)

may be hidden, while others may be observable.

How a specific blogger action leads to mutual awareness may depend on (a) if the

action is mutually observed, and (b) the importance of the action for the blogger

who performs the action. Note that some blogger actions are not observable by

other bloggers. For example, let us consider two hypothetical bloggers, Mary and

John. Let us assume that Mary creates an entry with a hyperlink that points to

John’s blog. In this case John would be unaware of Mary’s entry. On the other

hand, if Mary leaves a comment on John’s entry, then John is immediately aware

of her presence. If Mary mostly leaves comments on other bloggers, and the im-

portance of a comment for Mary is low – while many bloggers are aware of Mary,

she may not feel that she in engaged in dialogue with them. The assessment of

mutual awareness is the first step toward the discovery of blog communities.

We thus characterize mutual awareness as follows (see Figure 2 for an illustra-

tion): mutual awareness between two bloggers is affected by the type of action, the

number of actions for each type, and when the action occurred. It depends on sus-

8

tained actions – it increases if there are follow-up actions that lead to mutual

awareness and decreases if actions are not sustained over time.

We represent the set of bloggers in the blogspace as a weighted directed graph G =

(V, E), where each node v∈V represents a blogger, each edge between any pair of

nodes u and v represents an action performed by u with respect to v. The weight

on each edge f(u,v) indicates the mutual awareness between two bloggers u and v.

The corresponding matrix M with each entry Muv= f(u,v) is called mutual aware-

ness matrix and is defined as follows.

Definition 1 (Mutual awareness matrix):

𝐌 = 𝛼𝑘min(𝐗 𝑘 , 𝐗 𝑘𝑇

)𝑘 (1)

where the index k is used to denote a specific action (e.g. leaving a comment, or

creating an entry-to-entry link) and αk represent the importance of the actions and

is usually empirically determined. 𝐗 𝑘 is aggregated action matrix for the action

type k. Since mutual awareness due to earlier actions will gradually diminish, the

temporal effect can be modeled as a decaying exponential function:

𝐗 𝑘 = 𝐗𝑘 ;𝑡𝑒−𝜆𝑘 (𝑇−𝑡)𝑇

𝑡=𝑡0 (2)

where λk is the decaying factor for the action type k. Different types of actions may

decay at different rate. Xk;t denotes all-pair type-k actions occurring at time t, and

𝐗 𝑘 aggregates these actions from time t0 to time T.

Mutual awareness is a bi-directional relationship indicating how well a pair of

bloggers is aware of each other. This semantics results in a symmetric mutual

awareness matrix. The reciprocity condition min(⋅,⋅) in Eq. (1) makes the possi-

bility of both bloggers being aware of each other to be high.

Figure 2: Mutual awareness between two bloggers is affected by the type of

actions, the number of such actions, and when such actions occur. The arrow

direction indicates the source and the destination blogger on whom the action

is performed. A mutual awareness curve is plotted to show the action impacts.

9

Empirical evaluation. We have studied the effectiveness of mutual awareness

(MA) matrix on real-world blog datasets [26]. We use the subgroup extraction

procedure described in [26] to extract subgroups, and evaluate the quality of these

subgroups in terms of different metrics. Compared with the baseline adjacency

matrices (with entries indicating the total number of entry-to-entry links), the sub-

groups extracted using mutual awareness matrices are usually of higher quality.

The quality evaluation is based on several metrics, including conductance, inter-

esting coefficient, etc. (see the definitions of the metrics in [26]). Figure 3(a)

shows the performance comparison of results from the WWE 2006 Workshop

Blog Dataset [26]. An example subgroup is shown in Figure 3(b) – the group is

observed to have cohesive topic about mystery novels based on the top keywords

from their blog contents.

2.2.2 Mutual Awareness Expansion

We extend the idea of mutual awareness to community extraction. Mutual aware-

ness quantifies the relationship between two bloggers. To extract a set of bloggers

having high mutual awareness, we hypothesize how mutual awareness expands in

a blog network:

Transitivity: One could become aware of a member without direct interaction

since he or she can observe his or her direct peers interacting with other

people. Thus awareness is transitive. (The transitivity property in social net-

work has been first examined in Travers and Milgram’s well-known small

world experiment [37], which motivates our proposed algorithm [27].)

Mystery Community

(―harry potter‖ ―buffy the

vampire slayer‖)

Number of nodes: 239

Conductance: 0.0004 Interest coefficient: 0.99

Coverage: 0.02

Figure 3: (a) Performance comparison between the Baseline communities and

the MA communities in terms of metrics conductance, interest coefficient, and

coverage. (b) An example subgroup cohesive topic about mystery novels, ex-

tracted using MA matrix.

(a) Performance comparison (b) Case study

10

Reciprocity: Such transitive awareness must be reciprocal. If expansion of

awareness is only one directional, one might not feel belonging to the com-

munity

Frequency: The amount of observed interaction must be sufficient for mem-

bers to feel connected to each other.

We characterize such mutual awareness expansion process by a random walk

model. The probability that two bloggers are aware of each other on the entire

network is quantified using the random walk expected length between two nodes

corresponding to the bloggers. We refer to this expected length as symmetric so-

cial distance. It is computed as follows: Given a direct graph G = (V,E) and the

mutual awareness matrix W associated with G, the random walk on G is defined

to be the Markov chain with state space V and the transition matrix P = D-1

W,

where D is a diagonal matrix with element dii = ∑j wij. A random walker at a node

i on G will follow the transition probability pij = Pij to visit the next node j. Note

that by construction wii = ∑j wij (i.e. pii = 1/2) for i≠j .

Let 𝜏𝑢→𝑣 denote the one-way social distance from node u to v, i.e. the expected

number of steps to reach node v from node u. We define 𝜏𝑢→𝑣 to have the transi-

tive awareness property:

Definition 2 (Transitive awareness property):

𝜏𝑢→𝑣 = 1 + 𝑝𝑢𝑥 𝜏𝑥→𝑣 𝑖𝑓 𝑢 ≠ 𝑣 𝑢 ,𝑥 ∈𝐸

0 𝑖𝑓 𝑢 = 𝑣 (3)

where pux is the transition probability from u to x. The equation can be illustrated

in Figure 4: To reach v from u, the random walker takes one step to get to the next

node x with transition probability pux, and then calculates the rest expected dis-

tance to v. The symmetric social distance is defined by 𝜏𝑢↔𝑣 = 𝜏𝑢→𝑣 + 𝜏𝑣→𝑢 .

Solution. Based on property in Eq. (3), the solution for 𝜏𝑢↔𝑣 ,denoted by 𝜏 𝑢↔𝑣 ,

can be derived by using Green’s function [10]:

Figure 4: Transitive awareness property – the social distance from node u to v

is defined by the expected number of steps before node v is visited, starting

from node.

u v

x

y

pux

puy

1 txv

tyv1

11

𝜏 𝑢↔𝑣 = 𝑣𝑜𝑙 1

𝜆𝑖(𝜙𝑖(𝑢) − 𝜙𝑖(𝑣))2𝑘

𝑖=2 (4)

where vol=∑u,v∈V wuv, ϕi’s and λi’s (0=λ1<λ2≤ … ≤λn) are the eigenvectors and cor-

responding eigenvalues of the Laplacian matrix L=D-W. The solution is com-

puted by truncating after the k-th smallest eigen-pair for k<n. Intuitively, 𝜏𝑢↔𝑣 is

the random walk expected path length from u to v and back to u, which takes into

account the indirect interactions between u and v derived by their interactions with

other nodes over the entire network.

Community extraction. We use the symmetric social distance as a criterion to

extract a community. Given a set of bloggers V, a subset S from V can be seen as a

community if the symmetric social distance among members of S is short com-

pared to those with non-members. Therefore, we iteratively split the set V into two

sets S and V\S by maximizing the symmetric social distance as follows:

𝑆 = argmax𝑆⊂𝑉 𝜔(𝑆, 𝑉) 𝜏𝑢↔𝑣𝑢∈𝑆,𝑣∈𝑉\𝑆 (5)

Where ω(S,V) is a weighted function used to obtained desirable properties (e.g.

balance partition) of the set of communities. Details of the algorithm can be found

in [27].

Empirical evaluation. We compare our community extraction method with well-

known baseline clustering algorithms, including the kernel k-means [11], norma-

lized cut [36] and iterative conductance cutting [21]. The results indicate that our

method outperforms all baseline methods in terms of low conductance, high cov-

erage, and low entropy [27].

2.3 Application: Query-sensitive Community Extraction

We apply the community extraction algorithm to the extraction of query-sensitive

communities, i.e. blog communities that have a strong content related theme with

respect to a given query. We summarize the idea as follows (see [27] for more de-

tails):

Step 1: Given a query topic Q, extract query-sensitive graph GQ to represent

interactions relevant to the topic.

Step 2: Given GQ, extract communities as described in Section 2.2.

In the first step, we construct a weighted action matrix with respect to a query Q.

Q contains query keywords that represent the given topics, e.g. ―Katrina‖, ―london

bomb‖, etc. The weight of an interaction is determined by the relevance score of

the blog content involved in the interaction. Because the query keywords in Q are

relatively short, in order to further incorporate relevant blogs in GQ, we compute

the ―query relevancy‖ by employing a web-based similarity function [27,35].

Figure 5 shows an example from our experiment. In this case, we extract com-

munities with respect to the query keyword is ―katrina‖, which is about a natural

disaster caused by the hurricane Katrina in August 2005. In order to understand

12

the ―meaningfulness‖ of the extracted communities, we employ a heuristic method

[27] to examine the relationship between the topic and the communities over time:

We connect those communities extracted from different time snapshots based on

an interaction similarity measure1. In Figure 5, each node represents a detected

community where the communities detected during the same week are aligned ho-

rizontally and the communities at different time snapshots are connected by ar-

rows. The grayscale of an arrow is proportional to the interaction similarity be-

tween the two communities. The node size reflects the number of community

members and the reddish shade of node is proportional to the query relevancy for

the keyword ―katrina‖. More saturated node represents more relevant community.

The interactions among the extracted communities are quite interesting. Two dom-

inated communities, one with a focus on politics (shown on the left) and the other

with a focus on technology (shown on the right), emerged and evolved due to the

Katrina event. When the event Katrina occurred at week 7, we found community

1 A more systematical solution will be presented in next section.

Figure 5: Communities with respect to the query ―katrina.‖ The node size re-

flects the number of community members and the reddish shade of node is

proportional to the query relevancy for the keyword ―katrina.‖

week 5

week 13

week 12

week 11

week 10

week 9

week 8

week 7

week 6

week 14

week 15

8/23 Hurricane Katrina forms

Political community about “katrina” emerges

Technical community about “katrina” emerges

Political community splits

9/17 Hurricane Rita forms

8/29 Levee failures in New Orleans

right wing

left wing

right wingleft

wing

13

merged from left-wing and right-wing members, due to debates about the govern-

ment response as well as cooperation of fund-raising. Later at week 11, the com-

munity split into stable communities that correspond to their political preferences.

The results suggest that for queries such as ―Katrina‖, community extraction help

identifies people with different viewpoints based on their sustained interactions.

Summary. A key idea in this work was that observable actions lead to the emer-

gence of human communities, and awareness expansion was critical to community

formation. We provide computational definition to quantify mutual awareness and

use it as a feature to extract subgroups in the blogosphere. The effectiveness of

mutual awareness features is verified on real-world blog datasets.

We showed how to detect blog communities based on mutual awareness expan-

sion, given a specific query. We proposed a symmetric social distance measure

that captures the expansion process and use it to detect communities. The commu-

nity evolution with respect to a query reveals interesting community dynamics.

There are some open issues in this work. (1) The communities are independently

extracted at consecutive timesteps and then the evolutions are characterizes to ex-

plain the difference between these communities over time. Such a two-stage ap-

proach may result in community structures with high temporal variation, and un-

desirable evolutionary characteristics may have to be introduced in order to

explain such high variation in the community structures. A more appropriate ap-

proach is to analyze communities and their evolutions in a unified framework. (2)

In order to extract communities with strong content related themes, we construct

networks with query-dependent edge weights with respect to given concepts.

However, the theme of a community may not be known in advance, and it may

emerge and evolve over time, depending on the content and context associated

with the interactions. This will require a new approach for extracting communities

from rich interaction contexts. We shall discuss these directions in next two sec-

tions.

14

3. Analyzing Communities and Evolutions in Dynamic Network

In this section, we present the FacetNet framework that analyzes communities and

their evolutions in a unified process. Traditional analysis of social networks treats

the network as a static graph, where the static graph is either derived from aggre-

gation of data over all time or taken as a snapshot of data at a particular time.

These studies range from well-established social network analysis [38] to recent

successful applications such as HITS [22] and PageRank [5]. However, this re-

search omits one important feature of communities in networked data – the tem-

poral evolution of communities.

3.1 Sustained Membership, Evolution and Community Discovery

If evolution is a nature characteristic of human communities, how are they differ-

ent from a chance meeting of casual individuals? Jones [Jones 1997] argued that a

virtual community is not a chance meeting of casual individuals but should in-

volve long term, meaningful conversations among humans, and this condition

suggests that there should be a minimal level of sustained membership. Lemke de-

scribed community ecology as follows: ―they have a relevant history, a trajectory

of development in which each stage sets up conditions without which the next

stage could not occur,‖ ―the course of their development depends in part on infor-

mation laid down (or actively available) in their environments from prior (or con-

temporary) systems of their own kind.‖

Recently, there has been a growing body of analytical work on communities and

their temporal evolution in dynamic networks (e.g. [2,27,32]). However, a com-

mon weakness in these studies, is that communities and their evolutions are stu-

died separately – usually community structures are independently extracted at

consecutive timesteps and then in retrospect, evolutionary characteristics are in-

troduced to explain the difference between these community structures over time.

Such a two-stage approach has two issues: (a) At each timestep, communities are

extracted without considering sustained membership (temporal smoothness of

clustering). (b) It may result in community structures with high temporal variation,

and undesirable evolutionary characteristics may have to be introduced in order to

explain such high variation in the community structures.

Sustained membership is the key to discovery time-evolving communities. We in-

troduce the FacetNet framework to extract sustained and evolving communities

from dynamic social networks.

3.2 Extracting Sustained Evolving Communities

We present the formulation of our model, and describe how to extract communi-

ties and their evolutions from the solution of our model.

15

3.2.1 Problem Formulation

We assume that edges in the networked data are associated with discrete time-

steps. We use a snapshot graph Gt=(Vt,Et) to model the interactions at time t, where in Gt, each node vi∈Vt represents an individual, each edge eij∈Et represents

the presence of interactions between vi and vj, and wt;ij = (Wt)ij denotes the edge

weight of eij. Note the edge weight can represent mutual awareness, or more gen-

erally, the frequency of interactions between nodes i and j observed at time t. As-

suming Gt has n nodes, Wt ∈ ℜ+𝑛×𝑛 (nonnegative matrix of size n×n) is the corres-

ponding weight matrix for Gt. Over time, the interaction history is captured by a

sequence of snapshot graphs G1, … , Gt, … indexed by time.

The basic principles (as illustrated in Figure 6) behind our models are the commu-

nity structure at time t is determined by (1) the data observed at time t (i.e. W,

which is short for Wt), and (2) the community structure at time t-1. We propose to

use the community structure at time t-1 (already extracted) to regularize the com-

munity structure at current time t (to be extracted). To incorporate such a regula-

tion, we introduce a cost function to measure the quality of community structure at

time t, where the cost consists of two parts—a snapshot cost and a temporal cost:

𝑐𝑜𝑠𝑡 = 𝛼 ⋅ 𝒞𝒮 + (1 − 𝛼) ⋅ 𝒞𝒯 (6)

This cost function is first proposed by Chakrabarti et al. [8,9] in the context of

evolutionary clustering. In this cost function, the snapshot cost 𝒞𝒮 measures how

well a community structure fits W, the observed interactions at time t. The tem-

poral cost 𝒞𝒯 measures how consistent the community structure is with respect to

historic community structure (at time t-1). The parameter a is set by the user to

control the level of emphasis on each part of the total cost.

A community structure at time t should fit W well, where W is the observed inte-

raction matrix at time t. This requirement is reflected in the snapshot cost 𝒞𝒮 in

the cost function Eq. (6). We adopt a stochastic block model first proposed in [39].

Figure 6: The community structure at time t (denoted as Ut) is determined by

(1) the data observed at time t (denoted as Wt), and (2) the community struc-

ture at time t-1 (denoted as Ut-1).

Wt

Wt-1

snapshot cost

temp

oral co

st

Ut-1

Ut

16

Assume that there exist m communities at time t, and that the interaction wij is a

combined effect due to all the m communities. That is, we approximate wij using a

mixture model wij= 𝑝𝑘 ⋅ 𝑝𝑘→𝑖 ⋅ 𝑝𝑘→𝑗𝑚𝑘=1 , where pk is the prior probability that the

interaction wij is due to the k-th community, 𝑝𝑘→𝑖 and 𝑝𝑘→𝑗 are the probabilities

that an interaction in community k involves node vi and vj, respectively. Written in

a matrix form, we have 𝑊 ≈ 𝑋Λ𝑋𝑇 , where X ∈ ℜ+𝑛×𝑚 is a non-negative matrix

with xik = 𝑝𝑘→𝑖 and ∑i xik = 1. In addition, Λ is an m×m non-negative diagonal ma-

trix with λk = pk, where λk is short for λkk. Matrices X and Λ (or equivalently, their

product XΛ) fully characterize the community structure in the mixture model.

Based on this model, we define the snapshot cost 𝒞𝒮 as the error introduced by

such an approximation, i.e.,

𝒞𝒮 = 𝐷(𝑊||𝑋Λ𝑋𝑇) (7)

where 𝐷(𝐴||𝐵) = (𝑎𝑖𝑗 log𝑎𝑖𝑗

𝑏𝑖𝑗− 𝑎𝑖𝑗 + 𝑏𝑖𝑗 )𝑖 ,𝑗 is the KL-divergence between A and

B. The snapshot cost is high when the approximate community structure XΛXT

fails to fit the observed data W well.

In the cost function Eq. (6), the temporal cost 𝒞𝒯 is used to regularize the com-

munity structure. We propose to achieve this regularization by defining 𝒞𝒯 as the

difference between the community structure at time t and that at time t-1. Recall

that the community structure is captured by XΛ. Therefore, with

Y= Xt-1Λt-1, the temporal cost is defined as:

𝒞𝒯 = 𝐷(𝑌||𝑋Λ) (8)

where D(||) is the KL-divergence as defined before. The temporal cost 𝒞𝒯 is high

when there is a dramatic change of community structure from time t-1 to t.

Putting the snapshot cost 𝒞𝒮 and the temporal cost 𝒞𝒯 together, we have an opti-

mization problem as to find the best community structure at time t, expressed by X

and Λ, that minimizes the following total cost:

𝑐𝑜𝑠𝑡 = 𝛼 ⋅ 𝐷(𝑊||𝑋Λ𝑋𝑇) + (1 − 𝛼) ⋅ 𝐷(𝑌||𝑋Λ) (9)

subject to X ∈ ℜ+𝑛×𝑚 , ∑i xik = 1, and Λ being a m×m non-negative diagonal matrix.

Solving this optimization problem is the core of our FacetNet framework.

Solution. We provide an iterative EM algorithm to find the optimal solutions for

Eq. (9) as follows:

𝑥𝑖𝑘 ← 𝑥𝑖𝑘 ⋅ 2𝛼 𝑤𝑖𝑗 ⋅𝜆𝑘 ⋅𝑥𝑗𝑘

(𝑋Λ𝑋𝑇)𝑖𝑗+ (1 − 𝛼) ⋅ 𝑦𝑖𝑘𝑗

then normalized such that 𝑥𝑖𝑘𝑖 = 1 ∀𝑘

𝜆𝑘 ← 𝜆𝑘 ⋅ 𝛼 𝑤𝑖𝑗 ⋅𝑥𝑖𝑘 ⋅𝑥𝑗𝑘

(𝑋Λ𝑋𝑇)𝑖𝑗+ 1 − 𝛼 ⋅ 𝑦𝑖𝑘𝑖𝑖𝑗

then normalized such that 𝜆𝑘𝑘 = 1 .

(10)

Details about the proposed model and the convergence of the solution can be

found in [29]. Different from the matrix factorization formulation presented here,

in [29], the problem is reformulated in terms of maximum a posteriori (MAP) es-

17

timation and we show a close connection between the optimization framework for

solving the evolutionary clustering problem and our proposed generative probabil-

istic model.

3.2.2 Extracting Communities and Evolutions

Community membership. Assume we have computed the result at time t-1, i.e.,

(Xt-1,Λt-1), and the result at time t, i.e., (Xt,Λt). We define a diagonal matrix Dt,

whose diagonal elements dt;ii=∑ij(XtΛt)ij. Then the i-th row of 𝐷𝑡−1𝑋𝑡Λ𝑡 indicates

the ―soft‖ community memberships of vi at time t.

Community evolution. To derive the community evolutions, we align the two bi-

partite graphs, that at time t-1 and that at time t, side by side by merging the cor-

responding network nodes vi’s (as illustrated in Figure 7). A natural definition of

community evolution (from community ci;t-1 at time t-1 to community cj;t at time t)

is the probability of starting from ci;t-1, walking through the merged bipartite

graphs, and reaching cj;t. A simple derivation shows that P(ci;t-1, cj;t)

=(Λ𝑡−1𝑋𝑡−1𝑇 𝐷𝑡

−1𝑋𝑡Λ𝑡)𝑖𝑗 and P(cj;t|ci;t-1) =(𝑋𝑡−1𝑇 𝐷𝑡

−1𝑋𝑡Λ𝑡)𝑖𝑗 . Each node and each

edge contribute to the evolution from ci;t-1 to cj;t. That is, all individuals and all in-

teractions are related to all the community evolutions, with different levels. This is

more reasonable compared to traditional methods where the analysis of communi-

ty evolution assumes all members having identical importance in a community.

3.3 Application: Time-dependent Ranking in Communities

We apply the FacetNet algorithm on the DBLP co-authorship dataset (see [29] for

more details). In Figure 8(a) we list the extracted top authors in two of the ex-

tracted communities, Data Mining (DM) and Database (DB), where the rank is de-

termined by the value xik, i.e., , 𝑝𝑘→𝑖 . Recall that 𝑝𝑘→𝑖 indicates to what level the k-

Figure 7: The evolution of communities from time t-1 to t, is obtained by

merging the bipartite graphs through corresponding nodes vi’s – as the proba-

bility of starting from ci;t-1 (community nodes at t-1), walking through the

merged bipartite graphs, and reaching cj;t (community nodes at t).

18

th community involves the i-th node, where the value is derived based on both the

current and the historic community structures. So from our framework, we can di-

rectly infer who the important members in each community are. Note that the im-

portance of a node in a community is determined by its contribution to the com-

munity structure.

We can also track the role each individual plays in a community by looking the

value of 𝑝𝑘→𝑖 over time. In Figure 8(b) and (c), We demonstrate one top author

(Philip S. Yu) in the DM community whose community membership remains sta-

ble over all the timesteps and another top author (Laks V. S. Lakshmanan) whose

community membership varies very much over the 5 timesteps. In the figure, each

Figure 8: (a) Top members in the Data Mining (DM) and Database (DB)

communities, sorted by pk→i. (b,c) The evolution of community memberships

for two top authors. In the compasses, an arrow close to the vertical axis indi-

cates a large value of the community membership in the DM community and

to the horizontal axis indicates a large value of the community membership in

the DB community. Hence, the first author consistently played an important

role mainly in the DM community, whereas the second author had a varying

role in both communities.

Data Min-

ing

Philip S. Yu, Jiawei Han, Jian Pei, Wei Wang, Haixun Wang, Beng Chin Ooi, Kian-Lee Tan, Charu C. Aggarwal, Jiong Yang, Hongjun Lu, Mong-Li Lee, Jeffrey

Xu Yu, Tok Wang Ling, Anthony K. H. Tung, Dimitris Papadias,Wynne Hsu, Bing

Liu, Ke Wang, Yufei Tao, Xifeng Yan, Wei Fan, Laks V. S. Lakshmanan, Sourav S. Bhowmick, Guozhu Dong, Jianyong Wang

Database

Divesh Srivastava, Nick Koudas, Divyakant Agrawal, Hans-Peter Kriegel, Surajit

Chaudhuri, Amr El Abbadi, H. V. Jagadish, Rajeev Rastogi, Minos N. Garofalakis,

S. Muthukrishnan, Jennifer Widom, Rakesh Agrawal, Elke A. Rundensteiner, Jeff-rey F. Naughton, Rajeev Motwani, Flip Korn, Michael J. Franklin, Johannes

Gehrke, Hector Garcia-Molina, Vivek R. Narasayya, Raghu Ramakrishnan, Laks V.

S. Lakshmanan, Walid G. Aref, Christos Faloutsos, Sihem Amer-Yahia

(a) Top members in two communities

(b) Philip S. Yu

(c) Laks V. S. Lakshmanan

19

compass indicates a pair (𝑝𝑘1→𝑖 , 𝑝𝑘2→𝑖) where k1 and k2 correspond to the DB and

the DM communities, respectively. So in a compass, a vertical arrow (which has a

large projection on the y-axis) indicates a large value of the community member-

ship in the DM community and a horizontal arrow (which has a large projection

on the x-axis) indicates a large value of the community membership in the DB

community. We can see that the first author consistently played an important role

mainly in the DM community, whereas the second author had a varying role in

both the two communities.

Compared to prior link analysis algorithms, such as HITS and PageRank, our Fa-

cetNet has two advantages: (a) Localized measures: Unlike most of the ranking

algorithms that give global measures, In FacetNet, we obtain individual impor-

tance (in terms of his/her participation in each community) and community mem-

bership simultaneously. The importance measures are localized (per community)

and can be aggregated as global measures on the entire network. (b) Temporal var-

iation: The importance of a node, and the context in which the node is deemed im-

portant, may vary over time. A simple function for discounting the historic data is

not sufficient to capture different types of variation. FacetNet allows understand-

ing how the nodes’ global and local importance change over time.

Summary. The analysis of communities and their evolutions in dynamic temporal

networks is a challenging research problem with broad applications. In this work,

we proposed a framework, FacetNet, that combines the task of community extrac-

tion and the task of evolution extraction in a unified process. To the best of our

knowledge, our framework is the first probabilistic generative model that simulta-

neously analyzes communities and their evolutions. The results obtained from our

model allow us to assign soft community memberships to individual nodes, to

analyze the strength of ties among various communities, to study how the affilia-

tions of an individual to different communities change over time, as well as to re-

veal how communities evolve over time. The experimental results on time-

dependent ranking in the DBLP communities demonstrate utility of our FacetNet

framework. It reveals the community membership evolution for an individual or

the evolutions of the communities, and to discover many interesting insights in

dynamic networks that are not directly obtainable from existing methods.

We are currently extending this framework in two directions. First, our current

model only considered the link information. In many applications, the content in-

formation (e.g., the contents of blog entries and the abstracts of papers) is also

very important. We are investigating how to incorporate content information into

our framework. Second, so far we only use our model to explain the observed da-

ta. To extend our model to predict future behaviors of individuals in a dynamic

social network is also an important research topic. We shall investigate some of

these directions in next section.

20

4. Community Analysis on Multi-Relational Social Data

This work aims at discovering community structure in rich media social networks,

through analysis of the time-varying multi-relational data. As an example scena-

rio, let us consider the use of social media in enterprises, which have increasingly

embraced social media software to promote collaboration. Such social media, in-

cluding wikis, blogs, bookmark sharing, instant messaging, emails, calendar shar-

ing, and so on, foster dynamic collaboration patterns that deviates from the formal

organizational structure (e.g. cooperate departments, geographical places, etc.).

People who are close in the formal organizational structure (e.g. formal collabora-

tion network) might be far apart in the communication network (e.g. the network

of instant messaging). On the other hand, users’ document access patterns might

be related to their corporate roles as well as personal interests. Figure 9(a) shows

an example of such multi-relational social data. The complex and dynamic inter-

play of various social relations and interactions in an enterprise reflects the day-to-

day collaboration practice – how people assemble themselves for a task or activi-

ty, how ideas are shared or propagated, through which communication means,

who are considered to be expert at some tasks, or what pieces of information are

relevant to a particular task, and so on.

Figure 9: (a) Users and related objects in an enterprise. (c) A metagraph that

represents the enterprise social context.

(a) multi-relational social data

(b) metagraph

u1

u2

j1

j2

users

documents

tags

projects

v(4) e(1)

v(1)

v(3)

v(2)

e(3)

e(2)

v(1): {user}

v(2): {tag}

v(3): {document}

v(4): {project}

e(1): {bookmark}

e(2): {join-project}

e(3): {instant-message}G

21

In this section, we introduce the first graph-based tensor factorization algorithm to

analyze the dynamics of heterogeneous social networks, which can flexibly dis-

cover communities along different dimensions (membership, content, etc.), and

can help predict users’ potential interests.

4.1 Embeddedness, Artifacts and Community Discovery

Studies have shown that individual behaviors usually result from mechanisms de-

pending on their social networks. Social embeddedness [17] indicates the choices

of individuals depend on how they are integrated in dense clusters or multiplex re-

lations of social networks. For example, social embeddedness in cohesive struc-

tures can lead people to make similar political contributions [31]. A similar idea

has been grounded in situated cognition. According to Dewey, an individual’s ac-

tions will always be interrelated to all others within certain social medium that

forms the individual membership in a community. Once membership is estab-

lished, the individual begins to share the same supply of knowledge that the group

possesses. Accordingly, this shared experience forms an emotional tendency to

motivate individual behavior in such a way that it creates purposeful activity evok-

ing certain meaningful outcomes [1].

This work has suggested the behavioral dynamics of individuals occur under com-

plex, social conditions that simultaneously give rise to the community structure

(i.e. the ―dense cluster‖ or ―community membership‖). While the conditions may

be ambiguous, situated cognition theorists have suggested that "artifacts holding

historic and negotiated significance within a particular context." Based on the

idea, we presume the community structure latent in multiplex relations based on

shared artifacts affects and is affected by individual choices. That is, community

structure that accounts for inherent dependencies between individuals embedded

in a multi-relational network can help understand and predict the behavioral dy-

namics of individuals.

4.2 Extracting Communities from Rich-context Social Networks

We focus on the multi-relational network observed from the social media. We de-

fine the problem as discovering latent community structure from the context of us-

er actions represented by multi-relational social networks. The problem has three

parts: (1) how to represent multi-relational social data, (2) how to reveal the latent

communities consistently across multiple relations, and (3) how to track the com-

munities over time.

4.2.1 Problem Formulation

We formally represent multi-relational social data through tensor algebra and me-

tagraph representation.

22

Tensor algebra. A tensor is a mathematical representation of a multi-way array.

The order of a tensor is the number of modes (or ways). A first-order tensor is a

vector, a second-order tensor is a matrix, and a higher-order tensor has three or

more modes. We use x to denote a vector, X denote a matrix, and 𝒳 a tensor. Each

entry (i,j,k) in a tensor, for example, could represent the number of times the user i

submitted an entry on topic j with keyword k.

Metagraph representation. We introduce metagraph, a relational hypergraph for

representing multi-relational and multi-dimensional social data. We use a meta-

graph to configure the relational context specific to the system features – this is

the key to make our community analysis adaptable to various social media con-

texts, e.g. an enterprise or a social media website like Digg. We shall an enterprise

example to illustrate three concepts: facet, relation, and relational hypergraph.

As shown in Figure 9(a), assume we observe a set of users in an enterprise. These

users might collaborate under different working projects, e.g. the user u1 and u2

work for the project j1, and user u2 belong to two projects j1 and j2 at the same

time. Collaboration can occur implicitly across different social media such as in-

stant messenger or email, e.g. user u3 frequently IM with u1 and u2. We denote a

set of objects or entities of the same type as a facet, e.g. a user facet is a set of us-

ers, a project facet is a set of projects. We denote the interactions among facets as

a relation; a relation can involve two (i.e. binary relation) or more facets, e.g. the

―join-project‖ relation involves two facets (user, project), and the ―bookmark‖ re-

lation involves three facets (user, document, tag). A facet can be implicit, depend-

ing on whether the facet entities interact with other facets, e.g. the set of bookmark

object might be omitted due to no interaction with other facets. Formally, we de-

note the q-th facet as v(q)

and the set of all facets as V. A set of instantiations of an

M-way relation e on facets v(1)

, v(2)

,…, v(M)

is a subset of the Cartesian product

v(1)…v

(M). We denote a particular relation by e

(r) where r is the relation index.

The observations of an M-way relation e(r)

are represented as an M-way data ten-

sor 𝒳(r).

Now we introduce a multi-relational hypergraph (denoted as metagraph) to de-

scribe the combination of relations and facets in a social media context (ref. Figure

9(b)). A hypergraph is a graph where edges, called hyperedges, connect to any

number of vertices. The idea is to use an M-way hyperedge to represent the inte-

ractions of M facets: each facet as a vertex and each relation as a hyperedge on a

hypergraph. A metagraph defines a particular structure of interactions among fa-

cets, not among facet elements. Formally, for a set of facets V={v(q)

} and a set of

relations E={e(r)

}, we construct a metagraph G=(V,E). To reduce notational com-

plexity, V and E also represent the set of all vertex and edge indices, respectively.

A hyperedge/relation e(r)

is said to be incident to a facet/vertex v(q)

if v(q)e

(r),

which is represented by v(q)

~e(r)

or e(r)

~v(q)

. E.g., in Figure 9(b), the vertex v(1)

23

represents the user facet, the hyperedge e

(1)={v

(1),v

(2),v

(3)} represents the ―book-

mark‖ relation.

Based on the discussed in Section 4.1, we assume the interaction between any two

entities (users or media objects) i and j in a community k, written as xij, can be

viewed as a function of the relationships between community k with entity i, and k

with j. If we consider the function to be stochastic, i.e. let pki indicate how likely

an interaction in the k-th community involves the i-th entity and pk is the proba-

bility of an interaction in the k-th community, we can express xij by xijk

pkipkjpk (as discussed in section 3.2). Likewise a 3-way interaction among enti-

ty i1, i2 and i3 is 𝑥𝑖1𝑖2𝑖3≈ 𝑝𝑘 ∙𝑘 𝑝𝑘→𝑖1

∙ 𝑝𝑘→𝑖2∙ 𝑝𝑘→𝑖3

. A set of such interactions

among entities in facet v(1)

, v(2)

and v(3)

can be written by:

𝓧 ≈ 𝑝𝑘𝐮𝑘(1)

°𝐾𝑘=1 𝐮𝑘

(2)°𝐮𝑘

(3)= [𝐳] ×𝑚 𝐔(𝑚)3

𝑚=1 (11)

where 𝒳∈ ℜ+𝐼1×𝐼2×𝐼3 , is the data tensor representing the observed three-way inte-

ractions among facet v(1)

, v(2)

and v(3)

. 𝑝𝑘→𝑖𝑞 is written as an (iq,k)-element of U(q)

for q=1,2,3. U(q)

is an IqK matrix, where Iq is the size of v(q)

. The probabilities of

communities are elements of z, i.e. pk=zk. We use [z] to denote a superdiagonal

tensor, where the operation [⋅] transforms a vector z to a superdiagonal tensor by

setting tensor element zk…k=zk and other elements as 0. The decomposition defined

in eq.(11) is similar to the CP/PARAFAC tensor decomposition [7,18], except that

the core tensor [z] and the factor matrices {U(q)

} are constrained to contain non-

negative probability values. Under the nonnegative constraints, the 3-way tensor

factorization is equivalent to the three-way aspect model in a three-dimensional

co-occurrence data [33].

The nonnegative tensor decomposition can be viewed as community discovery in

a single relation. The interactions in social media networks are more complex –

usually involving multiple two- or multi-way relations. By using metagraphs, we

represent a diverse set of relational context in the same form and define communi-

ty discovery problem on a metagraph, with the following two technical issues: (a)

how to extract community structure as coherent interaction latent spaces from ob-

served social data defined on a metagraph, and (b) how to extract community

structure as coherent interaction latent spaces from time evolving data given a me-

tagraph. The problems are formally stated as follows.

Definition 3 (Metagraph Factorization, or MF): given a metagraph G=(V,E) and a

set of observed data tensors {𝒳(r)}rE defined on G, find a nonnegative core tensor

[z] and factors {U(q)

}qV for corresponding facets V={v(q)

}. (Since E also

represents the set of all edge indices, the notations rE and e(r)E are exchangea-

ble. Likewise, qV and v(q)V are exchangeable.)

24

Definition 4 (Metagraph Factorization for Time evolving data, or MFT): given a

metagraph G=(V,E) and a sequential set of observed data tensors {𝒳t(r)

}rE defined

on G for time t=1,2,…, find nonnegative core tensor [zt] and factors {Ut(q)

}qV for

corresponding facets V={v(q)

}.

We will present our method in two steps: (1) present a solution to MF (next sec-

tion); (2) extend the solution to solve MFT (Section 4.2.3).

4.2.2 Metagraph Factorization

The MF problem can be stated in terms of optimization. Let us first consider a

simple metagraph case. Assume we are given a metagraph G=(V,E) with three ver-

tices V={v(1)

, v(2)

, v(3)

} and two 2-way hyperedges E={e(a)

,e(b)

} that describe the in-

teractions among these three facets, as shown in Figure 10. The observed data cor-

responding to the hyperedges are two second-order data tensors (i.e. matrices)

{𝒳(a), 𝒳(b)

} with facets {v(1)

, v(2)

} and {v(2)

, v(3)

} respectively. The facet v(2)

is

shared by both tensors.

The goal is to extract community structure from data tensors, through finding a

nonnegative core tensor [z] and factors {U(1)

, U(2)

, U(3)

} corresponding to the three

facets. The core tensor and factors need to consistently explain the data, i.e. we

can approximately express the data by 𝒳(a)[z]1U

(1)2U

(2) and

𝒳(b)[z]2U

(2)3U

(3), as illustrated in Figure 10. The core tensor [z] and facet U

(2)

are shared by the two approximations, and the length of z is determined by the

number of latent spaces (communities) to be extracted. Since both the left- and the

right-hand side of the approximation are probability distributions, it is natural to

use the KL-divergence (denoted as D(||)) as a measure of approximation cost.

Figure 10: An example of the metagraph factorization (MF). Given observed

data tensors {(a),(b)

} and a metagraph G that describes the interaction

among facets {v(1)

, v(2)

, v(3)

}, find consistent community structure expressed by

core tensor [z] and facet factors {U(1)

, U(2)

, U(3)

}.

( )a( )b

I1

I2

I3

U(2)

U(1)

U(3)z

K

K

≈ ≈I2

v(2)

v(1) v(3)e(a) e(b)G

I1 I3

I2

data data

K

25

We can generalize Figure 10 to any metagraph G, as: given a metagraph G=(V,E),

the objective is to factorize all data tensors such that all tensors can be approx-

imated by a common nonnegative core tensor [z] and a shared set of nonnegative

factors {U(q)

}, i.e. to minimize the following cost function:

𝐽 𝐺 = min𝐳,{𝐔(𝑞)} 𝐷(𝓧(𝑟)||[𝐳] ×𝑚 𝐔(𝑚)

𝑚 :𝑣(𝑚 )~𝑒 (𝑟) )𝑟∈𝐸

s.t. 𝐳 ∈ ℜ+1×𝐾 , 𝐔(𝑞) ∈ ℜ+

𝐼𝑞×𝐾 ∀𝑞, 𝐔𝑖𝑘

(𝑞)= 1 ∀𝑞∀𝑘𝑖

(12)

where K is the number of communities, and D(||) is the KL-divergence as de-

scribed above. The constraint that each column of {U(q)

} must sum to one is added

due to the modeling assumption that the probability of an occurrence of a relation

on an entity is independent of other entities in a community. This equation can be

easily extended to incorporate weights on relations.

Solution. By employing the concavity of the log function (in the KL-divergence),

we derive a local minima solution to Eq. (12). The solution can be found by the

following updating algorithm:

𝐳𝑘 ← 𝓧𝑖1⋯𝑖𝑀𝑟

(𝑟)𝜇𝑖1⋯𝑖𝑀𝑟𝑘

(𝑟)𝑖1⋯𝑖𝑀𝑟𝑟∈𝐸

𝑈𝑖𝑞𝑘(𝑞)

← 𝓧𝑖1⋯𝑖𝑀𝑙

(𝑙)𝜇𝑖1⋯𝑖𝑀𝑙

𝑘(𝑙)

𝑖1⋯𝑖𝑞−1𝑖𝑞+1⋯𝑖𝑀𝑙𝑙 :𝑒 (𝑙)~𝑣(𝑞)

where 𝜇𝑖1⋯𝑖𝑀𝑟𝑘(𝑟)

←𝐳𝑘 ×𝑚𝐔𝑖𝑚 𝑘

(𝑚 )

𝑚 :𝑣(𝑚 )~𝑒(𝑟)

([𝐳] ×𝑚𝐔(𝑚 )𝑚 :𝑣(𝑚 )~𝑒(𝑟) )𝑖1⋯𝑖𝑀𝑟

(13)

where z is a length K vector, L=|E| denotes the total number of hyperedges on G.

After updates, each column of U(q)

and the vector z are normalized to sum to one.

Because of this normalization step, we have omitted the scaling constant for up-

dating z and U(q)

. This iterative update algorithm is a generalization of the algo-

rithm proposed by Lee et al. [24] for solving the single nonnegative matrix facto-

rization problem. In metagraph factorization, the update for core tensor [z]

depends on all hyperedges on the metagraph, and the update for each facet factor

U(q)

depends on the hyperedges incident to the facet. The details of this algorithm

can be found in [30].

4.2.3 Time Evolving Extension

In the MFT problem, the relational data is constantly changing as evolving tensor

sequences. We propose an online version of MF to handle dynamic data. Since

historic information is contained in the community model extracted based on pre-

viously observed data, the new community structure to be extracted should be

consistent with previous community model and new observations, which is similar

to evolutionary clustering discussed in Section 3. To achieve this, we extend the

objective in Eq. (12) as follows.

A community model for a particular time t is defined uniquely by the factors

{Ut(q)

} and core tensor [zt]. (To avoid notation clutter, we omit the time indices for

t.) For each time t, the objective is to factorize the observed data into the nonnega-

26

tive factors {U

(q)} and core tensor [z] which are close to the prior community

model, [zt-1] and {Ut-1(q)

}. We introduce a cost lprior to indicate how the new com-

munity structure deviates from the previous structure in terms of the KL-

divergence. The new objective is defined as follows:

𝐽2 𝐺 = min𝐳, 𝐔 𝑞 1 − 𝛼 𝐷(𝓧 𝑟 ||[𝐳] ×𝑚 𝐔 𝑚

𝑚 :𝑣 𝑚 ~𝑒 𝑟 )𝑟∈𝐸 +

𝛼𝑙𝑝𝑟𝑖𝑜𝑟

with 𝑙𝑝𝑟𝑖𝑜𝑟 = 𝐷 𝐳𝑡−1||𝐳 + 𝐷(𝐔𝑡−1(𝑞)

||𝐔 𝑞 )𝑞

s.t. 𝐳 ∈ ℜ+1×𝐾 , 𝐔(𝑞) ∈ ℜ+

𝐼𝑞×𝐾 ∀𝑞, 𝐔𝑖𝑘

(𝑞)= 1 ∀𝑞∀𝑘𝑖

(14)

where α is a real positive number between 0 and 1 to specify how much the prior

community model contributes to the new community structure. lprior is a regulariz-

er used to find similar pair of core tensors and pairs of facet factors for consecu-

tive time. The new community structure will be a solution incrementally updated

based on a prior community model.

Solution. Based on a derivation similar to the discussion in Section 4.2.2, we pro-

vide a solution to Eq. (14) as follows:

𝐳𝑘 ← 1 − 𝛼 𝓧𝑖1⋯𝑖𝑀𝑟

𝑟 𝜇𝑖1⋯𝑖𝑀𝑟𝑘 𝑟

𝑖1⋯𝑖𝑀𝑟𝑟∈𝐸 + 𝛼𝐳𝑘 ;𝑡−1

𝑈𝑖𝑞𝑘(𝑞)

← (1 − 𝛼) 𝓧𝑖1⋯𝑖𝑀𝑙

(𝑙)𝜇𝑖1⋯𝑖𝑀𝑙

𝑘(𝑙)

𝑖1⋯𝑖𝑞−1𝑖𝑞+1⋯𝑖𝑀𝑙𝑙 :𝑒 (𝑙)~𝑣(𝑞) + 𝛼𝑈𝑖𝑞𝑘 ;𝑡−1(𝑞)

(15)

where z is a length K vector, 𝜇𝑖1⋯𝑖𝑀𝑙𝑘

(𝑙) is defined as in eq. (13). After updates, each

column of U(q)

and the vector z are normalized to sum to one. Because of this

normalization step, we have dropped the scaling constant for updating z and U(q)

.

It can be shown that the parameters in the previous model (zt-1 and {Ut-1(q)

}) act as

Dirichlet prior distribution to inform the solution search (ref. [28]), thus the solu-

tion is consistent with previous community structure.

4.3 Application: Context-sensitive Prediction in Enterprise

We design a prediction task to illustrate how our community tracking algorithm

can be utilized to predict users’ future interests based on the multi-relational social

data. Specifically, given data Dt at time t, we extract communities to predict us-

ers’ future use of tags, and compare the prediction with the ground truth in data

Dt+1. We collected collaboration relationships from the employee profiles and so-

cial media (e.g. bookmarks, wiki, etc.) in an enterprise. We then construct multiple

relations from the different data sources.

In our experiment, the time interval is one month. The overall prediction perfor-

mance is obtained by taking average prediction performance over 10-month data.

We compare our method with two baseline methods: (1) recurring interests – pre-

dicting future tags (at t+1) as the tags mostly frequently used by the user at t. (2)

collective interests (pLSA) – predicting future tags by using a well-known collec-

27

tive filtering method (probabilistic latent semantic analysis [19] or pLSA) on the

user-tag matrix.

We generate predictions base on the community structure extracted by our me-

thod, denoted by MF and MFT. The MF algorithm outputs community structure

from relational data of each time slot t-1. The MFT algorithm uses the same data

as MF, with an aid of prior community model extracted for time t-2 as an informa-

tive prior. Hence MFT gives results incrementally. From an extracted community

model we obtain the probability of a community k, i.e. p(k), and the probability of

a user u and a tag q, given community k, i.e. p(u|k) and p(q|k). Then a prediction is

made based on the condition probability p(q|u)p(u,q)k’p(k)p(u|k’)p(q|k’). The

detailed experiment setting can be found in [30]. Our method can also be applied

to a cold-start setting by incorporating a folding-in technique (ref. e.g. [33]) to

overcome the situation where p(q|k) may not be directly available from the model

parameters (e.g. q is a new tag which has not been used before t).

Figure 11 shows the relevant improvement compared with the first baseline me-

thod, i.e. the recurring interests. The results indicate the prediction given by our

community tracking algorithms outperform the baseline methods by 36-250% on

the average, which suggest that our method can better capture the cohesive struc-

tures of the contexts around users’ interests.

Summary. We proposed the MetaFac framework to extract community structures

from various social contexts and interactions. There were three key ideas: (1) me-

wiki

R1

user

tag

resource (URL)

R4

country

department

directory

R5

R3 R2

P@10 NDCG

1

1.2

1.4

1.6

1.8

2

rela

tive

imp

rove

men

t

recurring pLSA MF MFT

Figure 11: (a) IBM dataset: R1, ..., R5 are different relations among the 7 fa-

cets, e.g. bookmark (R1), join-wiki (R2), etc. The sizes of these relational data

from R1~R5 are: 3K×12K×61K, 3K×1K, 3K×2K, 3K×90 and 3K×42. (b) Pre-

diction performance: Our framework improves the prediction of users’ future

tag use.

(a) IBM dataset (b) Prediction performance

28

tagraph, a relational hypergraph for representing multi-relational social data; (2)

MF algorithm, an efficient non-negative multi-tensor factorization method for

community extraction on a given metagraph; (3) MFT, an on-line factorization

method to handle time-varying relations. To illustrate the utility of our method, we

design a tag prediction task in an enterprise context. We generated the predictions

based on the extracted community models and compare results with baselines. Our

method outperformed baselines up to an order of magnitude. We show significant

improvement of our method due to (a) incorporating a historic model and (b) leve-

raging diverse relations through a metagraph.

There are some open issues in this work. For example, there are different aspects

of community evolution, including change in the community size, change in the

number of communities and change in the community content or features (what

the community is about). To study the evolution within communities, our method

has assumed the number of communities does not change across time (i.e. we do

not consider the second aspect). Learning and comprehending several evolution

aspects in a unified process is a challenging issue.

Nevertheless, our work can lead to several interesting directions. (1) As our algo-

rithm does not tie to a specific data schema, it can be easily extended to deal with

schema changes. (2) By combining various social relations of data, it can be used

to identify effective social relations based on model selection approaches. As a po-

tential extension of this framework, we are interested in utilizing the relational

hypergraph to study the correlation between networks and the behavioral dynam-

ics of individuals.

5. Conclusions and Future Directions

In this article, we have discussed our current work on community analysis in dy-

namic, multi-relational social networks. Our work includes several key ideas: (1)

We introduce mutual awareness, a fundamental property of communities in online

social media, which is computationally defined based on observable individual ac-

tions within the social media context. The effectiveness of mutual awareness fea-

tures is empirically verified on large-scale real-world blog datasets. We propose

an efficient iterative mutual awareness expansion algorithm for community extrac-

tion using a random walk based distance measure that quantifying the amount of

mutual awareness expanding on the entire network. The community evolution

with respect to a query reveals interesting community dynamics. (2) We propose

FacetNet framework, the first probabilistic generative model that simultaneously

analyzes communities and their evolutions. The results obtained from our method

allow us to assign soft community memberships to individual nodes, to analyze

the strength of ties among various communities, to study how the affiliations of an

individual to different communities change over time, as well as to reveal how

communities evolve over time. Extensive experimental studies demonstrated that

29

by using our FacetNet framework, we are able to discover many interesting in-

sights in dynamic networks that are not directly obtainable from existing methods,

such as the evolution of individuals' contribution to different communities. (3) We

propose MetaFac, the first graph-based tensor factorization framework for analyz-

ing the dynamics of heterogeneous social networks. We introduce metagraph for

modeling multi-relational and multi-dimensional social data. Then we propose an

efficient non-negative multi-tensor factorization method for community extraction

on a given metagraph. In addition, we provide an on-line extension of this method

to handle time-varying multi-relations. Extensive experiments on enterprise and

large-scale social media data suggest that our technique is scalable and can help

predict users’ future interests based on the cohesive structure of contexts extracted

by our method.

Our current work has led to several interesting research directions. In social media

like Facebook, users often experience overload of online social connections,

shared interests and information, as well as the interplay of people and subject

matter. For people who are interested in certain topics, it is difficult to understand

how the topics (or media items) are, and have been, shared and discussed by dif-

ferent people. This leads to important technical questions on how to disentangle

and display the complex dynamic relationships between people and subjects over

time. Our methods can be used to support interactive visualization that allows us-

ers to explore and query multiple aspects of community activities, such as relevant

topics, representative users and artifacts (e.g. tweets, photos) of the communities,

as well as their relationships and evolutions.

Our work can also contribute to research in social science. For example, anthro-

pologists are interested in material cultural transition based on artifacts and their

association with time and space. Our work on community analysis has identified

several relevant structural elements from large scale observable interpersonal ac-

tivities in social media, including community awareness (mutual and transitive

awareness), community composition (degree of individuals’ participation in com-

munities), community inter-structure (the relationship among communities and

how it changes over time), community context (e.g. time and location associated

with community activities) and community artifacts (e.g. tags and photos generat-

ed by communities). We have used these structural elements to discover interest-

ing cultural patterns among communities, e.g. the interaction of right-wing and

left-wing communities in blog data. We plan to extend the current approach to

discover the structural changes of a community as well as the context where the

changes occur, to support the search and detection of emergent or transitional cul-

tural patterns. Ethnographic investigation is needed in order to reveal finer-grained

patterns of human interactions as well as qualitative understanding of the extracted

structures. Extracting the evolution of inter-structure among different communities

30

will help understand the condition and impact of social media as a new commu-

nicative practice.

6. Acknowledgement

This material is based upon work supported in part by NEC Labs America, an

IBM Ph.D. Fellowship and a Kauffman Entrepreneur Scholarship. We are pleased

to acknowledge Yun Chi, Shenghuo Zhu, Belle Tseng, Jun Tatemura and Koji Hi-

no, from NEC Labs America, for providing the invaluable advices on community

discovery and the NEC Blog dataset. We are indebted to Jimeng Sun, Paul Castro

and Ravi Konuru, from IBM T.J. Watson Research Center, for providing advices

on tensor analysis and the IBM enterprise data.

References

[1] A. AGOSTINO (1999). The relevance of media as artifact:

Technology situated in context. Educational Technology &

Society 2(4): 46-52.

[2] S. ASUR, S. PARTHASARATHY, et al. (2007). An Event-based

Framework for Characterizing the Evolutionary Behavior of

Interaction Graphs, Proceedings of the 13th ACM SIGKDD

international conference on Knowledge discovery and data

mining,

[3] L. BACKSTROM, D. HUTTENLOCHER, et al. (2006). Group

formation in large social networks: membership, growth,

and evolution, SIGKDD, 44-54, 2006.

[4] L. BACKSTROM, R. KUMAR, et al. (2008). Preferential beha-

vior in online groups. Proceedings of the international confe-

rence on Web search and web data mining: 117-128.

[5] S. BRIN and L. PAGE (1998). The anatomy of a large-scale

hypertextual Web search engine. Computer Networks and

ISDN Systems 30(1-7): 107-117.

[6] J. BROWN, A. COLLINS, et al. (1989). Situated cognition and

the culture of learning. Educational researcher 18(1): 32.

[7] J. CARROLL and J. CHANG (1970). Analysis of individual dif-

ferences in multidimensional scaling via an N-way generali-

zation of “Eckart-Young” decomposition. Psychometrika

35(3): 283-319.

31

[8] D. CHAKRABARTI, R. KUMAR, et al. (2006). Evolutionary

clustering, SIGKDD, 554-560,

[9] Y. CHI, X. SONG, et al. (2007). Evolutionary Spectral Clus-

tering by Incorporating Temporal Smoothness, SIGKDD,

2007.

[10] F. CHUNG and S. YAU (2000). Discrete Green's functions. J.

of Combinatorial Theory (A) 91(1-2): 191-214.

[11] I. DHILLON, Y. GUAN, et al. (2005). A unified view of kernel

k-means, spectral clustering and graph partitioning. Tech-

nical report, University of Texas at Austin, 2005.

[12] P. DOURISH (2001). Where the Action Is:: the Foundations of

Embodied Interaction, Mit Pr.

[13] N. FRIEDKIN and E. JOHNSEN (1999). Social influence net-

works and opinion change. Advances in Group Processes 16:

1-29.

[14] H. GARFINKEL (1984). Studies in ethnomethodology, Polity.

[15] D. GARLASCHELLI and M. LOFFREDO (2004). Patterns of link

reciprocity in directed networks. Physical Review Letters

93(26): 268701.

[16] M. GRANOVETTER (1973). The strength of weak ties. Ameri-

can journal of sociology 78(6): 1360.

[17] M. GRANOVETTER (1985). Economic action and social struc-

ture: A theory of embeddedness. American journal of sociol-

ogy 91(3): 481-510.

[18] R. HARSHMAN (1970). Foundations of the PARAFAC proce-

dure: Models and conditions for an" explanatory" multi-

modal factor analysis. UCLA working papers in phonetics

16(1): 84.

[19] T. HOFMANN (1999). Probabilistic latent semantic indexing,

SIGIR, 1999.

[20] Q. JONES (1997). Virtual-communities, virtual settlements &

cyber-archaeology: A theoretical outline. Journal of Com-

puter Mediated Communication 3(3): 35-49.

[21] R. KANNAN, S. VEMPALA, et al. (2004). On Clusterings:

Good, Bad and Spectral. J. of the ACM 51(3): 497-515.

32

[22] J. M. KLEINBERG (1999). Authoritative sources in a hyper-

linked environment. J. ACM 46(5): 604-632.

[23] T. KOLDA and J. SUN (2008). Scalable Tensor Decomposi-

tions for Multi-aspect Data Mining, ICDM, 2008.

[24] D. LEE and H. SEUNG (2001). Algorithms for non-negative

matrix factorization, NIPS, 556–562, 2001.

[25] J. LEMKE (1997). Cognition, context, and learning: A social

semiotic perspective. Situated cognition: Social, semiotic,

and psychological perspectives: 37–56.

[26] Y.-R. LIN, H. SUNDARAM, et al. (2006). Discovery of Blog

Communities based on Mutual Awareness, the 3rd Annual

Workshop on the Weblogging Ecosystems: Aggregation,

Analysis and Dynamics,

[27] Y.-R. LIN, H. SUNDARAM, et al. (2007). Blog Community

Discovery and Evolution Based on Mutual Awareness Ex-

pansion, 2007 IEEE/WIC/ACM International Conference on

Web Intelligence, 2007.

[28] Y.-R. LIN, Y. CHI, et al. (2008). FaceNet: A Framework for

Analyzing Communities and Their Evolutions in Dynamics

Networks, WWW, 2008.

[29] Y.-R. LIN, Y. CHI, et al. (2009). Analyzing Communities and

Their Evolutions in Dynamics Networks. Transactions on

Knowledge Discovery from Data (TKDD) 3(2).

[30] Y.-R. LIN, J. SUN, et al. (2009). MetaFac: Community Dis-

covery via Relational Hypergraph Factorization, SIGKDD,

2009.

[31] J. MOODY and D. WHITE (2003). Structural cohesion and

embeddedness: A hierarchical concept of social groups.

American Sociological Review: 103-127.

[32] G. PALLA, A. BARABASI, et al. (2007). Quantifying social

group evolution. eprint arXiv: 0704.0744.

[33] A. POPESCUL, L. H. UNGAR, et al. (2001). Probabilistic Mod-

els for Unified Collaborative and Content-Based Recom-

mendation in Sparse-Data Environments, UAI 2001, 437-

444,

33

[34] H. RHEINGOLD (1899). Virtual community: Homesteading on

the electronic frontier, Mit Press.

[35] M. SAHAMI and T. HEILMAN (2006). A web-based kernel

function for measuring the similarity of short text snippets,

Proc. of the 15th international conference on World Wide

Web, 377-386, 2006.

[36] J. SHI and J. MALIK (2000). Normalized Cuts and Image

Segmentation. IEEE Transactions on Pattern Analysis and

Machine Intelligence 22(8): 888-905.

[37] J. TRAVERS and S. MILGRAM (1969). An Experimental Study

of the Small World Problem. Sociometry 32(4): 425-443.

[38] S. WASSERMAN and K. FAUST (1994). Social Network Anal-

ysis: methods and applications, Cambridge University Press.

[39] K. YU, S. YU, et al. (2005). Soft Clustering on Graphs.

NIPS’05.

Index terms (alphabetically):

Community discovery

Community dynamics

Dynamic networks

FacetNet

Graph mining

MetaFac

Multi-relational mining

Mutual awareness

Social media

Social network analysis