RESEARCH Open Access

Techniques for comparing andrecommending conferencesGrettel Monteagudo García1, Bernardo Pereira Nunes1,2, Giseli Rabello Lopes3, Marco Antonio Casanova1*

and Luiz André P. Paes Leme4

Abstract

This article defines, implements, and evaluates techniques to automatically compare and recommend conferences.The techniques for comparing conferences use familiar similarity measures and a new measure based on co-authorshipcommunities, called co-authorship network community similarity index. The experiments reported in thearticle indicate that the technique based on the new measure performs better than the other techniquesfor comparing conferences, which is therefore the first contribution of the article. Then, the article focuses on threefamilies of techniques for conference recommendation. The first family adopts collaborative filtering based on theconference similarity measures investigated in the first part of the article. The second family includes two techniquesbased on the idea of finding, for a given author, the strongest related authors in the co-authorship network andrecommending the conferences that his co-authors usually publish in. The first member of this family isbased on the Weighted Semantic Connectivity Score—WSCS, which is accurate but quite costly to compute for largeco-authorship networks. The second member of this family is based on a new score, called the Modified WeightedSemantic Connectivity Score—MWSCS, which is much faster to compute and as accurate as the WSCS. The third familyincludes the Cluster-WSCS-based and the Cluster-MWSCS-based conference recommendation techniques, which adoptconference clusters generated using a subgraph of the co-authorship network. The experiments indicate as the bestperforming conference recommendation technique the Cluster-WSCS-based technique. This is the second contributionof the article. Finally, the article includes experiments that use data extracted from the DBLP repository and a web-based application that enables users to interactively analyze and compare a set of conferences.

Keywords: Conference comparison, Conference recommendation, Co-authorship networks, Social network analysis,Recommender systems, Linked data

IntroductionConferences provide an important channel for theexchange of information and experiences amongresearchers. The academic community organizes a largenumber of conferences, in the most diverse areas, gener-ating a rich set of bibliographic data. Researchersexplore such data to discover topics of interest, findrelated research groups, and estimate the impact ofauthors and publications [1–6]. Choosing a good confer-ence or journal in which to publish an article is in factvery important to researchers. The choice is usuallybased on the researchers’ knowledge of the publicationvenues in their research area or on matching the

conference topics with their paper subject. Indeed, theidentification of relevant publication venues presents noproblems when the researcher is working in his area. Itis less obvious, though, when the researcher moves to anew area.In this article, we define, implement, and evaluate

techniques to automatically compare and recommendconferences that help address the questions of selectingand evaluating the importance of conferences. From abroad perspective, techniques for comparing conferencesinduce clusters of similar conferences, when applied to aconference catalog. Therefore, when one finds one ormore familiar conferences in a cluster, he may considerthat the other conferences in the cluster are similar tothose he is familiar with. Techniques for recommendingconferences, on the other hand, select conferences

* Correspondence: casanova@inf.puc-rio.br1Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, RJ, BrazilFull list of author information is available at the end of the article

Journal of theBrazilian Computer Society

© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made.

García et al. Journal of the Brazilian Computer Society (2017) 23:4 DOI 10.1186/s13173-017-0053-z

according to a given criteria and rank them in order ofimportance. Thus, when one finds a conference closer tothe top of the ranked list, he may consider that the givenconference is more important than those lower down inthe list, within the bounds of the given criteria.The techniques for comparing conferences adopt

familiar similarity measures, such as the Jaccard similar-ity coefficient, the Pearson correlation similarity and theCosine similarity, and a new similarity measure, calledthe co-authorship network community similarity index.The experiments reported in the article indicate that thebest performing technique for comparing conferences isthat based on the new similarity measure, which istherefore the first contribution of the article.The article proceeds to define three families of confer-

ence recommendation techniques. The first family oftechniques adopts collaborative filtering based on theconference similarity measures investigated in the firstpart of the article. The second family includes two tech-niques based on the idea of finding, for a given author,the strongest related authors in the co-authorshipnetwork and recommending the conferences that his co-authors usually publish in. The first member of thisfamily is based on the Weighted Semantic ConnectivityScore—WSCS, an index for measuring the relatedness ofactors. However, since this index proved to be accuratebut quite costly for large co-authorship networks, thearticle introduces a second technique based on a newscore, called the Modified Weighted Semantic Connectiv-ity Score—MWSCS, which is much faster to computeand as accurate as the WSCS. The third family ofconference recommendation techniques includes theCluster-WSCS-based and the Cluster-MWSCS-basedtechniques, which adopt conference clusters generatedusing a subgraph of the co-authorship network, in-stead of the full co-authorship network. The experi-ments suggest that the WSCS-based, MWSCS-based,and Cluster-WSCS-based techniques perform betterthan the benchmark and better than the techniquesbased on similarity measures. Furthermore, betweenthese three techniques, the experiments permit us toconclude that the Cluster-WSCS-based techniqueshould be preferred because it is more efficient andhave no statistically significant differences when com-pared to the WSCS-based and MWSCS-based tech-niques. This is the second contribution of the article.The experiments mentioned in the previous para-

graphs use data extracted from a triplified version of thedblp computer science bibliography (DBLP) repository,which stores Computer Science bibliographic data formore than 4500 conferences and 1500 journals (as ofearly 2016). The experiments were performed using aweb-based application that enables users to interactivelyanalyze and compare a set of conferences.

The remainder of this article is structured as follows.The “Related work” section summarizes similar work.The “Techniques” section introduces the conferencecomparison and the conference recommendation tech-niques. The “Results and discussion” section presents anapplication that implements the techniques anddescribes their evaluation. Finally, the “Conclusions”section summarizes the main contributions of thisarticle.

Related workHenry et al. [1] analyzed a group of the four major con-ferences in the field of human-computer interaction(HCI). The authors discovered many global and localpatterns using only article metadata, such as authors,keywords, and year. Blanchard [2] presented a 10-yearanalysis of the paper production in intelligent tutoringsystems (ITS) and Artificial Intelligence in Education(AIED) conferences and showed that Western, Educated,Industrialized, Rich, and Democratic bias observed inpsychology may be influencing AIED research. Chen,Zhang, and Vogeley [3] proposed an extension of thecontemporary co-citation network analysis to identifyco-citation clusters of cited references. Intuitively, theauthors synthesize thematic contexts in which theseclusters are cited and trace how the research focusevolved over time. Gasparini, Kimura, and Pimenta [4]presented a visual exploration of the field of human-computer interaction in Brazil from a 15-year analysis ofpaper production in the Brazilian Symposium on HumanFactors in Computing Systems (IHC). Recently, Barbosaet al. [5] published an analysis of the same conferenceseries. Chen, Song, and Zhu [6] opened a wide range ofopportunities for research agendas and trends in EntityRelationship conferences.Zervas et al. [7] applied social network analysis (SNA)

metrics to analyze the co-authorship network of theEducational Technology & Society (ETS) Journal. Procó-pio, Laender, and Moro [8] did a similar analysis for thedatabases field. Cheong and Corbitt [9, 10] analyzed thePacific Asia Conference on Information Systems and theAustralasian Conference on Information Systems.Recently, Lopes et al. [11, 12] carried out an extensive

analysis of the WEBIST conferences, involving authors,publications, conference impact, topics coverage, com-munity analysis, and other aspects. The analysis startswith simple statistics, such as the number of papers perconference edition and then moves on to analyze the co-authorship network, estimating the number of commu-nities, for example. The paper also includes an analysisof author indices, such as the h-index, topics and confer-ence areas, and paper citations.Linked data principles to publish conference data were

also used in [13].

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 2 of 14

All the above references focus on metrics typicalof social network analysis mostly to compare differ-ent instances of the same publication venue and donot cover recommendation issues. Contrasting withthe above references, in this article, we propose, im-plement, and evaluate several techniques to compareconferences in general and not a specific conferenceseries. The current implementation works with thetriplified version of the DBLP repository, whichcovers the vast majority of Computer Scienceconferences.We now turn to conference recommendation, a prob-

lem that attracted attention due to the increase in thenumber of conferences in recent years.Medvet et al. [14] considered a venue recommendation

system based on the idea of matching the topics of apaper, extracted from the title and abstract, with thoseof possible publication venues for the paper. We adopteda simpler approach to obtain the topics of a conferencefrom the set of keywords and titles of the papers pub-lished in the conference and their frequency, after elim-inating synonymous keywords.Pham et al. [15] proposed a clustering approach

based on user social information to derive venuerecommendation based on collaboration filtering andtrust-based recommendation. The authors used datafrom DBLP and Epinion to show that the proposedclustering technique-based collaboration filteringperforms better than traditional collaboration filter-ing algorithms. In this article, we also explorecollaborative filtering and conference clustering todefine families of conference recommendationtechniques.Chen et al. [16] proposed a method for recommending

academic venues based on the PageRank metric.However, unlike the original PageRank method, whichinduces a relationship network model of these venues,the authors proposed a method that considers the effectsof authors’ preference for each venue. Thus, the PageR-ank metric is computed on a mixed network wherenodes are academic venues and authors and edges arethe co-authoring and publishing relationships (author-vehicle). The score of the nodes is then defined as thecombination of the effects of co-authoring and publica-tion. The propagation of punctuation across the networkalso suffers a variation from the original PageRank. Eachadjacent node propagates its effects proportionally to thesimilarity with its neighbor. If two authors are similar,the score is more intensely propagated, that is, authorswith similar interests influence the score of a venuemore strongly.Boukhris et al. [17] proposed a recommendation

technique for academic venues for a particular targetresearcher, TR. The technique prioritizes the venues

most used by the researchers that cite a TR. The citationintensities are adjusted with factors that intend to meas-ure the interest of a researcher by the work of TR sothat the venues of the researchers most strongly in-terested in the work of TR will have greater rele-vance. To solve the problem of target researcherswith few citations, the recommendation process usesco-authors and colleagues from the same institutionas TR. A final step in the recommendation processallows filtering the ranking results according to re-quirements reported by users.Yang and Davison [18] proposed an interesting ap-

proach for venue recommendation based on stylometricfeatures. They argue that the writing style and paper for-mat may serve as features for collaborative filtering-based methods. Their results show that the combinationof content features with stylometric features (lexical,structural, and syntactical) performs better than whenstylometric or content-based features are applied separ-ately. Although the accuracy reported is rather low,linguistic style and paper format remain as interestingfeatures to consider.Huynh and Hoang [19] proposed a simple network

model based on social network structure that mayserve to represent connections that goes beyondclassical “who knows whom” connections. Thus, forinstance, in their network model, the relationshipsbetween researchers can be based on co-authorshipmeasures and authors similarity. Their work canbenefit from ours by borrowing the metrics proposedhere.Asabere et al. [20] and Hornick et al. [21] addressed

the problem of recommending conference sessions toattendees. Similar to the venue recommendation prob-lem, recommendation techniques such as content andcollaborative-based methods are used to matchattendees and session presentations. The use of geoloca-tion information [20] and personal information providedat conferences [21] as features may also be incorporatedto improve venue recommendation. For instance, confer-ence and researcher locations can be used as featureswhen budget restrictions apply.Luong et al. [22] proposed and compared three rec-

ommendation methods for conferences. The methodsfind the most appropriate conference for a set of co-authors who want to publish a paper together. Thebest performing recommendation method, which wewill refer simply as the most frequent conference, isdivided into two stages. First, the method recursivelycollects the co-authors of the co-authors, until athree level deep network is created. Second, themethod weights the contributions of each co-authorby the number of papers they have co-authored withan author. It is defined as:

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 3 of 14

coauthor CONFi ¼XNm¼1

coauthors wi;m ð1Þ

where N is the set of co-authors who want to publish apaper together, i is a conference that might be recom-mended for the set N of co-authors, and coauthors_wi,m

is the weight of conference i for a co-author m∈N in theco-authorship network. This last function is defined as:

coauthors wi;m ¼XCoAk¼1

�nfreq CONFi;m

þ nfreq CONFi;kÞ � wCoAk;m

ð2Þ

where CoA is the set of co-authors of author m whohave published at conference i, w_CoAk,m is the numberof times author m co-authored papers with another au-thor k in the co-authorship network, nfreq_CONFi,m isthe probability of author m publishing in conference i,and likewise nfreq_CONFi,k is the probability of author kpublishing in conference i. In this article, we adoptedLuong’s most frequent conference technique as thebenchmark and, therefore, included a somewhat moredetailed account of their work.In this article, we propose two conference recommen-

dation techniques based on a social network analysis ofthe co-authorship network, but we adopt a measure ofthe strength of the connections between the authors inthe network which is computed differently from the pre-vious methods. We first propose to estimate the related-ness of actors in a social network by using a semanticconnectivity score [23], denoted SCS, which is in turnbased on the Katz index [24]. This score takes into ac-count the number of paths between two nodes of thenetwork and the accumulated weights of these paths.Then, we propose a second score that approximates theSCS score and uses the shortest path between twonodes. In addition to these two strategies, we alsopropose to construct a utility matrix and to implementrecommendation techniques based on collaborativefiltering using the utility matrix.

TechniquesIn this section, we introduce the conference comparisonand the conference recommendation techniques, whichare the main trust of the article. We refer the reader to[25] for illustrative examples of the techniques.

Conference comparison techniquesAs mentioned in the Introduction section, the tech-niques for comparing conferences induce clusters ofsimilar conferences, when applied to a conference cata-log. They adopt familiar similarity measures, such as theJaccard similarity coefficient, the Pearson correlation

similarity and the Cosine similarity, and a new similaritymeasure, called the co-authorship network communitysimilarity index.In what follows, we use the following notation:

� C is a set of conferences� A is a set of authors� P is a set of papers� pa : A→ 2P is a function that assigns to each

author i∈ A the set of papers pa(i)⊆ P that authori published (in any conference)

� pc : C→ 2P is a function that assigns to eachconference x∈ C the set of papers pc(x)⊆ P thatwere published in x

� pac : A × C→ 2P is a function that assigns to eachauthor i∈ A and each conference x∈ C the setof papers pac(i, x)⊆ P that author i published inconference x

� Ax and Ay are the set of authors that publishedin conferences x and y, that is, Ax = {i∈ A /|pac(i, x)| > 0} and, likewise, Ay = {i∈ A / |pac(i, y)| > 0}

� Ax,y is the set of authors that published in bothconferences x and y, that is, Ax,y = {i∈A / |pac(i, x)| > 0 ∧ |pac(i, y)| > 0}

� Gx = (Nx, Ex), the co-authorship network ofconference x, is an undirected and unweightedgraph where i∈Nx indicates that author ipublished in conference x and {i, j}∈ Exrepresents that authors i and j co-authored oneor more papers published in conference x

Similarity measures based on author informationIn what follows, we adapt familiar similarity measures toconferences and authors and introduce a new measurecalled community similarity.The Jaccard Similarity Coefficient for conferences x

and y is defined as

jaccard sim x; yð Þ ¼ Ax∩Ay� �

= Ax∪Ay� � ð3Þ

The utility matrix expresses the preferences of anauthor for a conference to publish his research. Moreformally, the utility matrix [rx,i] is such that a line xrepresents a conference and a column i represents anauthor and is defined as:

rx;i ¼ pac i; xð Þj jpa ið Þj j ð4Þ

Based on the utility matrix [rx,i], we define thePearson’s Correlation Coefficient between conferencesx and y as follows:

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 4 of 14

pearson sim x; yð Þ ¼X

i∈Ax;yrx;i− �rx� �

⋅ ry;i−�ry� �

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXi∈Ax;y

rx;i− �rx� �2⋅X

i∈Ax;yry;i−�ry� �2r

ð5Þwhere �rx is the average of the elements of line x of theutility matrix (and likewise for �ry).Again, based on the utility matrix [rx,i], we define the

Cosine Similarity between conferences x and y asfollows:

cos sim x; yð Þ ¼X

i∈Ax;yrx;i⋅ry;i� �

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiXi∈Ax;y

rx;i� �2⋅X

i∈Ax;yry;i� �2r ð6Þ

We introduce a new similarity measure betweenconferences based on communities defined over theco-authorship network of the conferences. Given theco-authorship network Gx = (Nx, Ex) of conference x,we define an author community cx of x as the net ofnodes of a connected component of Gx. Let cx and cybe author communities in the co-authorship networksof conferences x and y, respectively. We say that cxand cy are equivalent w.r.t. a similarity measure simand a threshold level α iff sim(cx, cy) ≥ α. For example,sim may be defined using the Jaccard similarity coeffi-cient between pairs of conferences introduced above.Let Cx and Cy be the sets of communities of confer-

ences x and y, respectively. Let EQ[sim, α](x, y) be the setof communities in the co-authorship network of confer-ence x that have an equivalent community in the co-authorship network of conference y (and symmetricallyEQ[sim, α](y, x)).The Co-authorship Network Communities Similarity

(based on a similarity measure sim and a threshold levelα) between conferences x and y is then defined as:

c sim sim; α½ � x; yð Þ ¼ EQ sim; α½ � x; yð Þj jmin Cxj j; Cy

�� ��� � ð7Þ

Note that |Cx| > 0 and |Cy| > 0 since Gx and Gy musthave at least one node each and therefore at least oneconnected component each.

Similarity measure based on conference keywordsIn the previous subsection, we proposed a utility matrixthat expresses the preferences of an author for a confer-ence to publish his research. However, we can alsoexpress the association of a topic with a conference.Therefore, in this section, we describe an algorithm toobtain the conference topics and introduce a new utilitymatrix that represents this information.To obtain the topics of the conference x, we first

extract, for each paper p ∈ pc(x), the set of keywords of

the paper, denoted by kwrds(p). Then, we define thefrequency of a keyword k for a conference x as:

f k; xð Þ ¼ p∈pc xð Þ= k ∈ kwrds pð Þf gj j ð8Þwhere the function kwrds(p) tries to eliminate synonym-ous keywords. In our implementation, we used the APIof the Big Huge Thesaurus1 to retrieve the synonyms ofa word, in English.The extraction of keywords for a paper, that is, the

computation of kwrds(p), is based on a lexical analysis ofpaper metadata. This process follows five steps:

1. Obtain the text for keyword extraction; in ourimplementation, we used the title and the keywordlist of the paper.

2. Tokenize the extracted text.3. Eliminate stopwords (i.e., the most common words

in a language).4. Eliminate suffixes to obtain the word lexeme.5. The resulting token list represents the keywords of

the paper.

We then define the set of keywords of a conference asfollows:

ckwrds xð Þ ¼ ∪p∈pc xð Þ

kwrds pð Þ ð9Þ

The database vocabulary is the union of all the rele-vant keywords for the conferences, that is:

K ¼ ∪x∈C

k ∈ ckwrds xð Þ=f k; xð Þ > βf g ð10Þ

where β is a frequency threshold, whose purpose is toeliminate keywords with low frequency.From the process of obtaining the keywords of a con-

ference, we can establish a new utility matrix thatexpresses the association of topics (keywords) with con-ferences. More formally, the utility matrix [sx,k] is suchthat a line x represents a conference and a column krepresents a keyword and is defined as:

sx;k ¼ f k; xð Þ ; if f f k; xð Þ > β0 ; otherwise

�ð11Þ

where β is the frequency threshold.The cardinality of the columns of the matrix [sx,k] is

the cardinality of the set K.The problem of comparing conferences using topics is

addressed by defining the similarity functions jaccard_sim_tpc(x,y), pearson_sim_tpc(x,y), cos_sim_tpc(x,y) andc_sim_tpc[sim,α](x,y), analogously to the functions jaccard_sim(x,y), pearson_sim(x,y), cos_sim(x,y), and c_sim[sim,α](x,y), respectively. To define the new functions,we apply the following transformations on the similarityfunctions introduced in the previous subsection:

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 5 of 14

� We substitute Ax and Ay by Kx and Ky, where Kx

and Ky are the sets of keywords that are relevant forconferences x and y, that is, Kx = {k∈ K / sx,k > 0}and Ky = {k∈A / sy,k > 0}.

� We substitute Ax,y by Kx,y, where Kx,y is the set ofkeywords relevant for both conferences x and y,that is, Kx,y = {k∈ K / sx,k > 0 ∧ sy,k > 0}.

Conference recommendation techniquesConference recommendation techniques based on classicalsimilarity measuresAs defined in [26], in a recommender system, there aretwo classes of entities—users and items. Users have pref-erences for certain items, which must be extracted fromthe data. The data itself is represented as a utility matrixgiving, for each user-item pair, a value that representswhat is known about the degree of preference or ratingof that user for that item. An unknown rating impliesthat there is no explicit information about the user’spreference for the item. The goal of a recommendationsystem is to predict the unknown ratings in the utilitymatrix.In our context, we recall from the “Conference com-

parison techniques” subsection that the utility matrix[rx,i] is such that rx,i expresses the preference (i.e., rating)of an author i for a conference x to publish his research.To predict an unknown rating, we compute the similar-ity between conferences and detect their nearest neigh-bors or most similar conferences. With this information,the rating of conference x for author i is defined asfollows:

CF x; ið Þ¼X

y∈Sxry;i� �

⋅sim x; yð ÞXy∈Sx

sim x; yð Þð12Þ

where Sx is the set of conferences most similar to x andry,i is the rating of conference y for author i.Therefore, we may immediately define a family of con-

ference recommendation techniques based on the utilitymatrix and the classical similarity measures introducedin the “Conference comparison techniques” subsectionthat we call CF-Jaccard, CF-Pearson, CF-Cosine, and CF-Communities, according to the similarity measureadopted. The “Results and discussion” section analyseshow they perform in detail.

Conference recommendation techniques based on theweighted authorship networkRecall from the “Conference comparison techniques”subsection that pa : A→ 2P is the function that assignsto each author i ∈ A the set of papers pa(i) ⊆ P thatauthor i published (in any conference). The weighted co-

authorship network based on pa is the edge-weightedundirected graph G = (N, E,w), where i ∈N representsan author, {i, j} ∈ E indicates that i and j are co-authors,that is, {i, j} ∈ E iff pa(i) ∩ pa(j) ≠∅, and w({i, j}) assigns aweight to the co-authorship relationship between i and j,and is defined as:

w i; jf gð Þ ¼ pa ið Þ ∩ pa jð Þj jpa ið Þ ∪ pa jð Þj j ð13Þ

Hence, the larger w({i, j}) is, the stronger the co-authorship relationship will be if authors i and j co-authored all papers they published, then w({i, j}) = 1;and if they have not co-authored any paper, then theedge {i, j} does not exist.The second family of conference recommendation tech-

niques explores the weighted co-authorship network andadopts two scores: the Weighted Semantic ConnectivityScore—WSCS and the Modified Weighted SemanticConnectivity Score—MWSCS. Hence, these techniquesare called WSCS-based and MWSCS-based recommenda-tion techniques.The Weighted Semantic Connectivity Score, WSCSe, is

defined by modifying the semantic connectivity scoreSCSe [7] to take into account the weight of the pathsbetween two authors i and j, computed as the sum ofthe weights of the edges in the path:

WSCSe i; jð Þ ¼XTw¼1

βw⋅ paths<w><i;j>

��� ��� ð14Þ

where paths<w><i;j>

��� ��� is the number of paths of weight equalto w between i and j, T is the maximum weight of thepaths, and 0 < β ≤ 1 is a positive damping factor.The conference recommendation technique based on

WSCSe works as follows. Given an author i, it starts bycomputing WSCSe(i, j), the score between i and anyother author j in the weighted co-authorship network.Then, it sorts authors in decreasing order of WSCSe,since authors that are more related to author i will havea higher WSCSe(i, j) value. For better performance, thetechnique considers only the first n authors in the listordered by WSCSe. Call this set Fi. For each author j inFi, the technique selects the conference c ∈ C with thehighest pac(j, c), denoted MaxCj. The rank of conferencex for author i is defined as follows:

rank x; ið Þ ¼Xj∈Fi

g x; ið Þ⋅WSCSe i; jð Þ ð15Þ

where g x; jð Þ ¼ 1 ; if f x ¼ MaxCj

0 ; otherwise

�

Since computing the WSCSe score can be very slowfor large graphs, we propose to compute only the

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 6 of 14

shortest paths from author i to other authors usingDijkstra’s algorithm. We then redefine the score as follows:

MWSCSe i; jð Þ ¼ βw ð16Þ

where w is a length of the shortest path from author i toauthor j. The recommendation technique remains basic-ally the same, except that it uses the MWSCS score.The results for the recommendation technique using

the MWSCSe score can be very different from those ob-tained using the WSCSe score. Indeed, it is easy to seethat, by using the MWSCSe score, we lose the informa-tion about all paths between the authors, except theshortest. For example, in the co-authorship network ofFig. 1, the pairs of authors (A1, A3) and (A1, A2), usingEq. 16, have the same MWSCSe, whereas the pair(A1,A3) should have a larger value; indeed, the path (A1,A4, A3) is ignored in the calculation of the MWSCSescore, using Eq. 16.

Conference recommendation techniques based onconference clustersIn the previous subsection, we presented two algorithmsto recommend conferences using the co-authorship net-work. The first algorithm, based on the WSCSe score, iscomputationally slower than the second, based on theMWSCSe score. Both algorithms are sensitive to the net-work size and, therefore, slower for large networks. Inthis section, we propose an algorithm to recommendconference that reduces the problem of recommendingconferences using the co-authorship network to theproblem of recommending conferences using a subgraphof the co-authorship network.We may immediately define a third family of confer-

ence recommendation techniques that contains twotechniques, called Cluster-WSCS-based and Cluster-MWSCS-based, if we use the WSCS and the MWSCSscores respectively to recommend conferences using a

subgraph of the co-authorship network, instead of thefull co-authorship network.Let u ⊆ C be a conference cluster. The co-authorship

network for u is the subgraph Gu = (Nu, Eu, w) of theweighted co-authorship network G = (N, E,w) such that:

Nu ¼ i∈N=∃c c∈u and pac i; cð Þj j > 0ð Þf g

Eu ¼ i; jf g∈E=∃c c∈u and pc cð Þ∩pa ið Þ∩pa jð Þð Þ≠∅ð Þf g

This family of recommendation techniques uses thefollowing pre-processing algorithm:

1. Obtain the set U of conferences clusters using asimilarity function s.

2. For each cluster u∈U, create the co-authorshipnetwork of the cluster.

3. For each cluster u∈U, obtain a vector Vu

representing cluster u.4. For each author i∈A, obtain a vector Vi

representing author i.

To define the algorithm, we need a function clus-ter_score(i, u) : A ×U→ℕ that assigns to each authori ∈ A and each cluster u ∈U a relationship score basedon the similarity between vectors Vi and Vu.Then, the general algorithm to recommend a confer-

ence to an author i is defined as:

1. Select ui such that ui ¼ maxu∈U clusterscore i; uð Þð Þ2. Apply a conference recommendation algorithm

(any of those proposed in the “Conferencerecommendation techniques based on classicalsimilarity measures” subsection) using theco-authorship network of cluster ui.

Steps 3 and 4 of the general algorithm and the defin-ition of cluster_score depend on the choice of the simi-larity function s used in step 1 of the pre-processingalgorithm. If we use one of the similarity functions intro-duced in the “Similarity measures based on author infor-mation” subsection, steps 3 and 4 and the cluster scoreare defined as:

� Step 3 computes, for each cluster u∈U, the vectorVu representing cluster u such that

Vu c½ � ¼ 1 ; iff c∈u0 ; otherwise

�

� Step 4 computes, for each author i∈ A, the vectorVi representing author defined exactly as thecolumn corresponding to author i in the utilitymatrix [rx,i] introduced in the “Similarity measuresbased on author information” subsection.

Fig. 1 Co-authorship network

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 7 of 14

� cluster_score is the similarity function s selected instep 1 of the pre-processing algorithm.

� However, if we use one of the similarity functionsintroduced in the “Similarity measure based onconference keywords” subsection, steps 3 and 4 andthe cluster score are defined as:

� Step 3 computes for each cluster u∈U, the vectorVu representing cluster u such that

Vu k½ � ¼ 1 ; if f k ∈ ∪c∈u Kc

0 ; otherwise

�

� Step 4 computes the keywords of the papers belongingto the author. The process is described in the“Similarity measure based on conference keywords”subsection for the case of the conference keywords.

� cluster_score is the Jaccard similarity functionbetween vectors Vu and Va.

Results and discussionExperimental environmentFigure 2 summarizes the architecture of the applicationdeveloped to run the experiments. The Conferences DataService handles queries to the triple store with conferencedata. The Co-authorship Network Service receives datafrom the Conferences Data Service and handles queries tothe Neo4j database. When an analysis is executed, the sys-tem stores the results for future reuse; the Previous Calcu-lation Service manages these functions. All experiments

that follow were executed in an Intel Core Quad 3.00GHz,with 6 GB RAM, running Windows 7.

Experiments with the conference similarity techniquesWe evaluated the conference similarity techniques assum-ing that the most similar conferences should fall in thesame category. We selected as benchmark the List of Com-puter Science Conferences defined in Wikipedia,2 whichcontains 248 academic computer science conferences, clas-sified in 13 categories. That is, the categories define a parti-tion P of the set of conferences. Then, we applied the sameclustering algorithm to this set of conferences but usingeach of the conference similarity measures. Finally, wecompared the clusters thus obtained with P. The best con-ference similarity measure would therefore be that whichresults in conference clusters that best match P.We adopted the hierarchical agglomerative clustering

algorithm, which starts with each conference as a single-ton cluster and then successively merges (or agglomer-ates) pairs of clusters, using similarity measures, untilachieving the desired number of clusters. To determinehow similar clusters are, and agglomerate them, a link-age criterion was used. The smallest value of these linksthat remains at each step causes the fusion of the twoclusters whose elements are involved.Let d(a, b) denote the distance between two elements

a and b. Familiar linkage criteria between two sets ofelements A and B are:

Fig. 2 Web application architecture

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 8 of 14

� Complete-linkage: the distance D(A, B) between twoclusters A and B equals the distance between thetwo elements (one in each cluster) that are farthestaway from each other:

D A;Bð Þ ¼ max d a; bð Þ=a∈A; b∈Bf g ð17Þ� Single-linkage clustering: the distance D(A, B)

between two clusters A and B equals the distancebetween the two elements (one in each cluster) thatare closest to each other:

D A;Bð Þ ¼ min d a; bð Þ=a∈A; b∈Bf g ð18Þ� Average linkage clustering: the distance D(A, B)

between two clusters A and B is taken as theaverage of the distances between all pairs ofobjects:

D A;Bð Þ ¼X

a∈A

Xb∈B

d a; bð ÞAj j⋅ Bj j ð19Þ

Before explaining the measures used to compare howwell different data clustering algorithms perform on aset of data, we need the following definitions. Given aset of n elements S and two partitions X and Y of S,where X is the correct partition and Y is the computedpartition, we define:

� TP (true positive) is the number of pairs of elementsin S that are in the same set in X and in the sameset in Y

� TN (true negative) is the number of pairs ofelements in S that are in different sets in X and indifferent sets in Y

� FN (false negative) is the number of pairs ofelements in S that are in the same set in X and indifferent sets in Y

� FP (false positive) is the number of pairs of elements inS that are in different sets in X and in the same set in Y

The measures to evaluate the performance of the cluster-ing algorithms using the proposed similarity functions are:

� Rand Index: measures the percentage of correctdecisions made by the algorithm:

RI ¼ TP þ TNTP þ TN þ FP þ FN

ð20Þ

� F-measure: balances the contribution of falsenegatives by weighting the recall through aparameter β > 0:

F ¼ β2 þ 1� �

P:R

β2P� �þ R

ð21Þ

where P ¼ TPTPþFP and R ¼ TP

TPþFN

Figure 3 shows the Rand Index obtained by execut-ing the hierarchical agglomerative clustering algorithmwith different linkages criteria, using the Jaccard,Pearson, Cosine, and Communities similarity mea-sures based on author information and conferenceskeywords. Note that, in general, the algorithm basedon the communities similarity had the best perform-ance, followed by the Jaccard similarity for similaritymeasures based on author information. In this case,the Cosine similarity had the worst behavior. Thesimilarity measures based on conference keywords arethe best, among which the Pearson and Cosine simi-larity achieved the best results.Figure 4 shows the F-measure obtained by executing

the same algorithms. By analyzing Fig. 4, we observe thatthe best performances, for the group of similarity basedon author information, were also obtained using thecommunities similarity and the Jaccard similarity mea-sures; the worst performance was obtained using thePearson similarity measure and the algorithm using theCosine similarity measure achieved the worst perform-ance only with the single-link linkage criterion. Again,all similarity measures based on conference keywordshad better results than the group based on author infor-mation, among which the Cosine similarity stands out.Therefore, these experiments suggest that the best per-

forming algorithm is that which adopts the communitiessimilarity measure.

Experiments with the conference recommendationtechniquesRecall that we proposed three families of recommendationtechniques. One family is based on similarity measuredefined in the “Similarity measures based on author infor-mation” subsection. These techniques are called CF-Jaccard, CF-Pearson, CF-Cosine, and CF-Communitiesbecause they use the Jaccard similarity, Pearson similarity,Cosine and a new similarity measure, the communitysimilarity respectively. The second family includes twotechniques based on the weighted and the modifiedweighted semantic connectivity, called WSCS-based andMWSCS-based recommendation techniques. Finally, thethird family uses the techniques based on the subgraph ofthe co-authorship network and are called Cluster-WSCS-based and Cluster-MWSCS-based. In view of the resultsof the previous subsection that evaluate the similaritymeasures for the clustering algorithms Cluster-WSCS-

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 9 of 14

based and Cluster-MWSCS-based, we selected asclustering technique the agglomerative algorithm withcomplete link and cos_sim_tpc(x,y) by its stability inthe results.We evaluated the conference recommendation tech-

niques using the same dataset as in the previous subsec-tion, with the 248 academic computer scienceconferences, and selected 243 random authors to predicttheir conferences ranking, for that we deleted all publi-cations of the author on the conferences that we want torank. We adopted Luong’s most frequent conferencetechnique as the benchmark (see the “Related work”section).Also recall that the mean average precision mea-

sures how good a recommendation ranking functionis. Intuitively, let a be an author and Ca be a rankedlist of conferences recommended for a. Let Sa be agold standard for a, that is, the set of conferencesconsidered to be the best ones to recommend for a.Then, we have:

� Prec@ k(Ca), the precision at position k of Ca, is thenumber of conferences in Sa that occur in Ca untilposition k, divided by k

� AveP(Ca), the average precision of Ca, is definedas the sum of Prec@ k(Ca) for each position k inthe ranking Ca in which a relevant conference fora occurs, divided by the cardinality of Sa:

AveP Cað Þ ¼X

kPrec@k Cað Þ

Saj j ð22Þ

� MAP, the Mean Average Precision of a rankscore function over all the authors used in theseexperiments (represented by set A) is then definedas follows:

MAP ¼ average AveP Cað Þ=a∈Af g ð23Þ

Moreover, in order to evaluate whether the differencesbetween the results are statistically significant, a pairedstatistical Student’s t test [27, 28] was performed. Ac-cording to Hull [29], the t test performs well even fordistributions which are not perfectly normal. A p valueis the probability that the results from the compareddata occurred by chance; thus, low p values are good.We adopted the usual threshold of α = 0.01 for statisticalsignificance, i.e., less than 1% that the experimental

Fig. 3 Rand index of the clustering algorithms

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 10 of 14

results happened by chance. When a paired t testobtained a p value less than α, there is a significantdifference between the compared techniques.Consider first the two conference recommendation

techniques based on the co-authorship network, theWSCS-based and MWSCS-based recommendation tech-niques. To compare them, we performed experimentsthat measured their runtime, accuracy, and average pre-cision of the top 10 conferences of an author (thus, inthis situation, the maximum |Sa| value used in the AvePcalculation is 10). Figure 5 shows the runtime results ofthe algorithms that implement these recommendationtechniques. Note that the WSCS-based algorithm is byfar the slowest, followed by the MWSCS-based. Theremaining algorithms had similar runtimes.Table 1 shows the accuracy and MAP of the 8 confer-

ence recommendation techniques we proposed, plus thebenchmark. Two of the proposed techniques (first tworows of Table 1) have very similar accuracy. In fact, ofthe 243 authors that we tested, the balance of the cor-rect predictions was 201 against 197. To better evaluatethe results, we applied a paired t test to investigatewhether there are statistically significant differencesbetween the MAP results of these conference

recommendation techniques. Table 2 shows the p valuesobtained by all t tests performed, where the boldface re-sults represent differences which are not statisticallysignificant.Based on these results, the three techniques with the

best results—WSCS-based, MWSCS-based, and Cluster-WSCS-based—do not have differences which are statisti-cally significant, based on their MAP results. The resultsalso indicate that these three techniques have better MAPresults than the benchmark (with statistically significantdifferences). The CF-Jaccard, CF-Communities, and theCluster-MWSCS-based techniques have results very closeto the benchmark (without statistically significant differ-ences when compared to the benchmark) but less thanthe three techniques with the best results—WSCS-based,the MWSCS-based, and Cluster-WSCS-based (with differ-ences statistically significant when compared to thesethree). The CF-Pearson and CF-Cosine techniques havepoor accuracy (with statistically significant differenceswhen compared to all other techniques).Thus, between the three techniques with the best

results—WSCS-based, MWSCS-based, and Cluster-WSCS-based—we may conclude that the Cluster-WSCS-based technique should be preferred because it is

Fig. 4 F measure with β = 1 of clustering algorithms

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 11 of 14

more efficient and maintains a MAP with no statisticallysignificant differences when compared to the WSCS-based and MWSCS-based techniques.

ConclusionsIn this article, we presented techniques to compare andrecommend conferences. The techniques to compareconferences are based on some classical similarity mea-sures and on a new similarity measure based on the co-authorship network communities of two conferences.

The experiments suggest that the best performance isobtained using the new similarity measure.We introduced three families of conference recom-

mendation techniques, following the collaborative filter-ing strategy, and based on (1) the similarity measuresproposed to compare conferences; (2) the relatedness oftwo authors in the co-authorship network, using theWeighted and the Modified Weighted Semantic Con-nectivity Scores; (3) conference clusters, using a sub-graph of the co-authorship network, instead of the fullco-authorship network. The experiments suggest thatthe WSCS-based, MWSCS-based, and Cluster-WSCS-based techniques perform better than the benchmarkand better than the techniques based on similarity mea-sures. Furthermore, between these three techniques, theCluster-WSCS-based technique should be preferredbecause it is more efficient and maintains a MAP withno statistically significant differences when compared tothe WSCS-based and MWSCS-based techniques.These conclusions should be accepted under the limi-

tations of the experiments, though, which we recalladopted a set of 248 academic computer science confer-ences as golden standard and used a random sample of243 authors. Further experiments ought to be performedwith other sets of conferences and authors, perhaps

Fig. 5 Runtime (milliseconds) of the recommendation algorithms for the different 243 authors

Table 1 Comparison of the accuracy and MAP of therecommendation techniques

Technique Accuracy (%) MAP (%)

(1) WSCS-based 82.72 80.93

(2) MWSCS-based 81.07 80.01

(3) Cluster-WSCS-based 80.66 79.83

(4) Cluster-MWSCS-based 77.78 76.82

(5) CF-Jaccard 78.19 77.73

(6) CF-Pearson 55.56 50.21

(7) CF-Cosine 56.79 51.89

(8) CF-Communities 79.02 77.93

(9) Benchmark 79.84 77.88

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 12 of 14

obtained from sources different from DPLP. However,the question of defining a golden standard remains anissue.In another direction, some of the techniques de-

scribed in the paper might be applied to other do-mains that contain essentially three types of objects,analogous to “conferences,” “papers,” and “authors”and two relationships, similar to “authored” and “pub-lished in.” One such domain would be that of “artmuseums,” “artworks,” and “artists”, with the relation-ships “created” and “exhibit in”. However, note thatthe notion of “co-authorship” would have no relevantparallel in the art domain. Again, the question offinding an appropriate data source and defining agolden standard would be an issue, which could beaddressed as in [30].A preliminary version of these results, except the tech-

niques described in the “Similarity measure based onconference keywords” and “Conference recommendationtechniques based on conference clusters” subsections,and the t test described in the “Experiments with theconference recommendation techniques” subsection,were presented in [31].As for future work, we plan to experiment with a

similarity measure based on conference keywords ex-panded to include semantic relationships between thekeyword, other than just synonymy. Also, we plan toexplore other strategies for recommending confer-ences, such as the complexity level, writing style, etc.Finally, we plan to expand the experiments to otherpublications datasets and other application domains,as already mentioned, and to make the tool and thetest datasets openly available.

Endnotes1https://words.bighugelabs.com/2https://en.wikipedia.org/wiki/List_of_computer_

science_conferences

AbbreviationsDBLP: dblp computer science bibliography; MSWCS: Modified WeightedSemantic Connectivity Score; SCS: Semantic Connectivity Score; SNA: Socialnetwork analysis; WSCS: Weighted Semantic Connectivity Score

AcknowledgementsThis work was partly funded by CNPq under grants 444976/2014-0, 303332/2013-1, 442338/2014-7, and 248743/2013-9 and by FAPERJ under grant E-26/201.337/2014.

Authors’ contributionsGMG defined the new similarity measures based on the co-authorshipnetwork, and implemented and evaluated all techniques, under thesupervision of MAC, BPN, GRL, and LAPPL. All authors read and approvedthe final manuscript.

Competing interestsThe authors declare that they have no competing interests.

Author details1Pontifical Catholic University of Rio de Janeiro, Rio de Janeiro, RJ, Brazil.2Federal University of the State of Rio de Janeiro, Rio de Janeiro, RJ, Brazil.3Federal University of Rio de Janeiro, Rio de Janeiro, RJ, Brazil. 4FederalFluminense University, Niterói, RJ, Brazil.

Received: 7 November 2016 Accepted: 15 February 2017

References1. Henry N, Goodell H, Elmqvist N, Fekete J-D (2007) 20 Years of four HCI

conferences: a visual exploration. Int’l J of Human-Comp Inter 23(3):239–285

2. Blanchard EG (2012) On the WEIRD nature of ITS/AIED conferences. In:Proceedings of the 11th Int’l. Conf. on Intelligent Tutoring Systems, Chania,Greece, 14-18 June 2012., pp 280–285

3. Chen C, Zhang J, Vogeley MS (2009) Visual analysis of scientific discoveriesand knowledge diffusion. In: Proceedings of the 12th Int’l. Conf. onScientometrics and Informetrics—ISSI 2009, Rio de Janeiro, Brazil, 14-17July 2009

4. Gasparini I, Kimura MH, Pimenta MS (2013) Visualizando 15 Anos de IHC.In: Proceedings of the 12th Brazilian Symposium on Human Factors inComputing Systems, Manaus, Brazil, 08-11 October 2013., pp 238–247

5. Barbosa SDJ, Silveira MS, Gasparini I (2016) What publications metadata tellUs about the evolution of a scientific community: the case of the Brazilianhuman-computer interaction conference series. Scientometrics, First Online

6. Chen C, Song I-Y, Zhu W (2007) Trends in conceptual modeling: citationanalysis of the ER conference papers (1979-2005). In: Proceedings of the11th Int’l. Conf. of the International Society for Scientometrics andInformatrics, Madrid, Spain, 25-27 June 2007., pp 189–200

Table 2 p values of the Student’s t test for MAP results of the recommendation techniques

(1) (2) (3) (4) (5) (6) (7) (8) (9)

(1) – 0.11590 0.02898 1.64E − 09 5.03E − 07 3.49E − 117 1.35E − 116 8.57E − 07 6.36E − 07

(2) – 0.75622 5.53E − 06 0.00023 1.43E − 106 4.98E − 108 0.00100 0.00073

(3) – 1.46E − 05 0.00105 1.44E − 112 2.68E − 108 0.00247 0.00126

(4) – 0.21574 5.40E − 88 1.16E − 86 0.09958 0.10754

(5) – 2.39E − 100 5.43E − 99 0.74710 0.80904

(6) – 0.04024 2.65E − 101 1.17E − 97

(7) – 4.46E − 100 6.32E − 99

(8) – 0.93732

(9) –

The boldface results represent differences which are not statistically significantLegend: (1) WSCS-based, (2) MWSCS-based, (3) Cluster-WSCS-based, (4) Cluster-MWSCS-based, (5) CF-Jaccard, (6) CF-Pearson, (7) CF-Cosine, (8) CF-Communities,(9) benchmark

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 13 of 14

7. Zervas P, Tsitmidelli A, Sampson DG, Chen N-S, Kinshuk (2014) StudyingResearch collaboration patterns via Co-authorship analysis in the field ofTeL: the case of educational technology & society journal. J Educ TechnoSoc 17(4):1–16

8. Procópio PS, Laender AHF, Moro MM (2011) Análise da Rede de Coautoriado Simpósio Brasileiro de Bancos de Dados. In: Proceedings of the 26thBrazilian Symposium on Databases, Florianópolis, Brazil, 3-6 Oct. 2011

9. Cheong F, Corbitt BJ (2009) A social network analysis of the Co-authorshipnetwork of the Australasian conference of information systems from 1990to 2006. In: Proceedings of the 17th European Conf. on Info. Systems,Verona, Italy, 8-10 June 2009

10. Cheong F, Corbitt BJ (2009) A social network analysis of the Co-authorshipnetwork of the pacific Asia conference on information systems from 1993to 2008. In: Proceedings of the Pacific Asia Conference on InformationSystems 2009, Hyderabad, India, 10-12 July 2009, Paper 23

11. Lopes GR, Nunes BP, Leme LAPP, Nurmikko-Fuller T, Casanova MA (2015)Knowing the past to plan for the future—an in-depth analysis of the first 10editions of the WEBIST conference. In: Proceedings of the 11th Int’l. Conf.on Web Information Systems and Technologies, Lisbon, Portugal, 20-22 May2015., pp 431–442

12. Lopes GR, Nunes BP, Leme LAPP, Nurmikko-Fuller T, Casanova MA (2016)A comprehensive analysis of the first ten editions of the WEBIST conference.Lect. Notes in Business Information Processing 246:252–274

13. Batista MGR, Lóscio BF (2013) OpenSBBD: Usando Linked Data paraPublicação de Dados Abertos sobre o SBBD. In: Proceedings of the 28thBrazilian Symposium on Databases, Recife, Brazil, 30 Sept. - 03 Oct. 2013

14. Medvet E, Bartoli A, Piccinin G (2014) Publication venue recommendationbased on paper abstract. In: Proceedings of the IEEE 26th InternationalConference on Tools with Artificial Intelligence, 10-12 Nov. 2014

15. Pham MC, Cao Y, Klamma R, Jarke M (2011) A clustering approach forcollaborative filtering recommendation using social network analysis.J Univers Comput Sci 17(4):583–604

16. Chen Z, Xia F, Jiang H, Liu H, Zhang J (2015) AVER: random walk basedacademic venue recommendation. In: Companion Proceedings of the24th International Conference on World Wide Web., pp 579–584

17. Boukhris I, Ayachi R (2014) A novel personalized academic venue hybridrecommender. In: Proceedings of the IEEE 15th International Symposiumon Computational Intelligence and Informatics, 19-21 Nov. 2014

18. Yang Z, Davison BD (2012) Venue recommendation: submitting your paperwith style. In: Proceedings of the 11th International Conference on MachineLearning and Applications, 12-15 Dec. 2012., pp 12–15

19. Huynh T, Hoang K (2012) Modeling Collaborative knowledge of publishingactivities for research recommendation. Computational collectiveintelligence. Technologies and applications. Volume 7653 of the seriesLNCS., pp 41–50

20. Asabere NY, Xia F, Wang W, Rodrigues JC, Basso F, Ma J (2014) Improvingsmart conference participation through socially aware recommendation.IEEE Trans Hum Mach Syst 44(5):689–700

21. Hornick M, Tamayo P (2012) Extending recommender systems for disjointuser/item sets: the conference recommendation problem. IEEE T KnowlData En 24(8):1478–1490

22. Luong H, Huynh T, Gauch S, Do L, Hoang K (2012) Publication venuerecommendation using author Network’s publication history. In:Proceedings of the 4th Asian Conf. on Intelligent Information andDatabase Systems - ACIIDS 2012, Kaohsiung, Taiwan, 19-21 March 2012.,pp 426–435

23. Nunes BP, Kawase R, Fetahu B, Dietze S, Casanova MA, Maynard D (2013)Interlinking documents based on semantic graphs. Procedia Comput Sci 22:231–240

24. Katz L (1953) A New Status index derived from sociometric analysis.Psychometrika 18(1):39–43

25. García GM (2016) Analyzing, comparing and recommending conferences. M.Sc. Dissertation. Department of Informatics, PUC-Rio, Rio de Janeiro, https://doi.org/10.17771/PUCRio.acad.27295

26. Leskovec J, Rajaraman A, Ullman JD (2014) Mining of Massive Datasets.Cambridge University Press, Cambridge

27. Baeza-Yates RA, Ribeiro-Neto BA (2011) Modern information retrieval—theconcepts and technology behind search, 2nd edn. Pearson Education Ltd.,Harlow, England

28. Manning CD, Raghavan P, Schütze H (2008) Introduction to InformationRetrieval. Cambridge University Press, New York

29. Hull D (1993) Using statistical testing in the evaluation of retrievalexperiments. In: Proceedings of the 16th Annual International ACMSIGIR Conference on Research and Development in InformationRetrieval, SIGIR ’93. NY, USA, ACM, New York, pp 329–338

30. Ruback L, Casanova MA, Renso C, Lucchese C (2017) SELEcTor: discoveringsimilar entities on LinkEd DaTa by ranking their features. In: Proceedings ofthe 11th IEEE International Conference on Semantic Computing, San Diego,USA, 30 Jan. - 2 Feb. 2, 2017

31. García GM, Nunes BP, Lopes GR, Casanova MA (2016) Comparing andrecommending conferences. In: Proceedings of the 5th BraSNAM—BrazilianWorkshop on Social Network Analysis and Mining, Porto Alegre, Brazil, 05July 2016

Submit your manuscript to a journal and benefi t from:

7 Convenient online submission

7 Rigorous peer review

7 Immediate publication on acceptance

7 Open access: articles freely available online

7 High visibility within the fi eld

7 Retaining the copyright to your article

Submit your next manuscript at 7 springeropen.com

García et al. Journal of the Brazilian Computer Society (2017) 23:4 Page 14 of 14