J Intell Inf SystDOI 10.1007/s10844-016-0424-5

Detecting abnormal profiles in collaborative filteringrecommender systems

Zhihai Yang1 ·Zhongmin Cai1

Received: 24 June 2015 / Revised: 4 July 2016 / Accepted: 6 July 2016© Springer Science+Business Media New York 2016

Abstract Personalization collaborative filtering recommender systems (CFRSs) are thecrucial components of popular E-commerce services. In practice, CFRSs are also partic-ularly vulnerable to “shilling” attacks or “profile injection” attacks due to their openness.The attackers can inject well-designed attack profiles into CFRSs in order to bias the rec-ommendation results to their benefits. To reduce this risk, various detection techniques havebeen proposed to detect such attacks, which use diverse features extracted from user pro-files. However, relying on limited features to improve the detection performance is difficultseemingly, since the existing features can not fully characterize the attack profiles and gen-uine profiles. In this paper, we propose a novel detection method to make recommendersystems resistant to such attacks. The existing features can be briefly summarized as twoaspects including rating behavior based and item distribution based. We firstly formulatethe problem as finding a mapping model between rating behavior and item distribution byexploiting the least-squares approximate solution. Based on the trained model, we designa detector by employing a regressor to detect such attacks. Extensive experiments on boththe MovieLens-100K and MovieLens-ml-latest-small datasets examine the effectiveness ofthe proposed detection method. Experimental results demonstrate the outperformance of theproposed approach in comparison with benchmarked method including KNN.

Keywords Recommender system · Shilling attack · Attack detection

The research is supported by the National Natural Science Foundation (61175039, 61375040),International Research Collaboration Project of Shaanxi Province (2013KW11) and FundamentalResearch Funds for Central Universities (2012jdhz08).

� Zhongmin Caizmcai@mail.xjtu.edu.cn

Zhihai Yangzhyang xjtu@sina.com

1 Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi’anJiaotong University, Xi’an, China

J Intell Inf Syst

1 Introduction

To overcome the phenomenon of information overload, recommender systems (RSs) areessential tools to guide the customers to the information they are seeking for without wast-ing time navigating irrelevant information. Personalization RSs help users to select the itemsthat they might like. They use information about user profiles to predict the relevance orutility of a particular product. Collaborative filtering is one of the most successful recom-mendation algorithm and used in different applications. Examples of collaborative filteringrecommender systems (CFRSs) including Amazon and Ringo (Burke et al. 2006; Bryanet al. 2008; Chung et al. 2013; Mehta et al. 2007). However, CFRSs are prone to manip-ulation from producers or attackers. The phenomenon is often called “shilling” attacks or“profile injection” attacks. In such attacks, the attackers deliberately insert attack profilesinto genuine profiles to bias the predict results to their benefits, which can decrease thetrustworthiness of recommendation. The attack profiles indicate the attacker’s intention thathe wishes a particular item can be rated with the highest score (called push attack) or thelowest score (called nuke attack). There are several reports about such attacks on Amazonor eBay. These events are that of a well-known company injecting malicious profiles intosystems to promote its films (Hurley et al. 2007; Jia et al. 2013). Therefore, constructing aneffective method to defend the attackers and remove them from the CFRSs is crucial.

An interesting question from this case is how to systematically and dynamically detect“shilling” attacks using existing features extracted from a large number of user profiles. Itis very important to answer this question, since detecting “shilling” attacks usually carryinteresting information of potential problems which requires real-time attention and shouldbe detected and dealt with at the early stage. However, existing detection methods showedan acceptable level of performance in a few of attack models, which is based on limitedfeatures (Burke et al. 2006; Bryan et al. 2008; Hurley et al. 2009; Mehta et al. 2007; Moridand Shajari 2013; Zhang and Zhou 2014). In addition, how to improve the detection per-formance when attack size1 and filler size2 are small is also a tricky problem (Burke et al.2006; Morid and Shajari 2013). In practice, relying on the limited features to improve thedetection performance is difficult seemingly. Hence, we either explore new and effectivefeatures to characterize user profiles or employ an advanced detection approach to improvethe detection performance.

As mentioned above, we assume that there is a correspondence relationship between theexisting features which can be roughly divided into two categories (such as rating behaviorbased and item distribution based) and it can be used to discriminate between attack profilesand genuine profiles. Compared with genuine users, attackers have different rating inten-tions on a target item. They either promote the target item with the highest rating or demotethe target item with the lowest rating. Take bandwagon attack for example, attackers selectpopular items and totally rate the highest or lowest score (in push or nuke attack), whichmay show an abnormal rating behavior. In contrast, genuine users may not totally rate thehighest score on the popular items. Based on the hypothesis, we try to find a mapping modelto characterize this relationship. In this paper, we propose a novel detection method to makerecommender systems resistant to “shilling” attacks. The objective of the proposed methodis to extract diverse features from different perspectives and construct a mapping model for

1The ratio between the number of attackers and genuine users.2The ratio between the number of items rated by user u and the number of entire items in the recommendersystem.

J Intell Inf Syst

detecting “shilling” attacks. To achieve this objective, the key challenge is how to constructa relationship based on the extracted features. To this end, we take both the features of rat-ing behavior and item distribution into consideration when designing the mapping model.By adopting the least-squares approximate solution, we train the mapping model and designa detector to detect such attacks. In addition, the effectiveness of the proposed approach isvalidated and benchmarked method are briefly discussed. Experimental results show thatour approach performs well in comparison with the benchmarked method.

The main purposes and major contributions of this paper are summarized as follows:

– 8 features are exploited to characterize the item distribution and 12 features areemployed to characterize the rating behavior for constructing a corresponding relation-ship between ratings and items.

– A novel detection approach is proposed to detect “shilling” attacks based on thepresented features, which adopts the relationship between rating behavior and itemdistribution to construct a mapping model.

– The proposed method is completely data-driven. Extensive experiments on both theMovieLens-100K and MovieLens-ml-latest-small datasets are conducted to demon-strate the effectiveness of the proposed method.

The remaining parts of this paper are organized as follows: Section 2 reviews the relatedwork on “shilling” attack detection in collaborative filtering recommender systems. Sec-tion 3 is used to introduce attack profiles and attack models. The proposed detection methodis introduced in Section 4. Experimental results are presented and discussed in Section 5.Finally, we briefly summarize our work and discuss the limitations of this paper and ourfuture work.

2 Related work

Collaborative filtering recommender systems (CFRSs) are known to be vulnerable to“shilling” attacks or “profile injection” attacks. Shilling attack is a method by construct-ing attack profiles based on knowledge of the CFRSs to achieve attacker’s intent. Publishedstudies of attack detection in CFRSs have investigated different detection methods. Burkeet al. (2006) presented several features extracted from user profiles for their utility in attackdetection. They exploited KNN classifier as their detection method. However, it was unsuc-cessful to detect attacks when filler size is small. Williams et al. (2007a, b) tried to extractfeatures from user profiles and utilized them to detect such attacks. But, they also sufferedfrom low detection accuracy and some genuine profiles are misclassified as attack pro-files. Then, He et al. (2010) incorporated the rough set theory into detect shilling attacksby means of taking features of user profiles as the condition attributes of the decision table.But, their method also faced with the low overall classification rate in some cases, especiallyfor bandwagon attack. After that, Wu et al. (2012) proposed a hybrid detection algorithmto defend shilling attacks, which combines the naive Bayesian classifiers and augmentedexpectation maximization base on several selected metrics. Unfortunately, their approachalso suffered from low F-measure (David 2011) when filler size is small. In addition, Zhangand Zhou (2012) proposed the idea of ensemble learning for improving predictive capabil-ity in attack detection. They constructed the base-classifiers (or weaker learner) with theSupport Vector Machine (SVM) approach and then integrate them to generate a high predic-tive ability learner for detection. Their proposed method exhibited better performance thansome benchmarked methods. Nevertheless, it still suffered from low precision especially

J Intell Inf Syst

when the attack size is small. And then, the same authors (Zhang and Zhou 2014) alsointroduced an online method, HHT-SVM, to detect profile injection attacks by combiningHilbert-Huang transform (HHT) and support vector machine (SVM). They created ratingseries for each user profile based on the novelty and popularity of items in order to pro-vide basic data for feature extraction. The precision of their method shows better resultsthan the benchmark, but the precision significantly decreased with the filler size increased.Previous studies showed that the detection results of “shilling” attacks are not reached atthe end and leave much to be desired, especially when the filler size or attack size is small.On the one hand, the existing features are limited so that it is difficult to fully characterizethe attack profiles and genuine profiles. On the other hand, the detection performance ofthese detection methods is limited. To get a better use of the existing features, we roughlydivide these features into two categories including rating behavior and item distribution andfind a mapping model to construct a relationship between the two categories for detectingthe “shilling” attacks, since attackers and genuine users have different rating intentions ondifferent items.

3 Attack profiles and attack models

The attackers have different attack intentions which bias the recommendation results to theirbenefits. In the literature, the shilling attacks are classified into two ways: push attack andnuke attack (Burke et al. 2006; Williams et al. 2007a, b). In push attack, attackers promotethe target items by rating the highest score, whereas in nuke attack, attackers demote thetarget items by rating the lowest score. The general form of attack profiles is showed inTable 1. The details of the four sets of items are described as follows:

IS : A set of selected items with specified rating using function σ(iSk ) (Cao et al. 2013);IF : A set of filler items, randomly received items with randomly assigned ratings using

function ρ(iFl

);

IN : A set of items with no rating;IT : A set of target items with singleton or multiple items, called single-target attack or

multi-target attack (Chung et al. 2013). The rating is γ (iTj ), generally rated the maximumor minimum value in the entire profiles.

In this paper, we introduce 8 attack models to generate attack profiles. The presentedattack models and corresponding explanations are listed in Table 2. The details of theseattack models are described as follows:

(1) AOP attack: A simple and effective strategy to obfuscate the Average attack is tochoose filler items with equal probability from the top x % of most popular itemsrather than from the entire collection of items (Seminario and Wilson 2014).

(2) Random attack: IS = φ and ρ(i) ∼ N(r, σ 2) (Hurley et al. 2009).(3) Average attack: IS = φ and ρ(i) ∼ N(ri, σi

2) (Hurley et al. 2009).

Table 1 General form of attack profiles

IT IS IF IN

iT1 ... iTj iS1 ... iSk iF1 ... iFl iN1 ... iNv

γ (iT1 ) ... γ (iTj ) σ (iS1 ) ... σ(iSk ) ρ(iF1 ) ... ρ(iFl ) Null ... Null

J Intell Inf Syst

Table2

Atta

ckmod

elssummary

Atta

ckmod

els

I SI F

I NI T

Item

sRating

Item

sRating

Push

ornuke

AOP

Null

x%

popularitems,ratin

gssetwith

norm

aldistaround

item

mean.

Null

r max

or

r min

Random

Null

Randomly

chosen

Normaldistaround

system

mean.

Null

r max

or

r min

Average

Null

Randomly

chosen

Normaldistaround

item

mean.

Null

r max

or

r min

Bandw

agon

(average)

Popularitems

r max

or

r min

Randomly

chosen

Normaldistaround

item

mean.

Null

r max

or

r min

Bandw

agon

(random)

Popularitems

r max

or

r min

Randomly

chosen

Normaldistaround

system

mean.

Null

r max

or

r min

Segm

ent

Segm

entedite

ms

r max

or

r min

Randomly

chosen

r min

or

r max

Null

r max

or

r min

Reverse

Bandw

agon

Unpopular

items

r min

or

r max

Randomly

chosen

System

mean

Null

r max

or

r min

Love/Hate

Null

Randomly

chosen

r min

or

r max

Null

r max

or

r min

J Intell Inf Syst

(4) Bandwagon (average) attack: IS contains a set of popular items. σ(i) = rmax or rmin

(push or nuke) and ρ(i) ∼ N(ri, σi2) (Zhang and Kulkarni 2013).

(5) Bandwagon (random) attack: IS contains a set of popular items, σ(i) = rmax or rmin

(push or nuke) and ρ(i) ∼ N(r, σ 2) (Zhang and Kulkarni 2013).(6) Segment attack: IS contains a set of segmented items. σ(i) = rmax or rmin (push or

nuke) and ρ(i) = rmin or rmax (push or nuke) (Gunes et al. 2012; Mobasher et al.2006).

(7) Reverse Bandwagon attack: IS contains a set of unpopular items, σ(i) = rmin or rmax

(push or nuke) and ρ(i) ∼ N(r, σ 2) (Gunes et al. 2012).(8) Love/Hate attack: IS = φ and ρ(i) = rmin or rmax (push or nuke) (Gunes et al. 2012).

4 The proposed approach

In this section, we firstly introduce the problem formulation and then describe the frame-work of the proposed method. The details of the proposed method are finally introduced.

4.1 Problem formulation

In this subsection, we respectively discuss model structure, problem statement, choice ofregressors and model identification with large amounts of data. To facilitate the discussions,we give the descriptions of notations used in this paper as shown in Table 3.

4.1.1 Model structure

From the relationship between rating and item of view, we explore a model structure toreveal the dependency relationship between rating behavior and item distribution. So, wecan write loosely

i = F(r) + ε (1)

where i is a vector of output, namely the attributes of item distribution, r is a vector ofinput, namely the attributes of rating behavior, and ε is some noise that is a consequenceof unaccounted measure attributes and turbulence. The nonlinear map F(r) describes the

Table 3 Notations anddescriptions Notation Description

ru,i The rating assigned by user u to item i (Burke et al. 2006).

ri The average rating value of item i (Burke et al. 2006).

NRi The set of users who have rated item i (Burke et al. 2006).

Nu The number of items rated by user u.

Uu The set of items rated by user u.

Pu,T Pu,T = {i ∈ Pu, such that ru,i = rmax} (Burke et al. 2006).nu The length of profile u.

n The average profile length in the system.

I The set of the entire items in the recommender system.

J Intell Inf Syst

dependency relationship between rating behavior and item distribution. In addition, thecomponents of the input vector r including 12 rating attributes (as shown in Table 4) includ-ing RDMA, WDMA and etc. The components of the output vector i include 8 attributes ofitem distribution (as shown in Table 5), such as FSTI, FSPI and etc. The goal is to solve forthe function F and use this model to detect “shilling” attacks. For rating attributes, we usethe existing rating metrics to measure rating patterns of “shilling” attacks. The presentedmetrics (see Table 4) provide a useful insight into making a distinction between attack pro-files and genuine profiles (Burke et al. 2006; Zhou et al. 2014; Li and Luo 2011; Morid andShajari 2013; Cacheda et al. 2011; Mobasher et al. 2007a). In this paper, we incorporate thetwelve rating attributes into the model structure. In the early stage, we calculate all ratingattributes for each user and normalize the results to construct the input vector.

To explore the internal relevance between rating and item, we firstly observe the distribu-tion of filler items or selected items in user profiles. We conduct a list of attack experimentsby utilizing 8 attack models (as shown in Table 2). One observation is that the selected itemscontain popular and unpopular items. In addition, from the items that rated maximum rating(for push attack) or minimum rating (for nuke attack) of view, there are three ratings (max-imum score 5, minimum score 1 and average score) that rate on the selected items or filler

Table 4 Rating attributes

Features Equations Descriptions

RDMA RDMAu =∑Nu

i=0|ru,i−ri |

NRi

NuRating Deviation from Mean Agreement(Burke et al. 2006).

WDMA WDMAu =∑Nu

i=0|ru,i−ri |

NR2i

NuWeighted Deviation from Mean Agreement(Burke et al. 2006).

WDA WDAu = ∑Nu

i=0|ru,i−ri |

NRiWeighted Degree of Agreement (Burke et al.2006).

FMD FMDu = 1|Uu|

∑|Uu|i=1

∣∣ru,i − ri∣∣ Filler mean difference (Morid and Shajari

2013).

MeanVar MeanV aru =∑

j∈Pu,F(ru,j −ru)2

|Pu,F | Mean Variance (Morid and Shajari 2013).

FMV FMVu = 1|UFu |

∑Ii∈UF

u(ru,i − ri )

2 Filler mean variance (Morid and Shajari2013).

FMTD FMT Du =∣∣∣∣

∑i∈P(u,T ) ru,i

|Pu,T | −∑

k∈Pu,Fru,k

|Pu,F |∣∣∣∣ Filler Mean Target Difference (Morid and

Shajari 2013).

TMF T MFu = maxj∈PTFj Target Model Focus attribute (Morid and

Shajari 2013).

LengthVar LengthV aru = |nu−n|∑k∈U (nk−n)2

Length Variance (Burke et al. 2006).

Entropy Entropyu = −∑rmax

r=1nru∑rmax

i=1 niu

log2nru∑rmax

i=1 niu

Entropy of user rating vector (Morid andShajari 2013).

FAC FACu =∑

i (ru,i−ri )√∑i (ru,i−ri )

2Filler average correlation (Morid and Shajari2013).

UNRAP Hv(u) =∑

i∈I (rui−rUi−ruI +rUI )2∑

i∈I (rui−ruI )2Unsupervised Retrieval of Attack Profiles(Burke et al. 2006).

J Intell Inf Syst

Table 5 Item attributes

Features Equations Descriptions

Filler Size withTotal Items (FSTI)

FST Iu =∑|I |

i=1 O(ru,i )

|I | The ratio between the number of items ratedby user u and the number of entire items inthe recommender system (Zhang and Zhou2014).

Filler Size with PopularItems (FSPI)

FSPIu =∑K

i=1 (O(ru,i )

KThe ratio between the number of popu-lar items rated by user u and the numberof entire popular items, K , in the recom-mender system (Zhang and Zhou 2014).

Filler Size with PopularItems in Itself (FSPII)

FSPIIu =∑K

i=1 (O(ru,i )∑|I |

j=1 O(ru,j )The ratio between the number of popularitems rated by user u and the number ofentire items rated by user u (Zhang andZhou 2014).

Filler Size withUnpopular Items(FSUI)

FSUIu =∑|I |

i=1 O(ru,i )

|I |−KThe ratio between the number of unpopu-lar items rated by user u and the number ofentire unpopular items in the recommendersystem (Zhang and Zhou 2014).

Filler Size withUnpopular Items inItself (FSUII)

FSUIIu =∑|I |

i=K+1 O(ru,i )∑|I |

k=1 O(ru,k )The ratio between the number of unpopu-lar items rated by user u and the numberof entire items rated by user u (Zhang andZhou 2014).

Filler size with max-imum rating in totalitems (FSMAXRTI)

FSMAXRT Iu =∑|I |

i=1 O(ru,i=rmax )

|I | The ratio between the number of items ratedby user u with maximum score and thenumber of entire items in the recommendersystem.

Filler size with min-imum rating in totalitems (FSMINRTI)

FSMINRT Iu =∑|I |

i=1 O(ru,i=rmin)

|I | The ratio between the number of items ratedby user u with minimum score and thenumber of entire items in the recommendersystem.

Filler size with aver-age rating in totalitems (FSARTI)

FSART Iu =∑|I |

i=1 O(ru,i=ravg)

|I | The ratio between the number of items ratedby user u with average score and the num-ber of entire items in the recommendersystem.

items. Based on the above mentioned, we propose 8 metrics to measure the distribution ofitems (such as FSTI, FSPI and etc. Zhang and Zhou (2014)) as shown in Table 5.

4.1.2 Problem statement

The problem consists of two phases: the first is to identify the model F ; the second is to usethe trained model to detect “shilling” attacks. These correspond to the two major blocks inFig. 1. We firstly describe our data of user profiles more formally. For an user profile, wehave a number of items that rated by the user and his ratings. An user might rate at leastone item. These items come from different item distribution, such as popular, unpopularand etc. In addition, the user has corresponding rating on each item, which constructs ratingbehavior of the user. So, the data can be represented as

{Ru, Iu}Nu=1, (2)

J Intell Inf Syst

The stage of detec�onThe stage of normalizing data

Training set(user profiles)

Test set(user profiles)

Construc�ng vectors of ra�ng

and item a�ributes

Normalize data and apply regressor Ψ

Anomaly detector Detec�on

resultsCalculate the

residuals

Vector RVector I

Vector RVector I

Normalize

Normalize

Trained model M and

empirical covariance C

CM

Fig. 1 The framework of the proposed method which consists of two stages: the stage of normalizing dataand the stage of detection

where N is the number of users, R and I are the input and output vector, respectively. Theinput-output pairs are independent and obey (1).

The first problem is to identify the nonlinear map F from the training set. We approachthis as a linear regression problem and replace (1) with

I = MΨ (R) + e, (3)

where M is a matrix that contains the model parameters, Ψ (R) is a regressor, and e denotesthe noise which combines ε in (1) and the error of the regression fit. In what follows, wewill discuss the choice of regressors.

The second problem is to use the trained model M to detect anomalies. We consider amodification of (3) and write it as

I = MΨ (R) + e + f, (4)

where f denotes the impact of the shilling attacks on the output I . Assuming that R, I

and M are known in (4), we want to determine if f is nonzero. Then we call this problemanomaly detection problem: we want to identify when there exists attack profile in an entireuser profiles given by (2). In addition, we will describe how to detect shilling attack in thefollowing section.

To be brief, the problem can be summarized as follows:

(1) Find a model that holds across a range of user profiles;(2) Use the trained model to detect shilling attacks.

4.1.3 Choice of regressors

To address our problem, we can use different functions Ψ to map the inputs and outputs. Asimple one might be the linear regressor

Ψlinear (R) = [R, 1]T , (5)

where R is the input vector. By using this regressor, the trained model M would describea linear relationship between the input R and the output I . In this paper, we also use analternative quadratic regressor that is constructed as follows:

Ψquadratic(x) = [{RjRk}j≤k, R, 1]T , (6)

where {RjRk}j≤kis the set of all distinct rating behaviors of the elements of the input

vector R. The subscript refers to the elements of R as opposed to a distinct rating behavior.

J Intell Inf Syst

Similarly, the trained model M would describe a quadratic relationship between the input Rand output I (Chu et al. 2010). Of course, we can choose other high-dimensional regressorsin our work. It’s worth noting that the choice of regressors affects the performance andinterpretation of the regression model, but not how the model is constructed.

In this paper, we conduct a list of experiments on different attack models to choose abetter regressor. As shown in Table 6, we report the predictive power (Chu et al. 2010)(as shown in (16)) of the trained model for both linear regressor and quadratic regressor indifferent attack strategies. In addition, we fix 12.5 % attack size and 5.2 % filler size in eachattack model. The results may indicate that quadratic regressor has better performance thanlinear regressor as illustrated in Table 6. Therefore, we choose the quadratic regressor as theregressor.

4.1.4 Model identification with large amounts of data

Model identification gives us an empirical model which can be used for anomaly detection.In this part, we illustrate how to train a model M from a large dataset. By using data withoutheavy turbulence (so that e is small) and without any impact of the attacks on the output I

(such that f = 0), we can find a model from the data. The solution can be constructed byfinding the least squares approximate solution for M by solving

minN∑

u=1

‖ Iu − MΨu ‖2 + λ‖ M ‖2F , (7)

where Iu and Ψu denote the output vector and the regressor, respectively. ‖ · ‖F denotes theFrobenius norm of a matrix. N denotes the number of users. The parameter λ is chosen toprevent model over-fitting and keep the coefficients in the model “small”. In addition, wecall the least-squares model obtained this way the linear model if we use a linear regressorfor the model. In the same way, a different regressor would be described similarly (such asa quadratic model) (Chu et al. 2010).

To compute quickly the solution (7) and overcome a distributed fashion for a large datasetthat do not fit into memory or store in large historical datasets, we take the derivative with

Table 6 The predictive power ofthe linear regressor and quadraticregressor in different attackmodels

Attack models Predictive power

Linear regressor Quadratic regressor

AOP 0.765 0.808

Random 0.778 0.812

Average 0.785 0.810

Bandwagon (average) 0.813 0.845

Bandwagon (random) 0.821 0.842

Segment 0.807 0.839

Reverse Bandwagon 0.793 0.831

Love/Hate 0.805 0.827

J Intell Inf Syst

respect to the matrix M in (7) and set the result equal to zero, and then we obtain the normalequations that provide the solution to (7) as follows:

∂∂M

(N∑

u=1‖ Iu − MΨu ‖2 + λ‖ M ‖2F

)= 0,

(λI +

N∑

u=1ΨuΨ

Tu

)MT =

N∑

u=1ΨuI

Tu ,

(8)

After the derivation of the formula, we can construct our model recursively and alsoeasily handle large datasets by updating the two matrices containing these running sums,

N∑

u=1

ΨuΨTu and

N∑

u=1

ΨuITu (9)

In the end, we calculate the empirical covariance for the residuals as a combination ofthe running sums

C = 1

N − 1

N∑

u=1

(Iu − MΨu)(Iu − MΨu)T

= 1

N − 1

(N∑

u=1

(IuI

Tu

)− 2M

N∑

u=1

(ΨuI

Tu

)+ M

N∑

u=1

(ΨuΨ

Tu

)MT

)

(10)

where N is the number of users, and M is our trained model which can be solved by addingthe regularization λI . Note that, we have already computed the matrices,

∑Nu=1

(ΨuI

Tu

)and

∑Nu=1

(ΨuΨ

Tu

). We also calculate the matrix

∑Nu=1

(IuI

Tu

)in a similar manner.

4.2 The framework of the proposed method

As shown in Fig. 1, our proposed algorithm consists of two stages: the stage of normalizingdata and the stage of detection. At the stage of normalizing data, we are firstly given the userprofile data, including training set and test set. To capture the essential information for ourproposed method in the early phase, we construct the vectors of rating attributes and itemattributes (see Section 4.1). And then we normalize our data and apply the regressor to thenormalized data. At the stage of detection, the training set in the normalized data is used totrain the model of classification and the empirical covariance for the purpose of detection.In addition, at the stage of detection, we train the model M by solving (7)–(11). And then,we calculate the empirical covariance for the residuals as a combination of the running sumsC by solving (10). By using the trained model M and the empirical covariance C, we firstlycalculate the residual part (as shown (11)) in test dataset. To detect anomaly profiles, we usea detector D to decide whether an user profile is anomalous (Chu et al. 2010). In this paper,we incorporate the Mahalanobis norm into the detection method. It can be summarized asa detector (as shown in (12)), which is a function of the individual residuals. The details ofdetection will be discussed in the following subsection.

Let I and Ψ denote the output vector and regressor, respectively. The proposed datanormalization algorithm is described in Algorithm 1. In this Algorithm, step 1 and step 2create respectively the vector of rating and item attributes for each user u. These attributesare described in Tables 3 and 4. Step 3 normalize the input data by utilizing (16) and applythe regressor Ψ to the normalized data. Step 4 raises a linear regression problem which isthe basis for the proposed method.

J Intell Inf Syst

4.3 Anomaly detection

To design detection method, we firstly use the trained model M and calculate the residualR in the test sets. The formula of the residual can be represented as

Resu = Iu − MΨu, (11)

where u denotes user in the test sets. The motivation for using this method inspired fromChu et al. (2010).

Based on the given residuals Res and the empirical covariance C, we could set a thresh-old T by using the norm of residual for an user profile. This is similar to calculating theHotelling H 2 statistic for all user profiles. The detector labels every user as attacker ornot by comparing the Mahalanobis norm of the residual to a threshold T . In addition, weuse the empirical covariance C to calculate the Mahalanobis norm. The detector can berepresented as

D(Resu) ={1, if ‖ ResuC

− 12 ‖2 ≥ T

0, otherwise.(12)

where C is the empirical covariance, and T is the chosen threshold.Let D denotes the detection results. The proposed detection method is described in Algo-

rithm 2. In Algorithm 2, step 1 is the process of training model. Step 2 calculates theempirical covariance C by using the trained model M . Step 3 calculates the residual using(11), which is prepared for the detector. Algorithm 2 can detect shilling attacks once thedetector is generated.

J Intell Inf Syst

5 Experiments and analysis

In this section, we firstly introduce the experimental data and settings on two real-worlddatasets. Then we describe attack profiles and evaluation metrics. In addition, a list of exper-iments on the two datasets are conducted to demonstrate the effectiveness of the proposedmethod. Finally, experimental results are briefly discussed.

5.1 Experimental data and settings

In our experiments, we use both the MovieLens-100K3 and MovieLens-ml-latest-small4

datasets as the datasets describing the behaviors of genuine users in recommender systems.MovieLens datasets were collected by the GroupLens Research Project at the University ofMinnesota. The two datasets are the most popular datasets used by researchers and devel-opers in the field of collaborative filtering and attack detection in recommender systems.The MovieLens-100K dataset consists of 100,000 ratings on 1,682 movies by 943 ratersand each rater had to rate at least 20 movies. All ratings are in the form of integral valuesbetween minimum value 1 and maximum value 5. The minimum score means the rater dis-tastes the movie, while the maximum score means the rater enjoyed the movie. According tothe information derived fromMovieLens website, the sparse ratio5 of the rating data approx-imates to 93.7 % and the average rating of all users is around 3.53. Besides, the AverageNumber of Items Rated (ANIR) by each user is approximately 7 %. For the MovieLens-ml-latest-small dataset, it describes 5-star rating and free-text tagging activity from a movierecommendation service. It contains 100,023 ratings and 2,488 tag applications across 8,570movies. These data were created by 706 users between April 02, 1996 and March 30, 2015.This dataset was generated on April 01, 2015. Users were selected at random for inclusion.All selected users had rated at least 20 movies.

In our attack experiments, attack profiles are created according to different attack mod-els as shown in Table 2. The attack profiles indicate the attackers intention that he wishesa particular item can be rated the highest or the lowest. In this paper, we mainly detectthe nuke attacks. Of course, our proposed approach can be used to detect the push attacks.Take MovieLens-100K dataset for example, we use a separate training and test set by par-titioning the MovieLens-100K dataset in half. The first half was used to construct trainingdataset (470 genuine users) for training classifier. For each test the second half of the datawas injected with attack profiles and then run through the classifier that had been built onthe augmented first half. For each attack model, we generate samples of nuke attack pro-files according to the corresponding attack model with different attack sizes {2.5 %, 7.5 %,12.5 %, 17.5 %, 22.5 %, 27.5 %, 32.5 %, 37.5 %, 42.5 %, 47.5 %} and filler sizes {1.3 %,3.2 %, 5.2 %, 7.1 %, 9.1 %, 11 %, 13 %}. To ensure the rationality of the results, the targetitem is randomly selected for each attack profile. In addition, we randomly select 10 moviesas the popular movies which are rated by more than 200 users in the system. For reversebandwagon attack, we randomly choose 10 movies as the selected movies which are ratedby one user in the system. And for segment attack, we select 5 top popular items as thesegmented items which are rated with the highest number of users.

3http://grouplens.org/datasets/movielens/.4http://grouplens.org/datasets/movielens/.5The ratio between the number of ratings and entire ratings in the rating matrix.

J Intell Inf Syst

The training data was created by inserting a mix of attack profiles which combine diverseattack profiles (8 attack models) with different filler sizes {1.3 %, 3.2 %, 5.2 %, 7.1 %,9.1 %, 11 %, 13 %}. To balance the proportion between genuine profiles and attack profilesin the training set, we construct 60 attack profiles (attack size is 12.5 %) for each of thementioned attack models corresponding to the filler sizes. For test datasets, the second halfof data was injected attack profiles with diverse filler sizes {1.3 %, 3.2 %, 5.2 %, 7.1 %,9.1 %, 11 %, 13 %} and attack sizes {2.5 %, 7.5 %, 12.5 %, 17.5 %, 22.5 %, 27.5 %,32.5 %, 37.5 %, 42.5 %, 47.5 %} for each attack model, respectively. Therefore, we have560 (8 × 10 × 7) test datasets including 8 attack models, 10 different attack sizes and 7different filler sizes. For the MovieLens-ml-latest-small dataset, we generate attack profilesin the same way. These processes are repeated 10 times and the average values of detectionresults are reported for the experiments. In addition, we should not forget that algorithms arestrongly dependent on the characteristics of the dataset, so the results obtained in the studymay not coincide with those obtained with data from other domains. All numerical studiesare implemented using MATLAB R2012a on a personal computer with Intel(R) Core(TM)i7-4790 3.60GHz CPU, 16GB memory and Microsoft windows 7 operating system.

To measure the effectiveness of the proposed detection methods, we use detection rateand false alarm rate in the paper.

detection rate = |D ∩ A||A| (13)

f alse alarm rate = |D ∩ G||G| . (14)

where D is the set of the detected user profiles, A is the set of attacker profiles, and G is theset of genuine user profiles (Zou and Fekri 2013; Mobasher et al. 2007b).

To avoid numerical problems and neutralize the effect of large variables (or attributes),we normalize our data and apply the regressor Ψ to the normalized data. Especially for thevector of rating attributes, the input vector consists of 12 attributes which have differentmeasurement scales. We use the formula as follows:

Ψu = Ψ

(2

(Ru − Rmin

Rmax − Rmin

)− 1

)(15)

where Rmin and Rmax are the minimum and maximum value in the rating attributes (input),respectively. In addition, we also normalize the attributes of item distribution (output) in thesame way (but do not apply the regressor). The normalization forces most of raw data to liein [-1, 1] and prevents large numbers from dominating (Chu et al. 2010).

To measure how good the trained model is, we want to know how of the output variationthe model explains. We calculate the predictive power of the model using following formula

pp = 1 −∑N

u=1 ‖ Iu − MΨu ‖2∑N

u=1 ‖ Iu ‖2 . (16)

Note that if pp = 1, then the numerator must have been 0. This only occurs if MΨu is aprefect predictor of Iu. If pp = 0, then we know that MΨu explains none of the variance inIu (Chu et al. 2010).

5.2 Experimental results and analysis

To validate the effectiveness of the proposed detection method in comparison with a bench-marked method KNN, we conduct a list of experiments on both MovieLens-100K and

J Intell Inf Syst

(a) (b)

0 % 5 % 10% 15% 20% 25% 30% 35% 40% 45% 50%0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te/F

alse

ala

rm ra

te

Attack size

Ours [Detection rate]KNN [Detection rate]Ours [False alarm rate]KNN [False alarm rate]

0 % 5 % 10% 15% 20% 25% 30% 35% 40% 45% 50%0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te/F

alse

ala

rm ra

te

Attack size

Ours [Detection rate]KNN [Detection rate]Ours [False alarm rate]KNN [False alarm rate]

Fig. 2 The comparison of detection rate and false alarm rate when filler size 5.2 % and attack size varies. aAverage attack. b Bandwagon (random) attack

MovieLens-ml-latest-small datasets in each attack model with diverse filler sizes {1.3 %,3.2 %, 5.2 %, 7.1 %, 9.1 %, 11 %, 13 %} and attack sizes {2.5 %, 7.5 %, 12.5 %, 17.5 %,22.5 %, 27.5 %, 32.5 %, 37.5 %, 42.5 %, 47.5 %}. For KNN classifier, we utilize all 20 fea-tures extracted from user profiles including attack profiles and genuine profiles (as shownin Tables 4 and 5). Based on the training data described in Section 5.1, KNN with k = 9 wasused to make a binary profile classifier with 0 if classified as authentic and 1 if classified asattack. Classification results and KNN classifier were created using Waka.6

To compare the detection performance of our proposed method with the benchmarkedmethod, we conduct a list of experiments with different attack sizes and filler sizes in twodifferent attack models (average and bandwagon (random) attacks are taken for examples).As shown in Fig. 2, by fixing 5.2 % filler size, the curves of detection performance showa relatively stable trend and perform no change obviously with the attack size increased inthe later stage (attack size>22.5 %) as illustrated in Fig. 2a and b. Notice that, the detec-tion performance significantly outperforms KNN in both average and bandwagon (random)attacks. By fixing 7.5 % attack size, the detection performance of the proposed method alsosignificantly outperforms KNN with the filler size increasing as illustrated in Fig. 3. Theresults may indicate that the proposed method is more effective to use the existing featuresthan KNN. In some cases, more features used for a classifier may cause bad results (thecurves of KNN as shown in Fig. 3).

To compare the detection performance of the proposed method in both nuke and pushattacks, we plot the receiver operating curves (ROC) in Fig. 4. Take bandwagon (random)attack for example, the curves show the dependence of the detection rate (13) and falsealarm rate (14) on the threshold T in (12). The curves were empirically obtained using thetest set data. In Fig. 4, we can observe that the detection achieves 91.2 % detection rate whenthe false alarm rate achieves 11.7 % in nuke attack. For the push attack, the curve showssimilarly phenomenon. In summary, the proposed method performs well by sacrificing falsealarm rate to improve detection rate as illustrated in Fig. 4. Our goal is to identify theeffectiveness of the proposed method in different attack intentions (nuke attack and pushattack).

6http://www.cs.waikato.ac.nz/∼ml/weka/.

J Intell Inf Syst

(a) (b)

1 % 2 % 3 % 4 % 5 % 6 % 7 % 8 % 9 % 10%0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te/F

alse

ala

rm ra

te

Filler size

Ours [Detection rate]KNN [Detection rate]Ours [False alarm rate]KNN [False alarm rate]

1 % 2 % 3 % 4 % 5 % 6 % 7 % 8 % 9 % 10%0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te/F

alse

ala

rm ra

te

Filler size

Ours [Detection rate]KNN [Detection rate]Ours [False alarm rate]KNN [False alarm rate]

Fig. 3 The comparison of detection rate and false alarm rate when attack size is 7.5 % and filler size varies.a Average attack. b Bandwagon (random) attack

To further evaluate the effectiveness of the proposed method, we also conduct a seriesof experiments by utilizing other 6 attacks to show the performance curves of the proposedmethod just as Figs. 5 and 6 illustrated. From these results, we can observe that the pro-posed method can partly detect diverse attacks. Compared with Figs. 5 and 6, the detectionrate increased more obvious with attack size varies (see Fig. 5) than filler size varies (seeFig. 6). The second observation is that the detection rate ultimately achieves the highestvalue with the attack size increasing. Nevertheless, the detection rate shows low value whenattack size is small. The results may indicate that it is difficult to fully solve the imbalancedclassification by the proposed method. In addition, the detection rate gradually increasedand achieved a stable rate with the filler size increasing. These results may indicate that thepresented features are also limited for the proposed method to detect shilling attacks.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te

False alarm rate

Push attackNuke attack

Fig. 4 ROC curves for the proposed method in nuke and push bandwagon (random) attack when attack sizeis 12.5 % and filler size is 5.2 %

J Intell Inf Syst

(a) (b)

0 % 5 % 10% 15% 20% 25% 30% 35% 40% 45% 50%0.8

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

1

Det

ectio

n ra

te

Attack size

AOPRandomBandwagon (average)SegmentReverse BandwagonLove/Hate

0 % 5 % 10% 15% 20% 25% 30% 35% 40% 45% 50%0

0.02

0.04

0.06

0.08

0.1

0.12

Fals

e al

arm

rate

Attack size

AOPRandomBandwagon (average)SegmentReverse BandwagonLove/Hate

Fig. 5 The comparison of detection rate and false alarm rate in diverse attack models when filler size is5.2 % and attack size varies. a Detection rate. b False alarm rate

To evaluate the detection performance of the proposed method on different datasets,we conduct extensive experiments on both the MovieLens-100K and MovieLens-ml-latest-small datasets. As shown in Fig. 7, the proposed method shows effective detectionperformance on both the two datasets regardless of the different filler sizes or attack sizes.Figure 7a combines the detection rate and false alarm rate when the filler size is 5.2 % andattack size varies. Figure 7b combines the detection rate and the false alarm rate when attacksize is 7.5 % and filler size varies. We can find that the detection rates increase very fastand achieve a stable level along with an increase in attack size or filler size. In Fig. 7a, thedetection rates reach close to 100 % when attack size reaches 22.5 %. The false alarm rates,on the other hand, consistently stay below 10 % when the attack size gradually increasing.Similarly, in Fig. 7b, the detection rates reach close to 90 % when filler size reaches 5.2 %and the false alarm rates also stay below 10 % regardless of diverse filler sizes.

(a) (b)

1 % 2 % 3 % 4 % 5 % 6 % 7 % 8 % 9 % 10%0.7

0.75

0.8

0.85

0.9

0.95

1

Det

ectio

n ra

te

Filler size

AOPRandomBandwagon (average)SegmentReverse BandwagonLove/Hate

1 % 2 % 3 % 4 % 5 % 6 % 7 % 8 % 9 % 10%0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

0.11

Fals

e al

arm

rate

Filler size

AOPRandomBandwagon (average)SegmentReverse BandwagonLove/Hate

Fig. 6 The comparison of detection rate and false alarm rate in diverse attack models when attack size is7.5 % and filler size varies. a Detection rate. b False alarm rate

J Intell Inf Syst

(a) (b)

0 % 5 % 10% 15% 20% 25% 30% 35% 40% 45% 50%0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te/F

alse

ala

rm ra

te

Attack size

MovieLens−100K [detection rate]MovieLens−ml−latest−small [detection rate]MovieLens−100K [false alarm rate]MovieLens−ml−latest−small [false alarm rate]

1 % 2 % 3 % 4 % 5 % 6 % 7 % 8 % 9 % 10%0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Det

ectio

n ra

te/F

alse

ala

rm ra

te

Filler size

MovieLens−100K [detection rate]MovieLens−ml−latest−small [detection rate]MovieLens−100K [false alarm rate]MovieLens−ml−latest−small [false alarm rate]

Fig. 7 The comparison of detection rate and false alarm rate in different datasets. All of them are nukeattacks. a The filler size is 5.2 % and attack size varies in random attack. b The attack size is 7.5 % and fillersize varies in random attack

5.3 Discussion

By summarizing all experimental results, we can briefly describe the following aspects fordiscussing the effectiveness of the proposed method:

– Although the detection performance of the proposed method can not reach a full levelwhen the attack size or filler size is small, the detection results demonstrate the outper-formance of the proposed method as compare to baseline. Furthermore, the extractedfeatures can not totally characterize user profiles is also a main reason.

– Similarity based feature such as DegSim (Burke et al. 2006; Zhou et al. 2014) is usefulto capture attack profiles, but the time consumption of calculating similarity betweenusers or items is a big challenge especially for large-scale datasets.

– The trained mapping relationship between rating and item partly relies on the extractedfeatures, therefore, the final prediction results may be affected.

6 Conclusion and future work

“Shilling” attacks or “profile injection” attacks are serious threats to the collaborativefiltering recommender systems (CFRSs). Since the existing features are unsuccessful tocharacterize the attack profiles and genuine profiles in some cases or published classifica-tion methods for detecting shilling attack are more relying on the existing features. In thispaper, we provided a novel detection approach for detecting shilling attacks, which con-structs a mapping model by exploiting the relationship between rating behavior and itemdistribution. Extensive experiments on both theMovieLens-100K andMovieLens-ml-latest-small datasets demonstrated the effectiveness of the proposed approach. Compared with thebenchmarked method, the proposed method shows more optimistic detection performance.

One of the limitations of the proposed method directly comes from new types of attacksincrementally, since they are generated over time in the context of real CFRSs. Besides,the detection performance of the proposed method directly depends on the attack types andfeature extraction in the training stage. In our future work, we intend to extend and improve

J Intell Inf Syst

attack detection in the following directions: (1) Considering more attack models such asPower user attack and Power item attack (Seminario and Wilson 2014; Seminario 2013);(2) Developing a theoretical grounded method for the detection problem is also of interest;(3) Extracting more simple and effective features from user profiles is also an open issue.

Acknowledgment The research is supported by the National Natural Science Foundation (61175039,61375040), International Research Collaboration Project of Shaanxi Province (2013KW11) and Fundamen-tal Research Funds for Central Universities (2012jdhz08). One anonymous reviewer has carefully read thispaper and has provided to us numerous constructive suggestions. As a result, the overall quality of the paperhas been noticeably enhanced, to which we feel much indebted and are grateful.

Compliance with Ethical Standards

Disclosure of potential conflicts of interest The authors declare that they have no competing interests.

Research involving Human Participants and/or Animals All two authors contributed equally to thiswork. They all read and approved the final version of the manuscript.

Informed consent School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an,Shaanxi 710049, P.R. China.

References

Bryan, K., OMahony, M., & Cunningham, P. (2008). Unsupervised retrieval of attack profiles in collaborativerecommender systems. In ACM conference on recommender systems (pp. 155–162).

Burke, R., Mobasher, B., & Williams, C. (2006). Classification features for attack detection in collaborativerecommender systems. In International conference on knowledge discovery and data mining (pp. 17–20).

Cacheda, F., Carneiro, V., Fernandez, D., & Formoso, V. (2011). Comparison of collaborative filtering algo-rithms: limitations of current techniques and proposals for scalable, high-performance recommendersystems. ACM Transactions on the Web (TWEB), 5(1).

Cao, J., Wu, Z., Mao, B., & Zhang, Y. (2013). Shilling attack detection utilizing semi-supervised learningmethod for collaborative recommender system.World Wide Web, 16, 729–748.

Chu, E., Gorinevsky, D., & Boyd, S. (2010). Detecting aircraft performance anomalies from cruise flightdata. In AIAA infotech aerospace conference (pp. 20–22).

Chung, C., Hsu, P., & Huang, S. (2013). βP : a novel approach to filter out malicious rating profiles fromrecommender systems. Journal of Decision Support Systems, 55(1), 314–325.

David, M.W. (2011). Evaluation: from precision, recall and f-measure to roc, informedness, markednesscorrelation. Journal of Machine Learning Technologies.

Gunes, I., Kaleli, C., Bilge, A., & Polat, H. (2012). Shilling attacks against recommender systems: acomprehensive survey. Artificial Intelligence Review, 42(4), 1–33.

He, F., Wang, X., & Liu, B. (2010). Attack detection by rough set theory in recommendation system. InIEEE international conference on granular computing (pp. 692–695).

Hurley, N., OMahony, M., & Silvestre, G. (2007). Attacking recommender systems: a cost-benefit analysis.IEEE Intelligent Systems, 64–68.

Hurley, N., Cheng, Z., & Zhang, M. (2009). Statistical attack detection. In Proceedings of the 3rd ACMconference on recommender systems (RecSys’09) (pp. 149–156).

Jia, D., Zhang, F., & Liu, S. (2013). A robust collaborative filtering recommendation algorithm based onmultidimensional trust model. Journal of Software, 8(1).

Li, C., & Luo, Z. (2011). Detection of shilling attacks in collaborative filtering recommender systems. InInternational conference of soft computing and pattern recognition (pp. 190–193).

Mehta, B., Hofmann, T., & Fankhauser, P. (2007). Lies and propaganda: detecting spam users in collaborativefiltering. In Proceedings of the 12th international conference on intelligent user interfaces (IUI’07)(pp. 14–21).

J Intell Inf Syst

Mobasher, B., Burke, R., & Sandvig, J. (2006). Model-based collaborative filtering as a defense againstprofile injection attacks. In American Association for Artificial Intelligence.

Mobasher, B., Burke, R., Bhaumil, B., & Williams, C. (2007a). Towards trustworthy recommender systems:an analysis of attack models and algorithm robustness. ACM Transactions on Internet Technology, 7(4),23–38.

Mobasher, B., Burke, R., Bhaumik, R., & Williams, C. (2007b). Toward trustworthy recommender sys-tems: an analysis of attack models and algorithm robustness. ACM Transactions on Internet Technology(TOIT), 7(4), 38.

Morid, M., & Shajari, M. (2013). Defending recommender systems by influence analysis. InformationRetrieval, 17(2), 137–152.

Seminario, C.E. (2013). Accuracy and robustness impacts of power user attacks on collaborative recom-mender systems. In ACM conference on recommender systems.

Seminario, C.E., & Wilson, D.C. (2014). Attacking item-based recommender systems with power items. InACM conference on recommender systems (pp. 57–64).

Williams, C.A., Mobasher, B., & Burke, R. (2007a). Defending recommender systems: detection of profileinjection attacks. SOCA, 1(3), 157–170.

Williams, C.A., Mobasher, B., Burke, R., & Bhaumik, R. (2007b). Detecting profile injection attacks in col-laborative filtering: a classification-based approach. In Advances in web mining and web usage analysis(pp. 167–186).

Wu, Z.A., Wu, J.J., Cao, J., & Tao, D.C. (2012). HySAD: a semi-supervised hybrid shilling attack detectorfor trustworthy product recommendation. In ACM SIGKDD conference on knowledge discovery anddata mining (pp. 985–993).

Zhang, Z., & Kulkarni, S. (2013). Graph-based detection of shilling attacks in recommender systems. InIEEE international workshop on machine learning for signal processing (pp. 1–6).

Zhang, F., & Zhou, Q. (2012). A meta-learning-based approach for detecting profile injection attacks incollaborative recommender systems. Journal on Computing, 7(1), 226–234.

Zhang, F., & Zhou, Q. (2014). HHT-SVM: an online method for detecting profile injection attacks incollaborative recommender systems. Knowledge-Based Systems, 65, 96–105.

Zhou, W., Koh, Y.S., Wen, J.H., Burki, S., & Dobbie, G. (2014). Detection of abnormal profiles on groupattacks in recommender systems. In Proceedings of the 37th international ACM SIGIR conference onResearch on development in information retrieval (pp. 955–958).

Zou, J., & Fekri, F. (2013). A belief propagation approach for detecting shilling attacks in collaborativefiltering. In Proceedings of the 22nd ACM international conference on conference on information &knowledge management (CIKM) (pp. 1837–1840).