1-HKUST: Object Detection in ILSVRC 2014∗

Cewu Lu† Hao Chen∗‡ Qifeng Chen∗§ Hei Law∗† Yao Xiao∗ Chi-Keung Tang††Hong Kong University of Science and Technology

‡Chinese University of Hong Kong§Stanford University

Abstract

The Imagenet Large Scale Visual Recognition Challenge(ILSVRC) is the one of the most important big data chal-lenges to date. We participated in the object detection trackof ILSVRC 2014 and received the fourth place among the 38teams. We introduce in our object detection system a num-ber of novel techniques in localization and recognition. Forlocalization, initial candidate proposals are generated us-ing selective search, and a novel bounding boxes regressionmethod is used for better object localization. For recogni-tion, to represent a candidate proposal, we adopt three fea-tures, namely, RCNN feature, IFV feature, and DPM fea-ture. Given these features, category-specific combinationfunctions are learned to improve the object recognition rate.In addition, object context in the form of background priorsand object interaction priors are learned and applied in oursystem. Our ILSVRC 2014 results are reported alongsidewith the results of other participating teams.

Keywords: Object Detection, Deep learning and ILSVRCchallenge.

1. Introduction

We present our system for the ILSVRC 2014 competi-tion. The big data challenge has evolved over the past fewyears as one for the most important forums for researchersto exchange their ideas, benchmark their systems, and push-ing breakthrough in categorical object recognition in large-scale image classification and object detection.

The 1-HKUST team made its debut in this year’s com-petition and we focused on the object detection track. Theobject detection problem can be divided into two sub-problems, namely, localization and recognition. Localiza-tion solves the “where” problem, while recognition solvesthe “what” problem. That is, we locate where the objectsare, and then recognize which object categories the detected

∗indicates equal contribution; all the authors are united under 1-HKUST although some of their current affiliations are elsewhere.

Figure 1. Our framework.

objects should belong to.We made technical contributions on both localization

and recognition. For localization, we exploit regression onbounding box using deep learning based on selective searchoutputs. For recognition, we focus on integrating the state-of-the-art computer vision techniques to build a more pow-erful category-specific category predictor. In addition, ob-ject background priors are also considered.

2. Framework

Figure 1 gives the overview of our system which sum-marizes our contributions in object localization and recog-nition.

2.1. Localization

We first extract candidate objectness proposals using se-lective search. As widely known, the output boundingboxes are almost never perfect and fail to coincide theground-truth object boxes with a high overlap rate (e.g.80%). To cope with this problem, we learn a regressor usingdeep learning.

1

2.2. Recognition

Given a set of candidate proposals in hand, we extractdifferent types of feature representation for recognition. Weadopt three types of feature, namely, CNN feature, DPMfeather and IFV feature, to measure the given candidate pro-posals.

For CNN feature, we first train the CNN mode, and theCNN features are set as the parameters of the sixth layer.We apply the SVM training to obtain 200 object categorySVM models, as similarly done in RCNN [2]. For DPMfeature [1] we also train 200 DPM models. For IFV fea-ture [3], we make use of the fast IFV feature extraction so-lution [5] to compute at a rate of 20 seconds per image. Wealso train 200 SVM category models as similarly done forthe above two features. After obtaining 200 CNN scores,200 DPM scores, and 200 IFV scores, these scores are con-catenated into a 600-dimensional feature vector. Finally, wetrain a 200-class SVM model on these 600D features.

2.3. Background Prior

Objects occur in context and are part of the scene. Back-ground scene understanding can definitely benefit object de-tection. The background can reject (or re-score) unreason-able objects. For example, a yacht does not appear in anindoor environment with high probability. In our imple-mentation, we train a presence prior model (PPM) under theDCNN framework on the object detection data of ILSVRC2014. Rather than producing a single label per image, thismethod outputs multiple labels for an image. Thus, falsepredictions can be removed if the evaluation based on ourtrained presence prior falls below a confidence threshold.Our experimental results show that the presence prior canhelp to filter false predictions where more context informa-tion is considered. Typical mAP improvement ranges 1–3%, with more improvement reported when the input imagecontains less object classes (e.g., 1–3).

3. Results

We discuss the performance of our entries in ILSVRC2014. In the object detection track, there are two sub-tracks: with and without extra training data. Our resultswere achieved without extra training data. We were rankedfourth in terms of number of winning categories. Table 1tabulates the top winners and we refer readers to [4] or theofficial website of ILSVRC 2014 for complete standings.Our mAP is 0.289. By analyzing the per-class results1, wefound that 1-HKUST is still ranked fourth among all theteams using and without using extra training data. Table 2shows that using extra training data gives a clear advantage.

1http://image-net.org/challenges/LSVRC/2014/results/ilsvrc2014_perclass_results.zip

Team name Number of object mAPcategories won

NUS 106 0.372MSRA 45 0.351

UvA-Euvision 21 0.3201-HKUST 18 0.289

Southeast-CASIA 4 0.304CASIA-CRIPAC-2 0 0.286

Table 1. Number of object categories won without extra trainingdata.

Team name Number of objectcategories won

GoogLeNet 138CUHK-DeepID-Net 28

Deep-Insight 271-HKUST (run 2) 3Berkeley-Vision 1

NUS 1UvA-Euvision 1

MSRA-Visual-Computing 0MPG-UT 0

ORANGE-BUPT 0Trimps-Soushen 0

MIL 0Southeast-CASIA 0CASIA-CRIPAC-2 0

Table 2. Number of object categories won with and without extratraining data. 1-HKUST did not use extra training data.

Sample visual results are demonstrated in Figure 2. Sur-prisingly, a number of difficult cases for human detectionsuch as the lizard in Figure3 can be reliably detected by oursystem.

Unlike other participating teams, 1-HKUST had verylimited computing budget and resources in our training andexperiments: one 24-core server PC (Dell PowerEdge R7202 x 12C CPU, 128GB RDIMM memory and one NVIDIAGRID K1 GPU), and one 6-core PC (Dell Alienware Aurora4.1Ghz, 6C CPU, 32GB DDR2 memory and one NVIDIAGeForce GTX 690 GPU). Due to limited computing re-sources, the parameter tuning might not have been opti-mized, and so we strongly suspect that our framework couldhave achieved a better mAP rating if we had more sufficientcomputing resources.

References

[1] P. F. Felzenszwalb, R. B. Girshick, D. McAllester, andD. Ramanan. Object detection with discriminativelytrained part-based models. TPAMI, 32(9):1627–1645,

airplane

armadillo

car

coffee maker

person

ping pong

Figure 2. Sample object detection results.

2010. 2

[2] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Richfeature hierarchies for accurate object detection and se-mantic segmentation. In CVPR, 2014. 2

[3] F. Perronnin, J. Sanchez, and T. Mensink. Improvingthe fisher kernel for large-scale image classification. InECCV’10, pages 143–156, 2010. 2

[4] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bern-stein, A. C. Berg, and L. Fei-Fei. Imagenet large scalevisual recognition challenge, 2014. 2

[5] K. E. A. van de Sande, C. G. M. Snoek, and A. W. M.Smeulders. Fisher and vlad with flair. In CVPR, June2014. 2

(a) (b)

Figure 3. (a) is a input image, (b) is our detection result. Somepeople found it difficult to recognize a lizard on pebbles.