Embed Size (px)
Citation preview
In the paper Figure 7, we claimed: “the best value for R depends on the amount of training data available.” Here are the results for Gun-Point Dataset and another dataset, Two_ Pat, which we randomly pick half size instances repeatedly. The observation is that with fewer objects in the dataset, the accuracy decreases and peaks at larger window size.
0 10 20 30 40 50 60 70 80 90 10060
65
70
75
80
85
90
95
100
Warping Window r(%)
Acc
ura
cy(%
)
6 instances
100 instances
50 instances
24 instances
12 instances
0 10 20 30 40 50 60 70 80 90 10060
65
70
75
80
85
90
95
100
Warping Window r(%)
Acc
ura
cy(%
)
6 instances
100 instances
50 instances
24 instances
12 instances
6 instances
100 instances
50 instances
24 instances
12 instances
Gun Point
0 10 20 30 40 50 60 70 80 90 10020
30
40
50
60
70
80
90
100
Warping Window Size(%)
Acc
urac
y(%
)
500 instances
120 instances
30 instances14 instances
6 instances
Two_Pat
Name # class # features # instances Evaluation Data type
JF 2 2 20,000 2,000/18,000 real
Letter 26 16 20,000 5,000/15,000 mixed
Pen Digits 10 16 10,992 7,494/3,498 real
Forest Cover Type 7 54 581,012 11,340/569,672 real
Iris 3 4 150 10-fold CV real
Ionosphere 2 34 351 10-fold CV real
Voting Records 2 16 435 10-fold CV Boolean
Australian Credit 2 14 690 10-fold CV 6 numerial/8 categorical
German Credit 2 24 1,000 10-fold CV real
Leaf 6 150 442 200/242 time series
Two_Pat 4 128 5,000 1,000/4,000 time series
Face 16 131 2,231 1,113/1,118 time series
In the paper Table 4, we list the datasets used in the paper, here we present additional datasets and show all the experiments that could not fit in the paper due the limit of space.
0 200 400 600 800 1000 1200 1400 1600 1800 2000
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 200 400 600 800 1000 1200 1400 1600 1800 2000
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
JF, 2 classed, 20,000 instances, 2,000/18,000
50000 500 1000 1500 2000 2500 3000 3500 4000 4500
20
30
40
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
Letter, 26 classes, 20,000 instances, 5,000/15,000
50000 500 1000 1500 2000 2500 3000 3500 4000 4500
20
30
40
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
0 1000 2000 3000 4000 5000 6000 7000
80
90
100
Number of instances seen before interruption, S
accu
racy
(%) Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 1000 2000 3000 4000 5000 6000 7000
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
Pen digits, 10 classed, 10,992 instances, 7,494/3,498
0 2000 4000 6000 8000 10000
30
40
50
60
70
80
90
Number of instances seen before interruption, S
acc
ura
cy(%
)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 2000 4000 6000 8000 10000
30
40
50
60
70
Number of instances seen before interruption, S
acc
ura
cy(%
)
Random Test
SimpleRank Test
Forest Cover Type, 7 classes, 581,012 instances, 11,340/569,672
0 100 200 300 400 500 600
70
80
90
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
Australian Credit, 2 classes, 690 instances, 10-fold Cross Validation
0 100 200 300 400 500 600
70
80
90
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
Australian CreditDataset
0 100 200 300 400 500 600
70
80
90
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
Random Train
Random Test
SimpleRank Train
SimpleRank Test
Australian CreditDataset
Number of instances seen before interruption, S
Random Train
Random Test
SimpleRank Train
SimpleRank Test
BestDrop Test
BestDrop Train
0 100 200 300 400 500 600
70
80
90
accu
racy
(%)
Australian CreditDataset
0 100 200 300 400 500 600
70
80
90
accu
racy
(%)
Australian CreditDataset
0 100 200 300 400 500 60040
50
60
70
80
90
100
data instances
acc
ura
cy(%
)
RandomTrainRandomTestSimpleRankTrainSimpleRankTestDROP1DROP2DROP3
0 50 100 150 200 250 300
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 50 100 150 200 250 300
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
0 50 100 150 200 250 300
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
BestDrop Test
0 50 100 150 200 250 30040
50
60
70
80
90
100
data instances
acc
ura
cy(%
)
RandomTrain
RandomTestSimpleRankTrain
SimpleRankTest
DROP1
DROP2DROP3
Ionosphere, 2 classes, 351 instances, 10-fold Cross Validation
Iris, 3 classes, 150 instances, 10-fold Cross Validation
0 50 100 150 200 250 300
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 50 100 150 200 250 300
50
60
70
80
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
0 20 40 60 80 100 12040
50
60
70
80
90
100
data instances
acc
ura
cy(%
)
RandomTrainRandomTestSimpleRankTrainSimpleRankTestDROP1DROP2DROP3
0 50 100 150 200 250 300 350
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 50 100 150 200 250 300 350
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
Voting records
0 50 100 150 200 250 300 350
90
100
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
BestDrop Test
0 50 100 150 200 250 300 35040
50
60
70
80
90
100
data instances
acc
ura
cy(%
)
RandomTrain
RandomTestSimpleRankTrain
SimpleRankTest
DROP1
DROP2DROP3
0 100 200 300 400 500 600 700 800 900
60
70
Number of instances seen before interruption, S
accu
racy
(%)
Random Train
Random Test
SimpleRank Train
SimpleRank Test
0 100 200 300 400 500 600 700 800 900
60
70
Number of instances seen before interruption, S
accu
racy
(%)
Random Test
SimpleRank Test
German Credit, 2 classes, 1,000 instances, 10-fold Cross Validation
0 100 200 300 400 500 600 700 800 90040
50
60
70
80
90
100
data instances
accu
racy
(%)
RandomTrain
RandomTestSimpleRankTrain
SimpleRankTest
DROP1
DROP2DROP3
10000 100 200 300 400 500 600 700 800 90030
40
50
60
70
90
100
accu
racy
(%)
4%5%11%
14%
9%
0 100 200 300 400 500 600 700 800 90030
40
50
60
70
80
90
100
4%
6%7%8%10%
12%13%
Number of instances seen before interruption, S
Random, Euclidean distance
Random, Fixed R = 4
SimpleRank, Fixed R = 4
SimpleRank, Adaptive R
Two Patterns Dataset
10000 100 200 300 400 500 600 700 800 90030
40
50
60
70
90
100
accu
racy
(%)
4%5%11%
14%
9%
0 100 200 300 400 500 600 700 800 90030
40
50
60
70
80
90
100
4%
6%7%8%10%
12%13%
Number of instances seen before interruption, S
Random, Euclidean distance
Random, Fixed R = 4
SimpleRank, Fixed R = 4
SimpleRank, Adaptive R
Random, Euclidean distance
Random, Fixed R = 4
SimpleRank, Fixed R = 4
SimpleRank, Adaptive R
Two Patterns Dataset
Two_Pat, 4 classes, 5,000 instances, 1,000/4,000 split
0 50 100 150 200 250 300 350 40030
40
50
60
70
80
90
100
accu
racy
(%)
8%
9% -
11%
12%
Random, Euclidean distance
Random, Fixed R = 8
SimpleRank, Fixed R = 8
SimpleRank, Adaptive R
Number of instances seen before interruption, S
Leaf Dataset
0 50 100 150 200 250 300 350 40030
40
50
60
70
80
90
100
accu
racy
(%)
8%
9% -
11%
12%
Random, Euclidean distance
Random, Fixed R = 8
SimpleRank, Fixed R = 8
SimpleRank, Adaptive R
Number of instances seen before interruption, S
0 50 100 150 200 250 300 350 40030
40
50
60
70
80
90
100
accu
racy
(%)
8%
9% -
11%
12%
Random, Euclidean distance
Random, Fixed R = 8
SimpleRank, Fixed R = 8
SimpleRank, Adaptive R
Random, Euclidean distance
Random, Fixed R = 8
SimpleRank, Fixed R = 8
SimpleRank, Adaptive R
Number of instances seen before interruption, S
Leaf Dataset
Leaf Dataset, 6 classes, 442 instances
0 200 400 600 800 100020
30
40
50
60
70
80
90
100
Number of instances seen before interruption,
accu
racy
(%)
Face Dataset
Random, Euclidean distance
Random, Fixed R = 4
SimpleRank, Fixed R = 4
SimpleRank, Adaptive R
Random, Euclidean distance
Random, Fixed R = 3
SimpleRank, Fixed R = 3
SimpleRank, Adaptive R
3%4%
Face dataset, 16 classes, 2,231 instances, 1,113/1,118 split
