Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Department of Computer ScienceCSCI 5622: Machine Learning
Chenhao TanLecture 15: VC Dimension
Slides adapted from Jordan Boyd-Graber, Chris Ketelsen
1
Logistics
• Project feedback• Peer feedback• Instructor feedback
• HW3 scores• Midterm
2
Learning objectives
• Learning about the VC dimension for infinite dimensional H
• See some examples of computing the VC dimension
• See some generalization bounds in terms of VC dimension
3
Assumptions and Definitions
4
Generalization Bound of Finite H
5
Generalization Bound of Infinite H
6
Generalization Bound of Infinite H
7
Complexity Measures - Counting Bits
8
Complexity Measures - Counting Bits
9
Complexity Measures – Growth function
10
Complexity Measures – Growth function
11
Complexity Measures – Growth function
12
Complexity Measures – Growth function
13
Complexity Measures – Growth function
14
Complexity Measures – Growth function
15
Complexity Measures – Growth function
16
Complexity Measures – Growth function
17
Complexity Measures – Growth function
18
Complexity Measures – Growth function
19
Complexity Measures – Growth function
20
Complexity Measures – Growth function
21
Complexity Measures – VC dimension
22
Complexity Measures – VC dimension
23
Complexity Measures – VC dimension
24
Complexity Measures – VC dimension
25
Complexity Measures – VC dimension
26
Complexity Measures – VC dimension
27
Complexity Measures – VC dimension
28
Complexity Measures – VC dimension
29
Complexity Measures – VC dimension
30
Complexity Measures – VC dimension
31
Complexity Measures – VC dimension
32
Complexity Measures – VC dimension
33
Complexity Measures – VC dimension
34
Complexity Measures – VC dimension
35
Complexity Measures – VC dimension
36
Complexity Measures – VC dimension
37
Complexity Measures – VC dimension
38
Complexity Measures – VC dimension
39
Complexity Measures – VC dimension
40
Complexity Measures – VC dimension
41
Complexity Measures – VC dimension
42
43
Complexity Measures – VC dimension
44
Complexity Measures – VC dimension
45
Complexity Measures – VC dimension
46
Wrap-up
• VC dimension is a convenient measure of the complexity of infinite hypothesis classes
• Nice generalization bounds that depend on the VC dimension
47