Title

Maximal Data PilingVisual similarity of & ?Can show (Ahn & Marron 2009), for d < n:

I.e. directions are the same!How can this be?Note lengths are differentStudy from transformation viewpoint

1Maximal Data PilingRecall Transfoview of FLD:

2Maximal Data PilingInclude Corres-pondingMDP Transfo:

Both giveSameResult!

3Maximal Data PilingDetails:FLD, seping plane normal vectorWithin Class, PC1 PC2Global, PC1 PC2

4Maximal Data PilingAcknowledgement:This viewpointI.e. insight into why FLD = MDP(for low dimal data)Suggested by Daniel Pea5Maximal Data PilingFun e.g:rotate from PCA to MDP dirns

6Maximal Data PilingMDP for other class labellings:Always existsSeparation bigger for natural clustersCould be used for clusteringConsider all directionsFind one that makes largest gapVery hard optimization problemOver 2n-2 possible directions7Maximal Data PilingA point of terminology (Ahn & Marron 2009):MDP is maximal in 2 senses:# of data piledSize of gap (within subspace gend by data)

8Maximal Data PilingRecurring, over-arching, issue:

HDLSS space is a weird place9Maximal Data PilingUsefulness for Classification? First Glance: Terrible, not generalizable HDLSS View: Maybe OK? Simulation Result:Good Performance forAutocorrelated Errors Reason: not known10Kernel EmbeddingAizerman, Braverman and Rozoner (1964) Motivating idea:Extend scope of linear discrimination,By adding nonlinear components to data(embedding in a higher dimal space)

Better use of name:nonlinear discrimination?11Kernel EmbeddingToy Examples:In 1d, linear separation splits the domaininto only 2 parts

12

Kernel EmbeddingBut in the quadratic embedded domain,

linear separation can give 3 parts

13Kernel EmbeddingBut in the quadratic embedded domain

Linear separation can give 3 partsoriginal data space lies in 1d manifoldvery sparse region of curvature of manifold gives: better linear separationcan have any 2 break points(2 points line)

14Kernel EmbeddingStronger effects for higher order polynomial embedding:E.g. for cubic,

linear separation can give 4 parts (or fewer)

15Kernel EmbeddingStronger effects - high. ord. poly. embedding:original space lies in 1-d manifold, even sparser in higher d curvature gives:improved linear separationcan have any 3 break points (3 points plane)?Note: relatively few interesting separating planes

16Kernel EmbeddingGeneral View: for original data matrix:

add rows:

i.e. embed inHigherDimensionalspace

17Kernel EmbeddingEmbedded Fisher Linear Discrimination:Choose Class 1, for any when:

in embedded space.Image of class boundaries in original space is nonlinearAllows more complicated class regionsCan also do Gaussian Lik. Rat. (or others) Compute image by classifying points from original space

18Kernel EmbeddingVisualization for Toy Examples:Have Linear Disc. In Embedded SpaceStudy Effect in Original Data SpaceVia Implied Nonlinear RegionsApproach:Use Test Set in Original Space(dense equally spaced grid)Apply embedded discrimination RuleColor Using the Result19Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

20PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

PC1OriginalData

21PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

PC1

22PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

PC1

23PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

PC1

24PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

PC1

25PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

PC1Note:EmbeddingAlways BadDont WantGreatestVariation

26PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

FLDOriginalData27PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

FLD28PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

FLD29PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

FLD30PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

FLD

All Stable& Very Good31PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

GLROriginalData32PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

GLR33PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

GLR34PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

GLR35PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

GLR36PEod1Raw.ps

Kernel EmbeddingPolynomial Embedding, Toy Example 1:Parallel Clouds

GLRUnstableSubject toOverfitting37PEod1Raw.ps