Unsupervised Learning: Clustering

UnsuperviseUnsupervised Learning: d Learning: ClusteringClusteringSome material adapted from slides by Andrew Some material adapted from slides by Andrew

Moore, CMU.Moore, CMU.

Visit Visit http://www.autonlab.org/tutorials/ for forAndrewAndrew’’s repository of Data Mining tutorials.s repository of Data Mining tutorials.

Unsupervised LearningUnsupervised Learning Supervised learning used labeled data pairs Supervised learning used labeled data pairs

(x, y) to learn a function f : X→Y.(x, y) to learn a function f : X→Y. But, what if we donBut, what if we don’’t have labels?t have labels?

No labels = No labels = unsupervised learningunsupervised learning Only some points are labeled = Only some points are labeled = semi-semi-

supervised learningsupervised learning Labels may be expensive to obtain, so we only get Labels may be expensive to obtain, so we only get

a few.a few.

ClusteringClustering is the unsupervised grouping of is the unsupervised grouping of data points. It can be used for data points. It can be used for knowledge knowledge discoverydiscovery..

Clustering DataClustering Data

K-Means ClusteringK-Means Clustering

K-Means ( k , data )• Randomly choose k

cluster center locations (centroids).

• Loop until convergence

• Assign each point to the cluster of the closest centroid.

• Reestimate the cluster centroids based on the data assigned to each.

K-Means AnimationK-Means Animation

Example generated by Andrew Moore using Dan Pelleg’s super-duper fast K-means system:

Dan Pelleg and Andrew Moore. Accelerating Exact k-means Algorithms with Geometric Reasoning.Proc. Conference onKnowledge Discovery in Databases 1999.

Problems with K-MeansProblems with K-Means VeryVery sensitive to the initial points. sensitive to the initial points.

Do many runs of k-Means, each with Do many runs of k-Means, each with different initial centroids.different initial centroids.

Seed the centroids using a better Seed the centroids using a better method than random. (e.g. Farthest-method than random. (e.g. Farthest-first sampling)first sampling)

Must manually choose k.Must manually choose k. Learn the optimal k for the clustering. Learn the optimal k for the clustering.

(Note that this requires a performance (Note that this requires a performance measure.)measure.)

Problems with K-MeansProblems with K-Means How do you tell it which clustering you How do you tell it which clustering you

want?want?

Constrained clustering techniquesConstrained clustering techniques

Same-cluster constraint(must-link)

Different-cluster constraint(cannot-link)

Unsupervised Learning: Clustering

Documents

Unsupervised Learning and Clustering - University of …rita/uml_course/add_mat/Clustering.pdf · Unsupervised Learning and Clustering. ... Iterative Optimization Algorithms ... In

CS 478 - Clustering1 Unsupervised Learning and Clustering In unsupervised learning you are given a data set with no output classifications Clustering is

Machine Learning II: Unsupervised Learning€¦ · 2 Our agenda today Unsupervised Learning and Applications • Clustering – Application: Image Segmentation • Density Estimation

Unsupervised Learning: Clustering 1 Lecture 16: Clustering Web Search and Mining

Unsupervised learning or Clustering – K-means …guestrin/Class/10701-S07/Slides/clustering.pdf · Unsupervised learning or Clustering – K-means Gaussian mixture models Machine

Unsupervised Clustering for Deep Learning: A tutorial survey

Lecture 12: Unsupervised learning. Clustering€¦ · Lecture 12: Unsupervised learning. Clustering Introduction to unsupervised learning Clustering problem K-means clustering Hierarchical

Neural Networks - Lecture 81 Unsupervised competitive learning Particularities of unsupervised learning Data clustering Neural networks for clustering

CC282 Unsupervised Learning (Clustering) Lecture 7 slides for

Chapter 9 UNSUPERVISED LEARNING: Clustering Part 2 Cios / Pedrycz / Swiniarski / Kurgan

STAT 408 - Statistical Learning Clustering€¦ · Statistical Learning Clustering Unsupervised Learning k-means clustering ## K-means clustering with 3 clusters of sizes 44, 26,

Quantum speed-up for unsupervised learning - Springer · Quantum speed-up for unsupervised learning ... Keywords Unsupervised learning · Clustering ·Quantum learning · Quantum

Reinforcement Learning Peter Bodík. Previous Lectures Supervised learning –classification, regression Unsupervised learning –clustering, dimensionality

Learning Bit by Bit Clustering. Supervised vs. Unsupervised Training vs. Exploring

Artificial Intelligence for Cybersecurity · Artificial Intelligence and Machine Learning. Machine Learning. Unsupervised Learning. Clustering. Clustering (2) •Can be aggregative

Chapter 9 UNSUPERVISED LEARNING: Clustering Part 1 Cios / Pedrycz / Swiniarski / Kurgan

Unsupervised Learning and Clustering - University of …rita/ml_course/lectures_old/Clustering.pdf · Unsupervised Learning and Clustering. ... Iterative Optimization Algorithms

Improving Unsupervised Image Clustering With Robust Learning

Unsupervised Learning: Clustering Rong Jin Outline Unsupervised learning K means for clustering Expectation Maximization algorithm for clustering

Clustering - cs.bilkent.edu.trcs.bilkent.edu.tr/~gunduz/teaching/cs550/documents/CS550... · Clustering / Unsupervised Learning In SUPERVISED learning ... Heterogeneous vs homogeneous: