Unsupervised Learning: Clustering and Dimensionality Reduction Explained

Scenario context

Unsupervised learning helps you find patterns when you do not have labels. Clustering groups similar data points into segments, and dimensionality reduction compresses large feature spaces into smaller, useful representations for visualization, search, and downstream models.

Spotify's "Discover Weekly" playlist groups listeners into micro-genres like "Indie Stomp and Holler" and "Vapor Twitch" — without an…

This walkthrough focuses on Finding Structure Where There Are No Labels.

Before making a design choice, clarify:

• Spotify's "Discover Weekly" playlist groups listeners into micro-genres like "Indie Stomp and Holler" and "Vapor Twitch" — without anyone ever labeling the data.
• Nobody told the algorithm what genres exist or how many to create.
• It found them by clustering millions of listening patterns, discovering structure that nobody had explicitly defined.

Concepts in play: deep-learning-foundations, vector-embeddings-and-similarity, sorting-and-searching-algorithms, tree-data-structures

Stage 1 — The Unsupervised Input: Unlabeled Feature Matrices

• Unsupervised pipelines start with a feature matrix X — rows are examples, columns are features — but no label column exists .
• Toy dataset — user segments: Even without labels, three groups are visible: high-spend frequent users, low-spend infrequent users, and a mid-range group.
• Preprocessing matters: Distance-based methods are sensitive to scale.
• If monthly spend ranges 0–1000 and sessions/week ranges 0–10, spend will dominate Euclidean distance unless features are standardized first.

Observable outcome: If monthly spend ranges 0–1000 and sessions/week ranges 0–10, spend will dominate Euclidean distance unless features are standardized first.

Stage 2 — Clustering Algorithms: K-Means, DBSCAN, and Hierarchical

• K-Means alternates two steps until convergence: 1.
• Assign each point to the nearest centroid.
• Recompute each centroid as the mean of its assigned points.
• K-Means minimizes: $$\min {{c k}} \sum {i=1}^{n} \ x i - c {k(i)} \ ^2$$ It is fast but assumes roughly spherical clusters and requires choosing $k$ in advance.

Observable outcome: K-Means minimizes: $$\min {{c k}} \sum {i=1}^{n} \ x i - c {k(i)} \ ^2$$ It is fast but assumes roughly spherical clusters and requires choosing $k$ in advance.

Stage 3 — Deep Dive: How K-Means Converges Step by Step

• K-Means alternates two steps until assignments stop changing.
• In the assignment step , each point is assigned to the nearest centroid by Euclidean distance.
• In the update step , each centroid moves to the mean of its assigned points.
• Total inertia (within-cluster sum of squares) can only decrease or stay flat each iteration — it never increases — guaranteeing convergence, though not necessarily to the global optimum.

flowchart LR
    A[Initialize K Centroids] --> B[Assign Points to Cluster]
    B --> C[Recompute Centroids]
    C --> D{Centroids Changed?}
    D -- Yes --> B
    D -- No --> E[Final Clusters]

Observable outcome: Total inertia (within-cluster sum of squares) can only decrease or stay flat each iteration — it never increases — guaranteeing convergence, though not necessarily to the global optimum.

Stage 4 — Dimensionality Reduction: PCA, t-SNE, and Autoencoders

• PCA (Principal Component Analysis): - Linear technique — finds the directions of maximum variance in the data.
• • Great baseline for tabular numeric data.
• • "Variance captured by first 2 components = 87%" is easy to communicate.
• t-SNE and UMAP: - Non-linear embeddings that preserve neighborhood relationships.

flowchart TD
    A[Raw Unlabeled Data] --> B[Clean and Scale Features]
    B --> C{Goal?}
    C -->|Grouping| D[Run Clustering]
    C -->|Compression or Viz| E[Run Dimensionality Reduction]
    D --> F[Validate Cluster Quality]
    E --> G[Validate Information Retention]
    F --> H[Business Interpretation]
    G --> H

flowchart TD
    A[High-Dim Data] --> B[Standardize Features]
    B --> C[Compute Covariance Matrix]
    C --> D[Eigen Decomposition]
    D --> E[Sort Eigenvectors]
    E --> F[Select Top K Vectors]
    F --> G[Project Data]
    G --> H[Reduced Dimensions]

Observable outcome: t-SNE and UMAP: - Non-linear embeddings that preserve neighborhood relationships.

Stage 5 — Real-World Applications: Unsupervised Learning in Production

• Customer segmentation: cluster purchase behavior to identify high-value, at-risk, and new users.
• Feed cluster labels into targeted marketing campaigns.
• Anomaly detection: DBSCAN marks sparse points as noise — those are your anomalies in log data or sensor streams.
• Search and recommendation: PCA or autoencoders compress item feature vectors into dense embeddings.

Observable outcome: Search and recommendation: PCA or autoencoders compress item feature vectors into dense embeddings.

Stage 6 — What to Learn Next

• • Supervised Learning Algorithms: Regression and Classification - Machine Learning Fundamentals - Deep Learning Architectures: CNNs, RNNs, and Transformers

Observable outcome: - Supervised Learning Algorithms: Regression and Classification - Machine Learning Fundamentals - Deep Learning Architectures: CNNs, RNNs, and Transformers

End-to-end outcome

Spotify's "Discover Weekly" playlist groups listeners into micro-genres like "Indie Stomp and Holler" and "Vapor Twitch" — without anyone ever labeling the data.

Alternatives considered

Decision Guide: Choosing the Right Method

Decision Guide: Choosing the Right Method

Failure and degraded modes

Trade-offs & Failure Modes: Trade-offs and Common Mistakes

• Detection: Trade-offs & Failure Modes: Trade-offs and Common Mistakes
• User or operator impact: Trade-offs & Failure Modes: Trade-offs and Common Mistakes