Topic Modeling vs Classification
Classification maps data into known buckets; topic modeling reveals patterns you did not define in advance. They answer fundamentally different questions.
Classification (Supervised)
Starts with a predefined label set. Every item gets assigned to an existing category. The goal is decision assignment — routing items into buckets you already understand.
Topic Modeling (Unsupervised)
No predefined labels. The algorithm discovers latent themes from the data itself. The goal is pattern discovery — revealing structure you did not know existed.
Key insight: Teams often use topic discovery before building a formal taxonomy. It helps reveal recurring issues, hidden subpopulations, and language used by real users. The discovered topics can then inform a classification system.
The Theme Discovery Pipeline
A practical pipeline from raw documents to validated topic labels. The last steps matter most because raw clusters do not explain themselves.
Gather the corpus: reviews, tickets, papers, chats. Define the unit of analysis.
Remove boilerplate, signatures, duplicates. Bad input = bad clusters.
Convert text to dense vectors. Semantic similarity, not keyword matching.
UMAP or t-SNE to denoise and reveal structure. Optional but often helpful.
K-means, HDBSCAN, or hierarchical. Group similar embeddings together.
LLM or human reviews representative examples and names each cluster.
Scalability note: Production pipelines need batching, approximate nearest-neighbor indexing, incremental updates, and sampling strategies for review. A topic modeling system must be designed like a data product, not just a notebook experiment.
Interactive: 2D Cluster Visualization
Explore how customer support tickets cluster by theme. Each point represents a ticket embedded into 2D space. Colors show discovered groups.
Hover to see ticket text. Each color is a discovered topic cluster. Grey points are noise (unclustered).
Why Embeddings Beat Keywords
Embedding-based methods represent each text unit in a semantic vector space, so clustering can group items that are conceptually related even if they share no keywords.
Keyword-Based (TF-IDF / LDA)
Traditional methods rely on shared vocabulary. People describe the same issue in many different ways, causing false separation.
Embedding-Based (Modern)
Embeddings capture semantic similarity. Both sentences land near each other in vector space because they're about billing problems, regardless of wording.
The modern combo: Embeddings give you better semantic grouping, and LLMs can then summarize or name the discovered clusters. That combination is often more useful to product teams than traditional topic-word lists alone.
Dimensionality Reduction: UMAP vs t-SNE
High-dimensional embedding spaces (384-3072 dims) are noisy. Reduction can reveal local structure, denoise the space, and make clusters easier to separate.
Interactive: See the effect of reduction
| Method | Strengths | Weaknesses |
|---|---|---|
| UMAP | Preserves global structure, faster, good for clustering downstream | Hyperparameter-sensitive (n_neighbors, min_dist) |
| t-SNE | Excellent local structure, reveals fine-grained clusters | Loses global distances, slower, non-deterministic by default |
| PCA | Fast, deterministic, linear | Misses non-linear structure in text embeddings |
Caution: Reduction may distort distances if applied carelessly. A strong engineering answer: reduction is a tool, not a default law. You use it when it improves cluster structure or interpretability, then verify the result with representative examples rather than trusting a plot alone.
When to skip reduction: If your clustering algorithm handles high dimensions well (e.g., you're using cosine K-means on normalized embeddings), reduction may hurt more than it helps. Always compare cluster quality with and without.
Clustering Algorithms Compared
The right algorithm depends on data distribution, scale, and the need for interpretability. No clustering method can rescue weak embeddings.
K-Means
Assumes roughly spherical clusters. You choose k (number of clusters) in advance. Fast and simple, but sensitive to initialization and outliers.
DBSCAN / HDBSCAN
Density-based: finds clusters of any shape and automatically identifies noise points. No need to specify k. HDBSCAN adapts to varying densities.
Hierarchical
Builds a tree (dendrogram) of clusters from fine to coarse. Cut at different levels to get different granularities. Great for exploring topic hierarchies.
| Feature | K-Means | DBSCAN | Hierarchical |
|---|---|---|---|
| Requires k? | Yes | No (uses min_samples, eps) | Cut threshold |
| Cluster shape | Spherical | Arbitrary | Varies by linkage |
| Noise handling | None (forces assignment) | Built-in noise label | Depends on cut level |
| Scale | Millions | 100Ks | 10Ks |
How LLMs Name Clusters
"Billing friction during checkout" is more useful than "payment issue words." Good labels are operational, not statistical.
Weak Labels (Automated Only)
Top-k terms from TF-IDF. Descriptive but not actionable. Product teams can't route work based on word lists.
Strong Labels (LLM + Human)
LLM summarizes representative examples. Human validates and enriches with operational context. Teams can act on these labels.
The naming pipeline
Caution: LLM-generated summaries can sound crisp even when the underlying cluster is messy. Cluster quality must still be validated against raw examples.
Monitoring Topic Drift Over Time
Topics drift as products change, events occur, and new language enters the corpus. Topic discovery is not a one-time report.
Interactive: Topic volume over time
Watch how topic proportions shift across months. Click "Simulate Drift" to see what happens when a new issue emerges.
Evaluating Topic Quality
The question is not only "Do the clusters exist?" but "Can a product, operations, or research team act on them?"
Automatic Measures
Human Evaluation (Essential)
Strong answer: Evaluation should combine statistical coherence with analyst usefulness. If a cluster contains semantically mixed examples, the label is likely misleading even if a metric looks acceptable. Manual inspection of representative examples remains essential.
Common Mistakes at Scale
Topic discovery should be treated as iterative sense-making. The goal is insight you can trust, not just an impressive chart.
Wrong Unit of Analysis
Clustering entire documents when the real signal is at the sentence or paragraph level. Or clustering individual sentences when the topic only makes sense across a full ticket. Match the unit to the question.
Clustering Boilerplate
Email signatures, auto-replies, and template text create false clusters that dominate the topic space. Always clean boilerplate before embedding. The biggest cluster is often "noise."
Overinterpreting Visualizations
2D plots are projections of high-dimensional space. Apparent clusters in 2D may not be real, and real clusters may be invisible in 2D. Always validate with representative examples, not just the plot.
Treating Labels as Truth
Automatically generated cluster labels sound authoritative but may be misleading. "Customer Frustration with Premium Features" sounds crisp even if the cluster mixes unrelated complaints. Labels are hypotheses.
Ignoring Temporal Drift
Running topic modeling once and assuming the same clusters remain stable indefinitely. Products change, seasons shift, and new issues emerge. Set up periodic re-analysis and drift monitoring.
Weak Embeddings
Using a general-purpose embedding model for domain-specific text. Medical, legal, or technical text may need domain-adapted embeddings. No clustering algorithm can rescue embeddings that don't capture your domain's semantics.
Key takeaway: "Good topic discovery pipelines treat clusters as hypotheses that must be interpreted and validated, not as automatic truth from the algorithm." The strongest answers emphasize validation as a first-class step, not an afterthought.