Lesson menuPython TutorialHideShow

Python Tutorial

Hierarchical Clustering

Group data into a nested tree of clusters without deciding the number of clusters up front.

← Confusion MatrixLogistic Regression →

The Idea

Hierarchical clustering is an unsupervised method: it finds structure in unlabeled data. The agglomerative (bottom-up) approach starts with every point as its own cluster, then repeatedly merges the two closest clusters until only one remains. The full history forms a tree called a dendrogram.

Unlike k-means, you do not need to choose the number of clusters beforehand — you cut the dendrogram at the height that gives the grouping you want.

Drawing a Dendrogram

import numpy as np
import matplotlib.pyplot as plt
from scipy.cluster.hierarchy import dendrogram, linkage

x = [4, 5, 10, 4, 3, 11, 14, 6, 10, 12]
y = [21, 19, 24, 17, 16, 25, 24, 22, 21, 21]
data = np.array(list(zip(x, y)))

# 'ward' minimizes the variance added by each merge
linkage_matrix = linkage(data, method="ward")

dendrogram(linkage_matrix)
plt.xlabel("sample index")
plt.ylabel("distance")
plt.show()

The vertical axis is the distance at which clusters merge. A long vertical jump suggests a natural place to cut.

Getting Cluster Labels

from sklearn.cluster import AgglomerativeClustering

model = AgglomerativeClustering(n_clusters=2, linkage="ward")
labels = model.fit_predict(data)
print(labels)   # e.g. [0 0 1 0 0 1 1 0 1 1]

plt.scatter(x, y, c=labels, cmap="viridis")
plt.show()

Linkage Methods

The linkage rule defines the distance between two clusters and changes the shapes you recover.

LinkageDistance between clustersTends to produce
singleclosest pair of pointsLong, chain-like clusters
completefarthest pair of pointsCompact, equal-diameter clusters
averagemean of all pairwise distancesA balance of the two
wardincrease in within-cluster varianceSimilar-sized, spherical clusters

Choosing the Number of Clusters

Cut the dendrogram where merges suddenly become expensive (large vertical gaps), or use a distance threshold instead of a fixed count:

model = AgglomerativeClustering(
    n_clusters=None, distance_threshold=15, linkage="ward")
labels = model.fit_predict(data)
print("clusters found:", labels.max() + 1)

Scale your features first (see Feature Scaling). Distance-based clustering is dominated by whichever feature has the largest range.

Best Practices

  • Standardize features before clustering so no single feature dominates.
  • Use ward linkage as a solid default for compact clusters.
  • Hierarchical clustering is O(n²) in memory — sample or switch to k-means for very large datasets.
  • Validate clusters with a silhouette score or domain knowledge, not just the picture.

Try It Yourself

Exercise 1: Cluster the sample points into 2 groups with ward linkage and print the labels.

Show solution
import numpy as np
from sklearn.cluster import AgglomerativeClustering
data = np.array([[1, 2], [1, 4], [8, 8], [9, 9]])
print(AgglomerativeClustering(n_clusters=2).fit_predict(data))

Exercise 2: What advantage does hierarchical clustering have over k-means regarding the number of clusters?

Show solution

You don't have to choose k in advance — you cut the dendrogram at whatever height gives the grouping you want.

📘 Real-World Deep Dive

Knowing <strong>ML Hierarchical Clustering (scikit-learn)</strong> well is what turns scikit-learn from a curiosity into a daily tool — you'll reach for it in nearly every real project.

Real-Life Scenario

An end-to-end usage of ML Hierarchical Clustering that you'd actually see in a data pipeline or analytics notebook.

Real-Life Example

import numpy as np
from scipy.cluster.hierarchy import linkage, fcluster
rng = np.random.default_rng(0)
X = np.vstack([rng.normal(0, 0.1, (5,2)),
               rng.normal(5, 0.1, (5,2))])
Z = linkage(X, method="ward")
labels = fcluster(Z, t=2, criterion="maxclust")
print(labels)

Expected Output

(see source)

Common mistakes

  • fit expects numeric arrays; OneHotEncoder / LabelEncoder are easy to forget for categorical features.
  • Calling predict on a model trained on unscaled data and then scaling inputs at inference time silently degrades accuracy.
  • train_test_split(X, y) requires both arrays; train_test_split(X) for unsupervised learning slips past static checkers.
  • Treating ML Hierarchical Clustering as a black box without reading the docs — the API has subtle defaults that bite when you scale.

🚀 Performance & Best Practices

  • Wrap preprocessing + estimator in a Pipeline so fit / predict stay reproducible.
  • Use joblib / pickle for serialising models, not the entire Python state.
  • For > 100 k rows, switch to HistGradientBoostingClassifier or cuML.
  • When working with scikit-learn, prefer vectorised / batched operations over Python loops.

🧪 Try It Yourself

  1. Reproduce the snippet on a representative slice of your own data.
  2. Profile the snippet with cProfile or timeit and find the single biggest improvement.
  3. Generalise the snippet into a small, reusable function you can drop into future projects.

FAQ: Hierarchical Clustering

Common questions about this page.

What is Hierarchical Clustering?

Hierarchical Clustering is a Machine Learning lesson that explains hierarchical clustering in Python. Group data into a nested tree of clusters without deciding the number of clusters up front. Copy the samples and run them in the Python editor. It is written for beginners who want a clear definition and working examples.

Should I run hierarchical clustering examples locally for better learning?

Yes. Use the browser editor on StudyGrid for a quick check, then Download the example and run it on your computer. Local runs show real errors and the real toolchain, which is one of the fastest ways to learn hierarchical clustering in this Machine Learning Python lesson (Hierarchical Clustering).

How do I use hierarchical clustering in Python?

To use hierarchical clustering in Python, follow the examples on this StudyGrid page. Copy a snippet, run it in the browser, then Download and run it locally for better learning. Change the values and compare the output.

What is the syntax of hierarchical clustering?

This Hierarchical Clustering tutorial shows hierarchical clustering syntax with short Python examples. Use the code blocks in this lesson for the exact statements, then try them in your editor.

Hierarchical Clustering example for beginners

Yes. This page includes a beginner hierarchical clustering example you can copy and run. It is designed for searches such as "hierarchical clustering for beginners", "hierarchical clustering example", and "how to use hierarchical clustering".

What are common mistakes with hierarchical clustering?

Common hierarchical clustering mistakes include wrong syntax, mixing types, and skipping practice. Work through this Machine Learning chapter in order, run every example, and check the output before moving on.

Why should I learn hierarchical clustering?

Hierarchical Clustering is used in real Python work. Learning hierarchical clustering helps you write clearer programs and continue the Machine Learning tutorial on StudyGrid.

Is Hierarchical Clustering free to learn online?

Yes. You can learn hierarchical clustering free on StudyGrid (studygrid.in). This chapter is part of the Machine Learning path and includes examples, syntax, and next-step links.