Scatter Plot
Visualize relationships between two variables to spot trends, clusters, and potential correlations.
Why Scatter Plots Matter
Scatter plots reveal linear or non-linear relationships, highlight outliers, and guide feature selection. Before training models, inspect scatter plots to understand whether a simple regression or more complex algorithm is appropriate.
Creating Basic Scatter Plots
Scatter plots are created using Matplotlib's scatter() function. They display individual data points as dots on a two-dimensional plane, making it easy to observe relationships between variables.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
# Generate sample data
np.random.seed(42)
sqft = np.random.normal(1500, 500, 100)
price = 100 * sqft + np.random.normal(0, 50000, 100) # Linear relationship with noise
df = pd.DataFrame({'sqft': sqft, 'price': price})
# Basic scatter plot
plt.figure(figsize=(8, 6))
plt.scatter(df["sqft"], df["price"], alpha=0.6)
plt.xlabel("Square Feet")
plt.ylabel("Price ($)")
plt.title("Housing Prices vs. Size")
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# With more customization
plt.figure(figsize=(8, 6))
plt.scatter(df["sqft"], df["price"], alpha=0.7, s=50, c='steelblue', edgecolors='black', linewidth=0.5)
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title("Housing Prices vs. Size", fontsize=14, fontweight='bold')
plt.grid(True, alpha=0.3, linestyle='--')
plt.tight_layout()
plt.show()Key parameters: alpha controls transparency (0-1), s sets point size, c sets color, and edgecolors adds borders. Use transparency (alpha) to mitigate overplotting when datasets are dense.
Color Encoding and Categorization
Color encoding adds a third dimension to scatter plots, helping identify patterns, clusters, or relationships with categorical or continuous variables.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from matplotlib.colors import ListedColormap
# Generate sample data with categories
np.random.seed(42)
sqft = np.random.normal(1500, 500, 200)
price = 100 * sqft + np.random.normal(0, 50000, 200)
condition = np.random.choice(['Poor', 'Fair', 'Good', 'Excellent'], size=200, p=[0.1, 0.3, 0.4, 0.2])
condition_score = np.where(condition == 'Poor', 1,
np.where(condition == 'Fair', 2,
np.where(condition == 'Good', 3, 4)))
df = pd.DataFrame({
'sqft': sqft,
'price': price,
'condition': condition,
'condition_score': condition_score
})
# Color by continuous variable (numeric)
plt.figure(figsize=(10, 6))
scatter = plt.scatter(df["sqft"], df["price"], c=df["condition_score"],
cmap="viridis", alpha=0.7, s=60, edgecolors='black', linewidth=0.5)
plt.colorbar(scatter, label="Condition Score")
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title("Housing Prices vs. Size (Colored by Condition)", fontsize=14)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Color by categorical variable
colors = {'Poor': 'red', 'Fair': 'orange', 'Good': 'green', 'Excellent': 'blue'}
for cat in df['condition'].unique():
mask = df['condition'] == cat
plt.scatter(df.loc[mask, 'sqft'], df.loc[mask, 'price'],
c=colors[cat], label=cat, alpha=0.7, s=60, edgecolors='black', linewidth=0.5)
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title("Housing Prices vs. Size (Colored by Condition Category)", fontsize=14)
plt.legend(title="Condition")
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Size encoding (bubble plot)
size = df['condition_score'] * 50 # Scale size based on condition
plt.figure(figsize=(10, 6))
plt.scatter(df["sqft"], df["price"], s=size, c=df["condition_score"],
cmap="coolwarm", alpha=0.6, edgecolors='black', linewidth=0.5)
plt.colorbar(label="Condition Score")
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title("Bubble Plot: Size and Color Encode Condition", fontsize=14)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()Color encoding helps identify clusters, trends, and outliers. Use consistent color palettes (viridis, plasma, coolwarm) across dashboards. Share color scheme guidelines through info.studygrid@gmail.com to maintain consistency across your team's visualizations.
Adding Trend Lines and Regression
Trend lines help visualize relationships and can reveal linear or non-linear patterns. Overlay regression lines to summarize the relationship between variables.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy import stats
from sklearn.linear_model import LinearRegression
# Generate sample data
np.random.seed(42)
sqft = np.random.normal(1500, 500, 100)
price = 100 * sqft + np.random.normal(0, 50000, 100)
df = pd.DataFrame({'sqft': sqft, 'price': price})
x = df["sqft"].values
y = df["price"].values
# Method 1: Using numpy polyfit (polynomial fitting)
coef = np.polyfit(x, y, deg=1) # Linear regression
trend = np.poly1d(coef)
plt.figure(figsize=(10, 6))
plt.scatter(x, y, alpha=0.6, s=50, label='Data points')
plt.plot(x, trend(x), color="crimson", linewidth=2, label=f'Trend: y={coef[0]:.2f}x+{coef[1]:.2f}')
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title("Scatter Plot with Linear Trend Line", fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Method 2: Using scipy stats.linregress
slope, intercept, r_value, p_value, std_err = stats.linregress(x, y)
line = slope * x + intercept
print(f"Slope: {slope:.2f}")
print(f"Intercept: {intercept:.2f}")
print(f"R-squared: {r_value**2:.4f}")
print(f"P-value: {p_value:.6f}")
print(f"Standard error: {std_err:.4f}")
# Method 3: Using scikit-learn
model = LinearRegression()
model.fit(x.reshape(-1, 1), y)
predicted = model.predict(x.reshape(-1, 1))
plt.figure(figsize=(10, 6))
plt.scatter(x, y, alpha=0.6, s=50, label='Data points')
plt.plot(x, predicted, color="crimson", linewidth=2,
label=f'Regression: y={model.coef_[0]:.2f}x+{model.intercept_:.2f}')
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title(f"Linear Regression (R² = {model.score(x.reshape(-1, 1), y):.4f})", fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Residual plot to check linearity
residuals = y - predicted
plt.figure(figsize=(10, 5))
plt.scatter(predicted, residuals, alpha=0.6, s=50)
plt.axhline(y=0, color='red', linestyle='--', linewidth=2)
plt.xlabel("Predicted Values", fontsize=12)
plt.ylabel("Residuals", fontsize=12)
plt.title("Residual Plot (Check for Patterns)", fontsize=14)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Polynomial trend (degree 2)
poly_coef = np.polyfit(x, y, deg=2)
poly_trend = np.poly1d(poly_coef)
x_sorted = np.sort(x)
plt.figure(figsize=(10, 6))
plt.scatter(x, y, alpha=0.6, s=50, label='Data points')
plt.plot(x_sorted, poly_trend(x_sorted), color="green", linewidth=2,
label='Polynomial trend (degree 2)')
plt.xlabel("Square Feet", fontsize=12)
plt.ylabel("Price ($)", fontsize=12)
plt.title("Scatter Plot with Polynomial Trend Line", fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()Always visualize residuals to ensure the trend line is appropriate. Random residuals suggest a good fit, while patterns indicate non-linear relationships or heteroscedasticity. R-squared ($R^2$) measures how well the line explains the variance (closer to 1 is better).
Correlation Analysis
Scatter plots are excellent for visualizing correlation. Correlation measures the strength and direction of a linear relationship between two variables, ranging from -1 to +1.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy.stats import pearsonr, spearmanr
# Generate data with different correlation strengths
np.random.seed(42)
# Strong positive correlation
x1 = np.random.normal(0, 1, 100)
y1 = 2 * x1 + np.random.normal(0, 0.5, 100)
corr1, p1 = pearsonr(x1, y1)
# Weak positive correlation
x2 = np.random.normal(0, 1, 100)
y2 = 0.3 * x2 + np.random.normal(0, 1, 100)
corr2, p2 = pearsonr(x2, y2)
# Negative correlation
x3 = np.random.normal(0, 1, 100)
y3 = -2 * x3 + np.random.normal(0, 0.5, 100)
corr3, p3 = pearsonr(x3, y3)
# No correlation
x4 = np.random.normal(0, 1, 100)
y4 = np.random.normal(0, 1, 100)
corr4, p4 = pearsonr(x4, y4)
# Create subplots
fig, axes = plt.subplots(2, 2, figsize=(14, 12))
datasets = [
(x1, y1, corr1, 'Strong Positive Correlation', axes[0, 0]),
(x2, y2, corr2, 'Weak Positive Correlation', axes[0, 1]),
(x3, y3, corr3, 'Negative Correlation', axes[1, 0]),
(x4, y4, corr4, 'No Correlation', axes[1, 1])
]
for x, y, corr, title, ax in datasets:
ax.scatter(x, y, alpha=0.6, s=50)
ax.set_title(f'{title}\nr = {corr:.3f}, p = {pearsonr(x, y)[1]:.4f}', fontsize=12)
ax.grid(True, alpha=0.3)
# Add trend line
z = np.polyfit(x, y, 1)
p = np.poly1d(z)
ax.plot(x, p(x), "r--", alpha=0.8, linewidth=2)
plt.tight_layout()
plt.show()
# Correlation matrix visualization
data = pd.DataFrame({
'sqft': x1,
'price': y1,
'age': x2,
'condition': y2
})
correlation_matrix = data.corr()
print("\nCorrelation Matrix:")
print(correlation_matrix)
# Visualize correlation matrix
import seaborn as sns
plt.figure(figsize=(8, 6))
sns.heatmap(correlation_matrix, annot=True, cmap='coolwarm', center=0,
square=True, linewidths=1, cbar_kws={"shrink": 0.8})
plt.title("Correlation Matrix Heatmap", fontsize=14)
plt.tight_layout()
plt.show()Pearson correlation measures linear relationships, while Spearman correlation captures monotonic relationships (including non-linear). Always report both the correlation coefficient and $p$-value to assess statistical significance.
Advanced Visualization Techniques
Enhance scatter plots with additional techniques to reveal more insights from your data.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
# Generate sample data
np.random.seed(42)
n = 200
sqft = np.random.normal(1500, 500, n)
price = 100 * sqft + np.random.normal(0, 50000, n)
category = np.random.choice(['A', 'B', 'C'], size=n, p=[0.3, 0.4, 0.3])
df = pd.DataFrame({'sqft': sqft, 'price': price, 'category': category})
# 1. Facet Grid (Multiple scatter plots by category)
g = sns.FacetGrid(df, col='category', col_wrap=3, height=4)
g.map(plt.scatter, 'sqft', 'price', alpha=0.6, s=50)
g.set_axis_labels('Square Feet', 'Price ($)')
g.set_titles('Category {col_name}')
plt.suptitle('Scatter Plots by Category', y=1.02, fontsize=14)
plt.tight_layout()
plt.show()
# 2. Pair Plot (Multiple pairwise relationships)
df_extended = pd.DataFrame({
'sqft': sqft,
'price': price,
'age': np.random.normal(10, 5, n),
'condition': np.random.normal(3, 1, n)
})
sns.pairplot(df_extended, diag_kind='kde', height=2.5)
plt.suptitle('Pair Plot: All Pairwise Relationships', y=1.02, fontsize=14)
plt.tight_layout()
plt.show()
# 3. Hexbin plot (for large datasets)
np.random.seed(42)
large_x = np.random.normal(0, 1, 10000)
large_y = 2 * large_x + np.random.normal(0, 1, 10000)
plt.figure(figsize=(10, 6))
plt.hexbin(large_x, large_y, gridsize=30, cmap='Blues', mincnt=1)
plt.colorbar(label='Count')
plt.xlabel('X', fontsize=12)
plt.ylabel('Y', fontsize=12)
plt.title('Hexbin Plot (Large Dataset)', fontsize=14)
plt.tight_layout()
plt.show()
# 4. Contour plot with density
x = np.random.normal(0, 1, 1000)
y = 2 * x + np.random.normal(0, 1, 1000)
plt.figure(figsize=(10, 6))
# Density estimation
from scipy.stats import gaussian_kde
kde = gaussian_kde(np.vstack([x, y]))
xx, yy = np.meshgrid(np.linspace(x.min(), x.max(), 100),
np.linspace(y.min(), y.max(), 100))
density = kde(np.vstack([xx.ravel(), yy.ravel()])).reshape(xx.shape)
plt.contour(xx, yy, density, levels=10, colors='black', alpha=0.5)
plt.contourf(xx, yy, density, levels=10, cmap='Blues', alpha=0.6)
plt.scatter(x, y, alpha=0.3, s=20, c='red', edgecolors='black', linewidth=0.5)
plt.xlabel('X', fontsize=12)
plt.ylabel('Y', fontsize=12)
plt.title('Contour Plot with Density Estimation', fontsize=14)
plt.colorbar(label='Density')
plt.tight_layout()
plt.show()
# 5. Joint plot (scatter + marginal distributions)
sns.jointplot(data=df, x='sqft', y='price', kind='scatter',
alpha=0.6, height=8, marginal_kws=dict(bins=20))
plt.suptitle('Joint Plot: Scatter with Marginal Distributions', y=1.02, fontsize=14)
plt.tight_layout()
plt.show()These advanced techniques help reveal patterns in complex datasets. Facet grids enable categorical comparisons, pair plots show all pairwise relationships, and hexbin/contour plots handle large datasets efficiently.
Outlier Detection
Scatter plots are excellent for identifying outliers—points that deviate significantly from the general pattern. Outliers can indicate data quality issues, measurement errors, or genuinely unusual observations.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from scipy import stats
# Generate data with outliers
np.random.seed(42)
x = np.random.normal(0, 1, 100)
y = 2 * x + np.random.normal(0, 0.5, 100)
# Add some outliers
outliers_x = np.array([-3, 3, -2, 2.5])
outliers_y = np.array([-1, -1, 5, 4])
x = np.concatenate([x, outliers_x])
y = np.concatenate([y, outliers_y])
df = pd.DataFrame({'x': x, 'y': y})
# Method 1: Z-score based outlier detection
z_scores_x = np.abs(stats.zscore(df['x']))
z_scores_y = np.abs(stats.zscore(df['y']))
outliers_mask = (z_scores_x > 3) | (z_scores_y > 3)
# Method 2: IQR based outlier detection
Q1_x, Q3_x = df['x'].quantile([0.25, 0.75])
IQR_x = Q3_x - Q1_x
Q1_y, Q3_y = df['y'].quantile([0.25, 0.75])
IQR_y = Q3_y - Q1_y
outliers_iqr = (df['x'] < (Q1_x - 1.5 * IQR_x)) | (df['x'] > (Q3_x + 1.5 * IQR_x)) | \
(df['y'] < (Q1_y - 1.5 * IQR_y)) | (df['y'] > (Q3_y + 1.5 * IQR_y))
# Visualize with outliers highlighted
plt.figure(figsize=(12, 5))
# Plot 1: Z-score outliers
plt.subplot(1, 2, 1)
normal = df[~outliers_mask]
outliers = df[outliers_mask]
plt.scatter(normal['x'], normal['y'], alpha=0.6, s=50, label='Normal', c='blue')
plt.scatter(outliers['x'], outliers['y'], alpha=0.8, s=100, label='Outliers (Z-score)',
c='red', marker='x', linewidths=2)
plt.xlabel('X', fontsize=12)
plt.ylabel('Y', fontsize=12)
plt.title('Z-Score Outlier Detection', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
# Plot 2: IQR outliers
plt.subplot(1, 2, 2)
normal_iqr = df[~outliers_iqr]
outliers_iqr_data = df[outliers_iqr]
plt.scatter(normal_iqr['x'], normal_iqr['y'], alpha=0.6, s=50, label='Normal', c='blue')
plt.scatter(outliers_iqr_data['x'], outliers_iqr_data['y'], alpha=0.8, s=100,
label='Outliers (IQR)', c='orange', marker='s', linewidths=2)
plt.xlabel('X', fontsize=12)
plt.ylabel('Y', fontsize=12)
plt.title('IQR Outlier Detection', fontsize=14)
plt.legend()
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Annotate outliers
plt.figure(figsize=(10, 6))
plt.scatter(df['x'], df['y'], alpha=0.6, s=50)
for idx, row in df[outliers_mask].iterrows():
plt.annotate(f'Outlier {idx}', (row['x'], row['y']),
xytext=(10, 10), textcoords='offset points',
bbox=dict(boxstyle='round,pad=0.3', facecolor='yellow', alpha=0.7),
arrowprops=dict(arrowstyle='->', connectionstyle='arc3,rad=0'))
plt.xlabel('X', fontsize=12)
plt.ylabel('Y', fontsize=12)
plt.title('Annotated Outliers', fontsize=14)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
print(f"Number of outliers (Z-score): {outliers_mask.sum()}")
print(f"Number of outliers (IQR): {outliers_iqr.sum()}")Always investigate outliers before removing them. They may represent important anomalies, data entry errors, or measurement issues. Document outlier handling decisions and share guidelines through info.studygrid@gmail.com.
Standardization and Scaling
When variables have significantly different ranges, standardizing them can make relationships clearer and improve visualization readability.
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.preprocessing import StandardScaler
# Generate data with different scales
np.random.seed(42)
feature1 = np.random.normal(1000, 200, 100) # Large scale
feature2 = np.random.normal(5, 1, 100) # Small scale
target = 0.5 * feature1 + 10 * feature2 + np.random.normal(0, 50, 100)
df = pd.DataFrame({
'feature1': feature1,
'feature2': feature2,
'target': target
})
# Original scale
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
axes[0].scatter(df['feature1'], df['target'], alpha=0.6, s=50)
axes[0].set_xlabel('Feature 1', fontsize=12)
axes[0].set_ylabel('Target', fontsize=12)
axes[0].set_title('Original Scale', fontsize=14)
axes[0].grid(True, alpha=0.3)
axes[1].scatter(df['feature2'], df['target'], alpha=0.6, s=50)
axes[1].set_xlabel('Feature 2', fontsize=12)
axes[1].set_ylabel('Target', fontsize=12)
axes[1].set_title('Original Scale (Different Range)', fontsize=14)
axes[1].grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
# Standardized scale
scaler = StandardScaler()
df_scaled = pd.DataFrame(
scaler.fit_transform(df),
columns=df.columns
)
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
axes[0].scatter(df_scaled['feature1'], df_scaled['target'], alpha=0.6, s=50)
axes[0].set_xlabel('Feature 1 (Standardized)', fontsize=12)
axes[0].set_ylabel('Target (Standardized)', fontsize=12)
axes[0].set_title('Standardized Scale', fontsize=14)
axes[0].grid(True, alpha=0.3)
axes[0].axhline(y=0, color='r', linestyle='--', alpha=0.5)
axes[0].axvline(x=0, color='r', linestyle='--', alpha=0.5)
axes[1].scatter(df_scaled['feature2'], df_scaled['target'], alpha=0.6, s=50)
axes[1].set_xlabel('Feature 2 (Standardized)', fontsize=12)
axes[1].set_ylabel('Target (Standardized)', fontsize=12)
axes[1].set_title('Standardized Scale (Comparable Ranges)', fontsize=14)
axes[1].grid(True, alpha=0.3)
axes[1].axhline(y=0, color='r', linestyle='--', alpha=0.5)
axes[1].axvline(x=0, color='r', linestyle='--', alpha=0.5)
plt.tight_layout()
plt.show()
print("Original scales:")
print(f"Feature 1: mean={df['feature1'].mean():.2f}, std={df['feature1'].std():.2f}")
print(f"Feature 2: mean={df['feature2'].mean():.2f}, std={df['feature2'].std():.2f}")
print(f"Target: mean={df['target'].mean():.2f}, std={df['target'].std():.2f}")
print("\nStandardized scales:")
print(f"Feature 1: mean={df_scaled['feature1'].mean():.4f}, std={df_scaled['feature1'].std():.4f}")
print(f"Feature 2: mean={df_scaled['feature2'].mean():.4f}, std={df_scaled['feature2'].std():.4f}")
print(f"Target: mean={df_scaled['target'].mean():.4f}, std={df_scaled['target'].std():.4f}")Standardization transforms variables to have mean 0 and standard deviation 1, making relationships clearer when variables have different scales. This is especially important when comparing relationships across features with vastly different ranges.
Best Practices
- Use appropriate point sizes: Adjust
sparameter based on dataset size. Larger datasets need smaller, more transparent points. - Handle overplotting: Use transparency (
alpha), jittering, or density plots for large datasets with overlapping points. - Standardize when needed: Plot standardized values when ranges differ significantly to make relationships comparable.
- Use facet grids: Compare multiple categories side by side using Seaborn's
FacetGridor Matplotlib subplots. - Annotate outliers: Mark and investigate notable outliers that might indicate data quality issues.
- Add trend lines: Include regression lines or LOESS curves to highlight relationships, but always check residuals.
- Color encoding: Use color to add a third dimension, but ensure color palettes are accessible and consistent across visualizations.
- Label clearly: Always include axis labels, units, titles, and legends for context.
- Check assumptions: Verify linearity, homoscedasticity, and normality when interpreting relationships.
- Document conventions: Share visualization guidelines and color schemes through
info.studygrid@gmail.com.
Next Steps
Advance to linear regression to learn how to model linear relationships discovered in scatter plots.
Try It Yourself
Exercise 1: Plot car age against speed for the given data and describe the trend.
Show solution
import matplotlib.pyplot as plt
age = [5, 7, 8, 7, 2, 17, 2, 9, 4, 11, 12, 9, 6]
speed = [99, 86, 87, 88, 111, 86, 103, 87, 94, 78, 77, 85, 86]
plt.scatter(age, speed)
plt.xlabel("age"); plt.ylabel("speed")
plt.show() # older cars tend to be slower -> negative trendExercise 2: Does a scatter plot showing correlation prove causation?
Show solution
No. Correlation shows two variables move together, but a third factor may cause both, or the link may be coincidental.
Key Takeaways
- Scatter plots reveal relationships between two numeric variables.
- Look for direction, strength, and clusters or outliers.
- Correlation is not causation.
📘 Real-World Deep Dive
Knowing <strong>ML Scatter Plot (scikit-learn)</strong> well is what turns scikit-learn from a curiosity into a daily tool — you'll reach for it in nearly every real project.
Real-Life Scenario
An end-to-end usage of ML Scatter Plot that you'd actually see in a data pipeline or analytics notebook.
Real-Life Example
import numpy as np
import matplotlib.pyplot as plt
x = np.random.rand(50); y = 2*x + np.random.normal(0, 0.2, 50)
plt.scatter(x, y); plt.show()Expected Output
(no output)Common mistakes
fitexpects numeric arrays;OneHotEncoder/LabelEncoderare easy to forget for categorical features.- Calling
predicton a model trained on unscaled data and then scaling inputs at inference time silently degrades accuracy. train_test_split(X, y)requires both arrays;train_test_split(X)for unsupervised learning slips past static checkers.- Treating ML Scatter Plot as a black box without reading the docs — the API has subtle defaults that bite when you scale.
🚀 Performance & Best Practices
- Wrap preprocessing + estimator in a
Pipelinesofit/predictstay reproducible. - Use
joblib/picklefor serialising models, not the entire Python state. - For > 100 k rows, switch to
HistGradientBoostingClassifierorcuML. - When working with scikit-learn, prefer vectorised / batched operations over Python loops.
🧪 Try It Yourself
- Reproduce the snippet on a representative slice of your own data.
- Profile the snippet with
cProfileortimeitand find the single biggest improvement. - Generalise the snippet into a small, reusable function you can drop into future projects.