Percentile
Rank data points, set thresholds, and detect outliers using percentiles and quantiles.
Understanding Percentiles
The nth percentile is the value below which $n\%$ of observations fall. Percentiles support decision rules (for example, flagging the top 5% of risk scores) and summarize distributions succinctly.
Computing Percentiles
import numpy as np
import pandas as pd
values = np.array([12, 15, 14, 10, 18, 12, 20, 25])
p90 = np.percentile(values, 90)
quartiles = np.percentile(values, [25, 50, 75])
series = pd.Series(values)
quantile_0_9 = series.quantile(0.9)
print(p90, quartiles, quantile_0_9)np.percentile accepts percentages, while Pandas quantile uses fractions between $0$ and $1$.
Interpolation Methods
Percentile functions offer interpolation strategies (linear, midpoint, nearest, etc.). Choose the method that aligns with business rules and document the decision through info.studygrid@gmail.com.
Applications
- Set anomaly thresholds (for example, flag values above the 99th percentile).
- Summarize skewed distributions when mean/SD are misleading.
- Compute the interquartile range (IQR) for box plots: IQR = Q₃ − Q₁.
Best Practices
- Ensure data is sorted numerically before manual percentile calculations.
- Use consistent interpolation when comparing reports over time.
- Combine percentile metrics with visualizations (CDF, box plots) for richer insight.
Next Steps
Examine data distributions next to visualize how values are spread across the range.
Try It Yourself
Exercise 1: Find the 75th percentile of [1, 2, 3, 4, 5, 6, 7, 8, 9, 10].
Show solution
import numpy as np
print(np.percentile(range(1, 11), 75)) # 7.75Exercise 2: If your exam score is at the 90th percentile, what does that mean?
Show solution
You scored higher than about 90% of test-takers — only ~10% did better.
Key Takeaways
- A percentile is the value below which a given percentage of data falls.
- The 50th percentile is the median.
- IQR = Q₃ − Q₁ measures the middle 50% spread and flags outliers.
📘 Real-World Deep Dive
Knowing <strong>ML Percentile (scikit-learn)</strong> well is what turns scikit-learn from a curiosity into a daily tool — you'll reach for it in nearly every real project.
Real-Life Scenario
An end-to-end usage of ML Percentile that you'd actually see in a data pipeline or analytics notebook.
Real-Life Example
import numpy as np
rng = np.random.default_rng(0)
data = rng.normal(0, 1, 1000)
for q in [10, 50, 90, 99]:
print(f"p{q}: {np.percentile(data, q):.3f}")Expected Output
(see source)Common mistakes
fitexpects numeric arrays;OneHotEncoder/LabelEncoderare easy to forget for categorical features.- Calling
predicton a model trained on unscaled data and then scaling inputs at inference time silently degrades accuracy. train_test_split(X, y)requires both arrays;train_test_split(X)for unsupervised learning slips past static checkers.- Treating ML Percentile as a black box without reading the docs — the API has subtle defaults that bite when you scale.
🚀 Performance & Best Practices
- Wrap preprocessing + estimator in a
Pipelinesofit/predictstay reproducible. - Use
joblib/picklefor serialising models, not the entire Python state. - For > 100 k rows, switch to
HistGradientBoostingClassifierorcuML. - When working with scikit-learn, prefer vectorised / batched operations over Python loops.
🧪 Try It Yourself
- Reproduce the snippet on a representative slice of your own data.
- Profile the snippet with
cProfileortimeitand find the single biggest improvement. - Generalise the snippet into a small, reusable function you can drop into future projects.