Lesson menuPython TutorialHideShow

Python Tutorial

Logistic Regression

Despite its name, a classification model — it predicts the probability that an observation belongs to a class.

← Hierarchical ClusteringGrid Search →

From Line to Probability

Linear regression outputs any real number, but a probability must lie between 0 and 1. Logistic regression passes the linear combination through the sigmoid function:

p = 1 / (1 + e^(−(β₀ + β₁x)))

The output is the probability of the positive class. Applying a threshold (usually 0.5) turns that probability into a class label.

A Binary Classifier

import numpy as np
from sklearn.linear_model import LogisticRegression

# tumor size (cm) -> malignant (1) or benign (0)
X = np.array([3.78, 2.44, 2.09, 0.14, 1.72, 1.65, 4.92,
              4.37, 4.96, 4.52, 3.69, 5.88]).reshape(-1, 1)
y = np.array([0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1])

model = LogisticRegression()
model.fit(X, y)

print(model.predict([[3.46]]))            # class label
print(model.predict_proba([[3.46]]))      # [P(benign), P(malignant)]

Use predict_proba when you need calibrated probabilities (e.g. ranking risk), and predict when you only need the label.

Interpreting Coefficients (Odds)

Exponentiating a coefficient gives the odds ratio — the multiplicative change in odds for a one-unit increase in the feature.

log_odds = model.coef_[0][0]
odds = np.exp(log_odds)
print(f"Each extra cm multiplies the odds of malignancy by {odds:.2f}")

Scaling and Regularization

Logistic regression is gradient-based, so scale your features. scikit-learn regularizes by default; C is the inverse of regularization strength (smaller C = stronger regularization).

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

clf = make_pipeline(
    StandardScaler(),
    LogisticRegression(C=0.5, penalty="l2", max_iter=1000))
clf.fit(X, y)

Multiclass Classification

For more than two classes, scikit-learn automatically fits a multinomial (softmax) model.

from sklearn.datasets import load_iris

iris = load_iris()
multi = LogisticRegression(max_iter=1000)
multi.fit(iris.data, iris.target)
print(multi.predict(iris.data[:3]))

Evaluating the Model

Judge a classifier with the confusion matrix, precision/recall, and the ROC-AUC score — not just accuracy, especially on imbalanced data. Use class_weight="balanced" when one class is rare.

from sklearn.metrics import roc_auc_score

probs = model.predict_proba(X)[:, 1]
print("AUC:", roc_auc_score(y, probs).round(3))

Best Practices

  • Scale features and increase max_iter if the solver warns about convergence.
  • Tune C with cross-validation to balance bias and variance.
  • Use class_weight="balanced" for imbalanced targets.
  • Adjust the probability threshold to trade precision against recall for your use case.

Try It Yourself

Exercise 1: Train logistic regression on iris (2 classes) and print the predicted probability for one sample.

Show solution
from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
mask = y < 2                       # keep only classes 0 and 1
m = LogisticRegression(max_iter=1000).fit(X[mask], y[mask])
print(m.predict_proba(X[mask][:1]).round(3))

Exercise 2: Despite its name, is logistic regression used for regression or classification?

Show solution

Classification — it predicts the probability of class membership, then thresholds it into a label.

📘 Real-World Deep Dive

Knowing <strong>ML Logistic Regression (scikit-learn)</strong> well is what turns scikit-learn from a curiosity into a daily tool — you'll reach for it in nearly every real project.

Real-Life Scenario

An end-to-end usage of ML Logistic Regression that you'd actually see in a data pipeline or analytics notebook.

Real-Life Example

from sklearn.datasets import load_iris
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
Xtr, Xte, ytr, yte = train_test_split(X, y, random_state=0, stratify=y)
clf = LogisticRegression(max_iter=200).fit(Xtr, ytr)
print("test score:", round(clf.score(Xte, yte), 3))

Expected Output

(see source)

Common mistakes

  • fit expects numeric arrays; OneHotEncoder / LabelEncoder are easy to forget for categorical features.
  • Calling predict on a model trained on unscaled data and then scaling inputs at inference time silently degrades accuracy.
  • train_test_split(X, y) requires both arrays; train_test_split(X) for unsupervised learning slips past static checkers.
  • Treating ML Logistic Regression as a black box without reading the docs — the API has subtle defaults that bite when you scale.

🚀 Performance & Best Practices

  • Wrap preprocessing + estimator in a Pipeline so fit / predict stay reproducible.
  • Use joblib / pickle for serialising models, not the entire Python state.
  • For > 100 k rows, switch to HistGradientBoostingClassifier or cuML.
  • When working with scikit-learn, prefer vectorised / batched operations over Python loops.

🧪 Try It Yourself

  1. Reproduce the snippet on a representative slice of your own data.
  2. Profile the snippet with cProfile or timeit and find the single biggest improvement.
  3. Generalise the snippet into a small, reusable function you can drop into future projects.

FAQ: Logistic Regression

Common questions about this page.

What is Logistic Regression?

Logistic Regression is a Machine Learning lesson that explains logistic regression in Python. Despite its name, a classification model — it predicts the probability that an observation belongs to a class. Copy the samples and run them in the Python... It is written for beginners who want a clear definition and working examples.

Should I run logistic regression examples locally for better learning?

Yes. Use the browser editor on StudyGrid for a quick check, then Download the example and run it on your computer. Local runs show real errors and the real toolchain, which is one of the fastest ways to learn logistic regression in this Machine Learning Python lesson (Logistic Regression).

How do I use logistic regression in Python?

To use logistic regression in Python, follow the examples on this StudyGrid page. Copy a snippet, run it in the browser, then Download and run it locally for better learning. Change the values and compare the output.

What is the syntax of logistic regression?

This Logistic Regression tutorial shows logistic regression syntax with short Python examples. Use the code blocks in this lesson for the exact statements, then try them in your editor.

Logistic Regression example for beginners

Yes. This page includes a beginner logistic regression example you can copy and run. It is designed for searches such as "logistic regression for beginners", "logistic regression example", and "how to use logistic regression".

What are common mistakes with logistic regression?

Common logistic regression mistakes include wrong syntax, mixing types, and skipping practice. Work through this Machine Learning chapter in order, run every example, and check the output before moving on.

Why should I learn logistic regression?

Logistic Regression is used in real Python work. Learning logistic regression helps you write clearer programs and continue the Machine Learning tutorial on StudyGrid.

Is Logistic Regression free to learn online?

Yes. You can learn logistic regression free on StudyGrid (studygrid.in). This chapter is part of the Machine Learning path and includes examples, syntax, and next-step links.