Classical Machine Learning
Linear & Logistic Regression
Linear regression predicts a numeric value by fitting a weighted sum of features; logistic regression predicts class probability by passing a weighted sum through a sigmoid or softmax. They are boring in the best way: fast, interpretable, and excellent baselines.
- Linear regression models a continuous target as a weighted sum
- Logistic regression is a classifier despite the name
- Coefficients are useful but easy to misread
- Regularization turns simple linear models into practical tools
- Linear models are calibration-friendly baselines
- Use them before heavier models
Linear/logistic regression should usually be the first thing you try on tabular data. Not because they are always best, but because they expose whether the signal is simple, whether features are sane, and whether the extra complexity of tree ensembles or neural networks is actually buying anything.
| Model | Target | Output | Typical loss |
|---|---|---|---|
| Linear regression | Continuous number | Numeric prediction | Mean squared error / absolute error |
| Logistic regression | Class label | Class probability | Cross-entropy / log loss |
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
preprocess = ColumnTransformer([
("num", StandardScaler(), numeric_columns),
("cat", OneHotEncoder(handle_unknown="ignore"), categorical_columns),
])
model = Pipeline([
("prep", preprocess),
("clf", LogisticRegression(max_iter=1000, class_weight="balanced")),
])
scores = cross_val_score(model, X, y, cv=5, scoring="f1")
print(scores.mean())