Classical Machine Learning

Feature Engineering

Feature engineering turns raw data into model-usable signal. In classical ML, better features often beat fancier algorithms; in deep learning, learned representations reduce but do not eliminate feature thinking.

Features define what the model can notice
Categorical variables need deliberate encoding
Aggregates are often high-value features
Time awareness prevents leakage
Pipelines protect train/serve consistency
Feature quality is often product knowledge

Feature engineering moves
Move	Example	Risk
Scaling	Standardize numeric inputs	Fitting scaler on all data leaks validation/test
Encoding	One-hot country or product tier	High-cardinality explosion
Rolling aggregate	purchases_last_30_days	Future leakage
Ratio	price_per_square_meter	Divide-by-zero / noisy denominator
Text feature	TF-IDF or embedding	Vocabulary drift / privacy
Interaction	plan_type × region	Overfitting rare combinations

Keep preprocessing inside the validation path

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression

preprocess = ColumnTransformer([
    ("num", StandardScaler(), numeric_cols),
    ("cat", OneHotEncoder(handle_unknown="ignore"), categorical_cols),
])

pipeline = Pipeline([
    ("features", preprocess),
    ("model", LogisticRegression(max_iter=1000)),
])

# cross_val_score fits preprocessing separately inside each fold.
scores = cross_val_score(pipeline, X, y, cv=5)

This avoids fitting preprocessing on validation/test data by accident.