Feature scaling and categorical encoding are essential data preprocessing steps in machine learning. In this chapter, we cover the core concepts, mathematical formulas, and practical Python implementation using Scikit-Learn.

1. Feature Scaling Techniques

Machine learning models that rely on distance metrics (such as KNN, SVM, K-Means) or gradient descent optimization (such as Linear Regression, Logistic Regression, and Neural Networks) are sensitive to the scale of input features.

Min-Max Normalization

Min-Max Scaling rescales the features to a fixed range, typically [0, 1].

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
X_scaled = scaler.fit_transform(X)

Z-Score Standardization (StandardScaler)

Standardization shifts the distribution to have a mean of 0 and a standard deviation of 1.

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

RobustScaler

RobustScaler uses the median and Interquartile Range (IQR), making it robust to outliers.

from sklearn.preprocessing import RobustScaler
scaler = RobustScaler()
X_scaled = scaler.fit_transform(X)

2. Categorical Encoding Methods

One-Hot Encoding

One-Hot Encoding creates binary indicator columns for nominal categorical variables.

from sklearn.preprocessing import OneHotEncoder
encoder = OneHotEncoder(sparse_output=False)
X_encoded = encoder.fit_transform(categories)

Ordinal Encoding

Ordinal Encoding assigns integers to categorical variables with an inherent order (e.g., Low, Medium, High).

from sklearn.preprocessing import OrdinalEncoder
encoder = OrdinalEncoder(categories=[['Low', 'Medium', 'High']])
X_encoded = encoder.fit_transform(ratings)