Machine Learning (scikit-learn) Documentation
Practical machine learning in Python — preparing data, regression, classification, validation, tuning and serving models.
Machine learning (ML) lets programs learn patterns from data instead of following hand-written rules. This page uses Python's scikit-learn to go from raw data to a tested model. You should know Python and Pandas & NumPy first.
1. Introduction
| Type | Learns from | Examples |
|---|---|---|
| Supervised | Data with answers (labels) | Spam detection, price prediction |
| Unsupervised | Data without labels | Customer groups, anomaly detection |
| Reinforcement | Rewards from trial and error | Game playing, robotics |
Supervised learning has two jobs:
- Regression predicts a number (house price).
- Classification predicts a category (spam or not).
| Term | Meaning |
|---|---|
| Feature | An input column (X) |
| Label / target | What you predict (y) |
| Training | Fitting the model to data |
| Inference | Using the trained model |
2. The ML Workflow
- Define the problem and how to measure success.
- Collect and explore the data.
- Clean it and prepare features.
- Split into train and test sets.
- Train a simple baseline, then better models.
- Evaluate on the test set.
- Tune, then deploy and monitor.
pip install scikit-learn pandas numpy matplotlib
3. Preparing Data
import pandas as pd
from sklearn.model_selection import train_test_split
df = pd.read_csv("students.csv")
df.info()
print(df.isna().sum())
X = df.drop(columns=["placed"])
y = df["placed"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
| Step | Tool |
|---|---|
| Missing values | SimpleImputer (mean, median, most frequent) |
| Text categories to numbers | OneHotEncoder |
| Same scale for numbers | StandardScaler |
A pipeline applies these steps the same way every time:
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
numeric = ["cgpa", "projects"]
categorical = ["branch"]
prep = ColumnTransformer([
("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), numeric),
("cat", OneHotEncoder(handle_unknown="ignore"), categorical),
])
model = Pipeline([("prep", prep), ("clf", LogisticRegression(max_iter=1000))])
model.fit(X_train, y_train)
print(model.score(X_test, y_test))
Common mistake (data leakage): fitting a scaler on all the data lets test information leak into training. Split first, and use a pipeline.
4. Regression
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score
reg = LinearRegression()
reg.fit(X_train, y_train)
pred = reg.predict(X_test)
print("MAE:", mean_absolute_error(y_test, pred))
print("R2:", r2_score(y_test, pred))
Linear regression fits y = w1·x1 + w2·x2 + … + b by minimising squared error.
| Metric | Meaning |
|---|---|
| MAE | Average size of the error, in real units |
| RMSE | Like MAE, but punishes big errors more |
| R² | Share of variation explained (1 is perfect) |
5. Classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix
clf = LogisticRegression(max_iter=1000)
clf.fit(X_train, y_train)
pred = clf.predict(X_test)
print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred))
Despite the name, logistic regression classifies: it outputs a probability.
| Metric | Formula | Use when |
|---|---|---|
| Accuracy | correct / total | Classes are balanced |
| Precision | TP / (TP + FP) | False alarms are costly (spam filter) |
| Recall | TP / (TP + FN) | Misses are costly (disease test) |
| F1 | Balance of precision and recall | Classes are imbalanced |
Why accuracy can lie: if 99% of transactions are genuine, a model that always says "genuine" scores 99% and catches no fraud.
6. Common Algorithms
| Algorithm | Idea | Good for |
|---|---|---|
| Linear / logistic regression | Weighted sum of features | Fast baseline, easy to explain |
| K-nearest neighbours | Vote of the closest points | Small datasets |
| Naive Bayes | Probabilities, assuming independent features | Text, spam |
| Decision tree | A series of if/else splits | Easy to explain; overfits alone |
| Random forest | Many trees voting | Strong default for tables |
| Gradient boosting (XGBoost, LightGBM) | Trees that fix earlier errors | Often the best on tabular data |
| SVM | Widest margin between classes | Medium-sized data |
from sklearn.ensemble import RandomForestClassifier
forest = RandomForestClassifier(n_estimators=200, random_state=42)
forest.fit(X_train, y_train)
print(forest.score(X_test, y_test))
7. Overfitting and Validation
- Underfitting: too simple; poor on training and test data.
- Overfitting: memorises the training data; great on training, poor on test.
Bias–variance trade-off: simple models have high bias, complex models have high variance. You want the balance.
Ways to fight overfitting: more data, a simpler model, regularisation (Ridge, Lasso), limiting tree depth, and cross-validation.
from sklearn.model_selection import cross_val_score
scores = cross_val_score(forest, X, y, cv=5, scoring="f1")
print(scores.mean(), scores.std())
K-fold cross-validation trains K times, each time testing on a different slice, which gives a more reliable score.
8. Tuning Hyperparameters
Hyperparameters are settings you choose (such as tree depth), not values the model learns.
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(
RandomForestClassifier(random_state=42),
{"n_estimators": [100, 300], "max_depth": [5, 10, None]},
cv=5, scoring="f1",
)
grid.fit(X_train, y_train)
print(grid.best_params_, grid.best_score_)
Use the test set once, at the very end. Tuning against it makes the score too optimistic.
9. Unsupervised Learning
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
km = KMeans(n_clusters=3, n_init="auto", random_state=42)
groups = km.fit_predict(X_scaled)
pca = PCA(n_components=2) # shrink many features to 2, e.g. to plot them
points = pca.fit_transform(X_scaled)
- K-means groups points around K centres. Scale the data first, and choose K with the elbow method.
- PCA reduces the number of features while keeping most of the variation.
10. Neural Networks in Brief
A neural network stacks layers of weighted sums with non-linear activations (such as ReLU). Training repeats these steps: predict, measure the loss, use backpropagation to find gradients, and update weights with gradient descent.
| Network | Used for |
|---|---|
| Feed-forward | Tabular data |
| CNN | Images |
| Transformer | Text; the basis of LLMs |
Use PyTorch or TensorFlow for deep learning. On ordinary tables, tree models often win with less work.
11. Saving and Serving a Model
import joblib
joblib.dump(model, "model.joblib")
model = joblib.load("model.joblib")
from flask import Flask, jsonify, request
import pandas as pd
app = Flask(__name__)
@app.post("/predict")
def predict():
row = pd.DataFrame([request.json])
return jsonify(prediction=int(model.predict(row)[0]))
After deployment, watch for data drift: real data changes over time, and accuracy falls until you retrain.
12. Practice Problems
- Predict house prices with linear regression, and report MAE.
- Classify the Iris flowers (
load_iris) with KNN, and try different K values. - Build a spam classifier using
TfidfVectorizerand Naive Bayes. - Predict Titanic survival with a full pipeline and cross-validation.
- Group customers with K-means, and describe each group.
13. Interview Questions
Supervised vs unsupervised? Supervised learns from labelled data; unsupervised finds structure without labels.
What is overfitting? Doing well on training data but badly on new data, because the model memorised noise.
Bias vs variance? Bias is error from too-simple assumptions; variance is sensitivity to the training data.
Precision vs recall? Precision: of those predicted positive, how many were right. Recall: of the real positives, how many were found.
Why split the data? To measure performance on data the model has never seen.
What is cross-validation? Training and testing on several splits for a more reliable estimate.
L1 vs L2 regularisation? L1 (Lasso) can drive weights to zero, removing features; L2 (Ridge) shrinks them all.
Bagging vs boosting? Bagging trains models independently and averages them (random forest); boosting trains them in order, each fixing earlier errors.
What is data leakage? Information from outside the training data getting into the model, which gives scores that are too good.
How do you handle imbalanced classes? Use F1 or recall, class weights, and over- or under-sampling.
14. Cheat Sheet & References
The scikit-learn pattern
model = SomeModel()
model.fit(X_train, y_train)
pred = model.predict(X_test)
model.score(X_test, y_test)
Pick a model
Number to predict → Linear Regression → Random Forest / Gradient Boosting
Category → Logistic Regression → Random Forest / Gradient Boosting
Text → TF-IDF + Naive Bayes or Logistic Regression
No labels → K-means, PCA
Pick a metric
Regression: MAE, RMSE, R²
Balanced classes: accuracy
False alarms costly: precision
Misses costly: recall
Imbalanced: F1