Getting everything ready for you…

Loading courses, problems and your progress. This takes just a moment.

0s

Machine Learning (scikit-learn) Documentation

Practical machine learning in Python — preparing data, regression, classification, validation, tuning and serving models.

Machine learning (ML) lets programs learn patterns from data instead of following hand-written rules. This page uses Python's scikit-learn to go from raw data to a tested model. You should know Python and Pandas & NumPy first.

1. Introduction

Type Learns from Examples
Supervised Data with answers (labels) Spam detection, price prediction
Unsupervised Data without labels Customer groups, anomaly detection
Reinforcement Rewards from trial and error Game playing, robotics

Supervised learning has two jobs:

  • Regression predicts a number (house price).
  • Classification predicts a category (spam or not).
Term Meaning
Feature An input column (X)
Label / target What you predict (y)
Training Fitting the model to data
Inference Using the trained model

2. The ML Workflow

  1. Define the problem and how to measure success.
  2. Collect and explore the data.
  3. Clean it and prepare features.
  4. Split into train and test sets.
  5. Train a simple baseline, then better models.
  6. Evaluate on the test set.
  7. Tune, then deploy and monitor.
pip install scikit-learn pandas numpy matplotlib

3. Preparing Data

import pandas as pd
from sklearn.model_selection import train_test_split

df = pd.read_csv("students.csv")
df.info()
print(df.isna().sum())

X = df.drop(columns=["placed"])
y = df["placed"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
Step Tool
Missing values SimpleImputer (mean, median, most frequent)
Text categories to numbers OneHotEncoder
Same scale for numbers StandardScaler

A pipeline applies these steps the same way every time:

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline, make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric = ["cgpa", "projects"]
categorical = ["branch"]

prep = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), numeric),
    ("cat", OneHotEncoder(handle_unknown="ignore"), categorical),
])
model = Pipeline([("prep", prep), ("clf", LogisticRegression(max_iter=1000))])
model.fit(X_train, y_train)
print(model.score(X_test, y_test))

Common mistake (data leakage): fitting a scaler on all the data lets test information leak into training. Split first, and use a pipeline.

4. Regression

from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, r2_score

reg = LinearRegression()
reg.fit(X_train, y_train)
pred = reg.predict(X_test)

print("MAE:", mean_absolute_error(y_test, pred))
print("R2:", r2_score(y_test, pred))

Linear regression fits y = w1·x1 + w2·x2 + … + b by minimising squared error.

Metric Meaning
MAE Average size of the error, in real units
RMSE Like MAE, but punishes big errors more
R² Share of variation explained (1 is perfect)

5. Classification

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix

clf = LogisticRegression(max_iter=1000)
clf.fit(X_train, y_train)
pred = clf.predict(X_test)

print(confusion_matrix(y_test, pred))
print(classification_report(y_test, pred))

Despite the name, logistic regression classifies: it outputs a probability.

Metric Formula Use when
Accuracy correct / total Classes are balanced
Precision TP / (TP + FP) False alarms are costly (spam filter)
Recall TP / (TP + FN) Misses are costly (disease test)
F1 Balance of precision and recall Classes are imbalanced

Why accuracy can lie: if 99% of transactions are genuine, a model that always says "genuine" scores 99% and catches no fraud.

6. Common Algorithms

Algorithm Idea Good for
Linear / logistic regression Weighted sum of features Fast baseline, easy to explain
K-nearest neighbours Vote of the closest points Small datasets
Naive Bayes Probabilities, assuming independent features Text, spam
Decision tree A series of if/else splits Easy to explain; overfits alone
Random forest Many trees voting Strong default for tables
Gradient boosting (XGBoost, LightGBM) Trees that fix earlier errors Often the best on tabular data
SVM Widest margin between classes Medium-sized data
from sklearn.ensemble import RandomForestClassifier

forest = RandomForestClassifier(n_estimators=200, random_state=42)
forest.fit(X_train, y_train)
print(forest.score(X_test, y_test))

7. Overfitting and Validation

  • Underfitting: too simple; poor on training and test data.
  • Overfitting: memorises the training data; great on training, poor on test.

Bias–variance trade-off: simple models have high bias, complex models have high variance. You want the balance.

Ways to fight overfitting: more data, a simpler model, regularisation (Ridge, Lasso), limiting tree depth, and cross-validation.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(forest, X, y, cv=5, scoring="f1")
print(scores.mean(), scores.std())

K-fold cross-validation trains K times, each time testing on a different slice, which gives a more reliable score.

8. Tuning Hyperparameters

Hyperparameters are settings you choose (such as tree depth), not values the model learns.

from sklearn.model_selection import GridSearchCV

grid = GridSearchCV(
    RandomForestClassifier(random_state=42),
    {"n_estimators": [100, 300], "max_depth": [5, 10, None]},
    cv=5, scoring="f1",
)
grid.fit(X_train, y_train)
print(grid.best_params_, grid.best_score_)

Use the test set once, at the very end. Tuning against it makes the score too optimistic.

9. Unsupervised Learning

from sklearn.cluster import KMeans
from sklearn.decomposition import PCA

km = KMeans(n_clusters=3, n_init="auto", random_state=42)
groups = km.fit_predict(X_scaled)

pca = PCA(n_components=2)              # shrink many features to 2, e.g. to plot them
points = pca.fit_transform(X_scaled)
  • K-means groups points around K centres. Scale the data first, and choose K with the elbow method.
  • PCA reduces the number of features while keeping most of the variation.

10. Neural Networks in Brief

A neural network stacks layers of weighted sums with non-linear activations (such as ReLU). Training repeats these steps: predict, measure the loss, use backpropagation to find gradients, and update weights with gradient descent.

Network Used for
Feed-forward Tabular data
CNN Images
Transformer Text; the basis of LLMs

Use PyTorch or TensorFlow for deep learning. On ordinary tables, tree models often win with less work.

11. Saving and Serving a Model

import joblib

joblib.dump(model, "model.joblib")
model = joblib.load("model.joblib")
from flask import Flask, jsonify, request
import pandas as pd

app = Flask(__name__)

@app.post("/predict")
def predict():
    row = pd.DataFrame([request.json])
    return jsonify(prediction=int(model.predict(row)[0]))

After deployment, watch for data drift: real data changes over time, and accuracy falls until you retrain.

12. Practice Problems

  1. Predict house prices with linear regression, and report MAE.
  2. Classify the Iris flowers (load_iris) with KNN, and try different K values.
  3. Build a spam classifier using TfidfVectorizer and Naive Bayes.
  4. Predict Titanic survival with a full pipeline and cross-validation.
  5. Group customers with K-means, and describe each group.

13. Interview Questions

Supervised vs unsupervised? Supervised learns from labelled data; unsupervised finds structure without labels.

What is overfitting? Doing well on training data but badly on new data, because the model memorised noise.

Bias vs variance? Bias is error from too-simple assumptions; variance is sensitivity to the training data.

Precision vs recall? Precision: of those predicted positive, how many were right. Recall: of the real positives, how many were found.

Why split the data? To measure performance on data the model has never seen.

What is cross-validation? Training and testing on several splits for a more reliable estimate.

L1 vs L2 regularisation? L1 (Lasso) can drive weights to zero, removing features; L2 (Ridge) shrinks them all.

Bagging vs boosting? Bagging trains models independently and averages them (random forest); boosting trains them in order, each fixing earlier errors.

What is data leakage? Information from outside the training data getting into the model, which gives scores that are too good.

How do you handle imbalanced classes? Use F1 or recall, class weights, and over- or under-sampling.

14. Cheat Sheet & References

The scikit-learn pattern

model = SomeModel()
model.fit(X_train, y_train)
pred = model.predict(X_test)
model.score(X_test, y_test)

Pick a model

Number to predict → Linear Regression → Random Forest / Gradient Boosting
Category          → Logistic Regression → Random Forest / Gradient Boosting
Text              → TF-IDF + Naive Bayes or Logistic Regression
No labels         → K-means, PCA

Pick a metric

Regression: MAE, RMSE, R²
Balanced classes: accuracy
False alarms costly: precision
Misses costly: recall
Imbalanced: F1

References