Lecture 12 — Classical Recognition & Neural Networks

Lecture 12

Dr. Muhammad Kazim

Department of Intelligent Systems, University of Lahore

Agenda

Part A — Classical recognition: the sliding-window paradigm, HOG + linear SVM, Haar cascades, bag-of-visual-words.
Part B — Neural network foundations: perceptron → MLP, loss functions, back-propagation, optimisers, regularisation, and the train/val/test discipline that underlies every model you will train for the rest of the course.

This lecture is the hinge between classical CV (weeks 1–11) and deep learning (weeks 13–15).

The Recognition Problem, Revisited

Classification: what is in the image? (single label)
Detection: what, and where? (class + bounding box, possibly multiple instances)
Segmentation: what, exactly which pixels? (pixel-level label)

Before deep networks, detection was solved by scanning a classifier over the image at many positions and scales — the sliding-window paradigm.

The Sliding-Window Paradigm

Build an image pyramid (resize the image at several scales) so objects of different sizes are all found at some pyramid level.
Slide a fixed-size window across each pyramid level with a stride.
Extract features from each window and run a binary classifier (object vs. background).
Collect all positive windows; suppress duplicates with Non-Maximum Suppression (NMS).

Complexity: for an W×H image, S scales and a w×h window, you evaluate on the order of S·W·H windows — this is why classical detectors demanded very cheap per-window features and classifiers.

Histogram of Oriented Gradients (HOG)

Compute image gradients (Gx, Gy) and per-pixel orientation/magnitude.
Divide the window into small cells (e.g. 8×8 px); accumulate a histogram of gradient orientations weighted by magnitude in each cell.
Group cells into overlapping blocks and L2-normalise the block's histogram — this makes HOG robust to local illumination changes.
Concatenate all block histograms into one feature vector per window.

import cv2
hog = cv2.HOGDescriptor()
hog.setSVMDetector(cv2.HOGDescriptor_getDefaultPeopleDetector())
boxes, weights = hog.detectMultiScale(
    frame, winStride=(8, 8), padding=(8, 8), scale=1.05)

HOG + Linear SVM: Pedestrian Detection

Dalal & Triggs (2005): HOG features feeding a linear SVM classifier was the state of the art for pedestrian detection before deep learning.
Linear SVM decision function: \(f(x) = w^\top x + b\); the window is positive if \(f(x) > 0\).
Because the classifier is linear, HOG detectors are fast enough to run on embedded CPUs — still a reasonable baseline on a Raspberry Pi when a full YOLO model is too heavy.
cv2.HOGDescriptor_getDefaultPeopleDetector() ships a pretrained linear SVM in OpenCV.

Haar-Like Features & the Integral Image

Haar-like features are simple rectangular filters (edge, line, and four-rectangle patterns) that sum pixel intensities under white regions minus black regions.
Computing a rectangle sum naively costs O(pixels); the integral image \(I_\Sigma(x,y)=\sum_{x'\le x,\,y'\le y} I(x',y')\) lets any rectangle sum be computed in O(1) with 4 lookups.
This is what makes Haar cascades cheap enough to scan every window at every scale in real time on 2001-era hardware — and still today on microcontrollers.

Viola–Jones: The Cascade Classifier

AdaBoost selects a small set of the most discriminative Haar features from a huge pool and combines them into a strong classifier (weighted vote of weak classifiers).
Strong classifiers are chained into a cascade of stages, each cheaper and less accurate than a single strong classifier alone.
A window is rejected as soon as any stage rejects it — most background windows are discarded after 1–2 cheap stages, so average work per window stays tiny.

face_cascade = cv2.CascadeClassifier(
    cv2.data.haarcascades + "haarcascade_frontalface_default.xml")
faces = face_cascade.detectMultiScale(gray, scaleFactor=1.1, minNeighbors=5)

Note: cv2.CascadeClassifier's constructor signature changes in OpenCV 5.0 — see Lecture 14.

Bag-of-Visual-Words (BoVW)

Detect local features (e.g. SIFT/ORB, Lecture 7–8) across a training set.
Cluster the descriptors with k-means into K clusters — the cluster centres form a visual vocabulary / codebook.
For a new image, assign every detected descriptor to its nearest codeword and build a K-bin histogram of codeword occurrences — this is the image's feature vector, fed to a classifier (SVM, k-NN).

BoVW discards all spatial layout — two images with the same features scattered differently look identical to it.

Spatial Pyramid Matching (SPM)

Fix BoVW's spatial blindness by partitioning the image into a pyramid of increasingly fine grids (1×1, 2×2, 4×4, …).
Compute a separate BoVW histogram per cell at every pyramid level, then concatenate (with level-dependent weights favouring finer levels).
Result: a single feature vector that captures both what visual words are present and roughly where.

Practitioner Notes: Why Classical Detectors Still Matter

Speed & footprint: a Haar cascade or HOG+SVM model is kilobytes, not megabytes — still the right choice on an MCU with no NPU.
No training data needed: pretrained cascades (face, eye, smile) ship with OpenCV and just work.
Interpretability: every stage/feature is inspectable, unlike a CNN's learned filters.
When to reach for deep learning instead: cluttered backgrounds, many object classes, pose/appearance variation, or when you have a labelled dataset and a GPU/NPU budget — covered next.

From Hand-Crafted to Learned Features

HOG / Haar / BoVW           Neural network
  engineer designs the        network LEARNS the
  feature extractor      →    feature extractor
  (fixed, by hand)            (trained, from data)

The rest of this lecture builds the machinery — perceptron, MLP, loss, back-propagation, optimisers — that lets a network learn its own features. Lecture 13 scales this up to convolutional networks.

The Perceptron

Rosenblatt's perceptron (1958): \(\hat{y} = \text{step}(w^\top x + b)\) — a linear decision boundary with a hard threshold.
Learning rule: nudge \(w\) toward misclassified points, \(w \leftarrow w + \eta(y-\hat{y})x\).
Limitation: a single perceptron can only separate linearly separable data — famously, it cannot learn XOR.
Fix: stack layers of perceptrons with non-linear activations → the Multi-Layer Perceptron.

Multi-Layer Perceptron (MLP)

An MLP alternates linear layers (\(z = Wx+b\)) with non-linear activations \(a=\sigma(z)\), stacked in depth.
The Universal Approximation Theorem: an MLP with one sufficiently wide hidden layer and a non-linear activation can approximate any continuous function on a compact domain — depth is what makes it practical (fewer parameters for the same expressiveness).
Common activations: Sigmoid \(\sigma(z)=1/(1+e^{-z})\), Tanh, ReLU \(\max(0,z)\) (default choice — cheap, avoids vanishing gradients for \(z>0\)), Leaky ReLU, Softmax (multi-class output layer).

Loss Functions

Regression — Mean Squared Error: \(\mathcal{L} = \frac{1}{N}\sum_i (y_i-\hat{y}_i)^2\)
Classification — Cross-Entropy: \(\mathcal{L} = -\frac{1}{N}\sum_i \sum_c y_{i,c}\log \hat{y}_{i,c}\), paired with a softmax output layer.
The loss is the single scalar that back-propagation differentiates to produce every weight update — get the loss wrong (e.g. MSE on a classification head) and training silently under-performs.

Gradient Descent & Back-Propagation

Forward pass: push a batch through the network, compute the loss.
Backward pass: apply the chain rule layer-by-layer, from the loss back to the inputs, to get \(\partial \mathcal{L} / \partial W\) for every weight.
Update: \[ W \leftarrow W - \eta \, \frac{\partial \mathcal{L}}{\partial W} \] where \(\eta\) is the learning rate.
Back-propagation is just the chain rule applied systematically — every autodiff framework (PyTorch loss.backward()) implements exactly this.

SGD, Momentum & Adam

SGD: update using the gradient of a mini-batch (not the full dataset) — noisy but cheap, and the noise itself helps escape shallow local minima.
Momentum: \(v \leftarrow \beta v + \nabla\mathcal{L}\), \(W \leftarrow W - \eta v\) — accumulates a velocity, damping oscillation and speeding convergence in consistent directions.
Adam: keeps per-parameter running estimates of the first moment (mean) and second moment (variance) of the gradient and adapts the effective learning rate per parameter — the default optimiser for most modern training.
Practical default: start with Adam, \(\eta \approx 10^{-3}\), and only reach for tuned SGD+momentum if you need the last bit of generalisation.

Over-Fitting vs. Under-Fitting

error
  │  \                              ___ val error (overfit gap)
  │   \                        ___-''
  │    \__            ___-----
  │        ''---___---           ___ train error
  │____________________________________ model capacity / epochs
   under-fit        ↑ sweet spot        over-fit

Under-fitting: both train and validation error are high — the model is too simple, or hasn't trained enough.
Over-fitting: train error keeps dropping while validation error rises — the model has memorised training noise instead of learning general structure.

Regularisation

Weight decay (L2): add \(\lambda \lVert W \rVert^2\) to the loss, discouraging large weights.
Dropout: randomly zero a fraction of activations during training, forcing redundancy so the network can't rely on any single unit.
Early stopping: stop training when validation loss stops improving, even if training loss keeps falling.
Batch normalisation (previewed here, detailed in Lecture 13): normalises layer activations per mini-batch, stabilising and regularising training.

Data Augmentation

Synthesise label-preserving variations of training images so the model sees more diversity than the raw dataset contains: random crop/resize, horizontal flip, rotation, colour jitter, cutout/random-erasing, mixup/CutMix.
Augmentation is itself a form of regularisation — it directly reduces the overfitting gap from the previous slide.
This week's lab uses augmentation when fine-tuning a classifier by transfer learning, and again when training a custom YOLO detector (Lab 12).

Train / Validation / Test Discipline

Train set: what the optimiser sees — weights are fit here.
Validation set: used to tune hyper-parameters (learning rate, architecture, when to stop) and to catch over-fitting — never used to update weights directly.
Test set: touched exactly once, at the very end, to report the final number. If you tune based on the test set, it has silently become a validation set and your reported number is optimistic.
Common leakage bugs: augmenting before splitting, duplicate/near-duplicate images across splits, splitting video frames instead of clips (adjacent frames leak into both sets).

Code Walkthrough: A Minimal PyTorch Training Loop

class SmallCNN(nn.Module):
    def __init__(self, num_classes):
        super().__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(), nn.AdaptiveAvgPool2d(1))
        self.head = nn.Linear(64, num_classes)

    def forward(self, x):
        return self.head(self.features(x).flatten(1))

crit = nn.CrossEntropyLoss()
opt = torch.optim.Adam(model.parameters(), lr=1e-3)
for x, y in train_loader:
    opt.zero_grad()
    loss = crit(model(x), y)
    loss.backward()      # back-propagation
    opt.step()            # Adam update

Full trainer with a real train/val loop: applied-cv-course/code/ml/train_classifier_pytorch.py.

Common Pitfalls

Forgetting optimizer.zero_grad() — gradients silently accumulate across batches.
Forgetting model.eval() / torch.no_grad() at validation time — dropout/batch-norm behave differently in train vs. eval mode, and you waste memory tracking gradients you'll never use.
Learning rate too high (loss diverges/NaNs) or too low (nothing happens in the time you have) — always sanity-check with a short run first.
Not normalising inputs (e.g. ImageNet mean/std) when using a pretrained backbone — the network was trained expecting a specific input distribution.
Reporting validation accuracy from the best epoch as if it were the test accuracy.

This Week’s Lab

Lab 12 — Deep Learning: Classification & YOLO Detection.

Classify images with a pretrained model, then train a custom classifier by transfer learning on Google Colab (free GPU).
Detect objects on image/video/camera with YOLO; train a custom detector.
Try SAHI (Slicing-Aided Hyper Inference) for small objects, and a pose model.

Deliverable: a trained custom classifier + a custom YOLO detector running on the Raspberry Pi.

Key Takeaways

Classical detectors (HOG+SVM, Haar cascades) trade accuracy for extreme speed and tiny footprint — still the right tool on the smallest hardware.
BoVW + spatial pyramids showed that "features + a simple classifier" scales to whole-image recognition, foreshadowing learned features.
An MLP is linear layers + non-linear activations; back-propagation is the chain rule; Adam is the practical default optimiser.
Regularisation, augmentation, and a clean train/val/test split are what separate a model that generalises from one that has memorised its training set.