9.2.2 Convolutional Networks

Big question

Why don’t we just use a giant feedforward network on images? A 224×224 RGB image has 150,528 input pixels. A single fully-connected hidden layer with 1000 neurons would have 150 million weights — for one layer. Most of those weights would learn redundant local-pattern detectors for every pixel position. Yann LeCun’s insight in 1989 was that the same edge detector should work at every location: share the weights across positions. The convolutional layer is born — a tiny 3×3 or 5×5 filter, slid across the image, learning one feature detector that runs everywhere [1, 2]. This single architectural idea is what made computer-vision deep learning practical.

Learning objectives

1. Connect convolution-as-kernel (3.4.1) to convolution-as-CNN-layer — same math, learned weights.
2. Implement a forward convolution in NumPy and apply learned filters to an image.
3. Understand the CNN architecture: convolution → activation → pooling, stacked.
4. Visualise feature maps — the per-filter response at each layer — to see what the network is “looking at.”

Part 1 — Shared weights, sliding filters

A fully-connected layer treats every pixel as an independent input feature: weights w(i, j) connect input pixel (i, j) to every hidden neuron. A convolutional layer shares weights across positions: one 3×3 filter is applied at every location of the input.

def conv2d(image, kernel, stride=1, pad='same'):
    H, W = image.shape
    k = kernel.shape[0]
    if pad == 'same':
        p = k // 2
        image = np.pad(image, p, mode='edge')
    out = np.zeros((H, W))
    for y in range(H):
        for x in range(W):
            out[y, x] = np.sum(image[y:y+k, x:x+k] * kernel)
    return out

This is the same convolution as 3.4.1. The difference in a CNN: the kernel weights are learned by backpropagation rather than hand-designed [3].

Part 2 — Convolutional layer ingredients

A practical convolutional layer has four ingredients:

• Filters — K filters of shape (k, k, C_in) produce K output channels.
• Stride — step size when sliding the filter. Stride=1 keeps spatial size; stride=2 halves it.
• Padding — same keeps the output size equal to input; valid shrinks by k-1.
• Activation — typically ReLU, applied element-wise after convolution.

out_height = (in_height - kernel + 2*pad) / stride + 1

After the convolution + activation, a max-pooling layer reduces spatial size by taking the maximum over each 2×2 block. Pooling is what gives CNNs their translation invariance — the same feature at a slightly different position still fires the same downstream units [4].

Part 3 — Feature maps

After applying K filters, you have K feature maps — each one a 2D image showing where that filter “fires.” Visualising these is how you build intuition about what a CNN has learned.

Astonishingly, early-layer filters in trained CNNs look like Gabor filters — the same oriented-edge detectors that the 1980s computer-vision community had already designed by hand. The network rediscovers these features from scratch, by gradient descent on a classification objective [2, 5].

Synthesis project

MODIFY II.

Artistic filters via convolution

Apply each of these hand-crafted kernels to an image and observe the artistic effects:

Goals

1. Edge detection — Sobel kernels (3.4.2) for grayscale outline.
2. Emboss — [[-2, -1, 0], [-1, 1, 1], [0, 1, 2]] for 3D-looking shading.
3. Sharpen — [[0, -1, 0], [-1, 5, -1], [0, -1, 0]] for crisp detail.

Implementation

Same conv2d from Part 1. Apply each filter; save the result. The emboss filter is asymmetric (the sum is non-zero) so the output has a directional shading effect — a kind of pseudo-3D look.

import numpy as np

def conv2d(x, W, b, stride=1):
    """Multi-filter 2D convolution with bias.
    x: (H, W, C_in), W: (K, k, k, C_in), b: (K,)
    Returns: (H_out, W_out, K)
    """
    H, W_in, C = x.shape
    K, kh, kw, _ = W.shape
    # TODO: implement using nested loops or im2col.

def maxpool2d(x, size=2, stride=2):
    """2x2 max pool with stride 2."""
    # TODO

def relu(x):
    return np.maximum(0, x)

# Build the network
def forward(image):
    x = image[..., None]                # (28, 28, 1)
    x = relu(conv2d(x, W1, b1))         # (28, 28, 4)
    x = maxpool2d(x)                    # (14, 14, 4)
    x = relu(conv2d(x, W2, b2))         # (14, 14, 8)
    x = maxpool2d(x)                    # (7, 7, 8)
    x = x.flatten()
    return W3 @ x + b3                  # logits

Downloads

References