11.1.1 Webcam Processing

Overview

A webcam is a NumPy array generator. Every 30-millisecond tick it produces a new (H, W, 3) array, the same data structure you have manipulated since lesson 1.1.1. The job of an interactive system is to put a transformation between capture and display — anything from a colour filter to motion detection to a full pose-estimation network. The pipeline is identical regardless of complexity:

while True: frame = capture(); processed = transform(frame); display(processed)

This lesson covers the canonical loop with OpenCV’s VideoCapture, three classes of real-time operators (channel, blur, edge), and the most-used interactive-systems primitive: frame differencing for motion detection. The code is OpenCV-flavoured because that’s the dominant ecosystem; the operators are the NumPy ones you already know.

Learning objectives

1. Open and read a webcam stream via OpenCV’s VideoCapture API, releasing the device cleanly.
2. Apply real-time per-frame transformations (grayscale, blur, edge detection) without dropping below 30 fps.
3. Implement frame-differencing motion detection: store previous frame, absdiff, threshold, overlay.
4. Recognise that every interactive-system input — webcam, mic, MIDI, Kinect — produces the same pipeline shape.

Quick start — capture loop

import cv2

cap = cv2.VideoCapture(0)   # 0 = default webcam; can also be a video filename

while True:
    ok, frame = cap.read()
    if not ok:
        break
    cv2.imshow('Webcam', frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Core concepts

Concept 1 — The capture-process-display loop

Every interactive video application has the same three-step inner loop:

1. Capture. Read one frame as a NumPy array. ret, frame = cap.read() returns False on disconnect or end-of-file; always check ret before processing.
2. Process. Apply zero or more transformations. The frame is uint8 BGR (OpenCV’s quirk — not RGB); convert to grayscale, blur, threshold, anything from your NumPy toolkit.
3. Display. cv2.imshow(window, image) paints a window; cv2.waitKey(1) gives the GUI 1 ms to update and polls for a keypress. Without waitKey, no frame ever displays.

After the loop, cap.release() and cv2.destroyAllWindows() clean up. Leaving the device locked is a common bug — the camera is unusable until your Python process exits or the OS recovers.

Concept 2 — Real-time per-frame operators

Since each frame is a NumPy array, every Module 03 transformation works in real time. The OpenCV equivalents are usually one line:

gray    = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)   # luma
blurred = cv2.GaussianBlur(frame, (21, 21), 0)      # 21x21 Gaussian
edges   = cv2.Canny(gray, 50, 150)                  # Canny edges
mirror  = cv2.flip(frame, 1)                        # horizontal flip

The constraint is frame budget: at 30 fps you have ~33 ms per frame for capture + process + display. A 640×480 Gaussian blur takes ~2 ms; a Canny edge detector takes ~5 ms; reading the frame and showing the result eats another ~10 ms. Most interactive applications fit a handful of operators per frame comfortably.

Concept 3 — Frame-differencing motion detection

The cheapest motion detector compares consecutive frames. Pixels that change significantly between $t-1$ and $t$ are “moving”:

prev_gray = cv2.GaussianBlur(cv2.cvtColor(prev, cv2.COLOR_BGR2GRAY), (21, 21), 0)

while True:
    ok, cur = cap.read()
    cur_gray = cv2.GaussianBlur(cv2.cvtColor(cur, cv2.COLOR_BGR2GRAY), (21, 21), 0)

    diff = cv2.absdiff(prev_gray, cur_gray)         # per-pixel |a - b|
    _, motion = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)

    out = cur.copy()
    out[motion > 0] = [0, 255, 0]                   # green where motion

    prev_gray = cur_gray.copy()
    cv2.imshow('Motion', out)
    if cv2.waitKey(1) & 0xFF == ord('q'): break

Two pre-processing tricks are doing real work here. First, convert to grayscale before comparing — colour comparisons would triple the work and add no useful information for motion. Second, blur before differencing — a 21×21 Gaussian removes per-pixel sensor noise that would otherwise produce false-positive “motion.”

Exercises

Three exercises in Execute → Modify → Create order: run the capture loop, add a filter, then build a motion detector.

sepia_matrix = np.array([[0.272, 0.534, 0.131],
                         [0.349, 0.686, 0.168],
                         [0.393, 0.769, 0.189]])
sepia = cv2.transform(frame, sepia_matrix)
sepia = np.clip(sepia, 0, 255).astype(np.uint8)

small = cv2.resize(frame, (64, 48), interpolation=cv2.INTER_AREA)
pixelated = cv2.resize(small, (frame.shape[1], frame.shape[0]),
                       interpolation=cv2.INTER_NEAREST)

mirror = cv2.flip(frame, 1)   # 1 = horizontal, 0 = vertical, -1 = both

import cv2
import numpy as np

cap = cv2.VideoCapture(0)
ok, prev = cap.read()
prev_gray = cv2.GaussianBlur(cv2.cvtColor(prev, cv2.COLOR_BGR2GRAY), (21, 21), 0)

while True:
    ok, cur = cap.read()
    cur_gray = cv2.GaussianBlur(cv2.cvtColor(cur, cv2.COLOR_BGR2GRAY), (21, 21), 0)

    # TODO 1: per-pixel absolute difference between current and previous gray frames

    # TODO 2: threshold the diff to a binary motion mask at value 25

    # TODO 3: copy the current frame, paint green wherever the mask is set

    prev_gray = cur_gray.copy()
    cv2.imshow('Motion', out)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

diff = cv2.absdiff(prev_gray, cur_gray)
_, motion = cv2.threshold(diff, 25, 255, cv2.THRESH_BINARY)
out = cur.copy()
out[motion > 0] = [0, 255, 0]

Downloads

Summary

Common pitfalls to avoid

• Forgetting cap.release(). Locks the device until process exit.
• Skipping cv2.waitKey. The window never updates; debugging is mysterious.
• Ignoring ret. A False return on disconnect crashes the next line.
• Comparing colour frames directly. 3× the work for no benefit; convert to grayscale first.

References