Capstones · 3

Watch a network learn a boundary

A classifier's whole job is to carve the input space into regions, one per class. For points on a plane you can see that carving: colour every point by the class the network would give it, and the split between the colours is the decision boundary. A linear model can only draw a straight line; a network with a hidden layer can bend it. Here a small MLP separates two interleaving moons — the boundary is drawn before training (random) and after (bent to fit).

The boundary, before and after

Two half-moons, each a class, that no straight line can separate. The grid below reads the network's decision at every point on the plane and draws it as a heatmap: the two shades are the two classes, and the seam between them is the boundary. Press Run — the first heatmap is the untrained net's random split, the second is what 140 steps of training bent it into, and the curve is the loss coming down.

What to remember

• A classifier partitions the input space; for 2-D data you can draw that partition as a heatmap over a grid — the seam is the decision boundary.
• A linear model draws a straight boundary. A hidden layer with a non-linearity (here ReLU) is what lets it bend, so it can separate classes no line could.
• Reading the model on a dense grid and drawing it is a general trick for seeing what any small model has learned.
• Built from nn.Sequential, CrossEntropyLoss and Adam — the same loop as every other trained model here, just in two dimensions so you can watch it.