Foundations · 3

Broadcasting & shapes

Broadcasting lets an operation between different shapes work without copying data — a row is stretched across a matrix, a scalar across everything. Getting the shapes right is most of the battle in real code.

A row stretches across the rows

Adding a shape-[3] vector to a [2,3] matrix adds it to every row. Trailing axes line up from the right.

keepdim: the shape you subtract back

A reduction drops the axis it collapses. keepdim=true keeps it as size 1 — the shape you need to broadcast the result back against the original.

Seeing a broadcast: the sum table

A column [8,1] plus a row [1,8] broadcasts to an [8,8] grid — every cell is row + column. The smooth gradient is the broadcast made visible: neither input was copied, yet every pair was combined.

Your turn: centre the columns

Centring means subtracting each column's mean so every column ends up averaging zero. The block subtracts the wrong mean — fix it to the per-column mean.

What to remember

• Broadcasting combines different shapes without copying; trailing axes line up from the right.
• A size-1 (or missing) axis stretches to match the other.
• keepdim keeps a reduced axis at size 1, so the result can broadcast back against the original.
• In borch: elementwise ops broadcast like torch's; reach for mean(dim, keepdim) when you need the shape back.