Tutorial · 2

From scratch

The same problem as the quickstart, built up from nothing: raw tensors and a hand-written gradient step first, then nn.Linear, then optim, then a DataLoader. Each step deletes code that was doing something you have now seen for yourself.

This is the tutorial that answers "what is nn actually doing". Nothing below is magic, and by the end the loop looks like every other training loop you will write.

0 · The problem

Fit a straight line to points that follow y = 2x − 1 with noise on top. Small enough to check by eye, real enough that every piece has to work.

1 · By hand

Two parameters, a loss you write yourself, and a step you apply yourself. The only thing borch does here is compute the derivatives — that is what autograd is, and it is the one piece you would not want to write by hand.

Note noGrad around the update: changing a parameter is not part of the computation whose gradient you want, and torch's optimizers do exactly this.

2 · Let nn.Linear hold the parameters

The layer keeps weight and bias and knows the forward pass. What disappears: creating the two tensors, remembering their shapes, and writing x @ w + b. What stays the same: everything else.

3 · Let optim take the step

zeroGrad() and step() replace the two loops you wrote. The optimizer is now a thing you can swap: change SGD to Adam on the line below and watch the curve change shape.

4 · Let a DataLoader hand out batches

The last piece. Full-batch training was fine for 64 points; real data does not fit, so you iterate. The loop body does not change at all — only where x and y come from.

What each layer bought. Autograd bought the derivatives. The module bought parameter bookkeeping. The optimizer bought the update rule as something you can replace. The loader bought batching. None of it changed the arithmetic — you can go back to block 1 and see the same numbers coming out.