Learn · 2
Autograd
Mark a tensor with requiresGrad, do arithmetic, call
backward(), read .grad. Backward is synchronous — only
reading the number back is awaited.
The smallest possible gradient
Pull w toward 5 and check the derivative by hand:
d/dw (w − 5)² = 2(w − 5), which at w = 3 is −4.
Gradients accumulate
Calling backward() twice adds into .grad rather than
replacing it — the same as torch, and the reason every training loop starts with
zeroGrad(). Try running this block twice in a row.
Turning it off
noGrad(fn) runs a function with the tape switched off — evaluation,
and anything an optimizer does to parameters in place. There is also
gradMode for callers that need to hold the switch themselves;
a binding in another language cannot pass a callback across the boundary.
Chains, not just single steps
The graph is built as you go — eager, like torch. Every operation that has a
backward rule records its parents, and backward() walks them.
roll and maskedSelect were both silently cut while the
golden table was entirely green. 53 of the conformance cases now ask only
"did a gradient arrive", which is a different question from "is the value right".