Learn

Notation

The shorthand the lessons lean on, in one place. Nothing here is unusual — it is torch's vocabulary — but it helps to have the symbols named once so a chapter can move quickly.

Reading the lessons

• Run — execute the box; ⌘/Ctrl + Enter does the same. Edit first if you like; Reset restores the original.
• JavaScript / Python tabs — the same work in both; press to compare the numbers.
• ○ → ✓ — a lesson's completion circle on the course home; press it, or pass the lesson's exercise, to mark it done.
• Your turn — an exercise that ships with one line to fix; the page says learned when it is right.
• await — reading a value back from the GPU is asynchronous in JavaScript (await x.item()); Python reads directly.

Tensors and shapes

• [N, C, H, W] — a shape: batch, channels, height, width. A vector is [n], a matrix [m, n].
• dtype — the element type (e.g. float32); device — where it lives (the GPU).
• [m, k] @ [k, n] → [m, n] — the shape rule for matrix multiply: inner sizes match, outer two become the result.
• A scalar is a shape-[] tensor; in JavaScript a constant is Tensor.full([], k).

Vectors and matrices

• a · b — the dot product, a.dot(b): matching entries multiplied and summed.
• ‖v‖ — the L2 norm (length), v.norm(); distance is ‖a − b‖.
• Aᵀ — transpose, A.transpose(); A @ B — matrix multiply, A.matmul(B) (or mm).
• A v = λ v — an eigenvector v and eigenvalue λ; A = U diag(S) Vᵀ — the SVD, with singular values S.

Training

• logits — a model's raw output scores, before they are turned into probabilities.
• softmax — logits to a probability distribution that sums to 1; cross-entropy — the loss −log p of the true class.
• requiresGrad, backward(), .grad — mark an input, run the backward pass, read the gradient.
• train / eval — a module's mode; dropout and batch norm behave differently in each.