Tutorial · 5

Adversarial examples

Gradients point at whatever you ask them about. Ask about the input instead of the weights and you can walk an image just far enough to change the answer while it still looks the same to you. That is the whole of FGSM, and it is six lines.

Needs the CIFAR subset. Run python3 site/fetch_data.py once — the same data as tutorial 4. This page trains its own copy of that model in the first block (a few seconds); a page that depended on another page's state would break the moment you opened it directly.

1 · A gradient with respect to the picture

Everything so far marked parameters as requiring gradients. Mark the input instead and backward() fills x.grad — one number per pixel saying which way to move it to make the loss worse.

2 · One step along the sign

FGSM keeps only the direction of each pixel's gradient and moves a fixed distance ε along it. Small ε, invisible change; the question is how small ε can be and still flip the answer.

3 · Look at what changed

Three rows: the original, the perturbed version, and the difference amplified so you can see it at all. If the middle row looks like the top row to you and the model disagrees, that is the point of the exercise.

Why this matters beyond the trick. The attack needs no access to the training data and no search — one backward pass through a model you have. It also runs entirely in the reader's browser, which is the unusual part: the gradient with respect to an input is the same call as the gradient with respect to a weight, and borch does not treat them differently.