Learn · 7

Data

Real training reads batches, shuffles them, and holds out a validation split. The names are torch's — TensorDataset, DataLoader, randomSplit — and they live under data, where torch.utils.data would be.

A dataset and a loader

TensorDataset takes tensors that share a first dimension and hands back one row at a time. DataLoader turns that into batches, and it is iterable — for (const [x, y] of loader) reads the way the Python does.

Shuffling, and the last short batch

shuffle reorders every epoch. dropLast decides what happens to the remainder — keep it and the final batch is smaller, which matters the moment anything in your model assumes a fixed batch size.

Holding a split back

randomSplit gives you views onto the same data — no copies. Reporting a number measured on rows the model trained on is the most common way to be wrong and feel right.

Images

vision is the torchvision.transforms position and holds only transforms — there are no pretrained weights here and there will not be. Normalize takes per-channel mean and standard deviation, and Compose chains transforms in order.

A batch, drawn

Six samples stacked on axis 0, drawn as a strip. A model does not see one example at a time — it sees a whole batch at once, which is what makes training on a GPU fast.

What to remember

A dataset yields samples; batching stacks them on axis 0 and shuffling reorders them each epoch.
Hold out a validation split and never train on it — that is how overfitting is caught.
Transforms (crop, flip, normalise) augment the training data.
The network sees a whole batch at once.

Augmentation is measured, not assumed. On the CIFAR ResNet-18 benchmark this repository reports 64.6% with augmentation on and 59.3% with it off, and both numbers are stated because the second one is worse than the sister implementation's. A number quoted without its condition is not a number.