torchvision's seat
Images, boxes and datasets
663 of the library's 2950 names are the vision side. They are shaped
like torchvision and they are here for one reason: training needs a place
where an image becomes a tensor, and that place is where things go quietly wrong.
Every section below names a way it does.
transforms — 279 names
Where torchvision.transforms would be. An image is an
(H, W, C) array here, not a tensor. torchvision receives a PIL
image; there is no PIL in a browser, so an array stands in its place. Making a tensor
per image makes one GPU buffer per image, which looks like it works right up until it
collapses on memory.
These use the same rules as the Python side (borchvision.py).
If the two ever diverge, the same data becomes different values depending on which
library read it — a difference no type checks and only a value comparison sees.
v2 — 68 functional, 134 classes
v2 is torchvision's current recommended surface, and it is v1 with a different front. That was measured before any of it was written: on a plain image, v2's transforms give what v1's give to the last bit. So the arithmetic is not written twice. Everything v1 already had is re-exported, not copied, and the class twins extend the v1 class and override only what it prints.
One pair answers backwards from its neighbour. v2's
getSize gives [height, width]; getImageSize, one
namespace over, gives [width, height]. torchvision reversed it deliberately
and deprecated the old spelling rather than change it. Both are here, both keep their
own order, and reading the wrong one gives you a transposed image that still trains.
ops — 164 names
Box geometry and the losses built on it: boxIou and its generalized,
distance and complete variants, the three IoU losses, sigmoidFocalLoss,
nms and batchedNms. What is not here is everything
that wants a detector's feature maps or its predictions — there is no detector in
this library yet, and these take nothing but four numbers a box.
The rest is the same geometry applied to the things that ride along with an
image. Crop a picture and its boxes move; rotate it and its masks and keypoints
must turn with it. So each of the transforms — affine, crop, resize, resizedCrop,
rotate, perspective, elastic, pad and the two flips — has a kernel per target:
rotateBoundingBoxes, rotateMask, rotateKeypoints.
That is where the count comes from.
They are separate functions because they disagree. A rotated box is not the rotated corners — it is the upright box around them, so it grows. A mask goes through the same warp as the image but with nearest-neighbour sampling, because interpolating a label invents labels that were never there. A keypoint that leaves the canvas is clamped by one kernel and dropped by another. Handing all three to one function would make those choices invisible.
The arithmetic happens on the CPU, on purpose. Boxes come in tens, not
millions, and every function here sorts, iterates until a set stops shrinking, or reads
a mask's extent — shapes a GPU kernel is bad at and a loop is good at. Each call
reads its tensors back once, which is why they are async.
datasets — 18 names
The decoders, and only those. A dataset is an address and a format. The address half — fetching, caching, checksums — is not here, because a test case that downloads is a test case that fails on a train.
The format half is the part that fails quietly, and every function in this file is one of those. Each of these leaves a dataset that still trains when it is read wrong:
- STL-10 is stored column-major and its labels start at 1. Skip the transpose and every picture comes out rotated — still a picture.
- MovingMNIST's split cuts the frames, not the clips. A reader that cuts clips instead gets the right rank and half the count.
- FER2013 keeps its pixels as a string of integers inside a cell, and it has two layouts.
What is not here
No detector, so no roiAlign, no DeformConv2d, no feature
pyramid. No dataset downloads. No PIL, and therefore no operation that needs one.
The API reference is generated from the built declarations, so the
surest answer to “does this exist?” is to look it up there rather
than to assume torchvision's answer carries over.