Workbench

One call that trains on a folder of images

torch.workbench is a recipe, not a torch name — it sits one level in so that is visible. Hand it a folder: it reads the labels off the file names, trains, scores every label by how much the model doubts it, and hands back an ONNX file. The images stay in the tab. It is what the marimo notebook runs, and you can call it yourself in either notebook on this site.

Open it on your own folder → Walk through it first →

Try it on your own folder

Five steps, no Python needed for any of them. What you need is a folder of images you already know the answers to. A few hundred is enough for the frozen-backbone path, which is the default; a thousand is comfortable.

1. Name the files so the label is in front. Everything before the first _ is the label: ok_001.jpg, ng_002.jpg, crack_017.jpg. A zip of folders works too, and then the folder name is the label. Nothing else about the names matters.
2. Open the page and wait about ten seconds. Python and the notebook are downloading into the tab. There is nothing to install and no account to make.
3. Drop the folder in and press run (bottom right). The images are read in the tab and stay there — nothing is uploaded, and you can watch that in your browser's network panel.
4. Read the review table from the top. It is sorted by how much the model disagrees with the label you gave, worst first. That is where your mislabels are: on a public set with 9 % of labels wrong, the first twenty rows are about two-thirds mislabels, against nine in a hundred if you had read in a random order.
5. Take the model. The export button writes an ONNX file, which is the format ONNX Runtime reads on a phone, a server or a PC — it does not need borch or a browser afterwards.

Start at step 2 →

Nothing above is a screenshot of an ideal run: the same five steps are pressed nightly by tests/browser/marimo_probe.py on a real GPU, and that check refuses to report at all if the tab it opened has no GPU.

The same thing, called yourself

If you would rather write it, the five steps above are four lines. They run in either notebook on this site, and in any Pyodide.

import borch_webgpu as torch
s = torch.workbench.setup(files).fit()   # train on the folder
s.suspects()                             # the labels it doubts, worst first
s.onnx()                                 # take the model away

Everything below is what those four lines let you change, and what the review order is worth. The API reference →

The call

import borch_webgpu as torch          # or borch_cpu — same spelling, no device
wb = torch.workbench

s = wb.setup(files, val=0.2)         # train, holding a fifth back
print(s.accuracy, s.held_out)        # and what it was measured on
s.suspects()                         # the labels it doubts, worst first
s.onnx()                             # take the model away

Two more calls answer the question before that one — which backbone. wb.compare(files, budget_mb=60) trains every candidate that fits the budget and prints a board; wb.pick(files, at_least=0.9) returns the smallest one that clears a bar. Both mark the rows they cannot tell apart: with a hundred held out, two backbones four points apart are one result, and wb.resolution(held_out) says how far apart they would have to be before the difference is real. A board that hides that invites a decision the data does not support.

The same names are on both doors. borch_cpu has no nn and no autograd, so the frozen-backbone path runs there and the from-scratch CNN and the ONNX export do not — the surface is asked what it can do rather than what it is called, and a caller who wants the other path is told which door to open.

What it reads, and what comes back

• Labels come from the file names. A label is the part of the name before the first _ — ok_014.png, ng_003.png. A zip of class folders works too, and nothing in it is decoded until a batch asks for it, so thousands of images cost what a batch costs. Upload nothing and a synthetic three-class set runs instead.
• Two model paths. A frozen ImageNet EfficientNet-B0 with a linear head on top (one feature vector per image, computed once — hundreds of images are enough), or a small CNN where every weight learns. The frozen one is the field trainer's path and the default.
• Masks make it segmentation. A zip with images/ and masks/ — the same names in both — trains a U-Net instead, and the review table puts the given mask beside the model's, worst agreement first.
• What comes back is an ONNX file that ONNX Runtime reads anywhere, the review order, and the settings and the result flat enough to paste into a report. Nothing is installed and nothing is uploaded.
• The calls underneath are the library's, and you can use them without the recipe: torch.decode_images turns files into an NCHW batch with labels from the names, torch.ImageFiles is the same on demand, and torch.suspects scores each label by how many of its nearest neighbours in the model's features disagree with it — that score is the review order.

Does the review order hold up

• Does the review order find the real mislabels? Measured on CIFAR-10N, fifty thousand images with the labels people actually gave and the clean labels known, through this very path (frozen B0 pre-logits → torch.suspects): with 9 % of labels wrong, the queue's first 20 are 65 % mislabels (random order: 9 %), AUROC 0.948, and reading 18 % of the queue catches 90 % of them; with 40 % wrong, the first 20 are all mislabels. The browser path on a 5,000-image subset lands within a few points of the native run (AUROC 0.929 vs 0.917). Harder sets say where it thins: CIFAR-100N (a hundred fine classes, 40 % wrong) AUROC 0.842 with the first 100 of the queue 78 % mislabels; Food-101N (web-collected food, 18 % wrong, the wrong ones being other dishes that look alike) AUROC 0.767, the first 20 65 % mislabels — twice random, not seven times. Larger neighbourhoods help at scale: at fifty thousand rows k = 20 lifts CIFAR-100N with 10 % flipped from AUROC 0.913 to 0.962, and the workbench uses twenty from two thousand images up. Three domain sets, with 10 % of clean labels flipped (k = 20): NEU-CLS steel-surface defects AUROC 1.000 and the queue's first 20 all mislabels; ISIC 2019 dermoscopy 0.920; PatchCamelyon histology patches 0.918 — the inspection camera's kind of picture is where the frozen backbone is at home, and tissue is where a partial fine-tune starts to matter. tests/review_eval.py holds every number.
• Segmentation, the same two questions. A three-level U-Net on Kvasir-SEG (a thousand endoscopy images with a polyp mask each, at 96 px) with a fifth of the training masks corrupted — swapped with another image's, or shifted off the polyp. Held-out IoU against the clean masks, trained on clean masks: torch 0.55, the browser 0.51–0.57 across the day's kernel versions; trained with a fifth of the masks wrong: torch 0.40–0.50 over six seeds, the browser 0.46 (the number after thirty epochs on corrupted masks moves with the arithmetic's summation order, so it is reported as a range). 34 s for 800 images × 30 epochs on the Mac's GPU against torch's 20 s — it was 104 s two days earlier, before a day of kernel work measured with GPU timestamps (BatchNorm's statistics, the convolution's tiles and gathers, the broadcast fold, `cat`, BatchNorm paired with its ReLU, the loss in one kernel). One training step at batch 16 is 18 ms against torch's 12; of the 15 ms the GPU itself spends, nine are the convolutions — the narrow layers now run a direct kernel, and the weight gradient's GEMM is the largest single item. A mask has no neighbourhood to vote in, so the queue here is the model's own disagreement (one minus the IoU between its mask and the given one): AUROC 0.88–0.93 in torch across seeds, 0.91 in the browser, and reading about a third of the queue catches 90 % of the wrong masks. tests/seg_eval.py and tests/browser/seg_eval_py.py.

Every number above is held by a check that runs nightly — tests/review_eval.py and tests/seg_eval.py for the orders, tests/test_workbench.py for the recipe's 39 invariants, and tests/browser/marimo_probe.py for the whole path through a real tab, which refuses to report at all if that tab has no GPU.