📝 Terminology
A compact glossary for terms used across this site.
| Term | Definition | Example | Notes |
|---|---|---|---|
| Batch | A group of training examples processed together in one forward and backward pass | A batch size of 64 processes 64 examples together | In deep learning, “batch” usually means a mini-batch |
| Calibration | Predicted confidence matches observed frequency | Of 100 predictions made with 0.8 confidence, about 80 are correct | Fit a calibration mapping on a separate held-out split |
| Down-sampling | Deliberately keeping fewer examples from an overrepresented group or class | Keep a subset of negative examples when positives are rare | It can change class prevalence, so probability calibration may need correction |
| Epoch | One complete pass through all training batches | 50,000 examples with batches of 100 give 500 steps per epoch | Training data is usually shuffled each epoch |
| Iteration / step | One parameter update using one batch | optimizer.step() updates the model parameters | With gradient accumulation, one step can combine several micro-batches |
| Logit | An unnormalized model score before sigmoid or softmax converts it to a probability | A binary classifier can output a logit before applying sigmoid | A logit is not itself a probability |
| Mini-batch | A small subset of the training set used for one training step | 64 examples from a training set of 50,000 | Batch size is a training hyperparameter |
| Weight decay | A regularization method that penalizes large parameter values during training | An optimizer may use weight_decay=0.01 | Often equivalent to L2 regularization for plain SGD; adaptive optimizers commonly use decoupled weight decay |