Supervised learning in AI means training a model on labelled examples (inputs paired with the correct outputs) so that it can predict the output for new inputs it has never seen. The inputs are called features and the correct output is the label, the “answer” the model is meant to produce. Spam filters, fraud detectors and loan-default models are built this way. So is one stage of instruction-tuned models such as InstructGPT: an LLM is first pre-trained with a self-supervised objective on unlabelled text, then fine-tuned with supervised learning on labelled demonstrations.
How supervised learning in AI works, from labelled data to inference
Google’s machine learning course frames it as a student who prepares for an exam by studying old papers that contain both the questions and the answers; the system is “supervised” because a human has supplied data with the known correct results. Stanford’s CS229 notes state the same idea formally: given a training set, learn a function h from inputs X to outputs Y such that h(x) is a good predictor of the corresponding y.
The steps, in order:
- Assemble labelled examples. Each example carries features and a label, while unlabelled examples contain features but no label. Google’s guidance is that good datasets are both large and diverse, where diversity means the range the examples cover.
- Choose a model form. CS229 starts with a linear function of the inputs whose parameters (weights) are what training adjusts. A deep network is the same idea with far more parameters.
- Train by minimising loss. The model compares its prediction with the true label, and the difference (the loss) drives gradual updates to its parameters. For linear regression that loss is the familiar least-squares cost.
- Evaluate on held-out labelled data. The model sees only the features of an evaluation set, and its predictions are compared with the true labels it never saw.
- Run inference. Once evaluation is satisfactory, the model makes predictions, called inferences, on new unlabelled examples.
Classification or regression: which kind of supervised problem is it?
Supervised learning divides into two problem types, classification and regression, distinguished by the type of target variable. If the target is continuous, such as a house price, the problem is regression; if it takes a small number of discrete values, such as whether a dwelling is a house or an apartment, it is classification.
| Classification | Regression | |
|---|---|---|
| Output | A category, or the likelihood of belonging to one | A real, continuous number |
| Typical examples | Spam or not spam; whether a photo contains a cat | Salary from work experience; rainfall in millimetres; price from square footage and location |
| Example model | Logistic regression | Linear (least-squares) regression |
The two are closer than they look. CS229 shows that least-squares regression and logistic regression are both special cases of one family, generalised linear models, differing in the distribution they assume for y given x. Many classifiers also output a score rather than a hard verdict: Google describes its example spam model as assigning a score that indicates whether an email is real or spam.
Supervised learning algorithms most teams reach for
Wolfram’s documentation lists these methods: decision trees, logistic regression (probabilities from linear combinations of features), random forests (Breiman-Cutler ensembles of decision trees) and support vector machines.
- Linear regression fits weights by minimising squared error, either iteratively or in closed form through the normal equations.
-
Logistic regression passes a linear combination of features through the sigmoid function to estimate p(y x) for binary classification. - Decision trees and random forests: a random forest is an ensemble of decision trees.
- Support vector machines learn a separating boundary between classes.
- Deep convolutional networks, the type used in the retinopathy study below, are a neural network type optimised for image classification.
The details of training those networks well are covered in our best practices for training deep learning models.
Discriminative and generative classifiers
| Logistic regression and support vector machines are discriminative, and the contrast with generative models is worth stating precisely. CS229 defines discriminative algorithms as those that learn p(y | x) directly, like logistic regression, and generative algorithms as those that model p(x | y) and p(y). Naive Bayes and linear discriminant analysis are generative classifiers, while GANs, variational autoencoders, diffusion models and autoregressive models generate data through different objectives. |
The choice has a measurable trade-off. Ng and Jordan’s comparison of naive Bayes and logistic regression found that, under their assumptions, the generative model can approach its best performance with fewer examples, while the discriminative model can reach a lower error in the limit. The generative model families beyond LLMs are compared in what types of generative AI models exist beyond LLMs.
A real-world supervised learning example: diabetic retinopathy screening
A deep convolutional neural network was trained on a development set of 128,175 retinal images, each graded 3 to 7 times by a panel of 54 US licensed ophthalmologists and senior residents for diabetic retinopathy, macular oedema and image gradability. The retinal images are the inputs; the label is the ophthalmologists’ grade. Those labels came from licensed specialists, and the grading ran from May to December 2015.
In computer vision work, that grading protocol (who labels, how many times, how disagreements are resolved) is as much a part of the system as the network architecture.
Email spam filtering is one of the most common examples of classification in use. A model is trained on labelled spam and legitimate emails, extracting features such as the sender, subject line and body copy. Google says its machine learning models, together with its other protections, block more than 99.9 percent of spam, phishing and malware from reaching Gmail inboxes; that is Google’s own claim about its product, not an independent measurement. Fraud detection follows the same pattern, with models trained on fraudulent and legitimate activity and then used to flag suspicious activity in real time, as does loan-default risk assessment in banking.
Why supervised learning needs labelled data, and what labelling takes
Supervised learning needs labels because the loss is the gap between the prediction and the label; with no label there is nothing to compute the error against. The labels also have to exist for the evaluation set, not only for training.
Labelling is manual human labour, and individual workers make mistakes. A 2025 study of a commercial annotation pipeline describes data annotation as essential but laborious and costly, with errors driven by task ambiguity, varying worker skill and fatigue after long hours. Because individual annotators make mistakes, each task is repeated by several annotators and the responses are aggregated, which multiplies the cost of labelling a dataset. In that pipeline a majority vote over the repeats produces the consensus ground truth, and before any of it starts, annotators are trained and calibrated against written guidelines, for example a rule that a pedestrian is a human with both feet on the ground.
The cost can be managed, though not for free. The same study found that a conservative strategy could prune about 22% of assigned annotation tasks while preserving over 99% of the majority-vote labels, and an aggressive one could prune 60% while preserving over 96%. The authors are explicit that pruning always reduces dataset accuracy in exchange for lower cost, so the right level depends on how much error the downstream application tolerates.
Even widely used datasets carry label errors. Northcutt and colleagues (2021) estimated an average of at least 3.3% label errors across the test sets of 10 widely used computer vision, language and audio datasets, including at least 6% of the ImageNet validation set. If a test set contains errors at that scale, a small accuracy difference between two models may be noise in the labels rather than a real difference in the models.
Generative AI also relies on human labelling. TIME reported that the workers labelling toxic text for OpenAI’s ChatGPT safety classifier through the contractor Sama took home between about $1.32 and $2 per hour, and three of them said they were expected to read and label 150 to 250 passages per nine-hour shift. Annotation for reinforcement learning from human feedback is described as more time-consuming still, sometimes requiring specialised expertise.
Is an LLM trained with supervised or unsupervised learning?
An LLM is trained in stages, and the first stage, pre-training, is self-supervised, not supervised. Pre-training starts from random parameters and uses stochastic gradient descent to optimise a self-supervised objective over a large text corpus. Self-supervised learning takes its training signal from the data itself by predicting a hidden part of the input from the visible part, for example hiding words in a sentence and predicting them from the rest. Meta’s account of models such as BERT and RoBERTa describes exactly this sequence: pre-train in a self-supervised phase on large unlabelled text, then fine-tune for a particular task such as topic classification.
Supervised learning enters after pre-training. In the InstructGPT work (Ouyang et al., 2022), OpenAI collected labeller-written demonstrations of the desired behaviour and fine-tuned GPT-3 on them using supervised learning. A dataset of human rankings of model outputs was then used to fine-tune further with reinforcement learning from human feedback. That last stage borrows from a different learning paradigm, covered in how agents learn through trial and error with reinforcement learning.
| Stage | Data | Where the training signal comes from |
|---|---|---|
| Pre-training | Large unlabelled text corpus | The text itself: predict hidden or next words |
| Supervised fine-tuning | Labeller-written demonstrations | Human-provided target outputs |
| RLHF | Human rankings of model outputs | Human judgements of which outputs are better |
Self-supervised is not the same as unsupervised
Sources differ on vocabulary. Google’s introductory guide says generative models are initially trained with an unsupervised approach and sometimes trained further with supervised or reinforcement learning. Other sources, including Meta’s account, call the pre-training stage self-supervised. Meta’s account explains why the self-supervised label fits: the model still learns against a concrete target (the hidden word), a target that comes from the data rather than a human annotator. Unsupervised learning uses unlabelled data and learns its structure without explicit instructions.
Two neighbouring terms complete the picture. Self-training trains a teacher model on a small labelled set, pseudo-labels the unlabelled data it is confident about, and retrains on the expanded set, while pre-training learns representations from patterns in unlabelled data without propagating labels. A 2024 comparative study using BERT on downstream tasks found the pre-train-then-fine-tune paradigm gave the best overall performance, and that self-training offered no additional benefit when combined with semi-supervised pre-training. That result covers encoder models on downstream tasks, not text generation, so it supports the pattern rather than settling every LLM case.
Generative AI still runs on supervised components
The tidy split, with generative AI creating and supervised learning classifying, does not hold in a working system. Generative AI is itself a sub-field of machine learning powered by deep neural networks, it is fine-tuned on supervised data, and it is wrapped in supervised classifiers: the ChatGPT toxicity detector was built by feeding a model labelled examples of violence, hate speech and sexual abuse so it could detect them and filter them out. Meta reports deploying its self-supervised XLM-R model to improve hate speech classifiers in multiple languages, the same pre-train-then-classify pattern turned to moderation.
The paradigm described in the 2024 study is a large unlabelled pre-training set followed by fine-tuning on smaller labelled data, and Meta argues that supervised learning alone is a bottleneck for generalist models precisely because of its appetite for labelled data. InstructGPT’s demonstrations and rankings, ChatGPT’s toxicity detector and evaluation sets all used labelled data. That labelling is where generative and agentic AI R&D meets classical machine learning practice. How this layered stack maps onto the broader taxonomy of ML, deep learning and GenAI is set out in what the key feature of generative AI is for a production team.
FAQ
Is kNN supervised or unsupervised?
k-nearest neighbours is a supervised algorithm: it predicts the label of a new point from the labels of the closest training examples, so it needs labelled data. It is sometimes confused with k-means, which is an unsupervised clustering method that uses no labels.
Is machine learning really AI?
Yes: machine learning is a subset of artificial intelligence focused on systems that learn patterns from data instead of following pre-programmed rules. Supervised learning is one of four categories of ML systems, alongside unsupervised, reinforcement and generative AI.
