Feature extraction is the step that turns raw pixels into a compact numerical representation a downstream model can reason about. In production computer vision systems we still see classical extractors β SIFT, ORB, HOG, edge and contour operators β running alongside CNN and transformer backbones, not replaced by them. The current narrative treats classical feature extraction as obsolete; the reality on the engineering side is that hybrid pipelines, where each stage chooses between classical and deep components, regularly run 3β10Γ cheaper than uniformly-deep pipelines for the same task accuracy (observed pattern across our CV engagements; not a benchmarked rate). Knowing when to reach for which layer is the actual skill. This article walks through what the two layers do, where the classical methods still win in 2026, and how they compose with deep features in real deployments. What image processing actually does Image processing sits at the front of the pipeline. Its job is to turn whatever the sensor produced into something the next stage can rely on. That includes noise reduction (Gaussian blur, median filtering, non-local means), contrast normalisation (histogram equalisation, CLAHE), geometric correction (lens undistortion, rectification), and colour-space transforms (RGB to YCbCr or LAB depending on what you want to isolate). Edge detectors such as Sobel and Canny highlight boundaries; morphological operators clean up the result. None of this is glamorous, and none of it is optional. A clean, well-conditioned image lets every later stage β classical or deep β do less work to reach the same answer. We pay close attention to this front-end because most of the surprises in production CV systems trace back to an environment the preprocessing was not tuned for: a new lighting rig, a different camera batch, a sensor firmware change that altered the gamma curve. What feature extraction actually does Feature extraction is the next step. It selects and returns the most informative parts of an image, reducing the data set size while preserving what matters for the downstream task. Two broad families exist. Classical extractors detect geometric structure directly. Harris corners and FAST find points of interest. SIFT, SURF, and ORB build local descriptors around those points that survive scale and rotation changes. HOG aggregates gradient histograms over cells, which is why it works so well for pedestrian detection. PCA compresses high-dimensional patches into a small number of principal components, which is useful when you need to denoise or compare patches cheaply. Deep extractors use the intermediate activations of a trained backbone β ResNet, EfficientNet, ViT, DINOv2, CLIP β as the feature representation. Early layers learn edge-like and colour-blob filters that look remarkably similar to what classical operators compute by hand; deeper layers combine these into textures, parts, and object-level features. The trick most teams use is to freeze the early layers and fine-tune the later ones on the target data set, a transfer-learning pattern that has been the default starting point for image classification and detection for years. Where classical features still beat deep features in 2026 The three honest wins There are three situations where, in our experience, a classical feature stage is still the right engineering call: Situation Why classical wins Very little labelled data Classical descriptors generalise out of the box. A pretrained CNN can do this too, but only if the domain is close to ImageNet; in medical, industrial, or remote-sensing data the gap is often wide enough that hand-engineered features are more reliable. Microcontroller or low-power edge target ORB and HOG run in single-digit milliseconds on a Cortex-M class device. Even quantised MobileNets struggle there. For battery-powered cameras and embedded inspection rigs, this is the difference between a system that ships and one that does not. Interpretable, debuggable features In regulated applications β industrial inspection with audit trails, forensics, certain medical workflows β engineers need to point at a specific feature and explain why the system fired. Classical features support that directly; deep activations do not, without significant XAI scaffolding. For everything else β abundant labels, a GPU available at inference, no interpretability requirement β a pretrained deep backbone is the default starting point and the right one. What Nixon and Aguado get right The Nixon and Aguado framing of feature extraction as a layered choice β point features, region features, shape features, texture features β survives the deep-learning era because it describes what is being extracted, not which algorithm extracts it. A modern team building a hybrid pipeline still has to choose which feature category the task needs, then pick an implementation. The deep-only narrative skips that step and ends up extracting features the task does not need, at compute cost the deployment cannot afford. How the two layers compose In practice, classical and deep extractors sit in the same pipeline more often than either community admits. A common pattern: classical preprocessing and ROI extraction in front of a CNN. A cheap edge or contour pass crops the image to the region of interest β a license plate, a defect candidate, a product label β and a small CNN then classifies or reads what is inside. The CNN runs on a much smaller image, which is where most of the compute saving comes from. Another pattern: classical features for calibration, deep features for the task. Self-driving stacks use deep feature extractors (BEVFormer, transformer-based fusion networks) over multi-camera and lidar inputs to detect lanes, vehicles, and pedestrians. The calibration and ground-truth labelling steps that sit underneath those models still rely on classical checkerboard detection and SIFT-style matching. The two layers do not compete; they handle different problems in the same system. A third pattern: classical descriptors for sparse matching, learned descriptors for dense matching. Stereo and structure-from-motion pipelines mix ORB or SIFT for the sparse keypoint correspondences with learned descriptors such as SuperPoint or LoFTR for the dense regions. The depth map is then derived by triangulation across views. Which feature types translate into ML model inputs? Not every feature extraction technique is meant to feed a downstream model. Some β heatmaps, attention overlays, segmentation visualisations β exist for human inspection. The ones that translate cleanly into ML inputs share a property: they produce a fixed-size, numerically stable vector or tensor per region or image. Translate well into model inputs: SIFT/ORB/SURF descriptors, HOG vectors, PCA components, CNN activations, ViT embeddings, CLIP embeddings. Primarily for visualisation or analysis: raw edge maps, segmentation overlays, attention rollouts, Grad-CAM heatmaps. Mixing these up is a common source of pipeline bugs: a visualisation surface gets fed into a classifier and the model trains on artefacts of the rendering rather than the underlying image structure. When to write a classical-CV feature stage instead of fine-tuning The question we hear most often from engineering teams is the inverse of βshould we use deep learning here?β It is: when is fine-tuning a model the wrong answer? Reach for a classical feature stage when at least two of the following are true: The deployment target cannot run the deep model at the required frame rate, and quantisation or pruning is not enough. The labelled data set is small (low hundreds of examples) and synthetic augmentation cannot bridge the domain gap. The task is geometrically well-defined β corner matching, shape measurement, alignment β and a learned model would mostly relearn geometry it already has a closed-form solution for. The regulator or auditor needs to see, on paper, what the system is extracting and why. If only one is true, the hybrid pattern usually wins: classical front-end, deep back-end. If none are true, fine-tune a pretrained backbone and move on. Feature extraction next to segmentation and pattern recognition Image segmentation, feature extraction, and pattern recognition are sometimes presented as alternatives. They are not β they are sequential stages in a typical production pipeline. Segmentation isolates the region of interest (a tumour, a defect, an organ). Feature extraction describes that region in a way the next stage can use. Pattern recognition β classification, matching, regression β produces the decision. In a deep-only pipeline a single network can fold all three into one forward pass, which is part of why this distinction has blurred. In hybrid pipelines, and in any system where the stages are tuned independently for cost or auditability, the three remain distinct. We cover the segmentation side of this in our image segmentation methods walkthrough. Practical notes from production A few things we pay close attention to when designing these pipelines: Profile the front-end first. Image processing and ROI extraction often dominate the latency budget on edge devices. Optimise there before reaching for a bigger model. Match feature scale to task scale. SIFT at 128 dimensions is overkill for some matching tasks and underspecified for others. The same applies to choosing which layer of a CNN to read features from. Watch for distribution drift on the front-end. A change in camera, lens, or lighting often breaks classical preprocessing before it visibly degrades the deep model. Logging the intermediate statistics β mean intensity, edge density, ROI count β catches this earlier than logging accuracy alone. Tooling. OpenCV for the classical stack; PyTorch or TensorFlow for the deep stack; ONNX and TensorRT for deployment on NVIDIA hardware; quantisation-aware training when the target is a microcontroller. For the deeper architectural walkthrough on hybrid CV pipelines and where the classical layer sits in the stack, see Feature Extraction and Image Processing in Computer Vision: The Classical Layer That Still Matters. FAQ What is feature extraction in computer vision? Feature extraction is the step that turns raw pixels into a compact numerical representation a downstream model can reason about. Classical methods (SIFT, SURF, ORB, HOG) detect corners, edges, and gradient histograms; modern deep methods use the intermediate activations of a convolutional or transformer backbone (ResNet, EfficientNet, ViT, DINOv2, CLIP) as the feature representation. The classical methods remain useful for low-power, training-data-poor, or interpretability-critical applications; the deep methods dominate where labelled data and compute are available. How does feature extraction relate to image processing? Image processing is the front of the pipeline (denoising, contrast normalisation, geometric correction, colour-space transforms); feature extraction is the next step that produces the representation used for classification, detection, or matching. In a modern deep pipeline the two often blur together because the early layers of a CNN learn their own preprocessing; in classical and hybrid pipelines they remain distinct stages with separate tuning. When should you use classical feature extraction instead of a deep model? Three situations where classical features still win in 2026: when you have very little labelled training data and need a method that generalises out of the box; when you need real-time performance on a microcontroller or low-power edge device where a CNN will not fit; when you need interpretable, debuggable features for a regulated application (industrial inspection, forensics). For everything else, a pretrained deep backbone is the default starting point. How is feature extraction used in self-driving cars and 3D vision? Self-driving stacks use deep feature extractors (BEVFormer, transformer-based fusion networks) over multi-camera and lidar inputs to detect lanes, vehicles, and pedestrians, with classical features still used for calibration and ground-truth labelling. Stereo and structure-from-motion pipelines use a mix of classical descriptors for sparse matching and learned descriptors (SuperPoint, LoFTR) for dense correspondence; the depth map is then derived by triangulation across views. Image credits: Freepik.
