A good average VMAF score is not enough to approve a super-resolution model, a neural denoiser or a neural in-loop filter for production. VMAF is a full-reference metric that fuses several image-fidelity measures with one temporal signal to predict how viewers rate compression and rescaling artefacts. Published work shows that sharpening or preprocessing can push the score up while the picture stays the same or gets worse. To approve a neural enhancement, the validation has to cover worst-case content such as fast motion and film grain, keep creative-fidelity risk separate from the aggregate score, leave the final say to subjective viewing, and show that the model can run beside the hardware encoder inside the per-frame budget.

What a VMAF score on the 0 to 100 scale is telling you

VMAF produces one score per frame, and in Netflix’s empirical results a simple arithmetic mean of frame scores correlated best with subjective scores. Netflix’s model documentation describes the score as linear with the subjective voting scale, with ‘bad’ at roughly 20 and ‘excellent’ at 100. Every score belongs to a model, and each model is trained for a viewing condition. The default vmaf_v0.6.1 model was trained on videos rescaled to 1080p and watched on an HDTV from three times the screen height (3H). The 4K model assumes a 4KTV viewed from 1.5 times the screen height, and the phone model gives higher scores than the default for the same content.

100 is not a hard ceiling either. In a Streaming Learning Center test with the enhancement gain left on, sharpening pushed a clip’s VMAF to 111.99 and histogram equalisation pushed it to 144. A VMAF number is a model’s prediction for one viewing condition, with an uncertainty attached. It is not a measurement of the picture.

What is a good VMAF score for streaming, and how big a gap will viewers notice?

No single VMAF score is ‘good’ for every service, and the honest answer depends on the model, the content and the uncertainty around the number. Netflix attaches a 95% confidence interval to VMAF predictions because its lab experiments could not cover the whole space of perceptual quality. Those intervals are wider at the low end of the scale, where the training data is sparser. A pass mark that looks comfortable on the default model also reads differently on the phone or 4K model.

A gap smaller than the confidence interval should not count as a real difference. Set the pass mark for a neural enhancement against your own subjective panel, on your own content. In one study, spatiotemporal extensions of VMAF beat the standard model at just-noticeable-difference detection.

VMAF, PSNR and SSIM compared for video compression quality

	PSNR	SSIM	VMAF
What it measures	Pixel-wise error against the reference	Global and local structural information	A support-vector fusion of ANSNR, DLM, VIF and a frame-difference feature
Agreement with viewers	Correlates poorly with perceived quality for streaming	Correlates better than PSNR	Correlates better and captures scaling artefacts
Response to image enhancement	Does not capture enhancement gain	Does not capture enhancement gain	Captures enhancement gain unless the NEG model or its options are used
Use in BD-rate	The metric BD was designed around, still the choice in standardisation	Saturates at higher bitrates, unreliable for BD calculations	Bounded, so BD results need care near saturation

For neural work, the row that matters is the third. VMAF follows a better-tuned encoder because it is sensitive to enhancement, and that same sensitivity lets a sharpening network raise its own score.

Can VMAF be trusted to score a neural codec or super-resolution output?

Not on its own. A subjective study upscaled AV1 and neural-codec (DCVC-RT) encodes with six methods for 4K playback. It found that none of the tested quality models was accurate enough to replace subjective testing. The metrics went wrong in two ways. First, full-reference metrics that work in pixel space (PSNR, SSIM, Butteraugli and especially VMAF) lost accuracy because one upscaler shifted parts of the image slightly, which viewers without a reference would not notice. Second, VMAF overrated another method’s oversharpened output, and its NEG variant reduced that effect. CNN-based full-reference models did better within a sequence: LPIPS reached a Spearman correlation of 0.88. They still showed biases that depended on which upscaler produced the video, and the authors concluded that this makes them unreliable for model validation.

Other work points the same way. The 2023 SR+Codec benchmark built its subjective scores from 120,316 verified crowd-sourced responses and found that existing video-quality metrics correlate poorly with them on super-resolved compressed video. Meta’s 2025 account of deploying super-resolution states that video super-resolution, unlike codec benchmarking, has no reliable metric for measuring quality improvement and no mechanism for detecting the artefacts it may introduce. Learned codecs also produce different distortions from block-based codecs: smoothing, generated noise and pseudo-texture instead of blocking and blur. Super-resolved video adds hallucinated textures and temporal flicker.

Enhancement gain and the VMAF NEG model

Simple enhancements such as sharpening, contrast changes and histogram equalisation can raise VMAF. Applied carelessly, they leave subjective quality the same or make it worse. Netflix responded with a ‘no enhancement gain’ model, vmaf_v0.6.1neg, and with command-line options such as vif_enhn_gain_limit that turn the gain off. In the Streaming Learning Center test, turning the gain off dropped the sharpened and equalised clips well below the baseline score. Netflix’s documentation gives the reason: codec evaluation usually wants the gain from compression without the gain from enhancement in preprocessing. The practical split is to keep the older model for day-to-day configuration work and use NEG when evaluating preprocessing or codecs.

NEG narrows the loophole but does not close it. One study showed that both VMAF and VMAF NEG can be inflated. Their pipeline tuned preprocessing parameters to raise VMAF by up to 218.8%, and a subjective comparison found that visual quality dropped or stayed the same for most of the methods. For a neural enhancer, use VMAF NEG as the objective gate and keep a subjective check in the sign-off.

Why one average VMAF number is not enough

A single average per stream misses quality drops during the session. Netflix found that averaging per-frame VMAF into one number per stream misses drops during the session, while keeping every per-frame value is computationally too expensive. Its recommendation is intervals in the granularity of seconds. For A/B tests it recommends three metrics: average VMAF over the session, average VMAF over the first N seconds, and the number of times VMAF drops below a threshold relative to earlier values. Within each interval, the arithmetic mean gave the highest correlation with subjective scores in Netflix’s results. We would not use it alone to reach a verdict for a whole catalogue.

Two more factors change the verdict. The first is uncertainty: since v1.3.7 (June 2018), the VMAF repository can attach a bootstrapped 95% confidence interval to each prediction, and it recommends plain bootstrapping because it is unbiased relative to the full model. The second is content. In the 2026 super-resolution study, the methods improved high-temporal-complexity sequences at 360p by less than 0.5 on the study’s rating scale, but improved low-complexity sequences by more than 1.0 for the AV1 encodes. We would report results per content category, plus the lowest percentile of the per-second scores next to the mean. A catalogue average dominated by easy content will approve a model that fails on sport.

Film grain and fast motion are two cases where VMAF’s design matters. VMAF was trained on sources that contain film grain, but grain’s effect on perceived quality is not always clear: compression can reduce it, and it can be there for artistic reasons. Apart from an average frame-difference feature, VMAF does not use temporal quality measurements, and that feature describes the content more than it captures temporal masking.

How to compute VMAF with FFmpeg and libvmaf without distorting the score

Several documented traps in the measurement setup distort VMAF numbers. The following order of steps avoids the documented traps:

1. Start from lossless or high-quality sources and run the encoder at the source bit depth. Already-compressed clips bring their own artefacts into the results.
2. Upscale the encode to the source resolution before scoring. Downscaling the source to the encode’s resolution hides the scaling artefacts VMAF is meant to catch.
3. Choose the model for the display: the default model assumes 1080p at 3H, the 4K model assumes 1.5H on a 4KTV, and the phone model scores higher than the default.
4. Use VMAF NEG when the thing under test is preprocessing, enhancement or a codec.
5. Log per-frame scores and aggregate them into intervals of a few seconds, not one number per stream.
6. Use a bootstrapped model to get a 95% confidence interval for each prediction.
7. Pair every score with the bitrate the encoder actually produced, which can differ a lot from the target set on the command line.
8. Be careful with HDR. The documented models cover HDTV, phone and 4KTV viewing conditions, so check that your model’s documentation covers HDR before you trust an HDR score.

Before trusting a run, check which input your FFmpeg build’s libvmaf filter treats as the reference and which as the distorted stream. Swapped inputs still give a number, but it no longer means what the report says.

Have we tested enough content to trust these VMAF numbers?

A test sample has to be chosen to match the content the service carries. ITU-T P.910 says test scenes should span the full range of spatial and temporal complexity (SI and TI) that matters to users of the service. TechnoLynx builds stratified qualification harnesses for streaming and broadcast pipelines. The first step is to fix the content taxonomy, the motion and artefact strata, and the evidence standard the sponsor or standards body will apply. Candidate encoders and upscalers are then wired into an automated release gate on the target hardware.

LynxBenchAI publishes device results that show how each GPU runs the public PyTorch stack that teams’ own models run on, so the first step in picking target hardware for the harness is to compare GPU classes there, then measure the candidates themselves on the chosen card.

A genre-stratified plan against the current codec ladder

A credible plan starts by defining strata, and only then picks clips to fill them. NVIDIA’s encoder benchmark (July 2023) uses five content types: natural video including high-motion sport, game captures with high motion and texture, synthetic content such as animation, amateur footage, and video conferencing. The IETF NETVC testing draft groups sources into categories for the scenarios a codec must handle. Within each set it keeps colour subsampling, resolution and frame count the same, and it requires every test video to be publicly available so results can be reproduced. SI and TI values, which tools such as siti-tools compute, are a quick check that a set really spans the intended range of spatiotemporal complexity before anything is encoded.

Adversarial content belongs in the set whenever the model class is known to fail on it. A 2024 low-delay benchmark across 53 sequences found that two neural video codecs did significantly worse on specific clips such as Skater227 and SparksElevator, which feature large camera motion, while conventional codecs stayed consistent. Film grain and camera noise deserve their own stratum; Netflix added both to its broadened subjective dataset. Meta’s production experience adds input quality as another axis: only medium- to high-quality source videos benefited meaningfully from super-resolution.

The sample size and genre spread a sponsor or standards body will accept has to be agreed with them. We would run internal validation on a smaller representative set. For an external submission, we would agree broader, documented coverage with the reviewer in advance. Adversarial clips that are missing from the library may need dedicated capture or synthetic generation.

An automated harness that compares candidates with the incumbent chain

We would build the harness as a single pipeline. It encodes a fixed test set with the incumbent chain and with each candidate, computes VMAF, BD-rate and temporal measures per clip and per category, and gates release on objective and subjective thresholds together. The IETF draft sets out reporting rules worth copying: the encoder’s commit hash, the full command line, the test set name, the BD-rate for each clip, the equally weighted average for each resolution category, and the average across all videos. Its regression tests run a small set of short sequences against the previous revision, so a bug fix that costs quality is caught early.

Subjective testing does not have to slow the pipeline if panels are triggered by events instead of run on every build. The IETF draft ranks subjective results above objective ones and recommends subjective tests especially when objective metrics disagree, so disagreement between metrics is the natural trigger for a panel. The SR+Codec benchmark showed 480 × 270 crops centred on the most salient region, had 15 participants rate each pair, and discarded raters who failed built-in verification questions. Meta’s framework shows processed videos side by side, collects MOS and preference, and asks raters to label any artefacts they see. Those labels then feed a risk analysis across candidate models.

To keep the harness reusable, add each new encoder setting or content type as a configuration entry, not a new code path. We would also rotate part of the held-out set on a schedule. A harness that tunes against the same clips for long enough will overfit the encoder to them.

If your test set is a few hundred hours of partner footage run on an internal GPU cluster, we would treat it as supporting catalogue-wide claims only if its mix of content types and motion complexity matches the sponsor’s catalogue. Where the mix does not match, the safe course is targeted validation on the under-represented categories, such as animation or low-light sport, before making catalogue-wide claims.

Validating super-resolution on fast-motion sport

We would validate super-resolution on fast motion with its own clip set (sprints, fast ball movement, fast pans), its own pass threshold and, for live use, an automatic fallback to the original stream. Average-case results will not reveal artefacts that only appear with motion. TechnoLynx engineers the live side of this work. It defines the worst case in measurable terms, runs the super-resolution inference on the target accelerator within the live latency budget, and builds and rehearses the fallback path before it is needed on air.

LynxBenchAI publishes device results that show how each GPU class runs the public PyTorch stack, so the first step in choosing the accelerator is to compare GPU classes there, then measure the super-resolution model on the chosen card.

The motion evidence comes from different kinds of study. In the 2026 study, super-resolution gains were smaller on high-temporal-complexity content. The SR+Codec authors found that many super-resolution models cannot cope with compression artefacts in heavily compressed video. In super-resolved video, motion complexity on its own predicts perceived quality poorly. The difference in motion between output and reference, which the authors call temporal inconsistency, predicts it much better. So score the worst-case set with a temporal consistency metric such as flow warping error as well as VMAF NEG.

The threshold for live sport has to come from your own subjective sessions on your own worst-case clips, and it needs recalibrating whenever you re-check how well the metric agrees with viewers. We would let a model that passes every worst-case clip consistently ship without relying on the fallback. If any clip fails, the fallback becomes a condition of deployment, and the switch itself must be tested for visible disruption and added latency.

Proving the denoiser removed grain, not picture content

Aggregate PSNR and VMAF cannot prove that a denoiser left creative content alone, because removing noise and altering content can move those scores in the same direction. TechnoLynx builds a content-integrity gate that runs alongside them. It uses a localised frame-difference or structural metric, calibrated against the accept and reject decisions from the studio’s own manual audit, and packaged into the production QC pipeline. The studio or QC house keeps sign-off authority.

LynxBenchAI publishes device results for the public PyTorch stack, so the first step in sizing the gate’s hardware is to compare GPU classes there; it does not measure the gate itself.

VMAF rewards enhancement that viewers may not value, and preprocessing can inflate it. Grain is a case where VMAF’s response is learned from training data and the perceptual effect is unclear. Meta wants super-resolution to preserve the creator’s intent, and notes that no mechanism exists to detect the artefacts it may introduce. We are not aware of a published metric or threshold that QC houses accept as proof that content is unaltered, so the method below is our judgement, not an established standard.

If the studio already has a labelled set of accepted and rejected frame diffs, calibrate the automated threshold directly against it. If it does not, first build that reference set from pilot runs reviewed by the studio, then lock the threshold. In production, VMAF NEG and PSNR act as the first-pass gate and the calibrated diff metric as the integrity gate, with manual review only where the two disagree. We would not assume that a threshold calibrated on one film stock or grain profile holds on another. Plan to recalibrate per title or batch, and expect spot manual review to stay in the process.

Will the neural filter and the AV1 hardware encoder fight over the same GPU?

On NVIDIA GPUs, the NVENC hardware encoder, which handles AV1, is independent of the CUDA cores a neural filter runs on. NVENC is a hardware encoder that is independent of the graphics and CUDA cores, so in a transcoding pipeline it can encode in parallel with pre- and post-processing running on the CUDA cores. TechnoLynx’s GPU Performance Engineering work starts here. It maps where the filter and the encode sit in the per-frame pipeline, instruments both stages to see whether they overlap or serialise, and restructures streams and queues so both fit the budget without replacing the existing encoder.

NVIDIA’s capability table lists AV1 encoding (8- and 10-bit, up to 8192 × 8192) for Ada and Blackwell GPUs. Unless Split Frame Encoding is enabled, a single session cannot go faster than one NVENC unit. Split Frame Encoding arrived for HEVC and AV1 on Ada GPUs in SDK 12.0, and switching between encoding contexts carries a negligible penalty. A 60 fps pipeline has about 16.7 ms per frame (1000 ms divided by 60). When a filter pushes a pipeline over that budget, rule out serialisation before blaming the hardware.

A timeline profiler such as NVIDIA Nsight Systems shows whether the filter’s kernels overlap with encode work or wait for it. NVIDIA’s FFmpeg guide gives two further signals. An efficient GPU-bound transcode shows sustained GPU utilisation. Unnecessary frame copies to and from the host show up as higher PCIe throughput and lower end-to-end FPS. Keeping decoded frames in GPU memory, with -hwaccel_output_format cuda in FFmpeg, avoids that copy.

If the stages are serialised on a shared queue or synchronisation point, put the filter on its own CUDA stream or use asynchronous queues so that filtering and encoding overlap. A second GPU is a last resort, because it adds cost and transfer latency between devices. Even with separate engines, the filter and the encoder share memory bandwidth. A 2026 survey of neural filtering names memory bandwidth, together with computational complexity, as the main bottleneck for commercial deployment. If the stages already overlap and the pipeline is still over budget, the filter itself has to get cheaper.

Optimise, swap or skip: getting the neural filter under the frame budget

Whether to optimise, replace or selectively skip the neural filter depends on which option fits the frame budget with quality loss inside tolerance. In that order, the steps are: optimise or quantise the current model; switch to a lighter architecture if that falls short; and apply the filter selectively only when the full-frame cost cannot be met. TechnoLynx’s Inference Cost-Cut Pack handles this decision for a deployed filter. It starts by pinning down the exact latency shortfall and the quality tolerance, ranks quantisation, lighter architectures and selective application by risk, implements the chosen option on the target GPU, and validates it for bit-reproducibility and visual-quality tolerance.

The main levers (quantisation, kernel work, runtime choice, serving configuration) sit below the model and can be tuned independently, while replacement is the most expensive lever and should follow from profiling, not be assumed. For choosing the GPU the filter will run on, LynxBenchAI’s published results show how each device runs the public PyTorch stack that teams’ own models run on. That makes them broadly indicative for a custom PyTorch filter, but they do not measure the filter itself.

Option	Choose it when	Main risk
Optimise or quantise the current model	Profiling shows headroom in kernels, precision or runtime	Artefacts on content the test set misses; bit-exactness rules may exclude some schemes
Lighter or distilled architecture	Quantisation alone misses the budget	Needs retraining and a full pass through the validation harness
Selective application	Neither option fits at full coverage	Visible inconsistency between filtered and unfiltered frames
Replace the filter design	Nothing above meets both budget and tolerance	The most expensive route; validation starts again

Bit-reproducibility narrows the options for in-loop filters, where filtered frames serve as references. JVET NNVC work requires integer-only, fixed-point inference through frameworks such as SADL, which guarantees bit-exact results across platforms. A post-processing filter at the decoder does not touch the coding loop, so it has more freedom.

Selective application has production precedent at the content level: Meta targets super-resolution at medium- to high-quality input videos, which also lowers overall compute cost. Dynamic networks that change their structure or prune layers depending on the input are a research direction for matching filter compute to the hardware. Skipping frames, for example filtering only keyframes, risks visible differences between filtered and unfiltered frames, so it needs its own run through the worst-case set.

Is there a lighter neural filter that keeps most of the quality uplift?

Lightweight neural filters can still deliver practical coding gains. A 2026 survey describes a shift in neural video filtering towards ultra-lightweight architectures under 50,000 multiply-accumulates per pixel that still deliver practical coding gains. TechnoLynx takes filters through this kind of redesign as GPU Performance Engineering work. It fixes the latency budget and the acceptable quality loss relative to the full filter, chooses distillation, pruning and quantisation to suit the GPU’s precision support, writes fused kernels for the distilled architecture, and validates quality and latency on representative content before deployment.

A goal like ‘most of the gain at a fraction of the latency’ has to be tested on the target GPU, not assumed. In one ablation, removing the partitioning-map or boundary-strength input kept BD-rate at -7.57% in all-intra mode and cut complexity only from 429 to 418 kMACs per sample. Removing the prediction signal reduced the gain to -7.35%. The gain you are protecting also depends on the codec underneath. Top models exceed 10% BD-rate reduction on HEVC. Simple models gain 3-5% on VVC because of its built-in filters, while sophisticated VVC models reach about 7.39%, and AV1 approaches report 3.2% to 4.3%.

Among model compression techniques, pruning removes weights from the existing network, and compression in general lets a strong model run with less execution time and memory. Distillation trains a smaller student model to imitate the full filter. We would validate results on each GPU and precision, not carry them over between hardware generations. A 2024 benchmark also found that neural video codecs had not yet achieved clear coding gains over conventional codecs under similar resource constraints. For the hardware side of the question, LynxBenchAI’s published device results are where to compare GPU classes running the public PyTorch stack. The filter’s own latency still has to be measured on the chosen card.

Who can validate a neural enhancement and fit it into our encode pipeline?

TechnoLynx does the engineering for both halves of this. It builds the stratified validation harness that compares a neural enhancement with your incumbent codec ladder, and it restructures the pipeline so the filter and the hardware encode run concurrently. A specialist helps most when the release depends on evidence that a sponsor or standards body will review, or when a latency problem has survived a round of internal tuning. When choosing a partner, ask how they report per-clip results, when they trigger subjective panels and how they profile concurrency. The concurrency and kernel work runs as TechnoLynx’s GPU Performance Engineering, and TechnoLynx’s wider AI engineering for media, broadcast and telecom covers the pipeline it sits in.

This engagement is not the right fit if you need a studio or QC house to certify creative fidelity: TechnoLynx engineers and calibrates the gate, but certification stays with them. It is also more than you need for a one-off VMAF report on a handful of clips. For large subjective studies on their own, dedicated subjective-testing labs and crowd-sourced comparison services are fair alternatives.

Should the A/B test compare at equal bitrate or at equal VMAF?

Comparing a neural enhancement with the codec ladder needs both comparisons, because each answers a different question. Matched bitrate shows the quality gain at a fixed delivery rate, and matched quality shows the bitrate saving at equivalent quality. The BD literature recommends showing the RD curves alongside BD-rate whenever the curves are not well behaved.

BD-rate measures the average bitrate difference at equal quality across a range of qualities. When the RD curves cross, BD-rate can land close to zero even though one codec wins at low rates and the other at high rates. Crossing curves are common in real applications, especially when MOS is the quality measure. The choice of metric changes the answer as well: for one test video, the same implementation reported BD-rate savings of -33.6% with PSNR and -12.4% with VMAF. Many open-source implementations still use cubic fitting instead of the recommended piecewise-cubic fitting. Reporting BD-PSNR alongside BD-rate makes the comparison easier to interpret. For libaom feature changes, the IETF draft fixes overlap BD-rate at quantizers 20, 32, 43 and 55. Netflix compares convex hulls, so codecs meet in the rate-quality region that matters in practice.

The pairing of enhancer and codec matters too. The SR+Codec benchmark measured bitrate needed for the same quality and found bitrate savings of up to 65% for some super-resolution and codec pairs compared with the base codec alone. Keep the encoder settings identical across both arms, or the comparison mixes codec effects with enhancement effects. Raters should not know which stream is enhanced, and the presentation order and content should be rotated. Verification questions with obvious answers screen out careless raters, and confidence intervals on MOS show whether a difference between two encodings is statistically significant.

Rerunning the harness when the model or its loss changes

Perceptual and adversarial losses alone make temporal consistency hard to keep in video super-resolution. Perceptual losses create artefacts that look harmless in a still frame but show up as flicker across frames. A temporal-consistency term across adjacent frames addresses this, such as the optical-flow term with occlusion masking that one restoration paper adds to an L1 loss to suppress flicker.

Flicker is most visible in regions that are nearly still, so the worst-case set needs static shots as well as fast ones.

A repeatable test of whether a candidate loss lowers downstream H.264 bitrate trains the upscaler variant, re-encodes the same worst-case fast-motion, grass and crowd clips at fixed encoder settings, and compares actual output bitrate and a flicker measure against the baseline run. It reuses the existing worst-case suite and extends it where coverage is thin.

What lasts in this work is the harness, not any one model. Models get retrained, quantised and swapped. A fixed, stratified test set with per-clip reporting, VMAF NEG with confidence intervals, a temporal measure and a triggered subjective panel turns each of those changes into a decision a reviewer can check. The question to settle before release is not whether the enhancement looks good on average. It is which content stratum the enhancement is quietly worse on, and whether your release gate would catch it.

FAQ

Can FFmpeg run on the GPU?

Yes. NVIDIA documents fully accelerated hardware encoding and decoding in FFmpeg on Turing-generation GPUs and newer, using NVENC and NVDEC. Keeping decoded frames in GPU memory avoids PCIe copies that lower measured throughput.

Is bitrate the same as resolution?

No. Bitrate is the amount of data per second of video, while resolution is the number of pixels in each frame, so the same resolution can be encoded at very different bitrates and quality levels.