VR rendering on a headset is a budget problem before it is a graphics problem. The renderer has to sustain 72 to 120 frames per second in stereo while keeping motion-to-photon latency under 20 ms, and the target XR systems research uses for AR is tighter at 5 ms. Foveated rendering, variable rate shading (VRS) and reprojection are the tools that make that budget reachable, but each buys something different. On a standalone headset, all three sit under a thermal ceiling that decides what the device can hold once it is warm, not what it shows on a cold demo.
VR rendering budgets per refresh rate
The display refresh interval sets the per-frame budget. The application and reprojection have to finish inside one display interval, because missing vsync loses an entire frame.
| Refresh rate | Per-frame budget | Half-rate interval (frame missed) |
|---|---|---|
| 72 Hz | 13.9 ms | 27.8 ms |
| 90 Hz | 11.1 ms | 22.2 ms (2 × 11.1) |
| 120 Hz | 8.3 ms | 16.6 ms (2 × 8.3) |
Motion-to-photon latency is a separate quantity from the frame budget. The ILLIXR open-source XR testbed study (arXiv, 2020) computes it as the age of the IMU motion sensor sample behind the pose, plus the time reprojection takes, plus the wait until pixels start to appear, so a reprojection pass that misses vsync shows up directly as added latency. That formula leaves out the time for all pixels to reach the screen, which the same study puts at 4.2 ms on its setup.
A measurement study of the HTC Vive, Oculus Rift CV1, Rift S and Valve Index found mean latencies of 21 to 42 ms at the start of a sudden movement, falling to 2 to 13 ms once motion prediction could account for the delay, within about 25 to 58 ms of movement onset. For sudden movements, where recent tracking shows no consistent trend, prediction’s improvement is nullified. The same study traces the residual delay to a pipeline in which simulation, render and presentation each take a frame, with prediction running roughly two frames ahead.
One practical consequence sits in application code. Driving game logic from the headset or controller pose read in the Update loop, rather than the before-render loop, adds one HMD frame of latency.
GPU utilisation misleads once frames drop
Meta’s missed-frames documentation shows why a utilisation graph can mislead. When an app on a 72 Hz display falls to half rate, the GPU is measured across a 27.8 ms interval, so the 13.9 ms of work needed to return to full rate only fits once utilisation drops below roughly 50%. A team reading “60% GPU” at half rate as headroom would be wrong, since the frame still does not fit.
How much eye-tracking latency foveation tolerates, and what it leaves of the frame budget
Foveation depends on eye-motion latency, whose budget of up to 50 ms is far looser than the 20 ms for head motion. Albert et al. (ACM TAP, 2017) found no significant drop in acceptable foveation for added eye-tracking latency of 20 to 40 ms, while 80 to 150 ms caused a significant reduction, suggesting total system latency of 50 to 70 ms can be tolerated. Singh et al. (IEEE VR, 2023) work to a tighter figure: up to 50 ms of total eye-motion-to-photon latency, with the gaze tracker itself needing to stay under 15 ms.
Shipping hardware sits close to that edge. Meta (February 2023) measured end-to-end ETFR pipeline latency of 46 to 57 ms in its UE4 test app, depending on rendering load, and UploadVR reports Quest Pro’s first-generation eye tracking at around 50 ms end to end. A 2021 i-Perception comparison of the Fove-0, Varjo VR-1 and HTC Vive Pro Eye measured eye-tracking latencies from 45 to 81 ms. Measured eye-tracking latencies therefore sit at or near the edge of that tolerance, and some exceed it. That is a reason to treat the foveal radius as conservative rather than shrinking it to the limit.
Foveation does not change the frame budget itself. At 90 Hz the app still has 11.1 ms. What eye-tracked foveation changes is how much fragment work fits inside it, and the gaze tracker’s own compute is a cost the savings must outweigh. Meta notes that its ETFR pipeline keeps the eye cameras running at display rate, fully synced to rendering, specifically to keep that latency down.
Fixed versus eye-tracked foveation: measured shading savings on Quest and PCVR
Foveated rendering cuts fragment shading, so it helps fragment-heavy apps (complex materials, high render resolution) and does little for vertex-bound ones. Fixed foveated rendering (FFR) assumes the user looks near the centre and lowers resolution towards the edges, while tracked or eye-tracked foveated rendering (ETFR) moves the full-resolution region with the gaze. Because the low-density region stays in peripheral vision, ETFR can use more aggressive foveation maps and save more GPU time than FFR.
The measured gains on Meta’s standalone hardware are real but bounded. In Meta’s performance test app as reported by UploadVR, FFR saved 26 to 36% at default resolution and ETFR 33 to 45%; at 1.5× resolution FFR saved 34 to 43% and ETFR 36 to 52%. In that test app, higher base resolution meant larger savings. Sony claims larger figures for PlayStation VR2 (around 60% for FFR and 72% for ETFR), which UploadVR attributes to different GPU architectures and higher resolution; those figures are a vendor claim, not a matched test.
Hardware decides which kind you can use. Meta’s documentation lists ETFR on Meta Quest Pro and Meta VR Glasses only; Quest 2, Quest 3 and Quest 3S have no eye-tracking hardware. When eye tracking is unavailable or switched off, an ETFR app falls back to Dynamic High FFR.
A key standalone-versus-tethered difference is the cost of running gaze tracking. Singh et al. (2023) found that with a baseline gaze tracker, tracked foveation used more energy than FFR, and an optimised gaze tracker co-designed with the renderer brought total energy up to 40% below FFR, but only in the Bistro and Sponza scenes; San-Miguel and Classroom showed no saving. The same paper notes that most headsets supporting tracked foveation are tethered and run on high-end GPUs. Its benefit over FFR was at most 1.5× and appeared in only two of its scenes, against 1.5× to more than 3× over unfoveated rendering.
Meta recommends spending the savings on eye-texture size: an app on FFR-3 can move to ETFR-2 with a larger eye texture and 2× MSAA at around the same GPU time.
What is the per-eye fragment-shading cost difference between standalone Quest 3 and tethered PCVR for the same content, and how should that change which workloads are baked versus rendered live?
Meta Quest 3’s Snapdragon XR2 Gen 2 runs in a 4 to 6 W typical envelope, up to 10 W peak, with passive cooling, according to a September 2025 arXiv feasibility study. The ILLIXR study’s desktop (an RTX 2080 system) has a thermal design power far above anything acceptable for mobile XR and met the VR motion-to-photon target for virtually all frames. On its low-power Jetson configuration, the application and reprojection stages were severely degraded, sometimes by more than an order of magnitude.
The XRgo study (ACM MMSys, 2025) gives a comparable picture on a newer embedded board: rendering locally on a Jetson AGX Orin at a 30 W profile meant to emulate headset thermal design power, its Bistro scene averaged 20 application FPS and Spaceship 8 FPS, both missing target. The authors also note that mobile-class devices remain largely limited to low polygon counts with forward shading. Epic’s Unreal documentation likewise describes forward rendering as the faster baseline for VR.
That gap should drive what runs live. Our rule of thumb for content that has to ship on both:
| Workload | Standalone (Quest-class) | Tethered PCVR |
|---|---|---|
| Static lighting and global illumination | Bake | Bake or render live, depending on headroom |
| Complex materials, heavy post-processing | Simplify or bake into textures; these are fragment-bound and thermally expensive | Render live; foveation buys resolution |
| Dynamic lighting, particles, always-on physics | Budget tightly; Meta lists them as thermal-pressure patterns | Render live |
| Geometry density | Keep low; forward shading baseline | Higher counts viable |
| Gaze tracking for foveation | Only where the tracker fits its own energy budget | Most tracked-foveation headsets run tethered on a high-end GPU |
On the tethered side, the host GPU sets the ceiling; our page on GPU workstations for tethered XR rendering covers what desktop-class hardware handles, and the headset-to-host link is covered in direct attach copper cabling in tethered XR. A third option, offloading rendering to a remote machine, changes the power arithmetic: in XRgo the offloaded thin client drew about 15 W, while local rendering at the board’s 60 W maximum drew around 44 W. Offloading only pays when transmission, decoding and reprojection cost less power than rendering locally. The strategic question of which AR workloads run on the headset GPU versus a tethered or cloud renderer has its own treatment. Whichever host you choose, benchmarking GPUs against your own workload tells you more than a spec sheet.
How do ASW and reprojection headroom interact with a dropped-frame budget, and when does reprojection become visible?
Reprojection hides an occasional missed frame. When an app misses a frame on Meta’s runtime, the compositor re-displays the old frame with rotational reprojection (TimeWarp), which works well for head rotation because rotation introduces no parallax, but does not correct head translation. Occasional misses are hard to notice; sustained drops produce judder, flickering black regions at the periphery when the head turns far enough to reveal unrendered areas, and growing latency.
Whether reprojection becomes visible depends on how long drops last and on the type of head motion. Translational reprojection has a harder problem: objects that were hidden become visible (disocclusion), and good hole-filling is usually too expensive to run at target frame rate.
There are two ways to buy headroom on purpose. Application SpaceWarp (AppSW) lets the app render at half the refresh rate, submit motion vectors and depth, and have the compositor synthesise the in-between frames, which doubles the time budget per rendered frame. Meta’s ASW 2.0 combines Asynchronous Spacewarp with Positional Timewarp (PTW), a depth-based 6-DoF reprojection whose GPU overhead Meta describes as usually lost in frame-timing noise. Meta’s guide also warns that the same reprojection type must be used in both the ASW and timewarp steps, because correcting head movement twice produces artefacts.
In the XRgo user study, OpenWarp (6-DoF reprojection) scored an average 2.94 against 2.56 for TimeWarp while using under 2% more power. When latency spiked, OpenWarp showed stretching artefacts while TimeWarp users felt abrupt movements, and participants preferred the stretching on average.
Reprojection is also on the latency critical path. The ILLIXR study found that although reprojection used no more than 10% of total cycles, it was a dominant contributor to motion-to-photon latency. A frame budget that leaves the compositor no room is a latency bug waiting for the first heavy scene.
How do foveation, ASW/reprojection and variable rate shading fit into one frame?
The three techniques act at different stages of one frame. A typical XR frame runs in this order:
- Pose and simulation. The app reads a predicted pose; SteamVR’s Unity plugin predicts two frames ahead.
- Stereo render with foveation. Instanced stereo (desktop) or mobile multiview reduces the cost of drawing both eyes. The foveation map sets per-region shading rates through VRS, with rates from 1×2 up to 4×4, where 4×4 means one pixel-shader invocation for 16 pixels. Fixed foveation applies an image-based VRS pattern that lowers shading rate away from the centre, where lens distortion discards detail anyway. With ETFR, the gaze sample moves that map each frame.
- Submission. The app submits colour, and for positional reprojection or AppSW also depth and motion vectors.
- Compositor. Asynchronous timewarp runs at the display refresh rate, independent of the app’s submission schedule, reprojecting the latest content even when the app misses frames. The reprojection pass also corrects lens distortion and chromatic aberration.
- Scan-out. Low-persistence displays illuminate only briefly at the end of each refresh, from 0.33 ms on the Valve Index to 2 ms on the Oculus Rift.
When the app falls behind, the runtime adds its own levers on top. With dynamic resolution enabled, Meta’s OS lowers render scale after dropped frames; with dynamic foveation enabled, it can raise the FFR level under GPU pressure; and under thermal pressure it can drop the refresh rate back to 72 Hz or halve the effective frame rate. These are recovery paths that trade image quality for continuity, so a title that relies on them in normal play has already spent its margin.
What thermal and power limits cap throughput on mobile XR SoCs?
Sustained heat sets the real ceiling on a standalone headset. The September 2025 Quest 3 feasibility study reports that sustained high utilisation typically leads to thermal throttling within 5 to 10 minutes, and that lightweight native mixed-reality compositing at 720p30 left only about 5% headroom. Mixed reality makes it worse: the study reports 17% lower GPU and 14% lower CPU performance in MR mode than in VR-only mode because of sensor-processing overhead, and cites MR applications using up to 40% more GPU power than VR-only equivalents.
Meta’s thermal documentation describes a staged response. The OS first budgets sustained workload through CPU and GPU levels, then reduces clock speeds as sensors approach thresholds, which raises per-frame cost and makes frame timing more variable, and finally can force a cool-down state with a user notification. Its FrameSync timing algorithm is designed to absorb some of that variability. The documentation states the trade plainly: an app near the thermal ceiling drops frames in long sessions, while one that leaves headroom stays smooth.
A few constraints are checkable before any profiling. Quest 3 and 3S cannot use GPU levels 3 or 4 when passthrough is enabled. GPU levels rise automatically at 91% utilisation and fall at 85% or below, and they are hints the runtime can override for thermal and power reasons, with no guarantee that matching levels mean the same clocks across headset models. Power is also not only a GPU matter: on the ILLIXR study’s low-power Jetson, SoC and system components consumed more than 50% of total power.
Meta lists failing to use foveated rendering or App Spacewarp, continuous full-resolution passthrough alongside a heavy 3D scene, and always-on physics, particles or dynamic lighting among the common causes of thermal pressure. Choosing a lower sustained level leaves room for short bursts.
Which XR rendering pipelines ship today, and where do they break under sustained load?
The building blocks on standalone hardware include a forward renderer, multiview stereo, FFR through VRS (ETFR where eye tracking exists), the compositor’s asynchronous timewarp, and AppSW as an opt-in budget doubler. Meta’s ASW 2.0 combines Asynchronous Spacewarp with positional timewarp.
They break in three recognisable ways. The first is thermal: a renderer tuned on a cold device runs into the throttling window and its frame timing degrades. The second is content variance: in the ILLIXR study, motion-to-photon mean and variability both grew with scene complexity on the low-power Jetson until the Sponza scene was practically unusable at 19.3 ms average before display time. The third is measurement: ILLIXR found per-component metrics such as VIO frame rate insufficient to judge the final user experience. Profiling the renderer in isolation, at the start of a session, on the lightest scene, tests none of these.
Who can audit our XR renderer against the headset budget?
TechnoLynx audits XR renderers against the headset compositor budget through its GPU Performance Engineering work, starting with a GPU Audit that measures whether the current renderer operates inside that budget or relies on best-case conditions that collapse under content variance.
Which renderer design choices will survive the next generation of headsets?
The Quest 3 feasibility study compares the XR2 Gen 2 with the Snapdragon 8 Gen 3 phone chip, citing roughly 1.5 to 2× the CPU and GPU performance based on AnTuTu scores and about 34% headroom on the same MR workload. That is a synthetic benchmark comparison, not a test of a future headset’s sustained thermal behaviour. Meta argues that ETFR’s main effect is cutting pixel throughput and that it could enable high pixels-per-degree rendering on future HMDs.
What the research does support is that the gap grows as features arrive. The ILLIXR authors expect higher-fidelity displays and added components (scene reconstruction, eye tracking, hand tracking) to widen the performance and power gap further. Some of that load is ML: Meta’s research on offloading models to DSPs or NPUs pipelined with the GPU targets the 11 ms rendering budget at 90 Hz. For teams training gaze or super-resolution models, we cover ML infrastructure in W&B pricing for experiment tracking and what Accelerate and DeepSpeed do for XR ML workloads.
So the decision that ages well is architectural, not hardware-specific. Keep foveation, depth and motion-vector submission and a reprojection path in the pipeline from the start, because each future headset seems likely to spend new silicon on sensors and resolution before it hands the renderer spare milliseconds. Plan for the thermal envelope you can measure today.
FAQ
Can Quest 3 do foveated rendering?
Quest 3 supports fixed foveated rendering, but Meta’s documentation states it has no eye-tracking hardware and does not support eye-tracked foveated rendering. With dynamic foveation enabled, the OS can raise the FFR level when the GPU is under pressure.
Is VR CPU or GPU heavy?
Both. In the ILLIXR study, the application and visual-inertial odometry dominated CPU cycles, while eye tracking and scene reconstruction ran on the GPU and reprojection used both. In mixed reality on Quest 3, depth segmentation is reported to push CPU utilisation to 60 to 70%.
