The plain answer to “what is mixed reality” depends on who you ask, because the terms are used inconsistently. In the original 1994 sense from Paul Milgram and Fumio Kishino, any blend of real and virtual content counts, which is why Microsoft says the AR filters people use on Instagram are already mixed reality experiences. In a stricter sense, an experience becomes mixed reality when digital objects interact spatially with physical ones: a virtual sphere rolling behind a real table, or an overlay that disappears when your hand passes in front of it.
Three mechanisms you can check for yourself serve that stricter version: the device maps the room, hides virtual objects behind real ones (occlusion), and shows the real world with little enough delay that the virtual content looks anchored rather than floating. How mixed reality fits into the whole AR vs VR vs XR decision is covered in our overview of how VR, AR and XR fit together; this page stays with the definition and with examples you have probably already used.
What is mixed reality, in plain words?
Mixed reality is the middle of a spectrum. Milgram and Kishino anchored one end with a purely real environment and the other with a purely virtual one, and called any environment that blends real and virtual objects mixed reality. Inside that middle ground they named two regions: augmented reality (AR), where the real world is augmented with virtual content, and augmented virtuality (AV), where most of the content is virtual but some real-world objects are included. Microsoft uses the same picture and calls the whole spectrum the “mixed reality spectrum”, with physical reality at one pole and digital reality at the other.
Researchers have not stood still on this. In a 2021 revisit published in Frontiers in Virtual Reality, Skarbez, Smith and Whitton argued that the continuum is actually discontinuous, because perfect virtual reality cannot be reached, and that mixed reality is broader than previously believed. Their reasoning is that even a perfect headset cannot control your interoceptive senses, such as balance and proprioception, so a person in “full” VR is still perceiving something real. On their definition, conventional VR is itself a subset of mixed reality.
A common way to lay out the family, from least to most virtual, is AR, then MR, then VR.
Where XR fits around mixed reality
Extended reality (XR) is the umbrella over all of it. IEEE describes XR as covering the technologies that combine digital content with the physical environment or replace it entirely, including VR, AR and MR. The 3GPP technical report on XR uses the same scope and adds “the areas interpolated among them”. Rauschnabel and colleagues argued that XR should not stand for “extended reality” at all, but for “xReality”, with the X as an open variable.
Is mixed reality just augmented reality with a new name?
No, though the overlap is large and the vocabulary is genuinely inconsistent. The term “mixed reality” dates from the 1994 Milgram and Kishino paper, so it is not a recent invention. The Rauschnabel framework notes that academics and professionals have used AR, VR, MR and XR inconsistently, which has led to conceptual confusion.
The useful distinction is about what the device knows about the room. The 3GPP report defines MR as an advanced form of AR in which virtual elements are inserted into the physical scene with the intent of making them look like part of it. The Frontiers paper records a common shorthand: MR systems possess knowledge about the physical world, while AR systems do not. Rauschnabel’s framework treats it as a continuum within AR, running from assisted reality to mixed reality according to the level of local presence.
Our reading is that MR is a real category with a fuzzy edge: the label earns its keep when the experience depends on a model of the room, and it is marketing when a flat overlay gets called MR. For the operational differences between AR and VR themselves, our guide to distinguishing AR and VR goes further, and what augmented reality is and where it is applied covers the AR side on its own.
How does mixed reality work in everyday smartphone apps?
IEEE describes smartphone AR as camera pass-through: the phone shows its camera feed with graphics overlaid. To keep those graphics in the right place, the app has to estimate the pose of the physical environment relative to the display, which is solved by simultaneous localisation and mapping (SLAM) running on the device. Visual-inertial odometry is another common route to markerless tracking, with fiducial markers used where a known pattern is available.
IKEA’s Place app, announced in 2017, let people place 3D, true-to-scale models of furniture in their own living space. Google added an AR feature to Google Maps on Pixel phones that identifies the user’s location and places signs and arrows on screen for navigation. Camera-based AR devices that do 3D tracking depend on analysing their surroundings in real time.
A reviewer of the Meta Quest 3 contrasted its anchored virtual objects with the jittering, drifting or floating sometimes seen in phone-based AR. Occlusion is the other gap: depth cues such as occlusion and shadows are what keep virtual graphics looking real.
Is Pokémon Go mixed reality or augmented reality?
Pokémon Go is a borderline case, and the experts disagree. Wikipedia’s article on mixed reality games lists Pokémon GO and Harry Potter: Wizards Unite as examples of mixed reality games. When Undark asked researchers in July 2016, one said the game shows digital graphics on top of real physical space, “so at the very least it is mixed reality”. Blair MacIntyre, in the same piece, called it AR, “just a really weak form of it”.
As Undark reports, Ronald Azuma wrote in 1997 that a true AR application must combine the real and the virtual, be interactive in real time, and be in 3D. At launch, Pokémon Go characters appeared as 2D images overlaid on the camera view without really interacting with their real-world surroundings; having a Pokémon hide behind an actual tree would require the computer to understand the real objects.
Niantic later moved the game towards that line. It announced a “reality blending” beta, due to begin in June, that would let Pokémon hide behind real objects or be occluded by a tree or table. Niantic also said the work would give Pokémon spatial and contextual awareness of their surroundings. Our verdict: the original game is mixed reality only in the loose sense, and the occlusion feature is the part that moves it towards true MR, on the devices that support it.
How a headset blends virtual objects with your real room
A mixed reality headset such as Meta Quest 3 shows the room through cameras, builds a 3D model of the room, and uses depth to decide what should hide what.
Passthrough. Meta’s documentation describes passthrough as a real-time, perceptually comfortable 3D visualisation of the physical world, which developers combine with their virtual content through the Passthrough API. Apps never see the camera images: they submit a placeholder layer, and the XR compositor swaps in the passthrough rendition and blends it above or below the app’s own layers. Surface-projected passthrough lets an app supply the geometry the images are projected onto, which gives a more stable result where the environment is known. Video passthrough gives better control over occlusion and blending than optical see-through, but can add latency, distortion and resolution loss.
Spatial mapping. Microsoft’s HoloLens documentation describes spatial mapping as a triangle-mesh representation of real-world surfaces that lets apps place holograms on real surfaces and occlude them behind real objects. On Quest 3, the headset scans the room and generates a mesh of floor, walls, ceiling and furniture in around 10 to 20 seconds, according to UploadVR’s review, after which virtual objects can collide with, emerge from or appear behind them. That review contrasts it with the manual wall-marking setup on Quest 2 and Pro, which only supported rectangular cuboids.
Depth-based occlusion. Meta’s Depth API provides real-time, per-eye, per-frame depth estimates, which developers use to implement dynamic occlusion between virtual and physical content. Meta describes Quest 3’s depth engine as AI-powered.
Latency. Low passthrough latency prevents nausea and matters for safety. When Optofidelity measured photon-to-photon passthrough latency, Apple Vision Pro came in at about 11 milliseconds, while the other headsets tested, including Meta Quest 3 and Quest Pro, measured 35 to 40 milliseconds. That is a different measure from motion-to-photon latency, which IEEE says should stay below about 20 milliseconds to avoid discomfort, a requirement explained further in our guide to distinguishing AR and VR. The numbers and the lived experience do not line up neatly: UploadVR’s reviewer called Quest 3’s passthrough extremely low latency and spent over an hour in it comfortably. Our take is that comfort depends on depth correctness and tracking as well as raw delay, so one latency figure does not settle whether a headset feels right.
All of this costs compute. Microsoft notes that storing, rendering and processing dense spatial meshes consumes significant compute and storage resources, and the memory side of that budget is covered in our piece on unified virtual memory and XR rendering budgets.
Where headset mixed reality still falls short
Occlusion is one weak point. At Quest 3’s launch, virtual objects could hide behind the scanned room mesh but not behind moving things: hands, arms and other people always appeared behind virtual objects, which the reviewer called jarring, with Meta promising dynamic occlusion in a later software update that developers would then need to adopt. Spatial mapping has its own failure modes: a small dark object that fails to scan leaves a hole that holograms show through, and reflections or bright lights can leave “hallucinations” that wrongly occlude content. On HoloLens, the mapping camera sees only 3.1 metres ahead, and mapped data is kept available only in a limited “bubble” around the user.
What are everyday examples of mixed reality in 2026?
The examples below are scored against the strict test.
| Example | Real and virtual blended | Room model | Occlusion | Our verdict |
|---|---|---|---|---|
| Instagram AR filters | Yes; Microsoft counts them as mixed reality | Unclear | Unclear | Mixed reality in the broad sense only |
| Pokémon Go (2016 launch) | Yes, 2D characters over the camera view | No interaction with surroundings reported | No interaction with surroundings reported | Weak AR or loose MR |
| Pokémon Go with reality blending (2020 beta) | Yes | Spatial awareness planned | Yes, behind trees or tables | Close to true MR on supported devices |
| IKEA Place | Yes | Yes, true-to-scale in the room | Unclear | AR with spatial placement |
| Google Maps AR navigation | Yes, signs and arrows on screen | Tied to location | Unclear | AR |
| HoloLens RoboRaid | Yes | Yes, enemies on the walls adapt to each room | Yes, by real objects | True MR |
| Meta Quest 3 MR apps | Yes, colour passthrough | Yes, scanned room mesh | Behind the mesh; dynamic occlusion missing at launch | True MR, with gaps that depend on the app |
The pattern is the one to carry: the more of the right-hand columns an experience fills in, the more it behaves like mixed reality and the less like a sticker on a video feed. For the virtual end of the spectrum, our list of virtual reality use cases and examples covers what this table leaves out.
Which mixed-reality applications are people actually using daily?
The ones on phones, by Microsoft’s account. Microsoft says mobile AR offers the most mainstream mixed reality solutions today, through social media filters that people may not even realise are mixed reality. Pokémon Go became one of the most popular smartphone applications after its July 2016 release and drove up interest in AR games. The Frontiers paper makes the same point about reach: millions of consumers can access AR experiences such as Pokémon GO on their phones.
UploadVR’s Quest 3 reviewer recommended the headset for VR but described its mixed reality as first-generation, with the technology and content still in their earliest days.
Will mixed reality replace smartphones for everyday use?
Not on this evidence, and the gaps are specific. Dynamic occlusion was missing at Quest 3’s launch, and developers would need to integrate it once Meta shipped it. Measured passthrough latency spans roughly 11 milliseconds on Vision Pro to 35 to 40 milliseconds on the other headsets tested. HoloLens spatial mapping has a limited range and leaves holes and phantom surfaces. A phone, meanwhile, already runs the broad form of mixed reality that people use without thinking about it.
MacIntyre told Undark in 2016 that headset technology might in a few years be used for directions, for repairing machinery by overlaying instructions on the equipment, and even for helping with remote surgery. Those are task-shaped uses rather than a replacement for everything a phone does.
The question we would watch is narrower than “headset or phone”: when does strict mixed reality (room model, dynamic occlusion, depth-correct low-latency passthrough) become standard across affordable devices instead of a feature of a few? The 3GPP report expects that, in truly immersive MR, users cannot tell virtual objects from real ones. The TechnoLynx Generative & Agentic AI R&D practice builds generative and agentic AI into products.
