Editor’s note: This article, originally published on May 20, 2022, has been updated.
Advances in extended reality have already changed the way we work, live and play, and it’s just getting started.
Extended reality, or XR, is an umbrella category that covers a spectrum of newer, immersive technologies, including virtual reality, augmented reality and mixed reality.
From gaming to virtual production to product design, XR has enabled people to create, collaborate and explore in computer-generated environments like never before.
What Is Extended Reality?
Virtual, augmented and mixed reality are all elements of XR technology.
Virtual reality puts users inside a virtual environment. VR users typically wear a headset that transports them into a virtual world — one moment they’re standing in a physical room, and the next they’re immersed in a simulated environment.
The latest VR technologies push these boundaries, making these environments look and behave more like the real world. They’re also adding support for additional senses, including touch, sound and smell.
With VR, gamers can become fully immersed in a video game, designers and customers can review building projects to finalize details prior to construction, and retailers can test virtual displays before committing to a physical one.
Augmented reality is when a rendered image is overlaid onto the real world. The mobile game Pokémon GO famously brought AR to the mainstream by showing computer-rendered monsters standing on lawns and sidewalks as players roam their neighborhoods.
AR graphics are visible through cell phones, tablets and other devices, bringing a new kind of interactive experience to users. Navigating directions, for example, can be improved with AR. Rather than following a 2D map, a windshield can superimpose directions over one’s view of the road, with simulated arrows directing the driver exactly where to turn.
Mixed reality is a seamless integration of the real world and rendered graphics, which creates an environment in which users can directly interact with the digital and physical worlds together.
With MR, real and virtual objects blend, and are presented together within a single display. Users can experience MR environments through a headset, phone or tablet, and can interact with digital objects by moving them around or placing them in the physical world.
There are two types of MR:
Mixing virtual objects into the real world — for instance, where a user sees the real world through cameras in a VR headset with virtual objects seamlessly mixed into the view.
Mixing real-world objects into virtual worlds — for example, a camera view of a VR participant mixed into the virtual world, like watching a VR gamer playing in a virtual world.
The History of XR
To understand how far XR has come, consider its origins in VR.
VR began in the federal sector, where it was used to train people in flight simulators. The energy and automotive design industries were also early adopters. These simulation and visualization VR use cases required large supercomputers. It also needed dedicated spaces, including powerwalls, which are ultra-high-resolution displays, and VR CAVEs, which are empty rooms that have the VR environment projected on each surface, from the walls to the ceiling.
For decades, VR remained unaffordable for most users, and the small VR ecosystem was mainly composed of large institutions and academic researchers.
But early in the previous decade, several key component technologies reached a tipping point, which precipitated the launch of the HTC Vive and Oculus Rift head-mounted displays (HMDs), along with the SteamVR runtime.
Individuals could now purchase personal HMDs to experience great immersive content. And they could drive those HMDs and experiences from an individual PC or workstation with a powerful GPU.
Suddenly, VR was accessible to millions of individuals, and a large ecosystem quickly sprung up, filled with innovation and enthusiasm.
In recent years, a new wave of VR innovation started with the launch of all-in-one (AIO) headsets. Previously, fully immersive VR experiences required a physical connection to a powerful PC. The HMD couldn’t operate as a self-contained device, as it had no operating system and no ability to compute the image.
But with AIO headsets, users gained access to a dedicated device with a simple setup that could deliver fully tracked VR anywhere, anytime. Coupled with the innovation of VR streaming technology, users could now experience powerful VR environments, even while on the go.
Latest Trends in XR
High-quality XR is becoming increasingly accessible. Consumers worldwide are purchasing AIOs to experience XR, from immersive gaming to remote learning to virtual training. Large enterprises are adding XR into their workflows and design processes. XR drastically improves design implementation with the inclusion of a digital twin.
And one of today’s biggest trends is streaming XR experiences through 5G from the cloud. This removes the need to be tethered to workstations or limit experiences to a single space.
By streaming over 5G from the cloud, people can use XR devices and get the computational power to run XR experiences from a data center, regardless of location and time. Advanced solutions like NVIDIA CloudXR are making immersive streaming more accessible, so more XR users can experience high-fidelity environments from anywhere.
AR is also becoming more common. After Pokémon GO became a household name, AR emerged in a number of additional consumer-focused areas. Many social media platforms added filters that users could overlay on their faces. Organizations in retail incorporated AR to showcase photorealistic rendered 3D products, enabling customers to place these products in a room and visualize it in any space.
Plus, enterprises in various industries like architecture, manufacturing, healthcare and more are using the technology to vastly improve workflows and create unique, interactive experiences. For example, architects and design teams are integrating AR for construction project monitoring, so they can see onsite progress and compare it to digital designs.
And though it’s still fairly new, MR is developing in the XR space. Trends are shown through the emergence of many new headsets built for MR, including the Varjo XR-3. With MR headsets, professionals in engineering, design, simulation and research can develop and interact with their 3D models in real life.
Varjo XR-3 headset. Image courtesy of Varjo.
The Future of XR
As XR technology advances, another technology is propelling users into a new era: artificial intelligence.
AI will play a major role in the XR space, from virtual assistants helping designers in VR to intelligent AR overlays that can walk individuals through do-it-yourself projects.
For example, imagine wearing a headset and telling the content what to do through natural speech and gestures. With hands-free and speech-driven virtual agents at the ready, even non-experts will be able to create amazing designs, complete exceedingly complex projects and harness the capabilities of powerful applications.
Platforms like NVIDIA Omniverse have already changed how users create 3D simulations and virtual worlds. Omniverse allows users from across the globe to develop and operate digital twin simulations. The platform provides users with the flexibility to portal into the physically accurate, fully ray-traced virtual world through 2D monitors, or their preferred XR experience, so they can experience vast virtual worlds immersively.
Entering the next evolution of XR, the possibilities are virtually limitless.
What Is Spatial Computing?
Unlike traditional digital experiences, which are confined to screens, spatial computing places virtual elements directly into the physical world, creating more natural and intuitive interactions. The technology combines sensors, cameras and AI-driven software to recognize and respond to real-world elements, so users can interact with digital objects as if they were tangible.
Supported by platforms like NVIDIA Omniverse, spatial computing has broad applications, from industrial design and training to navigation and entertainment. For example, designers could use it to visualize 3D prototypes in real space, or field teams could harness the technology to receive AR guidance overlaid onto real equipment.
As spatial computing evolves, it’s poised to reshape interactions with digital information, making it part of everyday physical environments and expanding the possibilities for XR in practical and creative ways.
Learn more about how organizations can use NVIDIA XR technologies.
