The Metaverse: And How It Will Revolutionize Everything

by Matthew Ball

The success of Discord is another good example. Historically, gaming platforms such as Nintendo, PlayStation, Xbox, and Steam tightly protect their player networks and communication services. This is why someone on Xbox Live cannot “friend” someone on PlayStation Network, nor speak to them directly. Instead, users on other platforms are only available inside cross-platform games, such as Fortnite, and via their game-specific IDs. While this approach worked well enough when two players knew which single game they wanted to play before logging on, it didn’t work well for unplanned hanging out, or enjoying ad hoc play. The more central gaming was to someone’s lifestyle, the less this solution suited them.

Discord emerged to meet this demand, and it has offered gamers numerous benefits. It operates across all major computing platforms—PC, Macs, iPhones, and Androids—meaning every gamer can access a single social graph (and non-gamers can join,

Discord has been able to build a gamer communication network larger—and far more active—than any single immersive gaming platform.

Establishing Common 3D Formats and Exchanges

Witness the current universe of 3D assets. Billions of dollars have been spent on non-standardized virtual objects and environments across film and video games, civil and industrial engineering, healthcare, education, and more. There are no signs that this level of spending will do anything but increase in the near future. Constantly remaking these objects for a new file format or engine is financially impractical and often wasteful; the greatest attribute of a digital “thing” is that it can be endlessly re-used without additional cost.

Interchange solutions are already emerging to tap into the “virtual gold mine” of previously created and fragmented asset libraries.

Nvidia’s Omniverse, which launched in 2020 and enables companies to build and collaborate in shared virtual simulations built upon 3D assets and environments from different file...

Omniverse is built on Universal Scene Description (USD), an interchange framework developed by Pixar in 2012 and open sourced in 2016. USD provides a common language for defining, packaging, assembling, and editing 3D, with Nvidia likening it to HTML, but for the Metaverse.4 In short, Omniverse is driving both an interchange platform and a 3D standard.

If interoperability truly has value, then financial incentives and competitive pressure will eventually solve for it. Developers will eventually figure out how to technically and commercially support Metaverse business models. And they’ll use the Metaverse’s larger economy to surpass “legacy” game makers.

This is one lesson of the rise of free-to-play game monetization. In this business model, players are charged nothing to download and install a game—or even to play it—but are presented with optional in-game purchases such as an extra level or a cosmetic item. When it was first introduced in the 2000s, and even a decade later, many believed free-to-play would, at best, lead to lower revenues for a given game and at worst cannibalize the industry. Instead, it proved to be the best way to monetize a game and a core driver behind video gaming’s cultural ascendance. Yes, it led to many non-paying players, but it substantially increased the total number of players and even gave paying players a reason to spend more. After all, the more people you can show a customized avatar off to, the more you’ll pay to do so.

Not all virtual worlds will move to widely interoperable model, of course. Despite the prevalence of free-to-play multiplayer online games today, many titles are still paid, single-player, offline, or all three.

FOR MANY OF US, THE MOST EXCITING ASPECT OF the Metaverse is the development of new devices that we might use to access, render, and operate it. This usually leads to visions of super-powerful, yet lightweight, augmented reality and immersive virtual reality headsets. These devices are not required for the Metaverse, but are often assumed to be the best or most natural way to experience its many virtual worlds.

Google’s first foray came in 2014 and was named Google Cardboard and had the stated goal of inspiring interest in immersive virtual reality.

In 2016, Google launched its second VR platform, Daydream, which was intended to improve upon Cardboard’s foundation.

According to press reports, Google plans to release a new VR and/or AR headset platform in 2024.

The first Echo Frames released in 2019, with an updated model edition released a year later. Neither seems to have sold well.

One of the most outspoken proponents of AR and VR devices is Mark Zuckerberg. In 2014, Facebook acquired Oculus VR for $2.3 billion, more than twice the sum it paid for Instagram two years earlier, even though Oculus had yet to release its device to the public.

Facebook announced that the Oculus Quest 2 had sold over 10 million units between October 2020 and December 2021—a figure that beat Microsoft’s new Xbox Series S and X console, which released around the same time. However, the device has yet to replace the PC, of course, and Facebook has yet to release an AR device. Still, the bulk of Facebook’s $10 billion–$15 billion in annual Metaverse investments are believed to focus on AR and VR devices.

Apple has been, as usual, secretive about its plans for or even belief in AR or VR—but its acquisitions and patent filings are revealing. Over the past three years, Apple has bought start-ups such as Vrvana, which produced an AR headset called Totem; Akonia, which produced lenses for AR products; Emotient, whose machine learning software tracked facial expressions and discerned emotions; RealFace, a facial recognition company; and Faceshift, which remapped a user’s facial movements to a 3D avatar. Apple also purchased NextVR, a VR content producer, as well as Spaces, which created location-based VR entertainment and VR-based experiences for videoconferencing software. On average, Apple is granted over 2,000 patents per year (and files for even more). Hundreds of those relate to VR, AR, or body tracking.

Snap’s first AR glasses, 2017’s Spectacles, received more acclaim for their pop-up vending machine sales model than their technical, experiential, or sales success, the company has released three new models over the last five years.

companies such as Microsoft, Facebook, Snap, and Niantic see the ongoing struggles with AR and VR as proof that they may be able to displace Apple and Google, which operate the most dominant platforms of the mobile era, while Apple and Google understand that they must invest to avoid disruption. There are early signals validating the belief that AR and VR are the next big device technology,

With AR and VR, one key constraint is the device display. The first consumer Oculus, released in 2016, had a resolution of 1080 × 1200 pixels per eye, while the Oculus Quest 2, released four years later, had a resolution of 1832 × 1920 per eye (roughly equivalent to 4K). Palmer Luckey, one of Oculus’s founders, believes that more than twice the latter resolution is required for VR to overcome pixilation issues and become a mainstream device. The first Oculus peaked at a 90-Hz refresh rate (90 frames per second), while the second offered 72–80 Hz. The most recent edition, 2020’s Oculus Quest 2, defaults to 72 Hz, but supports most titles at 90 Hz and provides “experimental support” for 120 Hz on less computationally intensive games. Many experts believe 120 Hz is the minimum threshold for avoiding the risk of disorientation and nausea.

“the hardest technology challenge of our time may be fitting a supercomputer into the frame of normal-looking glasses.”

Increasing both the number of pixels rendered per frame as well as the number of frames per second requires substantially greater processing power. This processing power also needs to fit inside a device that can be comfortably worn on your head, rather than be stored inside your living room credenza or held in the palm of your hand. And crucially, we need AR and VR processors to do more than just render more pixels.

Facebook’s VR device has a battle royale game, Population: One. But this battle royale doesn’t support 150 concurrent users, as Call of Duty Warzone does, nor 100 like Fortnite, or even Free Fire’s 50. Instead, it’s limited to 18. The Oculus Quest 2 can’t handle much more. Furthermore, the graphics of this game come closer to those of the PlayStation 3, which released in 2006, rather than 2013’s PlayStation 4, let alone 2020’s PlayStation 5.

iPhone—the most successful product of the mobile era.

Today, Apple designs many of the chips and sensors inside of its devices, but its first several models were entirely comprised of components created by independent suppliers. The first iPhone’s CPU came from Samsung, its GPU from Imagination Technologies, various image sensors were from Micron Technologies, the touchscreen’s glass was from Corning, and so on.

There are no “best practices” within the brand-new device category. In fact, many of the choices we consider obvious today were once controversial—not just the iPhone’s touchscreen. For example, some early Android builds and apps used Apple’s “pinch-to-zoom” concept, but believed it was backward

Snap’s AR glasses cost less than $500 and target consumers, while Microsoft’s run for $3,000 or more and focus on enterprises and professionals. Google believed that rather than sell a multi-hundred- or multi-thousand-dollar VR headset, consumers should instead place the expensive smartphone they already own into a “viewer” that costs less than $100.

Amazon’s augmented reality glasses don’t even have a digital display, and instead emphasize its audio-based Alexa assistant and trendy form factor. Facebook, unlike Microsoft, seems to be focusing on VR before AR,

Most commonly, gamers imagine wearing smart gloves and even bodysuits that can provide physical (that is, “haptic”) feedback to simulate what’s happening to their avatar in a virtual world. Many such devices exist today, though they’re so costly and functionally limited that they’re typically used exclusively for industrial purposes.

Gloves and bodysuits can also be used to capture a user’s motion data, rather than just relay feedback, thereby allowing the wearer’s body and gestures to be reproduced in a virtual environment in real time. This information can also be captured using tracking cameras. However, such cameras require unobstructed views, relative proximity to the user, and can struggle if they need to track more than a single user in rich detail.

Facebook’s highest price acquisition since Oculus VR in 2014 was CTRL-labs, a neural interface start-up that produces armbands that record electrical activity from skeletal muscles (a technique called electromyography). Although CTRL-labs’ devices are worn more than six inches away from the wrist and even farther away from the fingers, CTRL-labs’ software enables minute gestures to be reproduced inside virtual worlds

Google’s fifth-largest acquisition ever was the smart wearables company Fitbit, which the company bought for over $2 billion in early 2021. Wearables will shrink in size and increase in performance, and as the technology improves they will be integrated into our clothes.

These developments will help users enhance their interactions with the Metaverse, and enable them to interact with it in more places.

In 2015, Elon Musk founded Neuralink, of which he remains CEO, and announced that the company was working on a “sewing machine–like” device which could implant sensors that are four-to-six micrometers thick (roughly 0.000039 inches, or one-tenth the width of human hair) into the human brain. In April 2021, the company released a video in which a monkey played the game Pong using a wireless Neuralink implant.

In 2021, Google unveiled Project Starline, a physical booth designed to make video conversations feel like you’re in the same room with the other participant. Unlike a traditional monitor or videoconferencing station, Starline’s booths are powered by a dozen depth sensors and cameras (together producing seven video streams from four viewpoints and three depth maps), as well as a fabric-based, multilayered light-field display, and four spatial audio speakers. These features allow participants to be captured and then rendered using volumetric data, rather than flattened 2D video.

Storied camera manufacturer Leica now sells $20,000 photogrammetric cameras that have up to 360,000 “laser scan set points per second,” and that are designed to capture entire malls, buildings, and homes with greater clarity and detail than the average person would ever see if they were physically on-site. Epic Games’ Quixel, meanwhile, uses proprietary cameras to generate environmental “MegaScans” comprising tens of billions of pixel-precise triangles.

Such scanning devices make it easier and cheaper for companies to produce high-quality “mirror worlds” or “digital twins” of physical spaces—and to use scans of the real world to produce higher-quality and less-expensive fantasy worlds.

Consider Amazon’s cashier-less, cashless, and auto-pay grocery stores, Amazon Go. These stores deploy scores of cameras that track every customer by way of facial scanning as well movement tracking and gait analysis. A customer can pick up and put down whatever they like, then simply walk out of the store, having paid for only what they took with them. In the future, this sort of tracking system will be used to reproduce these users, in real time, as digital twins.

Technologies such as Google’s Starline might simultaneously allow workers to be “present” in the store (potentially from an offshore “Metaverse call center”), jumping ...

humans find 3D environments and data presentation far more intuitive

Google’s second-largest acquisition ever (the largest if we exclude Motorola, which Google divested after three years) is that of Nest Labs, which develops and operates smart sensor devices, for $3.2 billion in 2014.

Google spent another $555 million to acquire Dropcam, a smart-camera maker, which was then folded into Nest Labs.

Most experts, including Unity Technologies CEO John Riccitiello, estimate that by 2030, there will be fewer than 250 million active VR and AR headsets in use.

Oculus VR, meanwhile, uses the high-resolution, multi-angle iPhone camera to produce mixed reality experiences. For example, an Oculus user playing Beat Sabre#x2021; can place their iPhone several meters behind them, so they can see themselves inside a VR environment from within their VR headset, and all from a third-person perspective.

In January 2022, Sony announced its PlayStation VR2 platform, which boasted 2000 × 2040 pixels per eye (roughly 10% more than the Oculus Quest 2) and a 90–120-Hz refresh rate (compared to the OQ2’s 72–120), with 110° field of view (versus 90°), plus eye tracking (not available). However, the PSVR2 requires users to own and physically connect to Sony’s PlayStation 5 console, which costs more than the cheapest Oculus Quest 2, and does not come with the PSVR2 headset.

Hardware as the Gateway The many devices required and expected to support the Metaverse can be grouped into three categories. First, the “primary computing devices,” which for most consumers are smartphones, but may be AR or immersive VR at some point in the future. Second, the “secondary” or “supporting computing devices,” such as a PC or PlayStation, and likely AR and VR headsets. These devices may or may not rely on a primary device, or be complemented by them, but they will be used less frequently than a main device and for more specific purposes. Finally, we have the tertiary devices, such as a smartwatch or tracking camera, which enrich or extend a Metaverse experience, but will rarely operate it directly.

The Metaverse is a mostly intangible experience: a persistent network of virtual worlds, data, and supporting systems. However, physical devices are the gateway to accessing and creating these experiences. Without them there is no forest to be known, heard, smelled, touched, or seen.

THE METAVERSE IS ENVISIONED AS A PARALLEL plane for human leisure, labor, and existence more broadly. So it should come as no surprise that the extent to which the Metaverse succeeds will depend, in part, on whether it has a thriving economy.