10 COMMENTS

Google Aims to Make VR Hardware Irrelevant Before ItEven Gets Going

Smartphones have sidelined digital cameras and other special-purpose devices. Now Google thinks mobile phones will shove

virtual-reality headsets like the Oculus Rift into the shadows, too.

By Tom Simonite on November 3, 2015

The bright black sand at Diamond Beach, Iceland, is dotted with smooth gray pebbles and glassy chunksof ice. The crash of foamy waves fills my ears as I turn my head to look up and down the beach and seethe dark strip of sand fade into white fog as if it goes on forever. I don’t feel the cold, because I’m notreally there. Moments earlier, I had been standing in a cable car climbing over Onomichi, Japan.

I’m in a windowless room at Google’s headquarters, holding a cardboard box against my face in a waythat positions a smartphone two inches from my eyes. Google Cardboard, as both the phone holder andthe accompanying app are called, transforms the computer in your pocket into a virtual-reality headset.It’s something of a gimmick. But it’s also a serious attempt to create a new mass communication andentertainment medium. “Virtual reality will have an important role to play in entertainment,communications, work, and learning,” says Clay Bavor, who leads Google’s virtual-reality project.“Cardboard will be the way that we make these immersive experiences available for everyone.”

Mark Zuckerberg, CEO of Facebook, also thinks virtual reality will change the world. He has predictedthat it is “the next major computing and communication platform,” a very serious claim in Silicon Valleybusiness-speak. But Zuckerberg and Google are trying to make it come true in almost opposite ways. In2013, Facebook spent $2 billion to acquire Oculus VR, a company developing a sophisticated virtual-reality headset called the Rift that is held to your face with a thick elasticated strap and attaches via acable to a powerful PC. The Rift goes on sale early next year. The price is unknown, but a version thatuntil recently was available to developers cost $350, and an Oculus-approved PC to power itsforthcoming headset costs at least $950. Facebook has also collaborated with Samsung on a $99headset called the Gear VR, which can use some Samsung smartphones as a screen, and boasted thatit will widen virtual reality’s audience this way. But Samsung is unlikely to make the Gear VR compatiblewith its competitors’ devices, and Facebook’s main project is the high-end Rift experience.

In contrast, Cardboard works with iPhones and almost any phone running Google’s Android software.Google has released the design of the Cardboard phone holder for free, and other companies sellversions for $10, or more if you want one in plastic, metal, or wood. Google estimates that it and othercompanies have sold or given away more than a million Cardboard kits. This weekend the New YorkTimes will send out over a million more to its subscribers, to promote a virtual-reality documentary onchildren displaced by war. Google has also begun sending Cardboard kits to schools with a specialversion of the app that lets a teacher take a class on a 3-D tour of a coral reef or Machu Picchu.

Students in a classroom at the Bronx Latin School in New York City tried Cardboard as part of Google’s Expeditions Pioneer Program forschools.

The Rift offers a far better experience than Cardboard and has spurred tech and media companies toinvest hundreds of millions in virtual-reality startups, content, and copycat headsets from companiessuch as Sony and phone manufacturer HTC. But the technological tide that made the Rift possible alsoworks against it—in Cardboard’s favor. Even as the smartphone industry slashed the costs of thedisplays and sensors needed to build a good virtual-reality headset, smartphones have made people lessinclined to spend money on PCs or on single-purpose gadgets such as cameras or GPS devices—andperhaps virtual-reality technology. “There’s a set of enthusiastic users, me being one of them, that’ll bewilling to charge and plug in and assemble these things, and we’re going to have a great experience,”says Google’s Bavor. And indeed, many people on his team are working on applications for high-end VRtechnologies, he says. But for virtual reality to go anywhere right now—and to answer the question ofwhat on earth it’s good for—it has to spread beyond that small niche, says Bavor.

Mark Zuckerberg says virtual reality is “the next

Mark Zuckerberg says virtual reality is “the nextmajor computing and communication platform.”But Facebook and Google are trying to make thatstatement come true in almost opposite ways.

Google has several initiatives designed to help its lightweight take on virtual reality take off. It is workingwith phone makers to ensure that future smartphones support Cardboard better. It has invented a newkind of camera to make it easier to produce virtual-reality movies, and it is developing YouTube into aplatform to deliver them. If it works, Google’s gimmick might neutralize virtual-reality headsets beforethey even get properly established.

No head strap required

Cardboard originated in a sequence of events that seem divinely chosen to help Google executivesboast of their company’s openness to new ideas. In the spring of 2014, David Coz, a software engineer inGoogle’s Paris office, traveled to company headquarters in Mountain View, California, carrying the firstprototypes of Cardboard, which he’d made with a colleague, Damien Henry, in the time Google allows forside projects. Coz demonstrated it to a few people, who liked it and showed it to others. Within hourssomebody handed Cardboard to Google cofounder and CEO Larry Page after he stepped off stage atthe company’s weekly all-hands meeting. He liked it too. The next day, Bavor’s boss, Sundar Pichai (whois now Google’s CEO), called him in to try it. Bavor lifted the cardboard box to his face and wastransported to Paris, thanks to imagery from Google’s Street View service. When Bavor’s mind returnedto Mountain View, Pichai told him to turn the prototype into a real product, to be launched two monthslater at Google’s annual conference for developers.

That June, thousands of developers, Google employees, and journalists shuffling out of the almost three-hour keynote were each handed a brown cardboard packet the size of a slim paperback. It opened toreveal a set of cardboard goggles; a few Velcro patches, two plastic lenses, a magnet, and a rubber bandwere also included. Simple instructions explained how to fold the device together, download an app, andput your phone inside to jump into virtual space. Cardboard’s design was updated this summer, but itremains essentially the same. The app splits a phone’s display in two, and the goggles’ lenses project aslightly different viewpoint to each of your eyes, so your brain perceives depth. To interact with what yousee on the screen—to select from a virtual menu, say, or pause a movie—you press a button on top ofthe goggles. It swings a piece of metalized fabric against the phone’s touch screen, which registers thetap of a fingertip.

Cardboard is easy to fold and put to use with many kinds of smartphones.

Putting your expensive glass-and-metal pocket computer into a dowdy brown box you just foldedtogether and raising it to your face feels a little silly. But watch someone try Cardboard for the first time,and you will usually see a grin quickly appear beneath the plain brown goggles. “There is magic in thedifference between your expectations and what it delivers,” says Bavor. “You already have this device inyour pocket that, when you add some cardboard, turns into a good VR viewer.”

He is both right and wrong. It is somewhat surprising that your phone can function as virtual-realityheadset. But right now Cardboard is only okay, not good. For a few minutes it’s great, but the discomfortand even nausea that VR can cause—when your body’s sense of how it is moving conflicts with what itis seeing—strike more readily than they do when you use a headset like the Rift. Current smartphonehardware can’t update the virtual world in close enough synchrony with the movements of your head asyou look around, which is what causes motion sickness. The Rift works well in part because it has cutthat lag to less than 20 milliseconds, and even then, viewing some kinds of content can still leave someusers nauseated. Cardboard’s latency is significantly greater.

Google designed Cardboard without a head strap in recognition of that deficiency—and to make it moresocial. The idea is that your arms will get tired before you overexpose yourself or your friends get antsyfor their turn. But one reason the Google VR team is now large enough to occupy its own building on theGoogle campus is that Bavor thinks smartphones offer a good enough experience for some purposesand will soon get much better. Google is talking with manufacturers about how their devices could helpfix Cardboard’s limitations—for example, by upgrading their motion sensors to be more accurate and toupdate their readings more frequently, like those in Samsung’s Gear VR devices. Because its Androidoperating system powers 1.4 billion smartphones around the world, Google has significant influence onthe companies that make them and how they perform.

Qualcomm, which leads the market in smartphone processors, has made better support for virtual realitya selling point of its new flagship chip, the Snapdragon 820, which is due to appear in many Androiddevices in 2016. Manufacturers see support for virtual reality as a priority for high-end handsets, saysJay Wright, who until this week was a vice president of product management at Qualcomm who workedon virtual and augmented reality. He joined the 3-D design software company PTC, which bought aQualcomm group working on augmented reality software. “It’s been proven there is a viable experience,”he says. “Now it’s about improving the performance.”

“You already have this device in your pocket that,when you add some cardboard, turns into a goodVR viewer.”

Cardboard could also benefit from a Google research effort known as Project Tango, which isdeveloping 3-D sensors and software to help phones and tablets accurately track their position in space.Qualcomm and competing chipmakers Infineon and Intel are all on board. Google is unlikely to get muchhelp from its competitor Apple, although its devices currently work with Cardboard as well as high-endAndroid devices do. Nonetheless, with the progress being made, add-on devices like Samsung’s Gear VRaren’t necessary, according to Bavor. “We prefer to tune the software and components of thesmartphone to work well as opposed to adding complexity and things you need to charge,” he says.

Strategy games

When you ask virtual-reality developers to predict what we all will do with the technology and why, youoften hear predictions about how we will physically engage with imaginary worlds and the objects andpeople in them. The Oculus Rift will ship with a games controller from Microsoft’s Xbox, and Facebookwill also sell complicated controllers that can be used to grab and manipulate virtual objects. Google isinstead trying to make virtual reality into a new kind of casual, mostly passive, entertainment likewatching TV or surfing online videos—one perhaps more likely to win a mass audience and thus becomea lucrative platform for ads or premium content.

The Cardboard app offers a handful of such experiences, including a version of the Google Earth 3-Dmap program where you fly over cities or mountain ranges, and a cute animated short by a former Pixardirector. Other companies can create mobile apps to display their own content through Cardboard andwill take you climbing in Yosemite, freak you out with a short horror movie, or put you on stage next toPaul McCartney’s piano. And this summer, YouTube was upgraded to host what are described as “360-degree videos,” which don’t present your eyes with 3-D imagery but give a sense of being somewhereelse by enveloping you with a panoramic video. They can surround you with frenetic dancers in a musicvideo, or take you to snowy Svalbard to watch a solar eclipse. Soon, much more impressive 3-D videoslike the one that transported me to Iceland will appear on YouTube. For this purpose, Google invented acamera called Jump; it’s built from a ring of 16 small cameras positioned so that every point is capturedfrom three different perspectives, allowing software to merge them into a 3-D view.

Jump’s basic design has been released for free, and GoPro will begin selling a version of the cameralater this year. Google will offer an “assembler” service that converts raw Jump footage into 3-Dexperiences ready for viewing or editing at no charge. Bavor predicts that the cameras will becomestandard for film crews large and small; indeed, major media companies such as ABC and the DiscoveryChannel are already producing content for this nascent virtual-reality platform. Mercedes and Volvo havemade Cardboard apps for promotional purposes. “Anyone with a smartphone and a Cardboard will beable to see the most beautiful places on earth and participate in important events and moments inhistory,” Bavor says.

The students in New York viewed Aztec ruins through Cardboard.

Google is taking on a chicken-and-egg problem common to any company trying to establish itself as themiddleman in a new marketplace. Consumers won’t buy into virtual reality unless there’s something toexperience; content companies won’t provide it without a ready audience. “The only way to get around itis to subsidize one side of the market,” says Michael Cusumano, a management professor at MIT whohas studied how software and consumer electronics companies conjure up new businesses. “Give away3-D viewers, or pay software developers to create applications or other content.” Facebook is trying tofill the content gap by hosting 360-degree video as YouTube does, partnering with content companiessuch as Netflix, and building an in-house production effort called Oculus Story Studio. Google iseffectively subsidizing both sides of the market. By making handsets more or less for free out of deviceswe already own, it’s offering its content partners a potentially much larger audience.

Philip Rosedale, founder of the virtual world Second Life and now the CEO of High Fidelity, a companythat has raised $16 million to develop ways for people to socialize in virtual-reality environments, predictsthat both Google’s and Facebook’s projects will initially disappoint people who expect it to becomepopular. “Next year there’s going to be a big push to make VR a success, and it’s not going to succeed,”he says. Rosedale thinks it will take a few years for Oculus to reach beyond a niche gaming audience, or

Credit: Illustration by Sophia Foster-Dimino, photos courtesy of Google

Tagged: Computing, Google, virtual reality

Reprints and Permissions | Send feedback to the editor

for smartphones to be capable of providing a widely acceptable experience. Referring to Bavor’s group,he says: “I have a lot of respect for that team. But I don’t think that doing VR in software on top of amobile device can be compelling today.”

Bavor acknowledges that Google and other companies investing in virtual reality have much to learn, buthe says the only way to do it is to get the technology to a significant audience. And the device for doingthat is already—perhaps all too often—in our hands and in front of our eyes. “The technology’s here,” hesays. “We’ve seen the proof that you can create wonderful VR experiences with today’s smartphonehardware; the sensors and the hardware and the content are only going to get better.”

© 2015 v1.13.05.10MIT Technology Review