Developing VR in Unity

DEVELOPING VR IN UNITYADITYO PRATOMO - LABTEK INDIE

DEVELOPING VR IN UNITY

ABOUT ME

▸ Chief Technology Officer at Labtek Indie

▸ Chief Academic Officer at Froyo Framework

▸ Certified Unity Developer #20167UCD724

▸ Loves to tinker with tech


VR IN LABTEK INDIE

It’s Strange (2015) Oculus Rift & Leap Motion, developed in Unity

https://irunsiregar.itch.io/its-strange https://www.youtube.com/watch?v=oTii7SSfyLM

https://irunsiregar.itch.io/its-strange

https://www.youtube.com/watch?v=oTii7SSfyLM


VR IN LABTEK INDIE

Combat League (2016) Oculus Rift, developed in Unreal Engine

https://www.youtube.com/watch?v=1sHrSOe8JM8

https://www.youtube.com/watch?v=1sHrSOe8JM8


VR IN LABTEK INDIE

99 Names (2015) Web VR, developed in JS with ThreeJS

99names.adityo.net

http://99names.adityo.net


VR IN LABTEK INDIE

Mixed Reality Experiment Google Cardboard devloped in Unity with Vuforia SDK

http://www.adityo.net/?p=1146

http://www.adityo.net/?p=1146


WHAT I’LL TALK

▸ VR Tech Overview

▸ Desktop VR vs Mobile VR

▸ VR in Unity

▸ Interaction in VR

▸ Towards Better VR Experience

▸ Future Prediction

NOW IS THE TIME TO START MAKING SOMETHING IN VR, EVEN IF YOU DON'T PLAN ON LAUNCHING IT. IT IS A REALLY GOOD IDEA TO START EXPERIMENTING AND SEE WHAT YOU CAN LEARN. EVEN IF YOU'RE NOT GOING TO LAUNCH SOMETHING, IT IS GOOD TO HAVE EXPERIENCE IN VIRTUAL REALITY SO THAT IF IT MAKES SENSE FOR YOU TO GET INTO IT, YOU'RE ABLE TO

Palmer Luckey (Co-founder of Oculus)



VR TECH OVERVIEW

▸ VR is a tech that creates new perception of reality in the mind of its users

▸ Achieved through the use of HMD (Head Mounted Display) that tricks its user into thinking he/she is in a different place

▸ Closely related to AR, but not similar nor has hierarchical relation


VR TECH OVERVIEW


VR TECH OVERVIEW

1. Lens

‣ Emulating depth via focal point adjustment

2. Display

‣ Stereo rendering

‣ FOV emulates real life view

3. Tracking Technology

‣ Tracking head movement

‣ Uses gyro and accelerometer in smartphone

4. 3D Audio

‣ Immersive 3D audio


CURRENT VR PLATFORMS

PC

ANDROID

CONSOLE

OCULUS RIFT HTC VIVE

GOOGLE VR (CARDBOARD & DAYDREAM)

SAMSUNG GEAR VR

PS VR

OSVR


DESKTOP VR VS MOBILE VR: FROM THE TECH SIDE

▸ Desktop VR requires more horse power, but has potential for many kind of interaction and contents

▸ Mobile VR experience is hampered by the device’s spec

▸ Desktop VR can do positional tracking

▸ Mobile VR can only do head positional tracking

▸ Batteries vs Cable

▸ Both do VR content rendering in roughly the same way


DESKTOP VR VS MOBILE VR: FROM THE BUSINESS SIDE

▸ Mobile VR has lower entry barrier, mid-high end Android phone is sufficient

▸ Desktop VR requires high end gaming rig

▸ Mobile VR has way larger potential market than Desktop

▸ Desktop VR will interests hardcore gamer

▸ Mobile VR will interests budget-oriented gadget enthusiasts


DESKTOP VR VS MOBILE VR

▸ Mobile VR will help promoting VR to the masses

▸ Desktop VR will help pushing the boundaries, tech wise

▸ Both is ready for mass development, NOW!


WHY DEVELOP VR NOW?

▸ Relatively new tech

▸ Still no best practice sets in stone

▸ Experiments are highly appreciated

▸ Big or small developers, everybody are still finding its feet

▸ YOU can make the difference


VR IN UNITY PC

OCULUS RIFT HTC VIVE OSVR

OCULUS RIFT DK2 / CV1 WIN 7+

UNITY 5.X


VR IN UNITY PC


HTC VIVE WIN 7+

UNITY 5.X


VR IN UNITY PC


OSVR HMD WIN 7+

UNITY 5.X


VR IN UNITY ANDROID


SAMSUNG GEAR VR

ANDROID VR HMD ANDROID 5.1 +

UNITY 5.X GOOGLE CARDBOARD SDK

NEXUS 6 (DAYDREAM)


VR IN UNITY ANDROID


SAMSUNG GEAR VR

ANDROID VR HMD ANDROID 5.1 +

UNITY 5.X VUFORIA SDK


VR IN UNITY ANDROID


SAMSUNG GEAR VR

ANDROID 5.1 + UNITY 5.X

SAMSUNG GALAXY S7 SAMSUNG GEAR VR


VR IN UNITY: GENERAL ARCHITECTURE

▸ Default camera is replaced by vendor-specific VR camera (usually comes as prefab)

▸ VR Camera will render the game scene to be VR-ready

▸ Other game objects are used as usual

GAME SCENE

VR CAMERA

OTHER GAME OBJECTS


VR IN UNITY: TIPS FOR THE FIRST TIMER

▸ Most of the time, you’ll wrestle with setting up specific environment

▸ Oculus: Get the latest SDK and fulfil the requirements, use Rift DK2 at least, don’t use laptop

▸ Android: setup Android SDK properly, Google and Vuforia SDKs are generally easy to set up


VR IN UNITY

▸ After replacing the Main Camera object with the vendor-specific, AR Camera prefab, you’re done.

▸ EASY!

▸ Demo…

WHAT ELSE TO DO?

if getting VR to up and run is easy, then

LET’S TALK ABOUT INTERACTION


INTERACTION IN VR

▸ With headset strapped on, it’s hard to look around

▸ Mouse + keyboard, won’t be the best way to interact


INTERACTION IN VR

LOOK AROUND

GAMEPAD INTERACTION

GAZE INTERACTION

NATURAL INTERACTION (AVAILABLE NOW)

OCULUS TOUCH

VIVE CONTROLLER

GOOGLE DAYDREAM

LEAP MOTION

EXPERIMENTAL INTERACTION (DIY)

VOICE CONTROLLER

CUSTOM ELECTRONIC (ARDUINO / RPI / ETC) ?

easy to develop, minimal interaction

harder to develop, rich interaction


INTERACTION IN VR: LOOK AROUND

▸ The most basic thing you can do inside VR

▸ Replacing traditional FPS camera, controlled by mouse, with your head

▸ Results in a natural, immersive experience

▸ Used in many VR videos

▸ Minimum interaction, but can still produces excellent experience


360 PHOTOS / VIDEOS IN UNITY

▸ Generally speaking, here are the steps:

▸ Obtain 360 photo/video

▸ Make a skybox using sphere/cube

▸ Use the photo/video as the texture for the skybox





▸ Plugin:

▸ Easy Movie Texture (https://www.assetstore.unity3d.com/en/#!/content/10032)

▸ Discussions:

▸ http://bernieroehl.com/360stereoinunity/

▸ http://forum.unity3d.com/threads/projecting-a-360-degree-vr-video-on-a-sphere.376522/

▸ http://talesfromtherift.com/use-a-360-degree-panorama-as-a-skybox/

https://www.assetstore.unity3d.com/en/#!/content/10032

http://bernieroehl.com/360stereoinunity/

http://forum.unity3d.com/threads/projecting-a-360-degree-vr-video-on-a-sphere.376522/

http://talesfromtherift.com/use-a-360-degree-panorama-as-a-skybox/


GAZE INTERACTION

▸ Using eye as an input

▸ Activating an object by looking at it for a certain period

▸ (Optional) use Gaze Reticle to help user find it’s view’s crosshair


GAZE INTERACTION


GAZE INTERACTION

▸ Google Cardboard SDK:

▸ Use GvrGaze script

▸ Drag the script from Assets/GoogleVR/Scripts/UI/GvrGaze onto to the camera

▸ Implement IGvrGazePointer and IGvrGazeResponder to the interactive object

▸ Vuforia SDK uses Google’s Gaze


NATURAL INTERACTION WITH LEAP MOTION

▸ Leap Motion offers the ability to track hand gestures

▸ It can be sticked in front of Oculus/Vive to enable user to see his/her hands

▸ Thus, enables a natural interaction using hands, while inside VR



DEVELOPING VR WITH UNITY


▸ Download and Import the Leap Motion Core Assets Unity Package

▸ Use the LeapOVRPlayerController Prefab from the Leap Motion + OVR directory as the main VR Camera

▸ The hand will be added to the screen and acts as a 3D objects, just like the others (with colliders too!)


EXPERIMENTAL INTERACTION: VOICE CONTROL

▸ Use voice to give input to the VR world

▸ Closely tied to the deployment platform

▸ iOS, Android, Windows, Mac, has its own voice recognition system

▸ Unity 5.4 can natively do voice recognition in Windows (https://developer.microsoft.com/en-us/windows/holographic/voice_input_in_unity)

https://developer.microsoft.com/en-us/windows/holographic/voice_input_in_unity


EXPERIMENTAL INTERACTION: VOICE CONTROL

▸ Unity-only solution:

▸ Ispikit: https://github.com/ispikit/ispikit-unity

▸ Word Detection: https://www.assetstore.unity3d.com/en/#!/content/4518

▸ Unity + other:

▸ Read Serial in Unity

▸ Do voice recognition in other using Python/NodeJS/etc and send serial from there

https://github.com/ispikit/ispikit-unity



EXPERIMENTAL INTERACTION: CUSTOM ELECTRONIC

▸ If you have something different, why not make it yourself

▸ Use Arduino / Raspberry Pi / etc. to read from certain sensor and send its data to Unity

▸ Vital to create haptic feedback and experimenting with different kinds of interaction

▸ Make one of a kind immersive experience





▸ Use Unity to receive Serial data

▸ Android & Microcontroller Bluetooth: https://www.assetstore.unity3d.com/en/#!/content/16467

▸ SimpleSerialPort: https://www.assetstore.unity3d.com/en/#!/content/53594

▸ Attach sensors to Arduino/Raspberry Pi/other

▸ Send data via Serial/Bluetooth

▸ Watch out for latency!




TOWARDS BETTER VR EXPERIENCE: REMOVING SIMULATION SICKNESS

▸ Simulation sickness is VR’s biggest enemy

▸ Cause: brain is tricked to feel movement, while static

▸ When that trick is broken, nausea attacks, users opt out from the experience (and probably won’t come back)



▸ Technical causes:

▸ Latency (maximum should be 20ms)

▸ Many research has gone into how to solve this issue

▸ Many solutions, but none has quite hits the mark



▸ Hardware design solution:

▸ More comfortable lens

▸ More comfortable HMD when strapped on



▸ Content design:

▸ Adding nose and feet

▸ Shows a fixed reference point in a frame



▸ Game design - movement:

▸ In-game movement have more latency potential than standing still

▸ objects are continuously and dynamically rendered in real-time

▸ more processing power

▸ To move around, use teleport

▸ Teleport can be done with gaze interaction, no controller is required



Keep Talking and Nobody Explodes www.keeptalkinggame.com

http://www.keeptalkinggame.com



▸ Game design - movement:

▸ To move around:

▸ use teleport

▸ use pre-determined route

▸ Teleport can be done with gaze interaction, no controller is required

▸ Pre-determined route will ease us in optimizing assets

▸ Don’t add unnecessary vertical movement



Teleport: https://developer.oculus.com/blog/introducing-the-

oculus-sample-framework-for-unity-5/

INMIND VR: https://developer.oculus.com/blog/introducing-the-

oculus-sample-framework-for-unity-5/

https://developer.oculus.com/blog/introducing-the-oculus-sample-framework-for-unity-5/

https://developer.oculus.com/blog/introducing-the-oculus-sample-framework-for-unity-5/



▸ Game design - camera view:

▸ It’s natural to have first person view, but it’s easier to result in simulation sickness

▸ To reduce that, use third person view, it will still have good experience



Chronos: www.chronosvr.com

Colosse: colossevr.tumblr.com

http://www.chronosvr.com

http://colossevr.tumblr.com



▸ Tech tips:

▸ Use low poly objects, with high-res textures to compensate

▸ Toon-shading is okay, photorealistic can probably causes latency



Job Simulator: www.jobsimulatorgame.com

Land’s End: www.landsendgame.com

http://www.jobsimulatorgame.com

http://www.landsendgame.com



▸ Tech tips:

▸ Use as many static object as possible (uses least RAM)

▸ Use LOD, culling and batching to prevent rendering too many object at the same time

▸ Use as many baked lighting as possible

▸ https://unity3d.com/learn/tutorials/topics/virtual-reality/optimisation-vr-unity

https://unity3d.com/learn/tutorials/topics/virtual-reality/optimisation-vr-unity




TOWARDS BETTER VR EXPERIENCE

▸ Experience design:

▸ Design experience from the ground up, don’t just adapt already available 3D game to VR

▸ VR is now best experienced in short but meaningful time (5-20 minutes)

▸ Design focal point, because the user can look into any direction



▸ UI design:

▸ Make sure the user can see the UI at any time

▸ Go for “no-UI” style, embed the required information in the objects, just like in normal life

▸ Enable the player to look around, even when in pause mode




FUTURE PREDICTION

▸ Lower cost of desktop VR

▸ WebVR

▸ VR beyond entertainment

AT THE SAME TIME, I'D SAY DEVELOPERS SHOULD BE REALISTIC. DON'T BELIEVE THE MOST OPTIMISTIC THINGS THE ANALYSTS SAY. ESPECIALLY WHEN THOSE ANALYSTS DON'T ACTUALLY KNOW ANYTHING ABOUT VR.

IT'S GOING TO TAKE TIME FOR VIRTUAL REALITY TO BECOME TRULY MASS MARKET AND SUCCESSFUL, SO PEOPLE SHOULD SCALE THEIR EFFORTS APPROPRIATELY.

Palmer Luckey (Co-founder of Oculus)


THANK [email protected] @kotakmakan

mailto:[email protected]?subject=