Immersive Connected Experiences - Blogs@Intel ?· Immersive Connected Experiences How connected visual…

  • Published on

  • View

  • Download

Embed Size (px)


  • Immersive Connected Experiences How connected visual computing will change the

    virtual world as well as the actual world

    Jim Held, Sean Koehl, John Hengeveld, Mic Bowman and Inga Vailionis

    Intel Corporation

    Revision: 11/20/2008

    Executive Summary

    Immersive Connected Experiences (ICE) are a new category of connected visual computing

    usage models in which people will share experiences and information online within highly

    intuitive, interactive visual interfaces. Initial examples of ICE include two main categories:

    Simulated Environments such as virtual worlds, online multiplayer games and 3-D movies,

    and Augmented Reality where images from the real world are combined with digital

    information to provide an enhanced view of the globe around us. This paper describes the

    key technical challenges in bringing these usages to the mainstream, including new

    client/server platform innovations, more robust distributed computing techniques, tools to

    facilitate user-generated 3-D content, and techniques to improve experiences on mobile

    devices. It also describes Intels research agenda aimed at removing key technical barriers to

    the widespread adoption of ICE.

    White Paper Research @ Intel

  • White Paper: Immersive Connected Experiences



    A frequent driver of innovation is convergence. The convergence of radio and computing, for instance, brought us

    wireless computing and opened up a world of connected mobility as wireless Internet access grew. Today the

    convergence of several Internet trends involving connected visual computing technologies is enabling a new set of

    usage models.

    Visual computing combines real world and digital world experiences in fascinating ways. The potential of connected

    visual computing as a transformative technology is just beginning to be explored, but when combined with

    broadband internet connectivity, it holds great promise for connecting people within highly intuitive, interactive

    visual interfaces that will enhance the ability to share experiences and information. Intel calls such connected visual

    computing usages Immersive Connected Experiences (ICE). Intel is taking a leading role in driving adoption of ICE and removing the technical barriers to its widespread usage.

    This paper will describe ICE and its potential, plus discuss the technology and ecosystem required to enable easy

    creation of rich, immersive and scalable experiences.

    The Convergence of the Visual

    and the Connected More and more people today use computers and many mobile devices

    (handhelds, cellular phones, etc.) that have integrated cameras that allow

    them to capture the world around them. At the same time, broadband

    connectivity is making it easier to share visual experiences and giving rise

    to social networks where people congregate for exchanging personal

    content and other aspects of their lives. With the rapid growth of

    Facebook, YouTube, Picasso, and similar personal content-oriented sites,

    the Internet is becoming as much or more about user-generated content

    as it is about communications. Whats more, the growth and popularity of

    virtual worlds, such as Habbo and Second Life, is beginning to

    demonstrate how similar these worlds can be to the real world and how

    these sites (and the people participating in them) take on lives of their


    Looking at these Internet trends (social networking, user-generated

    content, broadband connectivity, mobile computing, and visual, immersive

    worlds), Intel sees an ever quickening convergence of visual and

    connected computing that is blurring the traditional divisions between

    real world activities and the digital world. This convergence is the basis

    for ICE, an emerging usage model that offers great opportunities for

    expanding ways we interact and experience the world and each other.

    To better understand ICE and its potential, lets look deeper at its


    Connected Computing

    Connected computing, as a usage model, is easy to comprehend. Weve all

    grown accustomed to, even dependent on, being able to connect to each

    other and exchange data, voice and video through the Internet. But

    connectivity as a computing trend has really just begun. The increase and

    diversity of mobile computing devices, faster broadband speeds, plus the

    growth in input devices such as cameras and various types of sensors,

    are not just making it easier to stay connected all the time always on,

    always connected but also spawning new uses and making it more

    important to have a continuous connection.

    Visual Computing

    Most of us think of computing as visual. After all, we look at a screen. But

    what we mean here by visual computing is immersive, real-time, life-like

    computing experiences. Were at the dawning of an age of such visual

    computing experiences. Examples of visual computing can already be

    seen in the 3D visual models used in financial services, interactive medical

    imaging, and the virtual worlds of Internet gaming.

    Visual computing applications are evolving to deliver ever more

    immersive visual experiences that look real, act real, and feel real. Many of

    the elements necessary to bring such holistic visual experiences into

    mainstream computing are in development and, with improvement, will

    make visual computing an integral part of daily life.

    Some of these elements include:

    Greater photorealism

    Improved modeling (better physics and intelligence)

    Better multimedia (improved video/audio editing and


  • White Paper: Immersive Connected Experiences


    More visual and tactile interactivity (advanced user

    interfaces including new touchscreen and haptic

    technologies [1] and other devices)

    Immersive Connected Experiences

    Technologies that combine connected and visual computing enable

    people to share experiences and information within a highly intuitive,

    interactive visual interface. Through connected visual computing

    technologies, well be able to use computing devices to enhance and

    extend our experiences of the real world, create new virtual worlds, and

    bridge the digital and real world with increasingly rich, Immersive

    Connected Experiences.

    ICE usages will include enhanced social networking, workplace

    collaboration, retail sales, virtual tourism, marketing, distance learning,

    training, gaming, and the arts, to name a few.

    Current examples of ICE fall into two main categories.

    Simulating Environments These are multi-user graphical

    environments where large numbers of people interact for the

    purposes of gaming, socialization, learning, or content creation.

    Examples include virtual worlds, massively multiplayer online

    games (MMORGs), and 3-D cinema.

    Augmented Reality There are environments where

    information from the actual world is combined with digital

    information to provide an enhanced view of the real world.

    Examples include Earth mapping applications such as Google*


    The rapid growth of virtual worlds is particularly indicative of the

    potential for ICE. In June 2008, the research firm Strategy Analytics

    predicted that the percentage of registered users that will go on to

    become active users in virtual worlds will increase from 10 percent now

    to 27 percent by 2017, creating a market base of almost 1 billion users

    [2]. This is up from todays 137 million active users. Pre-teens make up

    more than half the users, spending 16 hours per week (more than on TV

    or magazines) and purchasing items in the virtual worlds [3]. Many

    convert their parents into virtual world users. Even more interesting will

    be what happens if many of these pre-teens carry their virtual world

    habits into adulthood. Strategy Analytics puts the 2008 total market for

    user services at $310 million. The research group predicts that it will

    grow to $8 billion in 2017 [4].

    Augmented reality is also growing and evolving. Many people are familiar

    with how online maps have evolved to include the overlay of satellite

    images and recently street-level surround images. Combining this mixture

    of real and virtual will be particularly compelling with the connectivity and

    sensors of a mobile Internet device (MID). Images, video, audio, GPS,

    acceleration sensors could all provide rich input that could be used to

    create and then present to the user an augmented form of reality.

    Consider being able to take a photo of a monument and then doing a

    visual search using the image for real-time recognition of the monument

    and information about its significance. With the processing power,

    database and communication capability of a continual Internet

    connection, there are many exciting opportunities that can be enabled.

    Meeting the Challenges of ICE Removing key technical barriers to the adoption of Immersive Connected

    Experiences will require research into hardware and software innovations

    that improve end user experience, as well as the development of

    standards that improve interoperability. We want cinematic quality in the

    graphics, said Intel CTO Justin Rattner in his Fall 2007 IDF R&D keynote.

    We want to make user content creation much easier. Of course we want

    to retain the persistence, but we want that persistence to apply across

    virtual worlds, not to be contained in a single virtual world. And higher

    levels of social interaction, better human-computer interfaces to enable

    that, and we'd like the behavior -- the way things act in the virtual world -

    - to be much more natural.

    The desired qualities for future ICE environments include the following:

    Immersive visuals. Because theyre based on connected visual

    computing technologies, the user experience should be greatly

    enriched by immersive, highly realistic visuals.

    User-generated 3D content. Many ICE applications will be

    forms of social media. Thus, people will need and want to

    create or modify visual content within ICE environments. This

    will require new tools such as 3D character designers that are

    as easy to use as the blogging, photo, and video editing tools

    of today.

    Interoperability. Like the Internet, ICE applications based on

    broadly accepted standards will enable data and personas to

    be shared between various ICE environments in creative ways.

    ICE Example 1: Simulated Environments

    Imagine if instead of having to waste a day traveling to a

    conference and another day traveling back, you could attend a

    virtual conference in a simulated environment. Take that a step

    further and imagine, between presentations, being able to have

    conversations with other attendees avatars where facial

    expressions and body language actually help you and others make

    your points with the correct emphasis.

    Example: Qwaq* Forums

  • White Paper: Immersive Connected Experiences


    In fact, according to Strategic Analysts vice president and

    research director Barry Gilbert, "What's holding back the

    numbers [participants] is the reliance on any single virtual

    world because they're not yet portable."

    Robust distributing computing. Because ICE applications are

    distributed, good performance requires high quality, well-

    managed interactions among a wide variety of clients and

    servers. ICE applications should adapt to a diverse array of

    clients so that the user experience scales smoothly from

    performance-optimized home or office systems to compact

    mobile devices optimized for energy-efficiency.

    Scalability. ICE Environments should be able to scale over time

    to incorporate new technologies which improve their quality as

    well as to add more users as they grow in popularity.

    Intel is driving ICE development through its continuing research into

    many of the technologies and advancements that will be required to

    bring ICE into the mainstream. This includes research in redefining

    mobility, workload analysis, simulation, model-based computing, and tera-

    scale computing.

    Intel divides the challenges into four areas: platform optimization, visual

    content, distributed computation, and mobility.

    Platform Optimization

    ICE applications are very compute-intensive. They demand a lot of

    servers, clients and the network. Consider the complexity of the objects

    and the need for accurate simulations, particularly realistic object

    behavior. Then multiply that by the number of users and, the scene

    complexity as more users participate and interact, and its easy to see

    how ICE can be a tremendous challenge to system and network scaling.

    Whats needed are new architectures that scale well and deal with a

    range of client performance and capabilities, dynamically adapting to

    changing demands and virtual world circumstances. Intel believes that ICE

    applications will follow the same evolution the rest of social networking

    and Internet applications have, evolving from proprietary solutions to

    standard protocols and building blocks that can be used to build more

    sophisticated services. Such modular design will also allow for scaling

    through distribution and replication of components, as well as

    repartitioning functionality among types of servers and the diversity of


    Intel has a number of internal teams, projects and software-related

    efforts underway to speed the transition to connected visual computing

    technologies and future usage models like ICE. One extremely relevant

    example here is Intels Tera-scale Computing Research Program. This is a

    worldwide effort to advance computing technology by scaling multi-core

    architectures to 10s to 100s of cores and embracing a shift to parallel

    programming. Through this effort, Intel aims to enable applications and

    capabilities like ICE (see Figure 1). Other research includes redefining

    mobility, workload analysis, simulation and model-based computing

    most of which will benefit from tera-scale computing.

    Figure 1: Tera-scale microprocessors based on many cores will be

    essential to computing the elements of ICE applications in parallel.

    Visual Content

    One of the distinguishing features of ICE applications, such as Second

    Life, is the existence of player-generated content. Enabling easier end-

    user creation and use of rich visual content is essential to ICE. In ICE

    worlds based on the premise of user-customized 3D interactive content,

    such data-rich content wont be conducive to distribution at startup or

    even on a DVD ahead of time. It must be distributed real-time as users

    explore and enter newly created regions. This will place severe demands

    on the network requiring innovations in how content can be concisely

    represented as well as cached and pre-fetched. This will also require a

    new generation of technologies and tools designed to enable


    To simplify content creation, Intel is researching a variety of areas. One is

    parameterized content creation. This is the development of sets of

    parameters for configuring and controlling an object, such as an avatar.

    End-user creation is simplified to selecting and adjusting parameters,

    such as the...


View more >