Immersive Connected Experiences How connected visual computing will change the
virtual world as well as the actual world
Jim Held, Sean Koehl, John Hengeveld, Mic Bowman and Inga Vailionis
Immersive Connected Experiences (ICE) are a new category of connected visual computing
usage models in which people will share experiences and information online within highly
intuitive, interactive visual interfaces. Initial examples of ICE include two main categories:
Simulated Environments such as virtual worlds, online multiplayer games and 3-D movies,
and Augmented Reality where images from the real world are combined with digital
information to provide an enhanced view of the globe around us. This paper describes the
key technical challenges in bringing these usages to the mainstream, including new
client/server platform innovations, more robust distributed computing techniques, tools to
facilitate user-generated 3-D content, and techniques to improve experiences on mobile
devices. It also describes Intels research agenda aimed at removing key technical barriers to
the widespread adoption of ICE.
White Paper Research @ Intel
White Paper: Immersive Connected Experiences
A frequent driver of innovation is convergence. The convergence of radio and computing, for instance, brought us
wireless computing and opened up a world of connected mobility as wireless Internet access grew. Today the
convergence of several Internet trends involving connected visual computing technologies is enabling a new set of
Visual computing combines real world and digital world experiences in fascinating ways. The potential of connected
visual computing as a transformative technology is just beginning to be explored, but when combined with
broadband internet connectivity, it holds great promise for connecting people within highly intuitive, interactive
visual interfaces that will enhance the ability to share experiences and information. Intel calls such connected visual
computing usages Immersive Connected Experiences (ICE). Intel is taking a leading role in driving adoption of ICE and removing the technical barriers to its widespread usage.
This paper will describe ICE and its potential, plus discuss the technology and ecosystem required to enable easy
creation of rich, immersive and scalable experiences.
The Convergence of the Visual
and the Connected More and more people today use computers and many mobile devices
(handhelds, cellular phones, etc.) that have integrated cameras that allow
them to capture the world around them. At the same time, broadband
connectivity is making it easier to share visual experiences and giving rise
to social networks where people congregate for exchanging personal
content and other aspects of their lives. With the rapid growth of
Facebook, YouTube, Picasso, and similar personal content-oriented sites,
the Internet is becoming as much or more about user-generated content
as it is about communications. Whats more, the growth and popularity of
virtual worlds, such as Habbo and Second Life, is beginning to
demonstrate how similar these worlds can be to the real world and how
these sites (and the people participating in them) take on lives of their
Looking at these Internet trends (social networking, user-generated
content, broadband connectivity, mobile computing, and visual, immersive
worlds), Intel sees an ever quickening convergence of visual and
connected computing that is blurring the traditional divisions between
real world activities and the digital world. This convergence is the basis
for ICE, an emerging usage model that offers great opportunities for
expanding ways we interact and experience the world and each other.
To better understand ICE and its potential, lets look deeper at its
Connected computing, as a usage model, is easy to comprehend. Weve all
grown accustomed to, even dependent on, being able to connect to each
other and exchange data, voice and video through the Internet. But
connectivity as a computing trend has really just begun. The increase and
diversity of mobile computing devices, faster broadband speeds, plus the
growth in input devices such as cameras and various types of sensors,
are not just making it easier to stay connected all the time always on,
always connected but also spawning new uses and making it more
important to have a continuous connection.
Most of us think of computing as visual. After all, we look at a screen. But
what we mean here by visual computing is immersive, real-time, life-like
computing experiences. Were at the dawning of an age of such visual
computing experiences. Examples of visual computing can already be
seen in the 3D visual models used in financial services, interactive medical
imaging, and the virtual worlds of Internet gaming.
Visual computing applications are evolving to deliver ever more
immersive visual experiences that look real, act real, and feel real. Many of
the elements necessary to bring such holistic visual experiences into
mainstream computing are in development and, with improvement, will
make visual computing an integral part of daily life.
Some of these elements include:
Improved modeling (better physics and intelligence)
Better multimedia (improved video/audio editing and
White Paper: Immersive Connected Experiences
More visual and tactile interactivity (advanced user
interfaces including new touchscreen and haptic
technologies  and other devices)
Immersive Connected Experiences
Technologies that combine connected and visual computing enable
people to share experiences and information within a highly intuitive,
interactive visual interface. Through connected visual computing
technologies, well be able to use computing devices to enhance and
extend our experiences of the real world, create new virtual worlds, and
bridge the digital and real world with increasingly rich, Immersive
ICE usages will include enhanced social networking, workplace
collaboration, retail sales, virtual tourism, marketing, distance learning,
training, gaming, and the arts, to name a few.
Current examples of ICE fall into two main categories.
Simulating Environments These are multi-user graphical
environments where large numbers of people interact for the
purposes of gaming, socialization, learning, or content creation.
Examples include virtual worlds, massively multiplayer online
games (MMORGs), and 3-D cinema.
Augmented Reality There are environments where
information from the actual world is combined with digital
information to provide an enhanced view of the real world.
Examples include Earth mapping applications such as Google*
The rapid growth of virtual worlds is particularly indicative of the
potential for ICE. In June 2008, the research firm Strategy Analytics
predicted that the percentage of registered users that will go on to
become active users in virtual worlds will increase from 10 percent now
to 27 percent by 2017, creating a market base of almost 1 billion users
. This is up from todays 137 million active users. Pre-teens make up
more than half the users, spending 16 hours per week (more than on TV
or magazines) and purchasing items in the virtual worlds . Many
convert their parents into virtual world users. Even more interesting will
be what happens if many of these pre-teens carry their virtual world
habits into adulthood. Strategy Analytics puts the 2008 total market for
user services at $310 million. The research group predicts that it will
grow to $8 billion in 2017 .
Augmented reality is also growing and evolving. Many people are familiar
with how online maps have evolved to include the overlay of satellite
images and recently street-level surround images. Combining this mixture
of real and virtual will be particularly compelling with the connectivity and
sensors of a mobile Internet device (MID). Images, video, audio, GPS,
acceleration sensors could all provide rich input that could be used to
create and then present to the user an augmented form of reality.
Consider being able to take a photo of a monument and then doing a
visual search using the image for real-time recognition of the monument
and information about its significance. With the processing power,
database and communication capability of a continual Internet
connection, there are many exciting opportunities that can be enabled.
Meeting the Challenges of ICE Removing key technical barriers to the adoption of Immersive Connected
Experiences will require research into hardware and software innovations
that improve end user experience, as well as the development of
standards that improve interoperability. We want cinematic quality in the
graphics, said Intel CTO Justin Rattner in his Fall 2007 IDF R&D keynote.
We want to make user content creation much easier. Of course we want
to retain the persistence, but we want that persistence to apply across
virtual worlds, not to be contained in a single virtual world. And higher
levels of social interaction, better human-computer interfaces to enable
that, and we'd like the behavior -- the way things act in the virtual world -
- to be much more natural.
The desired qualities for future ICE environments include the following:
Immersive visuals. Because theyre based on connected visual
computing technologies, the user experience should be greatly
enriched by immersive, highly realistic visuals.
User-generated 3D content. Many ICE applications will be
forms of social media. Thus, people will need and want to
create or modify visual content within ICE environments. This
will require new tools such as 3D character designers that are
as easy to use as the blogging, photo, and video editing tools
Interoperability. Like the Internet, ICE applications based on
broadly accepted standards will enable data and personas to
be shared between various ICE environments in creative ways.
ICE Example 1: Simulated Environments
Imagine if instead of having to waste a day traveling to a
conference and another day traveling back, you could attend a
virtual conference in a simulated environment. Take that a step
further and imagine, between presentations, being able to have
conversations with other attendees avatars where facial
expressions and body language actually help you and others make
your points with the correct emphasis.
Example: Qwaq* Forums
White Paper: Immersive Connected Experiences
In fact, according to Strategic Analysts vice president and
research director Barry Gilbert, "What's holding back the
numbers [participants] is the reliance on any single virtual
world because they're not yet portable."
Robust distributing computing. Because ICE applications are
distributed, good performance requires high quality, well-
managed interactions among a wide variety of clients and
servers. ICE applications should adapt to a diverse array of
clients so that the user experience scales smoothly from
performance-optimized home or office systems to compact
mobile devices optimized for energy-efficiency.
Scalability. ICE Environments should be able to scale over time
to incorporate new technologies which improve their quality as
well as to add more users as they grow in popularity.
Intel is driving ICE development through its continuing research into
many of the technologies and advancements that will be required to
bring ICE into the mainstream. This includes research in redefining
mobility, workload analysis, simulation, model-based computing, and tera-
Intel divides the challenges into four areas: platform optimization, visual
content, distributed computation, and mobility.
ICE applications are very compute-intensive. They demand a lot of
servers, clients and the network. Consider the complexity of the objects
and the need for accurate simulations, particularly realistic object
behavior. Then multiply that by the number of users and, the scene
complexity as more users participate and interact, and its easy to see
how ICE can be a tremendous challenge to system and network scaling.
Whats needed are new architectures that scale well and deal with a
range of client performance and capabilities, dynamically adapting to
changing demands and virtual world circumstances. Intel believes that ICE
applications will follow the same evolution the rest of social networking
and Internet applications have, evolving from proprietary solutions to
standard protocols and building blocks that can be used to build more
sophisticated services. Such modular design will also allow for scaling
through distribution and replication of components, as well as
repartitioning functionality among types of servers and the diversity of
Intel has a number of internal teams, projects and software-related
efforts underway to speed the transition to connected visual computing
technologies and future usage models like ICE. One extremely relevant
example here is Intels Tera-scale Computing Research Program. This is a
worldwide effort to advance computing technology by scaling multi-core
architectures to 10s to 100s of cores and embracing a shift to parallel
programming. Through this effort, Intel aims to enable applications and
capabilities like ICE (see Figure 1). Other research includes redefining
mobility, workload analysis, simulation and model-based computing
most of which will benefit from tera-scale computing.
Figure 1: Tera-scale microprocessors based on many cores will be
essential to computing the elements of ICE applications in parallel.
One of the distinguishing features of ICE applications, such as Second
Life, is the existence of player-generated content. Enabling easier end-
user creation and use of rich visual content is essential to ICE. In ICE
worlds based on the premise of user-customized 3D interactive content,
such data-rich content wont be conducive to distribution at startup or
even on a DVD ahead of time. It must be distributed real-time as users
explore and enter newly created regions. This will place severe demands
on the network requiring innovations in how content can be concisely
represented as well as cached and pre-fetched. This will also require a
new generation of technologies and tools designed to enable
To simplify content creation, Intel is researching a variety of areas. One is
parameterized content creation. This is the development of sets of
parameters for configuring and controlling an object, such as an avatar.
End-user creation is simplified to selecting and adjusting parameters,
such as the...