DMS - Florence - 2011 Ambient intelligence: a walkthrough S. Levialdi IEEE Life Fellow where are we going?

DMS - Florence - 2011

Ambient intelligence: a walkthrough

S. LevialdiIEEE Life Fellow

where are we going?


mashup computing


AmI: Wikipedia’s definition I– originally developed in the late 1990s for the time frame 2010–

2020 – refers to electronic environments that are sensitive and

responsive to the presence of people

– is a vision on the future of consumer electronics, telecommunications and computing

– devices support people in carrying out their everyday life activities, tasks and rituals in easy, natural way using information and intelligence hidden in the network connecting these devices

– devices grow smaller, more connected and more integrated into

our environment, the technology disappears into our surroundings until only the user interface remains perceivable by users.

página 3


AmI: Wikipedia’s definition II

• The ambient intelligence paradigm builds upon ubiquitous computing, profiling practices and human-centric computer interaction design and is characterized by systems and technologies that are:

• embedded: many networked devices are integrated into the environment

• context aware: these devices can recognize you and your situational context

• personalized: they can be tailored to your needs• adaptive: they can change in response to you• anticipatory: they can anticipate your desires

without conscious mediation.

página 4


AmI: ISTAGEuropean Community Standards

• AmI emphasizes on greater – user-friendliness, – more efficient services support, – user-empowerment, – and support for human interactions

• people will be surrounded by intelligent and intuitive interfaces embedded around everyday objects

• us and an environment recognising and responding

to the presence of individuals in an invisible way by year 2010.

página 5


What Would You Ask to Your Home if It Were Intelligent? Exploring User Expectations about

Next-Generation Homes?by Bonino and Corno

• separation is due to the natural distance between research and engineered applications or to the mismatching of needs and solutions?

• user perceptions about what intelligent homes can do• current research solutions or commercially available systems?• effectively and unobtrusively infer human activities in their

environments• first label and decompose activities as sequences of actions

with certain probabilities• then use these predefined activity models for recognition and

prediction •

página 6


Seamless integration

• Physical world • Virtual world • Networking persons to objects to other

persons• Connections may be

– Pre-established– Personally controlled– Varying in time– Varying with context

• The ambient is the interfaceKarich, 2011

página 7


the human senses

sight

touch

hearingtastesmell

balance &accelerationtemperaturekinesthetic sensepain

página 8


exploiting the human senses

Interacting with 1. humans - face-to-face communication2. computers – hand/eye utilization3. devices – generally tactile/sight4. appliances – mostly via touch5. environments – this is the challenge of

ambient intelligence

página 9


Smart homes

• Safe environment for people• Catering for special needs (senile dementia)• diagnosing interesting situations and• possibly advising the carer when intervention is

required • message can be generated automatically and• sent to carer (who may live remotely)• Existing smart homes:• Domus, Aware Home, MavHome, Gator Tech

Smart Home.

página 10


3 technologies(the claims)

convergence of

• Ubiquitous Computing (chips everywhere) – 3rd paradigm 1 X, 1 1,X Y

• Ubiquitous Communications • (info exchange from/to everywhere)

– PAN Personal Area Networks– VAN Vehicle Area Networks– MAN Metropolitan Area Networks

• Intelligent User Friendly Interfaces(variety of humane interfaces)

flexible, usable, accessiblepágina 11


AmI sample applications1. Touch and gesture-based interfaces2. Haptics and biometrics based navigation;3. 3D human activity recognition and monitoring;4. Interacting through 3D displays;5. Machine learning in behavior understanding;6. Real-time, de-centralized media-processing

achitectures7. Human-centered and location-aware multimedia8. Sensor perception and context awareness9. System personalization

Applications: Online browsing for heritage, e-learning and collaborative education, tourism and e-commerce, autonomous navigation,security and surveillance, mobile navigation.

página 12


Optimistic view


NTT: a vision for 2050

AmI is seen as a “carrier of memories”

Decrease anxiety – global warmingIncrease mental stability –

educational issuesYoshinobu Tonomura

E. Maeda

NTT Labs, Japanpágina 14


AmI: global communication

Allow communication with• Machines• Persons• Objects• Nature• Universe…

NTT Labs, Japan página 15


MIT: fluid interfaces…

• rethink the human-machine interactive experience

• more immersive, intelligent & interactive interfaces

• systems which are more responsive to people’s needs & actions

Pattie MaesMIT leads 53 projects


challenges in Just-in-time information

–likely relevant to the userChallenges in:

–user profiling–detecting context of user–recommendation algorithms

–offered unobtrusivelyChallenges in:

–Subtle interfaces

–Require minimal user effort to accessChallenges in:

–Natural, “on-the-move” interfaces

–safeguards the user’s privacy Pattie Maes, 2005

página 17


approaches

• Integrating the rich content of the digital world into our everyday physical existence

• Integrating the rich content of the analog world into our ever-growing number of gadgets, devices, computers, …

Pattie Maespágina 18


invisible media

•augment objects •sensitive, informative, •focus of our attention to provide relevant

content

Engine-info

página 19

⌘ F5Ctr 1

Move to the right.Check the pulley.


ReachMedia

On-the-move Interaction with Augmented Objects

Wireless RFID reader wristband reads tags in objects held by user

Touching an object resultsin a menu of services andinformation

Assaf Feldman, Sajid Sadi, 2005página 20


SubTextile

Sadi & Mihkak, 2005/9

enabling technology for I/O fabrics + applicationsin furniture & everyday objects

Anti-snoring, alarm clock, snooze, etc

página 21


From ink to digital data

página 22


Intelligent “Quickies”‘Quickies’ enriches the experience of using sticky notes by linking hand-written sticky-notes to the mobile phones, digital calendars, task-lists, e-mail and instant messaging clients. Augmenting sticky-notes,‘Quickies’ leverages existing patterns of behavior, merging paper-based sticky-note usage with the user's informational experience. AI, Natural Language Processing (NLP), RFID, and ink recognition technologies can make it possible to create intelligent sticky notes that can be searched, located, can send reminders and messages…

página 23


University of Ottawa

• an integrated approach to AmI• two precursory laboratories:

– 1984 – MCRLab: Multimedia Communications Research Laboratory

– 2002 - DISCOVER: Distributed Collaborative Virtual Environments Research Laboratory

Nicolas D. Georganaspágina 24


MCRLab I

• vision of human-centred research: – "i have aims" meaning: interactive Hapto-

Audio-Visual Environments for Ambient Intelligent Media Systems

• Environments• Multimedia Security (Watermarking) and

Tele-Surveillance• Ambient Intelligence• Collaborative Virtual Environments

Nicolas D. Georganas

página 25


MCRLab II

página 26


DISCOVER

• 3D Physical Modelling and Animation• Distributed and Collaborative Virtual

Environments• Intelligent Sensor Networks and

Ubiquitous Computing• Multimedia Computing and

Communications• Tele-Haptics

página 27


GUIs

spokendialoguegestural

multimodal

facialexpressions

haptic

biometricsignals

página 28


AmI @ IkerlanHumans surrounded by intelligentinterfaces supported by computing andnetworking technology embedded ineveryday objects such as furniture, clothes,vehicles, roads and smart materials

enabling and facilitating participation by theindividual – in society, social businesscommunities, and in the administration andmanagement of all aspects of their lives.

Rosa Iglesias, 2009

página 29


Ikerlan

Present research lines on AmI:

• Embedded systems for AmI• Advanced management systems• Design of mechanical systems• Mass personalization of products and

continuous innovation

Ikerlanpágina 30


Work Home

Health

Entertainment

Transportation

AmIimpact map

software

sensors

electronicscommunications

microsystems

Ikerlan’s view

página 31


Smart materials

IKERLAN-IK4 develops the first solar module in plastic in Spain

Smart windows Piezoelectric device generateselectricity from footsteps

Ikerlan

página 32


Applied technology

Virtual butler Wall display

Integratedmultimodal interaction

Ikerlan

página 33

Corning Glass


Killjoy view


Brin’s vision: 2 citiesCity of control - CoC

– deployment of radio frequency identification tags (RFID) have become not just commonplace but ubiquitous

– objects, spaces and, even people are tagged and given a unique number

– notions of public and private have begun to dissolve

– every item you buy at the supermarket is being tracked and potentially data-mined

– your movements are watched, not by the use of crude cameras but by tags embedded in your gadgets or in your clothes or even under your skin

David Brin

from “A Tale of Two Cities”

página 35Sean Dodson


Brin’s vision: 2 cities

City of trust - CoT– on the surface, very similar to CoC– pervasive systems are embedded but more

control is given to users– citizens may switch off surveillance cameras– Mark Weiser’s quote: future of information

society is to consider it “an utility”– ubicomp should be visible even if integrated

página 36


Shenzhen2010 – pop.10,357,938 (47.8%)

• a blueprint of CoC is already here• a string of small existing fishing villages• China’s first special economic zoneiPods, laptops, sneakers, cars are made• 200.000 surveillance cameras – closed

circuit• 2.000.000 CCTV’s will be installed• an all-seeing system national-wide• everybody is tracked!

Naomi Kleinpágina 37


CoC or CoT?

página 38


the world is changing

from humans with tools for information handlingto information spaces

4 levels for introducing new technologies:

1. code to implement distributed, global systems

2. node servers log, store & track vast data amounts

3. link to ensure digital connectivity4. network a policy: control, security, safety

Rob Van Kranenburg, 2008página 39


Rob’s ideas I

• Buildings, cars, houses become information spaces

• Objects and places are “augmented” with info processing – as electricity is today

• In a mediated environment it is not clear what is mediatedmediated and what mediatesmediates

• What connectedness do we really want?• The environment has become the interface

Jakob Weehjert

página 40


Rob’s observations on AmI

• No more public – only audience• No forgetting – no memory loss in digital

territory (right or wrong?)• There are no more humans – only

information spaces• From intelligence to EXtelligence• Submersed in information spacesinformation spaces• Personal agency will be lost

página 41


An example of AmI implementation


Song-Do City South Korea

Digital Life:Public recycling bins that use RFID to credit recyclersevery time they toss in a bottle; pressure-sensitive floors in the homes of older peoplethat can detect a fall and contact help; phones that store healthrecords and canbe used to payfor prescriptions...

página 43

Cisco Connected Town


A prediction:

“The most profound technologies are those that disappear.

They weave themselves into thefabric of everyday life until they areindistinguishable from it.”

T. Weiser, 1991página 44


limitations

• AmI aims at hiding computing technology

• Avoiding citizens to modify, expand, play…

• No creativity is allowed• “children cannot change lines of

code”: a metaphor for the AmI scenario

David Brinpágina 45


Two initiatives

Hive & Bricolab


Hive Networks IMiami University, 207

• is an organization that is liberating commonly available embedded computers for use by digital artists

• transcends the boundaries between engineering and art

• a work of art as well as a platform for other artists to create works.

• It combines the element of content with the element of networking.

Wolfgang Hauptfleisch, Alexei Blinov, Cieron Edwards página 47


Hive Live Social networking platform

• HiveLive is a platform with a lot of standard social networking functionality

• gives ordinary users the ability to control the modules deployed within a social network

• social networks often incorporate many Web 2.0 tools that could stand alone elsewhere, such as blogs, forums, and wikis

• with most platforms, the network administrator determines how these tools are deployed on the site: drop in a forum here, add a wiki there, and so on

• the network users for the most part then have equal access to these predefined areas of interaction

página 48


HLS platform I• HiveLive introduces customizable components they call

“Hives”.

• Network users can choose to add their own hives to an existing social network, configure the type of hive (blog, forum, wiki, etc.), and then determine who within the network gets to use them (everyone or just a subset of friends and colleagues).

• Want to join with friends to run a blog about a shared interest?

• Create a new blog hive that everyone can see but only a few can edit. Want to collaborate on a project with team members using a wiki?

página 49


HLS platform II• Create a wiki hive that only a few people can access at all

• The HiveLive platform is flexible enough so that you don’t even have to create a hive that’s easily categorized. If you just want to share a string of items with friends, you can create a blog-like page with a variety of data types.

• Users can also create their own hives by copying other hives with a few buttonclicks.

• HiveLive was self-funded for its first couple of years but raised $1.6M from institutional angels towards the end of 2006. They’re calling their product the “LiveConnect Community Platform.”

• •

página 50


Hive Networks: facilities• Each Hive device is capable of gathering

content (through webcams, microphones, sensors) and disseminating it (web server, audio/video live streams, bluetooth, wlan).

• At the same time each Hive device also acts as a node in the network, which means that it is capable of storing and forwarding data.

página 51


Bricolab• Stems from the metaReciclagem or

• “metarecyling” idea

• Mouvement de réappropriation des objets, qui se bricole lui même

• bringing people together with new technologies and distributed connectivity,

• unlike the dominant focus of IT industry on security, surveillance and monopoly of information and infrastructures

página 52


Bricolab: a shared network

• A distributed network for global and local development of generic infrastructures incrementally developed by communities.

• A global platform to investigate the new loop of

– open content,

– software and

– hardware for community applications,

R. Pontonepágina 53


≠ towns

página 54


AmI: where to?

• possible forms of connectivity:

– people to people (voluntary)

– people to machines (still voluntary)

– objects to people (unintended)

– ambient to people (uncalled)

página 55


The bottom line:• we need control over connectivity to ensure:

– Safety

– Privacy

– Creativity

• Two movements, support the above:– Hivenetworks– Bricolab

página 56

Documents

DMS - Florence - 2011 Ambient intelligence: a walkthrough S. Levialdi IEEE Life Fellow where are we going?