Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
© 2004 MIT Media Laboratory, Hiroshi Ishii
Tangible Bits Beyond Ubiquitous GUIs
2004 Tokyo
Hiroshi Ishii Tangible Media Group MIT Media Laboratory
1
© 2004 MIT Media Laboratory, Hiroshi Ishii
Tangible Bits Designing the Seamless Interface between People, Bits, and Atoms
February 2004
Hiroshi Ishii Tangible Media Group Things That Think MIT Media Laboratory
2
Ubiquitous Computing Mark Weiser, Xerox PARC, 1991
• Computers should be " ." •
, Pads, and Boards,
each other.
© 2004 MIT Media Laboratory, Hiroshi Ishii
transparentComputational services are delivered through a variety of computational devices such as Tabswith the infrastructure to allow these devices to talk with
Mark Weiser at Xerox PARC presented his vision of “Ubiquitous Computing” in 1991. He claimed computers should be transparent and invisible.
He claimed that the most profound technologies are invisible. They weave themselves into the fabric of everydaylife until they are indistinguishable from it.His team designed a variety of computational devices including Tabs, Pads, and Boards along with theinfrastructure to allow these devices to talk with each other.
Recently, he is introducing a concept of “calm technology” citing the LIVE WIRE by Natalie Jeremijenko as an example.
3
Misunderstood Ubiquitous • • • Multiple Computational Devices / User
–
© 2004 MIT Media Laboratory, Hiroshi Ishii
Anytime & Anyplace Mobile, wireless, broadband, RFID, ….
c.f. Mainframe (TSS) computing: one computer / many users Personal computing: one computer / user
Mark Weiser at Xerox PARC presented his vision of “Ubiquitous Computing” in 1991. He claimed computers should be transparent and invisible.
He claimed that the most profound technologies are invisible. They weave themselves into the fabric of everydaylife until they are indistinguishable from it.His team designed a variety of computational devices including Tabs, Pads, and Boards along with theinfrastructure to allow these devices to talk with each other.
Recently, he is introducing a concept of “calm technology” citing the LIVE WIRE by Natalie Jeremijenko as an example.
4
1997… i
i i l
..........
i
..........
incl
ject. i
project! ..........
i
© 2004 MIT Media Laboratory, Hiroshi Ishii
Wieser wrote to me in
Date: Sun, 26 Jan 1997 23:34:10 PST Subject: "Tangible Bits"
Dear Hirosh and Brygg,
I recently had a chance to read your CHI 97 paper "Tangible Bits"! Great work! In my opinion this s the k nd of work that will characterize the technologica
landscape in the twenty-first century.
My request is that you help me stop the spread of misunderstanding of ubiquitous computing based s mply on its name. Ubicomp was never just about making "computers" ubiquitous. It was always, like your work, about awakening computation mediation into the environment.
I tried to stop using ubiquitous computing because of its misleading implication, but it keeps cropping up again, so I keep returning to it as my umbrella name for lots of work,
uding Things That Think. Augmented reality was in use for awhile, but again got balkanized in meaning. I have started to talk about Calm Technology as a theme, but it better names a goal than a research pro "Tangible Bits" is very n ce, and maybe could serve as an overall umbrella, but then you might lose it as the name of your research
-mark (Dr.) Mark We ser Chief Technologist, Xerox PARC
5
LIVE WIRE
• 8 foot piece of plastic rope that hangsfrom a small electric motor mounted in the ceiling. The motor is linked to a
• Bits tangible
through motion, sound, and touch.
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
Natalie Jeremijenko, 1995
nearby LAN, so that passing bits cause twitching of the motor.
flowing through the wires of a computer network become
Mark Wieser called “Calm Technology”
Natalie Jeremijenko designed a beautiful instrument called Live Wire while an artist in residence at Xerox PARC.It is a piece of plastic cord that hangs from a small electric motor mounted on the ceiling. The motor is linked to the Ethernet, and each passing digital packet causes a tiny twitch of the motor.
Bits flowing through the wires of a computer network become tangible through motion, sound without being obtrusive, taking advantage of peripheral cues. This work encouraged us to think about ambient media as a general mechanism for displaying activities in cyberspace.
6
“The Computer for the 21st Century”
“The most profound technologies are those thatdisappear.
Mark Weiser
© 2004 MIT Media Laboratory, Hiroshi Ishii
They weave themselves into the fabric of everyday life until they are indistinguishable from it.”
July 23, 1952 - April 27, 1999
Mark Weiser wrote as follows in the 1st paragraph of his landmark paper on Ubiquitous Computing published in 1991 Scientific America . “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” If you learn the task quite well using the right tools, and become the master, the tools cease to be a center of users’ attention. �Tools disappear into the background and you can concentrate on the tasks themselves. This is the definition of transparent or invisible computers by Mark. Invisible does not mean you can not see with your eyes. It is the matter of user’s focus of attention and consciousness. It is the state in which tools do not get in your way and you can concentrate on the task. Mark showed a series of ubiquitous computers including Live Board, Pads, and Badges. All of them come with screens and GUIs. But I do not feel Ubiquitous GUI is the path to achieve invisible interface. Today, I would like to show my approach: Tangible Interfaces.
7
At the Border
Where the land meets the sea, there is a border.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Where the land meets the sea, there is a border. This borderline is constantly moving and changing the shape of the shore. However, it is not only a simple line of a landscape, but also a critical biological locale.
8
Living at the Border
Harsh, but also fertile environment.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Whenever the tides go in and out, the inhabitants in the intertidal zone experience harsh environmental changes. When the tidal animals are exposed, they must withstand the heat and drying. When they are submerged, they must face constant pounding of waves. Living at this border is hard, but it is also a fertile environment for many kinds of life and activity. Where the land meets the sea, countless creatures have evolved diverse and creative approaches for flourishing among the turbulence and constant changes of these competing worlds. Our ancestors came from the sea and landed crossing this border a million years ago.
9
At the Border
We live on the border where bits meet atoms. In the flood of pixels from the ubiquitous GUIscreens, we are losing our sense of body andplaces.
© 2004 MIT Media Laboratory, Hiroshi Ishii
between Physical and Digital
Pixels impoverish human senses.
We are now crossing another border. Where bits meet atoms, there is another shoreline. Today, we live on the boundary between physical space and cyberspace. The constant pounding by waves of bits from screens distorts our experience of space. People are losing theirsense of body and places in the flood of pixels from the ubiquitous rectangular screens. Living on this border is hard because our bodies remain in the physical world, even while we are immersed within digital information. It is also hard because current interfaces are not designed for amphibians. We are halfdrowning in the sea of bits, squeezing plastic mice instead of breathing and swimming. People are standing at the water's edge at a loss.
10
© 2004 MIT Media Laboratory, Hiroshi Ishii
At the Boundary Where the sea meets the land, life has blossomed into a myriad of unique forms in the turbulence of water, sand, and wind.
At another seashore between the land of atoms and the sea of bits, we are now facing the challenge of reconciling our dual citizenships in the physical and digital worlds. Windows to the digital world are confined to flat square screens and pixels, or "painted bits." Unfortunately, one can not feel and confirm the virtual existence of this digital information through one's body.
Tangible Bits seeks to realize seamless interfaces between humans, digital information, and the physical environment by giving physical form to digital information and computation, making bits directly manipulable and perceptible.
Hiroshi Ishii | Tangible Media Group | MIT Media Lab
11
• for the Boundaries
• Seamless Collaboration Media at NTT Human Interface Laboratories
• • )
• Tangible User Interfaces (96-00)
• • ( )
© 2004 MIT Media Laboratory, Hiroshi Ishii
Seamless Interfaces
(88-94)
Beyond “Being There” CSCW (Computer-Supported Cooperative Workcommunities: ACM SIGOIS, SIGCHI, IFIP
at Tangible Media Group, MIT Media Laboratory Beyond “Painted Bits” HCI Human-Computer Interactionscommunities: ACM SIGCHI and SIGGRAPH
12
My Art Work in 1959
© 2004 MIT Media Laboratory, Hiroshi Ishii
Let me show another art work. This is my drawing on a fridge door in 1959. I loved drawing. Since then, drawing has been a very important medium in which I think, express myself and communicate with my mother.
13
•
•
Shared Workspace Interpersonal Space © 2004 MIT Media Laboratory, Hiroshi Ishii
Goal of ClearBoard Design Seamless Integration of Shared Workspace and Interpersonal Space Natural and smooth transition between them using everyday cues
ClearBord, the shared drawing media I designed, is deeply rooted in my drawing practices and my liking for the whiteboard.
Our goal was to create a collaboration medium that seamlessly integrated the IPS and the SWS.
We designed ClearBoard to allow users to shift easily between these spaces by using familiar everyday cues such as the partner's gestures, head movements, eye contact, and gaze direction.
14
Beyond “Being There” Beyond “Talking Heads” • Tacit Goal of Telecommunication:
distance – But this goal is a mirage. –
• New Goal: Creation of added value
cannot provide – e.g. Virtual Shared Workspace
© 2004 MIT Media Laboratory, Hiroshi Ishii
Imitation of "Being There" over
It can never be achieved 100%.
that an ordinary face to face meeting
Most current video telephony systems have been designed to see "talking heads," and to create an illusion as if all the participants sit in an imaginative conference room. But this goal to simulate as close as to the real face to face meeting, however, it is something like a mirage. It can never be achieved 100%. The gap between the recognizable reality and virtuality cannot be filled by technology.As long as we try to find the raison d'etre of communication media only in its being "tele", we shall never be able to solve this contradiction. Conventional logic that it saves travel expenses and time should be reviewed.Our goal is to go beyond this model and demonstrate new usage of video communication technologies. To do so, we have integratedgroupware technology and real-time video communication technology to go beyond the imitation of physical proximity.Instead of pursuing the illusion of "Being there", we should focus on the creation of added value which cannot be obtained from face to face conversation.
15
Beyond Multimedia
• Multimedia
•
– –
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
= more than 3 cables behind a computer? "Multi-ness" is not main issue; rather seamless media integration which hides
low level representations and complexity of underlying technologies.
Multimedia = a premature label that represents a stage of media evolution from the mono-media to the seamless media.
"Multimedia" is now becoming a big buzz word in the computer and communication industries. As a result, the number of cables behind a computer, the number of features users need to understand, and the number of incompatible data formats are increasing beyond the limits of human cognitive capability.
The communication channels of human beings are inherently multi-modal and seamless. It does not make much sense to decompose the representation of information into primitive data types such as text, video, audio, and graphics, and stress the "multi-ness" of the media. For example, we are speaking, gesturing, and drawing simultaneously in a design meeting. We have great skills to express ideas and understand each other in everyday contexts using all these media as a whole. We believe the multi-ness of media is not the main issue, but how to integrate them into a seamless media hiding the various low level representations, discontinuities among primitive media, and complexity of underlying technologies is the core issue in designing new applications.
"Multi-media" sounds like a premature label that represents a stage of media evolution from the mono-media to the seamless media.
16
Vision Video “Seamless Media Design”
• TED4 Kobe (94), Door of Perception 2 (94)
© 2004 MIT Media Laboratory, Hiroshi Ishii
17
© 2004 MIT Media Laboratory, Hiroshi Ishii
PDA in 1958 - Abacus
When I was 2 years old, I started playing with a PDA called the "abacus".It was more than my mother's financial calculator. The abacus became my musical instrument, imaginary toy train, even back-scratcher. I enjoyed the sound and tactile interaction with this simple physical artifact. The abacus also became a medium of awareness. When my mother used the abacus to manage household accounts, I learned she could not play with me while her abacus made its music. My childhood abacus is suggesting us a future direction of HCI. That is the reason why I started the Tangible Bits project at MIT.
18
Eyes are in charge, but hands are underemployed. By pointing, by pushing and
pulling, by picking up tools,hands act as conduits through which we extendour will to the world.
They serve also as conduitsin the other direction: hands bring us knowledgeof the world. Hands feel. They probe. They practice.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Malcolm McCullough “Abstracting Craft: The Practiced Digital Hand ” 1996
Eyes are fully employed, but hands are underusedHands are underemployed. GUIs (Graphical User Interfaces) are designed for human eyes, not for hands. Telepresence, or interpersonal communication, is not the exception. Communication media such as telephones, video conferencing, and e-mail are for human ears and/or eyes, but not for hands. McCullough articulated the importance of hands as follows, “By pointing, by pushing and pulling, by picking up tools, hands act as conduits through which we extend our will to the world. They serve also as conduits in the other direction: hands bring us knowledge of the world. Hands feel. They probe. They practice.” [McCullough, 1996 #1].
19
Drawing Instruments What we have lost in the flood of PCs
© 2004 MIT Media Laboratory, Hiroshi Ishii
Aesthetics which value haptic interaction with specialized physical objects ... but much richness has been lost.
David Brewster’s Edinburgh Encyclopedia (1808-1830)
This slide shows the various architectural drawing instruments used in 19th century. Long before the invention of personal computers, our ancestors developed a variety of specialized physical artifacts to draw geometric shapes. Grasping and manipulating those instruments, users of the past must have developed unique and rich languages, cultures, and aesthetics which value haptic interaction with real physical objects. We began our investigation of "looking to the future of HCI" at a museum by looking for what we have lost in the flood of personal computers. Our intention is to find a way to rejoin the richness of the physical world in HCI.
20
Tangible User Interfaces
• digital information & computation, making bits –
) – perceptible through
our peripheral senses)
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
Giving physical forms to
directly manipulable with our hands (in manipulatory
(in ambulatorySeamless coupling between digital and physical worlds.
Tangible interface is an interface design approach to give physical forms to digital information to make it tangible. It makes bits directly manipulable with our hands, and perceptible through our peripheral senses. The first path to the Tangible Interface is the directmanipulation with hands. Second path is the use of ambient media to make people aware of digital information at the periphery of perception. Pinwheels is such an example.The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context.
21
Center and Periphery
• to grasp & manipulate bits center
• to be aware of bits at the peripheryusing
© 2004 MIT Media Laboratory, Hiroshi Ishii
Architectural Space as Interface
in the of user's focus by coupling bits with physical objects and surfaces, and
ambient display media such as light, sound, airflow, and water movement.
This slide illustrates the framework of “center and periphery” of human perceptions. The ambientROOM was designed to achieve two goals: 1) to grasp & manipulate bits in the center of user's focus by coupling bits with physical objects and surfaces, and 2) to be aware of bits at the periphery using ambient display media such as light, sound, airflow, and water movement. We get information in two main ways. First, we get information from what we are focusing on, where our center of attention is directed. Butat the same time, we also get information from ambient sources.We may have a sense of the weather outside from ambient cues such as light, temperature, sound, and air flow from the open windows. We may also have an idea of the activities of our colleagues from the ambient sound and the shadows of passers-by.
Our brain is not only capable of processing this ambient information while maintaining a focus elsewhere, but will naturally shift such information to our foreground when our subconscious deems it important. By using ambient media, we can take advantage of our brain's abilities as both a parallel processor and an attention manager. Current HCI research is primary focusing on foreground activity and neglecting the background. We would like to support seamless transitions of the user's focus of attention between the periphery (background) and the center (foreground) of human awareness.
22
Pinwheels: wind of bits
• Ambient information
• Architectural space
humans and online digital information.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Ren, Frei, Dahley, Wisneski, and Ishii, 1997-2000
display spinning in a "wind of bits.”
will be an ambient interface between
23
© 2004 MIT Media Laboratory, Hiroshi Ishii
Pinwheels: wind of bits Ren, Frei, Dahley, Wisneski, and Ishii, 1997-2000
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Ambient information display spinning in a"wind of bits.”
Architectural space will be an ambient interface.
QuickTime™ and a decompressor
are needed to see this picture.
24
created by a "rain of bits."
QuickTime™ and a decompressor
are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Water Lamp: rain of bits Dahley and Ishii, 1997
Water ripple shadow
25
Foreground --> Background
ambient
PCs
phone
© 2004 MIT Media Laboratory, Hiroshi Ishii
Peripheral Awareness using Ambient Media
Always on, real-time Peripheral awareness Seamless with environment
Time-consuming Requires navigation Complex
Interruptive Intrusive
26
Orb
•
portfolio, traffic on
mountains, pollenindex, etc.
• remapped to
© 2004 MIT Media Laboratory, Hiroshi Ishii
by Ambient Devices (Media Lab Spinoff) www.ambientdevices.com
This light glows different colors to help you monitor your
your commute, new snow in the
The behavior can be
summarize whatever information you’d like in your periphery.
New York Times Magazine, Dec. 2002
27
27
© 2004 MIT Media Laboratory, Hiroshi Ishii
Information Gauges 情報計器courtesy of Ambient Devices
Market 市場 Pollen 花粉 Weather 天気
• Ever-present information gauges log data and show trends in pollen index, weather forecast, snow survey, today’s market, blood pressure, blood sugar, exercise, weight, etc.常時表示の情報計器がデータを収集記録し表示する。例えば花粉、天気、降雪量、市場、血圧、血糖値、体重など。
Ambient DisplaysDesign Principles
• –
• Calm –
• Persistent connection –
• Decision-driven data –
• Private –
© 2004 MIT Media Laboratory, Hiroshi Ishii
Browser-less interface Glance-able, requires no navigation and no analysis, simple.
Non-intrusive, seamless with environment
Information is continuously updated.
Personalized and summarized data feeds to make a decision.
Encoded data
29
29
© 2004 MIT Media Laboratory, Hiroshi Ishii
Wireless products will diversifyワイアレスプロダクトは進化の初期段階
• Like wireless, electricity started out with one application in mind: the light bulb (電気の最初のアプリケーションは電球)
• It took 20 years for the first new products using electricity to be introduced (最初の製品導入まで 20年を要した)
• Today electricity is an invisible carrier to virtually every consumer product(今日電気は遍在キャリア)
Electricity電気
Wirelessワイアレス
Light Bulb 電球 Mobile Phone 携帯電話
?
ambientROOM
on ceiling
Light projection
Ambient sound of rain drops
Bottles as containers of bits
Clock to navigate time
© 2004 MIT Media Laboratory, Hiroshi Ishii
Architectural Space as Interface
Ripple shadows
on side wall
Open a bottle to release bits into air
Let me summarize what you saw in this video.We demonstrated three types of ambient media.First, ripple shadow display. We made a thin water tank outfitted with a small robot which taps water being "pulled by bits." With each pull, the robot creates a ripple on the surface of water. Light projected onto the water tank from above casts a subtle but poetic image of rippleson the ceiling of the room.Second, subtle light projection on a side wall provide awareness of activities in a remote working place.Third, ambient sound of rain drops or water stream is used to let people be aware of activities in cyberspace such as network traffic.
31
inTouch: Tangible Telepresence
create an illusion of touching the same object
inTouch-0: inTouch-1: inTouch-2:
© 2004 MIT Media Laboratory, Hiroshi Ishii
Brave, Dahley, Frei, Su, and Ishii, 1998
“Synchronized Distributed Physical Objects”
using force-feedback technology.
mechanical mockup
early electronic prototype
distributed prototype
Touch is underdeveloped. Touch is a fundamental aspect especially of interpersonal communication. Physical contact is a basic means through which people achieve a sense of connection and express emotion. Communication through touch, however, has been left largely unexplored in telepresence research. Visual and auditory extension of space was dominant approach of telepresence as represented by ClearBoard. In this summer, we have started the implementation of a distributed prototype via a computer network. The goal of this design is to allow the virtual connection of inTouch-1 to extend over any arbitrary distance.
In experiments using inTouch-1, in conjunction with audio communication, for longer periods of two-person interactions, subjects often noted that the haptic interaction with inTouch-1 was distracting at first, but later fell more into the background of perception. When this happened subjects felt that the interaction became more natural and smooth. Subjects also general felt that the haptic channel presented a “subtle but deep and revealing” addition to audio communication. More specifically users commented that the haptic channel allowed them to get a better sense of the other person’s emotional state and sincerity. Finally, some users expressed feelings of awkwardness when interactions with an acquaintance became more rhythmical, since they associated such movements with more intimate relationships.
32
© 2004 MIT Media Laboratory, Hiroshi Ishii
inTouch: Haptic Interpersonal Communication Medium
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
illusion of touching the same object using force-feedback technology.
QuickTime™ and a YUV420 codec decompressor
are needed to see this picture.
Touch is also a fundamental aspect of interpersonal communication. We use touch to achieve a sense of connection, indicate intention, and express emotion. Though often subtle, touch can be a very powerful mechanism through which we express our thoughts and feelings to others. Current telecommunications technology, however, lacks the mechanism for expression through touch. The goal of our work is to enhance real-time remote collaboration and communication by bringing a greater sense of touch and physicality to distributed multi-user interactions.
33
“Ghostly Presence”
Remote users remain isolated behind computer screen.
Movement of local objects suggests the physical presence of remote users.
© 2004 MIT Media Laboratory, Hiroshi Ishii
shared physical work space user B’s physical space
traditional remote collaboration systems
user A’s physical space
tangible telepresence
We have primarily described Synchronized Distributed Physiacl Objects as a means for extending the Tangible Interface approach into the realm of distributed CSCW. In our observations of and discussions with users of both PSyBench and inTouch, however, we have found that this approach also suggests a method for providing a new type of awareness of remote users. In traditional telecommunication systems we get the distinct impression that the distant user is in a space that is separate from our own. They are, for example, isolated in the digital world on the other side of our computer screen, where we are unable to touch them or share our physical world with them. In providing distant users with a shared physical space, we begin to see a
new form of awareness of the physical presence of the distant user within our own space. As the objects on the surface of PSyBench or the rollers on inTouch move around "on their own", we are compelled to imagine the position and actions of the person causing that movement... much like a player piano compels us to imagine the a real body sitting at the piano bench with arms extending to the keys. And we tend to imagine that presence within our own space; a sort of ghostly presence in our environment which can see and manipulate the same objects that we have access to.
34
ComTouch:
• Goal: Use touch to enhance real-time remote communication
• tactile communication language can be developed
© 2004 MIT Media Laboratory, Hiroshi Ishii
A Vibrotactile Communication Device A. Chang, Z. Kanji, A. O'Modhrain, E. Gunther R. Jacob, and H Ishii, 2002
Motivation: Explore how
The ComTouch project is about designing a touch communication device for mobile use. We aim to enhance interpersonal communication by using touch. Touch conveys a wealth of personal information. Although we know that tactile communication devices do exist, we were interested in further development of touch communication, particularly assessing in what ways touch can enhance audio interaction.
35
Touch to vibration mapping
© 2004 MIT Media Laboratory, Hiroshi Ishii
We chose to implement the touch-to-vibration mapping for the hand as shown here. For each finger, there are three different areas under the finger. Fingertip pressure is converted into vibration frequency and intensity. Because this is such a non-intuitive mapping , we implement a feedback channel so that the sender can feel their squeeze. This vibration is sent to a corresponding partner, who can squeeze back. By pressing harder… stronger and faster vibrations will result.
36
PSyBenchBrave, Bulthaup, Ishii, 1998
QuickTime™ and a decompressor
are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physically Synchronized Workbench for Remote Collaborative Design (very early crude prototype)
37
CurlybotFrei, Su, ishii, 2000
motion. QuickTime™ and a
YUV420 codec decompressorare needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
A toy that can record and playback physical
Children establish an affective and body syntonic connection with curlybot, and develop intuitions for concepts such as differential geometry.
curlybot is an toy that can record and playback physical motion. As one plays with it, it remembers how it has been moved and can replay that movement with all the intricacies of the original gesture; every pause, acceleration, and even the shaking in the user's hand, is recorded. curlybot then repeats that gesture indefinitely creating beautiful and expressive patterns. === We introduce an educational toy, called curlybot, as the basis for a new class of toys aimed at children in their early stages of development - ages four and up. curlybot is an autonomous two-wheeled vehicle with embedded electronics that can record how it has been moved on any flat surface and then play back that motion accurately and repeatedly. Children can use curlybot to develop intuitions for advanced mathematical and computational concepts, like differential geometry, through play away from a traditional computer.
In our preliminary studies, we found that children learn to use curlybot quickly. They readily establish an affective and body syntonic connection with curlybot, because of its ability to remember all of the intricacies of their original gesture; every pause, acceleration, and even the shaking in their hand is recorded. Programming by example in this context makes the educational ideas implicit in the design of curlybot accessible to young children.
38
CurlybotFrei, Su, ishii, 2000
•
• intuitions for concepts such as differential geometry,
from a traditional computer.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Children readily establish an affective and body syntonic connection with curlybot. They can develop
through play away
curlybot is an toy that can record and playback physical motion. As one plays with it, it remembers how it has been moved and can replay that movement with all the intricacies of the original gesture; every pause, acceleration, and even the shaking in the user's hand, is recorded. curlybot then repeats that gesture indefinitely creating beautiful and expressive patterns. === We introduce an educational toy, called curlybot, as the basis for a new class of toys aimed at children in their early stages of development - ages four and up. curlybot is an autonomous two-wheeled vehicle with embedded electronics that can record how it has been moved on any flat surface and then play back that motion accurately and repeatedly. Children can use curlybot to develop intuitions for advanced mathematical and computational concepts, like differential geometry, through play away from a traditional computer.
In our preliminary studies, we found that children learn to use curlybot quickly. They readily establish an affective and body syntonic connection with curlybot, because of its ability to remember all of the intricacies of their original gesture; every pause, acceleration, and even the shaking in their hand is recorded. Programming by example in this context makes the educational ideas implicit in the design of curlybot accessible to young children.
39
topobo
• (• • • • •
© 2004 MIT Media Laboratory, Hiroshi Ishii
Building Blocks with kinetic memory Hayes Raffle, Amanda Parkes, and Hiroshi Ishii
made of active motorized) & passive (static) components passives geometry based on cubic & tetrahedral crystals coincident input & output space actives “programmed” by moving, pushing, twisting units recorded sequence automatically plays back repeatedly distributed computation and networking
40
topobo
• )• • • • •
QuickTimeý DzYUV420 ÉRÅ[ÉfÉbÉN êLí£ÉvÉçÉOÉâÉÄ
ǙDZÇÃÉsÉNÉ`ÉÉǾå©ÇÈǞǽDžÇÕïKóvÇ-Ç�ÅB
© 2004 MIT Media Laboratory, Hiroshi Ishii
Building Blocks with kinetic memory Hayes Raffle, Amanda Parkes, and Hiroshi Ishii
made of active (motorized & passive (static) components passives geometry based on cubic & tetrahedral crystals coincident input & output space actives “programmed” by moving, pushing, twisting units recorded sequence automatically plays back repeatedly distributed computation and networking
41
Coincidence of input and output spaces
© 2004 MIT Media Laboratory, Hiroshi Ishii
Principle of Tangible Interface Design
inTouch 98 interpersonal communication
curlybot 00 mathematics and expression / narrative
topobo 04 building block with kinetic memory
Since 1996, we have been exploring several key design principles of Tangible Interface. One of these principle is the “coincidence of input and output space.” It is critical to blur the boundary of input/output, and representation/control to make information tangible. An initial example was inTouch designed for haptic interpersonal communication. Today, the curlybot will demonstrate a realization of this principle with special focus on mathematical education and narrative expression.
42
Constructive Assembly:
Triangles 99 /
Blocks
MERL © 2004 MIT Media Laboratory, Hiroshi Ishii
Physical/Digital Building Blocks
Blocks, Cubes, Triangles, …
Physical Digital Construction Kit
[Aish 79] [Anagnostou 89, Frazer 94]
[Anderson 00]
Since 1996, we have been exploring several key design principles of Tangible Interface. One of these principle is the “coincidence of input and output space.” It is critical to blur the boundary of input/output, and representation/control to make information tangible. An initial example was inTouch designed for haptic interpersonal communication. Today, the curlybot will demonstrate a realization of this principle with special focus on mathematical education and narrative expression.
43
© 2004 MIT Media Laboratory, Hiroshi Ishii
Triangles:Physical / Digital Construction Kit Gorbet, Orth and Ishii, 1998
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
• Physical embodiment of digital information topology
• Tactile feedback of digital connection from magnetic edge connectors
• Physically persistent representation
The Triangles allow users to feel as if they are actually holding and rearranging data itself, with the magnetic connectors providing tactile confirmation when connections are made or broken. Triangles is Not 'digitally enhanced' version of existing object. Triangles is the new language, digital and physical language.
Now let me introduce the Triangles, the final example of Tangible Bits project. The Triangles system is a physical/digital construction kit, which allows users to grasp and manipulate complex digital information with two hands. The system consists of a set of identical flat, plastic triangles, each with a microprocessor inside and magnetic edge connectors, which enable the Triangles to be physically connected to each other. The connectors pass electricity, allowing the Triangles to communicate digital information to each other and to a desktop computer. When the pieces contact one another, specific connection information is sent back to a computer that keeps track of the configuration of the system. The Triangles can be used to make two and three-dimensional objects whose exact configuration is known to the computer.
44
Triangles:
• digital information topology
• connection from magnetic edge connectors
• representation
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physical / Digital Construction Kit Gorbet, Orth and Ishii, 1998
Physical embodiment of
Tactile feedback of digital
Physically persistent
The Triangles allow users to feel as if they are actually holding and rearranging data itself, with the magnetic connectors providing tactile confirmation when connections are made or broken. Triangles is Not 'digitally enhanced' version of existing object. Triangles is the new language, digital and physical language.
Now let me introduce the Triangles, the final example of Tangible Bits project. The Triangles system is a physical/digital construction kit, which allows users to grasp and manipulate complex digital information with two hands. The system consists of a set of identical flat, plastic triangles, each with a microprocessor inside and magnetic edge connectors, which enable the Triangles to be physically connected to each other. The connectors pass electricity, allowing the Triangles to communicate digital information to each other and to a desktop computer. When the pieces contact one another, specific connection information is sent back to a computer that keeps track of the configuration of the system. The Triangles can be used to make two and three-dimensional objects whose exact configuration is known to the computer.
45
Painted Bits (GUI)
intangible representation (pixels on a screen)
information
output
digital
physical
pixels soundinput
remote control
intangible representation
© 2004 MIT Media Laboratory, Hiroshi Ishii
General input devices as remote-controllers of
46
Tangible Bits (TUI)
Tangible representation as interactive control mechanism to manipulate the information represented in both tangible and intangible forms
control tangible
representation
digital
physical
physical objects e.g. building model e.g. video projection
of digital shadow
information
intangible representation
© 2004 MIT Media Laboratory, Hiroshi Ishii
47
I/O Bulb and Luminous Room
• I/O Bulb –
• Luminous Room –
• Give life to architectural
• Enable direct manipulation of digital
© 2004 MIT Media Laboratory, Hiroshi Ishii
Underkoffler and Ishii, 1997 - 1999
High resolution output, two-way information
Multiple I/O bulbs illuminating architectural space
surfaces and physical objects.
world by grasping and manipulating objects with digital shadows.
48
I/O Bulb Video
• –
• distributed
• seep –
• Urp –
© 2004 MIT Media Laboratory, Hiroshi Ishii
Underkoffler and Ishii, 1997-99
illuminating light Holography
illuminating light
Fluid dynamics
Urban simulation
49
Sensetable Patten, Ishii, Pangaro, 2000
• TUI platform to track multiple objects and their
video projection
• simulation for Supply
© 2004 MIT Media Laboratory, Hiroshi Ishii
states on a table with
System Dynamics
Chain Analysis
50
Sensetable: TUI platform
• TUI platform to trackmultiple objects and their
projection
• Applications –
– Music “Audiopad”
–
© 2004 MIT Media Laboratory, Hiroshi Ishii
James Patten, Dan Chak & Hiroshi Ishii
states on a table with video
System Dynamics Simulation for Supply Chain Analysis (& Intel and Sloan School)
(James Patten & Ben Recht) Event Driven Simulation for IP Network Design (& NTT Comware)
51
© 2004 MIT Media Laboratory, Hiroshi Ishii
System Dynamics Simulation for Supply Chain Analysis Patten, Hines, Malone, Murphy-Hoye & Ishii 00-03
QuickTime™ and a decompressor
are needed to see this picture.
QuickTime™ and a decompressor
are needed to see this picture.
Collaboration with Intel and MIT Sloan School
52
System Dynamics Simulation for
and MIT Sloan School
QuickTime™ and a Sorenson Video 3 decompressor are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Supply Chain Analysis Patten, Hines, Malone, Murphy-Hoye & Ishii 00-03
Collaboration with Intel
53
IP Network Design Workbench
• Event-Driven Simulation +
design consulting expertise
• TUI supports cooperative direct manipulation of IP
© 2004 MIT Media Laboratory, Hiroshi Ishii
NTT Comware + TMG
NTT Comware’s network
Network simulator.
54
IP Network Design Workbench:
•
consulting expertise •
effects of changing
• TUI supports cooperative
QuickTime™ and a YUV420 codec decompressor
are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
NTT Comware + TMG (sensetable)
Based on Event-Driven Simulation Engine and NTT Comware’s NW
This workbench helps designers to evaluate the
topology, bandwidth, server location in real time, to optimize the network performance.
direct manipulation of IP Network simulator.
55
QuickTime™ and a YUV420 codec decompressor are needed to see this picture.
IP Network Design Workbench
• •
Thanks to Mr. Kase, Mr. Hirano, Mr. Narita, Ms.
© 2004 MIT Media Laboratory, Hiroshi Ishii
NTT Comware + TMG Event-Driven Simulation TUI supports cooperative direct manipulation of simulator to evaluate the effects of changing topology, bandwidth, server location in real time, to optimize the network performance.
Kobayashi, Mr. Tanaka, and many other NTT Comware people.
56
From Physical World Model to Computational Abstract Model
© 2004 MIT Media Laboratory, Hiroshi Ishii
Principle of Tangible Interface Design
Urp 99 System Dynamics Simulation 03
57
AudiopadJames Patten and Ben Recht (
• •
• j
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physics & Media)
A new way to perform electronic music. Designed to combine the expressive power of traditional musical instruments with the modularity of a computer Based on the Sensetable pro ect.
58
AudiopadJames Patten and Ben Recht (
• •
• j
QuickTime™ and a H.263 decompressor are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physics & Media)
A new way to perform electronic music. Designed to combine the expressive power of traditional musical instruments with the modularity of a computer Based on the Sensetable pro ect.
59
AudiopadJames Patten, Ben Recht (Physics & Media)
• •
modularity of a computer •
QuickTime™ and a H.263 decompressor
are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
A new way to perform electronic music. Designed to combine the expressive power of traditional musical instruments with the
Based on the Sensetable project.
60
AudiopadJames Patten and Ben Recht*
• •
• jQuickTimeý Dz
YUV420 ÉRÅ[ÉfÉbÉN êLí£ÉvÉçÉOÉâÉÄǙDZÇÃÉsÉNÉ`ÉÉǾå©ÇÈǞǽDžÇÕïKóvÇ-Ç�ÅB
© 2004 MIT Media Laboratory, Hiroshi Ishii
(*Physics & Media)
A new way to perform electronic music. Designed to combine the expressive power of traditional musical instruments with the modularity of a computer Based on the Sensetable pro ect.
61
CircuiTUI:
•
• Immediate feedback –
• qualitative data
• Classroom testing in MIT
© 2004 MIT Media Laboratory, Hiroshi Ishii
Real-time circuit simulation and analysis Dan Chak and Hiroshi Ishii
A middle ground between a laboratory and a text book
“Oscilloscope” voltage and current shown by each component
Displays all quantitative and
physics course this fall semester
62
• each component
• • Positive voltages are blue, negative are orange • • Circuit behavior can be understood immediately
© 2004 MIT Media Laboratory, Hiroshi Ishii
CircuiTUI Visualizations
“Oscilloscope” voltage and current shown by
Wire thickness varies with the amount of current
Color intensity analogous to voltage intensity
63
Sensetable James Patten & Hiroshi Ishii
• TUI platform to trackmultiple objects and their
video projection
• Applications – Music “Audiopad”
–
–
QuickTime™ and a Sorenson Video decompressorare needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
states on a table with
in collaboration with Ben Recht System Dynamics simulation for Supply Chain Analysis Chemistry
64
Concrete and Abstract
specific / concrete
Puck of
© 2004 MIT Media Laboratory, Hiroshi Ishii
Principle of Tangible Interface Design
generic / abstract
Phicon of metaDesk 97 Sensetable 00
Since 1996, we have been exploring several key design principles of Tangible Interface. One of these principle is the “coincidence of input and output space.” It is critical to blur the boundary of input/output, and representation/control to make information tangible. An initial example was inTouch designed for haptic interpersonal communication. Today, the curlybot will demonstrate a realization of this principle with special focus on mathematical education and narrative expression.
65
Actuated Workbench
•
•
QuickTime™ and a Photo - JPEG decompressor
are needed to see this picture.
–
–
© 2004 MIT Media Laboratory, Hiroshi Ishii
Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii
Augments Sensetable platform providing an additional physical dynamic display capability using a grid of electromagnets to move objects on surface of table in two dimensions. Pucks on table contain LC tag, permanent magnet, and toggle switch.
Applications Synchronization of distributed “Sensetables” in realtime remote collaboration Clearing up inconsistencies that arise from the computer's inability to move the objects on the table
66
Actuated Workbench
QuickTime™ and a H.263 decompressor
are needed to see this picture.
QuickTime™ and a H.263 decompressor
are needed to see this picture.
Function Application
© 2004 MIT Media Laboratory, Hiroshi Ishii
Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii 02
Magnetic forces to move objects on a table in two dimensions.
Augment existing “Sensetable” providing an additional physical dynamic display capability.
67
Actuated Workbench
•
using a grid of electromagnets to move objects on surface of table.
• Pucks on table contain LC tag, permanent
© 2004 MIT Media Laboratory, Hiroshi Ishii
Dan Maynes-Aminzade, Gian Pangaro & Hiroshi Ishii 02-03
Augments Sensetable platform providing an additional physical dynamic display capability
magnet, and toggle switch.
68
Actuated Workbench
Application 1 •
© 2004 MIT Media Laboratory, Hiroshi Ishii
Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii 02-03
Clearing up inconsistencies that arise from the computer's inability to move the objects on the table
with actuation without actuation
"In the video on the left, we see several people positioning cell phone towers on an overhead map. They can scroll the map by locking down one of the towers and moving it. However, notice how this creates an inconsistency between the physical and the digital states of the system. The user must manually adjust the positions of the other towers on the table to fix it. Users can also rotate and zoom the map, but this creates an inconsistency as well."
"By adding actuation to the system, we can automatically correct these inconsistencies. In the video on the right, we see the same scroll, rotate, and zoom operations, but the positions of the other pucks on the table are automatically updated using magnetic actuation."
69
Actuated Workbench
Application 2 •
remote collaboration
© 2004 MIT Media Laboratory, Hiroshi Ishii
Dan Maynes-Aminzade and Gian Pangaro & Hiroshi Ishii 02-03
Synchronization of distributed “Sensetables” in realtime
Actuation can also be used to synchronize two workbenches in different locations, facilitating remote collaboration. Notice how when a puck is moved on one table, it moves in the same way on the other table. This helps people to work together remotely using the Sensetable.
By placing a camera above the workbench, we can capture silhouettes of the users' hands. We then broadcast these hand silhouettes to the other workbench to show the activity of the remote user.
70
Senseboard Jacob, Ishii, Pangaro, Patten 2001
•
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
TUI platform to allow users to arrange small magnetic pucks bound to digital data and function on a grid of tag readers with video projection
Application: organizing and grouping information
71
Senseboard Jacob, Ishii, Pangaro, Patten
QuickTime™ and a decompressor
are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
2001 (video)
72
Illuminating ClayPiper, Ratti, and Ishii 2001
•
understanding of spatial relationships • Landscape Simulation
–
• 3D Laser Scanner + Video Projector
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physical Clay as 3-D Physical Input & Visual Display for intuitive manipulation and
land erosion, visibility, shadow casting, etc.
73
Illuminating Clay
QuickTime™ and a decompressor
are needed to see this picture.
•
and understanding of
• 3D Laser Scanner + Video Projector
© 2004 MIT Media Laboratory, Hiroshi Ishii
Ben Piper, Carlo Ratti & Hiroshi Ishii
Physical Clay as 3-D Physical Input & Visual Display for intuitive manipulation
spatial relationships
74
Systemvideo projector
glass beads
stone table
mode selection block
IR camera
A ceiling mounted IRcamera captures theradiance of the lightpassing through thesand model to determine
surface. The resulting landscape
back on to the surface.
© 2004 MIT Media Laboratory, Hiroshi Ishii
3D view
IR light sources
the geometry of the
analysis is projected
75
Physical Design Media � � Cardboard � Wooden Blocks �
© 2004 MIT Media Laboratory, Hiroshi Ishii
Clay
Found Objects
Frank O. Gehry, Architect
Physical Outcomes Stata Center 2002
76
and Digital Representation in Design
Clearer communication
Digital
Greater precision
© 2004 MIT Media Laboratory, Hiroshi Ishii
Lack of Continuity Between Physical
Physical
Ease of manipulation
Aids spatial understanding Easy distribution Quantitative analysis
How can we merge these media?
77
Form Giving + Computational Reflection: Refresh Rate of Iterative Design Cycle
Digital
Upper Stream
simultaneous form giving +
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physical
Lower Stream
Rough and rapid form giving Precise and quantitative with hand for ideation computational reflection
computational reflection
78
Representation of Idea Matters …
• • • •
© 2004 MIT Media Laboratory, Hiroshi Ishii
… because the mental operations are made possible by the representation. … GUI/CAD is not for ideation.
e.g. Mathematical representation Drawings Physical models Computational models
79
Media for Design Thinking
• Visual Thinking – sketch
• Tangible Thinking – tactile manipulation of
–
© 2004 MIT Media Laboratory, Hiroshi Ishii
physical representations coupled with digital computation design + analysis
80
PingPongPlus
• Digital augmentation of
"reactive table." • Ball tracking using
microphone array underneath table.
• From competition to collaboration
© 2004 MIT Media Laboratory, Hiroshi Ishii
Ishii, Lee, Wisneski, Orbanes 1999
ping pong play with
81
PingPongPlus
•
"reactive table." •
microphone array underneath table.
From competition to collaboration
QuickTime™ and a YUV420 codec decompressor are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Ishii, Lee, Wisneski, Orbanes 1999
Digital augmentation of ping pong play with
Ball tracking using
82
Augmentation of existing objects
© 2004 MIT Media Laboratory, Hiroshi Ishii
Principle of Tangible Interface Design
Another principle is the augmentation of existing objects to develop on top of existing understanding and skills. The bottles demonstrated in SIGGRAPH 99 emerging technologies is such an example. HandSCAPE to be presented by Jay Lee is another example to augment existing tape measure to bridge to the world of digital modeling.
Now I would like to introduce Phil Fre who is presenting and demonstrating curlybot.
83
musicBottles
Jazz Techno Classical Weather
• Glass bottles as "containers" and "controls" for digital information
• Seamless extension of metaphors and
© 2004 MIT Media Laboratory, Hiroshi Ishii
Ishii, Fletcher, Mazalek, Lee, Choo, Berzowska, Paradiso, 98-00
physical affordances into the digital domain
musicBottles proposes a new tangible interface using glass bottles as containers for digital information. Imagine an array of perfume bottles. Instead of scent, the bottles have been filled with music classical, jazz, and techno music. Opening each bottle releases the sound of a specific instrument.
84
Origin: Weather Bottle
• • Model: soy sauce bottle
in her kitchen © 2004 MIT Media Laboratory, Hiroshi Ishii
Present for my mother
The idea of a bottle interface originated from the concept of a “weather forecast bottle,” which I envisioned as a present for my mother.
Upon opening the weather bottle, she would be greeted by the sound of singing birds if the following day’s weather was forecasted to be clear. On the other hand, hearing the sound of rainfall from the bottle would indicate impending rain.
Such an interface would be consistent with the everyday interactions with her familiar physical environment, such as opening a bottle of soy sauce. She never clicked a mouse or typed a URL in her life, but opened soy sauce bottles thousands of times.
85
Origin: Weather Bottle
QuickTime™ and a YUV420 codec decompressor are needed to see this picture.
© 2004 MIT Media Laboratory, Hiroshi Ishii
86
Tangible Bits •
digital information and computation, making bits –
• and digital representation indesign
• Supporting multi-usercollaboration and “tangible thinking”
© 2004 MIT Media Laboratory, Hiroshi Ishii
Giving physical forms to
directly manipulable with two hands
Continuity between physical
Tangible interface is an interface design approach to give physical forms to digital information to make it tangible. It makes bits directly manipulable with our hands, and perceptible through our peripheral senses. The first path to the Tangible Interface is the directmanipulation with hands. Second path is the use of ambient media to make people aware of digital information at the periphery of perception. Pinwheels is such an example.The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context.
87
Painted Bits (GUI) and Tangible Bits (TUI)
•
• controllers”
Tangible User Interface •
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
Graphical User Interface Intangible representation (pixels on a screen) + Generic input devices as “remote
Tangible representation as interactive control mechanism to manipulate the information and computation Continuity between physical and digital representation in design
88
Tangible Bits
• Reconciliation of our dual citizenship in the worlds of bits and atoms.
• Interaction Design – – – shaped by industrial design, media arts and
© 2004 MIT Media Laboratory, Hiroshi Ishii
informed by sciences (HCI), materialized by technologies (CS, EE, ME), and
practical real-world applications.
89
Arts & Design Technologies
Tangible Interface Design
Tangible Bits Design Space
© 2004 MIT Media Laboratory, Hiroshi Ishii
Sciences &
concept, philosophy, scientific theory, implementation, aesthetics evaluation, analysis
Interaction Design Computer Sci. Cog. Sci. Industrial / Product Design Mechanical Eng. Soc. Sci. Architectural / Env. Design Electronic Eng. Media Arts / Interactive Arts
90
Design?
Conceptual “eyeglasses” –
© 2004 MIT Media Laboratory, Hiroshi Ishii
Inspire people’s imagination and creativity
91
“The Computer for the 21st Century”
“The most profound technologies are thosethat disappear.
Mark Weiser
© 2004 MIT Media Laboratory, Hiroshi Ishii
They weave themselves into the fabric of everyday life until they are indistinguishable from it.”
July 23, 1952 - April 27, 1999
Mark Weiser wrote as follows in the 1st paragraph of his landmark paper on Ubiquitous Computing published in 1991 Scientific America . “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.” If you learn the task quite well using the right tools, and become the master, the tools cease to be a center of users’ attention. �Tools disappear into the background and you can concentrate on the tasks themselves. This is the definition of transparent or invisible computers by Mark. Invisible does not mean you can not see with your eyes. It is the matter of user’s focus of attention and consciousness. It is the state in which tools do not get in your way and you can concentrate on the task. Mark showed a series of ubiquitous computers including Live Board, Pads, and Badges. All of them come with screens and GUIs. But I do not feel Ubiquitous GUI is the path to achieve invisible interface. Today, I would like to show my approach: Tangible Interfaces.
92
Exhibitions
Linz, Austria
“Get in Touch”
Centre Pompidou Paris, France,
PingPongPlus,
© 2004 MIT Media Laboratory, Hiroshi Ishii
Ars Electronica Center
musicBottles SandScape Audiopad
11/19/03-1/6/04
Le Design Interactif
93
Acknowledgments • Tangible Media Group
Graduate students and UROPs • • •
© 2004 MIT Media Laboratory, Hiroshi Ishii
Things That Think, Digital Life @ MIT Media Lab Friends in ACM SIGCHI, SIGGRAPH, IDSA, ICSID Ars Electronica Center, NTT-InterCommunication Center
We thank Bill Buxton and George Fitzmaurice at the University of Toronto for countless discussions about graspable UI, skill-based design, and foreground & background issues, through which many of the ideas in this paper were developed and shaped. Thanks are also due to Bill Verplank at Interval Research for his insightful comments and discussions on haptic interfaces and tangible media. We appreciate Mark Weiser for his inspiring Ubiquitous Computing work and the introduction of Live Wire to me. We thank TTT (Things That Think), a new consortium at the MIT Media Lab, for its ongoing support of the Tangible Bits project. We also would like to acknowledge the contribution of many hardworking graduate and undergraduate students at MIT for work on the implementation of the Tangible Bits platforms. In particular, we thank graduate students Scott Brave, Andrew Dahley, Matt Gorbet, and many other students for their work on the metaDESK and ambientROOM prototypes. Thanks are finally due to Steelcase, Inc. for their donation of Personal Harbor which became a plat form of the ambientROOM.
94
Thanks!
Hiroshi Ishii Tangible Media Group
/
© 2004 MIT Media Laboratory, Hiroshi Ishii
MIT Media Laboratory http://tangible.media.mit.edu
95
Thanks!
Hiroshi Ishii Tangible Media Group
/
© 2004 MIT Media Laboratory, Hiroshi Ishii
MIT Media Laboratory http://tangible.media.mit.edu
Thanks for your kind attention!
96
© 2004 MIT Media Laboratory, Hiroshi Ishii
At the Boundary Where the sea meets the land, life has blossomed into a myriad of unique forms in the turbulence of water, sand, and wind.
At another seashore between the land of atoms and the sea of bits, we are now facing the challenge of reconciling our dual citizenships in the physical and digital worlds. Windows to the digital world are confined to flat square screens and pixels, or "painted bits." Unfortunately, one can not feel and confirm the virtual existence of this digital information through one's body.
Tangible Bits seeks to realize seamless interfaces between humans, digital information, and the physical environment by giving physical form to digital information and computation, making bits directly manipulable and perceptible.
Hiroshi Ishii | Tangible Media Group | MIT Media Lab
97
Ubiquitous Computing Mark Weiser, Xerox PARC, 1991 • Computers should be "transparent." • Computational services are delivered through a
Tabs, Pads, and Boards
© 2004 MIT Media Laboratory, Hiroshi Ishii
variety of computational devices such as , with the infrastructure to allow
these devices to talk with each other.
Mark Weiser at Xerox PARC presented his vision of “Ubiquitous Computing” in 1991. He claimed computers should be transparent and invisible.
He claimed that the most profound technologies are invisible. They weave themselves into the fabric of everydaylife until they are indistinguishable from it.His team designed a variety of computational devices including Tabs, Pads, and Boards along with theinfrastructure to allow these devices to talk with each other.
Recently, he is introducing a concept of “calm technology” citing the LIVE WIRE by Natalie Jeremijenko as an example.
98
Research?
• – Local optimization –
problems – Reflection-driven
• Secure home – Orthodox – Established discipline –
© 2004 MIT Media Laboratory, Hiroshi Ishii
Increment
Solve known
Homogeneity
99
Research?
Yes • Innovation
– Mutation –
problems – Invention-driven
• No home – – – Diversity
No •
– Local optimization –
problems – Reflection-driven
• Secure home – Orthodox – Established discipline –
© 2004 MIT Media Laboratory, Hiroshi Ishii
Identify new
Heterodox - heretic Inter-disciplinary
Increment
Solve known
Homogeneity
100
Design
• Highly interdisciplinary – –
© 2004 MIT Media Laboratory, Hiroshi Ishii
Enjoy identity crisis. Appreciate diversity.
101
Design
• Highly interdisciplinary – –
• New vision, no roadmap – Invent new paradigms, rather than do
© 2004 MIT Media Laboratory, Hiroshi Ishii
Enjoy identity crisis. Appreciate diversity.
incremental work, because life is short.
102
Design
• Highly interdisciplinary – –
• New vision, no roadmap – Invent new paradigms, rather than
doing incremental work. • Success is a danger
– © 2004 MIT Media Laboratory, Hiroshi Ishii
Enjoy identity crisis. Appreciate diversity.
Do not stop. Keep redefining yourself.
103
To the Researchers •
–
• – –
• Ground to the earth –
• Success is a danger –
© 2004 MIT Media Laboratory, Hiroshi Ishii
New vision, no roadmap Invent new paradigms, rather than doing incremental work, because life is short.
Highly interdisciplinary Enjoy identity crisis. Appreciate diversity.
Learn through real world problem solving
Do not stop. Keep redefining yourself.
104
Beyond Painted Bits, Toward Tangible Bits
•
• representation and interfaces for
GUI TUI
© 2004 MIT Media Laboratory, Hiroshi Ishii
Beyond GUI (Graphical User Interface) Toward TUI (Tangible User Interface) Physical objects, surfaces, and spaces as
computationally mediated information.
Taking advantage of the multi-modality of human interactions with the physical world, we are trying to go beyond “painted bits”, and establish a new paradigm of “Tangible Bits.” Coupling bits with physical objects and phenomena is the key approach, we are trying to make the interfaces ubiquitous and transparent. Ultimately, we believe the architectural space itself should be an interface between people and digital information space
105
Thanks!
Hiroshi Ishii Tangible Media Group
http://tangible.media.mit.edu/
© 2004 MIT Media Laboratory, Hiroshi Ishii
MIT Media Laboratory
106
Thanks!
Hiroshi Ishii Tangible Media Group
© 2004 MIT Media Laboratory, Hiroshi Ishii
MIT Media Laboratory http://tangible.media.mit.edu/
107
Tangible Interface Design (1)
Coincidence of input and output spaces
interpersonal communication
computation / mathematics
© 2004 MIT Media Laboratory, Hiroshi Ishii
Principles of
inTouch 98 curlybot 00
and expression / narrative
Since 1996, we have been exploring several key design principles of Tangible Interface. One of these principle is the “coincidence of input and output space.” It is critical to blur the boundary of input/output, and representation/control to make information tangible. An initial example was inTouch designed for haptic interpersonal communication. Today, the curlybot will demonstrate a realization of this principle with special focus on mathematical education and narrative expression.
108
Tangible Interface Design (2)
Augmentation of existing objects
Bottles 99
containers © 2004 MIT Media Laboratory, Hiroshi Ishii
Principles of
Another principle is the augmentation of existing objects to develop on top of existing understanding and skills. The bottles demonstrated in SIGGRAPH 99 emerging technologies is such an example. HandSCAPE to be presented by Jay Lee is another example to augment existing tape measure to bridge to the world of digital modeling.
Now I would like to introduce Phil Fre who is presenting and demonstrating curlybot.
109
Oullim • Aufheben (• People, Bits, and Atoms
– • Arts, Design, Sciences,
and Technologies –
© 2004 MIT Media Laboratory, Hiroshi Ishii
German)
Body, Cyberspace, and Physical Space
Analysis and Synthesis
110
Real + Virtual, but mainly Visual •
• –
synthetic environment. • Augmented Reality
– ld,
| digital
© 2004 MIT Media Laboratory, Hiroshi Ishii
Personal Computing
Virtual Reality completely immerses a user inside a
allows the user to see the real worwith virtual objects superimposed upon the real world. supplements reality, rather than completely replacing it.
physical
111
Augmented Reality Def. by Ron Azuma Augmented Reality
l
1. 2. 3.
•
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
allows the user to see the real wor d, with virtual objects superimposed upon or composited with the real world. supplements reality, rather than completely replacing it.
Blends real and virtual, in real environment Real-time interactive Registered in 3-D
Azuma, A Survey of Augmented Reality. Presence, 6 (4), p.355-385, Aug. 1997 SIGGRAPH 2001 Course Notes 27, Augmented Reality: The Interface is Everywhere
112
Augmented RealityComputer-Augmented Environments
• media?
• by visual overlay
•
– – –
© 2004 MIT Media Laboratory, Hiroshi Ishii
How to integrate “real world” and computational
Integrating “real world” and computational media of digital information onto real
world image with see-through display, hand-held display, or direct video projection Redefining “Human Interactions” with computational (virtual) artifacts within the social and physical settings of their use
From Desktop Computing to Ubiquitous Computing From Desktop UI to Ubiquitous UI From Virtual Reality to Augmented & Mixed Reality
Augmented Reality is a new research stream which tries to integrate the real world and computational media.
There are many important work including Myron Krueger’s pioneering work of VIDEOPLACE in 1985.DigitalDesk by Wellner is one of early work which demonstrated a way to merge physical and digital document by using video projection ofcomputer display onto a real desk with physical documents.
The most common AR approach is the visual overlay of digital information onto real-world imagery with see through head-mounted display, hand-held display, or direct video projection.
113
Eyes are in charge, but hands are underemployed.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Eyes are fully employed, but hands are underusedHands are underemployed. GUIs (Graphical User Interfaces) are designed for human eyes, not for hands. Telepresence, or interpersonal communication, is not the exception. Communication media such as telephones, video conferencing, and e-mail are for human ears and/or eyes, but not for hands. McCullough articulated the importance of hands as follows, “By pointing, by pushing and pulling, by picking up tools, hands act as conduits through which we extend our will to the world. They serve also as conduits in the other direction: hands bring us knowledge of the world. Hands feel. They probe. They practice.” [McCullough, 1996 #1].
114
Bottles as Interface
© 2004 MIT Media Laboratory, Hiroshi Ishii
The core concept utilizes glass bottles as "containers" and "controls" for digital information. Physical manipulation of the bottles - opening and closing is the primary mode of interaction with digital information.
musicBottles proposes a new tangible interface using glass bottles as containers for digital information. Imagine an array of perfume bottles. Instead of scent, the bottles have been filled with music - classical, jazz, and techno music. Opening each bottle releases the sound of a specific instrument.
115
Sensetable & Senseboard as Generic TUI Platforms
TUI Applications
Urban Planning others
TUI Application Program Interface
© 2004 MIT Media Laboratory, Hiroshi Ishii
Supply Chain Vis.
Sensetable / Senseboard Hardware
116
------
Tangible User Interfaces
• information, making bits
– our hands
– perceptible through our peripheral senses
•
© 2004 MIT Media Laboratory, Hiroshi Ishii
Giving physical forms to digital
directly manipulable with
Seamless coupling between digital and physical worlds.
Tangible interface is an interface design approach to give physical forms to digital information to make it tangible. It makes bits directly manipulable with our hands, and perceptible through our peripheral senses. The first path to the Tangible Interface is the directmanipulation with hands. Second path is the use of ambient media to make people aware of digital information at the periphery of perception. Pinwheels is such an example.The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context.
Tangible interface is our challenge to design seamless interface between the land of atoms and the sea of bits for the people.
117
People have developed sophisticated skills for sensing and manipulating their physical environments. However, most of these skills are not employed by the traditional Graphical User Interface. GUIs (Graphical User Interfaces) are designed for human eyes, not for hands. GUIs are in the boxes detached from our physical environment and social context. Tangible interface is an interface design approach to give physical forms to digital information to make it tangible. It makes bits directly manipulable with our hands, and perceptible through our peripheral senses. The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context. The Tangible Interface seeks to build upon the skills of manipulating the physical world, by giving physical form to digital information and situating it in the architectural spaces. The Tangible Interface explores the way to weave interfaces with the digital world into the fabric of a physical and social context.
TouchCounters: Containers
© 2004 MIT Media Laboratory, Hiroshi Ishii
Interactive Electronic Labels for Physical (Yarin and Ishii 99)
“Distributed visualization of usage history”: Physical objects and surfaces that display their history of use
118
mediaBlocks: Ullmer, Ishii, and Glas, 1998
• Small, electronically tagged
• containment, transport, and manipulation of online media.
•
• collections of media elements.
© 2004 MIT Media Laboratory, Hiroshi Ishii
Physical Containers, Transports, and Controls for Online Media
wooden blocks Physical icons (phicons) for the
Rapid “copy” and “paste” from a media source into a media display. Racks allow navigation and sequencing of
119
Design Evolution
Paradigm Principles
Idea Hunch Hack
Product market
GUI, Ubiquitous Comp. WYSIWIS (GUI) /
NLS, Alto, Collab
Lisa, Mac, Navigator
PC, Web
© 2004 MIT Media Laboratory, Hiroshi Ishii
Bill Verplank 1998
Prototype
WYSIWIS (CSCW)
120
© 2004 MIT Media Laboratory, Hiroshi Ishii
Tangible Bits Designing the Seamless Interface between People, Bits, and Atoms
Conference Month Day, 2002
Hiroshi Ishii Tangible Media Group MIT Media Laboratory
121
Augmented RealityComputer-Augmented Environments
• media by of digitalinformation onto real world image with see-
video projection • Examples:
– – KARMA (Steven Feiner) – Chameleon (George Fitzmaurice) – Wearable Computing (Thad Starner)
© 2004 MIT Media Laboratory, Hiroshi Ishii
Integrating “real world” and computational visual overlay
through display, hand-held display, or direct
DigitalDesk (Pierre Wellner)
Augmented Reality is a new research stream which tries to integrate the real world and computational media.
There are many important work including Myron Krueger’s pioneering work of VIDEOPLACE in 1985.DigitalDesk by Wellner is one of early work which demonstrated a way to merge physical and digital document byusing video projection of computer display onto a real desk with physical documents.
The most common AR approach is the visual overlay of digital information onto real-world imagery with see through head-mounted display, hand-held display, or direct video projection.
122
Graspable User InterfaceG. Fitzmaurice, H. Ishii, & W. Buxton, 1995
• Direct control of virtual objects through
• Blend physical and virtual affordances •
© 2004 MIT Media Laboratory, Hiroshi Ishii
physical handles called “bricks”
Distribute cognitive load more evenly from visual to tactile system
This work of Graspable User Interface was done in collaboration with G. Fitzmaurice and Bill Buxton during my one year stay at the University of Toronto in 1994.The Graspable User Interface provide direct control of virtual objects throughphysical handles called “Bricks.”With Bricks, users can take advantage of our everyday skills in manipulating physical objects.
123