Upload
ralf-richards
View
220
Download
2
Embed Size (px)
Citation preview
1
Telepresence: Telepresence: An Umbrella Research TopicAn Umbrella Research Topic
Jim GrayMicrosoft [email protected]://research.Microsoft.com/~Gray/
2
Federal Research Support Federal Research Support Nerve Center of ScienceNerve Center of ScienceIf it’s not broke, don’t fix it.If it’s not broke, don’t fix it.But….But….
US Science is the engine of progressBUT…..
Best and brightest are spending
increasing time fund raising Seems excessive to me. Venture capital community is
richer and more generous
than Federal Support
THE
LONG
BOOM
3
Cyberspace is a Cyberspace is a New WorldNew World.. We have discovered a “new continent”. It is changing how we learn, work, and play.
– 1 T$/y industry– 1 T$ new wealth since 1993– 30% of US economic growth since 1993
There is a gold rush to stake out territory. But we also need explorers:
Lewis & Clark expeditions Universities to teach the next generation(s)
Governments, industry, and philanthropists should fund long-term research.
THELONG
BOOM
4
19701960 1980 1990
WorkstationsLisp machine, StanfordXerox Alto
Apollo, Sun
NetworkingArpanet, InternetEthernet, Pup, Datakit
DECnet, LANs, TCP/IP
GraphicsSketchpad, UtahGM/IBM, LucasFilm
E&S, SGI, PIXAR,..
WindowsEnglebart, RochesterAlto, Smalltalk
Star, Mac, Microsoft
Research Investments Pay OffResearch Investments Pay OffCSTB –NRC Evolving the High-Performance Computing and Communications Imitative to Support the nations Information Infrastructure, NA Press, CSTB –NRC Evolving the High-Performance Computing and Communications Imitative to Support the nations Information Infrastructure, NA Press, Washington DC, 1995.Washington DC, 1995.
Time-sharing CTSS, Multics, SSDUnix
SDS 940, 360/67 VMS
Government fundedIndustrial
Billion Dollar/year Industry
5
1970 1980 1990 2000
Relational Data BasesBerkeley, Wisc,… IBM
Oracle, IBM,…
Parallel DBs Tokyo,Wisconsin, UCLAICL, IBM
ICL, Teradata, Tandem
Research Investments Pay OffResearch Investments Pay Off
Data Mining (complex queries)
Wisc, Stanford, …IBM, Arbor,…
IRI, Arbor, Plato, …
6
Why Can’t Industry Fund IT Research?Why Can’t Industry Fund IT Research? It does: IBM (5.8%), Intel(13%), Lucent (12%), Microsoft(14.%), Sun (12%), ...
– R&D is ~5%-15% (50 B$ of 500 B$) AD is 10% of that (5 B$)
– Long-Range Research is 10% of that 500 M$2,500 researchers and university support
– Compaq: 4.8% R&D (1.3 B$ of 27.3 B$).AOL: 3.7% D, ?R (96 M$ of 2.6 B$) – Dell:1.6% R&D (204 M$ of 12.6 B$), EDS, MCI-WorldCom, ….
To be competitive, some companies cannot make large long-term research investments.The Xerox/PARC story:
created Mac, Adobe, 3Com…
7
PITAC ReportPITAC ReportPresidential IT Advisory CommitteePresidential IT Advisory Committeehttp://www.ccic.gov/ac/report/http://www.ccic.gov/ac/report/ Findings:
– Software construction is a mess: needs breakthroughs.– We do not know how to scale the Internet 100x
Security, manageability, services, terabit per second issues.
– USG needs high-performance computing (Simulation) but market is not providing vector-supers – just providing processor arrays.
– Trained people are in very short supply. Recommendations:
– Lewis & Clark expeditions to 21st century.– Increase long-term research funding by 1.4B$/y.– Re-invigorate university research & teaching.– Facilitate immigration of technical experts.
8
Outline (ambitious!)Outline (ambitious!) Microsoft Research (census) Tele-Presentations (Gordon Bell, Jim Gemmell) Microsoft Research initiative on Telepresence What if you could record everything you see & hear? The architecture revolution:
processing moves to transducers
9
Microsoft Research -- 1991Microsoft Research -- 1991 Founded in 1991 Goal:
pursue strategic technologies for Microsoft
Original research groups:– Natural Language Processing– Operating Systems– Programming Languages
Overall size < 20 at the end of 1992
10
Microsoft Research -- 1999Microsoft Research -- 1999 400 Researchers in 25 areas
– Operating systems to Statistical Physics Research lab locations:
– Redmond, Cambridge, San Francisco, Beijing Internationally recognized research teams
– Hundreds of publications, presentations– Leadership roles in professional societies,
journals, conferences
11
MS Research AreasMS Research Areas Operating systems, languages, compilers,
virtual machines, networking, wireless computing, fault-tolerance, large scale servers, security
Natural language, speech, vision, graphics, decision theory, information retrieval, UI, collaboration, statistics, signal processing
Cryptography, statistical physics and discrete mathematics
12
Growing FastGrowing Fast
Grew 20x from ‘92 to ‘99 Decided in ‘97 to grow by a 3x in 3 years
– 200 in FY97 => 600 in FY00, primarily in Redmond
Major impact on MS products– Virtually all MS products shipped today use
technology from MS Research Key role in MS growth
– Pioneering research in software that allows computers to see, hear, speak and understand
13
Microsoft Research Microsoft Research PhilosophyPhilosophy
University organizational model– Flat structure, critical mass groups
Open research environment– Aggressive publication of research results
in literature and on world wide web
– Frequent visitors, daily seminars
– Over 100 visiting professors and interns in 1998
– Over 110 visiting researchers in 1998
17
What I Do.What I Do.
Work for the government!– CSTB, PITAC(software, ngi), LoC study, ....
Work on scaleable systems:– 1 Billion Transactions Per Day Cluster
– TerraServer
– New: Sloan Digital Sky Survey
18
Outline (ambitious!)Outline (ambitious!) Microsoft Research (census) Tele-Presentations (Gordon Bell, Jim Gemmell) Microsoft Research initiative on Telepresence What if you could record everything you see & hear? The architecture revolution:
processing moves to transducers
19
Gordon Bell on Gordon Bell on Tele PresentationsTele Presentations
http://research.microsoft.com/barc/GBell/http://research.microsoft.com/barc/GBell/
20
Motivation:Motivation:TelepresentationsTelepresentations
• Presenter and/or audience telepresent
NOT: meeting or collaboration settings
Forget the nasty social issues!
Mostly one-way
21
TelepresentationTelepresentationElementsElements Slides Audio Video Script,
text comments, hyperlinks,etc.
22
Telepresentations:Telepresentations:The EssentialsThe Essentials
Slide and audio a must Add some video
(low quality) to make us feel good
Storage and transmission costs low
23
Telepresentations:Telepresentations:The Killer AppThe Killer App
Increased attendance & lower travel costs
Practical and low-cost NOW e.g. ACM97 - 2,000 visitors in real
space, 20,000 visitors on Internethttp://research.microsoft.com/acm97
24
Today’sToday’sExperimentExperiment
Would you like to pause, rewind, browse? Do you wish you could have seen this
– At home?– At another time?
How much does a present speaker add? How much would you pay for real presence?
25
University Lectures OnlineUniversity Lectures Online Research lectures on-line & on-demand http://murl.microsoft.com/ Will get UVC content Available to anyone anywhere
– T1 good, 28.8 OK Generated by CMU, MIT, MSR, Stanford, UW, Xerox Hosted by MSR
26
Outline (ambitious!)Outline (ambitious!) Microsoft Research (census) Tele-Presentations (Gordon Bell, Jim Gemmell) Microsoft Research initiative on Telepresence What if you could record everything you see &
hear? The architecture revolution:
processing moves to transducers
27
Changing role of computationChanging role of computation Past: Computers for:
– computing (Cray)– business data processing (IBM)– “document” creation (PC)
Future: Computers for: – understanding & learning– communicating– consuming & entertaining
Requires new User Interface to machines
FlowsFlows
29
Making “Flows” a RealityMaking “Flows” a Reality Computer Graphics
– Creating realistic looking environments, people
Computer Vision– Analyzing posture, gaze, gestures
Speech input/output Natural Language
– Analysis, IR Implicit requests for information
Building life-like human Building life-like human characterscharacters
31
How to fail at Tele-ConferencesHow to fail at Tele-Conferences
1. Eliminate gaze awareness and sense of space of a normal group setting
2. Have long audio latencies & poor audio quality
3. Use incompatible equipment4. Make it much harder to initiate the call
to make a phone call
32
Gaze Awareness & Gaze Awareness & Sense of SpaceSense of Space
Is anyone paying attention?
Who is talking (where is sound coming from?
33
Gaze AwarenessGaze Awareness
Looking at screen:the forehead shot
Looking at camera:the glowering shot
Looking at YOU.
34
You can’t just move the eyesYou can’t just move the eyes
Glowering
Surprise
Boredom
Interest
35
Mona Lisa EffectMona Lisa Effect Eyes and nose indicated gaze
36
Spatialized Audio & VideoSpatialized Audio & VideoPointing “nose vector” at targetPointing “nose vector” at target Map video onto
wire frame Rotate frame to
point in space Move (fake) eyes
in frame (>30°)to point at target
Project voice on that vector.
Recognizing gesturesRecognizing gestures
Live videoLive videoArea of Area of motionmotion H flowH flow V flowV flow
38
Generating life-like speech Generating life-like speech from textual datafrom textual data Data-driven stochastic speech
– Natural sounding– Rapid, automatic customizability
Examples– Synthetic voice w/ transplanted speech
contours
39
AT&T Voder, 1962, by Homer Dudley– Daisy (Inspiration for HAL’s voice in 2001)
Microsoft Research Whistler, 1997– Scarborough Fair
Artificial singingArtificial singing
40
Analyzing languageAnalyzing language Language recognition shipped in Word 97 General purpose text-critiquing,
summarization, Japanese word-breaking
41
Inside The Office Inside The Office Grammar CheckerGrammar Checker
42
Understanding language: Understanding language: MindNetMindNet
A huge language knowledge base
Automatically created from dictionaries
Words (nodes) linked by relationships
Millions of links Recently added
(Encarta) encyclopedia knowledge
Is_amouthLocn_of
MindNet -- “Going to the birds”MindNet -- “Going to the birds”
face
peck limbgoose
creaturemake
Is_a
Typ_obj sound
Typ_subj
preen
Is_a
Part feather
Not_is_a plant
Is_agaggle Is_a
Is_a
PartPart_of
catch
Typ_subj_ofTyp_obj
claw
wing
Is_a
turtle
beak
Is_a
Is_a
strike
Means
hawk
Typ_subj
opening
Is_a
chatter
Means
Typ_subjTyp_obj
Is_a
Is_a
clean
smooth
billIs_a
duck
Is_a
Typ_0bj_of keep
animal
quack
Is_a
CausePurpose
bird
meat
egg
poultry
Is_a
supplyPurpose Typ_obj
Quesp
hen
chicken
Is_a
Is_a
leg
arm
Is_a
Is_a
Is_a
Typ_subj_of
fly
44
Changing balance between Changing balance between user & software systemsuser & software systems
Yesterday:– Applications were single programs running in
isolation– Users used to (more or less) understand systems
that they used Today:
– Componentized applications operate in concert – Sophisticated users understand only small
percentage of systems they use
45
Tomorrow’s Systems and Tomorrow’s Systems and ApplicationsApplications Users will not be able to predict
– where computations will be performed, – when they will be performed or – by what software components
Gap between system capabilities and user understanding will grow to the point that the only way user will be able to use system is through assisting agents
46
Examples of user agents & Examples of user agents & implicit actionsimplicit actions Lumiere (Office 97)
– Monitoring user and program events to provide user help and assistance
Implicit queries– Inferring information needs from browsing
Lookout/SpamKiller– Monitoring mail activity to auto-categorize it
User ModelingUser Modeling Models of a user’s informational goals
– User’s query (when available…)– User’s background– Acute and long-term search activity– Acute actions with objects and documents– Program data structures
Explicit and implicit information access and display
48
Outline (ambitious!)Outline (ambitious!) Microsoft Research (census) Tele-Presentations (Gordon Bell, Jim Gemmell) Microsoft Research initiative on Telepresence What if you could record everything you see &
hear? The architecture revolution:
processing moves to transducers
51
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
A novel A letter
Library of Library of Congress Congress (text)(text)
All Disks
All Tapes
A Movie
LoC (image)
All Photos
LoC (sound + cinima)
All Information!
52
Alan Newell’s & Michael Lesk’s Alan Newell’s & Michael Lesk’s PointsPoints
www.lesk.com/mlesk/ksg97/ksg.htmlwww.lesk.com/mlesk/ksg97/ksg.html Soon everything can be recorded and kept
Most data will never be seen by humans
Precious Resource: Human attention Auto-SummarizationAuto-Search
will be a key enabling technology.
53
Outline (ambitious!)Outline (ambitious!) Microsoft Research (census) Tele-Presentations (Gordon Bell, Jim Gemmell) Microsoft Research initiative on Telepresence What if you could record everything you see &
hear? The architecture revolution:
processing moves to transducers
54
Put Everything Put Everything in Future (Disk) Controllersin Future (Disk) Controllers(it’s not “if”, it’s “when?”)(it’s not “if”, it’s “when?”)
AcknowledgementsAcknowledgements::
Dave PattersonDave Patterson explained this to me a year ago explained this to me a year ago
Kim KeetonKim Keeton
Erik RiedelErik Riedel
Catharine Van IngenCatharine Van Ingen
Helped me sharpen these arguments
55
Remember Your RootsRemember Your Roots
56
Technology Drivers: DisksTechnology Drivers: Disks Disks on track 100x in 10 years
2 TB 3.5” drive Shrink to 1” is 200GB Disk replaces tape?
Disk is super computer!
Kilo
Mega
Giga
Tera
Peta
Exa
Zetta
Yotta
57
Data GravityData Gravity Processing Moves to TransducersProcessing Moves to Transducers
Move Processing to data sources Move to where the power (and sheet metal) is Processor in
– Modem– Display– Microphones (speech recognition)
& cameras (vision)– Storage: Data storage and analysis
58
It’s Already True of PrintersIt’s Already True of PrintersPeripheral = CyberBrickPeripheral = CyberBrick You buy a printer You get a
– several network interfaces– A Postscript engine
cpu, memory, software, a spooler (soon)
– and… a print engine.
60
Basic Argument for x-DisksBasic Argument for x-Disks Future disk controller is a super-computer.
– 1 bips processor– 128 MB dram– 100 GB disk plus one arm
Connects to SAN via high-level protocols– RPC, HTTP, DCOM, Kerberos, Directory Services,…. – Commands are RPCs– Management, security,….– Services file/web/db/… requests– Managed by general-purpose OS with good dev
environment Apps in disk saves data movement
– need programming environment in controller
61
The Slippery SlopeThe Slippery Slope
If you add function to server Then you
add more function to server Function gravitates to
data.
Nothing = Sector Server
Everything = App Server
Something =
Fixed App Server
66
Why Not Everything?Why Not Everything?
AllowAllow Everything on Disk Server Everything on Disk Server(thin client’s)(thin client’s)
Tried and true design– Mainframes, Minis, ...– Web servers,…– Encapsulates data– Minimizes data moves– Scaleable
It is where everyone ends up. All the arguments against are short-term.
68
Disk = NodeDisk = Node has magnetic storage (100 GB?) has processor & DRAM has SAN attachment has execution
environment
OS KernelSAN driver Disk driver
File System RPC, ...Services DBMS
Applications
69
Technology Drivers:Technology Drivers: System on a ChipSystem on a Chip
Integrate Processing with memory on one chip– chip is 75% memory now– 1MB cache >> 1960 supercomputers– 256 Mb memory chip is 32 MB!– IRAM, CRAM, PIM,… projects abound
Integrate Networking with processing on one chip– system bus is a kind of network– ATM, FiberChannel, Ethernet,.. Logic on chip.– Direct IO (no intermediate bus)
Functionally specialized cards shrink to a chip.
70
Technology Drivers: What if Technology Drivers: What if Networking Was as Cheap As Disk IO?Networking Was as Cheap As Disk IO? TCP/IP
– Unix/NT 100% cpu @ 40MBps
Disk– Unix/NT
8% cpu @ 40MBps
Why the Difference?Host Bus Adapter does
SCSI packetizing, checksum,…flow controlDMA
Host doesTCP/IP packetizing, checksum,…flow controlsmall buffers
71
Technology Drivers: Technology Drivers: The Promise of SAN/VIAThe Promise of SAN/VIA:10x in 2 years:10x in 2 years http://www.ViArch.org/http://www.ViArch.org/
Today: – wires are 10 MBps (100 Mbps Ethernet)
– ~20 MBps tcp/ip saturates 2 cpus– round-trip latency is ~300 us
In the lab– Wires are 10x faster Myrinet, Gbps Ethernet,
ServerNet,…
– Fast user-level communication tcp/ip ~ 100 MBps 10% of each processor round-trip latency is 15 us
72
Gbps Ethernet: 110 MBps
SAN: SAN: Standard InterconnectStandard Interconnect
PCI: 70 MBps
UW Scsi: 40 MBps
FW scsi: 20 MBps
scsi: 5 MBps
LAN faster than memory bus?
1 GBps links in lab.
100$ port cost soon
Port is computer
RIPFDDI
RIPATM
RIPSCI
RIPSCSI
RIPFC
RIP?
75
Technology DriversTechnology Drivers
Plug & Play SoftwarePlug & Play Software RPC is standardizing: (DCOM, IIOP, HTTP)
– Gives huge TOOL LEVERAGE– Solves the hard problems for you:
naming, security, directory service, operations,...
Commoditized programming environments – FreeBSD, Linix, Solaris,…+ tools– NetWare + tools– WinCE, WinNT,…+ tools– JavaOS + tools
Apps gravitate to data. General purpose OS on controller runs apps.
76
Basic Argument for x-DisksBasic Argument for x-Disks Future disk controller is a super-computer.
– 1 bips processor– 128 MB dram– 100 GB disk plus one arm
Connects to SAN via high-level protocols– RPC, HTTP, DCOM, Kerberos, Directory Services,…. – Commands are RPCs– management, security,….– Services file/web/db/… requests– Managed by general-purpose OS with good dev
environment Move apps to disk to save data movement
– need programming environment in controller
77
SummarySummary Microsoft Research (census) Tele-Presentations (Gordon Bell, Jim Gemmell) Microsoft Research initiative on Telepresence What if you could record everything you see &
hear? The architecture revolution:
processing moves to transducers