11

MyLifeBits: Attempting to realize the Memex Vision

Gordon BellGordon Bell

February 2003February 2003http://research.microsoft.com/barc/MediaPresence/MyLifeBits.aspxhttp://research.microsoft.com/barc/MediaPresence/MyLifeBits.aspx

With Jim Gemmell & Roger LuederWith Jim Gemmell & Roger Lueder

22

Outline … MyLifeBits

Background…fulfilling the Memex visionBackground…fulfilling the Memex visionCyberizing everythingCyberizing everythingFile to database transitionFile to database transitionUse…beyond searchUse…beyond searchLong-term agenda and outlookLong-term agenda and outlook

33

MemexPosited by Vannevar Bush in “As We May Think”

The Atlantic Monthly, July 1945

““A memex is a device in which an individual stores A memex is a device in which an individual stores all his books, records, and communications, and all his books, records, and communications, and which is mechanized so that it may be consulted which is mechanized so that it may be consulted with exceeding speed and flexibility”with exceeding speed and flexibility”

Supports: Annotations, links between documents, Supports: Annotations, links between documents, and “trails” through the documentsand “trails” through the documents

““yet if the user inserted 5000 pages of material a day yet if the user inserted 5000 pages of material a day it would take him hundreds of years to fill the it would take him hundreds of years to fill the repository, so that he can be profligate and enter repository, so that he can be profligate and enter material freely”material freely”

44

Sketch of memex

Bush’s camera on the head

66

Capturing what you see

77

Memory Overload

As hard drives get bigger and cheaper, As hard drives get bigger and cheaper, we're storing way too much.we're storing way too much.

By Jim LewisBy Jim Lewis

There's a famous allegory about a map of the There's a famous allegory about a map of the world that grows in detail until every point in world that grows in detail until every point in reality has its counterpoint on paper; the twist reality has its counterpoint on paper; the twist being that such a map is at once ideally accurate being that such a map is at once ideally accurate and entirely useless, since it's the same size as and entirely useless, since it's the same size as the thing it's meant to represent.the thing it's meant to represent.

88

"The PC is going to be the place where you store the information and really the center of control“ Billg

1/7/2001MyLifeBits is a project to “cyberize” everything!MyLifeBits is a project to “cyberize” everything!

What? Recall of all articles, books, CDs, What? Recall of all articles, books, CDs, photos, video, communication (e.g. mail, phone), photos, video, communication (e.g. mail, phone), web web Why? …Why? …“because we can”“because we can”

Office: communicate, store, & workOffice: communicate, store, & work Home & Media Center: ambiance &entertainmentHome & Media Center: ambiance &entertainment Immortality for progeny. Memory aidsImmortality for progeny. Memory aids

Goal: to understand the 1 TByte PC c2006: Goal: to understand the 1 TByte PC c2006: need, utility, cost, feasibility and tools.need, utility, cost, feasibility and tools.

99

Gordon: Researcher, consumer,

computer system tester,

nerd wanna-be, and average man

Melissa: middle manager

Patrick: Consultant

Nicholas: Analyst

Sondra: Office manager

Knowledge worker scenarios

1010

The guinea pig Gordon Bell is digitizing his lifeGordon Bell is digitizing his life Has now scanned virtually all:Has now scanned virtually all:

Books written (and read when possible)Books written (and read when possible) Personal documents (correspondence including memos and email, Personal documents (correspondence including memos and email,

bills, legal documents, papers written, …)bills, legal documents, papers written, …) PhotosPhotos Posters, paintings, photo of things (artifacts, …medals, plaques)Posters, paintings, photo of things (artifacts, …medals, plaques) Home movies and videosHome movies and videos CD collectionCD collection And, of course, all PC filesAnd, of course, all PC files

Now recording: phone, radio, TV (movies), web pages… Now recording: phone, radio, TV (movies), web pages… conversations?conversations?

Paperless throughout 2002. 12” scanned, 12’ discardedPaperless throughout 2002. 12” scanned, 12’ discarded.. Only 30 GB!!!Only 30 GB!!!

1111

Capture and encoding

1212

I mean everything

1313

Input: tools, time, and cost Scanners: HP Digital Sender, flat beds with ADF, Scanners: HP Digital Sender, flat beds with ADF,

2-HP photo, faxing. (Duplex, color, feed-thru, etc.)2-HP photo, faxing. (Duplex, color, feed-thru, etc.) A good commercial scanner costs 2K-10KA good commercial scanner costs 2K-10K

Photos: $1 or 0.5-5 min. Photos: $1 or 0.5-5 min. Large posters: ~ 1-5 hr.Large posters: ~ 1-5 hr.Artifacts: ~ 10 min. including photoArtifacts: ~ 10 min. including photo

Scanning to TIF, PDF: <1 min/page or .10/page Scanning to TIF, PDF: <1 min/page or .10/page OCR: for MODI or PDF: ~3-5 pages/min (old data)OCR: for MODI or PDF: ~3-5 pages/min (old data) OCR: to recreate an editable “original” 10 min/page!OCR: to recreate an editable “original” 10 min/page!

OCR (Volume paper files): 400 pages/hr. 7 ppm.OCR (Volume paper files): 400 pages/hr. 7 ppm. Books: scanned at CMU ($10 - 100/book) in 1997Books: scanned at CMU ($10 - 100/book) in 1997 Videos: tbdVideos: tbd

1414

Music6.9 GB

1.8K files180 CDs

Working2.3 GB

432 folders2.9K files

Archive5.1 GB

477 folders18.7 K files

Video2.6 GB

10 hoursLow res

My Books98 MB

27.1K files & 42K .msg17.7 GB (by size) Files (by number)

.xls.jpg

.doc/html

.pdf .ppt/ppt albums

.tif

CyberAll Nov.1, 2001

Mail.7 GB

43K msgs

.doc/html.jpg

.gif

.xls

.pdf

.ppt

.tif

.gif

1616

gbell wag: 67 yr, 25Kday life

1

1 0

1 0 0

1 , 0 0 0

1 0 , 0 0 0

1 0 0 , 0 0 0

1 , 0 0 0 , 0 0 0

1 0 0 -5 K B

M s g s

1 0 0 -5 0 K B

p a g e s

5 -1 0 0 K B

T i f s

0 . 1 -1 M B

B o o k s

1 0 -4 0 0 K Bj p e g s

4 0 K s1 K B p ss o u n d

0 . 1 -1 0 0 M Bs o n g s

1 -1 0 G B

V i d e o s

L i f e t i m e s t o r a g e ( G B )

1717

MyLifeBits organization: time and space

Timeline/Context(space)

Personal(some $s)

GB Co.(angel, etc.)

ProfessionalACM, etc., …@Microsoft.com,

New co’s.

Archival (time) Working

1818

MyLifeBits: Some Lives(t) PersonalPersonal

Parents, children, grandkidsParents, children, grandkids CGB himselfCGB himself Close friendsClose friends

GB $sGB $s Personal incl. several legal Personal incl. several legal

structuresstructures Investments & boardsInvestments & boards

Past companies/organiz’nsPast companies/organiz’ns DECDEC Carnegie-Mellon U.Carnegie-Mellon U. DEC, NSF, Encore, Ardent, DEC, NSF, Encore, Ardent,

GB_consulting, GB_consulting,

CGB@ MicrosoftCGB@ Microsoft MLBMLB ClustersClusters TelepresenceTelepresence WWW presenceWWW presence

Computer History MuseumComputer History Museum BOD memberBOD member Fund-raisingFund-raising CyberMuseumCyberMuseum

StartupsStartups Bell-Mason DirectorBell-Mason Director Diamond & Vanguard Brds.Diamond & Vanguard Brds.

1919

MyLifeBits is:

Memex and more (audio and video)Memex and more (audio and video) Universal store for all personal stuffUniversal store for all personal stuff Guiding principles for the system:Guiding principles for the system:

1.1. Full text search & Full text search & collectionscollections (> than hierarchy) (> than hierarchy)

2.2. Visualizations for search, display, insightVisualizations for search, display, insight

3.3. Annotations and links add value and essentialAnnotations and links add value and essential Increase search ability and value of information.Increase search ability and value of information. So make many kinds and them easy to create!So make many kinds and them easy to create! Stories are the ultimate annotationStories are the ultimate annotation

4.4. Keep the links when you author: “transclusion”Keep the links when you author: “transclusion”

2020

MLB database: size and content?

Database features are essential: Database features are essential: Consistency, Indexing, Consistency, Indexing, Pivoting, Queries, Speed/scalability, Backup, replication.Pivoting, Queries, Speed/scalability, Backup, replication.

Folders &Files were the starting point >> database into sets Folders &Files were the starting point >> database into sets aka “collections” that are identical to the folder structureaka “collections” that are identical to the folder structure

Outlook (msgs, attachments, calendar, contacts)Outlook (msgs, attachments, calendar, contacts) Web trails including voice message annotation Web trails including voice message annotation Journal (Outlook), trails: every document use & transactionJournal (Outlook), trails: every document use & transaction What about? What about?

Money (transactions, payees, etc.)…is their lifelog/trailMoney (transactions, payees, etc.)…is their lifelog/trail Streets and trips to cross-index to all docsStreets and trips to cross-index to all docs Attributes for photos for retrieval? Location, time, settingsAttributes for photos for retrieval? Location, time, settings Presentations as a report or trail. Each slide an object!Presentations as a report or trail. Each slide an object!

2121

2222

Media center 2

2323

CD

VCR

Cassette

Plasma Panel

DVD

MediaCenter

Computer

Set top

Set top

Kbd Mse

Wfr

Spkr

SpkrIR

Cable/Satellite

Ethernet SVHS-wide

5.1 digital

5 speakers

stereo

stereo

stereoVideo*

5.1 digitalcomp.

stereoVideo*

Video*

Cables/linksSpeaker 5+1Plasma 2 or 3Cable/Enet 2IR 8Stereo 45.1 digital 2Comp./S-video 3Plasma panel 1Power 10Kbd/mse 2Monitor II (opt.) 4Camera 2Total 42 – 46 Things 18+remotes

*Video = composite or S-videoCamera

Mic

Receiver

Legacy

Legacy

Legacy

Redundant

2424

Photos

2525

Caneel Bay Vacation Jan. 1998

Gordon, Gwen, Brig, Pam, Gordon, Gwen, Brig, Pam, Fiona, Bob, Laura and KolbeFiona, Bob, Laura and Kolbe

2626

Searching: the most useful app?

Challenge: What questions for useful results? Challenge: What questions for useful results? Lots of ways to look at what you retrieveLots of ways to look at what you retrieveNeed for breaking the returns into segmentsNeed for breaking the returns into segmentsSearching for an indexer and search engine: Searching for an indexer and search engine:

index service, Enfish, dtSearchindex service, Enfish, dtSearchStuff I’ve Seen MSR’s index & search… Stuff I’ve Seen MSR’s index & search…

evolving in the right direction. evolving in the right direction. Productizing would remove the pressure for Productizing would remove the pressure for

LonghornLonghorn

2727

2828

2929

3030

Detail view

3131

Resource explorerAncestor (collections), annotations, descendant

& preview panes turned on

3232

Interface to xls

3333

3434

Statistics of use

3535

Synchronized timelines with histogram guide

3636

Visualization Browsing & searching. “Get me what I want|need!”Browsing & searching. “Get me what I want|need!”

Help the user find things among possible items versusHelp the user find things among possible items versus Waiting for an ideal system that can find “what I want”Waiting for an ideal system that can find “what I want”

Publication: Conventional & web, presentations, Publication: Conventional & web, presentations, etc.etc.

Helps understand the nature of the content e.g. Helps understand the nature of the content e.g. histogram of objects in timehistogram of objects in time

Context: Links to help understand the relationship Context: Links to help understand the relationship between objects. Provides more search handles.between objects. Provides more search handles.

Information density: what is it? Information density: what is it? What is its relationship to others?What is its relationship to others?

Content important. Flash and form, less useful.Content important. Flash and form, less useful.

3737

Value of media depends on annotations

““Its just bits until it is annotated”Its just bits until it is annotated”

3838

System annotations provide base level of value

Date 7/7/2000Date 7/7/2000

3939

Tracking usage – even better

Date 7/7/2000. Opened 30 times, emailed to 10 Date 7/7/2000. Opened 30 times, emailed to 10 people (its valued by the user!)people (its valued by the user!)

4040

Get the user to say a little something is a big jump

Date 7/7/2000. Opened 30 times, emailed to 10 people. “BARC Date 7/7/2000. Opened 30 times, emailed to 10 people. “BARC dim sum intern farewell Lunch”dim sum intern farewell Lunch”

4141

Getting the user to tell a story is the ultimate in media value

A story is a “layout” in time and spaceA story is a “layout” in time and space Most valuable content (by selection, and by being well annotated)Most valuable content (by selection, and by being well annotated) Stories must include links to any media they use (for future navigation/search – Stories must include links to any media they use (for future navigation/search –

“transclusion”).“transclusion”). Cf: MovieMaker; Creative Memories PhotoAlbumsCf: MovieMaker; Creative Memories PhotoAlbums

Dapeng was an Dapeng was an intern at BARC intern at BARC for the summer for the summer of 2000of 2000

We took him to We took him to lunch at our lunch at our favorite Dim Sum favorite Dim Sum place to say place to say farewellfarewell

At table L-R: Dapeng, Gordon, Tom, Jim, Don, At table L-R: Dapeng, Gordon, Tom, Jim, Don, Vicky, Patrick, JimVicky, Patrick, Jim

4242

Value of media depends on annotations

Auto-annotate whenever Auto-annotate whenever possible e.g. GPS cameraspossible e.g. GPS cameras

Make manual annotation Make manual annotation as easy as possible. XP as easy as possible. XP photo capture, voice, photo capture, voice, photos with voice, etcphotos with voice, etc

Support gang annotationSupport gang annotation Make stories easyMake stories easy

no

ne

au

to

au

to-u

sag

e

use

r-ba

sic

use

r-story

Annotations

“Its just bits until it is annotated”

4343

4545

The Agenda for the Tbyte(s), Lifetime, PC:The killer app after office and mail.

1.1. Guarantee that data will live forever! “dear appy” problemGuarantee that data will live forever! “dear appy” problem2.2. Cheap, easy, and data-rich (e.g. time, place) capture:Cheap, easy, and data-rich (e.g. time, place) capture:

GPS and time everywhereGPS and time everywherePaper capture has to be as easy as discard (scanner/shredder)Paper capture has to be as easy as discard (scanner/shredder)E-book…e-magazines & journals need to have critical mass! E-book…e-magazines & journals need to have critical mass! Telephony and audio capture with indexingTelephony and audio capture with indexingMedia Center compatible for entertainment (photos, video, TV, radio)Media Center compatible for entertainment (photos, video, TV, radio)

3.3. One?One? dbase for all books, conversations, mail, web pages … dbase for all books, conversations, mail, web pages …vs. long-term use of hierarchical files. vs. long-term use of hierarchical files. Is dbase intuitive?Is dbase intuitive?

4.4. Annotations/meta-information add every-increasing valueAnnotations/meta-information add every-increasing valueEase of annotation because it aids search and becomes the contentEase of annotation because it aids search and becomes the contentContent analysis (critical for photo & video!)Content analysis (critical for photo & video!)

5.5. Information control: privacy, security, expunge/deniability,… Information control: privacy, security, expunge/deniability,… 6.6. New “killer apps”: alzheimer, immortality, surrogate memory?New “killer apps”: alzheimer, immortality, surrogate memory?7.7. Any GUI to improve use (e.g. time to learn, use, retention)Any GUI to improve use (e.g. time to learn, use, retention)

4646

The End

4747

The “dear appy” problemDear Appy, Dear Appy, How committed are you? How committed are you?

Please come back to me, Please come back to me, Lost and forgotten dataLost and forgotten data

Who’s responsible?Who’s responsible?mediamediaplatform, file, and databasesplatform, file, and databasesevolving standards and formatsevolving standards and formatsevolving and/or disappearing appsevolving and/or disappearing apps

  

4848

Digitizing our lives Right now, it is affordable to buy 100 GB/yearRight now, it is affordable to buy 100 GB/year In 5 years In 5 years 1TB/year is afforadable!1TB/year is afforadable! It’s hard to fill a terabyte/year just by keeping what you see or It’s hard to fill a terabyte/year just by keeping what you see or

hear, but you can:hear, but you can: Look at 9800 pictures Look at 9800 pictures a daya day (300 KB JPEGs) (300 KB JPEGs) Read 2900 documents Read 2900 documents a daya day (1MB files) (1MB files) Listening to audio or view compressed video 24 hours/day (it takes Listening to audio or view compressed video 24 hours/day (it takes

more than 256 kb/s to fill a TB in a year)more than 256 kb/s to fill a TB in a year) Watch 1.5 Mb/s video 4 hours each day.Watch 1.5 Mb/s video 4 hours each day.

As Bush said, we can “be profligate and enter material freely”As Bush said, we can “be profligate and enter material freely”

0

0.2

0.4

0.6

0.8

1

2002 2003 2004 2005 2006 2007

Dis

k ca

pac

ity

(TB

)