2004.12.02 - SLIDE 1IS 202 – FALL 2004 Lecture 26: Future of Information Systems Prof. Ray Larson & Prof. Marc Davis UC Berkeley SIMS Tuesday and Thursday

2004.12.02 - SLIDE 1IS 202 – FALL 2004

Lecture 26: Future of Information Systems

Prof. Ray Larson & Prof. Marc Davis

UC Berkeley SIMS

Tuesday and Thursday 10:30 am - 12:00 pm

Fall 2004http://www.sims.berkeley.edu/academics/courses/is202/f04/

SIMS 202:

Information Organization

and Retrieval

2004.12.02 - SLIDE 2IS 202 – FALL 2004

Lecture Overview

• Review of Last Time– Mobile and Context-Aware Multimedia

Information Systems

• Past and Future of Information Systems– Early Visions: Memex– Future Directions– Discussion Questions

• Action Items for Next Time

2004.12.02 - SLIDE 3IS 202 – FALL 2004

Lecture Overview


Information Systems



2004.12.02 - SLIDE 4IS 202 – FALL 2004

Moore’s Law for Cameras2000

Kodak DC40

Nintendo GameBoy Camera

$400

$ 40

2002

Kodak DX4900

SiPix StyleCam Blink

2004.12.02 - SLIDE 5IS 202 – FALL 2004

Capture+Processing+Interaction+Network

2004.12.02 - SLIDE 6IS 202 – FALL 2004

Camera Phones as Platform

• Media capture (images, video, audio)

• Programmable processing using open standard operating systems, programming languages, and APIs

• Wireless networking• Personal information

management functions• Rich user interaction modalities• Time, location, and user

contextual metadata

2004.12.02 - SLIDE 7IS 202 – FALL 2004

Space – Time – Social Space

SPATIAL

SOCIAL

TEMPORAL

2004.12.02 - SLIDE 8IS 202 – FALL 2004

Campanile Inspiration

2004.12.02 - SLIDE 9IS 202 – FALL 2004

Mobile Media Metadata Idea

• Leverage the spatio-temporal context and social community of media capture in mobile devices– Gather all automatically available information at the

point of capture (time, spatial location, phone user, etc.)

– Use metadata similarity and media analysis algorithms to find similar media that has been annotated before

– Take advantage of this previously annotated media to make educated guesses about the content of the newly captured media

– Interact in a simple and intuitive way with the phone user to confirm and augment system-supplied metadata for captured media

2004.12.02 - SLIDE 10IS 202 – FALL 2004

MMM Demo Video

2004.12.02 - SLIDE 11IS 202 – FALL 2004

What is “Location”?

2004.12.02 - SLIDE 12IS 202 – FALL 2004

Camera Location vs. Subject Location

• Camera Location = Golden Gate Bridge

• Subject Location = Golden Gate Bridge

• Camera Location = Albany Marina

• Subject Location = Golden Gate Bridge

2004.12.02 - SLIDE 13IS 202 – FALL 2004

Kodak Picture Spot

2004.12.02 - SLIDE 14IS 202 – FALL 2004

Future of Mobile Imaging

• Integrate media capture and analysis at the point of capture and throughout the media lifecycle

• Leverage contextual metadata (spatial, temporal, social, etc.) about the capture and use of media content

• Design systems that incorporate human beings as functional components and aggregate user behavior

• Leveraging the spatio-temporal-social context of media capture and use can enable us to infer media content and solve key problems in mobile media management

2004.12.02 - SLIDE 15IS 202 – FALL 2004

Lecture Overview


Information Systems



2004.12.02 - SLIDE 16IS 202 – FALL 2004

“Drawing the Circles”

2004.12.02 - SLIDE 17IS 202 – FALL 2004


2004.12.02 - SLIDE 18IS 202 – FALL 2004


2004.12.02 - SLIDE 19IS 202 – FALL 2004


2004.12.02 - SLIDE 20IS 202 – FALL 2004


2004.12.02 - SLIDE 21IS 202 – FALL 2004


2004.12.02 - SLIDE 22IS 202 – FALL 2004


2004.12.02 - SLIDE 23IS 202 – FALL 2004


2004.12.02 - SLIDE 24IS 202 – FALL 2004


2004.12.02 - SLIDE 25IS 202 – FALL 2004

Looking Backward Looking Forward

“History is an angel being blown backwards into the

future.”

Janusis the Roman god of

gates and doors (ianua), beginnings and

endings, and hence represented with a double-faced head,

each looking in opposite directions.

2004.12.02 - SLIDE 26IS 202 – FALL 2004

Lecture Overview


Information Systems



2004.12.02 - SLIDE 27IS 202 – FALL 2004

“What Dr. Bush Foresees”

Cyclops CameraWorn on forehead, it would photograph anything you see and want to record. Film would be developed at once by dry photography.

MicrofilmIt could reduce Encyclopaedia Britannica to volume of a matchbox. Material cost: 5¢. Thus a whole library could be kept in a desk.

VocoderA machine which could type when talked to. But you might have to talk a special phonetic language to this mechanical supersecretary.

Thinking machineA development of the mathematical calculator. Give it premises and it would pass out conclusions, all in accordance with logic.

MemexAn aid to memory. Like the brain, Memex would file material by association. Press a key and it would run through a “trail” of facts.

2004.12.02 - SLIDE 28IS 202 – FALL 2004

Memex

2004.12.02 - SLIDE 29IS 202 – FALL 2004

Memex Detail

2004.12.02 - SLIDE 30IS 202 – FALL 2004

Cyclops Camera

2004.12.02 - SLIDE 31IS 202 – FALL 2004

Vocoder: “Supersecretary”

2004.12.02 - SLIDE 32IS 202 – FALL 2004

Investigator at Work

• “One can now picture a future investigator in his laboratory. His hands are free, and he is not anchored. As he moves about and observes, he photographs and comments. Time is automatically recorded to tie the two records together. If he goes into the field, he may be connected by radio to his recorder. As he ponders over his notes in the evening, he again talks his comments into the record. His typed record, as well as his photographs, may be both in miniature, so that he projects them for examination.”

2004.12.02 - SLIDE 33IS 202 – FALL 2004

Memex

• “A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.”

2004.12.02 - SLIDE 34IS 202 – FALL 2004

Associative Indexing

• “[…] associative indexing, the basic idea of which is a provision whereby any item may be caused at will to select immediately and automatically another. This is the essential feature of memex. The process of tying two items together is the important thing.”

2004.12.02 - SLIDE 35IS 202 – FALL 2004

The WWW circa 1945

• “It is exactly as though the physical items had been gathered together from widely separated sources and bound together to form a new book. But it is more than this; for any item can be joined into numerous trails, the trails can bifurcate, and they can give birth to side trails.”

• “Wholly new forms of encyclopaedias will appear, ready-made with a mesh of associative trails running them, ready to be dropped into the memex and there amplified.”

2004.12.02 - SLIDE 36IS 202 – FALL 2004

Selection

• “The heart of the problem, and of the personal machine we have here considered, is the task of selection. And here, in spite of great progress, we are still lame.

• Selection, in the broad sense, is still a stone adze in the hands of a cabinetmaker.”

—“Memex Revisited” (Bush 1965)

2004.12.02 - SLIDE 37IS 202 – FALL 2004

Interaction Paradigms for IR

• Direct manipulation– Query specification– Query refinement– Result selection

• Delegation– Agents– Recommender systems– Filtering

2004.12.02 - SLIDE 38IS 202 – FALL 2004

The “Adaptive” Memex

• “In an adaptive Memex, the owner has delegated to the machine the ability to propose or effect changes in the stored information. By analogy to business practice, the Memex is said to be functioning as an agent (Kay, 1984). The machine is playing an autonomous role within a restricted charter: to attempt a more effective organization of the information based on observations of actual use and topical similarities.”

2004.12.02 - SLIDE 39IS 202 – FALL 2004

Lecture Overview


Information Systems



2004.12.02 - SLIDE 40IS 202 – FALL 2004

Future Directions

• Multimedia information systems• Context-aware information systems• Commonsense systems• New user interfaces• Structured metadata in IO and IR• Model-based applications• Metadata creating and using applications• Social software and recommender systems• Advances in probabilistic methods and machine

learning

2004.12.02 - SLIDE 41IS 202 – FALL 2004

Lecture Overview


Information Systems



2004.12.02 - SLIDE 42IS 202 – FALL 2004

Discussion Questions

• Yun Kyung Jung on Goldberg– Bush’s indexing idea is ‘Associative trails’ as proposed in his

visionary paper, ‘As we may think’. Bush claims that retrieval should not function as conventional indexes do but as the human brain does, and that the human brain operates on perceived relevance to some theme instead of indexing documents directly by their contents or characteristics. The author criticizes Bush’s idea of indexing because relevance judgments are subjective, inconsistent, situational, and likely change as one’s knowledge evolves. Also, personalized information systems have limited usefulness for others. But the assumption is that the general usefulness is more important than personal usefulness. Is this a valid assumption? Isn’t it better to customize the computer to support the way you think, rather than customize everyone to support the way the computer thinks?

2004.12.02 - SLIDE 43IS 202 – FALL 2004


• Yun Kyung Jung on Goldberg– According to the paper, one of the Bush’s

contribution is that Bush’s article, “As we may think” effect in stimulating other people to think about future information technology. It is now 60 years later, can we say that all of Bush’s ideas for the Memex have been solved?

2004.12.02 - SLIDE 44IS 202 – FALL 2004


• Sarai Mitnick on Bush– Bush writes that the human mind works

through association, that "with one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts." This complex and unpredictable way of thinking is echoed by Bates' Berry-picking model, but the conclusions are quite different. How are the models different, and what might account for these differences?

2004.12.02 - SLIDE 45IS 202 – FALL 2004


• Sarai Mitnick on Bush– Bush seems to take the permanency of

records for granted in his discussions of the memex and in his dismissal of indexing techniques. Clearly, the web brings that permanence into question. While hypertext links resembling Bush's "selection by association" are critical on the web, what advantages do indexing techniques have for any collection of changing, impermanent documents?

2004.12.02 - SLIDE 46IS 202 – FALL 2004


• Sarai Mitnick on Bush– While Bush demonstrates prescience in his depiction

of future technology, his foresight also demonstrates that the computer "revolution" has a lengthy history. Many people speak of computing as altogether new, but to what degree can personal computers be separated from their forebears, and from emerging computerized devices (ie, cell phones, programmable television)? Is the PC just one device in a history of computing tools, or is there something qualitatively different about it that will keep it at the forefront of technology?

2004.12.02 - SLIDE 47IS 202 – FALL 2004


• Joshua Chao on Bush

– In many ways, the MEMEX machine is already available today in the form of the personal computer and the World Wide Web. How have our lives changed and are these changes in-line with those predicted by Bush?

2004.12.02 - SLIDE 48IS 202 – FALL 2004



– Bush mentions that the biggest hurdle facing a useful MEMEX is the problem of "selection" (or information retrieval). Do you think the current methods of IR would meet Bush's requirements?

2004.12.02 - SLIDE 49IS 202 – FALL 2004



– Bush mentions that the biggest hurdle facing a useful MEMEX is the problem of "selection" (or information retrieval). Do you think the current methods of IR would meet Bush's requirements?

2004.12.02 - SLIDE 50IS 202 – FALL 2004


• Callie Jones on Memex II– The tone of Bush’s writing in “Memex II” is visionary and almost

Darwinistic. He suggests that new, improved versions of the original Memex will bring us to higher evolutionary levels of human thought and accomplishment. “The race progresses as the experience and reasoning of one generation is handed on to the next” (183). He says we will think in “newer and higher ways” as we enter this “new revolution, where the power of man’s thought is enhanced….” (166) Do you agree that a machine like Memex II could lead to higher and higher levels of thinking with each generation that inherits the previous one’s experience and reasoning? Can the notion of “reasoning” be fully expressed through Bush’s trails of association – even with “evaluation networks” like in the medical diagnosis example – or does “reasoning” involve more than weighting and associating elements with one another?

2004.12.02 - SLIDE 51IS 202 – FALL 2004


• Callie Jones on Memex II– The main reason Bush gives for the claim that Memex

II will enhance the power of man’s thought is that it “relieves his mind of much that is formulated and repetitive,” freeing him to concentrate on creative or “top-level” problems that advance civilization (166). Common sense suggests that “formulated and repetitive” tasks are good to delegate to machines, but is there ever a value in carrying them out? Long division is taught in schools, even though it is such a task, more efficiently performed by a calculator. Could a culture that mechanizes these tasks too thoroughly lead to an over-reliance on the machines that perform them and to intellectual laziness, rather than higher-level thinking?

2004.12.02 - SLIDE 52IS 202 – FALL 2004


• Callie Jones on Memex II– How re-usable and universal could the “associative

trails” in Memex II ever be? Bush asks: “Can a son inherit the memex of his father, or the disciple that of his master, refined and polished over the years, and go on from there?” He also suggests making massive memexes that bring vast numbers of associative trails together rather than the personalized, individual memexes originally conceived. How do the IR problems of predicting relevance judgments for multiple users in various subject areas apply to Bush’s concept of “associative trails”?

2004.12.02 - SLIDE 53IS 202 – FALL 2004


• Callie Jones on Memex II– How would you apply Bush’s concept of

“associative trails” to the metadata frameworks and syntax rules you coded in Protégé for your project?

2004.12.02 - SLIDE 54IS 202 – FALL 2004


• Callie Jones on Memex II– The innovator’s dilemma concerns the trade-off between the

exploration of new possibilities and the exploitation of old certainties. How might the innovator’s dilemma affect the use of a system like Memex II with so many trails? Bush writes: “After years of use the record will be everywhere permeated by a complex network of trails. These intersect at a multitude of points, a single item of importance may be tied into a dozen trails that pass through it for one reason or another” (175). Could the complexity and high number of trails associated with particular elements raise problems for usability? Could it be overwhelming for users if, for instance, every word in an encyclopedia entry displayed a set of possible trails linking that element to other elements related on the basis of some aspect of that highlighted word? Could too many choices lead to inaction or poor decision-making?

2004.12.02 - SLIDE 55IS 202 – FALL 2004


• Callie Jones on Memex II– One feature of Memex II is the incorporation of machine

learning, such that it could assist a physician by providing its suggestions for diagnosing a patient by comparing symptoms with previous cases recorded and learning from an evaluation network that would be periodically adjusted and revised. Bush discusses whether this version of Memex II could render sound judgments in other areas. “Whenever judgment can be based on explicitly stated criteria,” he writes, “the machine can judge more precisely, and far more rapidly, than any group of men. It becomes relatively helpless when both the data and criteria are vague” (182). For what kinds of problems can we develop such “explicitly stated criteria” on which we all agree? Could it really work in the profession of law, as Bush suggests?

2004.12.02 - SLIDE 56IS 202 – FALL 2004

Lecture Overview


Information Systems



2004.12.02 - SLIDE 57IS 202 – FALL 2004

Assignments 8 & 9

• Assignment 8 Photo Annotation– 5 photos per person in your ontology– Detailed annotation– In MMM2 browser

• Assignment 9 Project Presentation– Presentation of Application Idea– Presentation of Classification and Annotated Photos– Lessons Learned– Q&A

2004.12.02 - SLIDE 58IS 202 – FALL 2004

Next Time: Project Presentations

• Each group will have 10 minutes to present– This time limit will be rigorously enforced in order to

accommodate all groups

• We recommend structuring your presentation as follows:– 2 minutes: Presentation of Application Idea– 6 minutes: Presentation of Classification and

Annotated Photos and Lessons Learned– 2 minutes: Time for Q&A

• We also require that every member of the team be given the opportunity to present

2004.12.02 - SLIDE 59IS 202 – FALL 2004

Next Time: Project Presentations

• Presentation tools– PowerPoint slides or a web page for your

presentation– Protégé or the MMM2 Web Site or PowerPoint slides

or some other tool or medium to present your facetted classification

– MMM2 web site or PowerPoint slides or some other tool or medium to present your photo annotation examples

• All presentations should be linked to and accessible from your group web page– This will facilitate a quick transition from one group’s

presentation to the next

Documents

2004.12.02 - SLIDE 1IS 202 – FALL 2004 Lecture 26: Future of Information Systems Prof. Ray Larson & Prof. Marc Davis UC Berkeley SIMS Tuesday and Thursday