Michael J. BlackAugust 2002 Department of Computer Science Industrial Partners Program August 2002 Brown University

Michael J. BlackAugust 2002

Department of Computer Science

Industrial Partners Program

August 2002

Brown University


Brown University

Founded 1764;

Third college in New England and 7th in the country.

556 faculty members

16,606 undergraduate applicants – 1413 admissions.

55% female, 28% minority.


Brown Students5,674 undergraduates,

1,343 graduate students

316 medical students

Intended freshmen concentrations:

* 36% science and math:

* 26% humanities

* 21% social sciences

* 9% engineering.

Many involved in research.


CS Department

22 Tenured and tenure-track faculty and growing

194 ugrads

94 graduating

900 students/term in CS classes

26 masters students

52 PhD students

17 graduating masters

7 graduating PhDs


Industrial Partners ProgramFocused program with a small number of partners.

Customize relationships with each partner company (varies over time).

Close interactions throughresearch

recruiting

symposia and meetings


Industrial PartnersIndustrial Partners


MembershipAnnual fee: $25,000.

- directly supports research and instruction

Recruiting assistance

- job fairs, guaranteed place, special events, IPP colloquia, recruiting events, fliers, announcements….

Information dissemination

- conduit! newsletter with research news, patenting, partner interactions, …


IPP SymposiaHeld twice a year (spring and fall)

- speakers from academia and industry- topics of current interest- opportunity for pre-competitive cooperation with other partners- discussion and social opportunities

Recent symposia:

Computer and Network Security; Component Software and Technologies; Web Technologies; E-Commerce; Computing in a Wireless World; …


MembershipInvitations to CS seminars and colloquia.

Visits to campus and meetings with faculty and students.

Visits by faculty or students.

Regular contact with Prof. Michael Black (Director) and Suzi Howe (Manager) to customize the relationship.

http://www.cs.brown.edu/industry/ipp/


Research (today)Comp. Linguistics and Natural Language Understanding

(E. Charniak, T. Hofmann, M. Johnson, I. Kontoyiannis)

Database Management Systems

(U. Cetintemel, D.Goldin, S. Zdonik)

Theory of Networking, Concurrency, and Distributed Comp.

(M. Herlihy, S. Krishnamurthi, A. Lysyanskaya, E. Upfal)

Security (Distributed Data Authentication)

(R. Tamassia, P. Klein, A. Lysyanskya, T. Doeppner)

New User Interfaces

(M. Black, J. Hughes, D. Laidlaw, A. van Dam, D. Zelter)


ResearchAgents and Ecommerce –

- statistical learning, decision theory, game theory

- agent-based information retrieval systems

- auctions, commodity trading, portfolio optimization…

- facilitate human-computer interaction

(Tom Dean, Amy Greenwald, Thomas Hofmann, Eli Upfal, Pascal Van Hentenryck)


ResearchMobile and Ubiquitous Computing –

- data and resource management

- sensor networks, mobile ad hoc networks, Internet-scale information systems

- decentralized replication (overlay networks) for mobile and weakly connected environments

- adaptive data dissemination in wireless networks

- mobile computing and collaboration tools, note-taking, annotation, etc.

(Ugur Cetintemel, Tom Doeppner, Don Stanford, Stan Zdonik)


Other ResearchArtificial Intelligence

Combinatorial Algorithms

Combinatorial Optimization

Computational Biology

Computational Geometry and

Graph Drawing

Computer Graphics

Educational Software

Electronic Documents and Hypermedia

Internet Computing

Nanotechnology

Neuroinformatics and Brain Science

Operating Systems and Distributed Systems

Parallel Computation

Programming Languages

Randomized Algorithms and Probabilistic Analysis

Robotics and Computer Vision

Scientific Visualization

Scientific Computing

Software Engineering

Theory of Computation

User Interfaces and VR

Verification and Reliable Systems



Computing out of the BoxComputing out of the Box

Michael J. Black

Department of Computer Science

Brown University


WHITHER THE COMPUTER?

Dell workstation, 2001

XEROX Alto 1973


New Types of Interaction

Give machines more information (awareness) about their users and their users’ environment.

* speech and natural language

* vision

- faces, identity, expressions, gaze, …

- pose, motion, gesture

- action, setting, tasks, environment

- interactions between people


ApplicationsApplications• Human-Computer Interaction• Surveillance • Motion capture (games and animation)• Video search/annotation• Work practice analysis.

Social display of puzzlement

* detect moving regions* estimate motion* model articulated objects* model temporal patterns of activity * interpret the motion


Analysis of Facial MotionAnalysis of Facial Motion

Recognizing facial expressions from motion* passive user interfaces* active user interfaces

With Yaser Yacoob


Recognizing Facial MotionRecognizing Facial Motion


Gestural InterfacesGestural Interfaces


Capturing MeetingsCapturing Meetings

Motion and gestureanalysis

Web interfaceInput video

With Shanon Ju


Why is it Hard?Why is it Hard?

Low contrast

Self occlusion. Deformation.

2D view of 3D world

Unusual poses. Large Motion

Ambiguous matches


ApproachApproachBayesian formulation

p(model | cues) = p(cues | model) p(model)

3. Model/Search: particle filtering.

p(cues)1. Likelihood:

a. probabilistic, learned from examples.b. model both people and generic scenes (explain

entire image).

2. Prior: “learned” implicit probabilistic model uses ideas from texture synthesis.


Tracking 3D MotionTracking 3D Motion

With H. Sidenbladh


Modeling/Tracking ActivitiesModeling/Tracking Activities

With H. Sidenbladh


Forbidden Planet, 1956Forbidden Planet, 1956


BRAIN-COMPUTER INTERFACES

“If I could find … a code which translates the relation between the reading of the encephalograph and the mental image …the brain could communicate with me.”

“Donovan’s Brain”, Curt Siodmak, 1942

Brain

“Mad” scientist Nancy Davis (Reagan)


LANGUAGE OF THE BRAIN

“If spikes are the language of the brain, we would like to provide a dictionary… perhaps even providing the analog of a thesaurus.”

Rieke, et al 1997.


A NEURAL PROSTHETIC

EEG

single and multi-neuronactivity

Voluntary control signal

mathematical algorithm

Computer cursorand

keyboard entryRobotic arm

Stimulation of Muscles, Spinal Cord, and Brain

From: Mijail Serruya


CELL ENSEMBLES

B

PMA

CentralSulcus

Arcuate

SMAMIMI

5 mm

Implanted in the MI arm area of motor cortex

PMA

1 ms

80µV

100 electrodes, 400m separation4x4 mm

Utah Array


REAL TIME NEURAL CONTROL

Target

Neural control

Mijail Serruya

Linear filters built on-line.


ROBOT CONTROL

From: Mijail Serruya


Beyond the Desktop Metaphor

Biologically-embedded computing.

new physical pathways for interacting with the world

sensing and acting inside the body

the computer learns about the brain while the brain is constantly changing

Basic research in perception enabling machines to “see” their users.


Perceptual Perceptual User User InterfacesInterfaces

with Francois Berard


NEURAL IMPLANT

500 µm

Bone

Connector

Silicone

Dura

White Matter

400 µm

Cortex

I

III

V

VI

Arachnoid space

Connector Acyrlic

Chronically implanted.Stable recording for 2-3 years

(but not necessarily the same cells every day)Spikes as well as local field potentials.


SINGLE UNIT ACTIVITY

1/10 mm

2/1000’s second

Spikes

David Sheinberg


CURRENT/FUTURE WORK* Learning low-dimensional linear models of conditional firing

(“receptive fields”).

* Comparison of Bayesian and non-Bayesian decoding methods

(Kalman filter, particle filter, linear filter)

* Incorporating local field potentials.

* Analysis of more complex motions and statistical models.

* Recognizing patterns of motion.

* Plasticity.

* Robot control (service robots, semi-autonomous).

* Recording from multiple brain areas.

* Human studies.


SUMMARYBayesian model of neural activity in MI

* probabilistic relationship between neural activity and events in the world.

Non-parametric model computed using regularization

* extensive cross-validation experiments

* superior to previous methods

Introduced particle filtering for the Bayesian inference of hand motion in non-overlapping 50 ms intervals

* non-Gaussian likelihood and non-linear dynamics

* supports more sophisticated analysis


THANKSM. Fellows, Neuroscience

D. Sheinberg, NeuroscienceN. Hatsopoulos, NeuroscienceW. Patterson, EngineeringA. Nurmikko, EngineeringG. Friehs, Brown Medical School

S. Geman, Division of Applied Mathematics S. Shoham, Princeton

L. Paninski, NYU, Center for Neural ScienceSupport:

National Science Foundation, ITR Program.The Keck FoundationThe National Institutes of Health
