Raskar Entrepreneurship in Imaging, Imaging Ventures, CVPR 2012

Raskar, Camera Culture, MIT Media Lab

Camera Culture

Ramesh Raskar MIT Media Lab

http://raskar.info http://FB.com/rraskar

Ramesh Raskar Associate Professor

Imaging Ventures

Ramesh Raskar, MIT Media Lab

MAS.533 Imaging Ventures ~ Ramesh Raskar & Joost Bonsen © Spring 2010 ~ Tue 6-8pm ~

http://imagingventures.media.mit.edu/

Business Plan Executive Summaries Hadzima’s Pyramid

Core Concept

Elevator Speech

Executive Summary

Powerpoint Presentation

Full Business Plan

Detailed Support Foundation

Phrase

Paragraph

Few Pages

5-15 Slides

20-50 Pages

Binders

Ramesh Raskar, MIT Media Lab MAS.533 Imaging Ventures ~ Ramesh Raskar & Joost Bonsen © Spring 2010 ~ Tue 6-8pm ~ http://imagingventures.media.mit.edu/

Tech “Push” vs Market “Pull”

Problem

Solution

• Refining a business idea is iterating between “what you can do (the How)” and “what’s most worth doing (the Why)”

• Converging top-down and bottom-up thinking

After X, what is neXt

How to Invent?

MAS.533 Imaging Ventures ~ Ramesh Raskar & Joost Bonsen © Spring 2010 ~ Tue 6-8pm ~ http://imagingventures.media.mit.edu/

Exploring Cameras, Displays & Visual

Computing Innovations

Spring 20xx Tue 4-6p

Media Lab e14-525

Professor Ramesh Raskar

Joost Bonsen

http://imagingventures.media.mit.edu

Imaging Ventures MAS.533

Joost Bonsen

The Full MIT Innovation Ecology

Basic+Applied

Research

Commercial

Promise &

Viability

University

Spin-Off Emerging

Growth

Established

Company

Support /

Offices:

Academic /

Research

Offices

Student /

Informal

Organizations

QuickTime™ and a decompressor

are needed to see this picture.

Basic+Applied

Research

Commercial

Promise &

Viability

University

Spin-Off Emerging

Growth

Established

Company

Tiger Teams

D-Lab Energy,

ICT, Health,

Cycles, Design…

MarketLabs

New Enterprises

Ventures

D-Lab Development

Ventures, Dissemination

Imaging Ventures

VentureSIGs

Africa Health Delivery Lab

India Lab

China Lab

Tech Testbeds

Leadership Lab

Action Labs @ MIT Both For-Credit Curricular & Non-Credit Extracurricular Offerings

T h e M I T I n n o v a t I o n P I p e l I n e

Neuro Ventures

Goals of Class

1. Quality Connections with Fellow Classmates, Course Advisors, & Guest Speakers

2. Survey-Level Awareness of Imaging Technology Business Challenges & Opportunities

3. Lessons-Learned from Live-Cases 4. A Specific Venture Proposal written up as a Business

Plan Executive Summary 5. Increased Skills for Analyzing, Communicating, and

Synthesizing Information towards Imaging Venture Creation

6. Global Perspective on Imaging Innovations

MIT Alum Venture Legacy

The MIT $100K Entrepreneurship

& IDEAS Competitions • Elevator Pitch Competition -- October

– 1 minute, $1000 (October)

• Executive Summary Competition -- November

– 2 pages, $1000

– Announced next week Thurs 2/11 7:30pm 10-250

• Business Plan Competition -- Spring

– 2 page executive summary + business plan DUE 3/4!

– $100k Grand Prize + Track Prizes

http://www.mit100k.org/

• IDEAS Competition

– Mid-Month Rolling Submission; Grant Funds

http://web.mit.edu/ideas

MIT $100K Alumni Companies

Camera Culture Creating new ways to capture and share visual information

MIT Media Lab Ramesh Raskar

http://cameraculture.info

Facebook.com/cameraculture

1.Light-Field Camera A new camera design

exploiting the fundamental

dictionary of light-fields for a

single-capture capture of

light-fields with full-

resolution refocusing

effects.

2. Color Primaries A new camera design with

switchable color filter arrays

for optimal color fidelity and

picture quality on scene

geometry, color and

illumination.

3. Flutter-Shutter A camera that codes the

exposure time with a binary

pseudo-sequence to de-

convolve and remove

motion blur in textured

backgrounds and partial

occluders.

4. Compressive

Capture We analyze the gamut of

visual signals from low-

dimensional images to light-

fields and propose non-

adaptive projections for

efficient sparsity exploiting

reconstruction.

Computational Photography

1. Looking around corners Using short laser pulses and fast detector, we

aim to build a device that can look around

corners with no imaging device in the line of

sight using time resolved transient imaging.

2. Reflectance Recovery We demonstrate a new technique that allows

a camera to rapidly acquire reflectance

properties of objects 'in the wild' from a single

viewpoint, over relatively long distances and

without encircling equipment.

3. Trillion Frames per Second Imaging A camera fast enough to capture light pulses

moving through objects. We can use such a

camera to understand reflectance, absorption

and scattering properties of materials.

Femtosecond Imaging 3D Displays

1. Tensor Display A family of compressive light

field displays comprising all

architectures employing a stack

of time-multiplexed, light-

attenuating layers illuminated by

uniform or directional

backlighting

2. Layered 3D Tomographic techniques for

image synthesis on displays

composed of compact volumes

of light-attenuating material.

Such volumetric attenuators

recreate a 4D light field or high-

contrast 2D image when

illuminated by a uniform

backlight.

3. Glasses-free 3D HDTV Light field displays with

increased brightness and

refresh rate by stacking a pair of

modified LCD panels, exploiting

rank and constraint of 3D

displays

4. BIDI Screen A thin, depth-sensing LCD for

3D interaction using light fields

which supports both 2D multi-

touch and unencumbered 3D

gestures.

5. Living Windows 6D

Display A completely passive display

that responds to changes in

viewpoint and changes in

incident light conditions.

May 2012

Can you look around the corner ?

2nd Bounce

Multi-path Analysis

1st Bounce

3rd Bounce

Compressive Displays

mask 2

mask 3

mask K

light box

mask 1

Layered 3D

1. Augmented Light Fields Expands light field representations

to describe phase and diffraction

effects by using the Wigner

Distribution Function

2. Hologram v Parallax Barrier Defines connections between

parallax barrier displays and

holographic displays by analyzing

their operations and limitations in

phase space

3. Ray–Based Diffraction Model Simplified capture of diffraction

model for computer graphics

applications.

Post-Doctorial Researchers: Doug Lanman, Gordon Wetzstein, Alex Olwal, Christopher Barsi

Research Assistants: Matthew Hirsch, Otkrist Gupta, Nikhil Naik, Jason Boggess, Everett Lawson, Aydın Arpa, Kshitij Marwah

Visiting Researchers & Students: Di Wu, Daryl Lim

1. Retinal Imaging With simplified optics and cleaver

illumination we visualize images of

the retina in a standalone device

easily operated by the end user.

2. NETRA/CATRA Low-cost cell-phone attachments

that measures eye-glass

prescription and cataract information

from the eye.

3. Cellphone Microscopy A platform for computational

microscopy and remote healthcare

4. High-speed Tomography A compact, fast CAT scan machine

using no mechanical moving parts or

synchronization.

5. Shield Fields 3D reconstruction of objects from a

single shot photo using spatial

heterodyning.

6. Second Skin Using 3D motion tracking with real-

time vibrotactile feedback aids the

correct of movement and position

errors to improve motor learning.

Health & Wellness

1. Bokode Low-cost, passive optical design

so that bar codes can be shrunk

to fewer than 3mm and read by

ordinary cameras several meters

2. Specklesense Set of motion-sensing

configurations based on laser

speckle sensing . The underlying

principles allow interactions to

be fast, precise, extremely

compact, and low cost.

3. Sound Around Soundaround is a multi-viewer

interactive audio system,

designed to be integrated into

multi-view displays presenting

localized audio/video channels

with no need for glasses or

headphones.

Human Computer Interaction Visual Social Computing

1. Photocloud A near real-time system for

interactively exploring a

collectively captured moment

without explicit 3D

reconstruction.

2. Vision Blocks On-demand, in-browser,

customizable, computer-vision

application-building platform for

the masses. Without any prior

programming experience, users

can create and share computer

vision applications.

3. Lenschat LensChat allows users to share

mutual photos with friends or

borrow the perspective and

abilities of many cameras.

Light Propagation Theory and Fourier Optics

Visit us online at

Cameraculture.info fb.com/cameraculture

Computational Camera

Pixels

Images

Illumination

Lenses

Image Processing

Photons

Light Fields (4D/6D/8D)

Computational Light Transport

Algebraic Rank

Scene Priors/sparsity

Transforms (signal proc)

Smart Cameras

Bit Hacking

Computer Vision

Optics

Sensors

Signal Processing

Displays

Machine Learning

Computational Light Transport

Computational Photography Illumination

Co-designing Optical and Digital Processing

After X, what is neXt

How to Invent?

Simple Exercise ..

• Image Compression

– Save Bandwidth and storage

What is neXt

Simple Exercise ..

• Image Compression

– Save Bandwidth and storage

.. Video Compression ..

– Extend the idea to time dimension

Strategy #1: Xd

Extend it to next (or some other) dimension

• Idea you just heard

• Concept

• Patent

• New Product

• Product feature

• Design

• Art

• Algorithm

Strategy #1: Xd

• Extend it to next dimension (or some other) dimension

– Flickr to Youtube

– Wikipedia to .. ?

• Generalize the concept

(common in patent applications)

• Text, Audio (Speech), Image, Video .. Whats next ?

– CD ..

• Images to infrared, sound, ultrasound to EM spectrum

• Macro scale to microscale

• Airbag for car to airbag for .. ?

Data Text Audio Images Video ..

Strategy #2: X+Y

• Fusion of the dissimilar

– More dissimilar, more spectacular the output

• Example

– Scientific imaging + Photography

• Eye-care

• Coded aperture

• Tomography

Slit Lamp for Cataracts

Slit Lamp Exam Retinal Scan

Retinal Scan

Phoropter

Reading Charts

Vitor Pamplona, Ankit Mohan, Manuel Oliveira, Ramesh Raskar SIGGRAPH 2010

NETRA Near Eye Tool for Refractive Assessment

7 Billion people

4.5B cell phones

2B refractive

errors

0.6B without glasses

NETRA at

LVP Eye Institute

Diagnostics Dispensing Glasses

Bulky Equipment , Trained Professionals Increasingly cheap and easy

Shack-Hartmann WS

Wavefront aberrometer

Refraction Map using Wavefront Sensor

Spot Diagram on LCD

NETRA = Inverse of Shack-Hartmann wavefront sensor User interactively creates the Spot Diagram

1. Displace 25 spots with smart UI

CellPhone LCD

EyePiece

2. Displace spots till single dot

perceived

300+ DPI LCDs

NETRA Worldwide

Confidential 43

Thermometer for Eye

Brazil

Strategy #3: X

Do exactly the opposite

Fosbury Flop

http://en.wikipedia.org/wiki/File:Bundesarchiv_Bild_183-S0305-

0030,_Rolf_Beilschmidt.jpg

http://en.wikipedia.org/wiki/Dick_Fosbury

Straddle Method for High Jump

Fosbury Method

Replacement of landing

surfaces with foam rubber

1968 Olympics: 2.24m

• Toll Free calls

• Reverse Auction

Strategy #3: X Do exactly the opposite

• Processing, Memory, Bandwidth

– In Computing world, in any era, one of this is a bottleneck

– But overtime, they change

– E.g. bandwidth is now considered virtually limitless

• Business Process Reengineering (BPR)

– Michael Hammer, James Champy, 1990s

• In imaging:

– SLR: Faster mirror flip or no mirror flip

• Companies spent years improving mirror flip speed

• Why not just remove it?

• More computation

• Less light

Strategy #4: X

• Given a Hammer ..

– Find all the nails

– Sometimes even screws and bolts

Strategy #4: X

• Given a Hammer ..

– Find all the nails

– Sometimes even screws and bolts

• Given a cool solution/technique/Opportunity

– Find other problems

– (Where to find them?)

• Examples

– Compressive sensing

– Mobile phone opportunity

Strategy #5: X

• Given a nail,

– Find all hammers

– Sometimes even screwdrivers/pliers may work

• Given a problem,

– Find other solutions

– (Where to find them?)

• Examples

– App store (Apple) ..

– How to create Open platform for all devices

– How to create a ‘hardware’ app store

Strategy #6: X++

• Pick your adjective ..

• Making it faster, better, cheaper

neXt = adjective + X

X++ : Add your favorite adjective

• Context aware,

• Adaptive

• (temporally) Coherent,

• Hierarchical,

• Progressive

• Efficient

• Parallelized

• Distributed

• Example: Image or video compression schemes

• Personalized/Customized

• Democratized

X++ : Add your favorite adjective

• Example: Image or video compression schemes

• But X++ is a sign

– The field is maturing in terms of research

– But booming in business impact

• Kaizen

– Small incremental changes

– Japanese Management styles (6sigma, Kanban)

Where to find the ‘X’

• Awards (best papers, product, researchers)

• Talks abstract

– no need to attend, subscribe to mailing lists

• Network and talk to people

• Avoid small-talk .. Ask ‘what is the latest X’

• Patents

• Table of Contents

• Index pages

Pitfalls

• These six ways are only a start

• They are a good mental exercise and will

allow you to train as a researcher

• Great for projects

• But

– Maynot produce radically new ideas

– Sometimes a danger of being labeled incremental

– Could be into ‘public domain ideas’

What are Bad ideas to pursue

• X then Y (then Z)

– X+Y is great with true fusion, fusion of dissimilar is best

– But avoid a ‘pipeline’ systems, where the output of one is

THEN channeled into the input of the next stage, and non

of the components are novel (idea is easy to scoop)

http://www.flickr.com/photos/rsilfver/178134761/

What are Bad ideas to pursue

• X then Y (then Z)

– X+Y is great with true fusion, fusion of dissimilar is best

– But avoid a ‘pipeline’ systems, where the output of one is

THEN channeled into the input of the next stage, and non

of the components are novel (idea is easy to scoop)

• Follow the hype (too much competition)

• Do because it can be done

– (Why do we climb a mountain? because it is there! )

– But only the first one gets a credit.

– May make you strong, and give you a sense of

achievement but not a research project.

– My personal triangle of criteria

• Fun, Impact, Money

• 2 out of 3

– Intersection of interests, skills, demand

– Maybe another talk ..

Is the idea worth pursuing?

Is project worthwhile? Heilmeier's Questions

• What – What are you trying to do? Articulate your objectives using absolutely no jargon.

• Related work – How is it done today, and what are the limits of current practice?

• Contribution – What's new in your approach and why do you think it will be successful?

• Motivation – Who cares?

– If you're successful, what difference will it make?

• Challenges – What are the risks and the payoffs?

– How much will it cost?

– How long will it take?

• Evaluation – What are the midterm and final "exams" to check for success?

• Why now? (why not before, what’s new that makes possible)

• Why us? (wrong answers: I am smart, I can work harder than others)

http://en.wikipedia.org/wiki/George_H._Heilmeier#Heilmeier.27s_Catechism

Great Research: Strive for Five

1. Before Five teams Be first, often let others do details

2. Beyond Five years What no one is thinking about

3. Within Five layers of ‘Human’ Impact Relevance

4. Beyond Five minutes of description Deep, iterative, participatory

5. Fusing Five+ Expertise Multi-disciplinary, proactive

Ramesh Raskar, http://raskar.info

Acknowledgements

• Members of Camera Culture, MIT group

• Vitor Pamplona

• Kari Pulli, Nokia

• Asmita Joshi

• Rupesh Nasre, IISc

• Mark Bolas, USC

• Rajiv Narayan, Broad Institute, MIT

• Joost Bonsen, MIT

Research ..

• http://raskar.info

– How to come up w ideas: Idea Hexagon

– How to write a paper

– How to give a talk

– Open research problems

– How to decide merit of a project

– How to attend a conference, brainstorm

– Facebook.com/ rRaskar

• Tips

– Get on Seminar/Talks mailing lists worldwide

– http://www.cs.virginia.edu/~robins/YouAndYourResearch.html

– Why do so few scientists make significant contributions and so many are forgotten in the long run?

– Highly recommended Hamming talk at Bell Labs

Unfair Advantage

Extras ..

Prepare

• Internships

• Incubator (YC, TechStars)

• Move to cluster

• Find a mentor (but impress them first)

• If you do good stuff, good stuff will happen to you

• Journal paper, conf (get your ideas validated)

• Dont worry about discussing your plans w hundreds – Ideas don’t matter as much, team and execution matters

• Talk to people in very diff areas – (e.g. arial survey of powerlines)

• Should you have Prof. on board?

• Kickstarter (video demo)

Unfair Advantage

• Why now? (why not before, what’s new that makes possible)

• Why us? (wrong answers: I am smart, I can work harder than others)

• Do asmuch in univ, file provisionals (talk to TLO)

• Why not to file – To enforce rather than defend (5 years, $3M)

– Prevent someone else from patenting (write journal paper instead)

– License (too much effort and luck)

• Why to file – Stimulate investment or acquisition

– Deter patent infringements lawsuits

– Have leverage wrt partners (before JV)

Productivity

• Biz Card

• LinkedIn

• Website

• CardMunch

• Use Google Doc for Exec Summary – (read only mode)

• Many online resources – biz plan, Founders Workbench (Goodwin Procter)

MIT Media Lab Ramesh Raskar http://raskar.info facebook.com/rraskar

Raskar Entrepreneurship in Imaging, Imaging Ventures, CVPR 2012

Business

Weakly supervised object detection (WSOD) · Diba CVPR 2017 (VGG16) Bilen CVPR 2016 (VGG16) Bilen CVPR 2016 (AlexNet) Cinbis PAMI 2016 (AlexNet+FV) Bilen CVPR 2015 Wang ECCV 2014

CVPR 2019 Tracking and Detection Challenge CVPR 2019 · CVPR 2019 Tracking and Detection Challenge 16/06/2019 CVPR 2019 Long Beach Tracker MOTA IDF1 First Name Surname Affiliation

CVPR Program Rev15web.media.mit.edu/~raskar/MySlides/08CVPR/CVPRProgram.pdf · 2008-12-26 · 24. Consistent Image Analogies using Semi-supervised Learning, Li Cheng, S. V. N. Vishwanathan

Raskar, Camera Culture, MIT Media Lab Camera Culture Ramesh Raskar MIT Media Lab Ramesh Raskar Associate Professor

Ramesh Raskar Associate Prof, Media Lab, MIT Camera ...web.media.mit.edu/~raskar/Slides/02LectureFeb13.pdf · Ramesh Raskar Associate Prof, Media Lab, MIT ... • Beyond Visible Spectrum

New insight to vp plaque imaging cvpr

Motion Deblurring Using Hybrid Imaging Moshe Ben-Ezra and Shree K. Nayar Columbia University IEEE CVPR Conference June 2003, Madison, USA

CVPR Underwater Final

Tuto cvpr part4

Vijay Balu Raskar E.T. Notes

Raskar Ilp Oct08 Web

Introduction to Camera Challenges - Ramesh Raskar

Raskar, Camera Culture, MIT Media Lab Camera Culture Ramesh Raskar Camera Culture Associate Professor, MIT Media Lab

Raskar Npar04final

Raskar Next Billion Cameras Siggraph 2009

Raskar Sig05 Display Panel July05

CVPR Tutorial Final

Raskar ideahexagonapr2010

Raskar Pocket Proj Fid2005presented

Tuto cvpr part1