Embed Size (px)
Citation preview
Why is Computer Vision on a Mobile Device
Different? Instructor - Simon Lucey
16-423 - Designing Computer Vision Apps
Example of SLAM for AR
Taken from: H. Liu et al. “Robust Keyframe-based Monocular SLAM for Augmented Reality”, ISMAR 2016.
Example of SLAM for AR
Taken from: H. Liu et al. “Robust Keyframe-based Monocular SLAM for Augmented Reality”, ISMAR 2016.
Example of SLAM for AR
Taken from: H. Liu et al. “Robust Keyframe-based Monocular SLAM for Augmented Reality”, ISMAR 2016.
Source: http://www.slashgear.com/iphone-7-potential-wanes-as-android-n-starts-to-tango-20440932/
Source: http://www.slashgear.com/iphone-7-potential-wanes-as-android-n-starts-to-tango-20440932/
Today
• Course Logistics
• Philosophy to Mobile Computer Vision R&D
About this Course
• Team
• Office hours: by appointment (use Piazza). • 16423.courses.cs.cmu.edu has ALL information. • Questions: Please use Piazza. • Finding a project partner: Please use Piazza.
Me
(Instructor)(TA)
Ming-Fang Chang (Allie)
Assignments
• There will be 4 assignments - (4 + 12 + 12 + 12) = 40% • Each assignment will relate to the topics of the previous
lectures, but ALSO take us closer to the task of building our OWN augmented reality app.
• Assignment 0 will be released on Thursday January 19th. • Assignment 0 is due Friday February 3rd. • See course website (16423.courses.cs.cmu.edu) for full
schedule.
Assignments
• Goal is that every assignment takes you a step closer to building your OWN augmented reality app.
• Assignments are designed to take us (step-by-step) towards an augmented reality app.
Late Hand in Policy
• Late hand-in policy. Each student is allotted a total of five late-day points for the semester. Late-day points are for use on assignments only (They cannot be used for midterm or final projects). Late-day points work as follows:
• A person can extend an assignment deadline by one day using one point.
• If a person does not have remaining late day points, late hand-ins will incur a 10% penalty per day (up to three days per assignment).
• No assignments will be accepted more than three days after the deadline.
Mid-Term
• Mid-Term is worth 15%. • Will take place just before Spring Break on the 9th of March. • Focus will be on geometry and matrix calculus, and
theoretical concepts in vision.
Real camera image is inverted
Instead model impossible but more convenient virtual image
Final Project
• Final project is worth 45% of final grade. • 5% for project proposal & checkpoint. • 40% for final project report and presentation.
• Teams 1-2 (if it is something big we could discuss 3). • Topic: efficient or novel implementation of CV algorithm on a
mobile device. • If you have a MAC, project must be in iOS. (special
exception for Android can be made in rare cases). • See 16423.courses.cs.cmu.edu/ideas for project ideas. • Until March 26th,
• think about a topic • find a partner.
Project Ideas
See more ideas at 16423.courses.cs.cmu.edu/ideas
Background Material
• Most other parts of course cannot be found in books.
• I post all slides, and notes in the course on the course website.
• If you are completely new to OpenCV and Xcode you should consider getting this book too (link to Amazon.)
• Good beginners guide to using OpenCV in Xcode, so you can build up additional experience during the course.
Background Material
Resources
• You will need access to a MAC. • If you do not have a MAC, do not panic CMU has ample
MAC clusters on campus. • See:- https://www.cmu.edu/computing/clusters/facilities/index.html • We have iPADs for everyone in the class so that is cool
(yay!!!) so everyone should have an iOS device.
If you have a MAC
• Please ensure your MAC has the latest version of El Capitan. • Please ensure your iOS device has the latest version 10.2. • This will make life easy for you (less headaches for me).
Class Participation
• I’ll start on time. • It is important to attend.
• I will use part slides, part tutorial, part on board.
• Do ask questions. • Use Piazza or come to my office (by appointment only).
Today
• Course Logistics
• Philosophy to Mobile Computer Vision R&D
OpenCVMATLAB
OpenCVMATLAB
OpenCVMATLAB
Why is Mobile CV Different?
27
Why is Mobile CV Different?
27
Why is Mobile CV Different?
28
Why is Mobile CV Different?
28
Why is Mobile CV Different?
28
Connection to Robotics…..
29
Balancing Power versus Perception
30
Balancing Power versus Perception
31
Low Power Image Recognition Challenge• “Many mobile systems (smartphones, electronic glass,
autonomous robots) can capture images. These systems use batteries and energy conservation is essential. This challenge aims to discover the best technology in both image recognition and energy conservation. Winners will be evaluated based on both high recognition accuracy and low power usage.”
33
?
Moore’s Law
35
Moore’s Law is Dead??
36Source: “Moore's Law Is Dead! (But Not In Mobile)” ReadWrite, April 2015.
Not in Mobile!!!
37Source: “Moore's Law Is Dead! (But Not In Mobile)” ReadWrite, April 2015.
