View
7
Download
0
Category
Preview:
Citation preview
CMPT888–HumanActivityRecognition
GregMori
Outline
• Introtoclass• Administrativedetails
Overview
• Thisclassisaboutvision‐basedactionrecognition– Inputisimagesorvideos– Outputisdescriptionofwhatpeoplearedoingintheimages/videos
ActionRecognitionExample
• Recognizehumanactionsfromrawvideodata
Gatheringactiondata
• 3components:– detecthumans,track,recognizeaction
6
ApplicationsI• Automatedvideosurveillance– Drawattentiontoactionsofinterest– Savehumanoperatortime
ApplicationsII
• Collectdataonpedestrianbehaviour– CollaborationwithSaunierandSayed(UBCCivilEngineering)
ApplicationsIII
Automaticallydetectfalls,near‐falls(withS.RobinovitchSFU)
WhyuseComputerVision?
• Competingapproaches– Wearablesensors
– Manuallabour
• Non‐intrusive– Donotneedcooperativesubjects
• Inexpensive,nooperatorfatigue– Semi‐automatictechniques
PROBLEMDEFINITION
WhatisActionRecognition?
• Terminology– Whatisan“action”?
• Outputrepresentation– Whatdowewanttosayaboutanimage/video?
Unfortunately,neitherquestionhassatisfactoryansweryet
Terminology
• Theterms“actionrecognition”,“activityrecognition”,“eventrecognition”,areusedinconsistently– Findingacommonlanguagefordescribingvideosisanopenproblem
TerminologyExample
• “Action”isalow‐levelprimitivewithsemanticmeaning– E.g.walking,pointing,placinganobject
• “Activity”isahigher‐levelcombinationwithsometemporalrelations– E.g.takingmoneyoutfromATM,waitingforabus
• “Event”isacombinationofactivities,ofteninvolvingmultipleindividuals– E.g.asoccergame,atrafficaccident
• Thisiscontentious– Nostandard,rigorousdefinitionexists
OutputRepresentation
• Thisimagecontainsamanwalking– Actionclassification/recognition
• Themanwalkingishere– Actiondetection
• Giventhisimagewhatisthedesiredoutput?
OutputRepresentation
• Thisimagecontains5menwalking,4jogging,2running
• The5menwalkingarehere
• Thisisasoccergame
• Giventhisimagewhatisthedesiredoutput?
OutputRepresentation
• Giventhisvideowhatisthedesiredoutput?
• Frames1‐20themanrantotheleft,thenframes21‐25heranawayfromthecamera
• Isthisanaccuratedescription?• Arelabelsandvideoframesin1‐1correspondence?
ChallengesinRecognition
• Intra‐classvariation• Objectposevariation• Backgroundclutter• Occlusion• Lighting
TRIMESTERPREVIEW
Week2
• Preliminaries– Humandetection
– Backgroundsubtraction– Opticalflow
Dalal+TriggsCVPR05
Weeks3‐4
• MotionTemplates
BobickandDavisPAMI01
Efrosetal.ICCV03
Weeks5‐6
• Localfeaturevideorepresentations
Schuldtetal.ICPR04Dollaretal.VSPETS05
Week7
• Unsupervisedandweaklysupervisedmethods
Laptevetal.CVPR08
Week8
• Temporalmodels
?? ? ? ? ? ? ? ? ?
WangandMoriPAMI09
Week9
• Humanposeestimationandposeretrieval
Yangetal.CVPR10
Week10
• Discriminativemethods
Runright Walkleft Runright45
FathiandMoriCVPR08
Week11• Humanactionsinstillimages
SLAGWangetal.CVPR06
ADMINISTRIVIA
CoursePlan
• Readresearchpapers– ForeachtopicIpresentimportantpapers
– Studentseachpresentarecentpaper– Wediscuss
• Doaproject– Gainin‐depthexperienceonaproblemandalgorithm
Introductions
Prerequisite
• Noformalprerequisites– Butitwouldbebestifyouknowsomecomputervision/imageprocessingandsomemachinelearning
• Youwillneedtodotheusualthings– Math(continuous),programming,reading,writing,presenting
• Askmeifyouareconcerned
GradingScheme
• 10%Classparticipation– Participateindiscussionsaboutpapers,ask/answerquestions
• 10%Readingassignments– 1or2paperseachweek;subsetoftheonesIpresent
• 10%Paperpresentation– Choosefromlistofpapersonline
• 10%Assignment– Smallprogrammingassignmentonmotionanalysis
• 60%Project– Individualorinsmallgroups– Presentation,writtenreport
ReadingAssignments
• Similartominipaperreview– Oneparagraphsummarizingpaper
– Criticaldiscussion(whatyoulike/don’tlike)– Questionsyouhave(formetoexplain)
• Duebeforestartoflectureviaemail– FirstonedueMonday
• Thesedetailsandlistofpapersareonline
PaperPresentations
• Chooseonepaperthatinterestsyou– Fromlistonline/insyllabus
• 20minutepresentation– 10+minutesquestions/discussion– Feelfreetouseslidesprovidedbyauthors
Assignment
• Shortprogrammingassignment– Backgroundsubtraction– Motion‐basedactionrecognition
• Outnextweek,due2weekslater
Project
• Majorcomponentofcourse– Recognizeactions
• Implementexistingtechnique– Orvariantthereof– Canusesomethingyou’reworkingoninyourresearch• Mustrecognizeactions• Mustdosomethingthatdidn’texistbeforethiscourse
• Proposal,presentation,report
CoursePlan
• Nextweek– Preliminaries
• Backgroundsubtraction,humandetection,motion
• Afterthat– Papers,papers,papers
Recommended