View
347
Download
0
Category
Preview:
Citation preview
Basic Study on Life-‐‑‒logging Video Capture via Neuraltalk22016/1/15
-‐‑‒ Research Note -‐‑‒
About myself
• Motohiko Takeda(@kusojig / takeda@rabbitsdata.jp )
• Interests:• Data Based Marketing• Machine Learning, Data Mining• Data Strategy Planning & Execusion
• Currently freelance consultant for data utilizationand sensor technologies. Mainly experienced consulting projects for mobile, baking and advertising industries.
Motivation: to estimate the time for housekeeping
• Some households have conflict about burdensharing about housekeeping.
• I spend a lot of time in kitchen…• Thatʼ’s not true. I also help dishes!• Oh it doesnʼ’t take much time. I pay more!
• OK, So why donʼ’t we analyze housekeeping cost quantitatively by using technologies?
Approach: Computer Vision with AI (Neuraltalk2)
• Set web camera at the top of dinning room and take photos for each 15 seconds.
• Caption the picture by Neuraltalk2(Neuraltalk2 gives caption by describing the picture).
• Estimate housekeeping time by categorizing the caption data.
Setup WEB camera around the celling and regulated by PC
Kitchen Dinning table
Refrigerator
Overview of room and camera
Camera keeping• Camera connected with laptop PC via USB
Camera and laptop PC
Caption results were almost collect except detailed actions• Neuraltalk2 recognized “a man/woman standing in a kitchen” when somebody stands around the kitchen.
• However, the same capture were given when somebody is sitting in the dinning table near the kitchen.
• It could not recognized more detailed action like opening the refrigerator.
Estimated time of staying around the kitchen could explainthe amount of activity in a day
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23
Estimated time of “man/woman standing in a kitchen” per hour
Prepare for tomorrow meal* 48min/hour at a peak
Wake up
Prepare for a supper
Lunch andhaving a tea
• Largely measured the day activity around the kitchen.
Supper
* Stayed home for whole day
Issues: individual identification and intersection
• Issues 1: individual identifiation• We have not implemented individual identification.• For identifying spent time for each task, implementation of individual identification including detection of side-‐‑‒face is necessary.
* In case of husband/wife identification, only gender identification might be enough.
• Issues 2: Intersection • Pictures in house generally happens intersection since room has small space compared with public space.
• Attention towards the tilt of camera is needed.
Issues: individual identification and intersection
Conclusion and future tasks
• Even no customized AI program can identify the activity time and patterns in the room largely.
• By implementing individual identification, time spent in housekeeping could be identified in particular.
• By taking life-‐‑‒log in more detail, we can develop the recommendation for daily life.
• This project is still on progress (Jan. 2016)
For more information…
• We have developing the research and development for sensor technologies, machine learning and deep learning.
• For more information, please feel free to contact: takeda@rabbitsdata.jp
Recommended