8/14/2019 Smart Ring: Controlling Call Alert Functionality Based
on Audio and Movement Analysis
1/2
Smart Ring: Controlling Call Alert Functionality Basedon Audio
and Movement Analysis
Hamed Ketabdar
Quality and Usability Lab, Deutsche Telekom
Laboratories, TU BerlinErnst-Reuter-Platz 7, 10587, Berlin,
Germany
[email protected]
Kamer Ali Yksel
Technical University of Berlin
Ernst-Reuter-Platz 7, 10587, Berlin,
[email protected]
ABSTRACT
In this work, we present a method for controlling call
alertfunctionality in mobile phones. It has happened for almost
everybody experiencing a situation that call alert
functionality is not proper for actual ambient context,leading
to missing a phone call or disturbing others by a
loud ring. In this work, we use audio and physical
movement analysis to distinguish between different
situations in which a mobile phone may ring, and adjust thecall
alert functionality accordingly. Considering the factthat mobile
phones are usually carried in a pocket or bag,
capturing ambient audio is not usually practically perfect.
The novelty in our work is using information about
physicalmovements of user of mobile device in addition to
analysis
of ambient audio. Analysis of user movements is based on
information captured by acceleration sensors integrated in
mobile phone. The call alert functionality is then adjusted
based on a combination of ambient audio level and physical
activities of user.
Author Keywords
Call Alert Functionality, Ambient Audio, Physical
Movements, Acceleration Sensors, Ambient Context
ACM Classification Keywords
I.5.4 [Computing Methodologies]: Pattern
Recognition,Applications Signal processing.
General Terms
Algorithms
INTRODUCTION
Mobile phones have become an essential in our lives. They
are used frequently in many situations and places ranging
from home, work, outdoor, partying, restaurants, etc.
Besides such a widespread use of mobile phones, someunpleasant
experiences are also associated with them.
Almost everybody has experienced missing some important
calls, or disturbing others by a very loud phone ring in an
improper situation [1]. These unpleasant cases are usuallydue to
the fact that the call alert functionality is not proper
for the actual situation of mobile phones user. If the call
alert is in vibration mode, or too low volume ring, the
user may not realize the call alert if he is in a noisy
place
such as a restaurant or party, or just walking in a crowded
street. The user may decide then to increase the ringvolume,
however this does not solve the problem. Soon
when he is back at work or a quiet place, the loud ring
starts
disturbing his colleagues! A potential solution for this
problem would be adjusting call alert functionality based onthe
actual situation of user and ambient context.
There has been research on context aware mobile devices
which can adapt to actual context [2, 3]. Controlling call
alert functionality can be a case of such
adaption/adjustment. Contextual data is these approachesare
usually collected using several external sensors (e.g.
microphone for ambient noise, acceleration sensors for
movements). However, using internally integrated sensoryinputs
(such as embedded microphone) can be challenging,
and on the other hand more practical as it does not impose
wearing extra sensory units. For instance in [4] ambientnoise
level is measured using embedded microphone in
mobile device, and loudness of the noise is then estimated.The
call alert functionality is then adjusted based on a
reverse relation with loudness of ambient noise. However,
the main practical drawback in such an approach (usingembedded
microphone) is capturing ambient audio. As the
phone is usually carried in a pocket or bag, audio
information captured by the mobile phones microphone is
not always a proper representative of the ambient audio.
According to our experiments, a majority of captured audio
content is components caused by friction between the phoneand
materials inside a pocket or bag. These components can
practically hide ambient audio information. This issue can
become more problematic especially when the user is
engaged in physical activities such as walking.Additionally,
physical activity itself may also affect user
perception of call alerts.
Considering above argument, in this work we propose a
method for adjusting call alert functionality (using
embedded sensors in a mobile phone) which takes intoaccount
physical activities of user, in addition to ambient
audio analysis. Information about physical activities of
user
is captured using acceleration sensor integrated in mobile
Copyright is held by the author/owner(s).IUI10, February 710,
2010, Hong Kong, China.ACM 978-1-60558-515-4/10/02.
415
8/14/2019 Smart Ring: Controlling Call Alert Functionality Based
on Audio and Movement Analysis
2/2
phone. The alert is adjusted based on a weighted
combination of activity level of user (movement level ofmobile
device) and ambient audio captured by embeddedmicrophone. Ambient
audio gains a higher share when
mobile device movements are insignificant, and hence the
captured audio content is more reliable.
ADJUSTING CALL ALERTS
As already mentioned, we use a combination of audio data
captured by microphone, and movement related data
captured by accelerometers integrated in mobile
phone.Accelerometer sensor which is integrated in many new
mobile devices (e.g. iPhone, G1 Android, etc.) captures
acceleration of device along the three axis. It is
traditionallyused for rotating screen when the phone is
rotated.
Acceleration data can also deliver some information about
movements of the device, and user movements assuming
that the device is carried by the user. Activity level of
user
can affect his perception of call alerts, as being engaged
in
high physical activities such as running or walking canincrease
the chance of missing calls.
Combination of audio and movement related data provides
a score which is used for call alert adjustment. Call alert
strength has a reverse relation with this score. Incombination
of audio and movement data for call alert
adjustment, the basic idea is to weight ambient audio data
based on its reliability. As a mobile phone is usually
carried
in a pocket or bag, audio data during carriage can beheavily
biased by the components caused by rubbing
against materials inside the pocket or bag. In contrast,
ambient audio can be captured in a more reliable when the
phone is outside e.g. on a table.
The reliability is determined based on movement level of
device, i.e. activity level of its user assuming that the
phoneis carried by the user. If movement of the device is not
significant, we assume that audio data is reliable as a
measure of ambient noise. On the other hand, if movementof
device is significant, the weight of ambient audio data is
reduced in the adjustment process, and movement level of
device (user) is used as main factor for adjusting call
alerts.
For instance, a long continues walk of user can indicate
being outdoor, therefore need for stronger call alert. If
theuser is running, call alert should be even stronger!
Estimating movement level of device, and ambient audio
level are described in the next sections.
ESTIMATING MOVEMENT LEVEL OF MOBILE DEVICEIn order to estimate
movement level of mobile device, low pass components of the
acceleration signals which are
mainly due to gravity force are removed (high pass filter).
The magnitude of acceleration is then calculated using 3
axis acceleration components. The magnitude acceleration
is then processed over a window of 10 seconds, and some
features mainly based on average magnitude and averagerate of
change is calculated. A weighted sum of these
features is used as basis for estimating movement level of
device.
AMBIENT AUDIO ESTIMATION
Audio samples are obtained from microphone integrated in
the device. We use root-mean-square (RMS) of the audio
signal over a window of 10 seconds as a measure of
loudness.
IMPLEMENTATION
We have implemented the presented method on iPhone 3G
mobile device. Audio is captured at 8KHz, and
acceleration data is captured at 50Hz rate. The strength ofcall
alert for actual situation is obtained in reverse relation
with the score estimated from ambient audio and device
movement. In this relation, there is also a tuning factor
which can be adjusted to enhance the relation. If the callalert
strength is below a threshold, the alert functionality is
switched to vibration mode, otherwise the strength of call
alert is mapped to level of ring volume. The demoapplication
automatically simulates call alert situations
every 20 seconds, and the call alert functionality is
adjusted
accordingly. The demo application has been tested in
different indoor and outdoor situations providing
highersatisfaction with call alert functionality.
ACKNOWLEDGMENTS
We would like to thank Tim Polzehl, Mehran Roshandel
and Matti Lyra for helpful discussions.
REFERENCES
[1] Monk, A., Carroll, J., Parker, S., Blythe, M., Why aremobile
phones annoying? Behaviour & Information
Technology, Vol. 23, no. 1, pp. 33-41. Jan.-Feb. 2004
[2] Daniel Siewiorek, Asim Smailagic, Junichi Furukawa,
Neema Moraveji, Kathryn Reiger, and Jeremy Shaffer,
SenSay: A Context-Aware Mobile Phone, Proceedings
of the 7th IEEE International Symposium on WearableComputers,
2003.
[3] A Khalil, K Connelly, Context-aware Configuration: A
study on improving cell phone awareness, Lecture
Notes in Computer Science, 2005, Springer.
[4] Chris Mitchell, Adjust Your Ring Volume For Ambient
Noise, MSDN Magazine, October 2007.
416
