Energy Efficient Collaborative Sensing-Based Design: Soft Keyboard Case Study

IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 1, FEBRUARY 2014 115

Energy Efficient Collaborative Sensing-BasedDesign: Soft Keyboard Case Study

Mahsan Rofouei, Miodrag Potkonjak, and Majid Sarrafzadeh

Abstract—With our approach, for the synthesis and energy-efficient operation of a pressure sensor-based soft keyboard, energyis optimized at five levels of abstraction: 1) architecture of sensorsystems, 2) sensing schedule, 3) sensor data processing, 4) use ofsemantic information, and 5) customization. We have been able toachieve up to a factor of 63 energy reduction in a keyboard casestudy over the standard using various techniques. For example,when each sensor senses several keyboard events and each event issensed by three sensors, sensing and data processing results in a 12-to 42-fold energy savings depending on typing speeds. Customiza-tion techniques accounting for user typing speed and duration oftyping pulses support another 1.5-fold savings. We demonstrate us-ing a proof of concept prototype evaluation with five participants.

Index Terms—E-Textiles, sensing, soft keyboard.

I. INTRODUCTION

THE productivity of the user can be greatly improved byproviding energy efficient, full-scale, and customizable

keyboards that are light, low cost, and easy to transport anddeploy. In addition, minimizing the health risks associated withshared keyboards, E-Textile keyboards enable much better se-curity and user authentication. Finally, the new keyboards canalso collect a variety of statistics about the user that can be usedto improve productivity and even possible detection of healthproblems. Our goal is twofold: 1) to design an E-Textile-basedportable keyboard and energy efficient algorithms for its use,and 2) to study methods and techniques for design of layers inthe sensing and processing stack of pressure sensor systems.

While one-to-one event-sensor mapping sensing seems to bea natural way to create soft keyboards, we show that a differentdesign structure coupled with appropriate data collection andprocessing strategy is actually more energy efficient. Samplingthe pressure for each keyboard key using a separate sensor isnot energy efficient unless each key is equipped with equipmentthat can send an interrupt to the operating system. If a pollingcommunication strategy is employed, it is crucial to separatetwo events: one that indicates that any key is activated, and onethat indicates that a particular key is pressed. By polling a singleglobal sensor that indicates that any key is pressed, we can

Manuscript received April 29, 2013; revised August 6, 2013; acceptedNovember 5, 2013. Date of current version January 16, 2014. This paper wasrecommended by Associate Editor M. Jeng of the former IEEE Transactionson Systems, Man and Cybernetics, Part A: Systems and Humans (2012 ImpactFactor: 2.183).

The authors are with the Department of Computer Science, Univer-sity of California, Los Angeles, CA 90095 USA (e-mail: [email protected]; [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TSMC.2013.2290503

reduce the energy consumption. Once the event is detected, wequery individual keys. Motivated by this observation, we havedeveloped a sensing architecture that groups individual sensors.The partitions are organized in such a way that by iterativelyor simultaneously querying a small number of them, one candeduce which individual key is pressed.

The rate of sampling is dictated by two bounds. The first is aconsequence of the need to conduct sampling at least at a rate atwhich two consecutive keys are pressed. We address this issueby conducting a study of representative subjects. The secondbound is created by the need for additional sampling to identifya specific key using sensor that cover sets of geometrically con-nected areas below several keys. These two bounds are mergedso that the allocated time between two consecutive samplingsis sufficiently small to allow for the highest expected speed oftyping. We next describe how we have addressed the sensingand processing stack of common embedded sensing systems.

A. Sensing System Architecture

Our sensing architecture follows binary, many-to-many, andoverlapping sensing strategies. We characterize each signal todeduce if it is or is not below an empirically measured thresholdthat indicates a particular event. We conduct additional mea-surements that are used for both increasing the robustness andidentifying events (specific key pressing). Each key on our key-board is exposed to two sensors in such a way that for eachkey we have a unique pair of sensors. The simplest potentialrealization of this type is a matrix of horizontal and verticalsensors.

B. Interleaved and Coordinated Data Collectionand Processing

In many sensor networks, applications data collection andprocessing are two separate phases. However, in the case of thesoft keyboard, we merge these phases to enable flexibility andefficient search for event identification (diagnosis). We query aparticular combination depending on the results of the alreadyqueried sensors. This strategy is possible only when data pro-cessing is faster than the maximal sampling rate. In our case,we created this situation by restricting signals to their binaryvalues.

C. Event Detection and Diagnosis

We separate event detection and diagnosis. This decision ismandated by highly nonuniform time between pressing twoconsecutive keys. Thus, event diagnosis is only performed whenan event is detected.

2168-2291 © 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

116 IEEE TRANSACTIONS ON HUMAN-MACHINE SYSTEMS, VOL. 44, NO. 1, FEBRUARY 2014

D. Semantic Information

We identified two types of relevant semantic information thatcan facilitate reduction of required energy. The first is any mea-sure defined over the set of typed characters. Specifically, inour soft keyboard realization, we use two simple measures: fre-quency and conditional frequency of characters. The measureenables us to better organize the search for the pertinent key bysearching first for the keys that are more likely to be activated.

The second semantic measure is related to physiology andtyping-level proficiency, and has two components: 1) a min-imum time between consecutive activation of two keys (ofcourse, the keys may be identical) and 2) a conditional mea-sure, where we estimate and bind the minimal time that any keyis activated after each particular key is pressed. This minimaltime guides our sensor querying (sampling) schedule.

E. User Customization

Because different individuals have significantly differentspeeds and characteristics of typing, our second semantic mea-sure, minimal time between two consecutive activations, is cus-tomized for individual users and supports the creation of cus-tomized sampling schedules.

The rest of this paper is organized as follows: Section IIpresents related work in sensor networks, sampling strategies,typing, and keyboard design. Section III discusses E-Textilesand their characteristics. Sections IV and V describe the archi-tecture design and processing methodologies that we use for theE-Textile-based keyboard. Experimental results are provided inSection VI. Conclusions appear in Section VII.

II. RELATED WORK

Energy use is a critical system constraint in the design ofsensor networks [17], [18]. The authors of [21] and [22] haveaddressed sampling problems related to event-driven sensorsampling activation. Wan and Lemmon [23] discuss distributedevent-triggered sampling strategies, and the authors of [24]and [25] have developed techniques for dynamic sensor nodesso that required energy is minimized.

With respect to typing characteristics, keyboards, soft key-boards, and the use of semantic information for keyboard op-timization, Fitt’s model and extensions [2]–[5] have been usedto show human performance relationships related to movementsuch as a high correlation (often above 0.8) between the Fitt’sindex of moving difficulty and actual measured time to com-plete a task. The soft keyboard design [7], [8] and the use ofsemantic information for more effective text entry [9], [10] andkeyboard optimization techniques [11]–[14] continue to showhuman performance benefits.

E-Textiles have been introduced as a platform for pervasivecomputing [27]. Del Prete et al. [26] discuss a study on metro-logical properties of E-Textiles and provide pressure–resistancemodels for various E-Textile materials. Energy efficient routingin E-Textile applications [6] and designing efficient sensor struc-tures and sampling techniques for E-Textile-based systems [28]are active research areas.

Fig. 1. Resistance response as function of applied force.

To the best of our knowledge, this is the first effort to create asoft keyboard using smart textile and pressure sensors. It is alsothe first study that targets low energy soft-keyboards. Finally, thenew approach is the first that uses global sensing for detectionof localized events.

III. E-TEXTILES AND THE SENSOR MODEL

E-Textiles are composite yarns made of fibers coated with aconductive polymer. E-Textiles’ foldable characteristics makethem suitable for many portable and wearable applications.Next, we describe how E-textiles can be used to create pres-sure sensors, and how one can model their characteristics in thepresence of pressure.

A. Sensor Fabrication

An E-Textile-based sensor has a three-stacked layer structure.The sensing material is sandwiched between two conductivepads. E-Textile acts like a pressure sensitive resistor. When forceis applied, the resistance of E-Textile decreases. The conductivelayer can be made of conductive fabrics, copper foil tape, orconductive threads. To maintain the flexibility of the sensor andtextile feel, we use conductive thread and conductive tape forthe conductive layer.

B. Sensor Pressure–Resistance Characteristics

We model the force–resistance relationship as shown in Fig. 1.Specifically, we show the curve for forces below 10 N for a 2 cm× 2 cm sensor. We use this curve to obtain resistance valuescorresponding to typing forces.

C. Sensor Sensitivity With Distance

In fabricating a sensor array for mass production, the mid-dle sensing layer is shared among all array sensors (Seesection IV-A. for details). One ramification is a dependenceamong readings of close sensors. When applying force on onesensor, a pressure is imposed on its surrounding sensors due tomechanical linkage. Therefore, we calculate the resulting pres-sure sensed on each location as both the direct and indirectforces. Hence, our model includes the neighboring effect.

ROFOUEI et al.: ENERGY EFFICIENT COLLABORATIVE SENSING-BASED DESIGN: SOFT KEYBOARD CASE STUDY 117

Fig. 2. Multilayer sensor array design with K layers and a maximum of m × nsensors on each layer.

IV. ARCHITECTURE DESIGN AND PROCESSING

METHODOLOGIES FOR E-TEXTILE-BASED SYSTEMS

A. Sensing System Design: Solution Space Exploration

The main concern in designing sensing systems is to createa sensing architecture that captures all events and requires min-imal energy. The common practice is to design a sensor arrayin which a single sensor is assigned to each sensing locationwith its dedicated signal line. However, energy savings may beaccomplished through alternative sensing architectures.

In an n by m sensor array, where each sensor has its ownsignal line, m × n sensor readings are required to determine thelocation of asserted pressure. An alternative design mechanismis one where sensors on the same row/column share the samewire. In this architecture, a voltage is applied at the columnof interest and the current at each row is measured. Using thisreading mechanism, the entire array is scanned column by col-umn. This design scheme makes the routing much simpler sincesensors on the same row/column share the same wire, but stillneeds n × m readings.

We propose a multilayer design, where each layer consistsof several sensing elements. The sensor array is constructed byplacing the layers by considering the virtual locations of eachsensor in other layers. Pressure asserted at the top layer is sensedin the layers below due to the nature of E-Textiles. Calibrationcan be performed to obtain equivalent pressure at all levels. Thisenables reconfigurable design based on application needs. Fig. 2shows this multilayer design with k layers and the potential ofmaximum m × n sensors in each of the layers. However, basedon specific design constraints and requirements, sensors in eachlayer can be combined to enable different sampling schemesand to form different size sensors. For example, they can becombined to produce coarse grain sensing at locations wheredetailed data are not needed (e.g., all sensors in one layer can beconnected to form one sensor). Fig. 2 shows several examplesof such sensor combinations.

Using a higher level view of sensing elements in each layer,Fig. 3 shows one way of constructing this sensing structure

Fig. 3. Two-layered design. (a) Layer arrangement. (b) Virtual view of sensingarray and sensing locations.

where each layer’s sensing elements are combined into parallelsensing elements. Each sensing element, here, is the combina-tion of multiple square sensors in one line. The placing of twolayers is in such a way that the sensing elements on the layersbecome orthogonal to each other. The touching layer of thesesensors is connected to Vcc and signals are read from each ofthe sensors on each layer. This design mechanism supports cre-ating a sensing circuit with more sensing locations per sensorelement and thus requires fewer readings. For n vertical and mhorizontal sensing elements, only n + m readings are requiredfor n × m sensors.

The spacing between sensing elements can be chosen basedon desired effect of sensors on each other. Distance betweensensors can be chosen so large so that the neighboring effect isnegligible or chosen small enough so that only first neighborsof the sensor are affected. Sensing elements can be either drawnor sewn on to the virtual locations.

The inclusion of additional layers can either enable additionalinformation processing techniques or provide features such asreliability for the sensing system. For example, two identicallayers can be constructed in order to provide consistent resultsin the case of failure of sensors on one of the layers. However,the design and decisions on combining sensors in each layeris nontrivial due to geometric constraints such as preservingthe neighboring arrangement of sensors as discussed in SectionV-B.

1) Group Sensor Readings: Each sensor has a dedicated sig-nal which is multiplexed and then read through A/D converters.However, in some situations it might be beneficial to combinesensors in groups and read through one A/D converter in orderto save energy. One situation was described previously. Othercases are when group readings can enable effective queries byallowing readings from a combination of sensors. To enablethis capability, sensor readings can be combined as neededusing electronic switches. For example, this can be accom-plished using summing amplifiers as they keep the interactionbetween inputs at a minimum and produce an output voltage


proportional to the algebraic sum of their inputs. The numberand size of summing amplifiers in the sensing circuit is staticallydetermined depending on application requirements.

B. Decoupled Event Detection and Identification

We decouple event detection from event identification. Tra-ditionally, in order to find the location(s) of asserted pressure,readings of all sensor values at each sampling period are re-quired. Consequences include high energy requirements andextremely fast sampling rates in order not to miss any events.With the use of group sensor readings, we first create event de-tection sensors where readings from all sensors are combined.Hence, only one reading is required at each sampling period.Additional sensor readings are only performed in the case of anevent. In this scheme, the energy saving comes from reducedsamples (use of A/D converters) while the energy consumed ata sensing subcircuit remains the same.

Specifically, the design in example of Fig. 3 would changeto a three-layer design where the third layer is a single sen-sor with the size of the whole sensing array and is in chargeof detecting any event (key activation). Next, we present tech-niques that reduce energy consumptions in this phase of signalprocessing.

C. Interleaved and Coordinated Sampling and DataProcessing

We use a new paradigm of interleaved and coordinated sam-pling and data processing to obtain flexibility in managing sam-pling strategies for energy minimization. To avoid building acomplex system of equations and to enable fast and robust rea-soning, we use a binary sensing scheme. Our sensor architectureis designed to enable us to conduct processing using binary in-formation about sensor readings due to an experimentally con-firmed sensing model (described in Section V-D). Therefore, forevent identification, we use combinatorial search that rapidlyeliminates possibilities that are inconsistent with the measure-ments from further consideration.

Only a subset of close sensors detects pressure changes. Weperform a binary mapping from sensed values using a platform-dependent threshold. Values above this threshold are mapped to1 (0 otherwise). A group reading of n sensors after the binarymapping stage will result in a “1” if one of its n sensors isequal to 1. We take advantage of the previously discussed groupreading ability and binary representation of sensors to minimizethe number of queries needed to identify events.

For each event detected, depending on the expected availabletime to perform processing, the number of sensor readings canbe reduced. If the expected available time allows two epochs(units of time) to process, using group readings, we can narrowdown the search space by a factor of two, three, or more in thefirst epoch and then perform search in the reduced search spacein the next epoch. This strategy can be extended to more epochcalculations using the same reasoning. Consider an example oftwo epoch processing of eight sensors. In the first epoch, weperform a single group reading of four sensors. If the result is 1,

Fig. 4. Sampling points A, B, and C shown on the event detection signal.

we conclude that the activated sensor is within that group. Sim-ilarly if it is 0, it should be in the other group. Based on thisresult, we perform search only on a group of four. Therefore, inthis example, instead of performing eight measurements we onlyperform five. Different combinations and more epochs availablecan result in further savings.

The key question is how to determine the number of availableepochs for measurement and reasoning. This number dependson the event detector sampling mechanism. A very fast samplingin the event detector sensor will capture events early enough toenable multiple epoch processing. Sampling at a very slow rate,that is, only fast enough to capture the presence of events, wouldallow one epoch processing. Consider the example in Fig. 4 withan event happening in the event detection signal where point Cis the end of the event. If the event detector sampling can assurethe discovery of an event at an early point such as A, the timeavailable for event identification is tAC . However, if the eventdetector sampling is only fast enough to guarantee to captureevents at late points such as B, the remaining time for eventidentification becomes only tBC .

To minimize the overall energy consumption of the sys-tem, we need to solve an optimization problem. More avail-able epochs enable more optimized measurements and, there-fore, less energy consumption in the event identification phase.On the other hand, increasing the number of epochs requires ahigher sampling rate on the event detection sensor and, there-fore, higher energy consumption in the event detection phase.This optimization problem can be solved depending on differ-ent parameters and constraints of applications. In the E-Textilekeyboard examples, these parameters consist of time of typingsession and duration of a typing pulse.

D. Exploiting Domain Specific Information

The use of semantic interpretations can enable optimizationsboth at the architecture and algorithm design levels. Domainknowledge provides information on rate of events, durationof events, location of events, and conditional probabilities ofevents. Rate of events dictates the sampling rate required for de-tection. These data, in conjunction with the duration of events,show how much processing time is available to perform inter-leaved sampling and data processing. Conditional properties ofevents provide information on how the sensor groupings can beperformed. Location of events on the sensing array impacts the


Fig. 5. System prototype composed of client and host part.

sensing circuit design. Studying user activity patterns can alsoprovide information to optimize querying mechanisms. Finally,customization can be considered as semantic information thatis related to a specific user and can also be used to optimizesampling strategies and minimize energy. Customization tech-niques used in the design of an E-Textile keyboard are describedin Section V-E.

V. PORTABLE E-TEXTILE KEYBOARD DESIGN

In this section, we use the strategies presented in Section IVto design and implement an E-Textile-based foldable keyboard.We describe the prototype of the new keyboard. We also presentthe power consumption model that is the basis for our energyconsumption measurements.

A. Power Consumption

1) Prototype Description: Fig. 5 shows that our prototype iscomposed of a client part and a host part. In the client part, thepressure sensor array is scanned with the use of multiplexers.After the sensing data are acquired by the microcontroller, theyare packaged and transferred to the host receiver side through awireless RF circuit. The microcontroller used is MSP430f2274.The resolution of the A/D converter is 10 bits. The samplingrate is 10 Hz. The communication protocol used to transferthe data is SimpliciTI that uses CC2500 as the wireless com-munication chip. The SimpliciTI network protocol is a pro-prietary low-power radio frequency protocol targeting simpleand small RF networks. This network protocol can be consid-ered a complement to ZigBee and is more suitable for largernetworks. The client is connected to the sensing circuit. Typ-ing information is transferred wirelessly to the host side (e.g.,a smart phone, PDA, or a laptop) where it is received by theRF chip.

Since the applications of this system and similar systemswould be in portable/wearable devices, we chose low-power,lightweight components for our prototype design. We specif-ically selected the EZ430-RF2500 module from Texas Instru-ments for our processing and data transfer.

The prototype (see Fig. 6) is powered by two AAA batteries.A UART to USB converter (based on MP2010) is designed forinterface compatibility.

2) Power Consumption Model: The total power consump-tion is composed of the power consumed in the sensing circuit

and the transmission subsystem as follows:

PT = PSens + Pcom . (1)

The sensing power (PSens) consists of the power driven bythe sensing circuit and sampling power. The former is calculatedusing resistance of asserted pressure. The latter is proportional tothe power consumed by ADCs as each sensor reading is througha 10-bit ADC. As it can be seen in Table I, PSens is dominatedby ADC power. Hence, group readings are beneficial as theyonly employ a single ADC unit. In the event of group readings,the power consumed by summing amplifiers which is in thesame order of sensing circuit is also added. Table I is calculatedaccording to datasheets [15], [16].

Since data transmission power is similar to the ADC power,we see that localized processing is beneficial and that minimiz-ing the number of sensing readings dominates overall energyconsumption.

B. Sensing Architecture Design

Using a two-layer design (see Fig. 3) requires 17 columns andsix rows to cover all keys. Rows and columns can be scannedsimultaneously to identify the location of the pressed key. Wesimplify the querying mechanism to consider the extreme caseof requiring one epoch (unit of time) for each reading. The num-ber of queries is a linear function of the number of sensors ineach group. Therefore, either columns or rows will dominate thequerying time depending on the number of sensors. Our proto-type needs 17 columns and six rows, therefore, the queryingis dominated by the columns. However, balancing the numberof sensor in each group is more energy efficient. Note that 17columns and six rows require 23 readings.

Consider a balanced architecture for sensing 102 keys(17× 6), where ten rows and 11 columns are used. Now, only 21readings are required. However, due to geometrical constraintssuch as the need to preserve the neighboring arrangement of eachsensor, a balanced design of ten and 11 is not feasible. Fig. 7shows the sensor arrangements of nine and 12 columns and rowsthat produce a semibalanced architecture. Fig. 7(a) shows thearrangements of 12 rows instead of the original six rows. Thislayer would become the bottom layer of the keyboard. Note thatthis arrangement results in a unique pair of sensors for each key.Fig. 7(b) shows the arrangement of nine columns. Imagining avirtual line in the middle of the layer, each two sensors fromeach side are merged to create one sensor. This imaginary lineis aligned with the line breaking the six rows in 12 rows in thebottom layer. Therefore, from Figs. 7(a) and 7(b), we can seethat we have eight columns with 12 rows and one column withsix rows.

The widths of the sensing elements are similar to those foravailable keyboards (around 1.3 cm). The choice of the dis-tance between parallel sensing elements impacts the detectionalgorithms. We choose the distance between consecutive sens-ing elements so that the neighboring effect is observable onlyon its direct neighboring sensors. In this design, a key presswould result in value changes on six readings (three columnsand three rows). By conducting experiments with five sensors


Fig. 6. Prototype keyboard.

Fig. 7. Balanced Sensor Array. (a) Rows. (b) Columns.

TABLE IENERGY CONSUMPTION OF SUBMODULES

placed within equal distances from each other and pressure as-serted on the middle sensor, we found that placing the center ofsensors within 1.5-m spacing produces the desired neighboringeffect. This distance was modified until the asserted pressurecould only be sensed on the direct neighbors of the middle sen-sor. Note that our balanced design still needs only 21 (12 + 9)sensors readings.

C. Decoupled Event Detection and Identification

Using a summing amplifier, we add all the signals from thecolumns to create an event detector signal. Fig. 8 shows the event

Fig. 8. Event detector sensor, column 3 and 4 readings while pressing “A” and“S” keys consecutively. X- and Y -axis represent the number of samples and thepressure sensed.

detector signal together with signals Column 1 and Columns 2(represented by V1 and V2). The continuous sampling is per-formed only on this signal, and further analysis on key iden-tification is performed only in the case of an event. The mainconcern is to capture events early enough to allow sufficienttime for event identification.

Once there is no event detected by the event detector sensorfor a specific amount of time (here, 5 min), the keyboard isconsidered idle and, thus, the event detector sensor stops sam-pling to save energy. For activation of the keyboard after an idleperiod, the user uses an activation button on the keyboard.

D. Interleaved and Coordinated Sampling and DataProcessing

Our prototype uses a sensing model where only the clos-est sensor and its two neighbors detect pressure changes.Fig. 8 shows data from two adjacent columns on the keyboard(V2 and V3). Keys “A” and “S” (each on one column), are pressedconsecutively. Due to the sensing model, in the case of activa-tion on V2 , a relatively smaller activation is observed on V3 andvice versa after applying the binary mapping stage; a single keypress will result in either a pattern of three consecutive “1”s or


Fig. 9. Four sensor example.

two consecutive “1”s at the boundaries. The binary representa-tion of nine columns in the presence of a single key press at twodifferent locations is

000011100 : Event in 6th column110000000 : Event in 1st column.

Using the interleaved sampling and the data processingmethod, if the available time restricts the reasoning to be per-formed in one epoch, instead of performing n sensor readingsto find a key press, reading every other sensor (plus 1, to coverboth boundaries in the case of an even number of sensors) willsuffice ([n/2]+1). For multiple epoch processing, this can bedone in the last step. For two epoch processing, after the searchspace is reduced by the order of two, three, or more in the firstepoch, in the second epoch, sensors can be measured in ev-ery other pattern. We assume that computations necessary forkey identification to be performed in one epoch what is easilyaccomplished on 1-MIPS processor.

We show the efficiency of group readings and multiple epochprocessing in improving the expected number of readings in thebelow example of four sensors (see Fig. 9).

In a 3 epoch available time, we arrange our readings asfollows:

Epoch 1: Group reading of sensors A and BEpoch 2: Reading of sensor DEpoch 3: Reading of sensor CIn epoch 1, if the result of the group readings of sensors A

and B is a “0,” we can conclude considering the neighboringeffect that the pressed key is D and we finish in the first epoch.If it results in “1,” we proceed to epoch 2 and perform a readingon sensor D. Using measurement results of epoch 1, a “1”reading from sensor D means that sensor C is the pressed keysince we sense its neighboring effect on its surrounding sensors.Therefore, in the case of C being our pressed key, readingsterminate in epoch 2. Otherwise, we proceed to epoch 3 wherewe perform a reading on sensor C. Using similar reasoning, a“0” and “1” reading indicates sensors A and B as the pressedkeys, respectively. Therefore, the expected number of readingsreduces to 2.25 using the calculation below

E(readings) =14× 1 +

14× 2 +

12× 3 = 2.25.

The traditional approach requires four readings. Using multi-ple epochs to read but not performing group readings will resultin the expected value of 2.5 readings.

Using similar reasoning, we have calculated using a oneepoch exhaustive algorithm, the optimal number of readingswith group reading arrangements for different epochs for bothrows and columns of our prototype keyboard. The expected

TABLE IICOMBINATORIAL SEARCH FOR COLUMNS AND ROWS

TABLE IIICOMBINATORIAL SEARCH FOR THE UNBALANCED ARCHITECTURE

number of readings required for rows and columns calculatedusing combinatorial search are shown in Table II.

The sampling mechanism in the event detector sensor deter-mines the number of epochs available for processing. We set thelongest interval without sampling in such a way that the time isminimal, but sufficient for both event detection and additionalmeasurements required for key identification (2). If TS repre-sents time of typing session and TP is considered minimumduration of a typing pulse, we can approximate the number ofsamples needed to be performed by the global event detectorsensor as follows:

NSamples =TS

TP − TAvailable. (2)

Until now, we assumed that a single key is pressed at a time.In some situations, a user may press two or more keys simul-taneously. We address the situation when two keys are pressedusing an optimal algorithm, similar to the case when a singlekey is activated. Resolving the situation when three or morekeys are simultaneously pressed induces high energy overhead.We leave this case up to the typist to correct such instances.Therefore, if three or more keys are pressed simultaneously nokey presses will be inferred. Table III shows the requirementsfor our combinatorial search methods for 17 columns and sixrows.

Comparison with Table II indicates that the new keyboardarchitecture is better in the first epoch. We actually have eightcolumns of 12 and one column of six sensors. Using the cal-culations in Table III for six sensors, we see that a 12 by 9arrangement is better than 17 by 6 overall. Higher energy sav-ings can be achieved if we change the column connection bysplitting one column to create seven columns of 12 sensors andtwo columns of nine sensors.

E. Exploiting Domain Specific Information

We now show how the use of semantic information can beused to optimize sensing strategies and how we perform cus-tomization.

1) Semantic Information: We exploit the semantic informa-tion of the typing language in order to further reduce the system


TABLE IVCOMBINATORIAL SEARCH WITH SEMANTIC INFORMATION FOR COLUMNS AND

ROWS

energy consumption. Results presented in Table II assume equalprobabilities of events per sensor. Using semantic information,columns and rows have unequal probabilities of being activated.We leverage this property to modify our querying scheme to findmore likely events at an earlier time and thus reduce the totalexpected required number of readings. Consider Fig. 9. Sincedifferent sensors have different probabilities of being pressed, itis beneficial to place the key with highest frequency in locationD so that only one reading is required to identify it. Similarly,the next key with highest frequency is placed in location C andso on.

We used the relative frequency of letters in English from [1]to calculate probabilities for rows and columns of our prototypekeyboard by adding probabilities of keys in each row/column.Using this frequency information, we found the optimal sam-pling strategy for different epochs. Table IV shows higher sav-ings accomplished in the rows compared with columns. Thereason is that, considering the standard layout of keyboards andour sensing architecture, keys are better distributed in the rowsin terms of their frequency of occurrence relative to columns.One can also consider the conditional frequency of letters toeach other to calculate search strategies.

2) Customization: Users of keyboards have different typing-level proficiency and typing characteristics. This informationcan be used to perform customization on the sampling strate-gies. Note that only the event detection phase is impacted. Weperform customization by exploiting typing speed informationto adjust the event detection sampling. We also leverage cus-tomized information about latency of the next event as a condi-tional value that depends on the previous event.

The minimum time between consecutive activations of twocharacters is beneficial in adjusting the event identification sam-pling. For example, for a typist with the speed of 1 char/s thereis no point in sampling every 0.1 s or less. From people used tostandard keyboards, we collected data for the minimum time be-tween two consecutive key presses using our prototype keyboardand found a range from 399 to 699 ms. We noticed relativelyslower typing speeds on our prototype keyboard compared withstandard keyboards which can be a minor limitation of suchkeyboards.

The conditional measure of the minimal time of an event aftera known event guides our sensor querying (sampling) schedule.We measured the minimal time of an event after each of thealphabet characters. Fig. 10 shows the minimum, median, andmaximum of this value for five participants who were askedto type a given text in a 2-min session. Fig. 10 indicates that

Fig. 10. Conditional measure of minimal time of an event after a known event.

TABLE VENERGY CONSUMPTION OF SAMPLING TECHNIQUES (NJ)

while delays after characters are different across users, somecharacters have relatively longer delays.

VI. RESULTS

We gathered typing data from five participants and performedstatistical analysis to find the necessary parameters for our pro-posed sampling scheme. One of these parameters is the durationof a key press pulse. Here, we consider the minimum pulse dura-tion computed over all people which is 250 ms. This parameteris used in determining the sampling frequency of the global sen-sor detector using (2). We compare four different strategies (seeTable V). Method 1 is energy results for nonintelligent samplingof all sensors at 10-Hz sampling intervals for key identification.This method is still better than a trivial nonoptimized methodsince it uses a near optimized sensing architecture that requiresonly n + m readings for n × m keys. The savings from Method 1stem from the methods in Section V-A. Method 2 uses statisticalinformation on duration of typing forces to reduce sampling in-tervals on the optimized sensing architecture. In Method 3, weuse our new combinatorial search in binary representation ofsignals with epoch time intervals. Methods 2 and 3 are based onconcepts described in Sections V-B and V-C. Method 4 (based onusing semantic information discussed in Section V-D) integratessemantic information into our combinatorial search mechanismtogether with statistical information about typing force dura-tions on our near optimized sensing structure. The time of eachsession was 20 s. To show the efficiency of our combinatorialsearch sensing, we show the different situations of Methods 3and 4 based on available epochs and their energy consumption.Tables VI and VII show that more epochs available result inlower energy consumptions.

As Table V shows, using our most optimized version thatuses four epochs, we are able to achieve savings from 12× tomore than 42× in energy consumption depending on the typingspeeds compared with the nonoptimized version (Method 1).Compared with Method 2 which uses statistical data of typing


TABLE VIENERGY CONSUMPTIONS FOR COMBINATORIAL SEARCH IN BINARY

REPRESENTATION (NJ)

TABLE VIIENERGY CONSUMPTIONS FOR COMBINATORIAL SEARCH USING SEMANTIC

INFORMATION IN BINARY REPRESENTATION (NJ)

TABLE VIIIENERGY CONSUMPTION OF CUSTOMIZATION SAMPLING TECHNIQUES (NJ)

force durations, we achieve 4× to 15× savings in energy con-sumption. As expected, for faster typists, the savings are lessthan for slow typists.

The lifetime of the system using Method 4 is more than 250 hwith two AAA batteries. This is significant energy reductionscompared with current available foldable keyboards [19], [20]which offer 90-h life time.

Our most optimized version is when we consider, customiza-tion in our sampling. We show two different customizationtechniques. First, we use minimum delay between charactersfor each person. This customization has more benefit for slowtypists (Customized A). In this method, we consider both the du-ration of typing pulses and minimum delay between key presses.In the second customization, we go further by using informationfrom conditional measures of the minimal time based on charac-ters identified (Customized B). The savings from customizationare because of more efficient sampling in the event detectionphase. Event identification energy remains the same. Table VIIIshows the result of customization on three of the users and com-pares it with Method 4 which was the most optimized methodwith no customization.

Table VIII indicates that we can achieve up to a further1.5× savings compared with our most optimized version be-fore customization. Therefore, considering altogether savings,we achieve up to 63× compared with the nonoptimized method,and 23× compared with Method 2. This means that theamount of energy expenditure for our most optimized version(Customized B) keyboard is 63 times less than the amount ofenergy consumed in the nonoptimized implementation of thekeyboard.

With respect to ease of use, participants were not trained to usethe E-Textile keyboard and they did not express any difficultyin using the E-Textile keyboard.

VII. CONCLUSION

We have developed an approach for synthesis and energy-efficient operation of a pressure sensor-based soft keyboard. Weused E-Textiles as our sensing material due to their foldable andwearable characteristics and thus good targets for mobile andportable systems. We optimized energy at five levels of abstrac-tion: architecture of sensor systems, sensing schedule, sensordata processing, use of semantic information, and customiza-tion. Our results indicated savings up to 63× in the overallsystem energy consumption compared with the system whereeach key is sensed by exactly one sensor. The procedure we usedcould likely be easily retargeted to other pressure sensor-basedsystems such as smart (medical) shoes, gloves, and bed sheets.

REFERENCES

[1] H. Beker and F. Piper, Cipher Systems: The Protection of Communications.Etobicoke, Canada: Wiley, 1982.

[2] S. Zhai, A. Sue, and J. Accot, “Movement model, hits distribution andlearning in virtual keyboarding,” in Proc. SIGCHI Conf. Human FactorsComput. Syst., 2002, pp. 17–24.

[3] P. M. Fitts, “The information capacity of the human motor system incontrolling the amplitude of movement,” J. Exp. Psychology, vol. 47,pp. 381–391, 1954.

[4] I. S. MacKenzie, “Fitts’ law as a research and design tool in human-computer interaction,” Human-Comput. Interaction, vol. 7, pp. 91–139,1992.

[5] W. Soukoreff and I. S. MacKenzie, “Theoretical upper and lower boundson typing speeds using a stylus and keyboard,” Behav. Inf. Technol., vol. 14,pp. 370–379, 1995.

[6] J. Kao and R. Marculescu, “Energy-aware routing for e-textile applica-tions,” in Proc. Conf. Design, Autom. Test Eur., 2005, vol. 1, pp. 184–189.

[7] I. S. MacKenzie and S. X. Zhang, “The design and evaluation of a high-performance soft keyboard,” in Proc. SIGCHI Conf. Human Factors Com-put. Syst., 1999, pp. 25–31.

[8] M. Hunter, S. Zhai, and B. A. Smith, “Physics-based graphical keyboarddesign,” in Proc. CHI Extended Abstracts Human Factors Comput. Syst.,2000, pp. 157–158.

[9] M. Silfverberg, I. S. MacKenzie, and P. Korhonen, “Predicting text entryspeed on mobile phones,” in Proc. SIGCHI Conf. Human Factors Comput.Syst., 2000, pp. 9–16.

[10] J. Goodman, Venolia, K. Steury, and C. Parker, “Language modelingfor soft keyboards,” in Proc. 7th Int. Conf. Intell. User Interfaces, 2002,pp. 194–195.

[11] S. Zhai, M. Hunter, and B. A. Smith, “The metropolis keyboard-an ex-ploration of quantitative techniques for virtual keyboard design,” in Proc.ACM Symp. User Interface Softw. Technol., 2000, pp. 119–128.

[12] M. Raynal and N. Vigouroux, “Genetic algorithm to generate optimizedsoft keyboard,” in Proc. CHI Conf. Human Factors Comput. Syst., 2005,pp. 1729–1733.

[13] X. Bi, B. A. Smith, and S. Zhai, “Quasi-qwerty soft keyboard optimiza-tion,” in Proc. Conf. Human Factors Comput. Syst., 2010, pp. 283–286.

[14] A. Gunawardana, T. Paek, and C. Meek, “Usability guided key-targetresizing for soft keyboards,” in Proc. Conf. Intell. User Interfaces, 2010,pp. 111–118.

[15] (2011). http://focus.ti.com/docs/prod/folders/print/msp430-adc10.html.[16] (2012). http://focus.ti.com/docs/toolsw/folders/print/ez430-rf2500 t.html.[17] D. Ganesan, B. Krishnamachari, A. Woo, D. Culler, D. Estrin, and

S. Wicker, “Complex behavior at Scale: An experimental study of low-power wireless sensor networks,” Univ. California, Los Angeles, Los An-geles, CA, USA, Tech. Rep. UCLA/CSD-TR 02-0013, 2002.

[18] J. Polastre, J. L. Hill, and D. E. Culler, “Versatile low power media ac-cess for wireless sensor networks,” in Proc. 2nd Int. Conf. EmbeddedNetworked Sensor Syst., 2004, pp. 95–107.

[19] (2011). http://www.wirelessground.com/blfoblke.html.


[20] (2011). http://www.onlypre.com/blfoke.html.[21] M. Malinowski, M. Moskwa, M. Feldmeier, M. Laibowitz, and J. Paradiso,

“CargoNet: a low-cost micropower sensor node exploiting quasi-passivewakeup for adaptive asynchronous monitoring of exceptional events,” inProc. 5th Int. Conf. Embedded Networked Sensor Syst., 2007, pp. 145–159.

[22] S. Jevtic, M. Kotowsky, R. P. Dick, P. A. Dinda, and C. Dowding, “Luciddreaming: reliable analog event detection for energy-constrained applica-tions,” in Proc. Inf. Process Sensor Netw., 2007, pp. 350–359.

[23] P. Wan and M. D. Lemmon, “Event-triggered distributed optimization insensor networks,” in Proc. Inf. Processing Sensor Netw., 2009, pp. 49–60.

[24] F. Koushanfar, N. Taft, and M. Potkonjak, “Sleeping coordination forcomprehensive sensing using isotonic regression and domatic partitions,”in Proc. IEEE 25th Int. Conf. Comput. Commun., 2006, pp. 1–13.

[25] J. Liu, P. Cheung, F. Zhao, and L. J. Guibas, “A dual-space approach totracking and sensor management in wireless sensor networks,” in Proc.1st ACM Int. Workshop Wireless Sensor Netw. Appl., 2002, pp. 131–139.

[26] Z. Del Prete, L. Monteleone, and R. Steindler, “A novel pressure arraysensor based on contact resistance variation: metrological properties,” Rev.Scientific Instruments, vol. 72, pp. 1548–1553, 2001.

[27] D. Marculescu, R. Marculescu, N. H. Zamora, P. Stanley-Marbell,P. K. Khosla, S. Park, S. Jayaraman, S. Jung, C. Luterbacj, W. Weber,T. Kirstein, D. Cottet, J. Grzyb, G. Troster, M. Jones, T. Martin, andZ. Nakad, “Electronic textiles: a platform for pervasive computing,” Proc.IEEE, vol. 91, no. 12, pp. 1993–1994, Dec. 2003.

[28] J. M. Rofouei, M. Potkonjak, and M. Sarrafzadeh, “Energy efficiente-textile based portable keyboard,” in Proc. Int. Symp. Low Power Elec-tron. Design, 2011, pp. 339–344.

Authors’ photographs and biographies not available at the time of publication.

Documents

Energy Efficient Collaborative Sensing-Based Design: Soft Keyboard Case Study