Intelligent Home-Appliance Recognition over IoTCloud Network

1Shih-Yeh Chen, 2Chin-Feng Lai, Member, IEEE, 1Yueh-Min Huang, Fellow, BCS and 3Yu-Lin Jeng1 Department of Engineering Science, National Cheng Kung University, Tainan, Taiwan.

2 Institute of Computer Science and Information Engineering and Department of Electronic Engineering, NationalIlan University, I-Lan, Taiwan.

3 Cloud System Software Institute, Institute for Information Industry, Taiwan, R.O.C.Email: n98014045@mail.ncku.edu.tw, cinfon@ieee.org, huang@mail.ncku.edu.tw, jackjeng@iii.org.tw

Abstract—In recent years, under the concern of energy crisis,the government has actively cooperated with research institutionsin developing smart meters. As the Internet of Things (IoT)and home energy management system become popular topics,electronic appliance recognition technology can help users identi-fying the electronic appliances being used, and further improvingpower usage habits. However, according to the power usagehabits of home users, it is possible to simultaneously switchon and off electronic appliances. Therefore, this study discusseselectronic appliance recognition in a parallel state, i.e. recognitionof electronic appliances switched on and off simultaneously.This study also proposes a non-invasive smart meter systemthat considers the power usage habits of users unfamiliar withelectronic appliances, which only requires inserting a smartmeter into the electronic loop. Meanwhile, this study solves theproblem of large data volume of the current electronic appliancerecognition system by building a database mechanism, electronicappliance recognition classification, and waveform recognition.In comparison to other electronic appliance recognition systems,this study uses a low order embedded system chip to providelow power consumption, which have high expandability andconvenience. Differing from previous studies, the experiment ofthis study considers electronic appliance recognition and thepower usage habits of general users. The experimental resultsshowed that the total recognition rate of a single electronicappliance can reach 96.14%, thus proving the feasibility of theproposed system.

Index Terms—Parallel electronic appliance recognition, Smartmeter, Waveform recognition algorithm.

I. INTRODUCTION

IN recent years, as cloud computing and the Internetof Things (IoT) develop, the development of the smart

home apparently enters a new stage, thus, the industries andacademia of various countries have focused on developingSmart Grids, Cloud Computing Services, and Green Build-ings. As the energy crisis and global warming problems arecontinuously proposed, the topic of energy crisis has becomean urgent international problem to be solved.

Appliance recognition utilizes the different current and volt-age characteristics in order to recognize different appliances.This method not only effectively manages the appliancesin their usage states but also achieve energy managementand smart home functions. Further step combines cloud andsmart grid to implement whole area electronic distribution and

advanced energy planning. Appliance recognition still facesnumerous challenges. Appliance recognition focuses on smartmeters to perform power feature calculations, however thereare still installation problems. Also most only offers powerconsumed, power bill, basic power limit, wave detection dataprinting. Other than that the appliance recognition researchescan only recognize one appliance at one time. Those do not tar-get the appliance behaviors of families, simultaneous applianceturning on or off. It also has the negative of building a largevault of power features. How to build samples and decreasecalculation to implement this in an embedded environmentbecomes the main issue this research will face.

This researches focuses on parallel state appliance recogni-tion. Therefore appliance on/off state recognition. Using thepower meter from this research to get current informationto build power feature database then lower the calculationrate as much as possible. Then using Dynamic Time Warping(DTW) to power appliance classification to perform the relatedwave data analysis for recognition. The household applianceconfiguration state can be provided by the cloud serviceprovider in the future, thus further developing and providingmore advanced energy control and management service, whichchange will not influence the power usage habit of users. Theresults shown under simultaneous singular recognition ratescan reach up to 96.14%.

The contributions of this study can be concluded as follows:

1) Development and design of smart meter

2) Implementation of lightweight electronic appliancerecognition algorithm

3) Parallel electronic appliance recognition

II. RELATED WORK

A. Home-appliance Recognition with Smart Meters

The smart meter is the last layer of a smart grid, whichprovides users with more detailed power usage information bythe automation of the electronic meter and provides automaticmeter reading service. At present, smart meter studies followtwo directions. The first one is power line design, where thepresent no-fuse switch general meter used by home users can

978-1-4673-2480-9/13/$31.00 ©2013 IEEE 639

bear a 50A current supply. This type of design aims at ahigh power bus, and thus, should be tested by a high-ordermeter. Another direction is an extension line design, whichdesign is also known as the smart socket. Its purposes are:1) to provide an extension line household appliance controlservice, including relay control and infrared remote control; 2)to reduce the current sensing range to provide a more completeand safer energy information system; 3) seamless integrationwith the environment, as a power line meter is difficult tobe installed, expandability is worse. Akbar et al. [1] usedFast Fourier Transform to transform the current waveformof time domain into a frequency domain in order to obtainspecial electronic energy parameters for recognition. However,using electronic appliance characteristics for recognition hasa defect; if electronic energy information is not obtainedfrom the socket of the individual electronic appliance, butfrom the general supply, it is difficult to recognize electronicappliances. Obtaining electronic appliance characteristics fromindividual socket is difficult for use by common families dueto its difficult installation. As this type of information iseasily obtained, it is applicable to systems with lightweightoperational capability or electronic meters, and is one of thebasic sorting parameters used in this study.

B. Smart Grid

The smart grid is introduced into Internet and Communica-tion Technology (ICT) to achieve the monitoring, protection,and automation control. It covers the power companies, powerdistribution stations, power grid distribution system, electricityand household electricity. Smart grid domain can be discussedin three stages[2]. 1) constructing an Advanced MeteringInfrastructure (AMI) in urban areas, including energy produc-tion, management, and transportation systems construction; 2)providing Automatic Meter Reading (AMR) between the dis-tributing substation and use locations to replace manual meterreading, which can result in human error and greater cost; 3)integrating the Smart Home control with energy issues throughsmart meter technology to provide real-time home energy useand household appliance control service for general homeusers. Most studies tend to smart meters, meter managementsystems, network systems, data security, and electricity storagedevice. Marcelo Saguan et al. [3] presents that ”smart gridwill eventually establish an energy system which combinednew tools and technologies of power generation, transmissionand distribution, as well as the client’s home appliances andelectrical equipment. For large amounts of data exchange insmart grid, in-network processing [4-7] focus on dispersionand parallel processing and load-balancing issues. Traditionaldistributed computing systems can be categorized into twotypes: the Parallel and Distributed systems. Both types ofsystems, in the spirit of task simplification, divide the originalproblem for computing into multiple sections to be processedby multiple processors or computers. To the parallel system[8,9], different processors share common memory blocks anddata. However, the implementation relationships between mul-tiple processors should be governed and controlled by multipleimplementation method.

III. OVERALL ARCHITECTURE

In this section, the work describe our proposed solution.

A. Smart Meter/Socket Design

This study implements a smart meter with a parallel elec-tronic appliance recognition function. Its system structure, asshown in Fig. 1, is consisted of a hardware layer, a data processlayer, and a recognition layer. The smart home and smart griddomains are covered by the application derived from this metersystem.

Square wave Generator

Current Sensor

Noise Suppression

Clamp Circuit

Appliance Recognition

Smart Home

Sub-tree Classifier Database Creation

Amplifier

Application

Recognition

Data Process

Hardware

Smart Grid

Waveform Interception

Slide Windows

Fig. 1. Proposed System Architecture

• Hardware LayerThe hardware layer is the hardware design layer of asmart meter, which is in charge of processing electronicenergy signals, including complete electronic energy sig-nal waveform extraction, waveform correction and regu-lation. It provides the data process layer with preliminaryprimary voltage and current signal. The principle andimplementation of this stage are introduced in detail inlater section.

• Data Process LayerThe data process layer is the advanced data processingpart inside the STM32, and includes internal currentwaveform extraction, noise reduction, and state detection.It is in charge of providing electronic appliance character-istic processing for electronic appliance recognition algo-rithm, and the data process at this stage can greatly reducethe noise caused by the hardware layer, while increasingthe recognition rate and accuracy of the system.

• RecognitionThe recognition layer is the core method of this paper,and includes a waveform recognition algorithm, databasecreation, and segmentation, it is in charge of calculatingand classifying the electronic energy characteristics ofthe data process layer, which uses fewer characteristicgroups for hierarchical characteristic classification andremoves very different characteristics out of the databasematrix, thus, providing a better recognition database forthe core waveform recognition algorithm to reduce thetime complexity of recognition calculation.

640

IV. ELECTRONIC ENERGY DATA PROCESSING

FRAMEWORK

In order to accurately capture the waveform of a surgecurrent and avoid different power frequencies under energypolicies of different countries, this study uses a transformerto normalize the waveform extraction. The EXTI interface ofthe STM32 system chip is used for initialization, where theinterrupt trigger point is set as EXTI Trigger Rising, and thepriority of this interrupt trigger point is maximized to avoidother interrupts influencing the operation. As shown in Fig.2, the voltage is the initial waveform of external voltage afterpassing through the transformer, and the square wave generatorgenerates a set of continuous square wave signals. Finally, theexternal interrupt captures the rising edge part of the squarewave as the trigger point. The system stores the complete surgecurrent waveform in the sequence window when the externalinterrupt is triggered, and waits for the next external interrupttrigger.

-4

-2

0

2

4

-0.5

1.5

3.5

-0.5

1.5

3.5

Voltage

Square wave

Generator

External

interrupt

Fig. 2. The Schematic of Interrupt Trigger Point

Waveform extraction is discussed in the chapter of elec-tronic energy extraction, in order to achieve real-time elec-tronic energy information acquisition and to improve therecognition rate. The signal processing aims at electronicappliance state detection and noise processing, thus, twomechanisms are designed in the signal processing layer, whichare state detection and noise reduction.

A. Waveform Recognition Algorithm

The electronic appliance recognition algorithm used in thisstudy is a waveform recognition algorithm, which is usedfor recognizing the variations and similarities of waveformsor patterns, which is the commonly used method of Dis-tance Measure, and is mostly applied to electrocardiography(ECG), speech recognition, and pattern recognition domains.This study uses a waveform recognition algorithm to analyzeinstantaneous value of a current, where the major differencebetween the current and the aforesaid speech recognition isspatial compression. This study discusses this type of problemin the following section.

1) Euclidean Distance: The Euclidean distance is the mostfundamental part of distance measurement, as well as thebasis of implementation of multiple distance algorithms, thediscussed information is the range difference between twosets in Euclidean space as the basis of measurement. If there

are two finite data sets in a p-dimensional Euclidean space,A = {a1, a2, . . . , an} ⊂ Rp and B = {b1, b2, . . . , bn} ⊂ Rp,the equation of Euclidean distance is defined as follow.

dist(A,B) = [(a1−b1)2+(a2−b2)

2+...+(an−bn)2]1/2 (1)

As the Euclidean distance has not processed the time axis,there will be system recognition error as a result of displace-ment and noise, which renders it inapplicable to accurateelectronic appliance recognition.

2) Dynamic Time Warping: The waveform recognition al-gorithm used in this study is Dynamic Time Correction orDynamic Time Warping (DTW), which algorithm a commonalgorithm for discussing the similarities between two timedependent series, and as the characteristic of dynamic pro-gramming (DP) is usually used in speech recognition, thecharacteristic of speech recognition is the scaling of time axis,and the DTW can compare the waveform similarity betweentwo sets in different matrix lengths, as shown in Fig. 3.

Fig. 3. The Telescopic Features of DTW

If two sets Q = {q1, q2, ..., qn−1, qn} and U ={u1, u2, ..., um−1, um} are given, a n ∗m distance matrix Dis created, where the matrix D(i, j) is the distance betweenqi and uj , i.e. all D(i, j) = d(qi, uj), and the warping path(W) in matrix D represents the coincidence relation betweenQ and U, as defined by Eq. 2.

W = {w1, w2, ..., wk},max(m,n) ≤ K < m+ n− 1 (2)

The limits to warping path are as listed below:• Boundary conditions: w1 = (1, 1) and wk = (m,n),

i.e. the start and end of warping path are a clinodiagonalpath.

• Continuity: if wk = (a, b), wk−1 = (a′, b′) meetsa − a′ ≤ 1 and b − b′ ≤ 1, i.e. the available directionof warping path must be the adjacent matrix, includingadjacent bevel matrix.

• Monotonicity: if wk = (a, b), wk−1 = (a′, b′) meetsa− a′ ≤ 0 and b− b′ ≤ 0, i.e. this warping path must goin the single direction of the time series.

As shown in Fig. 4, the left waveform (Sample Data) is thesampled electronic energy information, and the lower wave-form is the electronic lamp sample in comparison database,as the two path matrices match each other completely, it iswarped to a bevel linear path.

641

Fig. 4. The Distance Schematic of DTW Lamp Identification

The DTW length is defined as the minimum warping costof a slant path, as shown in Eq. 3.

DTW (Q,U) = min

√

√

√

√

K∑

k=1

wk/2) (3)

The time complexity for calculating DTW distance is aboutO(mn), namely, the time complexity of equidistant DTWcalculation is O(n2); therefore, many studies have optimizedthis algorithm, and the Path Constraints [10-12] can reducethe time complexity to O(n) by limiting the warped bound-ary in matrix D. Another method is local path constraint[13-15], where the warp angle in matrix D is limited to−27◦−45◦−63◦. This method supports a jumping processing,which provides better path calculation instead of counting thenoise in the total DTW distance, the optimal path of DTWdistance will go −27◦ and −63◦, as possible, in order to obtaina shorter distance, with the fundamental purpose reducing thecalculation load while limiting the reasonable image path.

V. SYSTEM PROTOTYPE AND VERIFICATION

This study uses the following five electronic appliancesas the analytic data set, as shown in Table 1, including acirculation fan, a notebook computer, an LCD screen, a tablelamp, and a hot melt gun.

TABLE ITHE LIST OF ELECTRICAL EXPERIMENT

Data Type ColumnFan Close Weak Medium Strong

Notebook Close Power saving High performance N/ALCD Monitor Close Open N/A N/A

Bulb Close Open N/A N/AHot-melt gun Close Open N/A N/A

The noise reduction and parallel electronic appliance recog-nition mechanisms, as stated in previous chapter, are used for

analysis in the experimental process. The above mentionedelectronic appliances will pass through the learning phase tocomplete the extraction and correction of electronic appli-ance waveform information; meanwhile, this step contains thewaveform reforming mechanism, thus, facilitating the rapidcreation of the recognition database. Taking this experimentas an example, 96 comparison data can be created, providingthat the 8 groups of information are learned.

A. Performance Metrics

In order to test the accuracy of the electronic appliancerecognition system, as proposed in this study, in the parallelelectronic appliance environment, this chapter analyzes thecombination of all electronic appliance states. In order to simu-late the simultaneous on-off of electronic appliances, this studyuses a multihole extension line, available on the market, asassistance, where electronic appliances can be simultaneouslyswitched on by using the switch for the multihole extensionline. The correctness evaluation method of data mining is usedfor data verification in this study.For the binary classificationproblem, if the presently tested recognition sample is elec-tronic appliance a, the results can be divided into four series:

• True Positive (TP): The measured electronic applianceis electronic appliance a, and identified as electronicappliance a.

• False Negative (FN): The measured electronic applianceis electronic appliance a, but identified as another elec-tronic appliance.

• False Positive (FP): The measured electronic applianceis another electronic appliance, but identified as electronicappliance a.

• True Negative (TN): It is another electronic appliancein fact, and identified as another electronic appliance.

The precision (p) and recall (r) of this recognition systemcan be determined by the aforesaid four states. The precisionmeans ”taking the information of real electronic appliance aout of the set of all the types identified as electronic appliancea”, higher precision means lower misrecognition rate.

p = TPTP + FP ) (4)

The recall represents the ”ratio of samples of actual elec-tronic appliance a identified as electronic appliance a”, asdefined as Eq. 5.2, this study uses recall as the basis ofrecognition rate in the sample space.

r = TPTP + FN) (5)

B. Test Cases

In the test for parallel electronic appliance recognition,this study randomly extracts different electronic appliancecombinations and uses the electronic appliance recognitionalgorithm adopted in this study to identify the electronicappliances.

According to the analysis of the state of the notebookcomputer, the low power consumption notebook computer,as learned at the electronic appliance learning stage, is the

642

0

20

40

60

80

100

120

1 2 3 4 5 6 7 8 9 10 11

(%)

Test Pattern

Precision & Recall

prerecall

Fig. 5. The Precision and Recall of combination of Two Electrical

”low power consumption mode of an operating system notexecuting any program”, and it is in ”low power consumptionmode of executing multiple software synchronously” at therecognition stage. As the video adapter embedded in thenotebook computer automatically switches the video adaptermode, the efficiency of the notebook computer is increasedto approximately high efficiency mode. In order to validatethe causes for this case, the difference between two currentwaveforms is analyzed according to the measurement resultof the current sensor. It is observed the current waveformapproaches high efficiency when the low efficiency notebookcomputer executes massive calculations; meanwhile, the highefficiency mode approaches to low efficiency mode becauseof different quantities of started services. Therefore, it ismisrecognized as a notebook computer high efficiency modein this experimental result.

VI. SUMMARY

This study proposes a smart meter design based on anextension of line smart meter design in order to providea simpler installation mode, with convenience and expand-ability, for home users. In addition, this study proposes alightweight electronic appliance recognition method for thedesigned smart meter. The average recognition rate of a singleelectronic appliance can reach 96.14%, where the parallelelectronic appliance recognition is carried out without pre-setting any power usage conditions, and the recognition ratecan be higher than 84.14%, thus, validating the feasibilityof a lightweight electronic appliance recognition system tolower the computing capacity of an embedded system. Theparallel electronic appliance recognition system proposed inthis study aims at the training part, which is usually neglectedin electronic appliance recognition. This study also proposesa current waveform merging mechanism to provide the rapidcreation of a recognition sample database. Finally, the subtreeclassification mechanism is used to cut the database parenttree into several cluster subtrees, thus, obtaining the computingtime of a recognition algorithm and its spatial balance point.

ACKNOWLEDGMENT

This study is conducted under the ”Cloud computing sys-tems and software development project(2/3)” of the Institutefor Information Industry which is subsidized by the Ministryof Economy Affairs of the Republic of China and the NationalScience Council of the Republic of China, Taiwan for support-ing this research under Contract NSC 101-2628-E-197-001-MY3, 101-2221-E-197-008-MY3 and 101-2219-E-197-004.

REFERENCES

[1] M. Akbar and D.Z.A. Khan, Modified nonintrusive appliance load moni-toring for nonlinear devices, In Proc. of the IEEE International MultitopicConference, pp. 1 - 5, 2007.

[2] C. Bennett and D. Highfill, Networking AMI Smart Meters, In Proc. ofthe IEEE Energy 2030 Conference, pp. 1-8, 2008.

[3] M. Saguan, Advanced Metering: summary and conclusion, In Proc. ofthe Smart Metering Workshop, pp. 1-12, 2009.

[4] I.D. Schizas, G. Mateos, G.B. Giannakis, Distributed LMS for Consensus-Based In-Network Adaptive Processing, IEEE Transactions on SignalProcessing, Vol. 57, No. 6, pp. 2365 - 2382.

[5] T. Bilski, A Model for Storage Processes in Network Environment and ItsImplementation, Fifth IEEE International Workshop on Storage NetworkArchitecture and Parallel I/Os, pp. 47 - 53, 2008.

[6] H.-H. Tran and K.-J. Wong, Real-time in-network Single Frequency GPSProcessing for Deformation Monitoring, 2012 IEEE Wireless Communi-cations and Networking Conference, pp. 3273 - 3277, 2012.

[7] L. Ying, Z. Liu, D. Towsley, and C.H. Xia, Distributed Operator Place-ment and Data Caching in Large-Scale Sensor Networks, IEEE The 27thConference on Computer Communications. pp. 977 - 985, 2008.

[8] R. Gupta, P. Gopinath, A Hierarchical Approach to Load Balancing inDistributed Systems, In Proc. of the Fifth Distributed Memory ComputingConference, Vol.2, pp. 1000 1005, 1990.

[9] I. Barazandeh, S. S. Mortazavi, Two Hierarchical Dynamic Load Balanc-ing Algorithms in Distributed Systems, Second International Conferenceon Computer and Electrical Engineering, Vol.1, pp. 516 521, 2009.

[10] R. E. Schantz and D. C. Schmidt, Research Advances in Middleware forDistributed Systems: State of the Art, In Proc. of IFIP World ComputerCongress Communications Systems: State of The Art, pp. 1-36, 2002.

[11] S. Groot, K. Goda, and M. Kitsuregawa, A Study on Workload Im-balance Issues in Data Intensive Distributed Computing, In the Proc. ofDatabases in Networked Information Systems, pp. 27 32, 2010.

[12] G.C. Fox, S. Kamburugamuve, and R.D. Hartman, Architecture andMeasured Characteristics of a Cloud Based Internet of Things, Inter-national Conference on Collaboration Technologies and Systems, pp. 6 -12, 2012.

[13] Y. Qiao and H. Chen, The Design of Smart Cloud Computing System:The Internet of Thing Model, International Conference on Computationaland Information Sciences, pp.185-188, 2011.

[14] F. Yang and F.-L. Zhang, Research for applications of RFID in Chengduand innovation designing based on TRIZ, International Conference onApperceiving Computing and Intelligence Analysis, pp.452-456, 2010.

[15] J.C. Hung, V. Hsu, and Y.-B. Wang, A Smart-Travel System Basedon Social Network Service for Cloud Environment, Third InternationalConference on Intelligent Networking and Collaborative Systems, pp.514-519, 2011.

643