IoTDots: A Digital Forensics Framework for SmartEnvironments

Leonardo Babun, Amit K. Sikder, Abbas Acar, and A. Selcuk UluagacCyber-Physical Systems Security Lab

Department of Electrical & Computer Engineering, Florida International University10555 West Flagler St. Miami, FL 33174

Email: {lbabu002, asikd003, aacar001, suluagac}@fiu.edu

Abstract—IoT devices and sensors have been utilized in a co-operative manner to enable the concept of a smart environment.In these smart settings, abundant data is generated as a resultof the interactions between devices and users’ day-to-day ac-tivities. Such data contain valuable forensic information aboutevents and actions occurring inside the smart environment and,if analyzed, may help hold those violating security policiesaccountable. Nonetheless, current smart app programmingplatforms do not provide any digital forensics capability toidentify, trace, store, and analyze the IoT data. To overcomethis limitation, we introduce IoTDots, a novel digital forensicframework for a smart environment such as smart homes andsmart offices. IoTDots has two main components: IoTDots-Modifier and IoTDots-Analyzer. At compile time, IoTDots-Modifier performs the source code analysis of smart apps,detects forensically-relevant information, and automaticallyinsert tracing logs. Then, at runtime, the logs are stored into aIoTDots database. Later, in the event of a forensic investigation,the IoTDots-Analyzer applies data processing and machinelearning techniques to extract valuable and usable forensicinformation from the devices’ activity. In order to test theperformance of IoTDots, we tested IoTDots in a realistic smartoffice environment with a total of 22 devices and sensors.Also, we considered 10 different cases of forensic activitiesand behaviors from users, apps, and devices. The evaluationresults show that IoTDots can achieve, on average, over 98%of accuracy on detecting user activities and over 96% accuracyon detecting the behavior of users, devices, and apps in asmart environment. Finally, IoTDots performance yields nooverhead to the smart devices and very minimal overheadto the cloud server. To the best of our knowledge, IoTDotsis the first lightweight forensic solution for IoT devices thatcombines the collection of the forensically-relevant data froma smart environment and the analysis of such data usingdata processing and machine learning techniques for forensicpurposes. Finally, we have made the IoTDots-Modifier availableonline for the community.

Keywords—Digital forensics, Internet of Things, source codeanalysis, logging.

I. INTRODUCTION

The Internet of Things (IoT) has quickly evolved asthe network of Internet-enabled physical devices. The IoTdevices communicate with each other and interact with theusers’ day-to-day activities through sensors. These capabil-ities have also enabled the concept of a smart environment.Such an environment improves the quality of the life of thepeople while handling a new set of data previously untapped.For instance, smart offices can be equipped with a set ofstate-of-the-art gadgets like smart lights, smart surveillance

cameras, smart smoke detectors, presence sensors, and smartthermostats, all controlled by a central hub. In such a setup,the presence sensor can detect users and trigger the camerasto start recording while turning on the lights. At the sametime, the smart thermostat can become active and startlowering the room temperature. In this example, the smartenvironment has control and gets feedback from the users’activities to change the general state of the surroundingsbased on (1) what the users do, (2) the smart environmentsetup policies, and (3) the state of the devices.

Indeed, the interactions between devices and users in asmart environment generate data with tremendous forensicvalue. For instance, in the smart office setup previouslydescribed, the changes in the presence sensor’s state mayindicate the existence of individuals inside the smart officeduring unauthorized hours. Further, the smart environmentdata can give insights about the exact location of abruptspikes in temperature values or the presence of smokehappening right before a fire incident.

Nonetheless, current IoT solutions do not provide anymeans for forensic analysis. In general, the limitation ofavailable computing resources in the majority of the smartdevices and the unique cloud-based architecture of IoTmakes it very challenging to store data inside the devices forforensic purposes. Indeed, the most popular IoT program-ming platforms (e.g., Samsung SmartThings, openHab, etc.)do not provide any means to have access and indefinitelystore data in the cloud. Previous works have used the ideaof logging to save data from smart apps and devices forfurther analysis of data provenance. For instance, in [1], theauthors propose an instrumental platform-centric approachthat logs activities from smart apps for data provenance. Inthat work, however, the analysis considered the relationshipbetween devices, users, and smart apps while assumingdevice integrity. This approach, while valid and useful,overlook the problem of tampered devices [2] that cancause the abnormal behavior of the smart environment.Without this analysis, the IoT forensic investigation wouldignore additional threats coming from the compromiseddevices [3], [4]. These threats can modify the state of thesmart environment by poisoning the forensic data or byleaking sensitive information that attackers can use to bypassthe security policies of the smart environment and gainaccess, for instance, to restricted areas. In this paper, weintroduce IoTDots, a novel digital forensic framework forsmart environments. IoTDots has two main components:IoTDots-Modifier (ITM) and IoTDots-Analyzer (ITA). The

arX

iv:1

809.

0074

5v1

[cs

.CR

] 3

Sep

201

8

ITM analyzes smart applications looking for forensically-relevant information inside the apps. Then, the smart appsare modified by inserting specific logs that will send theforensic data to the IoTDots Logs Database (ITLD) atruntime. Later, in a case of a forensic investigation, the ITAapplies data processing and machine learning techniques onthe ITLD data to comprehensively learn the state of the smartenvironment and the users’ behavior at the time of interestof the forensic analysis. IoTDots considers the events andactions inferred from the ITLD data and the security policiesdefined for the smart environment to detect potential securityviolations from users, devices, or smart apps.

Specifically, IoTDots will look for users performingactivities that potentially violate the security policies insidea smart environment. Also, IoTDots will use the relationshipbetween devices’ states to detect users trying to remove, dis-able, or tamper IoT devices and/or malicious apps modifyingthe data extracted from the smart environment.

We evaluate IoTDots in a smart environment con-taining different types of devices including smart lights,smart thermostats, motion sensors, etc. Experimental resultsdemonstrate that IoTDots performs very well on detecting,storing, and processing forensically-relevant data from thesmart environment. Specifically, for activity detection, ourframework achieves up to 98% accuracy for both time-dependent1 and time-independent2 activities. On the otherhand, for the case of behavior detection, our frameworkachieves 96% accuracy. Additionally, a detailed performanceanalysis shows that no overhead is imposed to the smartdevices and minimal overhead is imposed to the use ofphysical memory and latency on the cloud-based servers asa result of utilizing IoTDots.

Summary of Contributions: The contribution of this workare as follows:

• We propose IoTDots, a novel digital forensic frame-work for smart settings. Our framework automati-cally analyzes and modifies smart apps to detect andstore forensically-relevant data from smart devices,apps, and users. Then, in the case of a foren-sic investigation, IoTDots applies data processingand machine learning techniques to detect valuableforensic evidence from smart devices, apps, andusers’ activities and behavior.

• We made the IoTDots-Modifier freely available on-line at so IoTDots users can use our automatedsystem to perform source code analysis and modi-fication of their smart apps to enable IoTDots.

• We used a threat model that considered users per-forming activities that violate the security policiesinside a smart environment. Also, our threat modelprotects device and data integrity by considering thecases of malicious users tampering smart devicesand apps poisoning the smart environment data ormisreporting device states.

• We evaluated IoTDots in a realistic smart officesetup that contains 10 different types of smart

1Activities performed only over certain specific time frame t.2Activities performed freely without considering any specific time frame.

devices including smart lights, smart thermostats,motion sensors, etc. and 22 devices in total.

• Our results demonstrate that IoTDots achieves veryhigh accuracy on revealing forensic-relevant activi-ties and behavior inside the smart environment.

• Finally, since IoTDots focuses on forensic-relevantdata only, its performance does not represent ad-ditional overhead in terms of computing resourcesto the smart devices and minimal overhead to thecloud-based servers.

Organization: The rest of this paper is organized as follows.In Section II, we present the background information. Then,a use case and the threat model are described in Section III.Section IV details the architecture of IoTDots and SectionV describes the data processing and machine learning tech-niques utilized by the proposed forensic framework. Then,the evaluation results are analyzed in Section VI, followedby the benefits, challenges, limitations of IoTDots and futurework in Section VII. Finally, the related work in Section VIIIand the conclusions in Section IX complete the paper.

II. BACKGROUND

Digital Forensics: The term digital forensics refers to theinvestigation and the uncovering of any kind of evidencefrom electronic devices containing digital data. The mostbasic application of digital forensics investigations is toeither support or oppose the hypothesis proposed in a casebefore a court. Frequently, digital forensics requires theanalysis of physical evidence directly related to people’spresence in the event scene such as an audio or videofile. In other cases, a more deep investigation of moretechnical evidence such as network data or mobile deviceforensics may be required. Complex cases may require thecombination of both types of analysis.

Although there are different approaches, a standard dig-ital forensics process can be completely described with fivedifferent stages [5], [6]: (1) Identification, (2) Preservation,(3) Recovery, (4) Analysis, and (5) Presentation. Dependingon the type of device analyzed and the source of theevidence, the methodology used and the type of evidenceextracted may differ. For example, while a mobile device isa good source of information for tracking the location of aperson, a computer may have the capability to provide moredetailed and varied intelligence.

IoT Forensics: With the growth of IoT, the digital forensicsis not anymore limited to storage devices like USB drives,computers, or smartphones. Now, the data from devices likea smart lock, smartwatch, or motion sensors can also be usedfor forensic purposes [7], [8]. The heterogeneous data thatthese devices can provide based on their interaction withusers’ day-to-day activities are very valuable. For example,a smart lock can reveal when and who entered the buildingor a smart speaker may reveal the exact location of its userduring the time of an incident. However, the use of datafrom smart devices is not a straightforward exercise due tosome unique challenges present in the IoT:

Variety of network protocols: Although there are somepopular wireless communication protocols used for commu-nication purposes such as WiFi or ZigBee, there exists no

2

common standard for all smart devices [9]. This situation cangenerate inconsistencies in the smart setup topologies. Forinstance, in some cases of ZigBee-enabled devices, a smarthub is required to connect the devices to the internet whilein WiFi-enabled devices, a direct connection to a gatewayis possible. As a consequence of this diversity, it is difficultto identify the source of the evidence data in most cases.

• Limited computing resources: Most of the smartdevices are very limited in terms of power, compu-tational resources, and storage capacity [2]. Theserestrictions make it harder to store and process thedata inside the devices for forensic purposes. Tosolve this limitation, an alternative would be totransmit and store all the data to a remote server.

• High divergence and diversity in smart data: Intraditional digital forensics investigations, the data isnormally collected from specific devices of interest.However, in IoT forensics, the data can be collectedfrom a diverse set of devices [10]. Different datasources may have different impact and meaningduring a digital forensic investigation. Therefore, itis required to have a reliable mechanism that cansuccessfully handle the data collected from differenttypes of smart devices.

III. THREAT MODEL AND ASSUMPTIONS

In this section, we describe the assumptions focusing ona use case where IoTDots can be utilized to perform forensicinvestigations. Moreover, we explain the details of the threatmodel assumed to classify forensically-valuable anomaloususer activities and malicious behavior from users and smartapps in a smart environment.

A. Assumptions

This work assumes that there exists an office O. The of-fice has several devices integrated to create a fully equippedsmart environment. The topology of the smart environmentin O includes devices like smart thermostats, smart locks,smart lights, presence sensors, security cameras, and smartsmoke detectors. We also assume that the general managerBob is the only person in O with administrative rights tohandle the apps that control the smart devices. Such appshave been previously modified by the IoTDots frameworkand, by policy, they are the only ones authorized to be usedto manage the devices inside O. Finally, the security policiesof O prohibit the presence of any person between 8:00pm to 7:00 am from Monday through Friday and anytimeduring the weekends. At some point, a fire incident insideO has caused the loss of sensitive information along withimportant economic consequences. The fire department hasdetermined that the fire was generated during the night timeand a forensic investigation is requested.

We introduce IoTDots as a novel framework that canutilize the logs from the IoT-modified apps to perform theforensic analysis of the event in O. Indeed, IoTDots canbe used in conjunction with traditional forensic analysistools and techniques to hold the responsible person, smartapp, or device (if any) accountable if a case of negligenceor deliberate violation of security policies is detected. By

using IoTDots, several forensic-related questions can beanswered: (1) What was happening inside O right beforethe fire incident had occurred (e.g., right before the smartsmoke detector started sensing the smoke presence)?; (2)Was anyone inside the room (e.g., presence sensor statechanged)?; (3) Was the door opened/closed anytime beforethe fire incident (e.g., smart locker state changed)?. Further,the proposed framework would be able to evaluate thedifferent states of the smart environment inside O and matchthem with the security policies in place to detect any (in-tentional or involuntary) violation of security policies (e.g.,Bob accessing the office at night time) or malicious activitiesfrom users (e.g., Bob tampering the security camera toavoid video recording) or smart apps (e.g., apps containingmalicious functions to modify the values from the presencesensor possibly by a remote attacker).

B. Threat and Forensic Model

IoTDots analyzes data from a smart environment to pro-vide answers during the course of a forensic investigation.These answers should provide enough information to eitherexonerate innocent people or hold the responsible personaccountable for his/her actions. To achieve that, the frame-work needs to be able to successfully detect and characterizeany forensically-valuable information, including regular useractivities, anomalous user activities, and malicious behaviorfrom users, smart apps, and devices. Within the context ofthis work, we define these operations as follows:

• Regular User Activities: Any action performed bythe authorized users inside the smart environmentthat does not violate the security policies in place.

• Anomalous User Activities: Any careless and un-intentional action performed by an authorized userinside the smart environment that is considered aviolation of the security policies in place.

• Malicious Behavior: Any intentional action per-formed by authorized users, non-authorized users,smart apps, and/or smart devices that clearly violatesthe security policies and is considered a threat forother users and/or the state of the smart environ-ment.

Hence, for IoTDots, we consider two different sets offorensically-valuable threats: (1) anomalous user activitiesand (2) malicious behavior from users or smart apps.

To evaluate IoTDots in detecting anomalous user activi-ties, we consider the following time-based activities:

• Time-independent Activity 1: An authorized usercontrolling a smart device in an anomalous mannerinside the smart environment at any time.

• Time-independent Activity 2: An authorized usercontrolling a smart device in an anomalous mannerfrom outside of the smart environment at any time.

• Time-independent Activity 3: An authorized userhaving an unauthorized presence in a specific areawithin the smart environment at any time.

3

Threat Time-dependency Attack Method Specific Attack Examples

Activity-1 Time-independent Tampered device Bob changes the orientation of the presence sensorto fit in a new equipment

Activity-2 Time-independent Authorized user Bob manually lowers the temperature of smart thermostat fromhome the night before of a big meeting with the stakeholders

Activity-3 Time-independent Authorized user Bob is inside the office O at 8:45 pmActivity-4 Time-dependent Authorized/unauthorized user Bob is getting into the office at 8:45 pmActivity-5 Time-dependent Authorized/unauthorized user Bob is using the secure pin to unlock the smart lock

Behavior-1 Time-independent Tampered device Bob disables the smart camera to stop recording while he isperforming unauthorized activities inside the office O

Behavior-2 Time-independent Unauthorized user/attacker Alice gets access to the office O using the smart lock pinthat she obtained through a malicious app that leaks information

Behavior-3 Time-independent Malicious app The presence sensor app reports inverted states to IoTDots

Behavior-4 Time-independent Malicious app The smoke detector app triggers the smart windowsand locks open by reporting false user presence

Behavior-5 Time-independent Malicious app The smart light app disable the compromised smart cameraby creating a specific light on/off pattern

Table I: Summary of the threat model and forensically-valuable activities and behavior considered for IoTDots.

• Time-dependent Activity 4: An authorized/unautho-rized user moving inside the smart environment atan unusual time.

• Time-dependent Activity 5: An authorized/unautho-rized user accessing the smart environment at anunusual time.

Since IoTDots considers the security policies of a smartenvironment to perform its analysis, we categorize useractivities as time-independent and time-depended. In the firstgroup, we include all the user actions whose execution timeis irrelevant to IoTDots as it is not regulated as part ofthe security policies. From the previous list, Activity-1, -2, and -3 can be considered as time-independent actions asthey do not depend on time. On the other hand, the secondgroup gathers those actions whose execution time is strictlyregulated by the security policies of the smart environment.In this category, we include Activity-4 and -5 because theycan only be considered forensically-valuable during a certainperiod of time (e.g., the presence of an authorized user inO is only considered a violation between 8:00 pm and 7:00am).

Further, we also consider the following forensic behav-iors from users or smart apps as part of the threat model forIoTDots.

• Behavior 1: An authorized user can try to tamper,destroy, or remove devices to prevent that the IoT-Dots logs can be acquired and sent to the ITLD.

• Behavior 2: An attacker uses a malicious app toget unauthorized physical or logical access to thesmart environment. This threat represents a case ofimpersonation attack.

• Behavior 3: A malicious app reporting incorrectstates (e.g., fake logs) to IoTDots to cover maliciousactivities. This threat represents a case of false datainjection attack.

• Behavior 4: A malicious app installed in the smartenvironment changes the normal behavior of other

smart home devices and applications by enforcingincorrect device states. This threat represents a caseof denial-of-service attack.

• Behavior 5: A malicious app triggers maliciousactions in other devices and apps by executing aspecific activity pattern. This threat represents a caseof a side channel attack.

Finally, Table I summarizes the threat model specifyingattack examples, the time-dependency categorization forevery action/behavior, and the methods of the attacks.

IV. IOTDOTS ARCHITECTURE

In this section, we present the general architecture ofIoTDots. Also, we detail the analysis of smart applicationsto enable IoTDots and the use of IoTDots Log Database(ITLD) for forensic purposes.

A. Overview

Our general architecture divides IoTDots into two parts:First, the IoTDots-Modifier (ITM) automatically analyzesand modifies the smart apps at compile time to allowforensically-relevant data to be sent to the ITLD in the formof forensic logs; second, logs from the IoTDots-modifiedsmart apps stored in ITLD are processed by the IoTDots-Analyzer (ITA) to extract valuable forensic information.With this architecture, we envision the utilization of IoTDotsin a corporate smart environment where only the use ofIoTDots-modified smart apps is authorized. Figure 1 showsthe proposed architecture for the ITM. For the purpose ofthis work, we target open source IoT programming platformssince our implementation requires source code modification.Such modification requires that the ITM performs sourcecode processing via static analysis on the smart apps’ sourcecodes directly extracted from the smart app repositories [11].Hence, the ITM process includes (1) the analysis of thesource code of the smart apps [12], [13], (2) forensically-relevant information detection, and (3) smart apps modifi-cation. Then, upon utilization, the IoTDots-modified appssend forensically-relevant logs containing states and actions

4

SmartApp RepositorySmart App Modification

(log insertions)

Source Code Analysis

(sources, sinks, entry points)

Forensically-relevant Points

(device states, data flow)

Smart Environment Setup

(IoTDots-modified Apps at runtime)

Smart App

Source Code

IoTDots Modifier (ITM)

IoTDots Logs Database

(ITLD)

1

2

3

Figure 1: The architecture of the ITM. Smart apps areanalyzed to detect and log forensic-relevant data points atcompile time. Then, the logs are sent to the ITLD at runtime.

occurring in the smart environment to the ITLD at runtime.Finally, the logs are organized and kept in the ITLD.

The second part of the architecture involves ITA, whichis shown in Figure 2. This stage performs data processingand analytics on IoTDots. The purpose of this analysis isto extract forensically-relevant information from the smartapp logs. This information may potentially allow learningthe state of the smart environment setup at any time. Also,this analysis provides insights into the users’ activitiesoccurring inside the smart environment. IoTDots is a frame-work capable of performing data processing and applyingmachine learning techniques to successfully combine logsfrom different smart devices, obtain a full description ofthe smart environment setup state, and learn all the possibleactivities that the users were performing inside the smartenvironment exactly at the time of interest of the forensicanalysis. Finally, these activities are correlated with thesecurity policies defined for the smart environment to detectuser activities and potential malicious behaviors from usersand smart apps. In the next sub-sections, we detail theimportant aspects of these operations.

B. IoTDots Modifier (ITM)

In this sub-section, we detail the ITM. As mentionedbefore, this part of the IoTDots framework analyzes thesource code from the original smart applications to detectforensically-relevant information and automatically insertsspecific logs for tracking purposes. Then, at runtime, thelogs are stored in the ITLD where they are later analyzed forforensic purposes. In this work, we are targeting open sourceIoT programming platforms (e.g., Samsung SmartThings,

IoTDots Logs Database

(ITLD)

IoTDots Analyzer (ITA)

Behavior Detection

User Activity Inference

Security Policies

AuditorActivity Detection

Figure 2: The architecture for ITA. In the case of a foren-sic investigation, IoTDots logs are analyzed for potentialforensically-valuable activities from users, apps, and devicesin a smart environment.

OpenHAB, and Apple’s HomeKit) which make the sourcecode of smart apps available online [11], [14], [15], [16].However, before completing these steps, we first need toanswer some key questions: What does the smart app life-cycle look like? How to define forensically-relevant infor-mation inside the smart app’s source code? Where and howsmart apps need to be modified to implement IoTDots? Thefollowing dissection of smart apps provides the answers tothese questions.

1) Smart Apps Structure: In general, smart app pro-gramming platforms [17], [18], [19] define the means toaccess and handle the apps’ data and to transmit themout of the applications. Through these processes, smartapplications can utilize device sensor readings, user-definedinputs, and events to execute the application logic either inthe hub (i.e., hub-based setup) or in the cloud (i.e., cloud-based setup). Therefore, through the analysis of the smartapplications’ structure, one can identify and label smart appinformation points that could potentially contain relevantdata for forensic purposes. In the following, we describe fivedifferent smart app resources that are labeled by IoTDotsas they potentially contain relevant information for forensicanalysis.

• Events: These are used to respond to changes in thephysical environment. At install time and dependingon the app context, a smart application subscribes toa set of device events. Then, event handler methodsare called every time such events occur. For forensicanalysis, events are important since they definephysical changes in the smart environment setup.In our analysis, we include device events as part ofIoTDots’ forensically-relevant information.

• Actions: After an event occurs, a smart app calls anaction to control the affected device. Actions giveideas on how a smart environment setup respondto device changes. Often, these actions containvaluable timing information that can define changesin forensic timelines.

• User-defined Inputs: Smart applications often re-quire inputs from users either to manage the ap-plication logic or to control devices. These inputsare defined at install time and determine specificthresholds to trigger actions. Also, user inputs maydefine contact information to allow notification fromsmart apps to be sent to specific recipients. Changes

5

in user-defined inputs are critical for forensic pur-poses since unauthorized modifications will directlyimpact the execution of the apps’ events and actionsand its notification process.

• Device Information: Applications also grant accessto devices at install time to implement the appli-cation logic. Such devices complete the list of au-thorized smart entities in smart environment setups.Logging this information allows tracing authorizedor unauthorized changes in the environment.

• Time and Location: Successful forensic analysis ofa smart environment requires not only the timinginformation of events/actions, but also the physicallocation from where these events/actions were gen-erated. In that way, such events can be directly at-tached to specific geo-location information of smartdevices.

The first step toward analyzing the smart app sourcecode is to focus on the application’s structure. In general,a smart app’s structure follows the sensor-computation-actuator paradigm, which means, smart solutions are in gen-eral designed the same way regardless of their specific ap-plication and complexity. This common architecture highlysimplifies the step of modeling the application structure fromits source code. The benefits from modeling the smart appinclude the extraction of smart apps’ entry points, events,and control flow of data. Also, it allows to identify entry/exitpoints of data (i.e., sources of data and sink functions)that are used to first define the origin of forensic-relevantinformation, and second how this information is sent outfrom the smart apps and to where.

2) Smart Apps Data Access Points: Smart app program-ming platforms clearly define APIs [20], [21] to transmitdata outside the applications. These APIs can be categorizedinto two main groups:

• Internet: Smart apps logic normally run as cloudservices. This imposes a noticeable difference ifcompared to other domains (e.g., mobile) wherelocal solutions are preferred. Therefore, smart appsare designed to directly act as web services or makecalls to external services defined by the developers.These calls can make requests to smart apps via end-points to obtain information such as device states,events, or even propose specific actions to controlthe smart solution. With IoTDots, we trace theserequests through a custom logging mechanism.

• Messages: Smart applications can also define cus-tom messages to send notifications to users. Thesenotifications can be of the three different types:push notification in the mobile app, email, or shortmessage services (SMS). This allows the developerto create a dedicated notification system to alertthe user when specific events occur. Our frameworkalso traces and stores these notifications for forensicpurposes.

3) Source Code and IoTDots Modifications of SmartApps: The purpose of the source code analysis of smartapps in IoTDots is to automatically detect the previously

Listing 1: A sample code for a SmartThings App

1 /* A section of a code block of an original smart app*/

23 section("Via a push notification and/or an SMS message") {4 input("recipients", "contact", title: "Send notifications

to") {5 input "phone", "phone", title: "Enter a phone number to

get SMS", required: false6 }7 }

Listing 2: A sample code for an IoTDots-modified SmartThings App

1 /* A section of a code block of an IoTDots-modifiedsmart app */

23 section("Via a push notification and/or an SMS message") {4 input("recipients", "contact", title: "Send notifications

to") {5 input "phone", "phone", title: "Enter a phone number to

get SMS", required: false6 }7 log.iotdots (”New recipient defined: $phone”) //IoTDots log8 }

defined points of interest inside the source code and in-sert tracking logs for forensic purposes. The source codeanalysis starts by constructing an Inter-procedural Con-trol Flow Graph (ICFG) of the apps and by extract-ing the nodes that define events, actions, user inputs,etc. For the specific purpose of this work, we targetSamsung SmartThings applications which are written inGroovy. Groovy is a Java-based programming languagethat defines visitors for app methods and variables throughthe ASTTransformationCollectorCodeVisitorat compile time [22]. These visitors are then used to con-struct the Abstract Syntax Tree (AST) of the smart apps.The ITM contains a customizer that (1) visits each node ofthe ICFG looking for forensic-relevant points and (2) modifythe apps by inserting forensic logs in such points.

In Listing 1 and Listing 2, we show how a smartapplication is modified by IoTDots for forensic purposes.After performing the source code analysis of the originalapp (see Listing 1), IoTDots flags a user-defined inputrecipients. This input defines the phone number towhich all the notification from the smart apps will be sent to.As explained earlier in this section, these types of inputs arevery critical for forensic purposes since careless or malicioususers can modify this info anytime without being noticed.This can negatively impact the purpose of tracking the eventsand actions executed by apps being utilized in a smart envi-ronment. Going back to the previous example, after labelingthe user input, IoTDots modifies the app by inserting aforensic log (see Listing 2). Here, log.iotdots functiondefines an http request that pushes the log informationto the remote ITLD. Then, the information is perpetuallykept in the ITLD and can be accessed/processed anytime aforensic investigation request is received.

Algorithm 1 details the processes performed by IoTDotsto analyze and modify a smart app. During Detection, theapplication source code is loaded into IoTDots modifier(Line 1). Recall that in this work, we are targeting opensource smart app application platforms so availability of thesource code is assumed. Then, the ICFG is calculated in Line2. Once the ICFG is obtained, all the nodes are exploredand all the forensic-relevant points are flagged in Line 5.This step concludes the Detection process in Algorithm 1.

6

IoTDots Modifier Console

Iotdots-modifier.appspot.com

Modify SmartThings App Reset Console Publish This App View Recent Apps

. . .

ecobee.poll()

}

def initialize() {

subscribe(app, appTouch)

private void sendMsgWithDelay() {if (state?.msg) {

send state.msg}

}def appTouch(evt) {

def plugSeLngs = [holdType: "${givenHoldType}"]. . .

Analysis Result Stacktrace

Actions

def initialize() {

private void sendMsgWithDelay() {if (state?.msg) {

send state.msg

}

. . .

Log.iotdots (“Method invoked: initialize”)

Log.iotdots (“Message content: state?.msg”)Log.iotdots (“Message recipient: state?.msg”)

ecobee.poll()subscribe(app, appTouch)Log.iotdots (“Method invoked: subscribe”)

}. . .

Figure 3: We made IoTDots available online at https://iotdots-modifier.appspot.com/.

Finally, the Modification process customizes the smart appsby inserting the IoTDots logs in Line 10.

Algorithm 1: Steps in the IoTDots-Modifier (ITM).1: appSC ← app source code

Detection:2: ICFG← createICFG(appSC)3: if Exists ICFG then4: for nodes in ICFG do5: forensicPT ← forensic-relevant points6: end for7: end if

Modification:8: if forensicPT then9: for points in forensicPT do

10: Insert IoTDots Logs11: end for12: end if

Finally, we made ITM available online at:

https://iotdots-modifier.appspot.com/

The current version of the online tool automatically an-alyzes and modifies smart apps from Samsung SmartThingsplatform. We started implementing IoTDots for SmartThingsapps mainly because (1) this platform defines the highestamount of different devices in the market, (2) it is opensource; so the applications’ source code can be found online,and (3) extensive API documentation is available online [23].Figure 3 depicts details of the online version of ITM. On theleft side, the user simply types or paste the original sourcecode that needs to be modified to enable IoTDots. Then, onthe right panel, the tool returns the modified app logging allthe forensically-relevant source code information points.

C. IoTDots Analyzer (ITA)

ITLD stores logs obtained from smart apps at runtime.This allows the information from events and actions in asmart environment to be utilized later for forensic purposes.For successfully extracting information from the logs, wedefine ITA that performs the following actions on the ITLDdata.

• Labeling: This step is used to label the data inthe ITLD. Once classified, the data can be usedto feed machine learning models to complete theanalysis. The data processing step labels the ITLDdata based on timing information and other forensic-related features. These features may include the typeof devices generating the logs, location of thesedevices, etc.

• Detection: One can think that the labeling of theITLD data is good enough for forensic purposes.However, much more information can be extractedfrom the IoIDots logs. In this work, we introducea framework not only capable of labeling the logsbased on forensic criteria (i.e. timestamp, location,device), but also analyzing the data to infer useractivity and detect the behavior of users and smartapps. For this purpose, ITA applies machine learningtechniques to the ITLD-labeled data to classify andextract red-flagged forensic logs. Red-flagged logscan be used to define events and actions relatedto authorized user activity that change the smartenvironment topology (e.g., unauthorized device re-placement or tampering, changes on user-definedinputs, unauthorized changes on setup topology,misplace or disable of any part of the smart setup,etc.). Also, red-flagged logs can be used by ITAto detect the abnormal or unauthorized behavior ofusers (e.g., users entering to access-limited zonesof the building after work hours) or apps (e.g.,malicious or tampered apps that force the smartsetup to behave differently than expected).

• Infer User Activity: For the purpose of this work,user activity is defined as any intentional or un-intentional action performed by authorized usersthat cause changes in the smart setup and alsoviolates the security policies established for thesmart environment. This type of classification canhold users accountable for tampering or damagingthe smart setup to avoid forensic logs.

• Detect Behavior: For this work, behavior of usersor apps is defined as any user or app action thatintentionally changes the purpose of the apps andalso violates the security policies established forthe smart environment. This type of classificationhold users accountable of a possible misconduct ormalicious activity or, on the contrary, will hold ma-licious or tampered smart applications responsiblefor making the smart setup to behave differentlyand potentially jeopardize the security of the smartenvironment.

To further detail the operations of ITA, we introduce Algo-rithm 2. In Line 1, iotLogs variable is initialized with thecontent of ITLD. Additionally, the initialization step includesupdating the variable policy with the security policies thatwere valid for the smart environment at the time that thelogs were acquired. Then, in Line 5 the IoTDots logs are

7

organized and labeled based on the timing information, thetype of devices that generated the logs, and the locationinformation of the devices. With this information, a machinelearning model is applied on the data to (1) detect useractions in the smart environment (Line 9) and (2) detectbehaviour of users and smart apps, all depending on thesecurity policies established at the time the logs wereacquired (Line 10). Finally, if a forensic violation is detected,the flag is set to TRUE.

Algorithm 2: Steps in the IoTDots-Analyzer (ITA).1: iotdLogs← ITLD2: policy ← smart environment security policies3: anomaly ← FALSE

Labeling:4: for each Log in iotdLogs do5: label Log by T ime, Device, Location6: MLdata← labels7: end for

Detection:8: for data in MLdata do9: Evaluates user activity userAct

10: ML← MLanalyzer(userAct, policy)11: if Anomaly detected in ML then12: anomaly ← TRUE13: end if14: end for

V. IOTDOTS FORENSIC EVIDENCE DETECTION

In this section, we describe the analysis techniques thatwere used to identify user activities and behaviors in a smartenvironment from the collected forensic data. Specifically,we implemented a Markov Chain-based detection techniquein IoTDots.

A. IoTDots Data Characterization

As noted earlier, IoTDots collects data from a smartenvironment in an ITLD that includes timing information,sensor information, device state, location, etc. For a specifictime slot t, the data collected by IoTDots can be representedby:

Data array,Et = {S,D,M,L}, (1)

where S represents the set of sensors’ features, D is theset of device features, M is the features extracted from thecontrolling devices (smartphone/ tablet), and L is the set oflocation features of controlling devices extracted from thelog files. We describe the characteristics of these featuresbelow.

• Timing features (T): A smart environment consistsof several sensors and devices. These sensors anddevices change their states based on different useractivities and commands associated with the smartenvironment. In this context, some devices per-form instantaneous tasks (e.g., switching lights withmotion) while some devices perform a task overa period of time (e.g., changing temperature overtime). IoTDots considers this timing informationas a feature to infer the overall state of the smartenvironment at a specific time to detect the useractivity and behavior.

• Sensor features (S): Sensors in a smart environmentwork as a trigger to different devices. A smartenvironment can have several different sensors (e.g.,motion sensor, temperature sensor, presence sensor,etc.) connected to multiple devices. These sensorssense the changes in devices’ peripheral and helpthe devices to take autonomous decisions such asswitching lights on motion, triggering fire alarmafter smoke, etc. Depending on the nature of the sen-sors, sensor data can be both logical (active/inactive)or numerical values. For IoTDots, we collect bothnumerical values and logical states of the sensorsand create the state of the smart environment ata specific time. We represent the change in bothlogical state and a numerical value of a sensor as abinary input (1 if active/change and 0 otherwise) tocreate a forensic data matrix to train the detectionalgorithm.

• Device features (D): A smart environment supportsdifferent devices that are connected to a smart huband different sensors and also with each other. Thesedevices can perform multiple tasks as standalonedevices (e.g., smart thermostat) or as connecteddevices (e.g., automatic door monitoring with smartlock and smart camera). For different user activitiesand input commands, these devices change theirstates (active/inactive) in an autonomous way. IoT-Dots collects these device state data (active/inactivestate) from the stored log of the devices.

• Controller device features (M): In a smart environ-ment, users can use smartphones/tablets to controlany device from the associated smart apps. IoTDotscollects any control command generated from con-troller devices to understand user activities and on-going tasks in the smart environment.

• Location features (L): The devices connected ina smart environment can be controlled from bothinside and outside of the location. To understandthe on-going activities in a smart environment, itis necessary to observe the exact location of agiven command to the devices. IotDots considers thelocation of both the controller and the smart devicesas a feature to understand any activities occurringin the smart environment.

B. Analytical Model Used in IoTDots

ITA collects the information above and creates a statearray to represent the state of the smart environment atspecific time t. Each element of the state array representsthe value of the features collected by IoTDots mentionedabove. For a specific time t, we consider the combinationof all the features collected by IoTDots as binary numbers(1 for active status and 0 for inactive status). Thus, thestate of the smart environment can be represented as a n-bitbinary number, where n is the number of features extractedfrom the logs. Then, the total number of possible states ofa smart environment with n number of features (sensors’states, devices’ states, controller devices, and locations) canbe m = 2n. IoTDots collects the state information of a smartenvironment over time to train a Markov Chain detection

8

model to detect forensically-valuable behavior from users,smart apps, and devices in the system. The Markov Chainmodel has two basic assumptions which are as follows:(1) the occurrence probability of a specific state dependsonly on the previous one and (2) the transition betweentwo consecutive states is independent of time and doesn’tdepend on any condition. Based on these assumptions, theMarkov Chain model can be illustrated by the followingequation [24].

P (Xt+1 = x|X1 = x1, X2 = x2..., Xt = xt) =

P (Xt+1 = x|Xt = xt),

when, P (X1 = x1, X2 = x2..., Xt = xt) > 0,

(2)

where Xt and Xt+1 denotes state of the smart environmentat time t and t + 1, respectively. Let assume the smartenvironment has the state i at time t and j at time t+ 1. IfPij illustrates the transition probability between state i to j,the state transition matrix for m number of states of a smartenvironment can be represented by the following matrix.

P =

P11 P12 P13 . . . . . . P1m

P21 P22 P23 . . . . . . P2m

. . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . .Pm1 Pm2 Pm3 . . . . . . Pmm

(3)

To calculate each element of the transition matrix, let assumethe smart environment has X0, X1, . . . , XT states at a giventime t = 0, 1, . . . , T . Then, each element of the transitionmatrix can be represented by the following equation [24]:

Pij =Nij

Ni, (4)

where Nij is the total number of transition from Xt toXt+1 over a certain period. From the state transition ma-trix, Markov Chain model can calculate the probability ofoccurring a state or sequence of states. To predict the prob-ability of occurring a state, let assume the initial probabilitydistribution of the Markov Chain model as follows:

Q = [q1 q2 q3 . . . . . . qm,] (5)

where, qm is the probability that the model is in state m attime 0. The probability of observing a sequence of statescan be calculated by the following equation:

P (X1, X2, . . . , XT ) = qx1

T∏2

PXt−1Xt. (6)

C. IoTDots Data Binarization

Due to the nature of the IoTDots data, the acquired logsare not always Markov Chain-ready. In some cases, oneneeds to implement a binarizer that converts numeric data(e.g., sensor readings) into binary values that can be directlyinterpreted by the Markov Chain model. We found that, forthe types of devices and sensors utilized in our evaluation,only a few cases of sensor readings require binarization. Inthese specific cases, IoTDots utilizes user-defined inputs inthe smart apps to define threshold values that automaticallyconvert numeric readings into binary values. For instance,for a temperature sensor, IotDots logs the sensor value

IoTDots Logs Data Type RequiresBinarizer Comments

Location3 Binary –Device Binary –

Sensor states Binary –Sensor values Numeric user-define inputs

Controller Binary –

Table II: IoTDots implements a binarizer to convert numericlogs into binary values.

for every device state change. Then, during analysis, thesevalues are compared against the temperature value thatthe user set at install time. For sensor readings over thethreshold, IoTDots feeds a value 1 to the Markov Chainmodel for that specific variable. On the other hand, for sensorreadings below the threshold, IoTDots feeds a value 0 tothe Markov Chain model. The use of user-defined inputs toimplement the binarizer provides information to the IoTDotsframework that may help to determine if the devices werecompromised or were behaving in an unexpected way duringthe forensic incident. Table II summarizes the use of thebinarizer for different types of data in IoTDots.

We extract the state information of the smart environ-ment from the collected logs and train the Markov Chainmodel. IoTDots then determines the on-going activities onthe smart environment based on the state transition matrixexplained in Equation 3. IoTDots predicts the next state ofthe environment and then match the predicted value with thestate inferred from the dataset to determine if the currentstate has any forensic value or not.

VI. EVALUATION AND DISCUSSION

In this section, we evaluate the efficacy of IoTDots todetect forensic incidents such as regular and anomalous useractivities as well as malicious user and smart app behaviorsin a given smart environment. For a better analysis, weconsider a hypothetical situation where a number of usersperform regular activities inside a smart office environmentwhile some others behave anomalously (e.g., by accessingoffice locations during unauthorized hours). Also, we con-sider another group of users that try to disable some smartdevices to change the configuration of the smart environmentand some malicious apps that poison the data from a specificnumber of devices. As detailed in Section IV, our frameworkneeds to first detect the forensically-valuable user activitiesto then classify them in regular or anomalous activities basedon the security policies in place. In parallel, IoTDots has tobe able to detect any forensically valuable behavior. We builtthe Markov Chain-based detection method for this purposeand train our model with data collected by IoTDots froma realistic smart office environment. Also, we evaluated thegeneral overhead introduced by IoTDots to smart devicesand cloud-based servers.

A. Training Environment and Data Collection

To test the efficacy of the IoTDots performance, wecollected daily usage data from a smart office environment.First, for training purposes, we implemented an emulator

3Refers to the Location Modes ”Office” or ”Other”.

9

environment where day-to-day user activities could be sim-ulated in a timely order. Then, the same activities wereexecuted in real-life settings to collect log data from thereal smart apps.

1) Training Environment Setup: IoTDots is evaluatedusing smart apps and devices from Samsung SmartThingsplatform. This allows performing tests using an IoT pro-gramming platform that defines the highest number of smartdevices and has one of the largest market share [25]. Also,SmartThings allows user-specific apps to be installed ina system which enables a multiple apps environment. Webuilt a real-life setting with different devices and popularsmart home apps available in the SmartThings App Market.A detailed list of the different device types used in theexperiments is given in Table III.

DeviceType Model

Smart Home Hub Samsung SamrtThings Hub. Smart Light Philips Hue Light Bulb

Smart LockYale B1L Lock withZ-Wave Push Button

Deadbolt

Fire AlarmFirst Alert 2-in-1

Z-Wave Smoke Detector andCarbon Monoxide Alarm

Smart Monitoring System Arlo by NETGEARSecurity System

Smart Thermostat Ecobee 4 Smart ThermostatMotion SensorLight Sensor

Temperature SensorFibaro FGMS-001 Motion Sensor

Door Sensor Samsung Multipurpose Sensor

Table III: List of smart devices used during the experiments.

Our set of forensically-valuable activities includes thefollowing scenarios:

• Time-dependent access: We provide user access toall the devices during office hours to observe useractivities in the smart environment.

• Restricted access: For specific places in the smartenvironment, we enforce restricted access policies.

• Multi-user environment: We consider a multi-userenvironment where different users perform differentactivities inside the smart environment. In a multi-user scenario, we also consider time-dependent ac-cess and restricted access to emulate the real-lifesmart office environment.

2) Data Collection: To enable the data collection, weused ITM to automatically modify the SmartThing appsby performing source code analysis to detect forensically-relevant information inside the source code and insert spe-cific logs. Then, while utilizing the apps, IoTDots data wassent to the ITLD. We collected data from the interactionof 10 different users and 22 smart devices (10 differenttypes of devices and sensors, as shown in Table III) andsensors for seven days for a total of 84209 forensically-valuable data incidents. Our dataset included data collectedfrom benign and malicious environments. For the first case,the users were allowed to perform anomalous activitiesin clear violation of the security policies defined for our

experiments. These anomalous activities are described inSection III. In total, we collected benign forensic data from30 different experiments with over 3000 instances for fivedifferent forensically-valuable anomalous behaviors.

For collecting data from a malicious forensic environ-ment, we considered two different cases with users andsmart apps. For the case of users, we considered userstrying to modify, tamper, remove, or destroy the deviceswith the intention of changing the original setup of thesmart environment and prevent that forensic logs can besent to the ITLD for future analysis. On the other hand, forthe apps, we installed different popular SmartThings appsalong with multiple malicious apps [26]. We created fourdifferent forensic scenarios (see Section III) and built theircorresponding smart apps similar to [12], [27] based onthe threat model explained in Section III. For Threat 2, wecreated two different apps for a smart lock that leaked theaccess code to an unauthorized person. This also representsdevice tampering by authorized users. For Threat 3, we builtan app that injected forged data in a fire alarm and triggeredthe alarm maliciously. For Threat 4, we developed an appthat could shut down the smart thermostat for a specificinput temperature. Finally, for Threat 5, we created an appfor a smart camera that could be triggered by a specific lightpattern. In total, for these different scenarios, we collected50 different forensic datasets to test the effectiveness ofIoTDots.

We also note that we used 75% of the user data totrain the Markov Chain model and then combined the rest25% of the data along with malicious data to test ITA.For performance metrics, we utilized standard parameter,including accuracy, F-score, True Positive Rate (TPR), FalsePositive Rate (FPR), True Negative Rate (TNR), and FalseNegative Rate (FNR).

B. Forensic Activity Detection from Users

The state of the interconnected devices inside the smartenvironment depends on the on-going user activities. Forexample, while a user moves from one place to another,several devices and sensors become active. The changeson device states can be an instantaneous event (a specificevent at a specific time such as switching on a light) ora combination of sequential events over a period of time(motion from one place to another).

UserActivity TPR FNR TNR FPR ACC F-

Score

Activity-1 0.9914 0.0086 0.9371 0.0629 0.9823 0.9635Activity-2 0.9926 0.0074 0.9334 0.0666 0.9874 0.9621Activity-3 0.9886 0.0114 0.9619 0.0381 0.9830 0.9750Activity-4 0.9687 0.0313 0.8714 0.1286 0.9484 0.9175Activity-5 0.9555 0.0445 0.8708 0.1292 0.9388 0.9112

Table IV: Performance evaluation of IoTDots for inferringforensically-valuable user activities.

Table IV illustrates the detailed evaluation of IoTDotsfor user activity inference. One can observe that for time-independent actions (i.e., Activity 1-3), IoTDots can achieveboth accuracy (i.e., ACC) and F-score over 98% and 96%,respectively. True Positive Rate (TPR) and True NegativeRate (TNR) are also high (over 98% and 93%, respectively).

10

1 2 3 4 5 6 7 8 9 10

Number of Users

85

90

95

100

Acc

urac

y (%

)

Forensic Activity-1Forensic Activity-2Forensic Activity-3Forensic Activity-4Forensic Activity-5

(a)

2 4 6 8 10 12 14 16

Number of Tampered Devices

92

94

96

98

100

Acc

urac

y (%

)

Forensic Behavior-1

(b)

2 4 6 8 10 12 14 16

Number of Devices

90

92

94

96

98

Acc

urac

y (%

)

Forensic Behavior-2Forensic Behavior-3Forensic Behavior-4Forensic Behavior-5

(c)

Figure 4: IoTDots evaluation results for different forensic incidents: (a) Accuracy of IoTDots on inferring activities in multi-user scenarios, (b) accuracy of IoTDots on detecting forensic behavior from users versus the number of tampered devices,and (c) accuracy of IoTDots on detecting forensic behavior from smart apps versus different number of devices. The differentcurves represent the performance of IoTDots for activities and behaviour as described in Section III.

On the other hand, for time-dependent actions (i.e., Activity4-5), IoTDots achieves the highest accuracy and F-score of94.84% and 91.75%, respectively. Since the time-dependentactions are related to user motions, different users mayhave different patterns of motion which increases the FalsePositive rate (FPR) and False Negative rate (FNR). In sum-mary, IoTDots achieves over 93% of accuracy for detectingdifferent user activities forensically. User action inference inthe smart environment also depends on the number of userspresent in the environment.

For the case of a multi-user smart environment, IoTDotsrequires a different analysis since users can perform differenttasks at once which can directly impact the accuracy ofthe user action inference. In Figure 4(a), the accuracy offorensic action inference is shown with respect to the numberof present users in the smart environment. As analyzed, itcan be observed how the accuracy values decrease with theusers. From Figure 4(a), one can observe that for time-independent activities (Activity-1, Activity-2, Activity-3),IoTDots achieves accuracy in the range of 98% to 95%.For time-dependent activities (Activity-4, Activity-5), theaccuracy of IoTDots varies from 95% to 86% as the numberof users increases.

C. Detection of Forensic Behavior from Users

Smart devices installed in a smart environment are vul-nerable to device tampering which can lead to maliciousevents. In previous works that proposed the use of IoTdata for forensic investigations, device integrity was alwaysassumed and the results were vulnerable to insiders [1].IoTDots can detect tampered devices based on the collectedlogs from the smart environments. For this, the MarkovChain model analyses the state of all the devices in thesmart environment at any given time and detects maliciousor unexpected states by comparing data from similar de-vices sharing the same context (i.e., device cooperation).During device cooperation, if one device is compromisedor tampered, the information collected from other trusteddevices inside the environment is used to detect the onereporting fake data. On the other hand, if the majority ofthe devices are compromised, the overall system cannotbe trusted which will impact the evaluation. For example,consider a smart light that is controlled by a motion sensor.During normal operations, if the sensor detects any motion,

the light will be on. However, in the event of a compromisedstate of the smart light, the light will not operate properlywith the motion sensor. On the other hand, if the motionsensor is compromised too, it would be hard to define whichdevice is hampering the normal operation of the devices.Figure 4(b) depicts the accuracy of IoTDots on detectingtampered or modified devices in a smart environment. Forthis, we installed 22 different devices (including smartsensors) in a smart office environment. One can notice fromFigure 4(b) that IoTDots can achieve near 100% of accuracyon cases with 2 tampered devices in the environment. Ingeneral, the accuracy of IoTDots decreases as the numberof tampered devices increase in the system. Finally, Figure4(b) demonstrate how device cooperation is affected by thenumber of compromised devices. One can observe how theaccuracy values drop when more than 10 devices (near 50%of the total number of devices) are compromised.

D. Detection of Forensic Behavior from Smart Apps

IoT programming platforms offer customized apps tocontrol the smart devices. In recent years, researchers havereported several malicious apps that can perform maliciousactivities in a smart environment [28], [29]. In this section,we test the efficacy of IoTDots in detecting behaviors in asmart environment caused by apps installed on the devices.As noted earlier, we consider four different scenarios to eval-uate app behavior in IotDots (Section III). To evaluate thesescenarios, we installed malicious IoTDots-modified apps ina real-life smart environment (smart office). Table V showsthe evaluation of IoTDots in detecting app’s behavior in thesmart environment. One can observe that IoTDots achieveshigh accuracy and F-score for all the aforementioned cases(over 95% and 89%, respectively).

BehavioralModel TPR FNR TNR FPR ACC F-

Score

Behavior-2 0.9564 0.0436 0.8479 0.1521 0.9468 0.8989Behavior-3 0.9669 0.0331 0.9333 0.0667 0.9636 0.9498Behavior-4 0.9725 0.0275 0.9289 0.0711 0.9670 0.9502Behavior-5 0.9647 0.0353 0.8953 0.1047 0.9572 0.9287

Table V: Performance evaluation of IoTDots in detectingforenscially valuable behavior from smart apps.

In Figure 4(c), the accuracy of IoTDots is shown for

11

different forensic cases with respect to the number of devicespresented in the environment. IoTDots achieves the highestaccuracy of over 97% for Behavior-3 and the lowest accu-racy of 92.3% for Behavior-2 in the case of only one deviceinstalled in the system. With the increase in the numberof devices, the accuracy decreases to 95% and 90% forBehavior-3 and Behavior-2, respectively. The accuracy ofdetecting Behavior-4 and Behavior-5 varies between 97% to95% and 94.8% to 93% with the number of devices in thesystem, respectively.

http sync calls http async calls http async calls with0

500

1000

1500

2000

2500

Ave

rage

Lat

ency

(m

s)

2400ms

280ms25ms

AtomicState vars

Figure 5: Average latency imposed by IoTDots to smartapps’ execution times. The maximum latency (2.4s) is ob-tained when synchronous HTTP requests are utilized to sendthe logs to ITLD. Then, the latency is significantly reducedto 280 ms after using the asynchronous HTTP requests.Finally, the minimum latency (25ms) is obtained after com-bining the asynchronous HTTP request with AtomicStatequeuing.

E. System Overhead

Since most of the smart apps are cloud-based, we donot expect that IoTDots impose any overhead on the smartdevices. However, it is necessary to evaluate the impact ofIoTDots on the computing resources of the IoT smart appsservers in the cloud. To assess this, we selected two metricsthat are directly related with (1) the amount of physicalmemory occupied by the IoTDots-modified apps and (2) thelatency of the smart apps in their operations and responses.

Physical Memory: Since server space is costly, we needto evaluate how much more physical space the IoTDots-modified apps require from the cloud servers if comparedwith the original smart apps (previous to IoTDots modifica-tion). On average, ITM adds around 110 new lines of codeto the original smart apps source code, which represent anincrease of around only 5KB of physical memory space perapp.

Latency: For the purpose of this work, we define latencyas the extra delay imposed by the IoTDots-modified appssource code to the original execution time of the smart apps.In general, the latency depends on (1) the number of timesthat the IoTDots logs are sent to the cloud and (2) theIOTDots log function execution time. In terms of latency,the worst case scenario is obtained when an individualsynchronous http request is called for every single IoTDotslog. In that case, around 2.4s latency is added every timea log is sent from the app to ITLD, which represents abig overhead to the app operation. To avoid this latency,IoTDots utilizes asynchronous HTTP request to send data

to ITDL. Since we are evaluating apps from the SamsungSmartthings platform, we utilized the beta version of theasynchttp_v1 class [23] to asynchronously send the logsto the database. Further, we reduced the latency overheadeven more by avoiding http requests every time a newlog is generated. For this, we utilized the AtomicStatevariable in SmartThings to queue and map several logstogether before communicating with the ITDL. Finally, withthese modifications, the average latency overhead imposedby IoTDots is reduced to around 30 ms for every httprequest to the ITDL (see Figure 5).

F. Summary

Overall, our evaluation results showed that IoTDots caninfer and classify forensically valuable user activities byconsidering changes in sensor reading and device states. Forinferring such activities, IoTDots pairs user activities withthe security policies defined for the smart environment todetect the activities. This may help the forensic investigatorsto know what was happening before, during, and after thetime of the analyzed incidents. Furthermore, forensic resultscan be extended by considering the sequence of the differentdevice states in the smart environment over time, which mayhelp to create a timeline of the incidents. Our results showedthat forensically-valuable information can be easily detectedwith high accuracy using the previously mentioned features.This may help the investigator to further analyze if incidentswere caused due to users’ negligence or due to specificactivities (either benign or malicious).

VII. BENEFITS, CHALLENGES, LIMITATIONS, ANDFUTURE WORK

There are several benefits associated with the use ofIoTDots. Also, some challenges and limitations can behighlighted.

1) Forensic Framework for Smart Environments:Smart environments are equipped with a myriad ofdifferent smart devices and sensors. This representsa new domain of relevant data that can be used forforensic purposes. In this context, IoTDots emergesas the first practical solution that collects, stores,and processes smart environment data to instrumentforensic analysis.

2) Automatic Log Insertion: IoTDots-Modifier auto-matically analyzes smart apps source code to iden-tify and log forensic-relevant data. Then, the logsare sent to a database where the data is kept forfuture analysis. The current version of IoTDotsperforms automatic analysis and modification forSamsung SmartThings apps only. The new versionwith support to other IoT programming platformsis currently under development. Despite most IoTapps follow similar architectural paradigms, dif-ferences in the programming languages and otherplatform-specific features make the expansion ofIoTDots to other platforms a challenge.

3) Deep Data Analysis: IoTDots-Analyzer incorpo-rates data processing and machine learning tech-niques to label the IoTDots data and detect

12

Tool Name IFACrossApp

Analysis

ConsiderDevices

No PlatformModification

ConsiderTampered

DevicesGoals & Comments

FlowFence Protects data by enforcing data flow policies from users.

ContextIoT Detects malicious data flow by analyzing appcontext in a simulation environment.

ProvThingsDetects malicious data flow by analyzing app context in

real environments. Considers app-device and device-deviceinteractions.

SaINT Detects data exflitrations from smart apps.

IoTDotsEnables forensics analysis in smart environments.

Assumes device-device, app-device, and user-deviceinteractions. Considers tampered devices.

Table VI: Comparison between IoTDots and other IFA tools.

forensically-valuable anomalous activity and be-havior from users, smart apps, and devices in asmart environment. The current version has shownexcellent results on accuracy metrics for differentcases of numbers of users and devices. However,analysis of new evaluation cases may be necessary.For instance, different machine learning algorithmsmay be considered to avoid conversion of the IoTdata into Markov binary states.

4) Low overhead: IoTDots imposes minimal to nooverhead to the devices deployed in the smart envi-ronment and very low overhead to the cloud serverssupporting the smart apps. In general, IoTDots-modified apps contain around 110 lines more onaverage if compared with the original smart apps.Also, the latency imposed by sending the IoTDotslogs to the cloud is very low.

5) Applicability: The approach used in IoTDots canbe easily generalized to any IoT platform. Forthe current version, IoTDots takes advantage ofopen source platforms to enable simplicity. Closed-source IoT systems like HomeSeer [30] andHomeOS [31] may require modification on ITMto allow source code analysis of the apps and theimplementation of an effective logging system.

As future work, we aim to incorporate new smart appplatforms to IoTDots. Since different platforms may usedifferent programming languages and APIs, the ITM mayrequire some architectural modifications to enable sourcecode analysis. Also, future versions of our framework willincorporate new threats to the analyzer as new types ofdevices and user behaviors are considered in the smartenvironment.

VIII. RELATED WORK

Smart environments such as smart home systems havebecome popular in recent years. However, there exists nocomprehensive platform for forensic analysis of data froma smart environment that considers the interaction betweensmart devices, users, and the security policies in place.In this section, we discuss existing forensic data analysisplatforms for smart environments and their shortcomings.

1) Forensic Data Collection from the Smart Environment:Researchers have proposed different approaches to collect

forensic data from smart environment and smart devices.Some of these works only focus on vendor-specific de-vices [32], while others present general methods to onlycollect forensic data without any data analysis option [33],[34]. Kebande et al. proposed a generic approach, DFIF-IoT, to analyze digital forensic data in IoT settings [35].Here, the authors presented a generalized method to captureforensic data from IoT devices including cloud, network,and device-level forensic data. However, DFIF-IoT onlydiscusses the theoretical approach to collect forensic datafrom IoT environment without any real-life implementationand evaluation, which decreases the practicality of this work.Zawoad et al. proposed FAIoT, a forensic-aware eco-systemto collect forensic data from IoT platforms [36]. FAIoTpresents a general platform to collect forensic data in asystematic way and organizes data for further analysis. Oneshortcoming of this work is that there is no implementationof data analysis as part of the framework. Chung et al. pro-posed a forensic framework to collect and analyze forensicdata in a IoT eco-system [37]. However, this solution islimited to Amazon Alexa ecosystem.

2) Smart Data Logging: The idea of logging informationfrom smart devices and apps is not novel. Several previousworks have used this technique to have access to informationas a result of smart app executions. In fact, the mostpopular IoT programming platforms provide the ways forlogging smart app data [38], [39], [40], [41]. Personalizedopen-source solutions can also be found [42], [3]. In thiscontext, solutions like the proposed in [1] try to providea comprehensive analysis of smart apps and smart deviceslogs for security purposes. In this work, the authors proposea platform-centric approach that looks at activities fromsmart apps for data provenance purposes. In this case,although this work is instrumental, the analysis is limited toconsider the temporal relationship between devices and appsevents without considering compromised devices, essentialfor forensic analysis.

3) IoTDots and Information Flow Analysis in IoT:Recently, information flow analysis (IFA) has moved fromtraditional research domains like android to IoT. For fairness,we believe it is necessary to evaluate the differences betweenIoTDots and some of the most popular IFA tools in themarket today. In general, IoTDots achieves the capabili-ties of all the considered tools with high accuracy, lowoverhead, but while considering a comprehensive forensic

13

model that includes challenges related to careless users,malicious users, malicious apps, and tampered devices. Ad-ditionally, more than policy enforcement or data provenancein IoT, IoTDots provides intelligence to match securitypolicies in smart environments with interactions betweensmart apps, devices, and users. Table VI summarizes themajor differences between FlowFence [43], ContextIoT [44],ProvThings [1], SaINT [12], and IoTDots. For clarity, weused the analysis criteria similar to the one proposed in [1].Then, we expanded the table features based on the IoTDotscontributions.

Compared to these prior works, IoTDots presents acomprehensive solution that automatically collects/storesforensically-relevant data from a smart environment con-sidering (1) the interaction between users, devices, andapps inside the smart environment and (2) the securitypolicies defined for every different environment. Additionally,our framework provides the means to analyze this dataand detect anomalous activities from users and maliciousbehavior from users, smart apps, and devices. IoTDots flagsthose activities and behaviors that potentially violate thesecurity policies of the smart environment and may helpto hold the perpetrators accountable in a holistic mannerduring forensic investigations.

IX. CONCLUSIONS

Smart devices and sensors deployed in smart environ-ments have access to data that can be used for foren-sic purposes. Nonetheless, current smart app programmingplatforms do not provide any digital forensic capabilityto keep track of such information. Additionally, currentforensic analysis solutions do not use information fromsmart apps and/or smart devices to perform forensic in-vestigations. In this work, we introduce IoTDots, a novelframework used to extract forensically-relevant logs fromsmart apps and automatically analyze them later for foren-sic purposes. The framework has two main components:IoTDots-Modifier and IoTDots-Analyzer. The Modifier per-forms smart apps’ source code analysis, detects forensically-relevant data points inside the smart app’s source code, andinserts specific logs at compile time. Then, at runtime, thelogs are sent to a remote IoTDots server. In a case of aforensic investigation, the Analyzer applies data processingand machine learning techniques to extract valuable andusable forensic information from the IoTDots logs. Asper the evaluation results, IoTDots achieves over 98% ofaccuracy on detecting user activities and over 96% accuracyon detecting the behavior of users and smart apps in thesmart environment. Additionally, IoTDots performs withvery minimal to no overhead on the smart devices testedand very low overhead to the IoT cloud server resources.Finally, we have made IoTDots-Modifier available online.

REFERENCES

[1] Q. Wang, W. U. Hassan, A. J. Bates, and C. Gunter, “Fear andlogging in the internet of things,” in Network and Distributed SystemsSymposium (NDSS), Feb 2018.

[2] L. Babun, H. Aksu, and A. S. Uluagac, “Identifying counterfeit smartgrid devices: A lightweight system level framework,” in 2017 IEEEInternational Conference on Communications (ICC), May 2017, pp.1–6.

[3] mattjfrank. (2018, May) Smartthings logging. [Online].Available: https://github.com/krlaframboise/SmartThings/blob/\master/smartapps/krlaframboise/simple-event-logger.src/simple-event-logger.groovy

[4] Malware found in surveillance cameras sold through Amazon,https://www.techworm.net/2016/04/malware-found-surveillance-cameras-sold-amazon.html, 2017, [Online; accessed 10-January-2018].

[5] R. Hegarty, D. J. Lamb, and A. Attwood, “Digital evidence chal-lenges in the internet of things.” in INC, 2014, pp. 163–172.

[6] Y. Yusoff, R. Ismail and Z. Hassan, “Common phases of computerforensics investigation model,” International Journal of ComputerScience & Information Technology (IJCSIT), vol. 3, no. 3, June 2011.

[7] R. Udeshi. (2018) Why you need forensics in an iot world. [Online].Available: https://www.guidancesoftware.com/blog/digital-forensics/2017/10/03/why-you-need-forensics-in-an-iot-world

[8] B. P. Kondapally. (2018) What is iot forensics and howis it different from digital forensics? [Online]. Available:https://securitycommunity.tcs.com/infosecsoapbox/articles/2018/02/27/what-iot-forensics-and-how-it-different-digital-forensics

[9] H. Aksu, L. Babun, M. Conti, G. Tolomei, and A. S. Uluagac,“Advertising in the iot era: Vision and challenges,” IEEE Communi-cations Magazine, pp. 1–7, 2018.

[10] U. Salama. (2018) Smart forensics for the internet of things (iot).[Online]. Available: https://securityintelligence.com/smart-forensics-for-the-internet-of-things-iot/

[11] Samsung. (2018) Smartthings official app repository. [Online].Available: https://github.com/SmartThingsCommunity

[12] Z. B. Celik, L. Babun, A. K. Sikder, H. Aksu,G. Tan, P. McDaniel, and A. S. Uluagac, “Sensitiveinformation tracking in commodity iot,” in 27th USENIXSecurity Symposium (USENIX Security 18). Baltimore,MD: USENIX Association, 2018. [Online]. Available: https://www.usenix.org/conference/usenixsecurity18/presentation/celik

[13] L. Babun, Z. Berkay Celik and A. Kumar Sikder. (2018) Saintproject. [Online]. Available: http://saint-project.appspot.com/

[14] Samsung. (2018) Smartthings community forum for third-partyapps. [Online]. Available: https://community.smartthings.com/

[15] OpenHAB. (2018) Openhab iot app market (eclipse market place).[Online]. Available: http://docs.openhab.org/eclipseiotmarket

[16] Apple. (2018) Apple’s homekit app market. [Online]. Available:https://support.apple.com/en-us/HT204893

[17] Samsung. (2018) Smartthings code review guidelines and bestpractices. [Online]. Available: http://docs.smartthings.com/en/latest/code-review-guidelines.html

[18] OpenHAB. (2018) Openhab iot app submission guideline.[Online]. Available: https://marketplace.eclipse.org/content/eclipse-marketplace-publishing-guidelines

[19] Apple. (2018) Apple’s homekit submission guideline. [Online].Available: https://developer.apple.com/app-store/review/guidelines

[20] Samsung. (2018) Smartthings official api documentation.[Online]. Available: http://docs.smartthings.com/en/latest/ref-docs/reference.html

[21] OpenHAB. (2018) Openhab privacy statement. [Online]. Available:http://www.myopenhab.org/privacy

[22] GroovyCodeVisitor. (2018) Groovycodevisitor: An implemen-tation of the groovy visitor patterns. [Online]. Available:http://docs.groovy-lang.org/docs

[23] Samsung. (2018) Smartthings official developer documentation.[Online]. Available: http://docs.smartthings.com

[24] A. K. Sikder, H. Aksu, and A. S. Uluagac, “6thSense: A Context-aware Sensor-based Attack Detector for Smart Devices,” in USENIXSecurity, 2017.

[25] Samsung. (2018) Samsung smartthings. [Online]. Available: https://www.smartthings.com/

[26] L. Babun, Z. Berkay Celik and A. Kumar Sikder. (2018) Iotbenchrepository. [Online]. Available: https://github.com/IoTBench

[27] ContexIoT attacks for SmartThings programs using existing adver-

14

sary techniques, https://sites.google.com/site/iotcontextualintegrity/home, 2017, [Online; accessed 09-January-2018].

[28] E. Fernandes, J. Jung, and A. Prakash, “Security Analysis of Emerg-ing Smart Home Applications,” in IEEE Security and Privacy (SP),2016.

[29] J. Zhu. (2018) Android-based smart tvs hit bybackdoor spread via malicious app. [Online]. Available:https://blog.trendmicro.com/trendlabs-security-intelligence/android-based-smart-tvs-hit-by-backdoor-spread-via-malicious-app/

[30] HomeSeer. (2018) Homeseer. [Online]. Available: https://homeseer.com/

[31] Microsoft. (2018) Homeos. [Online]. Avail-able: https://www.microsoft.com/en-us/research/project/homeos-enabling-smarter-homes-for-everyone/

[32] H. Chung, J. Park, S. Lee, and C. Kang, “Digital forensic investiga-tion of cloud storage services,” Digital investigation, vol. 9, no. 2,pp. 81–95, 2012.

[33] E. Oriwoh, D. Jazani, G. Epiphaniou, and P. Sant, “Internet of thingsforensics: Challenges and approaches,” in Collaborative Computing:Networking, Applications and Worksharing (Collaboratecom), 20139th International Conference Conference on. IEEE, 2013, pp. 608–615.

[34] S. Watson and A. Dehghantanha, “Digital forensics: the missingpiece of the internet of things promise,” Computer Fraud & Security,vol. 2016, no. 6, pp. 5–8, 2016.

[35] V. R. Kebande and I. Ray, “A generic digital forensic investigationframework for internet of things (iot),” in Future Internet of Thingsand Cloud (FiCloud), 2016 IEEE 4th International Conference on.IEEE, 2016, pp. 356–362.

[36] S. Zawoad and R. Hasan, “Faiot: Towards building a forensics awareeco system for the internet of things,” in Services Computing (SCC),2015 IEEE International Conference on. IEEE, 2015, pp. 279–284.

[37] H. Chung, J. Park, and S. Lee, “Digital forensic approaches foramazon alexa ecosystem,” Digital Investigation, vol. 22, pp. S15–S25, 2017.

[38] Wink. (2018, May) Wink. [Online]. Available: https://www.wink.com/

[39] Lowe’s. (2018, May) Iris by lowe’s. [Online]. Available: https://www.irisbylowes.com/

[40] MiCasaVerde. (2018, May) Logs - micasaverde. [Online]. Available:http://wiki.micasaverde.com/index.php/Logs

[41] Samsung. (2018, May) Smartthings official logging. [On-line]. Available: http://docs.smartthings.com/en/latest/tools-and-ide/logging.html

[42] K. LaFramboise. (2018, May) Simple event logger. [Online].Available: https://github.com/krlaframboise/SmartThings/blob/\master/smartapps/krlaframboise/simple-event-logger.src/simple-event-logger.groovy

[43] E. Fernandes, J. Paupore, A. Rahmati, D. Simionato, M. Conti, andA. Prakash, “FlowFence: Practical Data Protection for Emerging IoTApplication Frameworks,” in USENIX Security, 2016.

[44] Y. J. Jia, Q. A. Chen, S. Wang, A. Rahmati, E. Fernandes, Z. M. Mao,A. Prakash, and S. J. Unviersity, “ContexIoT: Towards ProvidingContextual Integrity to Appified IoT Platforms,” in NDSS, 2017.

15