Mimic: Raw Activity Shipping for File Mimic: Raw Activity Shipping for File Synchronization in Mobile File SystemsSynchronization in Mobile File Systems

Tae-Young Chang, Aravind Velayutham, and

Raghupathy Sivakumar

School of Electrical and Computer Engineering

Georgia Institute of Technology

Atlanta, GA 30332, USA

2004-06-10 2 of 27

File Synchronization in Mobile File SystemsFile Synchronization in Mobile File Systems

• File synchronization in distributed file systems is a consistency

restoration process between a file server and a client

• Over mobile networks such as a WWAN, file synchronization in

traditional file systems cannot be performed effectively due to

limited bandwidth

Bandwidth usage efficiency of file synchronization is important

File Server Mobile Client

Internet

Base Station

File Access

File Synchronization

2004-06-10 3 of 27

BandwidthBandwidth--Efficient File Synchronization (1)Efficient File Synchronization (1)

• Data compression scheme

– Compresses each block of data or meta-data

– On-line data compression in a log-structured file system

[Burrows’92]

• Differential update scheme

– Exploits similarities between versions of the same file

• Based on the diff scheme of the UNIX systems

– Rsync [Tridgell’00]

– Low-bandwidth network file systems [Muthi’01]

2004-06-10 4 of 27

BandwidthBandwidth--Efficient File Synchronization (1)Efficient File Synchronization (1)

• Operation shipping

– Logs and ships user operations that update the files

• Session/application level logging and replaying

• Need modification for GUI-based interactive applications

– Corrects replaying errors by forward error correction (FEC)

• Minor re-execution discrepancies caused by non-repeatable

operations can be detected by fingerprint algorithms

– Operation shipping for mobile file systems [Lee’02]

2004-06-10 5 of 27

MotivationMotivation

236 B1543 B29341 B98 keystrokesInsert a line

848 B2111 B29356 B476 keystrokesInsert a paragraph

72 B1119 B33449 B6 keystrokes + 12 mouse-clicks

Copy and paste a paragraph

from the same file

30 B1660 B40611 B7 mouse-clicksChange

the font type of a paragraph

ActivitySize

diffSize

Full FileSize

Activity EventsActivity Description

Update size comparison in Microsoft Word

2004-06-10 6 of 27

Goal and OverviewGoal and Overview

• Goal of our work

– To design an application-unaware activity shipping scheme

for file synchronization in interactive applications

Mimic is a file synchronization strategy that records raw

activity at the client side, ships the records to the server

during synchronization, and replays the activity at the

server

2004-06-10 7 of 27

Mimic System OverviewMimic System Overview

Input Device Input DeviceDriver

Event

Local FileSystem

File

Record

MessageQueue

InputMessage

MimicRecorder

Application

InputMessage

File

Application

InputMessage

MimicFile Verifier

Figerprint/ FEC

File /Record

Network FileSystem

MessageQueue

InputMessageMimic

Replayer

Record

Client

Server

2004-06-10 8 of 27

Mimic Design ElementsMimic Design Elements

• Raw activity recording

– Mapping filename to process/session

– Record optimization

• Raw activity replaying

– Environment synchronization

– Replaying speed optimization

• Integration of Mimic with file systems

• Verification/error correction of replayed files

– Presented in [Lee’02]

2004-06-10 9 of 27

Mimic Client DesignMimic Client Design::

Mimic Client to Window Manager InterfaceMimic Client to Window Manager Interface

processToWindowHandle( )

system message

getEnvironment( )

trapSystemMessage( )

getClipboard( )

Record

OS

WindowManager

MimicClient

FileSystem

WindowManager

MimicServer

MimicSynchronization

Native FileSynchronization

Client Server

OS

FileSystem

2004-06-10 10 of 27

• WHT maps the process (or session) handle and filename

of an application to the corresponding set of window

handles

– When a file is opened with a filename, the mapping

information is acquired, and added on WHT

• through processToWindowHandles( )

– When the file is closed, the mapping is removed from WHT

• Mimic recorder captures system messages having the

window handles listed on the WHT

– through trapSystemMessage( )

Mimic Client DesignMimic Client Design::

Window Handle Table (WHT)Window Handle Table (WHT)

2004-06-10 11 of 27

• Each captured message is translated into a descriptor

• Activity Descriptor (AD)

– Describes an individual user input activity for an application

• Meta Descriptor (MD)

– Capture the changes of system environment during recording

• Includes screen resolution, color depth, keyboard layout, clipboard content, etc.

• Environment Descriptor (ED)

– Describes the initial system environment when recording begins

• Same structure as that of an MD

• through getEnvironment() and getClibboard()

Mimic Client DesignMimic Client Design::

DescriptorsDescriptors

2004-06-10 12 of 27

messageidentifier

0 31

lParam wParam time hwnd

32 63 64 95 96 127 128 15920

bytes

EVENTMSG in Windows

messagetype

0 3

virtual-key code

4 11

time

12 15

messagetype

0

x-coordinate

11

time

12 153 4

y-coordinate4 bytes

messagetype

0 3

MD link information

4 11

time

12 15

Activity Descriptor (Keyboard Activity)

2 bytes

Activity Descriptor (Mouse Activity)

Activity Descriptor (Meta Descriptor Link)

2 bytes

messagetype

0 3

clipboard size

4

clipboard information

15

messagetype

0 3keyboard layout

information

4

2 bytes

messagetype x-coordinate

timey-coordinate4 bytes

variable

messagetype color information time

11

time

12 15

time

Meta Descriptor (Clibboard Content)

Meta Descriptor (Keyboard Layout Change)

0 3 4 11 12 15

Meta Descriptor (Screen Resolution Change)

0 3 4 11 12 15

Meta Descriptor (Color Depth Change)

2 bytes

Mimic Client DesignMimic Client Design::

Descriptor StructuresDescriptor Structures

2004-06-10 13 of 27

• Each descriptor is recorded in a record

• File Activity Record (FAR)

– Maintained per file

– Consisting of file information, a set of environment descriptors (EDs), a sequence of activity descriptors (ADs), and all other information

– Linked to a meta descriptor (MD) of a meta activity record (MAR) when an environment change happens

• Meta Activity Record (MAR)

– Shared by file activity records (FARs)

– Consisting of a sequence of meta descriptors (MDs)

Mimic Client DesignMimic Client Design::

RecordsRecords

2004-06-10 14 of 27

Mimic Client DesignMimic Client Design::

Mimic Client to File System InterfaceMimic Client to File System Interface

open( )

close( )

rename( )

delete( )

finish( )

synch( )/Synch_Status

OS

WindowManager

MimicClient

FileSystem

WindowManager

MimicServer

MimicSynchronization

Native FileSynchronization

Client Server

OS

FileSystem

processToWindowHandle( )

system message

getEnvironment( )

trapSystemMessage( )

getClipboard( )

Record

2004-06-10 15 of 27

• open(filename,mode,process_handle)– Called when a shared file is opened

• close(filename), rename(filename), delete(filename)– Called when a shared file is closed, renamed, or deleted

• synch(filename,diff_size)– Called when a shared file needs to be synchronized

• Decides update mode by comparing FAR_size with diff_size

– Returns Synch_Status• SYNCH_FAIL if Mimic synchronization is failed or diff is chosen

• Updates again on diff mode when Mimic synchronization failed

• finish()– Called when the synchronization process is completed

Mimic Client DesignMimic Client Design::

Integration with File SystemsIntegration with File Systems

2004-06-10 16 of 27

executeDefaultApplication( )

waitForProcessIdle( )

playActivity( )

setEnvironment( )

setClipboard( )

Replay

Mimic Server DesignMimic Server Design::

Mimic Server to Window Manager/OS InterfaceMimic Server to Window Manager/OS Interface

OS

WindowManager

MimicClient

FileSystem

WindowManager

MimicServer

MimicSynchronization

Native FileSynchronization

Client Server

OS

FileSystem

open( )

close( )

rename( )

delete( )

finish( )

synch( )/Synch_Status

processToWindowHandle( )

system message

getEnvironment( )

trapSystemMessage( )

getClipboard( )

Record

2004-06-10 17 of 27

• Environment synchronization

– Based on the environment descriptor (ED) of the FAR

– Initial system environment synchronization

• through setEnvironment()

– Clipboard content synchronization

• through setClipboard()

• Application synchronization

– Opens a corresponding application of the file activity record

• Based on the file extension of the filename

• Sets the same environment such as window size and location

• through executeDefaultApplication()

– Moves the system focus to the application

Mimic Server DesignMimic Server Design::

InitializationInitialization

2004-06-10 18 of 27

• System message/function generation

– Activity descriptor (AD) is mapped into an input system

message

– Meta descriptor(MD) is mapped into a system function to set

the environment

– through playActivity()

• System message/function re-execution

– Deliver the messages to system message queue

– Run the system functions

Mimic Server DesignMimic Server Design::

ReplayingReplaying

2004-06-10 19 of 27

• User activity skipping and misinterpretation

– Certain inputs are relevant to the application only for

particular states

– Too fast replaying may not let the application wait for a

particular state, and cause replaying errors

• Replaying speed control in Mimic

– Monitors the CPU utilization of the process after every

message playback

– Playbacks the next AD, only when the process is idle

• through waitForProcessIdle( )

Mimic Server DesignMimic Server Design::

Replaying Speed ControlReplaying Speed Control

2004-06-10 20 of 27

Experiment SetupExperiment Setup

• Wireless wide area network (WWAN) – CDMA2000-1X cellular network

• Effective data rate: about 17 Kbps

• Round-trip time between the client and server: about 300ms

• Operating system/application– Microsoft Windows 2000 Professional

– Microsoft Office 2000 suite

• Metrics– Transfer size

– Synchronization latency• Includes shipping, replaying, and verification delays

– Compared with the differential update (diff)• xDelta for Windows

2004-06-10 21 of 27

Transfer Size Results (1)Transfer Size Results (1)

Transfer Size for Microsoft Word

0 2 4 6 8 10 12 14 16 18

Insert 1 line (894%)

Insert1 paragraph (182%)

Insert1 full page (96%)

Modify 1 paragraph (851%)

Modify 1 full page (104%)

Delete 1 paragraph (9587%)

Delete 1 full page (10567%)

Copy 1 paragraph from the same file (1554%)

Copy 1 full page from the same file (2389%)

Copy 1 page of an external file (29%)

Copy 1 picture from an external file (79%)

Change the font type of a paragraph (5533%)

Act

ivity

Des

crip

tion

(Rel

ativ

e M

imic

Per

form

ance

, %

)

Transfer Size [KB]

Diff

Mimic

2004-06-10 22 of 27

Transfer Size Results (2)Transfer Size Results (2)

Transfer Size for Microsoft Visio

0 10 20 30 40 50 60 70

Insert 1 new blank page (14155%)

Insert 1 small graph (655%)

Insert 1 large graph (452%)

Modify 1 small graph (12252%)

Modify 1 large graph (2146%)

Delete 1 small graph (89707%)

Delete 1 small graph (48456%)

Copy 1 small graph from the same file (16315%)

Copy 1 large graph from the same file (61602%)

Copy 1 small graph from an external file (38%)

Copy 1 small picture from an external file (393%)

Change the font type of a graph (66867%)

Act

ivity

Des

crip

tion

(Rel

ativ

e M

imic

Per

form

ance

, %

)

Transfer Size [KB]

Diff

Mimic

2004-06-10 23 of 27

• Transfer size in Mimic is generally equal or less than that

in the differential update

– Except when copying from outside the file

• Due to bandwidth-inefficient clipboard structure

• Mimic overhead is generally proportional to activity size

– Except when copying from outside the file

• Transfer size relies on the size of the copied object

– Diff overhead is not proportional to activity size

• Single line insertion in diff may consume more bandwidth than a

single paragraph insertion

Transfer Size Results (3)Transfer Size Results (3)

2004-06-10 24 of 27

Synchronization Latency Results (1)Synchronization Latency Results (1)

Latency for Microsoft Word

0 1 2 3 4 5 6 7 8

Insert 1 new blank page (121%)

Insert 1 small graph (39%)

Insert 1 large graph (36%)

Modify 1 small graph (125%)

Modify 1 large graph (68%)

Delete 1 small graph (1336%)

Delete 1 small graph (1423%)

Copy 1 small graph from the same file (449%)

Copy 1 large graph from the same file (564%)

Copy 1 small graph from an external file (32%)

Copy 1 small picture from an external file (82%)

Change the font type of a graph (887%)

Act

ivity

Des

crip

tion

(Rel

ativ

e M

imic

Per

form

ance

, %

)

Latency [s]

Diff

Mimic

2004-06-10 25 of 27

Synchronization Latency Results (2)Synchronization Latency Results (2)

Latency for Microsoft Visio

0 5 10 15 20 25 30

Insert 1 new blank page (929%)

Insert 1 small graph (97%)

Insert 1 large graph (67%)

Modify 1 small graph (1198%)

Modify 1 large graph (1990%)

Delete 1 small graph (11164%)

Delete 1 small graph (6475%)

Copy 1 small graph from the same file (3747%)

Copy 1 large graph from the same file (13592%)

Copy 1 small graph from an external file (39%)

Copy 1 small picture from an external file (389%)

Change the font type of a graph (9315%)

Act

ivity

Des

crip

tion

(Rel

ativ

e M

imic

Per

form

ance

, %

)

Latency [s]

Diff

Mimic

2004-06-10 26 of 27

• Mimic still shows better latency performance for certain

activity or applications

– Even though the latency in Mimic includes its playback time,

its total update time does not exceed that of diff

• Benefit of small transfer size for those operations is larger than

playback overhead

• However, for the other types of activities, the differential

scheme performs better in terms of latency

– For large insertions and external copies

Synchronization Latency Results (3)Synchronization Latency Results (3)

2004-06-10 27 of 27

ConclusionsConclusions

• We propose an application-unaware approach called Mimic

that depends on transferring raw user activity records to

the server, where the file is updated through a playback of

the raw user activity on the old copy of the file

• We show that Mimic performs much better than diff in

most scenarios in terms of the transfer file sizes

• We conclude that Mimic can be used in tandem with diff to

substantially improve file synchronization performance