Vi rtua 1 Vers U s P h y s ica i Prace s s - M i gmti OR S t rate g i e s i n M ad e I i n y M o b i I e - C u m put i n g N etw U r ks

Kuiun r a n , Sumit Chosh L"..."",.." ...'. " ...... ~ ...... " ...... ".."...1." ..... .... " .....,........... '.I... ~ ...... I ........ " ...... " ....... I .....

he large bulk and substantial power requirements of Iiigli-speed computers o i the pdst fot ced iiiciriy mobile- computing systems lo adopt the traditional. centralized dpprmch wherein sequuntidl dlg~rilhiiis were exrcuted

on cenlralizetl compukrs. The results ~ t r ~ inefficiency and high m s t s caused hy slow sequential processing dnd the need to ptopa- gate data and information from gcographically dispersed siles lo the central computet- dnd relay Lhe decisions bdck to the sites. Ex- mplcs include the CenLrdized Traffic Control (C'fC) (1 1 and the Advanced Train Control Systsm LU'CS) [a ] . both used for rdilway networks. In these systenis, the destination of w r y train is known a priori by the dispatcher-a uniprocessor computer. Ad- ditionally. a dispatcher receives. at regular intewnls of time. the currsnt po~ t ron . speed, and mo\scmcnt of cvery trdin m d the sta- lusts, i.e.. whether occupicd or cnipty. ot cvery track in the sys- tem. The dispatcher utilizes uniproccssor-optimizdtion

switch settings corr

approach distributes the overall task of scheduling among all stations and trains and achieves high efficiency, robustness, and scalability. The new distributed command and control algo- rithm for the battlefield [41, where the traditional information- gathering and decision-generating central headquarters is re- placed by semi-autonomous fighting vehicles, yields superior performance over the traditional scheme under realistic battle conditions. In reality, the new M1-A2 tanks are outfitted with powerful workstations and advanced communications gear to facilitate sophisticated target acquisition, decision-making, ma- neuvering, and firing. Other futuristic services that are cur- rently on the drawing board today include economical access to patient medical records by mobile physicians and a distributed approach for intelligent vehicle highway system.

In this article we focus on the high-level principles that un- derlie the distributed modeling and accurate simulation of mobile-computing networks on a parallel-processing testbed. We first review a technique proposed in the literature called vir- tual process migration (VPM) and analyze its limitations. We then propose a new strategy called physical process migration (PPM) that attempts to address the limitations ofVPM. The soft- ware techniques of both approaches are detailed and their re- spective performance under different scenarios is contrasted.

Mobile-Computing Networks In general, a mobile-computing network may be characterized as follows. It consists of multiple mobile agents that require ac- cess to (i) information generated at multiple geographically dis- persed sites and (ii) computing engines to execute their decisions. It may include one or more stationary agents that per- form information acquisition and propagation to the mobile agents. While a static interconnection network may link the sta- tionary agents, a dynamic interconnection network will connect the mobile agents to the stationary nodes. The mobile nodes may connect to specific stationary nodes asynchronously, i.e., at ir- regular intervals of time, to acquire information, and following completion they will disconnect. The use of the term connection in this context refers to the transport layer in the ISO-OS1 termi- nology [21]. The underlying physical layer, however, is at liberty to utilize either wired or wireless transmission. The mobile and stationary agents are located at geographically dispersed sites. While both stationary and mobile nodes may have computing and communication needs, the relative weights and frequency are problem-specific. In addition, the system must be designed to accommodate evolutionary growth. That is, the system must continue to function and deliver relatively undiminished per- formance as the cumulative number of stationary and mobile entities increases with time.

Review of the literature Koch, Krombholz, and Thee1 [7] introduced the concept of mo- bile computing, defined its key characteristics, and presented its scope of application. Imielinski and Badrinath [8, 181 identified the challenges in data management, arising from the issues of mobile hosts, wireless broadcasting, and frequent disconnec-

CIRCUITS & DEVICES JANUARY 1998

tions, and discussed the necessary structure of the distributed algorithms. Forman and Zahorjan [51 stressed network reliabil- ity, greater autonomy for the mobile agents, asynchronous op- erations, and flexible consistency-semantics toward successful mobile-computing networks. They also raised issues related to frequent and abrupt disconnections, high bandwidth variability, security, and portability.

Satyanarayanan [6, 91 noted important limitations of mobile systems including poor resources, vulnerability to catastrophic failures, and the need to operate under a broad range of network conditions. He proposed the Coda file system, which realizes user-transparent mobility of agents by permitting autonomous operations while disconnected and transparent reintegration changes upon reconnection. He also reported future plans to sup- port very low bandwidth or intermittent connections in Coda.

Duchamp, Feiner, and Maguire [ZO] reported on their early efforts in systems software development for wireless mobile computing. They focused primarily on the issue of software de- sign to permit the movement of mobile hardware platforms without interrupting the high-level software.

In contrast to the reported research efforts [5-9, 18,201, this article is concerned with the issues of modeling and simulation of real mobile-computing networks, which is a key phase that must precede the actual development of such networks. Specifically, this article focuses on the geheral principles underlying the mod- eling and simulation of a specific class of solutions [3, 4, 191 to real-world mobile-computing network problems. The solutions utilize asynchronous, distributed algorithms. The fundamental philosophy here is that the overall decision-making task is intelli- gently distributed among the mobile and stationary entities. This will maximize local computations, minimize communications, and yield a high throughput approach that is efficient, robust, and scalable. As the system experiences evolutionary growth, the ap- proach must continue to function and yield relatively undimin- ished performance. Every entity isviewed as an asynchronous and autonomous process with well-defined computational and com- munications needs. While some processes are “stationary,” others are “mobile” within the network,

The literature on process migration is extensive. However, these research efforts are primarily driven by the need to balance load on parallel-processor networks. Thus, processes are mi- grated by the system, either semi-automatically or automati- cally, from overloaded processors to idle or underloaded processors to achieve equitable distribution of workload, possi- bly leading to faster completion of the task. Artsy and Finkel [ lo] proposed a pre-emptive process-migration facility within the Charlotte distributed operating system wherein running pro- cesses are dynamically relocated among component machines to cope with dynamic fluctuations in loads and service needs, meet real time scheduling deadlines, and improve the system’s fault tolerance. They noted that, often, migrating processes fail and the cost for recovery is likely to incur large overhead.

Glazer and Tropper [ 111 presented a load-balancing algo- rithm for optimistic discrete event simulation that allows for re- adjustments when resource requirements vary or initial

1 1

resource predictions are incorrect. They reported strong per- formance gains in a few simulations where the simulation ad- vancement rate improved by 71% and the rollback rate reduced by 66%. Suen and Wong [la] presented an efficient task- migration algorithm that aims to balance load among proces- sors, reduce communication overhead, and improve resource utilization and average response time. They reported that their approach saves up to 60% of the protocol messages used by broadcast algorithms.

Chen and Shin [ 131 proposed subcube-allocation and task- migration strategies for efficient execution of parallel tasks on hy- percube multiprocessors. Milutinovic, Houstis, and Crnkovic [ 141 presented two distributed task-allocation strategies, LB and LOCO, to facilitate efficient processor utilization. Gait [ 151 proposed a two-tier process scheduler wherein a tunable time quantum is ad- justed so that the average process completes execution on the proc- essor on which it is first scheduled while relatively long-lived processes are rescheduled globally. Lin and Yang [ 161 proposed a load-balancing technique based on a combination of reducing stan- dard deviation of the execution time in each processor and increas- ing the correlation coefficient between every two processor’s execution times. Hac [17] presented a distributed algorithm for performance improvement through file and process migration.

The key characteristics of mobile-computing networks differ from those for load balancing [lo]-[ 171. For application problems of interest to this study, the primary objective is to model and effi- ciently simulate the components of the network, not necessarily to equitably distribute tasks to computing processors. The exact pattern of migration of the mobile agents, in truth, is dictated by the nature of the application and the actual input data. The migra- tion pattern is further complicated by the fact that every mobile process is autonomous, i.e., every mobile entity determines its own migration pattern based on its unique behavior, input stimu- lus, and dynamic interactions with the stationary entities. Every mobile and stationary entity is characterized by unique computa- tion and communication needs. Furthermore, the nature of the migration is asynchronous, i.e., it is initiated at irregular intervals of time and may not be known a priori. Finally, in many real-world mobile-computing networks, the number of mobile and station- ary agents will generally be large, which, in turn, necessitates a distributed, scalable approach.

We focus on a class of real-world, large-scale, mobile-computing networks with the following characteristics: (1) The number of stationary entities is relatively modest, but the number of mi- grating entities is large, ranging from 10s to 100s; (2) the system is likely to grow in size with time, requiring that the underlying approach be scalable; (3) while the stationary entities are geo- graphically dispersed, the mobile entities are autonomous, im- plying that their migration patterns are unique to every mobile agent and are unknown a priori; and (4) while the stationary agents are permitted to communicate directly between them- selves through a static interconnection network, the mobile

agents are assumed not to require direct communication be- tween themselves for the following reasons. First, given that the number of mobile agents is large, facilities to provide direct communication between any two agents are likely to incur large overhead. This may also adversely impact scalability. Second, the underlying distributed algorithms are intelligently designed so that the stationary nodes perform the function of coordinat- ing information between the mobile agents when necessary. The target class of application problems includes railway networks [3, 19 ], community health-care networks [24], battlefield net- works [4], and intelligent vehicle highway systems.

Thus, the computer model of a mobile-computing network will consist of stationary and migrating processes executing on computing engines and mechanisms to facilitate stationary- stationary agent and mobile-stationary agent communications. Every process owns its own thread of control and is thus autono- mous and asynchronous relative to other processes in the system. The capabilities of the processes are defined by the nature of the system. The stationary processes acquire necessary information from other stationary processes and mobile processes, which is subsequently downloaded and utilized by appropriate mobile pro- cesses. While the static network interconnecting the stationary processes is permanent, the mobile processes connect and discon- nect dynamically and asynchronously, i.e., at irregular intervals of time, with appropriate stationary processes. A migration occurs when a mobile process, Mi, chooses to disassociate itself from the stationary process, Sj, and associate itself with the stationary pro- cess, s k , for all legitimate values o f j and k.

In a real system, every stationary and mobile agent is pro- vided with its own computing engine and facilities to initiate communication with other agents. It is therefore logical to as- sume that in a simulation of a mobile-computing network, every stationary and mobile process will have access to its own com- puting engine. However, many parallel-processing testbeds, in- cluding the one utilized in this article, are likely to have far fewer available processors than the total number of stationary and mo- bile agents. This results in two principal strategies for represent- ing mobile entities through processes in the testbed. They are

Processor-2 Migrating Lo Processor-1

Processor-1 .. .... .... . .- 1

1. Illustrating virtual process migration.

termed virtual process migration (VPM) and physical process migration (PPM) and are detailed subsequently.

Virtual Process Migration The obvious logical choice is to represent the relatively modest number of stationary agents as actual processes, assign them to the processors of the parallel-processing testbed on a one-on- one basis, and represent the mobile agents through virtual pro- cesses. Astationary node represents an agent located at a specific geographic position. Avirtual process migrates between proces- sors, when necessary, and its computational needs are executed by the host processor underlying the stationary node, where it may happen to be located at that instant of time. By definition, a virtual process is not permanently associated with any proces- sor. From time to time, it is associated with a processor, corre- sponding to a stationary node, that executes its computing needs and temporarily assigns it the status of an actual process. This strategy is termed VPM and has been utilized in [3] and [ 191. In VPM, the number of processors utilized equals the number of stationary entities. The processors are interconnected in the same topology as the stationary entities through software proto- cols that are initiated at initialization time and remain un- changed throughout the simulation.

Avirtual process in VPM is similar to a “thread” of an operal- ing system. However, unlike a “thread that contains the code, stack, stack pointer, and the program counter, a virtual process only contains the essential parameters required for its execu- tion. The exacl parameters are defined by the application pro-

gram. As an example, in the modeling and simulation of the intelligent vehicle highway system, the parameters for the mo- bile automobiles may include the vehicle license plate, model, manufacturer, current speed, desired speed, location, heading, origin, and destination. When a mobile entity is located at a sta- tionary node, it “appears” at the node, i.e., it is manifested as an actual process and its computing needs are executed by the host processor. Utilizing relevant information contained at the sta- tionary node and within itself, the mobile entity determines its subsequent course of action, which may include the decision to migrate to a different stationary node. Then, the simulation mi- grates the corresponding virtual process with all of its parame- ters to the appropriate stationary node where the mobile entity again “reappears.” Thus, the behavior of a mobile agent is self- contained and is neither visible to the stationary node nor to other virtual processes that may be temporarily co-resident at the same stationary node. Also, at any given time, one or more virtual processes may be resident at a stationary node and com- pete for the computation and communication resources. Thus, a scheduler may be utilized to assign slots of computing and com- munication facilities to the processes. Communication of infor- mation between the stationary process and a virtual process is achieved simply through buffer copying.

Figure 1 describes a simple mobile-computing network with three stationary entities, represented through stationary pro- cesses SP-1, SP-2, and SP-3. The simulation consists ofthree un- der lying processors-Processor-1, Processor-2, and Processor-3. At a given time instant, virtual processes W-1

Processor-3 r- Processor-2 1 Processor-1

SP: Stationary Process PP Physical (Mobile) Process

I

Processor-4 Processor-5 Processor-6 Processor-7 Processor-8 Processor-9 I

2. Illustrating physical process migration.

CIRCUITS 8 DEVICES IANUARY 1998

through VP-5 are resident on the three processors as shown in Fig. 1, while VP-6 is being migrated from Processor-2 to Processor-1. During migration, the actual process correspond- ing to VP-6 that is resident on Processor-2 is first terminated, its essential parameters are encapsulated in a message, the message is propagated to the Processor-1, the message is decoded in Processor-2, and finally an actual process is synthesized corre- sponding to VP-6. Every processor is responsible for executing the stationary process and one or more virtual processes, the scheduler, and the communication primitives.

A close examination yields two important characteristics of VPM. First, in general, for a mobile entity to migrate from sta- tionary node A to stationary node B, A and B must be connected directly. While this implies reduced complexity, it does not pre- clude the design of facilities to allow more complex migration. Second, the number of mobile entities at a stationary node at any given time instant is limited by the maximum number of pro- cesses permitted by the operating system.

Physical Process Migration Despite its utilization in [3] and [19], VPM incurs important limitations. As the number of mobile entities increases, the com-

petition for the host proces- sors' computing and commu- nication resources is likely to become acute, thereby slowing down the simulation signifi- cantly. To address this limita- tion, this article proposes a competing approach called PPM. In PPM, every pro- cess-stationary O Y mo- bile-is allocated a unique processor. The allocation is en- gaged at the instant the pro- cess is initiated into the system and is disengaged when the process terminates.

When a mobile process de- sires to communicate with a stationary process at runtime, first a communication protocol is dynamically established be- tween the underlying proces- sors and then information exchange is initiated. Thereaf- ter, when the mobile process desires to interact with a differ- ent stationary process, the old protocol is first disconnected and a new connection is estab- lished. A mobile process is al- lowed to maintain a connection with a single stationary process at any time. Thus, the PPM

strategy is a more accurate model of reality. The static intercon- nection network between the stationary processes remains identi- cal to that for the VPM. Clearly, the computational need of every mobile process is executed by its underlying processor, and where the computational needs of the mobile agents are high, there is the potential for higher efficiency and throughput relative to VPM. Unlike VPM, a mobile process may easily migrate from sta- tionary node A to stationary node Z in PPM where a direct connec- tion from A to 2 may be lacking.

PPM's principal advantage is in the use of one processor per process. Unfortunately, this also results in a weakness in the context of the limitations of today's testbed technology. Since testbeds with 1,000s of processors are not yet ubiquitous, simu- lation under PPM is limited to modest-sized mobile-computing

I

' I i

I . .... .... : I 4. Structure for every mobile entity in VPM.

i L j . : i1 ( l o a d 0) (

do mi.nimun(100,’ load) for-loop iterations; !,’ time slice is 100 /: decrement load;

I L,2 : if (messagcsqer-hop-remain C.) I

write tio a variable 1.oca~i.on to emulate a message; decrement messages-per-hop ... rcmain;

I L3 : if (:Load equals 0 AND messanes_p~er-}iop..remd.ir~ aciua1.s 0 ) {

determine stochastically where this cntity must micrate; migra’ie the process arid remove :.L f r o m t h e scheduler-’s l . i s t ;

> I

5. A mobile entity remanifested as a process, at a node.

networks. PPM also inherits the limitation of high overhead for mobile-stationary process communication, which includes ex- plicit message communication following the dynamic establish- ment of a communications protocol.

Figure 2 describes the use of PPM for the simple mobile- computing network shown earlier in Fig. 1. In addition to the three processors that model the stationary processes, six processors, Processor-4 through Processor-9, constitute the underlying proc- essors for the six physical processes, PP-1 through PP-6, corre- sponding to the six mobile entities. In Fig. 2, solid lines between stationary and mobile processes represent protocols that are estab- lished at agiven time instantwhile a broken line represents a proto- col that is in the state of either being established or disconnected. Thus, the broken lines between W-6 and each of Processor-2 and Processor-1 reflect the fact that physical process PP-6 is migrating from stationary node SP-2 to the stationary node SP-1.

Software Techniques Underlying the Process-Migration Strategies

The static network interconnecting the stationarv mo- - _ _ cesses in both VPM and PPM is established during ini- tialization of the simulation. As indicated earlier, every stationary process is assigned a distinct processor or workstation (a node). During execution, first a process opens a unique external input file, utilizing the identi- fier of the underlyingworkstation. This file contains the node’s operating characteristics that include its con- nectivity to other stationary processes. Second, the pro- cess starts to build the point-to-point connections, one at a time, utilizing the Berkeley socket protocols. When establishing a point-to-point connection between two processors, the initiator process executes a “connect” while the corresponding receiving process executes an “accept.” Every connection is half-duplex, implying di- rected edges in the network, and there may be multiple, overlapping cycles in the network. Furthermore, “con- nect” requires the receiving process identifier as an ar-

gument and it is nonblocking, while “accept” is blocking and is designed to receive any “connect,” i.e., from any processor. This threatens the network initializa- tion with the possibility of dead- lock and, t o c o u n t e r i t , t h e following algorithm is utilized.

The underlying nodes possess unique identifiers. When a station- ary process at a node (identifier X) requires connection with a sta- tionary process at a different node (identifier Y, Y > X), X always exe- cutes a “connect” while Y will exe- cute an “accept.” Upon completion of the network configuration in its memory, every node initiates exe-

cution of the stationary and mobile processes, which differs for the VPM and PPM strategies. The pseudo-code in Fig. 3 under- scores the function at each node. In Fig. 3, the code at label L1 first determines the set of stationary nodes in the system with identifiers higher than that of the current node. Then, the code starting at label L2 attempts to establish a connection between the current node and each of the nodes in the set through exe- cuting “connect,” one at a time, until all the connections are successfully established. The code starting at label L3 corre- sponds to the connection establishment between the current node and all other nodes in the system with identifiers lower than that of the current node.

Software Techniques Underlying VPM In VPM, every mobile entity is represented through a set of pa- rameters that is organized into a structure. The size of the struc- ture is a function of the application and may be dynamic. For a comparative study of the performance of WM and PPM, the fol- lowing fields are assumed for every mobile entity structure and

struct tlmevai timcout ; t Lmeout.. t v-sec‘ = 0

tLmeouc.tv.uscc = 0 f t3- set fdvar ; FII-ZCRO ( & f dvar) ; FlI-SET(socket, bfdvar) ;

T,,].: j.5 (selecL(socket-. f I, kfdvsr - , 0 , 0 , htj.m€OEtl) VPM-.en t i ty ne!,<-ent i t y ; read(socket, hnev-entity, sizeor (‘JPPI-enLit:yj ) ;

r n t i : m new pnt. i t y : 1 else {

1 recum 0;

1

6. A node intercepts a message encapsulating a mobile entity.

CIRCUITS 8 DEVICES a JANUARY 1998

shown in Fig. 4. The first field is the identifier of the entity, the sec- ond reflects its computational need, the third encapsulates the re- maining number of messages that this entity must exchange with the host stationary process, and the fourth field stores the remain- ing number of hops in this entity’s migration pattern.

The computational load of a mobile unit is represented by an integer, ranging from 100 to 10,000,000, and it constitutes the index of a simple “for loop”; that is, the number of iterations in the “for loop” equals the load value and the execution time of the iterative loop emulates the actual computational time. In VPM, at any given time instant, one or more mobile entities may be co-resident at a stationary node, competing for the single thread of control. To ensure that every mobile entity receives its fair share of the thread of control, the simulation proposes to use “time slicing,” wherein every virtual processvoluntarily gives up control after executing the loop for every 100 iterations.

Upon arrival at a node, a mobile entity is remanifested as an ac- tual process and is enqueued in the scheduler’s list. The scheduler allocates a time slot and executes the body of the mobile entity in the time slot. The pseudo-code in Fig. 5 represents the body of the mobile entity where its activity is emulated through executing it- erations. The iterations are executed in sets of 100. In Fig. 5, the statement at label L1 checks whether any iterations remain to be executed. If affirmative, a number of iterations equal to the mini- mum of 100 and the value of load is executed. If negative, all of the scheduled iterations have been completed. Next, the statement at label L2 checks whether any of the scheduled number of messages that need to be communicated are outstanding. If affirmative, a message communication is emulated by writing into avariable lo- cation. Otherwise, all scheduled messages have been communi-

cated. The statement at label L3 detects the scenar io t h a t a l l scheduled iterations and messages have been completed. Then, the subsequent migration of the mo- bile entity is deter- mined and it is propagated to t h e subsequent destina- tion node.

During migration, the parameters of the mobile entity are encapsulated in a message. The address and size of the message are passed to the operating system through the “write” system call, which then writes it to the appropriate outgoing socket and executes the transfer.

At the receiving end, the node polls for the arrival of the mobile entity using the “select” system call with 0 timeout. Select main- tains the ability to monitor multiple socket connections from within a single function call. When the message arrives, the node remanifests it as an actual process and enqueues it in the schedul- er’s list. The pseudo-code in Fig. 6 underscores this function of the node. The code statement at label L1 in Fig. 6 checks for a new mes- sage that encapsulates the arrival of a mobile unit. If affirmative, the message is read and the corresponding process is synthesized.

The scheduler implements round-robin scheduling of the sta- tionaty process and one or more mobile processes that may be co- resident in the host processor. When it is scheduled for execution, a mobile entity is first dequeued, then executed, and then either re- queued into the scheduler’s list or marked for migration to a differ- ent node. The functionality of the scheduler is shown in Fig. 7 in pseudo-code. In Fig. 7, the statement at label L1 reflects the sched- uler dequeueing the first element from the queue of mobile entities and then executes it. If the entity has exhausted its iterations and is marked for migration, as detected by the code at label L2, the sched- uler encapsulates it in the form of a message and propagates it to the output. Otherwise, following execution, the entity is requeued back into the list of mobile units at the node.

The main body of the program, executed by the node, integrates all of the above functions to describe the overall operation and is shown in Fig. 8. First, the data structures are initialized as reflected

I : ~ .... .. . .... . .....

8. The main program corresponding to each node.

hy t h p statpmpnt At Iahel L1. l’hen the mobile enti-

irig i n i o r i n a h i coli tail-led in the extcrnal d a h file. ;is 1-rprcsenlecl by l h L > COLIC. at liibel L2. Thc remainder of the program esecules h e lollo\ving t \ \ ~ ) opera~ioiis. repeatedly and ii: the proper squence: ii i check ii il nebv mobile node has

1 :wid dj sc‘onricct ( ! I

send dlsconnec-; conmanc t

cal; closc t o terminarc tnc c o n r e c t i m ? in: tq-conncct (4 ta t ;onary n

ca l l c o r r e c t ( ) t o ~ i l

i L (5uccessl {

send Id o x t n i s mobile c n t i t y ; -fezurn OK;

1 c l s c I 1e:Ern hKR9k.c:

migrated from an adjacent stationary node. If affirmative, remanifest the mobile entity as an actual process using the network message, and then insert the process into the scheduler’s queue for execution at a later instant of time; (ii) execute the scheduler, which allocates time slices to the enqueued mobile nodes. The statements at labels L3 and L4 reflect tasks (i) and (ii), respectively.

Software Techniques Underlying PPM Mobile Entities. At initialization, every mobile entity is associated with a processing node that is responsible for executing the itera-

Circuits & Devices Magazine Readeir Service Card

the stationary node. Second, the mo- bile node awaits an acknowledgment from the stationary node, following which both processes execute the “close” system call. Next, a new connection is established with the target stationary node. The pseudo-code in Fig. 9 describes the migration, discon- nection, and reconnection functions. The migrate function exe- cutes calls to the disconnect and try-connect functions.

The mainloop(), shown in Fig. 10, integrates all of the sub- functions described earlier to present the overall function of the mobile node in PPM. First, the network topology of the station- ary nodes is read from an external data file, as reflected by the

Please send me more information on advertisements and new products: I 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40

Name

Address

CitylStatelZip

Title

Tclephone

Signature Date IEEE Society

Please help our reader profile update. We need to know more ahout you. Check all that apply. I am responsible for: 0 design 0 specification 0 planning 0 approval 0 purchasc at my company.

Please send me more information about: U Regular IEEE membership in CAS EDS LEOS 0 IEEE student membership in CAS EDS LEOS 0 lEEE C&D subscriptions.

Please rote the following artides : Outstanding Please rate each article: (choose one) poor fair good excellent

Modeling Mobile Compute :ssage transfer, and migra- Networks s.Amessage transferwith

ProgressinOptitolinterconneas 0 0 0 0 0 stationary node requires : of anetworkprotocol. ent of migration, first the

U0 - D/A (onversion for Teletom 0

;ion between the mobile I a stationary process, if )e terminated. Normally,

Tell us what new departments and feature articles you would like to see in Circuits and ,Devices

7d of the connection, a :m call is executed. How- execution of the system

* two processors are not :d, the connection may be partially and a SIGPIPE

ierated when a process at- rite to it. To avoid this un- de effect, cooperation is both ends. In this article,

.- 1 . , . .

1.2 :

L3 :

: 11. The main program corresponding to each stationary node in PPM,

statement at label L1. This information is uGlized to determine migration-related decisions during the simulation. Second, the internal data structures are created, represented by label L2. Third, the following three operations are executed repeatedly: (i) the iterative loop is executed to simulate actual computational operations; (ii) it exchanges dummy data elements with the sta- tionary node to which it is connected at the current time instant; and (iii) it examines whether the required number of data ele- ments have been exchanged and the loop executed for the de- sired number of iterations. If affirmative, the mobile entity determines a new stationary node, stochastically, and initiates migration. The statements starting at labels L3, L4, and L5 rep- resent tasks (i), (ii), and (iii), respectively.

It is pointed out that for the VPM paradigm, the software pieces in Figs. 9 and 10 that are executed by every mobile entity process in the PPM paradigm are contained within and executed by the stationary node. Also, the disconnect and try-connect functions in Fig. 9 that underlie the mobile units in PPM have no counterpart in VPM.

Stationay Entities. The behavior of a stationary node includes three functions: (i) accept connection from the mobile entities,

(ii) exchange data with mobile entities, and (iii) accept termina- tion request from a mobile entity.

Given that a connection from a mobile node to a stationary node is initiated asynchronously and dynamically by the mo- bile entity, every stationary node must necessarily provide an entry point where the mobile node can initiate a connection. To realize the entry point, every stationary node binds a spe- cial socket and periodically listens to it through a select sys- tem call to determine whether a mobile node desires connection with it. The statement at label L1 in Fig. 11 real- izes this task. If affirmative, the stationary node executes an accept system call, reads and stores the identifier, and initi- ates the establishment of a connection with the mobile node. Upon connection, two kinds of messages are communi- cated-data elements and disconnection request. Data ele- ments are exchanged between the mobile and stationary nodes and, in this article, the data is dummy and simply dis- carded. When the mobile node intends to disconnect, it propa- gates a disconnection message to the stationary node. In turn, the stationary node will propagate an acknowledgment of the disconnection and execute the “close” system call to discon- nect. The statements a t labels L2 and L3 in Fig. 11 correspond to the receipt of disconnection and data element transfer.

It is pointed out that the program executed by each stationary node in PPM, as shown in Fig. 11, differs from that in Fig. 8 in that it neither emulates nor executes the activities correspond- ing to the mobile entities. Instead, it accepts connection and dis- connection requests from the mobile entities.

Implementation Issues The WM and PPM strategies are implemented on a testbed of 65+ SUN Sparc 10/40 workstations that are configured as a loosely cou- pled parallel processor. While each workstation is outfitted with 32 Mbytes of memory and executes on a Solaris 2.3 operating system, they are interconnected by a 10 MbiVsec ethernet. In addition, the code design permits execution under both SUN OS 4.1.3 and the freely available Linux operating systems [22]. The code is written in C++, is approximately 2,000 lines in length, and is compiled by a public-domain GNU g++ compiler. While the code executes in the background while the user executes programs on the consoles, the data presented here is obtained from simulations that are run late at night when network load is minimal.

Simulation Results and Performance Analysis For a comparative analysis of their performance, both VPM and PPM strategies are modeled and simulated on an parallel- processing testbed. The testbed closely resembles reality, and a number of experiments are designed and executed. Correspond- ing to an actual mobile-computing network where the key pa- rameters include the size of the static network ( i.e., the number of stationary nodes, the interconnection topology of the static network, the number of mobile entities, the computational load of the mobile agents, the number of messages exchanged be- tween the mobile and stationary entities at each hop, and the mi- gration pattern of the mobile entities), the simulation represents these parameters through independent variables. The key measure of performance is the maximum over-the-wall clock times required by all processors in the testbed.

In the experiments, the number of entities chosen reflect the fact that the testbed is limited to 65workstations. The number of stationary nodes range from 5 to 10, the static interconnection topology is assumed to be fully connected, and the number of mobile agents ranges from 5 to 50. The computational load of a mobile unit is represented by an integer, ranging from 100 to 10,000,000, and it constitutes the index of a simple “for loop,” as explained in Figs. 5 and 10. That is, the number of iterations in the “for loop” equals the load value and the execution time of the iterative loop emulates the actual computational time. The number of data elements exchanged between a mobile and sta- tionary agent is assumed to range from 1 to 100, where each data element is a 128-byte-long dummy. The choice of the 128-byte size reflects the message size used in the railway network simu- lation [3 , 191. The message communications are also referred to in Figs. 5 and 10. Every mobile agent’s migration pattern, re- flected in Figs. 5 and 10, is: (i) stochastic, i.e., randomly deter- mined; (ii) unique, i.e., independent of the migration patterns of all other mobile nodes; and (iii) asynchronous, i.e., the mobile entity may migrate at irregular intervals of time. The only con-

straint imposed on the mobile agents is that an agent will not im- mediately reconnect to the stationary node to which it was con- nected most recently. In the simulation, unless otherwise specified, every mobile agent connects and disconnects with the stationary nodes a total of 1,000 times.

Given that the processors of a parallel-processing testbed are asynchronous, their execution rates differ, and their clocks are out of phase, the order in which events are executed in the simulation may, in general, differ from that in actual operation. To preserve the order of event execution, different synchronization techniques are often utilized. The use of such techniques, however, constitutes an artifact of the simulation and has no correspondence in reality. These synchronization techniques slow down the simulation sig- nificantly and, as they affect both VPM and PPM similarly, the VPM and PPM implementations in this article are deliberately designed without them, without any loss in generality.

The results presented here reflect a total of over 200 simula- tion runs, each requiring an average 1,000 seconds of wall-clock time, 65 concurrently executing workstations, and several Mbytes of data collected from the simulation. When a mobile entity con- nects with a stationary entity, it performs computations, defined by the load value, and then exchanges data with the stationary process. The simulation terminates when every mobile entity has completed the specified number of connections and disconnec- tions. The measured simulation time includes the time required for establishing the software protocol connection, the computa- tion time, time for exchange of data, and disconnection time.

The graphs in Fig. l2(a) present the variation of the simula- tion time as a function of the computational load of the mobile entities, for both VPM and PPM. The number of stationary nodes is set to 10 and the number of mobile nodes ranges from 5 to 10 to 50. The number of data elements exchanged at each hop is set to 1 and each mobile entity engages itself in 1000 connections and disconnections. The graphs are revealing in that, while the VPM simulation times rise sharply with increasing load, the PPM simulation times remain relatively constant. For 5, 10, and 50 mobile entities, the PPM simulation times remain unchanged at 110 sec, 300 sec, and 710 sec, respectively. For low computa- tional load values, VPM exhibits superior performance due to the high overhead of connections and disconnections in PPM. For 5 mobile entities, beyond a computational load value of 20,000, PPM exhibits superior performance relative to VPM. Similarly, for 10 mobile entities, PPM’s performance exceeds that of VPM for a computational load value beyond 50,000. Furthermore, while the PPM simulation executes successfully for 50 mobile entities with load values ranging up to 100,000 and requiring 710 seconds, the VPM simulation requires extraordinarily large run times beyond load value of 10,000. The comparative behav- ior of PPM versus VPM is similar when the number of data ele- ments exchanged at each hop is set at 10 for 5 stationary nodes and 5 mobile entities, as shown in Fig. 12(b). Thus, for smaller number of data elements exchanged, modest number of mobile entities, and computational load under 100,000, the PPM strat- egy is scalable and exhibits superior performance.

CIRCUITS 8 DEVICES JANUARY 1998 19.

900.00

- c

0

S S

.g 800.00

y 700.00 0 Oll

L1 a, - o m - $ 600.00 g-5 m x -U

E 2 500.00 F o c -

m c 2 5

.E 2 - 3 2 400.00

c- 300.00 0 r a, S 0 200.00 0

100.00

PPM: 50 Mobile Entities

p p ~ : 5 Mobile Entities . ~ _ . .

PPM: 5 Mobile Entities . . . . . . . . . . . . . . . . . . . .

0.00 20.00 40.00 60.00 80.00 100.00

Computational Load (XI 03)

500.00

C c 0 450.00 a, c

H 400.00 0 Oll

m a , u m C K

0 Y

-w

o m 350.00

(2: @ m .E FJ 300.00

t--13 6 c-

E2

'c 0 m 'c - =I 25000

m $ 0 e- 200.00 0 c

a, S c 0 150.00 0

100.00

VPM ,I

PPM i\ .:

00 50,000 100,000

Computational Load

(b) ~

2. VPM and PPM simulation time as a function of computational load o f mobile entities: (a) 10 stationary nodes and 1 data element exchange6 at each hop, (b) 5stationary nodes and 10 data elements exchanged at each hop. The number ofstationary nodes = 10; the number of data ele-

ments exchanged at each hop = 1; and the mobile unit engages in I000 connections and disconnections.

900 00

c

U

C c 0

g 80000

p 70000

od m m uln

2 60000 U C

-W

n - 2 :

F 2 - P g

2' 50000 c

- m = 2 $ 40000

:" 30000 P

G ;

- a, c 0 20000 0

100 00

1 i o 20 30 40 50

Number of Mobile Entities

Figure 13 reorganizes the data presented in Fig. l2(a) and plots the simulation time as a function of the number of mobile nodes for different values of computational load ranging from 1,000 to 100,000. The aim is to reveal the impact of computational load on PPM and VPM performance while the number of mobile entities isvaried. It may be observed that while the VPM performance for smaller computa- tion load values of 1,000 and 10,000 exceeds the corresponding PPM graphs, the trend re- verses for a high load value of 100,000. The PPM graphs are virtually overlapping, imply- ing that the high computational load is equita- bly and efficiently shared by the greater number of processors in PPM.

Figures 14(a) and l4(b) plot the VPM and PPM simulation times as functions of the number of data elements exchanged at each hop. The number of mobile entities is set to 10. The computational load values are set at 10,000 and 100,000 in Figs. 14(a) and 14(b), respectively.

13. VPMand PPMsimulation time as a function of the number o f mobile entities for vary- ing computational load values. The number ofstationary nodes = 10; the number o f data elements exchanged at each hop = 1; and the mobile unit engages in 1000 connections

and disconnections.

The number ofstationary nodes ranges from 5 to lo. The VPM graphs remain uniform with an increasing number of data elements ex-

20

.. . .. ~ . . . .

120.00

100.00

VPM Number of Stationary Nodes = 5

i 400

. .-_ Y

8000 4.- 1 50 100 1 20 40 60 75

Number of Data Elements Exchanged at Each Hop Number of Data Elements Exchanged at Each Hop

(4 (b)

14. VPMand PPMsimulation time as a function of number o f data elements exchanged at each hop: (a) load = 10,000; (b) load = 100,000. The number o f mobile units = 10, and the mobile unit engages in 1000 connections connections and disconnections.

changed since buffer copying is extremely fast. However, the ob- servation that the VPM graph corresponding to 10 stationary nodes requires more simulation time than that for 5 stationary nodes appears to be counter-intuitive. One would have normally expected the 10 processors, corresponding to the 10 stationary nodes scenario, to finish executing the computational burden imposed by the mobile entities faster than the 5 processors cor- responding to the 10 stationary nodes scenario.

The reason for the observed behavior is that the function within every stationary node that translates the processor identi- fier to the socket descriptor is implemented through a linked list. Although a general approach, the linked list must be searched se- quentially for every execution of the function. Given that the sta- tionary nodes are fully connected, the number of sockets increase rapidly, thereby slowing down the simulation for 10 stationary nodes relative to 5 stationary nodes. The Unix profiler, gprof, re- veals that 40% of the simulation time is spent in the function for 10 stationary nodes as opposed to 23% of the simulation time for 5 stationary nodes. Since the binding between processor identifiers

and socket descriptors is static, it is planned that array data structure will be utilized to provide direct and fast access.

In PPM, however, exchange of data elements involves explicit messages and an increase in their number will require increas- ing simulation time, as evident from the linear slope of the PPM graphs. For 10 stationary nodes, the PPM simulation time con- tinues to trail the VPM simulation time up to 90 data elements exchanged per hop. Clearly, the overall computational load is executed faster by 10+10 = 20 processors in PPM relative to only 10 processors in VPM. However, when the number of data ele- ments exchanged increases beyond 90, the overhead from ex- plicit message passing in PPM surpasses the advantage of the greater number of computing elements and VPM supersedes PPM in performance. Where the number of stationary nodes is 5, the total number of processors at the disposal of PPM is 5+10 = 15, in contrast to 5 for VPM. Given the modest load of 10,000, the increased stress of connections and disconnections on 5 station- ary processors in PPM, coupled with the high overhead of ex- plicit message passing, causes PPM to exhibit inferior

CIRCUITS 8 DEVICES m JANUARY 1998 21

performance relative to VPM. Thus, PPM loses its performance edge beyond 10 data elements exchanged at each hop.

When the computational load value is high, namely 100,000, the greater number of processors associated with PPM relative to VPM is likely to yield a superior performance for PPM. The graphs in Fig. 14(b) confirm this expectation even when the number of data elements exchanged at each hop is increased from 1 to 75. Unlike Fig. 14(a) where the slopes of the PPM graphs are greater than those for the VPM graphs, in Fig. 14(b), the slopes of VPM and PPM are comparable. This is due to the large computational load that lessens the influence of the mes- sage communication overhead in PPM. The PPM simulation with 10 stationary nodes employs 10+10 = 20 processors while the PPM simulation with 5 stationary nodes employs 5+10 = 15 processors, and, as expected, the performance of the former ex- ceeds that of the latter.

Conclusion Both VPM and PPM play useful and effective roles in the modeling and simulation of real-world, mobile-computing networks. While PPM is a more accurate model of reality, by its very nature, VPM helps realize the modeling of networks with large numbers of mo- bile entities on testbeds with a modest number of processors, al- though at greatly reduced performance. Utamaphethai and Ghosh [23] reported an IVHS simulation with 45,000 entities represent- ing autonomous vehicles on a network of 60+ SUN Sparc 10 work- stations. In contrast, a PPM implementation would be difficult to realize due to the large number of processors required and it would be unacceptably slow due to the large numbers of connec- tions and disconnections. PPM is unquestionably superior when the computational loads associated with the mobile entities are high. However, its performance suffers relative to VPM where the system demands intense data exchange between the mobile and stationary units. This study also underscores that, for scalability in performance, the algorithms underlying real-world mobile- computing networks must be designed to reduce communication while increasing local computation.

Kwun Han graduated from Brown University in 1996 and is pres- ently a graduate student in the Computer Science Department at Carnegie Mellon University. Sumit Ghosh currently serves as an associate professor and associate chairman of research and graduate programs in the Computer Science and Engineering D e p a r t m e n t at Arizona State University.

Reference 1. Britannica Software, Compton Multimedia Encyclopedia on CD-ROM,

1993

2. D.C. Coll, A.U. Sheikh, R.G. Ayers, and J.H. Bailey, “The communications system architecture of the North American Advanced Train Control Sys- tem,” IEEE Transactions on Vehicular Technology, Vol. 39, No. 3, pp. 244- 255, August 1990.

3. R. Iyer and S. Ghosh, “DARYN, Adistributed decision-making algorithm for railway networks: modeling and simulation,” Proceedings of the IEEE In- ternational Conference on Systems, Man, and Cybernetics, pp. 269-274, University of Virginia in Charlottesville, VA 22903, October 13-16, 1991.

4. T. Lee and S. Ghosh, “A novel approach to asynchronous, decentralized decision-making in military command and control,” IEEE Computational Science QndEngineering, Vol. 3, No. 4, pp. 69-79, Winter 1996.

5. G.H. Forman and J. Zahorjan, “The challenges of mobile computing,”IEEE Computer, Vol. 27, No. 4, pp. 38-47, April 1994.

6. M. Satyanarayanan, “Mobile computing,” COMPUTER, Vol26, Iss 9, pp 81- 82, 1993.

7. H. Koch, H. Krombholz, and 0. Theel, “A brief introduction to the world of mobile computing, technical report THD-BS-1993-03, Computer Science Department, University of Darmstadt, Germany, May 1993.

8. T. Imielinski and B.R. Badrinath, “Mobile wireless computing: solutions and challenges in data management,” Technical Report DCS-TR-296, De- partment of Computer Science, Rutgers University, New Jersey.

9. M. Satyanarayanan, J.J. Kistler, L.B. Mummert, M.R. Ebling, P. Kumar, and Q. Li, “Experience with disconnect operation in a mobile computing envi- ronment,” Proceedings of the 1993 USENX Symposium on Mobile and Location-Independent Computing, Cambridge, MA 1993.

10. Y. Artsy and R. Finkel, “Designing a process migration facility-The Char- lotte experience,”COMPUTER,Vol22, Iss 9, pp47-56 October 13-16,1991.

11. D.W. Glazer and C. Tropper, “On process migration and load balancing in time warp, ”IEEE TrQnSQCfiOnS On Parallel QndDistributedSystems, Vol4,

12. T.T.Y. Suen and J.S.K. Wong, “Efficient task migration algorithm for dis- tributed systems, ” IEEE TrQnSQCfionS on Parallel and Distributed Sys- tems, Vol3, Iss 4, pp 488-499, 1992.

13. M.S. Chen and K.G. Shin, “Subcube allocation and task Migration in Hy- percube Multiprocessors,” IEEE Transactions On Computers, 1990, Vol39, Iss 9, pp 1146-1155.

14. V.M. Milutinovic, C.E. Houstis, and J.J. Crnkovic, “A simulation study of 2 distributed task allocation procedures,” IEEE TrQnSQCtiOnS On Software Engineering, Vol 14, Iss 1, pp 54-61, 1988.

15. J. Gait, “Scheduling and process migration in partitioned multiproces- sors, ”Joumal OfParallel and Distributed Computing, Vol8, Iss 3, pp 274- 279,1990.

16. W-M. Lin and B. Yang, “Load balancing technique for parallel search with statistical model,” Proceedings of the Fourteenth Annual IEEE Intema- tional Phoenix Conference on Computers and Communications, March 28-31, Phoenix, Arizona, 1995.

17. A. Hac, “A distributed algorithm for performance improvement through file replication, file migration, and process migration,” IEEE TrQnSQCfiOnS on SoftwareEngineering, Vol 15, Iss 11, pp 1459-1470, 1989.

18. T. Imielinski and B.R. Badrinath, “Data Management for Mobile Comput- ing,” SIGMOD Record, Vol. 22, No. 1, pp. 34-39, March 1993.

19. R.V. Iyer and S. Ghosh, “DARYN, a distributed decision-making algorithm for railway networks: modeling and simulation,” IEEE Transactions on Ve- hicular Technology, Vol. 44, No. l, February 1995.

20. D. Duchamp, S.K. Feiner, and G.Q. Maguire, “Software technology for wireless mobile computing,” IEEE Network, Vol. 5, No. 6, pp. 12-18, No- vember 1991.

ISS 3, pp 318-327, 1993.

21. D. Bertsekas and R. Gallager, Datu Networks, Prentice Hall, New Jersey,

22. Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139,

23. N. Utamaphethai and S. Ghosh, “DICAF, a high performance, distributed, scalable architecture for IVHS utilizing a continuous function-“Conges- tion Measure,” IEEE Computer (to appear March 1998).

24. S. Ghosh, K. Han, R. Reddy, S. Reddy, S. Kankanahalli, J. Jagannathan, and B. Shank, “A distributed, scalable, community care network architec- ture for real-time access to geographically-dispersed patient medical rec- ords: Modeling and simulation,” Proceedings o f the 19Ih Annual Symposium on Computer Care Applications in Medical Care (SCAMC ’9.9, New Orleans, American Medical Informatics Association, DC, pp. 352-356, Oct 28 - Nov 1, 1995.

1992.

USA.

CD