View
215
Download
0
Category
Tags:
Preview:
Citation preview
3/14/2005 CS851: Data Services for Advanced System Applications
1
DFuse: A Framework for
Distributed Data Fusion
Rajnish Kumar, Matthew Wolenetz, Bikash Agarwalla, JunSuk Shin, Phillip Hutto,
Arnab Paul, and Umakishore RamachandranGeorgia Institute of Technology
Presented by Lei Fang, Mar 14, 2005
3/14/2005 2CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 3CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 4CS851: Data Services for Advanced System Applications
Future Wireless Sensor Networks
More powerful hardware Resource-demand applications
• Streaming media• Surveillance• Image-based tracking• interactive vision
Hierarchal data fusion
3/14/2005 5CS851: Data Services for Advanced System Applications
Future Wireless Sensor Networks
Hierarchal data fusion
3/14/2005 6CS851: Data Services for Advanced System Applications
Data Fusion Challenges
Why is developing fusion applications challenging?
Time sensitive• Synchronization of the data from multiple streams
Distributed computing• Distributed fusion
Limited power• In-network aggregation• Power-aware routing• Dynamic relocation of fusion functions
3/14/2005 7CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 8CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 9CS851: Data Services for Advanced System Applications
DFuseDFuse
Fusion API: Help to develop Fusion API: Help to develop distributed sensor fusion distributed sensor fusion applicationsapplications
Fusion Placement: Support Fusion Placement: Support automatic management of fusion automatic management of fusion point placement and migrationpoint placement and migration
3/14/2005 10CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 11CS851: Data Services for Advanced System Applications
DFuse Architecture - Input
Target applicationsTarget applications HierarchicalHierarchical fusion functions on fusion functions on time time
sequencedsequenced data items data items Input Input
Task GraphTask Graph• Vertex: Vertex: source, sink, or immediate fusion points• Assume data sources are known at query time
Fusion Function• Transform a correlated set of inputs to a “fused” output item• May contract or expand the streaming data• Each fusion point performs one fusion function
3/14/2005 12CS851: Data Services for Advanced System Applications
DFuse Architecture - Components
Data Fusion Module Data Fusion Module Implement fusion API: Implement fusion API: allows any
synthesis operation to on stream of data to be specified as a fusion function
Fusion Placement Module Fusion Placement Module Determine the best placement of fusion
functions
3/14/2005 14CS851: Data Services for Advanced System Applications
DFuse Architecture – Working principles
Fusion Module provides fusion API which Fusion Module provides fusion API which allows any synthesis operation to on stream of data to be specified as a fusion function. The user inputs the task graph and fusion functions
to DFuse at some designated node (root node)
The placement module at the root node maps the fusion functions onto the physical network Subject to some cost function Periodically re-evaluate the health of fusion point
nodes and their neighbors, and moves the fusion functions
3/14/2005 15CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 16CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 17CS851: Data Services for Advanced System Applications
DFuse – Fusion Module
Fusion API CapabilitiesFusion API Capabilities Structure ManagementStructure Management
• Fusion Channel – Abstracts inputs and Fusion Channel – Abstracts inputs and encapsulates a general fusion function encapsulates a general fusion function (filter, collage, …) (filter, collage, …)
• provide data buffering and synchronization provide data buffering and synchronization facilities facilities
• E.g. items are fused and accessed by E.g. items are fused and accessed by timestamptimestamp
3/14/2005 18CS851: Data Services for Advanced System Applications
DFuse – Fusion Module
Fusion APIFusion API Correlation controlCorrelation control
• Specify and collect correlation sets (related Specify and collect correlation sets (related input items supplied to the fusion function)input items supplied to the fusion function)
Computation managementComputation management• Handle the specification, application and Handle the specification, application and
migration of fusion functionsmigration of fusion functions Memory managementMemory management
• Caching, prefetching, and buffer Caching, prefetching, and buffer managementmanagement
3/14/2005 19CS851: Data Services for Advanced System Applications
DFuse – Fusion Module
Fusion APIFusion API Failure/latency handlingFailure/latency handling
• Allow partial fusionAllow partial fusion• Accept a variable number of input itemsAccept a variable number of input items
Status and feedback handlingStatus and feedback handling• Allow interaction between fusion functions Allow interaction between fusion functions
and data sourcesand data sources• Supply Supply status informationstatus information via “status” register via “status” register• Support Support a command seta command set via “command” via “command”
registerregister
3/14/2005 20CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 21CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 22CS851: Data Services for Advanced System Applications
Placement Module – Role Assignment
Distributed role assignment algorithmsDistributed role assignment algorithms Map a fusion function to a network nodeMap a fusion function to a network node Three roles: end point, relay, fusion pointThree roles: end point, relay, fusion point
3/14/2005 23CS851: Data Services for Advanced System Applications
Placement Module – Role Assignment
Role Assignment HeuristicRole Assignment Heuristic Phase 1: Naive role assignment
• Start at root node• Algorithm: Naive Tree Building
Upon receiving the application task graph:if (I can host the root fusion function) { accept the root fusion function; for each input {
delegate subtree to “richest” neighbors; }else delegate entire tree to the “richest” neighbor;
3/14/2005 24CS851: Data Services for Advanced System Applications
Placement Module – Role Assignment
Role Assignment HeuristicRole Assignment Heuristic Phase 2: Optimization (Role Transfer)
Improve overall health of the overlay network
• Health is valued by cost function (power, data transfer rate)• Each fusion node periodically broadcasts its health• If a neighboring node has better health, it informs the fusion node• The fusion node transfer fusion task to the neighbor with best
health
Phase 3: Maintenance (application is running)
3/14/2005 25CS851: Data Services for Advanced System Applications
Placement Module – Sample Cost Functions
The lower the cost function value, the better the node health!
Minimize transmission cost 1 (MT1):•
Minimize power variance (MPV):• CMPV( k ) = 1/power ( k )
Minimize Transmission cost vs. power (MTP):• CMTP ( k, f ) = CMT1 ( k, f ) * CMPV ( k )
Minimize transmission cost 2 (MT2):• CMT2( k, f ) = ( power ( k ) > threshold ? (CMT1 ( k, f ) :
Infinity )
How to transfer the role ?
3/14/2005 26CS851: Data Services for Advanced System Applications
Placement Module – Local Minima
Two simple metrics Simple hop-count Fusion data expansion or contraction information
Two types of optimization Linear: All inputs to a fusion node are coming from one
relay node Triangular: Multiple paths for inputs to reach a fusion
point
3/14/2005 27CS851: Data Services for Advanced System Applications
Placement Module –Optimization
Linear Optimization
1000
1000
1000 1000
1500
SourceRelay Fusion Point(data contraction)
Sink
1000
1000
1500 1500
SourceFusion Point
Relay Sink
1000
1000
1500 1500
SourceFusion Point(data expansion)
Relay Sink
1000
1000
1000 1000
1500
SourceRelay Fusion Point Sink
3/14/2005 28CS851: Data Services for Advanced System Applications
Placement Module –Optimization
Triangular Optimization
1000
1000
1000
1500
fp1
Relay
fp3(data contraction)
fp4
fp2
1000
1000
1000
1500
fp1
fp3
fp2
3/14/2005 29CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 30CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 31CS851: Data Services for Advanced System Applications
DFuse – Implementation
Built on top of Stampede Stampede provides distributed threads, timestamps,
registers with synchronization, automatic garbage collection, communication channels, etc.
All buffers implemented as Stampede channels Map the fusion attribute to the timestamp of channel Status and command registers implemented with
Stampede registers
Decouple the fusion and placement module Interface: Built in communication channel and protocol
3/14/2005 32CS851: Data Services for Advanced System Applications
D-Stampede
Distributed programming system for Stream-based ubiquitous computing
Key Features Index and correlate different data streams temporally. Perform automatic distributed garbage collection of
unnecessary stream data. Provide parallelism Support distributed heterogeneous components. Dynamic application
3/14/2005 33CS851: Data Services for Advanced System Applications
D-Stampede Hardware model
Skiff
Skiff
camera
camera
Data Aggregators
Sensors
Actuators Unix / Linux / NT cluster
Channels / queuesChannels / queues
SensorFusion
3/14/2005 34CS851: Data Services for Advanced System Applications
D-Stampede Computational model-Thread/Channel GraphComputational model-Thread/Channel Graph
thread
Channel
thread
thread
threadthread
Channel
Channel
Channel
i_conn
o_conn
•many to many connections
•time sequenced data
•correlation of streams
•put(ts, item)
•get(ts, item)
•consume(ts)
3/14/2005 35CS851: Data Services for Advanced System Applications
D-Stampede ImplementationImplementation
3/14/2005 36CS851: Data Services for Advanced System Applications
D-Stampede Programming with D-StampedeProgramming with D-Stampede
3/14/2005 37CS851: Data Services for Advanced System Applications
Implementation – Fusion Module
Built on top of Stampede Functions:
createFC/destroyFC attachFC/detachFC: connect/disconnect
application thread with specified fusion channel
getFCItem/putFCItem: request/put fused data items
3/14/2005 38CS851: Data Services for Advanced System Applications
Implementation – Placement Module
Event-based simulation Input: Application task graph, network
topology Assumes ideal MAC and perfect routing
knowledge Simulates the fusion placement heuristic
• Naïve tree building• Optimization• Maintenance
3/14/2005 39CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 40CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 43CS851: Data Services for Advanced System Applications
Evaluation – Placement Algorithm Implement the tracker application with fusion Implement the tracker application with fusion
API and deploy it on the iPAQ farmAPI and deploy it on the iPAQ farm The location of filter and collage determined by
placement module movePC API: role transfer
Data Sources
3/14/2005 44CS851: Data Services for Advanced System Applications
Evaluation – Simulation Placement module adjusts power
level based on number of the amount of data sent and Based on ORiNOCO 802.11b PC card specs Only consider network communication costs
Optimization phase: runs for 2s Re-evaluate health every 100ms
During maintenance phase, role transfer evaluated every 50s
3/14/2005 48CS851: Data Services for Advanced System Applications
Evaluation – Runtime/Energy/Role Transfers
3/14/2005 49CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 50CS851: Data Services for Advanced System Applications
OutlinesOutlines
MotivationMotivation DFuse DFuse
ArchitectureArchitecture Fusion ModuleFusion Module Placement ModulePlacement Module ImplementationImplementation EvaluationEvaluation
ConclusionConclusion
3/14/2005 51CS851: Data Services for Advanced System Applications
ConclusionsConclusions
Future sensor networks will support data fusion applications
DFuse provides fusion API to ease the development such applications
Distributed fusion placement increases application runtime by dynamic role assignment
3/14/2005 52CS851: Data Services for Advanced System Applications
CritiquesCritiques Inflexible:
• DFuse can’t be implemented in current sensor network• Assume data sources are known before• Stationary sources and sinks• Complicated, resource demand• Assume every two node in the network is reachable• Migrate a particular fusion channel is a local decision
and only immediate neighbor nodes are queried to see if their heath for hosting the fusion channel would be better.
Insufficient evaluation:• Only consider network communication cost when Only consider network communication cost when
evaluate power consumptionevaluate power consumption• Implemented on iPAQ farm, but most of the work via
event-based simulation on desktop
3/14/2005 53CS851: Data Services for Advanced System Applications
Related workRelated work--- Data Fusion --- Data Fusion
InterfaceInterface TAG (Tiny Aggregation): TAG (Tiny Aggregation): PProvides a simple SQL-like declarative language for expressing
aggregation queries over streaming sensor data. Intelligently distributes and executes aggregation queries in the sensor
network in a time and power-efficient manner. Example:
Difference between Tag query and SQL query Output of a Tag query is a stream of values Record : <group id, aggregate value>. Each group is time-stamped and the readings
used to compute an aggregate record all belong to the same time interval, or epoch.
SELECT AVG(volume),room FROM sensorsWHERE floor = 6GROUP BY roomHAVING AVG(volume) > thresholdEPOCH DURATION 30s
SELECT (agg(expr), attrs) FROM sensorsWHERE {selPreds}GROUP BY {attrs}HAVING {havingPreds}EPOCH DURATION i
3/14/2005 54CS851: Data Services for Advanced System Applications
Related workRelated work--- Data Fusion --- Data Fusion
InterfaceInterface TAG (Tiny Aggregation): TAG (Tiny Aggregation): Structure of Data aggregationStructure of Data aggregation
• Three pieces:– Merging function f– Initializer i– Evaluator e
• General form: <z> = f(<x>,<y>)– <x>, <y>, <z> are multi-valued partial state record– Example:
f- for Average Each partial state record consists of <SUM, COUNT>
f(<S1, C1>,<S2,C2>) = <S1+S2, C1+C2>i- How to initiate a state record for a single sensor value
Eg. i(x) = <x,1> :Average over a sensor value of xe- Compute the actual value of aggregation e(<S,C>)
Pros:Pros:• Proven to give more accurate results then centralized approach• In-network aggregation: Save powerIn-network aggregation: Save power• SimpleSimple
Cons:Cons:• Static mapping fusions rolesStatic mapping fusions roles
3/14/2005 55CS851: Data Services for Advanced System Applications
Related workRelated work--- Dynamic Role --- Dynamic Role
AssignmentAssignment Generic Role Assignment for WSN
ArchitectureArchitecture
3/14/2005 56CS851: Data Services for Advanced System Applications
Related workRelated work--- Dynamic Role --- Dynamic Role
AssignmentAssignment Generic Role Assignment for WSN
Example: In-network aggregation Example: In-network aggregation
Rules/AlgorithmRules/Algorithm
Recommended