View
213
Download
0
Category
Preview:
Citation preview
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Re-thinking Data Management for Storage-Centric Sensor Networks
Deepak GanesanUniversity of Massachusetts Amherst
With: Yanlei Diao, Gaurav Mathur, Prashant Shenoy
2UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Sensor Network Data ManagementLive Data Management: Queries on current or recent data.
Applications: Real-time feeds/queries: Weather, Fire, VolcanoDetection and Notification: Intruder, Vehicle
Techniques:Push-down Filters/Triggers: TinyDB, Cougar, Diffusion, …Acquisitional Query Processing: BBQ, PRESTO, …
Archival Data Management: Querying or Mining of past data
Applications:Scientific Analysis of past events: Weather, Seismic, …Historical trends: Traffic analysis, habitat monitoringOur focus is on designing an efficient archival data management architecture for
sensor networks
3UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Archival Querying in Sensor Networks
Data Gathering with centralized archival query processing
Efficient for low rate, small volume sensors such as weather sensors (temp, humidity, …).
Inefficient energy-wise for “rich” sensor data (acoustic, video, high-rate vibration).
Lossless aggregation
DBMS
Internet
Gateway
4UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Archival Querying in Sensor Networks
Acoustic stream
Store data locally at sensors and push queries into the sensor network
Flash memory energy-efficiency, cost, capacity.
Limited capabilities of sensor platforms.
Internet
Gateway
Image stream
Flash Memory
Push query to sensors
5UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Technology Trends in Storage
Generation of Sensor Platform
CC1000
CC2420
Telos STM NOR
Atmel NOR
Communication
Storage
Micron NAND 128MB
Energy Cost
(uJ/byte)
6UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Outline
Case for Storage-centric Sensor Networks
Challenges in a Storage-centric Sensor
Database
StonesDB Architecture
Local Database Architecture
Distributed Database Architecture
Conclusion
7UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Optimize for Flash and RAM Constraints
Flash Memory ConstraintsData cannot be over-written, only erasedPages can often only be erased in blocks (16-64KB)Unlike magnetic disks, cannot modify in-place
Challenges:Memory: Minimize use of memory for flash database.Energy: Organize data on flash to minimize read/write/erase operationsAging: Need to efficiently delete old data items when storage is insufficient.
1. 1. Load block 2. Into Memory
3. Save block back
Eraseblock
Memory
2. Modify in-memory
~16-64 KB
~4-10 KB
8UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
SQL-style Queries: Min, max, count, average, median, top-k,
contour, track, etc
Similarity Search: Was a bird matching signature S observed last
week?
Classification Queries: What type of vehicles (truck, car, tank, …) were observed in the
field in the last month?
Wireless Sensor Network
Support Rich Archival Querying Capability
Signal Processing: Perform an FFT to find the mode of vibration signal between time
<t1,t2>?
9UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
StonesDB Goals
Our goal is to design a distributed sensor database for archival data management that:
Supports energy-efficient sensor data storage, indexing, and aging by optimizing for flash memories.
Supports energy-efficient processing of SQL-type queries, as well as data mining and search queries.
Is configurable to heterogeneous sensor platforms with different memory and processing constraints.
10UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
StonesDB Architecture
11UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Example: Indexing in StonesDB
Naïve Design:Consider a value-based index on entire streamDeletion/Aging of data triggers in-place updates involving energy-intensive block read/write/erase operations.
12UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Indexed Storage
StonesDB Design:Split data stream into partitions and build index on each partition. Age partitions as a whole cheaply.
Flash
Block
13UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Outline
Case for Storage-centric Sensor Networks
Challenges in a Storage-centric Sensor
Database
StonesDB Architecture
Local Database Architecture
Distributed Database Architecture
Conclusion
14UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
StonesDB: Data Mining Queries
Similarity Search: Was a bird matching signature S observed
last week?
Proxy Cache of Image Summaries
15UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
StonesDB: System Operation
Similarity Search: Was a bird matching signature S observed
last week?
Query Engine
Partitioned Access Methods
16UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Research Issues
Local Database LayerImpact of RAM limitations on storage organizationEnergy-optimized indexing and aging. New cost models for self-tuning energy-efficient sensor databases.
Distributed Database LayerIntelligent split of query processing between proxy and sensor tiersAdaptively tuning quality of data cached at sensor proxy based on query needs
UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
The End
STONES: STOrage-centric Networked Embedded Systems
http://sensors.cs.umass.edu/projects/stones
18UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Sensor Data Management Taxonomy
Timeline vs Prior KnowledgeQ
uery
ing
Min
ing
Current Recent Past
Acquisitional Query Processing
(BBQ, …)
Pushdown Filters(TinyDB, Cougar, …)
Timeline of data being processed
Search/Mining on Archived Sensor Data
Type of data processing
19UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer ScienceUUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science
Technology Trends in Sensor Platforms
Cyclops Camera+ Mica2 Mote128 x 128 resolution images4 KB RAM, 10 MHz microcontroller
OmniVision Camera + iMote2128 x 128 resolution images64KB - 32MB RAM, 10 MHz microcontroller
Spectrum of sensing devices with different power, capability, resource constraints.
Recommended