University of Minnesota

FAST: A Framework for Flash-Aware Spatial Trees

Mohamed Sarwat, Mohamed Mokbel, Xun ZhouDepartment of Computer Science and Engineering

University of Minnesota

Suman NathMicrosoft Research

2

Flash Memory: Already exists

• Mobile Devices

• Enterprise Servers

3

Flash Memory Storage: Pros

Flash Memory Hard Disk

Electronic Device Mechanical Device

Fast Read Performance Slow Read Performance

High Shock Resistance Low Shock Resistance

Low Power Consumption High Power Consumption

Small Size and Light Weight Relatively Large Size and Heavy Weight

Flash Memory is Topnotch !

4

Flash Memory Storage: Cons

Erase-Before-Update Property Erase latency 1500 microsecondsRead latency 25 microsecondsWrite latency 200 microseconds

Limited Number of Erases per Block100,000 to 1000,000

5

DBMS Tree Indexes & Storage

All Database indexing algorithms were built with the implicit assumption that the underlying secondary storage is the hard disk.

Making Tree Index Structures Flash-Aware !

6

Existing Flash-Aware Trees

■ Chin-Hsien Wu, Li-Pin Chang, Tei-Wei Kuo. An Efficient R-Tree Implementation over Flash-Memory Storage Systems. In GIS 2003 Designed only for R-tree. No support for recovery

■ Chin-Hsien Wu, Li-Pin Chang, and Tei-Wei Kuo. An Efficient B-Tree Layer for Flash-Memory Storage Systems. In TECS 2007 Designed only for B-tree. No support for recovery

7

Existing Flash-Aware Trees

■ Devesh Agrawal, Deepak Ganesan, Ramesh Sitaraman, Yanlei Diao, and Shashi Singh. Lazy-Adaptive Tree: An Optimized Index Structure for Flash Devices. In VLDB 2009 Need to build a new kind of tree indexing. Not very practical..!!

■ Yinan Li, Bingsheng He, Robin Jun Yang, Qiong Luo, Ke Yi. Tree Indexing on Flash Disks. In ICDE 2009 (short), VLDB 2010(full) Need to build a new kind of tree indexing. Not very practical..!!

8

We need a Framework

1) Generic: One Framework works for all

2) Practical: No need to throw your already existing index structure away

3) Durable: Designed with database systems in mind (recovery)

9

FAST is an Efficient, Generic , Practical, and

Practical Framework for index structures on the flash

storage

FAST

10

FAST: A Framework for Flash-Aware Search Trees

Recent Version of the tree

Flash Storage

Flash Memory Index

In-memory Buffer

Update Module

Search Module

Crash Recovery Module

Flushing Module

Search Query

Answer

Update Transaction

Log File

Restart after Crash

Full MemoryRead

Read

In-memory Update

Update Log Entries

Flushing Policy

Read

Selected Memory

Pages

Write

Flush Log

Entry

Read Index & Log Entries

Write

11

FAST: A Framework for Flash-Aware Search Trees

Recent Version of the tree

Flash Storage

Flash Memory Index

In-memory Buffer

Update Module

Search Module

Crash Recovery Module

Flushing Module

Search Query

Answer

Update Transaction

Log File

Restart after Crash

Full MemoryRead

Read

In-memory Update

Update Log Entries

Flushing Policy

Read

Selected Memory

Pages

Write

Flush Log

Entry

Read Index & Log Entries

Write

12

Update Operation (Insertion/Deletion)

Main Memory Flash Memory

Operation Log

…….

Tree Modifications Table

Tree Index Structure

Insert (O1, MBR ….) || Nodes 1,2 Step 1

Step 2Node

Node 1

Node 2

MBR1 - value

Obj1 - Inserted

Example: Insert (Obj1) in R-tree

Step 3

13

FAST: A Framework for Flash-Aware Search Trees

Recent Version of the tree

Flash Storage

Flash Memory Index

In-memory Buffer

Update Module

Search Module

Crash Recovery Module

Flushing Module

Search Query

Answer

Update Transaction

Log File

Restart after Crash

Full MemoryRead

Read

In-memory Update

Update Log Entries

Flushing Policy

Read

Selected Memory

Pages

Write

Flush Log

Entry

Read Index & Log Entries

Write

14

Search Operation (Find/Range)

Main Memory Flash Memory

Operation Log

…….

Tree Modifications Table

Tree Index Structure

Insert (O1, MBR ….) || Nodes 1,2

Step1

Step 2

Node

Node 1

Node 2

MBR1 - value

Obj1 - Inserted

Example: Range query in R-tree

Step 4

Query Result

Step 5

Step 3

15

FAST: A Framework for Flash-Aware Search Trees

Recent Version of the tree

Flash Storage

Flash Memory Index

In-memory Buffer

Update Module

Search Module

Crash Recovery Module

Flushing Module

Search Query

Answer

Update Transaction

Log File

Restart after Crash

Full MemoryRead

Read

In-memory Update

Update Log Entries

Flushing Policy

Read

Selected Memory

Pages

Write

Flush Log

Entry

Read Index & Log Entries

Write

16

Flushing

Main Memory Flash Memory

FAST Log File

Tree Index Structure

Insert (O1, MBR) || Nodes 1,2

MBR1 - value

Node 1 2 3 4 5

Dirty 00 1

Flush 2,3,4,5

75 %

Nodes 2,3,4,5 flushed

Same Block

1

2

9

12

14

4

17

18

5

Node 1

2 3 4 5

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

1 10 0 0

Delete (O2, MBR) || Nodes 12

Insert (O4, MBR) || Nodes 1,4,14

Insert (O5, MBR) || Nodes 4,17

Insert (O4, MBR) || Nodes 1,5,18

Step 1

Step 2

Step 4

Step 3

Step 5

17

FAST: A Framework for Flash-Aware Search Trees

Recent Version of the tree

Flash Storage

Flash Memory Index

In-memory Buffer

Update Module

Search Module

Crash Recovery Module

Flushing Module

Search Query

Answer

Update Transaction

Log File

Restart after Crash

Full MemoryRead

Read

In-memory Update

Update Log Entries

Flushing Policy

Read

Selected Memory

Pages

Write

Flush Log

Entry

Read Index & Log Entries

Write

18

Crash Recovery

Main Memory Flash Memory

FAST Log File

.....

Tree Index Structure

Insert (O1, MBR) || Nodes 1,2

Nodes 1,2,3,4,5, 14,17 flushed

1

2 3 4 5

6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Delete (O2, MBR) || Nodes 12

Insert (O4, MBR) || Nodes 1,4,14

Insert (O5, MBR) || Nodes 4,17

Insert (O4, MBR) || Nodes 1,5,18

Log Maintenance

Compact Log

Insert (O4, MBR) || Nodes 18

Crash Recovery

Restart after crash

Restart after crash

MBR1 - value

1

2

9

12

14

4

17

18

5

Node

19

Experimental Evaluation (1/4)

■ We implemented FAST in PostGreSQL

■ We instantiate B-tree and R-tree instances of FAST, termed FAST-Btree and FAST-Rtree

■ We use two synthetic workloads: Lookup intensive workload (WL): 80% search and

20% update Update intensive workload, (WU): 20% search and

80% update.

■ number of workload operations to 10 million

■ main memory size to 256 KB

■ Tree index size to 512 MB, and

■ log file size to10 MB (~ 2% of Index size)

20

Experimental Evaluation (2/4)

Memory Size Log File Size

348 lines of code3070 lines of code

308 lines of code

9230 lines of code

8150 lines of code

21

Experimental Evaluation (3/4)

Memory Size Log File Size

22

Experimental Evaluation (4/4)

Log Compaction Recovery

23

Conclusion

■ FAST is a generic framework for flash-aware index structures.

■ FAST guarantees the durability of update transactions applied to the underlying index structure.

■ FAST has good performance compared to its counterparts (LA-tree, FD-tree, and RFTL)

24

Questions