An Eﬀective Flash Memory Manager for Reliable Flash Memory

950IEICE TRANS. INF. & SYST., VOL.E85–D, NO.6 JUNE 2002

PAPER

An Effective Flash Memory Manager for Reliable Flash

Memory Space Management

Han-joon KIM†, Regular Member and Sang-goo LEE†, Nonmember

SUMMARY We propose a new effective method of managingflash memory space for flash memory-specific file systems basedon a log-structured file system. Flash memory has attractive fea-tures such as non-volatility and fast I/O speed, but it also suffersfrom inability to update in situ and from limited usage (erase)cycles. These drawbacks necessitate a number of changes to con-ventional storage (file) management techniques. Our focus is onlowering cleaning cost and evenly utilizing flash memory cellswhile maintaining a balance between these two often-conflictinggoals. The proposed cleaning method performs well especiallywhen storage utilization and the degree of locality are high. Thecleaning efficiency is enhanced by dynamically separating colddata and non-cold data, which is called ‘collection operation.’The second goal, that of cycle-leveling, is achieved to the degreethat the maximum difference between erase cycles is below theerror range of the hardware. Experimental results show that theproposed technique provides sufficient performance for reliableflash storage systems.key words: flash memory, logging, cleaning algorithm, cycleleveling, data collection

1. Introduction

Flash memory is non-volatile solid-state memory witha number of powerful features. Recently, its densityand I/O performance have improved to a level at whichit can be used not only as auxiliary storage for mo-bile computers but also as mass storage in a generalcomputing system [1]–[4]. Flash memory is as fast asDRAM in reading operations and much faster than ahard disk in writing∗. Also, flash memory is about 30%smaller than an equivalent DRAM cell, yet it is morescalable than DRAM due to its simple cell structureand insensitivity to cell failure. This feature enablesthe creation of a large memory space. Furthermore,data in a flash memory-based storage system can beprocessed directly without the need to be brought intomain memory. In other words, flash memory space canbe used as an extended part of the main memory. Inthese respects, flash memory has prospective featuresuseful to a storage system which enables it to providemuch better performance than conventional memory.

Although flash memory has numerous advanta-geous features, it cannot be a complete solution forstorage systems. Before we employ flash memory as theprimary persistent storage device, we have to consider

Manuscript received March 12, 2001.Manuscript revised November 12, 2001.

†The authors are with the School of Computer Scienceand Engineering, Seoul National University, Seoul, Korea.

its two critical drawbacks. First, flash memory can-not be updated in situ; its contents should be erasedbefore new data can be stored. An ‘erase’ operationthat requires more time than a read or write operation(about 0.6–0.8 seconds) resets the memory cells with ei-ther all ones or all zeros. This is because flash memorytechnology only allows the toggling of individual bitsor bytes in one direction for writes. The new free spacemust have been created by an erase (or cleaning) oper-ation beforehand. Thus, updates to flash memory areslower than updates to conventional memory. The sec-ond drawback is that the number of rewrite operationsallowed on each memory cell is limited. Typically, thecycling limit is between 100,000 and 1,000,000 erases,depending on the manufacturer [5], [6]. A flash regionthat approaches its cycling limit will experience fre-quent write failure; each chip cannot be guaranteed toprogram and erase within specific time frames althoughexisting data will still remain readable. Thus, when allthe memory blocks are not evenly utilized, the size ofavailable memory space decreases rapidly. Therefore,this has a critical effect on the system’s lifetime (or themean-time-to-failure), so it becomes a great obstacle indeveloping a ‘reliable’ flash memory-based storage sys-tem that is as highly reliable as a disk-based system.

In order to solve the first drawback (inability toupdate in situ), we must consider how to update dataon flash memory blocks and how to manage the freememory space. For updates, it is necessary to de-couplewrite and erase operations; in other words, an updateshould be implemented as a write of the new data atnew free space and invalidation of the old version at theoriginal location. In addition, since the size of avail-able space is limited, a flash memory system requires acleaning operation or garbage collection that incurs acertain amount of cost to reclaim the invalidated spacewithin the segments for future usage. Moreover, thecleaning cost should be minimized.

The second drawback requires the flash storagesystem to wear down all memory blocks as evenly aspossible; this process is called ‘cycle-leveling’ or ‘wear-leveling.’ As the amount of flash memory space getslarger, cycle-leveling becomes more essential [3]. The

∗It has been reported that read operations in flash mem-ory takes about 85–100 ns, and write operations take 6–9µsper byte [5], [6].

KIM and LEE: AN EFFECTIVE FLASH MEMORY MANAGER FOR RELIABLE FLASH MEMORY SPACE MANAGEMENT951

basic principle of cycle-leveling is that frequently up-dated data is put on the less utilized memory blocks.Thus if cycles are too skew over segments, a large num-ber of data should be rewritten at other less utilizedregions for cycle-leveling.

Consequently, for the flash storage system, we haveto solve several issues related to the cleaning opera-tion and cycle-leveling. The problem is that the ob-jective of minimizing cleaning cost conflicts with thatof cycle-leveling. If flash memory management focuseson lowering the cleaning cost, cycles are skew over seg-ments, and so the system’s reliability is dramaticallyaffected. On the contrary, excessive cycle-leveling gen-erates a large number of invalidated blocks, which de-grade cleaning performance.

In this paper, we describe a new way of mem-ory management for flash storage systems that ac-commodate effectively these two often-conflicting ob-jectives. As a flash space management strategy, weadopt append-only logging (or logging) proposed inlog-structured file systems (LFS) [7], since free spaceshould be claimed before write operations. (In append-only logging, all updates are carried out on a sequentialstructure called a ‘log.’) Based on the cleaning mecha-nism of the LFS, we propose a mechanism that can re-duce cleaning cost and evenly utilize flash memory cells.The cleaning efficiency is enhanced through a ‘collec-tion’ operation whose activity is to collect cold data(infrequently updated) that are fragmented over thelog. The conflict between cleaning and cycle-leveling isresolved by integrating cycle-leveling into the cleaningprocess through a special cleaning criterion for selectingvictim segments.

The remainder of this paper is organized as follows.In Sect. 2, we state how free space is managed under thelogging strategy for flash memory and present perfor-mance metrics and related goals. In Sect. 3, we intro-duce a way of achieving the presented goals along withits analytic discussion. Section 4 presents performanceresults. Section 5 discusses related work. Section 6concludes the paper.

2. Flash Memory Management

As a basic strategy for flash memory space manage-ment, we adopt append-only logging as in the log-structured file system. In previous works such as in[3], [8], [11], a logging approach has been recommendedfor managing flash memory since logging is quite effec-tive in handling flash memory in several ways. First,logging automatically solves the inability to update insitu since an update results in a new write at the endof the log and invalidation of the old. The natural sep-aration of asynchronous erases (cleanings) from writesallows write operations to fully utilize the fixed I/Obandwidth, and thus prevents performance degradationthat may occur when writes and erases are performed

simultaneously. In addition, logging has the positiveside effect of cycle-leveling because append-only writeslead to sequential use of unused space. More impor-tantly, our work can use a cleaning mechanism proposedin LFS since a flash storage system should perform anerase operation for generating new free space.

In order to support logging, the flash memoryspace is partitioned into segments. Each segment isa set of erase blocks† that are logically clustered to beerased together in a single erase operation. In each seg-ment, a fixed number of blocks are organized; the blockis the unit of I/O operation, which is the amount of datatransferred between the flash memory and main mem-ory. The term ‘segment’ is analogous to the segmentdefined as a unit of cleaning operation in Sprite-LFS [7].However, the size of each segment in the flash storagesystem should be sufficiently large since an erase op-eration takes a fixed amount of time regardless of thenumber of memory cells being erased††.

In this section, we first describe a prototype of flashstorage system, and then present our goals for the man-agement of flash memory space.

2.1 Architecture

The flash storage system proposed in this paper is illus-trated as Fig. 1. First, the File Manager is respon-sible for determining and maintaining files as a set oflogical blocks. Next, the Flash Memory Managermanages the physical flash memory; it determines thephysical location of a logical block, and issues the nec-essary flash I/O operation. The activity of the flashmemory manager is similar to that of a disk manager

Fig. 1 Architecture of flash storage system.

†The erase block is an independently erasable elementwithin a flash chip and is 64–128Kbytes in size. The eraseblock is distinguished from the logical I/O unit, block.

††The erase unit size must be determined by taking clean-ing cost into consideration; if the size is too large, cleaningcost increases, and if it is too small, a large number of eraseoperations are issued due to frequent cleaning. The problemof deciding the optimal segment size is beyond the scope ofthis paper.


in disk-based storage systems. Finally, the Flash Me-dia Interface involves a set of low-level operations forflash memory array, including erasing, programming,and error detection. Among these components, ourwork is mainly concerned with the Flash MemoryManager.

The Flash Memory Manager is composed ofthree major components: Allocator, Cleaner, andCycle-Leveler. The Allocator is responsible forkeeping a pool of free segments. It decides which of thefree segments is to be assigned next. The Cleanerreclaims invalid blocks of segments to generate new freespace. Finally, the Cycle-Leveler is responsible formaintaining an even distribution of erase cycles overthe flash segments.

The Flash Memory Manager works as shownin Fig. 2. First, data is newly written to the blocks ina segment, and for updates, the current block contain-ing the old data is marked ‘dead’ and the new data iswritten to a new block, which is then marked ‘live.’ Asthis process goes on, there will be lots of dead blocksin the segments of the log, and the number of avail-able fresh segments will decline. When the number ofavailable fresh segments gets below some threshold, theCleaner is triggered. The cleaning operation on theflash memory log works in two phases: the rewritingand the erasing phases.

In the rewriting phase, the Cleaner first selectsa segment in the log ( 1©), allocates a segment ( 2©), col-lects valid data from the selected segment and rewritesthe data onto the end of log ( 3©). In this phase, thechoice of segments assigned by the Allocator to thelog has an effect on the cleaning and cycle-leveling effi-ciency (refer to Sect. 3.3.2). Then, in the erasing phase,the erase operation is performed in parallel for all eraseblocks in the selected segment. The cleaned segment isreclaimed into the free segments pool ( 4©). Addition-ally, the Cleaner checks whether segments are un-

Fig. 2 Flash memory manager.

evenly cycled. If the degree of skewness in cycles ishigher than a given criterion, the Cleaner activatesthe Cycle-Leveler.

2.2 Problem Definition and Performance Metrics

In order to quantify the efficiency of our methods, wefirst need to present performance measures in termsof cleaning cost, the degree of cycle-leveling, and thesystem’s lifetime. Using these proposed measures, wepresent our aims of flash memory management.

Our cleaning cost model is based on the cleaningcost formula defined according to the eNVy flash stor-age system [2]. The cleaning cost in [2] is defined as

u1−u , where u is the utilization of a segment (the frac-tion of space occupied by live data). That is, the clean-ing cost is the amount of data that need to be rewritten(u) per unit of new space claimed (1 − u). This costmodel reflects the fact that only the cost of rewritingthe valid blocks in a segment is considered since seektime and rotational delay addressed in disk storage arenot applicable in flash memory and the read speed ismuch faster than the write speed.

However, this cleaning cost does not consider thefact that in flash memory, the cleaning is accompaniedby a very expensive erase operation. Even though eraseoperations incur a fixed cost at every cleaning, frequentcleaning over a fixed period results in higher overallcosts. Thus, the Cleaner should minimize the numberof erase (cleaning) operations. In particular, it is veryessential to reduce the utilization of the segment to becleaned when storage utilization is high. The reason isshown in the following lemma.

Lemma 1: Let W denote the number of blocks of thewrite requests after cleaning starts, let uc denote theutilization of the selected segment for cleaning, and letSegSize be the number of blocks per segment. Then,an approximation of the cleaning frequency f is givenby W

(1−uc)SegSize .Proof: see Appendix A.

As shown in Fig. 3, the value of f sharply increases

Fig. 3 Cleaning frequencies from varying utilization of thesegment to be cleaned.


as uc is higher, which means that minimizing the uti-lization of the segment to be cleaned leads to signifi-cantly lower the cleaning frequency especially when theutilization is high.

Obviously, the cleaning cost model in flash memorymanagement should address the number of erase (clean-ing) operations as well as the pure cleaning (writing)cost. First, we need to express the erasure cost per seg-ment (EraseCost) in the context of the writing cost persegment (WriteCost) since the two cost values are notaddible to each other. It is known that the time to erasea whole segment is nearly proportional to the time towrite a whole segment [1]. Thus, we assume that thereexists a ratio ku that can express EraseCost accordingto the scale of WriteCost. Then, EraseCost = ω · ku,where ω is a (fixed) erasure cost (e.g., time cost) mea-sured with a certain criterion. As a result, the totalcleaning cost can be described as follows:

CleaningCost = f · (ω · ku) +f∑

i=1

ui

1− ui(1)

where f is the number of cleanings over a fixed amountof the write request stream, and ui is the utilizationof the segment selected for the ith cleaning. However,this cost formula is not practical due to difficulty incomputing ku. Thus, as an alternative to Eq. (1), weuse only the second term of Eq. (1),

∑fi=1

ui

1−ui, as the

flash cleaning cost. This is because Eq. (1) has almostthe same mathematical behavior as

∑fi=1

ui

1−uiregard-

less of the value of ku (see Appendix B). Intuitively,the alternative formula represents the erasure cost asthe degree of how frequently the cleaning cost at eachcleaning time is accumulated.

Definition 1 (Cumulative Cleaning Cost): The cu-mulative cleaning cost is

∑fi=1

ui

1−ui.

Next, as a measurement of how evenly cycles aredistributed over all the segments, we use the levelingdegree as defined below.

Definition 2 (Leveling Degree): The leveling degree(∆ε) is the difference between the maximum erasecount (εmax) and the minimum erase count (εmin), i.e.,∆ε = εmax − εmin.

Instead of the variance or the standard deviationof erase counts, we choose to use the difference as ameasure of the degree of cycle-leveling because the sys-tem’s lifetime is dependent on the lifetime of the mostworn-out segments.

Generally, perfect cycle-leveling, where the levelingdegree is kept at 0 or 1 at all times, need not be sup-ported to assure the reliability of flash storage systems.This is because erase limits vary from segment to seg-ment within the hardware error range†, and also exces-sive cycle-leveling degrades cleaning (writing) perfor-mance. Therefore, we attempt to achieve cycle-leveling

only to the extent that the leveling degree is within theerror range of the erasure limit.

Lastly, we propose a measurement for evaluatingthe system’s lifetime. We can define the capacity of aflash storage system (flash cycle capacity) as the max-imal amount of (block) write requests that the flashspace can endure. This is proportional to the totalnumber of segment cycles performed up to the failurepoint (failure point is when x% of the space has ex-ceeded its erase limit, where x depends on the systemarchitecture and the workloads in the system). Thus,the used portion of flash cycle capacity up to systemfailure, which is called the ‘running cycle capacity,’ de-termines the system’s lifetime.

Definition 3 (Running Cycle Capacity): The run-ning cycle capacity at time t is

∑Nseg

i=1 εt(Si), whereNseg is the number of segments in the system and εt(Si)is the erase count of segment Si at time t.

In relation to the Definition 3, our objective ofcycle-leveling is to keep the leveling degree below theerror range so that at the time of failure, tf , each ofεtf

(Si), i = 1, . . . , Nseg, is within the error range of theerasure limit, in which case, the flash memory wouldhave been used to its fullest erasure capacity (withinthe error boundary). By increasing this running cyclecapacity, we can lengthen the system’s lifetime.

Let us suppose that without cycle-leveling, the sys-tem fails after x (segments ·erases) are consumed, andwith cycle-leveling, the system fails after y (segments ·erases) are used. Then, the running cycle capacity hasbeen increased by a ratio y

x . Another way to increasethe lifetime is to consume fewer erase cycles by reduc-ing the cleaning frequency. If the cleaning frequencyis further reduced by the ratio a, the lifetime will beincreased by a ratio y

x(1−a) .

3. Optimization of Flash Memory Manage-ment

In this section, we first introduce a basic cleaningmethod mentioned in [7], [12], and then based on thismethod, incrementally optimize the method of manag-ing the flash memory. The main components of ourmethod include a ‘collection’ operation, cycle-levelingalgorithm through a new cleaning index and heuristicsfor segment allocation.

3.1 Limitation of the Greedy Cleaning Method

The performance of a system that requires a cleaningmechanism usually depends on the algorithm it uses [2],[7]. Furthermore, according to the proposed cleaning

†Currently, this error range has not been reported be-cause its measurement is difficult. We will assume that theerror range is a number ε (say, 100).


cost model, the cleaning cost of a segment dependsstrongly upon its utilization. Thus, the most importantaspect of the Cleaner is the segment selection policy,based on which the segments that are to be cleaned areselected. Intuitively, we can expect that it is better toselect the segment with the least utilization. This isknown as the greedy cleaning method [7], which we re-fer to as Greedy I. However, as mentioned in [2], [7],[8], the greedy cleaning method does not perform wellwhen the degree of locality is high. Especially when theutilization is high, locality has a much bigger impact oncleaning cost.

The primary cause of the increase in cleaning costwith locality is the co-existence of cold and non-colddata within each segment on log. After a large numberof logging and cleaning operations, cold data becomesmixed with non-cold data within each segment. Afterthat time, at every cleaning, the cold data fragmentedover the log area moves around uselessly together withnon-cold data, without further invalidation. For thisreason, the utilization of cleaned segments remains sta-ble at a high value. For the same reason, the cleaningfrequency increases as well; the segment whose utiliza-tion is higher due to cold data is more quickly filled upby writes, so it has more chance of being cleaned.

3.2 Separation of Cold Data from Non-cold Data

In order to overcome the limitation of the greedy clean-ing method, we propose a ‘collection’ operation thatisolates cold data from non-cold data. Actually, this isimplemented as another module, called Collector ofthe Flash Memory Manager. The Cleaner peri-odically invokes the Collector, which then collectsfragmented cold data over the log.

The Collector works in two phases: I) searchand collect fragmented cold data (file) blocks, and II)cluster them to the end of the log. Figure 4 illustrates

Fig. 4 Procedure of collection operation: Blocks to beappended are marked as the lower-case letter of the file identifier.

the procedure for the collection operation.In phase I, at each collection time, the Collec-

tor determines the collection size, and searches forcold data that are much fragmented. In the example inFig. 4, the collection size is 10, and as much fragmentedcold data in file ‘A’ and ‘B’ are collected as possible. Inphase II, it writes the blocks of that file back to the endof the log until the collection size is reached. If duringthe process of collection the free space is not enough,the Cleaner can be invoked to generate new space.

After invoking a collection, the segments with col-lected cold data will be less invalidated rather thanother segments, and the segments that are relativelyless utilized segments due to collection operation willbe are selected for cleaning.

This collection operation must have reasonablepolicies with regard to 1) which cold data are to becollected, 2) when cold data are to be collected (i.e.,collection period), and 3) how much cold data are tobe collected (i.e., collection size). As for issue 1, databeing collected should be cold data with a high degreeof fragmentation over the log. For this, we must iden-tify cold data and measure how it is fragmented. Issues2 and 3 are crucial factors that affect the efficiency ofthe Collector. There is a trade-off between collec-tion size (or collection period) and cleaning cost; thatis, too frequent collection or too large amounts of collec-tion would undermine its positive effect. Here, we needto define the terms ‘collection period’ and ‘collectionsize’ to deal with the above issues.

Definition 4 (Collection Period): The collection pe-riod is the number of cleaning operations performeduntil a collection operation is again invoked. If collec-tion period is n, collection operation is invoked onceevery n cleaning times.

Definition 5 (Collection Size): The collection size isthe number of blocks to be collected for cold data col-lection from the log at each collection time.

3.2.1 Identification of Cold Data

In order to tackle issue 1, the Collector uses what wecall the ‘file access window’ (FAW), which keeps a list offiles that are recently accessed, as shown in Fig. 5. Thisfile access window is analogous to a ‘working set’ thatis used for virtual memory management. According tothe principle of the working set, the same blocks tend tobe referenced repeatedly within a working set window(time) with locality [13]. Likewise, we can expect thatfiles kept in FAW will be actively re-accessed for sometime as long as locality exists. Thus, since the files inFAW are likely to be updated and then fragmented, thefiles outside the FAW are likely to be ‘cold.’

Next, among the cold files outside the FAW, higherpriority is given to ones that are more fragmented since


Fig. 5 File access window.

unfragmented cold data is unnecessary to collect. Thus,we have to check what extent the data is spread overthe log. To do this, we estimate the degree of frag-mentation of each candidate file. The fragmentationdegree F (whose range is between 0 and 1) is definedas follows:

F =s

b(if b > Lseg, then b = Lseg) (2)

where b is the number of blocks occupied by the file, sis the number of segments over which the file is frag-mented, and Lseg is the number of segments in the log.If b is larger than Lseg, b is replaced by Lseg since scannot be larger than Lseg. This avoids the problemthat for the same s value, the fragmentation degree ofa large file is relatively small compared to that of asmall file. Depending on this fragmentation degree, theCollector decides whether the candidates are to becollected; it selects, among the files not in the FAW,ones with higher fragmentation degree.

To identify cold data, we choose to use files insteadof blocks since it is impossible to identify the degree offragmentation at the block level. Thus, the mappingtable between a file and its blocks is necessary, whichis maintained in the file manager (see Fig. 4) and isreferenced to locate the blocks of the file.

3.2.2 Collection Size and Collection Period

As cold data grows and becomes fragmented, collectionis required more often. Basically, the appropriate timefor collection operation is when cold data is fragmentedover the log so much that it degrades the cleaning per-formance. Also, the collection size should be less thanthat of all the fragmented cold data. Unfortunately, theproblem of deciding the collection size (or collection pe-riod) requires a fine-tuning knob that is chosen on an adhoc basis. Thus, we propose a simple heuristics basedon utilization and locality.

First, we set the collection size as the followingformula:

ColSize = ks · uavg · lf ·Nseg · SegSize (3)

where uavg is the average utilization, Nseg is the num-ber of segments in flash memory space, SegSize is the

number of blocks in a segment, ks is a constant be-tween 0 and 1 which are experimentally adjusted so asto achieve good cleaning efficiency, and lf denotes thedegree of reference locality, whose range is between 0and 1.

The collection size is determined to be (nearly)proportional to the average utilization. This is becausewhen locality exists, high utilization yields more colddata, which is more likely to be fragmented by continu-ous logging and cleaning operations. Cold data is morelikely to be fragmented as the reference locality of dataincreases. Thus, when locality is high, the collectionsize needs to be set larger. By adding other factors, wemay obtain a more precise collection size.

To make this collection policy complete, we needto define the degree of locality. Since in a log-basedstorage system the memory locations of blocks in fileschange, we measure the degree of locality using the fileaccess window as follows:

lf =|FAW | − |FAW |f

|FAW | − 1(4)

where |FAW | is the length of the file queue in the FAW,and |FAW |f is the number of distinct files in the FAW.Intuitively, we can measure the degree of locality as|FAW ||FAW |f , but in order to map this to values between 0and 1, we use Eq. (4) above. In an extreme case, if onlyone file is accessed for some time, that is, |FAW |f is 1,becomes 1.

Lemma 2: Let u0 denote the utilization of the seg-ment to be cleaned just before collection, let ks be theconstant in Eq. (3), and let lf denote the degree of lo-cality. Then, when using the greedy cleaning method,the utilization of the segment selected for collection isreduced to u0(1− ks · lf ).Proof: see Appendix C.

As stated in the above theorem, the utilizationvalue sharply drops by a ratio ks ·lf due to collection.Then it will gradually grow large and return to thevalue of u0, since the cold data clustered is again evenlyfragmented over the log during continuous cleaning op-erations. In any case, the cleaning cost at each cleaningtime and the cleaning frequency are reduced until theutilization of cleaned segments again level off.

Similarly to the collection size, the collection pe-riod may also be determined according to the averageutilization and the degree of locality; that is, it may beset to be (nearly) proportional to the average utiliza-tion, and as locality is higher, it needs to be set smaller.Fortunately, given the collection size, the collection pe-riod can be determined by the following lemma.

Lemma 3: Let ColSize and SegSize, respectivelydenote the collection size and segment size, let lf denotethe degree of locality, and ucol denote the utilization ofthe segment selected for cleaning after performing a col-lection. Then, when using the greedy cleaning method,


an approximation of the collection period is given byColSize

(1−lf )(1−ucol)SegSize .

Proof: see Appendix D.

We shall refer to the algorithm that employs theabove collection operation on top of the greedy methodas CICL I.

3.2.3 Modification to Cleaning Cost

The previously defined cleaning cost needs to be modi-fied to accommodate the cost (named ‘collection cost’)incurred by the collection operation which also requiresrewriting valid data as with the cleaning operation.The cumulative cleaning cost is modified as:

f∑i=1

(ucol,i

1− ucol,i+

c

1− u0

),

c =ColSize

SegSize× ColPeriod(5)

where f is the cleaning frequency, ucol,i (resp. u0) de-notes the utilization of ith selected segment for cleaningafter performing collection (resp. just before perform-ing collection), and c

1−u0represents the collection cost,

which is the additional cost required to generate (1−u0)of fresh space. This value is best explained as the costrequired if one collection were to be performed at eachcleaning time; in reality, only after as many cleaningoperations as collection periods are executed, one col-lection occurs. Thus, the numerator c of the collec-tion cost can be computed as Eq. (5), where SegSizeis the number of blocks in a segment. For example,suppose SegSize is 500 blocks, and 2,000 blocks arerewritten for a collection when ColPeriod is 100, thenc is 2,000

500 × 1100 = 0.04; that is, a cost of as much as 0.04

should be added to each of the 100 cleaning operationsin order to account for the cost of one collection.

(a) Area of utilization of the selectedsegment for cleaning and locality forpositive collection gain

(b) Changes in collection gainfrom varying utilization (locality= 0.75)

Fig. 6 Two graphic views of Theorem 1.

3.2.4 Analysis of Collection Operation

In this section, we will discuss the collection operationby using analytic methods. For this, we first define so-called ‘collection gain,’ which means the extent of gainobtained from a single collection operation.

Definition 6 (Collection Gain): The collection gainis

∑pi=0(

u01−u0

− ( ucol,i

1−ucol,i+ c

1−u0)), where u0 denotes

the utilization of the selected segment for cleaning justbefore performing collection, ucol,i denotes the utiliza-tion of the segment selected for the ith cleaning afterperforming collection, c is the value of Eq. (5), and pdenotes the collection period.

In this definition, it is assumed that the utiliza-tion of the selected segment for cleaning after p previ-ous cleanings, i.e., ucol,p is equal to u0. In fact, thisassumption is found to be reasonable through our ex-periment. With this assumption and Lemma 3, the fol-lowing theorem can be derived, which indicates whenthe collection operation can give a positive effect on thecleaning performance.

Theorem 1: If (lf > u0 and lf > 1− u0) or (lf ≤ u0

and lf > 14(u0−1) + 1), then the collection gain is posi-

tive, where u0 is the utilization of the segment selectedfor cleaning just before performing collection, and lfdenotes the degree of locality.Proof: see Appendix E.

Theorem 1 is illustrated by the two graphs inFig. 6. Figure 6 (a) depicts that if a coordinate valuecorresponding to the utilization of the segment to becleaned and the degree of locality exists in the shadedarea, the collection gain is more than zero. That is,when the u0 is high (say, over 0.75), the collectiongain is always positive regardless of the degree of lo-cality. Conversely, when u0 is low (say, below 0.75),


the collection gain is positive only if locality increaseswith decreasing u0. Also, when locality is sufficientlyhigh (say, over 0.5), the collection gain increases nearlyby geometric progression as u0 gets higher as shownFig. 6 (b). This analytic result of the collection opera-tion is very encouraging since localized access patternsand high storage utilization are commonly observed inthe real-world systems.

3.3 Cycle-Leveling

For any pattern of data access, uneven distribution oferasures tends to occur. In the worst case, if there arestatic data that are not accessed at all, severe skewin cycles will occur. Although the logging strategyhas a potential effect on cycle-leveling, the flash sys-tem needs a refined leveling method, especially underwrite-intensive environments.

3.3.1 Cleaning Index

Cleaning index is the criterion for selecting segments tobe cleaned; the segment with the lowest index value†

is selected at cleaning time. In our work, a specialcleaning index is defined for both lowering the cleaningcost and promoting cycle-leveling as follows:

Cleaning Index = (1− λ)ui + λ

(εi

εmax + 1

)(6)

where ui and εi, respectively, are the utilization andthe erase count of segment i, and εmax is the maximumof erase counts. The cleaning index is composed of twofactors: utilization and erase count of each segment.These two factors are weighted by so-called ‘normalizedleveling degree,’ denoted as λ, which maps the levelingdegree to values between 0 and 1.

If the cycles are evenly distributed, the cleaningindex is used for lowering the cleaning cost. On thecontrary, when there is severe skew in the cycles, thecleaning index works for cycle-leveling. In other words,at normal times when the leveling degree is below atolerable threshold (the value of λ approaches 0), theutilization ui has to be the chief factor in selecting asegment. When the leveling degree exceeds the tolera-ble limit (the value of λ approaches 1), then the clean-ing index should be calculated either considering bothfactors, or being biased toward cycle-leveling.

Here, in order to prevent excessive cycle-leveling,the weighting variable λ needs to affect the cleaningindex only after there is more than a certain amount ofcycle skewness, so that a small measure of skew in cyclescan be ignored. Therefore, λ is represented by a formof sigmoid function, which increases monotonically withthe leveling degree, ∆ε (see Fig. 7).

λ =

{ 21 + e

kε∆ε

if ∆ε �= 0

0 if ∆ε = 0(7)

Fig. 7 Changes of λ from varying kε.

where kε is a constant that determines the steepnessof this function. Figure 7 shows that as kε gets larger,the value of λ increases more slowly in the initial stage.Thus, the constant can be used for controlling the de-gree of cycle-leveling. Should we wish to loosely achievecycle-leveling, we have only to make kε larger. For ex-ample, when kε is small, say 10, the value of λ rapidlyincreases as the leveling degree is small. Then, thecleaning index is computed considering the ‘erase count’factor, which will result in near-perfect cycle-leveling.When kε is greater than 500, the Cleaner almost doesnot care about cycle-leveling until ∆ε is close to thepoint where the curve begins to rise steeply. Like this,cycle-leveling is achieved within the Cleaner througha special cleaning index. We refer to the methodthat integrates this leveling method into CICL I asCICL II.

3.3.2 Segment Allocation Policy

In terms of the Allocator module, cycle-leveling canbe assisted by carefully selecting the next fresh segmentfrom free space. For this, segment allocation by theAllocator considers the following two cases: 1) whenappending data to the log for generating new free space,and 2) when collecting cold data and rewriting to thelog.

In case 1), a segment with the lowest erase countis selected for replacement to increase its usage. Thenew segment can be expected to reach its next cleaningcycle earlier because of its hot data. Correspondingly,in case 2), a new segment with the highest erase countis allocated to the log. The idea behind this heuristics isthat a large amount of cold data will be clustered ontothe segment allocated for a collection operation, andthus the segment will experience less invalidation thanthe other segments. The method that uses this policyon top of CICL II will be referred to as CICL III.

†In the case of the cost-benefit policy introduced in

Sprite-LFS, the segment whose cleaning index ( age×(1−u)u

)is the greatest is selected [7].


4. Performance Analysis

4.1 Simulator and Workloads

We have built a simulator with 1Gbyte of flash spacethat is divided into 64 flash segments. Each segmentis composed of 256 erase blocks. The cleaning time inTable 1 indicates that at every 100 write requests, theCleaner checks whether the ratio of the number offree segments to the number of total segments is be-low 0.1, in which case one segment is cleaned. TheCollector chooses among the files not in the FAW,the cold ones with a fragmentation degree that exceeds0.8. The constant kε in Eq. (7) is set below 100 toachieve cycle-leveling. The other details pertaining toflash memory space and the Flash Memory Managerare described in Table 1. As for metadata, the segmentusage information required by the Cleaner (includ-ing the number of valid blocks and the erase count ofeach segment) and the file-block mapping table main-tained by the File Manager are kept as internal tablesin the simulator without being explicitly stored in flashmemory.

In our work, for reliable performance evaluation,we have synthetically generated two types of workload,named UNIX and LFS. The UNIX workloads are basedon the size distribution of UNIX files described in [14],where the size of cold data is larger than that of hotdata [15]. The LFS workloads, on the other hand, arecharacterized by frequent modification to small datafiles (4Kbytes files); LFS (90 → 10) workload is thesame as the ‘hot-and-cold ’ workload used in [7].

Since our proposed method affects only updatesand new write operations, read traffic is not modeled inour workload. Moreover, in each workload, write accessexhibits certain amounts of reference locality. In thispaper, to denote the degree of locality, we use the nota-tion similar to that of [1]; that is, ‘x → y.’ This meansthat x% of write accesses go to y% of the data, and theother (100−x)% go to (100− y)% of the data and thus‘50 → 50’ means no locality. Moreover, to represent un-changing (static) data, we specify one more parameterfor static data. For example, ‘x → y, z’ means that x%of write accesses go to y% of the data, and the other(100− x)% of the references go to z% of the data with(100− y − z)% of data unchanged.

Table 1 Simulation parameters.

Flash Memory Space Parameters Flash Memory Manager Parameters

Flash memory type 1Mbit × 8 bytes Cleaning period Every 100 write requests

Size of flash memory 1Gbyte Cleaning timeFree flash spaceTotal flash space

< 0.1

Segment size 16Mbytes Cleaning size 1 segmentNumber of segments 64 Identification of cold data |FAW | = 200, F > 0.8

Number of erase blocks per segment 256 Collection size ks = 0.1–0.3Block size 512 bytes Leveling degree kε = 100

The total number of write requests is about 10millions and the number of data items generated fromthem is about 2,000 files; initially, the write requestsgenerate the files, which are then repeatedly overwrit-ten or deleted. The detailed procedure to develop theUNIX(x → y) workloads with locality of reference isas follows. Firstly, the initial files are generated basedon the size distribution of UNIX files. The generatedfiles are then arranged in ascending order of size. Thenx% of writes are requested, in a pseudo-random numberwith a dynamic seed value, over a set of relatively small-sized files covering the first y% blocks of the sortedfiles. Lastly, the write position within a selected fileis also pseudo-randomly decided. The development ofLFS workloads is the same as that of UNIX workloadsexcept that they do not require the file ordering.

4.2 Simulation Results

Since our focus is on lowering the cleaning cost and si-multaneously achieving satisfactory cycle-leveling, weuse different greedy methods with a simple cycle-leveling as a baseline for performance measurement. Ina simple bid to promote cycle-leveling, when allocatinga free segment to the log, the least worn-out segmentcan be chosen when the maximum difference amongcycles is beyond a threshold value. Greedy II usesthis policy on top of Greedy I. Another cycle-levelingsolution is to swap the data in the least worn-out seg-ment and the most worn-out segment. This swappingmakes the former segment have a better chance of be-ing erased at cleaning time. Greedy III is a methodthat integrates this swapping solution with Greedy I.

In addition to these greedy methods, to compareour methods with other log-based systems in terms ofonly the cleaning cost, we will evaluate the cost-benefitmethod (Cost-Benefit) proposed in Sprite-LFS. InTable 2, we enumerate the various methods of man-aging flash space described above.

4.2.1 Experiment 1: Tuning Collection Operation

The performance of the Collector is determined bythe size of the FAW, the collection size, and the collec-tion period. The FAW does not fulfill its function if itssize is too large or too small. Figure 8 illustrates thecumulative cleaning cost for different workloads when


Table 2 Comparison of various flash space management methods.

Methods Cleaning index Separation of cold data Segmentation allocation

Greedy I u Not Available Not AvailableGreedy II u Not Available Most worn-out segment allocationGreedy III u Not Available Data swapping

Cost-Benefitage×(1−u)

uAge-sorting Not Available

CICL I u Collection (refer to Sect. 3.2) Most worn-out segment allocation

CICL II (1 − λ)ui + λ εiεmax+1

Collection (refer to Sect. 3.2) Most worn-out segment allocation

CICL III (1 − λ)ui + λ εiεmax+1

Collection (refer to Sect. 3.2) Allocation heuristics (refer to Sect. 3.3.2)

Fig. 8 Cumulative cleaning cost vs. size of FAW.

using CICL I. As shown in this figure, when the size ofthe FAW is around 200, the system shows the smallestcleaning cost. Thus, most of our experiments assume asize of 200 for the FAW.

In addition, the Collector responds sensitivelyaccording to the collection size. Figure 9 shows thecumulative cleaning cost according to changes in col-lection size when using CICL I. From this figure, wecan see that in order for collection to reduce the clean-ing cost, the collection size should be greater than somethreshold, but excessive cold data causes an increase intotal cleaning cost. In our experiments, based on Fig. 9,ks, which determines the collection size, is set to 0.1–0.3. Then using the collection size, the collection periodis determined by Lemma 3.

4.2.2 Experiment 2: Cleaning Cost

Figure 10 shows the changes in utilization of theselected segment to be cleaned. In the case ofGreedy I, the utilization of cleaned segment first in-creases sharply, but after having processed a certainnumber of cleaning cycles, it stays almost constant. In[12], the phenomenon was analytically proved for thecase of uniform reference of data. The curve of utiliza-tion of cleaned segment approaches a plateau near thetime it takes for cold data to be fragmented over allsegments in the log. In the case of CICL I, the utiliza-tion of cleaned segment and the cleaning frequency arereduced, and the convergence value is also suppressedcompared to Greedy I. On average, the cumulativecleaning cost of CICL I is 30% smaller than that of

Fig. 9 Cumulative cleaning cost vs. collection size.

Fig. 10 Comparison of CICL I and Greedy I in terms of theutilization of the selected segment for cleaning.

Greedy I. In addition, we can also see that the uti-lization of cleaned segment undergoes fluctuation. Thisis why cold data is clustered by the collection opera-tion, and is then re-fragmented by continuous loggingor cleaning.

Figure 11 shows the cumulative cleaning cost ac-cording to changes in average utilization when usingGreedy I and CICL I. In this figure, we can observethat the benefit of collection is bigger as utilization anddegree of locality are higher.

Figures 12 and 13 illustrate the cumulative clean-ing cost incurred by different methods for each work-load. In most workloads, our method has significantbenefits over greedy methods. Note that particularlywhen locality exists and segment utilization is high, our


methods (CICL series) outperform the greedy methods(Greedy series) in terms of cleaning cost and levelingdegree. For example, by paying a collection cost ofonly about 8% of the total cleaning cost in the UNIX(95 → 5) workload, CICL III reduces the cumulativecleaning cost by about 35% compared to Greedy I andGreedy II.

In CICL II, the drop in cleaning cost achieved bycollection is less visible due to cycle-leveling. However,CICL III regains the lost cost reduction effect as a

Fig. 11 Comparison of CICL I and Greedy I in terms of thecumulative cleaning cost.

Fig. 12 Comparison of cumulative cleaning cost.(utilization = 75%)

Fig. 13 Comparison of cumulative cleaning cost.(utilization = 85%)

result of the proposed segment allocation policy. Inthe UNIX(95 → 5) workload, while CICL III levelserase cycles more strictly than CICL I and CICL II,CICL III lowers the cleaning cost further than theother CICL methods.

In the UNIX (90 → 10, 80) workload that includesunchanging data, the cleaning cost reduction due tocollection is smaller than in other workloads, since thedata are less fragmented. Also, in the LFS workloads,our methods show smaller cost reduction compared tothe UNIX workloads. Since an LFS workload containsonly small files, less cold data in segments moves aroundthan that in the UNIX workload.

The Cost-Benefit method incurs a high overheadin our system although it is known to have a good per-formance in disk-based storage systems. This is becausethe size of segments used in a flash storage system aremuch larger than in a disk-based storage system. Ac-cording to the cost-benefit policy, segments which haveless utilization and greater age are selected as victimsat cleaning time. However, even if blocks are arrangedaccording to their age, the age values within large seg-ments are not usually consistent, which results in a fail-ure to demonstrate the capability of the method.

4.2.3 Experiment 3: Cleaning Frequency

In order to evaluate how much the cleaning frequencycan be reduced regardless of the size of a given write re-quests, we has measured so-called ‘inter-cleaning writecapacity,’ which is computed by dividing the size of thewrite request stream by the number of cleaning oper-ations. This measure provides the write capacity be-tween cleanings. That is, a larger value indicates thatfor a given amount of writes, cleaning is delayed, anderase commands become less frequent.

Figures 14 and 15 illustrate that when there islocality of reference, our methods have larger inter-cleaning write capacity than the greedy methods inmany cases. However, CICL III shows smaller writecapacity in the LFS(80 → 20), and LFS(95 → 5) work-loads. Also, in the UNIX(80 → 20) workload, CICL

Fig. 14 Comparison of inter-cleaning write capacity.(utilization = 75%)


series perform worse than greedy method. Here, notethat small inter-cleaning write capacity does not nec-essarily mean large cumulative cleaning cost. Whenthe Collector works very effectively, that is, collectsan optimal amount of cold data at an optimal time,the cleaning cost incurred by each cleaning can becomevery small. At this time, although the inter-cleaningwrite capacity is small, the cumulative cleaning cost isnot very large.

4.2.4 Experiment 4: Cycle-Leveling

Figures 16, 17, and 18 show changes in the leveling de-gree as cleaning operations proceed for different work-loads with locality. Some fluctuating graphs with asmall value of leveling degree represent the process ofcycle-leveling.

In most workloads, CICL II and CICL III allow∆ε to fluctuate within a given error range. For work-load UNIX (90 → 10, 80) in particular, CICL II andCICL III provide stable leveling effects although thesemethods have a smaller cleaning cost, while Greedy Iand CICL I show severe skew in cycles. Greedy IIoutperforms Greedy I, but its effect is not satisfactory.Greedy III also has a leveling effect similar to CICLmethods, but among all the methods with the exceptionof Cost-Benefit, Greedy III has the worst cleaningefficiency; this is because a large amount of write cost isrequired to swap data between the youngest and oldestsegments.

Also, these figures suggest that the segment allo-cation policy that belongs to CICL III is helpful incycle-leveling compared to the simple policy of select-ing the segment with the lowest erase count; this pol-icy in effect contributes to lowering the cleaning costas well. Moreover, CICL I, which only employs col-lection without cycle-leveling, also shows some degreeof cycle-leveling effect. This is because the collectionoperation invalidates the blocks in the segments thatcontain cold data, which makes these segments morelikely to be cleaned.

As mentioned in Sect. 3.3.1, the constant kε in-

Fig. 15 Comparison of inter-cleaning write capacity.(utilization = 85%)

cluded in the proposed cleaning index can be usedfor controlling the degree of cycle-leveling, as shownin Fig. 19. As kε gets larger, cycle-leveling is morestrictly achieved. Even though we maintain a ratherstrict cycle-leveling, the cleaning efficiency does notdegrade. Furthermore, we project that, for less strictcycle-leveling, the cleaning cost and frequency will bereduced further. Actually, this constant kε needs to be

Fig. 16 Changes of leveling degree: workload UNIX(95 → 5).

Fig. 17 Changes of leveling degree: workloadUNIX(90 → 10, 80).

Fig. 18 Changes of leveling degree: workload LFS(90 → 10).


adjusted according to environment where the system isused. Moreover, if we could know the error range oflimited cycles of flash memory, this could be reflectedin the value of kε.

Figures 20 and 21 represent the pattern of erasecounts of all segments over time (write requests)for workload UNIX(90 → 10, 80). In the case ofGreedy II, some deep valleys are shown over writerequests, but in the case of CICL III, a generally flatplane is formed. This indicates that CICL III dis-tributes the erase cycles more evenly than Greedy II.

Fig. 19 Control of leveling degree by kε.

Fig. 20 Pattern of erase counts of segments: using Greedy II.

Fig. 21 Pattern of erase counts of segments: using CICL III.

Let us consider the system’s lifetime with thesefigures. Suppose that the cycling limit is about 100,and a system failure occurs when the portion of flashspace above this cycling limit exceeds 20% of the totalspace. Then at failure time, the running cycle capacityof Greedy II is about 5,700 (segment · erases), whilethat of CICL III is about 6,200 (segment · erases).Furthermore, for a given number of writes, the clean-ing frequency is reduced further by a ratio of about 0.1.Then, we can expect that CICL III increases the sys-tem’s lifetime by about 20%(≈ 6,200

5,700×(1−0.1) −1.0) fromthat given by Greedy II.

5. Related Work

The methods found in previous works on flash memorymanagement such as [1]–[3], [8], [9] have focused mainlyon lowering cleaning cost. To enhance cleaning per-formance, a number of techniques in proposed in LFS(disk-based log-structured file system) have been ap-plied to flash storage systems since cleaning in flashstorage systems is analogous to garbage collection inLFS. The greedy policy was used in [3], [9], and in [8]the cost-benefit policy was modified so as to dynami-cally adjust the threshold values of the number of freesegments and segment utilization for triggering cleaner.Also, as a solution to separating cold data from non-cold data, LFS proposed a technique by which validblocks in the buffer are grouped by age before beingflushed into the disk [7], [10]. This age-sorting methodwas used as a part of the DAC algorithm proposed in[1] in order to migrate accessed data to their write fre-quencies. This data migration method is similar to lo-cality gathering in eNVy system [2] in which valid datain cleaned segments are sorted out in approximate or-der of the degree of locality. Different from such datamigration methods, [8] employs separate segments forcleaning the cold segments (whose data is rarely up-dated) under the modified cost-benefit policy so thatcold data cannot be mixed with non-cold data. Thesedata separation methods make cold data and non-colddata separate at all cleaning (or data update) times.This constant separation activity can disturb normalI/O operation and incur extra overhead. In contrast,under our flash memory management policy, the mod-erate amount of cold data are segregated from non-colddata only at an appropriate time after recognizing cur-rent data access pattern.

As for cycle-leveling, most previous studies haveproposed naive and simple methods of a “flip-flop” na-ture. The cycle-leveling approach in [3] is similar toGreedy II, and [2] uses the swapping approach as inGreedy III. However, these methods consume a lot ofsystem resources and time, which disturbs normal I/Oexecution. For example, swapping data between twosegments requires providing buffer memory as large as asegment, erasing two segments, and rewriting swapped


data. Furthermore, the rewriting should wait until theerasing ends. In contrast, our method can avoid thisproblem by considering cycle-leveling at the cleaningtime.

6. Conclusion

We have proposed a way of managing flash memoryspace for flash storage system that copes effectivelywith the features of flash memory while overcomingits constraints. We submit that our flash memorymanagement objectives have been achieved; the pro-posed method overcomes several problems raised byflash memory’s one-way writing and limited cycling.

Simulation results show that the cleaning methodinvolving collection operation performs especially wellwhen storage utilization and the degree of locality areboth high. This feature was also discussed with theanalysis of collection operation. Moreover, we can en-sure that the proposed cycle-leveling method works welltogether with our cleaner while avoiding internal con-flicts between them. Previous cycle-leveling methodsdisturb cleaning operation and normal I/O, but ourmethod works harmoniously with other components.This effect is possible since logging, collection opera-tion, and segment allocation policy have the potentialfunctionality of cycle-leveling. Consequently, our cycle-leveling method allows maximal storage space to beavailable at all times, so as to allow the constructionof a highly reliable flash storage system.

The proposed method can be particularly attrac-tive for flash storage systems in an office/engineering orOLTP environment, where small random writes with lo-cality are common. Additionally, the core componentsof our method, namely the collection operation andcleaning index, do not depend on logging even thoughwe use a logging strategy for flash space management;other strategies can be easily applied in conjunctionwith our method.

Acknowledgements

This work was supported by the Brain Korea 21project. The RIACT at Seoul National University alsoprovided research facilities for this work.

References

[1] M.L. Chiang, Paul C.H. Lee, and R.C. Chang, “Usingdata clustering to improve cleaing performance for flashmemory,” Software Practice and Experience, vol.29, no.3,pp.267–290, 1999.

[2] M. Wu and W. Zwaenepoel, “eNVy: A non-volatile mainmemory storage system,” Proc. ASPL94, pp.86–97, 1994.

[3] B. Dipert and M. Levy, “Flash memory file systems,” in De-signing with FLASH MEMORY, pp.227–271, Annabooks,San Diego, 1994.

[4] R. Ceres, F. Douglis, K. Li, and B. Marsh, “Operating sys-tem implications of solid-state mobile computers,” Techni-cal Report MITL-TR-56-93, Matsushita Information Tech-nology Laboratory, 1993.

[5] Intel, Flash Memory Databook, 1995.[6] Intel Corp., Series 2+ Flash Memory Card Family

Datasheet, http://www.intel.com/design/flcard/datashts,1997.

[7] M. Rosenblum and J.K. Ousterhout, “The design and im-plementation of a log-structured file system,” ACM Trans.Computer Systems, vol.10, no.1, pp.26–52, 1992.

[8] A. Kawaguchi, S. Nishioka, and H. Motoda, “Flash memorybased file system,” Proc. USENIX95, pp.155–164, 1995.

[9] F. Douglis, R. Ceres, F. Kaashoek, K. Li, B. Marsh, andJ.A. Tauber, “Storage alternatives for mobile computers,”Proc. OSDI94, pp.25–37, 1994.

[10] J.N. Matthews, D. Roselli, A.M. Costello, R.Y. Wang,and T.E. Anderson, “Improving the performance of log-structured file systems with adaptive methods,” Proc.SOSP97, pp.238–251, 1997.

[11] D. See and C. Thurlo, “Managing data in an embed-ded system utilizing flash memory,” Intel Technical Note,http://www.intel.com/design/flcomp/papers/esc flsh.htm,1995.

[12] J.T. Robinson, “Analysis of steady-state segment storageutilization in a log-structured file system with least-utilizedsegment cleaning,” OS Review, vol.30, no.4, pp.29–32,1996.

[13] P.J. Denning, “Working sets past and present,” IEEETrans. Software Eng., vol.6, no.1, pp.64–84, 1980.

[14] G. Irlam, “Unix file size survey,”http://www.base.com/gordoni/gordoni.html, 1993.

[15] C. Ruemmler and J. Wilkes, “UNIX disk access patterns,”Proc. USENIX93, pp.405–420, 1993.

Appendix A: Proof of Lemma 1

The cleaning frequency is obtained by dividing allwrite requests W by the write requests for one clean-ing. Within the selected segment, there are a totalof uc ·SegSize in-use blocks, and after cleaning thatsegment, the resulting free space will be filled with(1 − uc)SegSize writes; that is, for every cleaning,(1 − uc)SegSize writes are requested. For simplicity,we assume the greedy cleaning policy (refer to Sect. 3.1).In that case, after performing sufficient cleaning opera-tions, the utilization of cleaned segments becomes con-stant [12]. We also assume that W equals the numberof blocks of the remaining rewrite requests after the uti-lization value becomes constant. Thus, f can be givenby W

(1−uc)SegSize . ✷

Appendix B: An Alternative of Cost Formula

Suppose ui for all i equals uc. Then substitut-ing the equation of Lemma 1 into Eq. (1) yieldsW{(1−ω·ku)uc+ω·ku}

(1−uc)2·SegSize . Similarly, the alternative to the

cost formula,∑f

i=1ui

1−ui, becomes W ·uc

(1−uc)2·SegSize . Wecan easily find that these two formulas have almost thesame mathematical behavior regardless of the value ofku by computing their derivatives.


Appendix C: Proof of Lemma 2

The greedy cleaning method allows cold data tobe uniformly distributed over the log area. Thus,the expected size of the cold data to be collectedfrom segment i (whose utilization is ui) is ui

uavg×

ks·uavg·lf ·Nseg·SegSizeNseg

= ks ·ui · lf ·SegSize. Then, forsegment i, whose cold data is collected, its size of validdata is ui(1−ks · lf )SegSize. Therefore, the utilizationof segment i is ui(1− ks · lf ). ✷

Appendix D: Proof of Lemma 3

By the Definition 4, the collection period is calculatedas a ratio of the size of write requests per collection (X )to the size of write request per cleaning (Y). First, forcomputing X , we consider the fact that as locality ishigher, the rate of fragmentation of the cold data getsslower. We assume that only (1−lf) of all accesses go tothe cold data. Hence, the write requests per collectionequals ColSize

(1−lf ) . Secondly, as for Y, we expect that acleaned segment whose utilization is ucol has valid datawith a ratio (1−ucol). Thus, Y equals (1−ucol)SegSize.

✷

Appendix E: Proof of Theorem 1

From Lemma 3 and Eq. (5), the value of c is calculatedas (1 − ucol,i) (1 − lf ). Thus, the collection gain

is rewritten by∑p

i=1

(u0−(1−ucol,i)(1−lf )

1−u0− ucol,i

1−ucol,i

),

which is simply denoted as∑p

i=1 gu0,lf (u) with ucol,i =u. Here, if gu0,lf (u) > 0, which is a sufficient condi-tion for positive collection gain, then

∑pi=1 gu0,lf (u) >

0. Thus, let us differentiate gu0,lf (u) with respect

to u, then∂gu0,lf

(u)

∂u = 1−lf1−u0

− 1(1−u)2 . By us-

ing this derivative, we find that gu0,lf has the max-

imum value at um = 1 −√

1−u01−lf

. Here, there ex-

ist two cases for gu0,lf (u) > 0. (i) If um < 0, thengu0,lf (0) > 0. (ii) If um ≥ 0, then gu0,lf (um) >0. Consequently, as a sufficient condition for∑p

i=1 gu0,lf (u) > 0, we have (lf > u0 and lf > 1− u0)

or(lf ≤ u0 and lf > 1

4(u0−1) + 1). ✷

Han-joon Kim received the B.S. andM.S. degrees in Computer Science andStatistics from Seoul National University,Seoul, Korea in 1996 and 1998 and thePh.D. degree in Computer Science andEngineering from Seoul National Univer-sity, Seoul, Korea in 2002, respectively.His current research interests include filesystems, databases, information systems,and data/text mining technologies.

Sang-goo Lee received the B.S. de-gree in Computer Science from Seoul Na-tional University, Seoul, Korea in 1985and the M.S. and Ph.D. degrees in Com-puter Science from Northwestern Univer-sity, Evanston, IL, in 1987 and 1990, re-spectively. He taught at the University ofMinnesota from 1989 to 1990, and workedas a research engineer at Electronic DataSystems, Troy, MI, from 1990 through1992. He is currently an associate pro-

fessor at the School of Computer Science and Engineering, SeoulNational University, Korea. His current research interests includedatabases, e-business solutions, information retrieval, and digitallibraries.

Documents

An Eﬀective Flash Memory Manager for Reliable Flash Memory