Seminar Report on File System in Linux

7/29/2019 Seminar Report on File System in Linux

1/14

File Systems in Linux

Gurdev Si ngh, New Del hi

Linux file system

The structure of the Linux file system

The Li nux f i l e syst em t r eat s ever yt hi ng as a f i l e. Thi si ncl udes i mages, t ext f i l es, pr ogr ams, di r ect or i es,par t i t i ons and har dwar e devi ce dr i ver s.

Each f i l e syst em cont ai ns a cont r ol bl ock, whi ch hol dsi nf or mat i on about t hat f i l e syst em. The ot her bl ocks i n t hef i l e syst em ar e i nodes, whi ch cont ai n i nf or mat i on abouti ndi vi dual f i l es, and dat a bl ocks, whi ch cont ai n t hei nf or mat i on st or ed i n t he i ndi vi dual f i l es.

Ther e i s a subst ant i al di f f er ence bet ween t he way t he usersees t he Li nux f i l e syst em ( f i r st sense) and t he way t heker nel ( t he cor e of a Li nux syst em) act ual l y st or es t hef i l es. To t he user , t he f i l e system appears as ahi er ar chi cal ar r angement of di r ectori es t hat cont ai n f i l esand ot her di r ect or i es ( i . e. , subdi r ector i es). Di rect or i esand f i l es ar e i dent i f i ed by t hei r names. Thi s hi er ar chy

start s f r om a si ngl e di r ect or y cal l ed r oot , whi ch i sr epr esent ed by a " / " ( f or war d sl ash) .

( The meani ng of r oot and " / " are of t en conf usi ng t o newuser s of Li nux. Thi s because each has t wo di st i nct usages.The ot her meani ng of r oot i s a user who has admi ni st r at i vepr i vi l eges on t he comput er , i n cont r ast t o or di nar y user s,who have onl y l i mi t ed pr i vi l eges i n or der t o pr ot ect syst emsecur i t y. The ot her use of " / " i s as a separ at or bet weendi r ector i es or bet ween a di r ectory and a f i l e, si mi l ar t ot he backward sl ash used i n MS- DOS. )

The Fi l e syst em Hi erar chy Standar d ( FHS) def i nes t he mai ndi r ect or i es and t hei r cont ent s i n Li nux and ot her Uni x- l i keoper at i ng systems. Al l f i l es and di r ect or i es appear undert he r oot di r ect or y, even i f t hey ar e st or ed on di f f er entphysi cal devi ces ( e. g. , on di f f er ent di sks or on di f f er entcomput er s) . A f ew of t he di r ect or i es def i ned by t he FHS are/ bi n ( command bi nar i es f or al l user s) , / boot ( boot l oaderf i l es such as t he ker nel ) , / home ( user s home di r ect or i es) ,/ mnt ( f or mount i ng a CDROM or f l oppy di sk) , / r oot ( homedi r ect or y f or t he r oot user ) , / sbi n ( execut abl es used onl y


2/14



by t he r oot user ) and / usr ( wher e most appl i cat i on pr ogr amsget i nst al l ed) .

To t he Li nux ker nel , however , t he f i l e syst em i s f l at . Thati s , i t does not :

have a hi er ar chi cal st r uct ur e di f f er ent i at e bet ween di r ect or i es, f i l es or pr ogr ams i dent i f y f i l es by names. I nst ead, t he ker nel uses

i nodes t o r epr esent each f i l e.

The df command i s used t o show i nf or mat i on about each oft he f i l e syst ems whi ch ar e cur r ent l y mount ed on ( i . e. ,

connect ed t o) a syst em, i ncl udi ng t hei r al l ocat ed maxi mumsi ze, t he amount of di sk space t hey ar e usi ng, t hepercent age of t hei r di sk space t hey ar e usi ng and wheret hey ar e mount ed ( i . e. , t he mount poi nt ) . ( Her e f i l e syst emsi s used as a var i ant of t he f i r st meani ng, r ef er r i ng t o t hepar t s of t he ent i r e hi er ar chy of di r ect or i es. )

df can be used by i t sel f , but i t i s of t en mor e conveni entt o add t he - m opt i on t o show si zes i n megabytes rat her t hani n t he def aul t ki l obyt es:

d f -m

A col umn showi ng t he type of each of t hese f i l e syst ems canbe added t o t he f i l e syst em t abl e pr oduced by t he abovecommand by usi ng t he - - pr i nt - t ype opt i on, i . e. :

df - m - - pr i nt - type

Thi s command generat es a col umn l abel ed Type. For a Red HatLi nux i nst al l at i on on a home comput er most of t he ent r i esi n t hi s col umn wi l l pr obabl y be ext 3 and/ or ext2.

File System Layout

For conveni ence, t he Li nux f i l e syst em i s usual l y t houghtof i n a t r ee st r uct ur e. On a st andar d Li nux syst em you wi l lf i nd t he l ayout gener al l y f ol l ows t he scheme pr esent edbel ow.


3/14



Figure 3-1. Linux file system layout


4/14



Thi s i s a l ayout f r om a RedHat syst em. Dependi ng on t hesyst em admi n, t he oper at i ng system and t he mi ssi on of t heUNI X machi ne, t he st r uct ur e may var y, and di r ect or i es may

be l ef t out or added at wi l l . The names ar e not evenr equi r ed; t hey ar e onl y a convent i on.

The t r ee of t he f i l e syst em st ar t s at t he t r unk or slash,i ndi cat ed by a f or war d sl ash ( / ) . Thi s di r ectory,cont ai ni ng al l under l yi ng di r ect or i es and f i l es, i s al socal l ed t he root directoryor "t he r oot " of t he f i l e syst em.

Di r ect or i es t hat ar e onl y one l evel bel ow t he rootdi r ect or y ar e of t en pr eceded by a sl ash, t o i ndi cat e t hei rposi t i on and pr event conf usi on wi t h ot her di r ect or i es t hat

coul d have t he same name. When st ar t i ng wi t h a new syst em,i t i s al ways a good i dea t o t ake a l ook i n t he r ootdi r ector y.


5/14



inode

An i node st ands f or i ndex- node. I node st or es basi ci nf or mat i on about a r egul ar f i l e, di r ector y, or ot her f i l esyst em obj ect . The i node number i s a uni que i nt egerassi gned t o t he devi ce upon whi ch i t i s st or ed. Al l f i l esar e har d l i nks to i nodes. Whenever a pr ogr am r ef er s t o af i l e by name, t he syst em concept ual l y uses t he f i l ename t osear ch f or t he corr espondi ng i node. Many comput er progr amsof t en gi ve i - node number s t o desi gnat e a f i l e. Popul ar di ski nt egr i t y checki ng ut i l i t i es f sck or pf i l es command mayser ve her e as exampl es.

The st at syst em cal l r et r i eves a f i l e' s i node number andsome of t he i nf or mat i on i n t he i node. I n t he Li nux ker nel ,t here ar e t hr ee st orage ar eas where i nodes ar e saved.Fi r st , t her e ar e t wo l i st s ( bot h mai nt ai ned as doubl yl i nked l i st s) . One of t hese l i st s saves used i nodes, whi l eanother st or es unused i nodes. Ther e i s a hash t abl e thatst ores al l used i nodes as wel l , si nce hash t abl es makessear chi ng f or i nodes f ast er . The hash val ue i s based onsuper bl ock addr ess and i node number of t he i ndex. Andt hen, f i nal l y, t her e i s a st r uct ur e t hat st or es t he number

of used and unused i nodes.

Structure of inode

Thi s st r uct ur e r esembl es t he f ol l owi ng:

nr _i nodes var i abl e st or es t he number of i nodes, whi l enr _f r ee_i nodes st ores t he number of unused i nodes.

The i nodes st r uct ur e i s wi del y used i n f i l e syst emi mpl ement at i on. I t l ooks somet hi ng l i ke t hi s:


6/14




7/14



Now t hat we have l ooked i nt o t he i node st r uctur e, howshoul d we cr eat e and dest r oy i nodes? Li nux ker neli mpl ement er s pr ovi de t wo f unct i ons f or t hi s:

I get ( ) and i put ( )

The def i ni t i on of t hi s f unct i on i s ( ent i r e code noti ncl uded) :

The i get ( ) f unct i on provi des t he i node vi a t he super bl ocksb and i node number i no of i node. An i node t hat wasr et r i eved wi t h i get ( ) must be opened agai n usi ng t hef unct i on i put ( ) .


8/14



A f i l e' s i node number can be f ound usi ng the l s - i command,whi l e t he l s - l command wi l l r et r i eve i node i nf or mat i on.

Inodes and Operations

Li nux keeps a cache of act i ve and recent l y used i nodes.Ther e ar e t wo pat hs by whi ch t hese i nodes can be accessed.

The f i r st i s t hrough t he dcache descr i bed above. Eachdent r y i n the dcache ref er s t o an i node, and ther eby keeps

t hat i node i n the cache.

The second pat h i s t hrough t he i node hash t abl e. Each i nodei s hashed ( t o an 8 bi t number ) based on t he addr ess of t hef i l e- syst em' s super - bl ock and t he i node number . I nodes wi t ht he same hash val ue are then chai ned t oget her i n a doubl yl i nked l i st .

Access t hough t he hash t abl e i s achi eved usi ng t he i getf unct i on. i get i s onl y cal l ed by i ndi vi dual f i l e- syst emi mpl ement at i ons when l ooki ng up an i node (whi ch wasn' t

f ound i n t he dcache) , and by nf sd.

Basi ng t he hash on t he i node number i s a bi t r est r i ct i ve asi t assumes t hat ever y f i l e- syst em can uni quel y i dent i f y af i l e i n 32 bi t s . Thi s i s a probl em at l east of t he NFSf i l e- syst em, whi ch woul d pr ef er t o use t he 256 bi t f i l ehandl e as t he uni que i dent i f i er i n t he hash.

The nf sd usage mi ght be bet t er ser ved by havi ng t he f i l e-syst em pr ovi de a f i l ehandl e- t o- i node mappi ng f unct i on whi chhas i nt er pr et t he f i l ehandl e however i s most appr opr i at e.

Inode Methods

st r uct i node_oper at i ons {

st r uct f i l e_oper at i ons * def aul t _f i l e_ops;


9/14



i nt ( *cr eat e) ( str uct i node *, str uct dent r y *, i nt ) ;

st r uct dent r y * ( *l ookup) ( st r uct i node *, st r uctdent r y *) ;

i nt ( * l i nk) ( str uct dent r y *, st r uct i node *, str uctdent r y *) ;

i nt ( *unl i nk) ( st r uct i node *, st r uct dent r y *) ;

i nt ( *syml i nk) ( st r uct i node *, st r uct dent r y*, const char *) ;

i nt ( *mkdi r ) ( st r uct i node *, st r uct dent r y *, i nt ) ;

i nt ( *rmdi r ) ( st r uct i node *, st r uct dent r y *) ;

i nt ( *mknod) ( st r uct i node *, st r uct dent r y*, i nt , i nt ) ;

i nt ( *r ename) ( st r uct i node *, st r uct dent r y *,

st r uct i node *, st r uct dent r y *) ;

i nt ( *readl i nk) ( str uct dent r y *, char * , i nt ) ;

st r uct dent r y * ( *f ol l ow_l i nk) ( st r uct dent r y *,st r uct dent r y *, unsi gned i nt ) ;

i nt ( *get _bl ock) ( st r uct i node *, l ong, st r uctbuf f er _head *, i nt ) ;

i nt ( *readpage) ( st r uct f i l e *, st r uct page *) ;

i nt ( *wr i t epage) ( st r uct f i l e *, st r uct page *) ;

i nt ( *f l ushpage) ( st r uct i node *, st r uct page *,unsi gned l ong) ;

voi d ( *t r uncat e) ( st r uct i node *) ;


10/14



i nt ( *per mi ssi on) ( st r uct i node *, i nt ) ;

i nt ( *smap) ( st r uct i node *, i nt ) ;

i nt ( *reval i dat e) ( st r uct dent r y *) ;

};

default_file_ops

Thi s poi nt s t o t he def aul t t abl e of f i l e oper at i ons f orf i l es opened on t hi s i node. When a f i l e i s opened, t he f _op

f i el d i n t he f i l e s t ructure i s i ni t i al i sed f rom t hi s , andt hen t he open met hod i n t he f i l e_oper at i ons t abl e i scal l ed. That method may choose t o change the f _op t o adi f f er ent ( non- def aul t ) met hod t abl e. Thi s i s done, f orexampl e, when a devi ce speci al f i l e i s opened.

create

Thi s, and t he next 8 met hods are onl y meani ngf ul on

di r ect or y i nodes.

cr eat e i s cal l ed when t he VFS want s t o cr eat e a f i l e wi t ht he gi ven name ( i n t he dent r y) i n t he gi ven di r ect or y. TheVFS wi l l have al r eady checked t hat t he name doesn' t exi st ,and t he dent r y passed wi l l be a negat i ve dent r y meani ngt hat t he i node poi nt er wi l l be NULL.

Cr eat e shoul d, i f successf ul , get a new empt y i node f r omt he cache wi t h get _empt y_i node, f i l l i n t he f i el ds andi nser t i t i nt o t he hash t abl e wi t h i nser t _i node_hash, mar k

i t di r t y wi t h mar k_i node_di r t y, and i nst ant i at e i t i nt o t hedcache wi t h d_i nst ant i at e.

The i nt ar gument cont ai ns t he mode of t he f i l e whi ch shoul di ndi cat e t hat i t i s S_I FREG and speci f y t he r equi r edper mi ssi on bi t s.

lookup

l ookup shoul d check i f t hat name (gi ven by t he dent r y)exi st s i n t he di r ect or y (gi ven by the i node) and shoul d


11/14



updat e t he dent r y usi ng d_add i f i t does. Thi s i nvol vesf i ndi ng and l oadi ng t he i node.

I f t he l ookup f ai l ed t o f i nd anyt hi ng, t hi s i s i ndi cat ed byr et ur ni ng a negat i ve dent r y, wi t h an i node poi nt er of NULL.

As wel l as r et ur ni ng an er r or or NULL, i ndi cat i ng t hat t hedent r y was cor r ect l y updated, l ookup can r et ur n anal t er nat e dent r y, i n whi ch case t he passed dent r y wi l l ber el eased. I don' t know i f t hi s possi bi l i t y i s act ual l yused.

link

The l i nk met hod shoul d make a hard l i nk f r om t he namer ef er ed t o by the f i r st dent r y t o t he name r ef er r ed t o byt he second dent r y, whi ch i s i n t he di r ect or y ref er ed t o byt he i node.

I f successf ul , i t shoul d cal l d_ i nst ant i at e t o l i nk t hei node of t he l i nked f i l e t o t he new dent r y ( whi ch was anegat i ve dent r y) .

unlink

Thi s shoul d r emove t he name r ef er ed t o by t he dent r y f r omt he di r ect or y r ef er r ed t o by t he i node. I t shoul d d_del et et he dent r y on success.

symlink

Thi s shoul d cr eat e a symbol i c l i nk i n t he gi ven di r ect or ywi t h t he gi ven name havi ng t he gi ven val ue. I t shoul dd_i nst ant i at e t he new i node i nt o t he dent r y on success.

mkdir

Cr eat e a di r ect ory wi t h t he gi ven parent , name, and mode.

rmdir

Remove the named di r ect ory ( i f empt y) and d_del et e thedent r y.

mknod


12/14



Cr eat e a devi ce speci al f i l e wi t h t he gi ven par ent , name,mode, and devi ce number . Then d_i nst ant i ate t he new i nodei nt o t he dent r y.

rename

The f i r st i node and ent r y r ef er t o a di r ect or y and namet hat exi st . r ename shoul d r ename t he obj ect t o have t hepar ent and name gi ven by t he second i node and dent r y. Al lgener i c checks, i ncl udi ng t hat t he new par ent i sn' t a chi l dof t he ol d name, have al r eady been done.

readlink

The symbol i c l i nk r ef er r ed t o by t he dent r y i s r ead and t heval ue i s copi ed i nt o t he user buf f er ( wi t h copy_t o_user )wi t h a maxi muml engt h gi ven by the i nt .

follow_link

I f we have a di r ect or y ( t he f i r st dent r y) and a name wi t hi nt hat di r ect or y ( t he second dent r y) t hen t he obvious r esul tof f ol l owi ng t he name f r om t he di r ect or y woul d ar r i ve att he second dent r y. I f an i node requi r es some ot her , non-obvi ous, r esul t - - as do symbol i c l i nks - - t he i node shoul d

pr ovi de a f ol l ow_l i nk met hod t o ret ur n t he appr opr i at e newdent r y. The i nt argument cont ai ns a number of LOOKUP f l agswhi ch are descr i bed i n t he sect i on on namei l ookups.

get_block

Thi s met hod i s used t o f i nd t he devi ce bl ock t hat hol ds agi ven bl ock of a f i l e. The i node and l ong i ndi cat e t he f i l eand bl ock number bei ng sought ( t he bl ock number i s t he f i l eof f set di vi ded by t he f i l e- syst em bl ock si ze) . get _bl ockshoul d i ni t i al i se t he b_dev and b_bl ocknr f i el ds of t he

buf f er _head, and shoul d possi bl y modi f y t he b_st at e f l ags.

I f t he i nt ar gument i s non- zero t hen a new bl ock shoul d beal l ocat ed i f one does not al r eady exi st .

readpage

Readpage i s onl y cal l ed by mm/ f i l emap. c I t i s cal l ed by:

t r y_to_read_ahead f r om gener i c_f i l e_readahead andf i l emap_nopage


13/14



do_gener i c_f i l e_r ead

sys_sendf i l e

f i l emap_nopage

gener i c_f i l e_mmap r equi r es i t t o be non- nul l .

Thus i t i s needed f or memor y mappi ng of f i l es ( as you woul dexpect ) , f or usi ng t he sendf i l e syst em cal l , or i f t hegener i c_r ead_f i l e i s t o be used f or t he f i l e: r ead met hod.

r eadpage i s not expect ed t o act ual l y read i n t he page. I tmust ar r ange f or t he r ead t o happen. Cl i ent s wai t f or t he

page t o be unl ocked bef or e usi ng the data.

r eadpage can be i mpl ement ed usi ng bl ock_r ead_f ul l _pagewhi ch i s def i ned i n f s/ buf f er . c. Thi s r out i ne assumes t hati node: get _bl ock has been def i ned and set s up a buf f er _headst o access t he bl ock i n quest i on. These buf f er _heads wi l l beset t o cal l ' end_buf f er _i o_async' on compl et i on, whi ch wi l lunl ock t he page when al l buf f ers on t he page compl ete.

writepage

Wr i t epage i s cal l ed f r om l i nux/ mm/ f i l emap. c t oo.

i t i s cal l ed by do_wr i t e_page f r om f i l emap_wr i t e_page, f r omf i l emap_swapout , f i l emap_sync_pt e, and f r omgener i c_f i l e_mmap.

Wr i t epage can be i mpl ement ed usi ng bl ock_wr i t e_f ul l _pagef r om f s/ buf f er . c. I t i s a cl ose t wi n of bl ock_r ead_f ul l page. The i mpor t ant di f f er ences bei ng:

bl ock_r ead_f ul l page i ni t i at es a r ead wi t h l l _r w_bl ock,

whi l e bl ock_wr i t e_f ul l page onl y set s up t he buf f er s, butdoesn' t i ni t i at e t he wr i t e.

bl ock_r ead_f ul l page cal l s i node: get _bl ock wi t h t he cr eat ef l ags set t o zer o, whi l e bl ock_wr i t e_f ul l page set s i t t oone, and

bl ock_r ead_f ul l page cal l s i ni t _buf f er t o getend_buf f er _i o_async cal l ed on compl et i on.


14/14



These t wo r out i nes coul d be cl eaned up a bi t so t hat t hesi mi l ar i t y and di f f er ences s t and out mor e.

flushpage

f l ushpage i s cal l ed f r ommm/ f i l emap. c and mm/ swap_st ate. c.

I n mm/ f i l emap. c i s cal l ed by t r uncat e_i node_pages t o makesur e no I / O i s pendi ng on a page bef or e the page i sr el eased. mm/ swap_st at e. c si mi l ar l y cal l s i t when a page i sbei ng r emoved f r om t he swap cache - - al l I / O must bef i ni shed.

Documents

Seminar Report on File System in Linux