Boot Linux Faster

8/12/2019 Boot Linux Faster

1/45

Boot Linux Faster

USTC

Fengguang Wu

AKA 2006 Linux kernel conference Sep 16, 2006


2/45

WE NEED FAST BOOT

everybody likesit

everybody cares

we are put to shameby Windows with Preetcher !ac "S # with Boot!ache


3/45

the e"pert way

less services

less fonts

less features

light weight GUI### false&&depmod -a

keep ...slim & fluent


4/45

the hacker way

kernel init=/bin/sh


5/45

the shortcut

suspend

$

resu%e


6/45

the comprehe$sive way

he&, let's do all possi(le

to ensure

fast (oot with all the goodies)


7/45

Embedded Systems

*+ Linux Foru%

http--treecelinuxforu%org-pu(wiki-%oincgi-.ootup/i%eesourceshttp--treecelinuxforu%org-pu(wiki-%oincgi-.ootup/i%eeductionowto

"LS2006

%pro3ing Linux Startup /i%e 4sing Software esu%e 5and other techniues7https--ols2006108redhatco%-reprints-ka%inaga9reprintpdf

Linux .ootup /i%e eduction for :igital Still *a%erahttps--ols2006108redhatco%-reprints-park9reprintpdf
http://tree.celinuxforum.org/pubwiki/moin.cgi/BootupTimeResourceshttp://tree.celinuxforum.org/pubwiki/moin.cgi/BootupTimeReductionHowtohttps://ols2006.108.redhat.com/reprints/kaminaga-reprint.pdfhttps://ols2006.108.redhat.com/reprints/park-reprint.pdfhttps://ols2006.108.redhat.com/reprints/park-reprint.pdfhttps://ols2006.108.redhat.com/reprints/kaminaga-reprint.pdfhttp://tree.celinuxforum.org/pubwiki/moin.cgi/BootupTimeReductionHowtohttp://tree.celinuxforum.org/pubwiki/moin.cgi/BootupTimeResources


8/45

%a&or 'oadblocks

()O seek re$*y 1k ; 10k files on startup

?0s

bu++y apps sill& spins - sleeps

:+ pro(e dela&s @ up to s :a3e Bones Wh& 4serspace Sucks

!P, ho+s
https://ols2006.108.redhat.com/reprints/jones-reprint.pdfhttps://ols2006.108.redhat.com/reprints/jones-reprint.pdf


9/45

-e$eral Solutio$s

scheme

bootchart

prelink

parallelization

preload(focus)

defrag

helps

Analyze

CPU

CPU I! idles

I! wait seeks

I! seeks


10/45

.er$el Tricks

kernel options kernelquietide3=noprobe

mount -o noatime kexecfor fast reboot

run a new kernel instantly

parallel device initialization ongoing work (& Creg K


11/45

kexec mini HOWTO

ker$el CONFIG_KEXEC=y

i$stallkexec-tools

hack/etc/init.d/reboot# try kexec, then fall back to reboot /sbin/kexec -e/sbin/reboot -d -f -i

load a ker$el a$d reboot # kexec -l /vmlinuz \ --append="$(


12/45

Preli$ki$+

idea prelink5%odif&7+LF li(s and apps

to speed up d&na%ic linking

be$eits *DD apps linking to %an& li(s

CE"!+ - K:+

load ti%e notrun ti%e

*4 (ound not-" (ound


13/45

Paralleli*atio$

"3erlapping execution with -"

"ost distributions on the way #U#$ already there

LS. initserv/chkconfigtools

Ubuntu shows the future upstart


14/45


15/45

,bu$tu ,pstart

eve$t based compo$e$ts

initGo(s

e3ents

+oal to replace init,

and cron, inetd, job life-cycle


16/45

Preload

startup is ()O bou$d +oal

reduce -" wait (etter -" utiliHation

steps

collect -" trace preload files

defrag files


17/45

Distributio$ Solutio$s

debia$ preload

ubu$tu readahead +e$too

readahead9list9earl& readahead9list

suse (ootpreloadIearl&

(ootpreload

earl&kd%

edora

readaheadIearl& readahead

xd% preload


18/45

'eports

preload#s#$et 6


19/45

(deal Preload

co"plete in background

read in order

ando%3s

Sorted


20/45

cacheby %ens A&boe

prime mode %irror read data to cache de3ice

$ormal mode ser3e data fro% cache de3ice merits a$d demerits

turn rando% accesses to linear ones perfect la&out, opti%al -"

not uite linux wa&


21/45

the Flavours

fcache

perfect

kernel solution

specialized

weird

for enduser


22/45

the Flavours

fcache

perfect

kernel solution

specialized

weird

for enduser

filecache

good enough

kernel'userland

general purpose

easy

deploy"ent


23/45

Proposed Solutio$

()O trace /proc/filecacheinterface and tools

preload reuest for allfiles ASAP, so that -"scheduler knows the +lobal picture

schedule the bulkreuests in kernel

data layout poor %an's defragger for ext


24/45

!ollecti$+ ()O Traces

strace

*P+$!A

proc,pid-"ap

fboot

(kernel "odule)

#U#$

gentoo.wiki/co"

preload/sf/net

Andrew 0orton


25/45

stracePractice

To trace KDE startup initiated from KDM, we

have to strace two kdmprocesses and the X

process (PDs !"#!, $%## and &'"!)

# strace-f-F\

-p3703 -p4100 -p9573 \ -e trace=open\-ostrace.kdm

/proc/filecache


26/45

/proc/filecachei$odes view

# echo-nindex> /proc/filecache# cat/proc/filecache

# filecache 1.0

# ino size cached cached% state refcnt dev file 472353 8 0 0 -- 29 03:42(hdb2) /lib1205314 91 92 100 -- 65 03:42(hdb2) /lib/ld-2.3.6.so 16289 8 0 0 -- 38 03:42(hdb2) /etc 472394 4 0 0 -- 30 03:42(hdb2) /lib/tls 233608 1242 1056 85 -- 65 03:42(hdb2)/lib/tls/libc-2.3.6.so 65203 651 496 76 -- 2 03:42(hdb2) /bin/bash1205315 261 160 61 -- 10 03:42(hdb2)/lib/libncurses.so.5.5

/proc/filecache


27/45

/proc/filecachepa+es view

# echo -n /bin/bash> /proc/filecache# cat/proc/filecache

# file /bin/bash

# flags R:referenced A:active U:uptodate D:dirtyW:writeback M:mmap# idx len state refcnt0 46 RAU___ 147 12 RAU___ 160 13 RAU___ 173 4 __U___ 177 4 RAU___ 181 2 __U___ 183 6 RAU___ 1


28/45

filecache%erits

con3enient no o3erhead

shows cache usage shows fs %etadata -" trace for an& task

takesnapshot0; run task;takesnapshot1

diffsnapshot1 snapshot0


29/45

Preload Steps

setup queue parameters

for each fs:

readaheadfs metadatawait for fs mountreadaheadfiles in parallel

wait for IO completerestore queue parameters


30/45

()O Schedule

read0ahead re1uests IOPRIO_CLASS_IDLE ser3ed on disk idle or in (atch

need fix deadline and anticipator&

o$ pe$di$+ read

find the read9ahead reuest ueue it for su(%ission

need fix all ele3ators


31/45

e"t2 data layout

allocatio$ +roup fs di3ided into eual siHed groups

default to 80x128! for a 10C fs

allocate strate+y 3orlov4 spread out top9le3el directories

ensure localit& for nor%al inodes %akes un(alanced use of groups

dumpe2fs/dev/hda2 | grep free


32/45

e"t2 top0level dirs

co$cept directories on the fs root

orchattr -Tdir

orlov find a %oderatespare group

oldalloc find the %ostspare group 5for all dirs7


33/45


34/45

Put it To+ether

eleme$tary tools /proc/filecache snapshot

#filecache --snapshot --dump ./fc # ls./fc

bdev hdb2 hdb3

readahead files in parallel

#readahead-fs ./fc/hdb2


35/45

Put it To+ether

ro$t0e$d tool collect -" trace

#bootcache startfirefox $firefox & $sleep 10s #bootcache stopfirefox

preload files

#bootcache preloadfirefox

$firefox &


36/45

/et5s +o###

:e(ian


37/45

lock co$te$tio$

$ ps-C readahead-fs m -o \ pid,tid,class,pri,pcpu,stat,wchan:14,comm

PID TID PRI %CPU STAT WCHAN COMMAND

9413 - - 1.1 - - readahead-fs - 9413 24 0.1 Sl+ futex_wait - - 9414 21 0.0 Dl+ sync_buffer - - 9415 21 0.0 Dl+ real_lookup - - 9416 21 0.0 Dl+ real_lookup -

- 9417 21 0.0 Dl+ sync_buffer - - 9418 21 0.0 Dl+ real_lookup - - 9419 21 0.0 Dl+ real_lookup -...

l k t ti


38/45

lock co$te$tio$

[] sync_buffer+0x60/0x77[] __wait_on_bit+0x58/0x61[] out_of_line_wait_on_bit+0x80/0x88[]__wait_on_buffer+0x31/0x33


39/45

iope$

problem call seuence open()O read() stuck in open(), (efore reuesting -" can't tell -" ele3ator the glo(al picture

ope$ by i$ode

ext inode nu%(er >> offset inside fs no path lookup

no lockup

S,SE B t h t


40/45

S,SE Bootcharts plainplain

preloadpreload

filecachefilecache

! l i


41/45

!o$clusio$

startup ti"e re"ains the sa"e

still suffers fro" a lock contention lea3e as future work

seek re$*ies reduced

1$ startup is now CPU bound ti%e to show off &our dual core *4 O97

'll go for prelink^_^


42/45

Thank you.


43/45

O$+oi$+ Pro&ects


44/45

O$+oi$+ Pro&ects

Coogle Su%%er of *ode 2006

%pro3e the :e(ian .oot rocesshttp--initscripts9ngaliothde(ianorg-

apid linux desktop startup through pre9cachinghttp--codegoogleco%-p-pagecache9tools-

upstart init dae%on replace%enthttp--wwwnetsplitco%-(log-work-canonical-upstartht%l

'esources
http://initscripts-ng.alioth.debian.org/http://code.google.com/p/pagecache-tools/http://www.netsplit.com/blog/work/canonical/upstart.htmlhttp://www.netsplit.com/blog/work/canonical/upstart.htmlhttp://code.google.com/p/pagecache-tools/http://initscripts-ng.alioth.debian.org/


45/45

'esources(ootchart .oot rocess erfor%ance JisualiHation

http--www(ootchartorg

Anal&Hing and %pro3ing CE"!+ Startup /i%ehttp--wwwgno%eorg-;lcolitti-gno%e9startup-anal&sis-

10 /hings Apple :id /o !ake !ac "S # Fasterhttp--wwwkernelthreadco%-%ac-ap%e-opti%iHations-

kernel facilities for cache prefetchinghttp--%arctheai%sgroupco%-Nt>11?62

Linux .oot /i%e Speedups /hrough recachinghttp--kerneltraporg-node-21

Documents

Boot Linux Faster