Workshop on Free/Open Source Software

8/14/2019 Workshop on Free/Open Source Software

1/38

Welcome

Workshop on Free/Open Source Software


2/38

We are Linux


3/38


4/38

Introduction to FOSS

Why F/OSS?

Who Uses F/OSS?


5/38

Why F/OSS

Freedom to Use For any Purpose

Freedom to Look under the hood

Freedom to Modify Freedom To Distribute

Freedom to Profit

Security & Reliability Believe it or not, it runs our life.


6/38

F/OSS Development

Release Early, Release Often

Reuse Existing Components

Better Quality Control Less Total Cost of Ownership


7/38

Release Early, Release Often

Incremental features do not wait for Years

Quick feedback of any, even smallmodifications

Incremental Bug Fixing


8/38

Reuse Existing Components

Open License Promotes Sharing code andIdeas

Do not reinvent the wheel

Multiple point of testing, so fewer bugs


9/38

Less TCO

For an Individual

No Upfront Cost of Operating System

(Mostly) all Applications Available with Linuxare F/OSS

(in General) Very Secure, so Less Money,Time and effort needed

Good Support for Old Hardware


10/38

Choice

Why do we need choice?

What choices are available for F/OSS

Operating System Active distributions : 312 (distrowatch)

Applications Office, Entertainment, Graphics

Development Editors, IDE, SDKs


11/38

Success Stories

All Startup Companies

All Large Enterprises

Google, Yahoo, Facebook Your Computer Lab

Your Operating Systems Handbook


12/38

Linux Boot Sequence

512 Bytes


13/38

Boot Loaders

Lilo

Grub

Uboot (Embedded Systems) Boot monitors (acts both as 1st and 2nd Stage)

Also can download kernel to the target

Performs error checking and initialization Dissecting MBR

# dd if=/dev/hda of=mbr.bin bs=512 count=1

# od -xa mbr.bin


14/38

Anatomy Of BootLoader

Stage 2 Bootloader loadedfrom active partition


15/38

Kernel

Compressed image

At the head a small program that Performssome minimal hardware setup

(./arch/i386/boot/head.S)

Loads kernel and ramdisk to memory anddecompress the kernel

[ decompress_kernel (located in./arch/i386/boot/compressed/misc.c) ]

yet another startup_32 function, but this function is in ./arch/i386/kernel/head.S.


16/38

Kernel Start


17/38

Partition Table

Can contain only 4 partitions

Extended partitions

FileSystem Types FAT/NTFS

Ext

JFS/XFS/ZFS/ReiserFS

More at http://tldp.org/HOWTO/Filesystems-HOWTO.html


18/38

Food for thought

What is journaling?

Rationale

Optimizations Physical

Logical


19/38

FileSystems

What?

Blocksize

Ext2/3/4

FAT/NTFS


20/38

ext2

BLOCKDecided when FS is Created 1,2,3,4,8K size

Smaller blocks, less wastage of spacebut more accounting overhead

BLOCK GROUPSCluster of Blocks To reduce Fragmentation

Stored immediately after Super BlocksFirst 2 Blocks reserved for bitmap and

Inode Information, size of bitmap=1blockSize, And next comes inode data, and

then data

Super BlockInformation about FS Configuration,

first stored in 1024bytes offset,

And then in the beginning of all blockgroups(later revised), also stores information aboutInodes, blocks, free inodes etc..

Inodes & bitmaps?Inode contains links to data block,

and other Information other than name.

Has pointer to 12 Blocks of data, andanother pointer to indirect block (which inturn contains address of other Blocks),

And and doubly indirect blocks(?)


21/38

What is Bitmap?

A bitmap is simply an array of bits, with theNth bit indicating whether the Nth block isallocated or free

Advantages Simple

Disadvantages

Scalable?

4B block, 1GB FS => 32KB Bitmap, 1TB=>?,1PB=>?


22/38

Ext2 diagram


23/38

Ext3

Journaling

Writeback mode No journaling of data, only meta-data

Ordered mode Logically groups meta-data and and data-

blocks as a single group of transaction, onwrite data-blocks are written first

Journal mode Slowest to write, replay of journal

Fastest to read


24/38

Ext4

Delayed allocation and more...


25/38

FAT File Systems

structdirectory { // Short 8.3 namesunsignedcharname[8]; // file nameunsignedcharext[3]; // file extensionunsignedcharattr; // attribute byteunsignedcharlcase; // Case for base and extension

unsignedcharctime_ms; // Creation time, milliseconds unsignedcharctime[2]; // Creation time unsignedcharcdate[2]; // Creation date unsignedcharadate[2]; // Last access date unsignedcharreserved[2]; // reserved values (ignored)

unsignedchartime[2]; // time stamp

unsignedchardate[2]; // date stampunsignedcharstart[2]; // starting cluster numberunsignedcharsize[4]; // size of the file

};


26/38

Filesystem Hierarchy Standard A Standard for Unix Like Operating Systems

/ /bin : Essential user command binaries (for use by all users)

/boot : Static files of the boot loader

/dev : Device files

/etc : Host-specific system configuration

/home : User home directories (optional)

/lib : Essential shared libraries and kernel modules

/media : Mount point for removeable media

/mnt : Mount point for a temporarily mounted filesystem

/root : Home directory for the root user (optional)

/sbin : System binaries

/srv : Data for services provided by this system

/tmp : Temporary files


27/38

Shell Scripting

Commands as steps

Variables

Variable interpolation Loops and Conditions


28/38

Shebang line

Script permissions

Sed AWK

Grep

Find


29/38

globbing

filename matching, doesn't match /

man glob

uses special characters such as * and ?

* == any or no characters

n* == any filename starting with an n followedby 0 or more other characters

? == any single character ls /tmp/intermediate_bash/?on.txt

note that on.txt is not matched


30/38

regular expressions

uses special characters such as *, . ,+, ?, |

globbing evaluated by shell before regex getsevaluated

regex are used by tools such as grep and sed

use quotes to protect regex pattern

two types of regular expressions

extended

basic


31/38

extended/modern

* == 0 or more of the preceding character

+ == one or more of the preceding character

? == zero or one of the preceding character

basic/obsolete

+, ?, | are not special characters

need to escape characters such as { and( to use them as special characters


32/38

arrays

${array[0]}

must use curly brackets when specifying the variable

0 is the first element of an array,but not all elements have to havevalues

assigning an element deletes the old element content or createsthe element

compound assignments wipe out previous array contents

${array[*]} lists all elements of the array as a single value

${array[@]} lists all elements of the array as seperate values

${#array[$i]} is the length of the value assigned to element $i

${#array[@]} gives the number of items in the array

${!array[@]} lists the indexes that have values


33/38

shell math

use let or arithmetic evaluation syntax

let j="i + 4"; echo $j

j=$(( i + 4 )); echo $j

they don't need the $ to denote a variable, butlet needs quotes to use spaces or specialcharacters

for i in {1..4}; do j=$(( j + i )); done; echo $j


34/38

command substitution

$( echo pwd )

don't use backticks unless you need /bin/shcompatability

`echo pwd`

$() syntax allows nesting, fixes quoting issues

j=$( ls -d $( pwd ) ); echo $j


35/38

Proc FileSystem

Mounted on /proc

proc on /proc type proc (rw)

/proc/cpuinfo Gives CPU Info

/proc/meminfo Gives Memory Info

/proc/uptime Individual Numbered directories are for

processes


36/38

Inside process directory

Cmdline

The command with which this process wasstarted

Status Name of the program, status, Virtual Memory

size, threads etc..

More at http://www.redhat.com/docs/manuals/linux/RH

L-7.2-Manual/ref-guide/ch-proc.html


37/38

Exercises

List top 10 directories (in terms of size in/home folder)

Hint Du -hs gives the total size occupied by the

folder

Find all xml files in a given directory andchange all occurance of ...

to ... Hint

find and then -exec sed


38/38

Project

A system monitor/Task Manager

Hints /proc

Assume Only 15 processes

Documents

Workshop on Free/Open Source Software