COMP075 OS2

Managing Linux Services

Services?

• Services are processes that start, usually when the system starts, and run, usually, until the system stops

• These processes often provide a service of some sort to other processes through a client-server relationship and some form of IPC (Inter Process Communication)

• Some, like the components of the network protocol stack, and the X windows system perform functions for the system as a whole.

Starting ServicesMany Different Approaches

• Unix System 5 (SysV), introduced in the mid 1980's, introduced a method of starting services called SysV init

• Until recently this was the dominant architecture used in most linux variants

• Others were used less widely

• Systemd is a new emerging standard method that is slowly being adopted by the major linux distributions

• We will look at both systems

Starting Services (SysV init)

• Services can be started manually by running their associated init script with an appropriate parameter

• But given the number of needed services, this needs to be automated

• The run levels concept makes automatic service initialization easier to configure, and a little bit flexible

• The init process starts first and then starts init scripts according to the default run level

Run Levels• A run level is basically a directory containing softlinks

into a collection of scripts

/etc/init.d/rc3.d or /etc/rc3.d for run level 3

• The linked scripts start or stop services for that run level

• The init process runs the scripts when that run level is 'entered' (or 'left')

• Names of scripts cause services to start or stop in a specified sequence

– K03dbus

– S09ntp

What Run Levels?

• Only really standard levels are 0, 1 and 6

– Halt, Single User and Restart

• Many systems also use

– 2 multi-user

– 3 multi-user with networking

– 5 full GUI system

• But it depends on the way the system is configured

– In Kali and Debian the default level, 2, is a full GUI system

Start/Stop (init) Scripts

• init scripts are bash scripts that use a case statement to start or stop a service depending on a parameter (verb) passed to the script

• Most can handle start, stop and status

• Usually contained in /etc/init.d

/etc/init.d/postgresql status

– Gets the status of the postgresql service

• Many scripts can also handle restart

• /etc/init.d/rc3.d or /etc/rc3.d will contain softlinks to scripts in /etc/init.d

Linux Standard Base init Scripts

• According to the LSB, init scripts should always handle start, stop, restart, force-reload, and status

– Restart starts if not running already

– Force reload restarts if service doesn't understand reload

• Optionally reload and try-restart

– Try-restart only restarts if service already running

• Scripts should behave reasonable in situations like starting a service that is already running

Configuring Run Levels

• Services can be added to a run level by creating softlinks to the service's init script in the /init.d/rcX.d/ directory for run level X

• This is not easy: must follow naming conventions

– K03dbus• Kills dbus

– S09ntp• Starts ntp

– Numbers (03 09) force scripts to run in a predetermined sequence

AutomaticallyConfiguring Run Levels

• When packages or entire linux systems are installed:

– init scripts are written to /etc/init.d and

– softlinks to them are placed in the /etc/init.d/rcX.d directory

• This configuration is based on standard init info comments placed in the init scripts by the service developers

init info Comments

• Near the start of the script

• Delimited by the lines:

### BEGIN INIT INFO

### END INIT INFO

• Formatted comment lines then provide the init information

– What run levels to start in

– What sequence to start in

Standard init info Dependency Comments

• Service start sequence is based on dependencies between services specified by:

Provides: service-name …

Required-Start: service-name …

– Must be available during start up

Required-Stop: service-name …

– Must be available during shut down

Should-Start: service-name …

Should-Stop: service-name …

Processing Dependency Comments

• Dependency information has no effect at run time

– Is not checked

• Information is used to sequence the execution of the scripts by devising appropriate names for the softlinks in the run level directories during installation

/etc/init.d/rc3.d/S02network

– Starts the network before starting

/etc/init.d/rc3.d/S03syslog

Run level init comments

• init info comments are used to specify the standard run levels to install the service in

Default-Start: run level 1 [run level 2...] Default-Stop: run level 1 [run level 2...]

• Indicate run levels to start or stop the service

• There are also

• Short-Description

– Limited to a single line of text.

• Description

– Multi-line description

Manually Configuring Run Levels• Creating the appropriate softlinks is not meant to

be done manually

• chkconfig command allows for manual installation of a service in a run level

chkconfig

– Lists services with off/on status

-l shows each run level

service 35 sets service to start in levels 3 and 5

service on starts service in default levels

• Most distributions also have a gui for this purpose

Configuring with update-rc.d

• On kali the command update-rc.d performs functions like chkconfig (or you can install chkconfig)

• Only works if the service has not already been installed

update-rc.d remove|defaults

update-rc.d enable|disable S|2|3|4|5

The init Program

• First program run at system start up is called init

• It is responsible for starting services based on run levels

• Runs the init scripts pointed to by softlinks in /etc/init.d/rcX.d (or /etc/rcX.d)

• Controlled by /etc/inittab

• Is the parent process of all other processes

inittab

• Contains the default run level

id:3:initdefault:

• Line format is

id:runlevel(s):action:process

• id is somewhat arbitrary but conventional

• Action is sometimes an action type, sometimes an event that will trigger an action

• Process is usually a script to run

inittab examples

• l3:3:wait:/etc/init.d/rc 3

– When runlevel 3 is reached runs rc script with parameter '3' and waits for completion

• ca::ctrlaltdel:/sbin/shutdown -r -t 4 now

– Runs shutdown command when Ctrl-Alt-Del is pressed

• 5:2345:respawn:/sbin/mingetty tty5

– Starts terminal on console 5 (/dev/tty5) in runlevels 2345 and respawns if it stops

Problems with init and Run Levels• Serial execution

– Services don't start in parallel making for longer boot time

• Large complicated init scripts

• Run level idea has limited flexibility

• Doesn't respect dependencies at init time

– Only when services are installed are dependencies consulted and then only to determine startup sequence

• Only invoked at startup (or manually)

– Limited ability to respond to events

Replacements for init

• Lots of not very widely used replacements have been tried

• Some are working:

– Service Management Facility

• Solaris 10 and on

– Launchd

• Mac OS X v10.4

– Upstart

• Ubuntu

– Systemd

• Suse 12.1 kali 2.0 and others

systemd

• Well defined dependencies

• Dependencies allow for safe parallel starting of daemons

• Avoids overhead of bash scripts

– Init requirements specified in declarative config files

• Unix domain sockets and D-Bus for IPC

– Allows closer monitoring of processes

• Aims to replace a lot of linux components

– cron, syslog, atd, udev …

– Some say too many

d-bus

• An IPC mechanism

• Includes a system daemon and per user daemons

• User daemons enable desktop applications to communicate conveniently

• System daemon can be used to ease interaction with processes such as system services

• d-bus presents an object interface to senders and receivers using objects located in the daemons

What does systemd do?

• mounting your disks (checking for errors if needed),

• starting the firewall and other security tools,

• starting services

• connecting the network and networked filesystems

• starting the graphical system and the login manager which brings you to your desktop

Targets and Units

• Instead of run levels there are targets

– When a target is started systemd starts everything the target needs

– Old style run levels are represented as targets in systemd

• Instead of services there are units

– A more general concept

– Allows services to be dependent on hardware units for example

– Targets are a type of unit

Types of Units

• Config file names are name.type

– service

– socket

– target

– device

– mount

– automount

– timer

– others

Unit Files Locations

• /lib/systemd/system

– Installed packages

• /run/systemd/system

– Created from templates at runtime

• /etc/systemd/system

– Local configs, override the above

• These unit files are used in system mode

– Like at startup

• Other files are used in user mode

User Unit Paths

• /usr/lib/systemd/user

• $HOME/.local/share/systemd/user

• /run/systemd/user

• /etc/systemd/user

• $HOME/.config/systemd/user

• Some others

Unit File Contents

• Many options

• Description=

• Documentation=

• Requires=

• RequiresOverridable=

• Before=, After=

• OnFailure=

Using sytemd

• Changing runlevel

– systemctl isolate runlevel5.target

– isolate means stop everything else

• Current runlevel

– systemctl list-units –type=target

– Only if your system uses targets to emulate run levels

• systemctl start sshd

– Starts sshd service

Using sytemd

• systemctl status sshd.service

• service command will still work but transfers request to systemctl

– service sshd status

• systemctl disable mysql.service

• systemctl enable mysql.service

• chkconfig still works in Suse but transfers to systemctl

• systemctl reboot/halt/poweroff/...

Compatibility

• Your system may have removed the SysVinit files from /etc/init.d

– If so you can't run these scripts directly

– But you shouldn't anyway

– Use of service will invoke systemctl for you

• If you do have these scripts, the dependency comments matter and will be respected by systemd

• And the init scripts have to follow standards

Compatibility

• runlevel command may produce odd results

hermes:/home/cmb # runlevel

N 3

– Actually running in level 5

• chkconfig may not work accurately

– OK in Suse

• But default levels 2 3 and 4 are identical

Some Fun

hermes:/home/cmb # systemd-analyze time

Startup finished in 10114ms (kernel) + 52972ms (userspace) = 63087ms

hermes:/home/cmb # systemd-analyze blame

16171ms ntp.service

13286ms metasploit.service

6402ms systemd-modules-load.service

3386ms ossec.service

….

References

• Man pages for init, inittab, chkconfig, service

• Man pages for systemd system

– http://www.freedesktop.org/software/systemd/man/

• An intro

– http://www.freedesktop.org/wiki/Software/systemd/

• Init script

– http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/tocsysinit.html

• D-bus– http://www.freedesktop.org/wiki/Software/dbus/