Nove
LXC, Cgroups and Advanced Linux Container TechnologyLab
www.novel l .comNovell Training Services
AT T L I V E 2 0 1 2 L A S V E G A S
S U S 1 5
ll, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Proprietary StatementCopyright © 2012 Novell, Inc. All rights reserved.
Novell, Inc., has intellectual property rights relating to technology embodied in the product that is described in this document. In particular, and without limitation, these intellectual property rights may include one or more of the U.S. patents listed on the Novell Legal Patents Web page (http://www.novell.com/company/legal/patents/) and one or more additional patents or pending patent applications in the U.S. and in other countries.
No part of this publication may be reproduced, photocopied, stored on a retrieval system, or transmitted without the express written consent of the publisher.
Novell, Inc.404 Wyman Street, Suite 500Waltham, MA 02451U.S.A.www.novell.com
Novell TrademarksFor Novell trademarks, see the Novell Trademark and Service Mark list (http://www.novell.com/company/legal/trademarks/tmlist.html).
Third-Party MaterialsAll third-party trademarks are the property of their respective owners.
Software PiracyThroughout the world, unauthorized duplication of software is subject to bothcriminal and civil penalties.
If you know of illegal copying of software, contact your local Software Antipiracy Hotline. For the Hotline number for your area, access Novell’s World Wide Web page (http://www.novell.com) and look for the piracy page under “Programs.”Or, contact Novell’s anti-piracy headquarters in the U.S. at 800-PIRATES (747-2837) or 801-861-7101.
DisclaimerNovell, Inc., makes no representations or warranties with respect to the contents or use of this documentation, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose.
Further, Novell, Inc., reserves the right to revise this publication and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. Further, Novell, Inc., makes no representations or warranties with respect to any software, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. Further, Novell, Inc., reserves the right to make changes to any and all parts of Novell software, at any time, without any obligation to notify any person or entity of such changes.
Any products or technical information provided under this Agreement may besubject to U.S. export controls and the trade laws of other countries. You agree to comply with all export control regulations and to obtain any required licenses or classification to export, re-export or import deliverables. You agree not to export or re-export to entities on the current U.S. export exclusion lists or to any embargoed or terrorist countries as specified in the U.S. export laws. You agree to not use deliverables for prohibited nuclear, missile, or chemical biological weaponry end uses. See the Novell International Trade Services Web page (http://www.novell.com/info/exports/) for more information on exporting Novell software. Novell assumes no responsibility for your failure to obtain any necessary export approvals.
This Novell Training Manual is published solely to instruct students in the use of Novell networking software. Although third-party application software packages are used in Novell training courses, this is for demonstration purposes only and shall not constitute an endorsement of any of these software applications.
Further, Novell, Inc. does not represent itself as having any particular expertisein these application software packages and any use by students of the same shall be done at the student’s own risk.
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Contents
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
3
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
List of Figures
4 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to Croup
Section 1 Introduction to Croup
In this section you install and begin using croup tools.
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
5
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
1.1 Use Linux Control GroupsIn this exercise you install, enable, and use Linux control groups (cgroups).
Objectives:Task I: Install and Enable cgroups
Special Instructions and Notes:
(none)
Task I: Install and Enable cgroups1. Log in as the root user
2. Enter the following command to install the cgroups package(s)
rpm -q libcgroup1 || zypper in -y libcgroup1
3. Enter the following command to activate cgroups:
/etc/init.d/boot.cgroup start
4. Enter the following commands to enable cgroups to start at boot time:
chkconfig boot.cgroup on
5. Enter the following command to see that cgroups are enabled:
mount
You should see several entries with /sys/fs/cgroup/subsystem as the
mountpoint.
Task II: Administer cgroups Directly via the File SystemNote: The use of taskset simplifies the example as it binds the xterm calls to the first core.
You could use the cpuset subsystem to achieve the same effect, but that would require
additional commands. If you don't bind them to the same core, you don't see the effect in
top, as they will probably run on different cores, using all the available cpu time.
1. Open a terminal window and su to the root user account.
2. Create two cgroups by creating directories in the /sys/fs/cgroup/cpu
directory:
cd /sys/fs/cgroup/cpu
mkdir higherload lowerload
3. To view the files that were automatically created in the directories, enter
6 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to Croup
ls higherload
4. Set the values you want to use in the different groups:
echo 6 > higherload/cpu.shares
echo 4 > lowerload/cpu.shares
5. Start processes and assign them to one of the groups by entering
taskset c 0 xterm bg orange &
taskset c 0 xterm bg green &
This will open two xterm windows with different background colors.
6. In the orange xterm, enter
echo $$ > /sys/fs/cgroup/cpu/higherload/tasks
This will add the shell running within the xterm to the tasks within the
cpu/higherload cgroup.
7. In the green xterm, enter
echo $$ > /sys/fs/cgroup/cpu/lowerload/tasks
This will add the shell running within the xterm to the tasks within the
cpu/lowerload cgroup.
8. In the the terminal window, enter top
9. In each of the xterms, enter
md5sum /dev/urandom
In the output of top, you should see one md5sum process with 60% CPU load, the
other one with 40%.
10. View the content of one of the tasks files by entering
cat /sys/fs/cgroup/cpu/higherload/tasks
You should see two PIDs - the one that you added using the echo command, and
that of the md5sum process that was added to the list automatically because child
processes become part of the cgroup of their parent process.
11. In the green xterm window, put the md5sum process in the background and start
another one by pressing Ctrl+z and then entering:
bg
md5sum /dev/urandom
12. Watch the output of top - the two md5sum processes in the lowerload cgroup
should now each use 20% of the CPU load.
13. Close the green xterm window and watch the output of top - the remaining
md5sum process should now use close to 100% of the CPU load.
14. Close the orange xterm window.
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
7
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
15. Remove the cgroups by entering
rmdir /sys/fs/cgroup/cpu/{higher,lower}load
Task III: Administer cgroups via cg* CommandsIn this task, you will use the cpuset subsystem to pin the processes to one CPU/core instead
of the taskset command.
1. Open a terminal window and su to the root user account.
1. Create two cgroups, using the cgcreate command:
cgcreate g cpu,cpuset:higherload
cgcreate g cpu,cpuset:lowerload
This will create subdirectories in the cpu and cpuset subsystem directories.
2. Set the values you want to use, using the the following cgset command:
cgset r cpu.shares=6 r cpuset.cpus=0 higherload
cgset r cpu.shares=4 r cpuset.cpus=0 lowerload
3. The following is required for cpusets.cpus to work properly (see man cpuset)
cgset r cpuset.mems=0 higherload
cgset r cpuset.mems=0 lowerload
4. Start processes and assign them to one of the groups, using the following cgexec
commands:
cgexec g cpu,cpuset:higherload xterm bg orange &
cgexec g cpu,cpuset:lowerload xterm bg green &
5. In the the terminal window, enter top
6. In each of the xterms, enter
md5sum /dev/urandom
In the output of top, you should see one md5sum process with 60% CPU load, the
other one with 40%.
7. Close the xterm windows.
In the terminal window, enter cgclear to remove all cgroups.
(End of Exercise)
8 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to Croup
1.2 Configure /etc/cgconfig.confIn this exercise you configure cgroups so specific groups get created when the system boots. Then you modify a start script so the daemon is assigned to a cgroup upon startup.
Objectives:Task I: Create the /etc/cgconfig.conf FileTask II: Modify the /etc/init.d/apache2 File
Special Instructions and Notes:
(none)
Task I: Create the /etc/cgconfig.conf File1. Log in as the root user and open a terminal window.
2. Copy the /usr/share/doc/packages/libcgroup1/cgconfig.conf
file to /etc:
cp /usr/share/doc/packages/libcgroup1/cgconfig.conf
/etc
3. Open the /etc/cgconfig.conf file in an editor of your choice, remove the
comment characters in front of the group section, and change the GIDs so the
content of the file looks similar to the following:
group daemons/www { perm { task { uid = root; gid = www; } admin { uid = root; gid = root; } } cpu { cpu.shares = 1000; } } group daemons/ftp { perm { task { uid = root; gid = ftp;
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
9
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
} admin { uid = root; gid = root; } } cpu { cpu.shares = 500; } } # #mount { # cpu = /mnt/cgroups/cpu; # cpuacct = /mnt/cgroups/cpuacct; #}
4. Start the /etc/init.d/cgconfig script and make sure it is executed when
the system boots:
/etc/init.d/cgconfig start
chkconfig cgconfig on
5. View the entries in /sys/fs/cgroups/cpu/ that were created based on the
above configuration, using ls and cat.
Task II: Modify the /etc/init.d/apache2 FileIn this task, you modify the /etc/init.d/apache2 file so that the Apache processes are automatically assigned to the daemons/www cgroup that you created in Task I.
1. Log in as the root user and open a terminal window.
2. Modify the Apache2 start script to assign the Apache process to the daemons/www
cgroup you created in Task I:
Search for the line within the start section of the case statement that contains the
startproc command and add
cgexec g cpu:daemons/www
to the line so it looks like the following (in one line):
eval cgexec -g cpu:daemons/www startproc -f -t ${APACHE_START_TIMEOUT:-2} $cmdline
Save the file and close the editor.
3. Start Apache by entering
rcapache2 start
4. Check if processes were added to the respective task file, using the following
command:
cat /sys/fs/cgroup/cpu/daemons/www/tasks
10 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to Croup
You should see a list of PIDs
5. Enter the following command to view the PIDs of the Apache processes
ps aux | grep apache
Compare the PIDs in the output of ps with those in the tasks file. They should be
the same.
(End of Exercise)
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
11
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
Section 2 Introduction to LXC
In this section you being using LXC.
12 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to LXC
2.1 Create a Simple LXC ContainerIn this exercise you create a simple LXC container using a template.
Objectives:Task I: Install LXCTask II: Create a Basic Configuration File for the ContainerTask III: Create the LXC ContainerTask IV: Enable root Logins to the New ContainerTask V: Test the New Container
Special Instructions and Notes:
A network bridge name br0 must exist before performing this exercise
Task I: Install LXC1. As the root user, use zypper to search for the "lxc" package to install:
zypper se lxc
You should see the "lxc" package available.
2. Assuming it is not installed, use zypper to install the "lxc" package:
zypper in lxc
Task II: Create a Basic Configuration File for the Container
To create an LXC container you must have a basic configuration file that defines the network configuration for the container.
1. As the root user, in the text editor of your choice, create and open the /root/basic-
sles.conf file to be edited
2. Add the following lines to the file:
lxc.network.type=veth
lxc.network.link=br0
lxc.network.flags=up
3. Save the file and close the text editor
Task III: Create the LXC ContainerBefore starting a container the container must be created. This can be done by running the lxc-create command and referencing a template. the template will create everything required for the container: config file, rootfs, etc.
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
13
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
1. As the root user, enter the following command to create the LXC container:
lxc-create -n basic-sles -f /root/basic-sles.conf -t
sles
When the command is finished running you should have a new container named
basic-sles in /var/lib/lxc/
2. Enter the following command to see that the container was created:
lxc-ls
The command should show a list of the existing container which in this case is
only basic-sles
3. Enter the following command to see the current state of the new container:
lxc-info basic-sles
The command should show that the container is state is STOPPED
Task IV: Enable root Logins to the New ContainerThe basic SLES container needs to have the root password set and the console added to the securetty file so that the root user can log in to the container.
1. To set the root password in the new container, enter the following commands:
chroot /var/lib/lxc/basic-sles/rootfs
passwd root
(enter password: linux)
exit
2. To allow the root user to log into the new container, In the text editor of your
choice, open the /var/lib/lxc/basic-sles/rootfs/etc/securetty file to be edited
3. Add the following line to the end of the file:
console
Save the file and close the text editor
Task V: Test the New Container1. Enter the following commands to ensure that the network bridge named br0 is up
and running:
brctl show
ip link show dev br0
You should see that the br0 bridge is created and up
Note: If br0 doesn't exist, enter the following commands to create it and bring it up:
brctl addbr br0
14 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to LXC
ip link set up dev br0
Re-run the brctl show and ip link show commands to verify it worked
2. Enter the following command to start the new container:
lxc-start -n basic-sles
You should see the boot messages while the container starts and then be placed at a
login prompt
3. Log into the container with the following credentials:
Username: root
Password: linux
You should be at shell prompt as root in the new container
4. In another terminal window, enter the following command to see the current state
of the new container:
lxc-info -n basic-sles
You should see that the container's state is RUNNING
5. Close the terminal window that you launched and are currently logged into the
container in
6. Enter the followng to view the current state of the container:
lxc-info -n basic-sles
You should see that the container is still running even though you close the
terminal that launched it
7. Open another terminal window and enter the following command to connect to the
console of the running container:
lxc-console -n basic-sles
You should be at a login prompt of the container
8. In another terminal window, enter the following command to shutdown the
container:
lxc-stop -n basic-sles
9. Enter the following command to see the current state of the container:
lxc-info -n basic-sles
You should see that the current state of the container is now STOPPED
(End of Exercise)
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
15
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
2.2 Mirror a System in LXCIn this exercise you using the lxc-jailbird.sh script to mirror an existing systrem into and LXC conainter.
Objectives:Task I: TitleTask II: TitleTask III: Title
Special Instructrisions and Notes:
You will need to obain the lxc-jailbird.sh script from the instructor.The script may already have been added to your virtual machine environment: check /root/bin to see if is there.
Task I: Prepare the lxc-jailbird.sh ScriptIn this task you prepare the script for execution.
1. After you have located the lxc-jailbird.sh script, copy it to /root/bin.
2. Make its permissions executable.
Task II: Prepare an LXC Container using the lxc-jailbird.sh script
1. Using a name of your own chosing, create a container with lxc-jailbird.sh:
cd /var/lib/lxc
lxc-jailbird.sh YOURNAMEHERE
• field1: value
• field2: value
2. Start the script using the methods you learned from previous labs and explore the
container you created.
3. Stop your container and sync the elements necessary for a full system.
4. This will likely take some trial and error but the instructor has additional tools to
do this if you would like some help.
16 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Introduction to LXC
After completion of this exercise you will have created, stopped, started,and explored LXC containers.
(End of Exercise)
Version 1 Copying all or part of this manual, or distributing such copies, is strictlyprohibited. To report suspected copying, please call 1-800-PIRATES
17
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.
Cgroups and LXC
18 Copying all or part of this manual, or distributing such copies, is strictly prohibited. To report suspected copying, please call 1-800-PIRATES
Version 1
Novell, Inc. Copyright 2012-ATT LIVE-1-HARDCOPY PERMITTED. NO OTHER PRINTING, COPYING, OR DISTRIBUTION ALLOWED.