7/27/2019 eucaimp3

1/11

http://cs.worcester.edu/wiki/index.php?n=Main.EucalyptusInstallation

Eucalyptus Installation

Overview

These are the steps that were taken to create a functional Eucalyptus 3.1 system. Theses

intstructions are for CentOS 6.1. They may or may not work on other distributions.

(Definitely won't work for any non-RedHat OS)

HardwareThis guide assumes that you have 2 or more computers to use in order to build a

Eucalyptus cloud. These computers must support hardware virtualization (Read the

"Prepare System" section for more info). This guide will be documenting how to set up a

Eucalyptus cloud using the MANAGED networking mode. In our case, this required the

machine that is running the Cluster Controller (CC) to have a second Network Interface

Card (NIC) so that all Node Controllers (NCs) were on their own subnet.

Prepare System

We need to do a few things to make sure that our system is ready for Eucalyptus to be

installed.

Update the BIOS is one of the things we needed to do, Here is how you can do it too:

Step 1: Downloading FreeDOS to use to boot the machine and run the BIOS .EXE

wgethttp://www.fdos.org/bootdisks/autogen/FDOEM.144.gz Then, gunzip FDOEM.144.gz

to unzip the file.

Step 2: Copy the BIOS flash utility and the BIOS image that needs to be used to upgrade,

and mount it to a floppy disk image. Here is what you do in order to complete this step:

modprobe vfat

modprobe loop

http://cs.worcester.edu/wiki/index.php?n=Main.EucalyptusInstallationhttp://cs.worcester.edu/wiki/index.php?n=Main.EucalyptusInstallationhttp://www.fdos.org/bootdisks/autogen/FDOEM.144.gzhttp://www.fdos.org/bootdisks/autogen/FDOEM.144.gzhttp://www.fdos.org/bootdisks/autogen/FDOEM.144.gzhttp://www.fdos.org/bootdisks/autogen/FDOEM.144.gzhttp://cs.worcester.edu/wiki/index.php?n=Main.EucalyptusInstallationhttp://cs.worcester.edu/wiki/index.php?n=Main.EucalyptusInstallation

7/27/2019 eucaimp3

2/11

mkdir /tmp/floppy

mount -t vfat -o loop FDOEM.144 /tmp/floppy

After mounting the Floppy you want to copy the EXE. that you downloaded for the BIOS:

cp DELLBIOSVERSION.exe /tmp/floppy (not actual name of the bios)

then unmount the floppy: umount /tmp/floppy

Step 3 is to burn the bootable CD which emulates a floppy device.

mkisofs -o bootcd.iso -b FDOEM.144 FDOEM.144 cdrecord -v bootcd.iso

Now you should have a bootable CD that can boot into FreeDOS where you can then runthe BIOS .EXE that you have put on it.

Disable SELinux

Eucalyptus does not work with SELinux (Security Enhanced Linux). Disable it! Open the

following file in your favorite text editor (I use vim):

vim /etc/selinux/config

Change the following line:

SELINUX=enforcing

To this:

SELINUX=permissive

Then run:

setenforce 0

Open Ports (or Disable Firewall)

You have a choice here. You can either open the following ports by editing iptables, or just

turn off the firewall. If you want to add firewall rules to allow traffic on those ports, the

easiest way is by running...system-config-firewall-tui

... and following the prompts to open the following ports on the machines you will be using

for the CLC/CC:

1. 8443

2. 8773

7/27/2019 eucaimp3

3/11

3. 8774

4. 9001

Open port 8775 on all Node Controllers.

Don't feel like opening ports? Select the option to disable the firewall in `system-config-

firewall-tui` and move on with your life.

Check for Hardware Virtualization

We are going to be using KVM (Kernel-based Virtual Machine) for our virtual machine

instances, as is this is what RedHat is supporting now rather than Xen. KVM requires that

your CPU have hardware virtualization support (Intel VT or AMD-V) in order to work.

To check for that, run the following command:

egrep '^flags.*(vmx|svm)' /proc/cpuinfo

If there is NO output then your CPU does NOT have hardware virtualization support, and

therefore cannot use KVM. If you're willing to sell your soul you may still be able to use

VMWare...

Sync with NTP Time Server

The system clocks need to synchronized across all machines. Install the Network Time

Protocol daemon for this. Install the ntp package if it is not already installed:

yum install -y ntp

Sync with time server:

ntpdate pool.ntp.org

Start the service:service ntpd start

Install Dependencies

Install Additional Repositories

Download the following .rpm package files More infohere.

wget http://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-

4.noarch.rpm

wget http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm

wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-

6-5.noarch.rpm

http://downloads.eucalyptus.com/devel/packages/3-devel/http://downloads.eucalyptus.com/devel/packages/3-devel/http://downloads.eucalyptus.com/devel/packages/3-devel/http://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-4.noarch.rpmhttp://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-4.noarch.rpmhttp://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpmhttp://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpmhttp://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpmhttp://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpmhttp://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-5.noarch.rpmhttp://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpmhttp://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-4.noarch.rpmhttp://yum.pgrpms.org/9.1/redhat/rhel-6-i386/pgdg-centos91-9.1-4.noarch.rpmhttp://downloads.eucalyptus.com/devel/packages/3-devel/

7/27/2019 eucaimp3

4/11

wget http://downloads.eucalyptus.com/devel/packages/3-

devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-

1.el.noarch.rpm

yum localinstall pgdg-centos91-9.1-4.noarch.rpm epel-release-6-

5.noarch.rpm \

elrepo-release-6-4.el6.elrepo.noarch.rpm \eucalyptus-nightly-release-3-1.el.noarch.rpm

Necessary Manual Edit

Note: DO THIS IF AND ONLY IF YOU ARE RUNNING A 32 BIT VERSION OF CENTOS.

It is HIGHLY recommended that you use a 64 bit OS, but making this change allows you to

get some .noarch packages from the Eucalyptus repository. Otherwise, the package list will

be 404 Not Found. Edit /etc/yum.repos.d/eucalyptus-nightly-release.repo This is the file for

the Eucalyptus 3 development repositories. The x86_64 repos had to be used because the

i686/i386 repositories gave a 404 error.

[euca-3-devel]

name=euca-3-devel

baseurl=http://downloads.eucalyptus.com/devel/packages/3-

devel/nightly/centos/$releasever/x86_64/gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY

gpgcheck=1

enabled=0

[euca-3-deps]

name=euca-3-depsbaseurl=http://downloads.eucalyptus.com/devel/packages/3-

devel/centos/$releasever/x86_64/gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLY

gpgcheck=1

enabled=1

The changes that should be made are in bold.

Install Packages

Source:http://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.html

yum install bzr python-boto euca2ools libvirt-devel openssl-devel gccant ant-nodeps \

java-1.6.0-openjdk-devel curl-devel libxslt-devel xalan-j2-xsltc

wsdl4j \

backport-util-concurrent httpd postgresql91-server libvirt PyGreSQL

make \

openssh-clients scsi-target-utils qemu-kvm axis2-codegen axis2-adb-

codegen \

http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpmhttp://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpmhttp://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpmhttp://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/$releasever/http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/$releasever/http://etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLYhttp://downloads.eucalyptus.com/devel/packages/3-devel/centos/$releasever/http://downloads.eucalyptus.com/devel/packages/3-devel/centos/$releasever/http://etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLYhttp://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.htmlhttp://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.htmlhttp://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.htmlhttp://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.htmlhttp://etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLYhttp://downloads.eucalyptus.com/devel/packages/3-devel/centos/$releasever/http://downloads.eucalyptus.com/devel/packages/3-devel/centos/$releasever/http://etc/pki/rpm-gpg/RPM-GPG-KEY-EUCALYPTUS-NIGHTLYhttp://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/$releasever/http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/$releasever/http://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpmhttp://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpmhttp://downloads.eucalyptus.com/devel/packages/3-devel/nightly/centos/6/x86_64/eucalyptus-nightly-release-3-1.el.noarch.rpm

7/27/2019 eucaimp3

5/11

httpd httpd-devel rampartc rampartc-devel axis2c axis2c-devel axis2

Create eucalyptus User

We need to add a eucalyptus user for the Eucalyptus services to run as. It is added to the

kvm group so machines that are node controllers can manipulate instances.useradd eucalyptuspasswd eucalyptus

Modify WSDL2C.sh

Open /usr/lib64/axis2c/bin/tools/wsdl2c/WSDL2C.sh in a text editor. Erase the existing lines

and add the following:

#!/bin/sh

java -classpath $(build-classpath axis2/codegen axis2/kernel

axis2/adb \axis2/adb-codegen wsdl4j commons-logging xalan-j2 xsltc \

backport-util-concurrent ws-commons-XmlSchema ws-commons-neethi \

ws-commons-axiom annogen ) org.apache.axis2.wsdl.WSDL2C $*

Compile Dependencies

DON'T ACTUALLY DO THIS ON A 64 BIT MACHINE. The eucalyptus repository that was

installed in a previous step has all of the packages you need. This is here for historical

purposes and will probably be deleted soon. Note: Compilation only needs to happen on

one machine. Once compiled, the binaries can be copied to the other servers

This section is copied pretty much verbatim from theEucalyptus 2.0 Installation Instructions.

Download Source Code

Download and extract tarball (2.0.3 dependencies work for 3.0 as far as I can tell)

wget http://eucalyptussoftware.com/downloads/releases/eucalyptus-

2.0.3-src-deps.tar.gz

tar -zxf eucalyptus-2.0.3-src-deps.tar.gz

Set shell variable for install location Dave's completely objective, non-biased Note: Installing

to /opt is really ugly, but this what the Eucalyptus install guide uses, so I will too.

export EUCALYPTUS=/opt/eucalyptus

Create directory for dependency binaries

mkdir -p $EUCALYPTUS/packages/

http://open.eucalyptus.com/wiki/EucalyptusInstallationSource_v2.0http://open.eucalyptus.com/wiki/EucalyptusInstallationSource_v2.0http://open.eucalyptus.com/wiki/EucalyptusInstallationSource_v2.0http://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gzhttp://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gzhttp://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gzhttp://eucalyptussoftware.com/downloads/releases/eucalyptus-2.0.3-src-deps.tar.gzhttp://open.eucalyptus.com/wiki/EucalyptusInstallationSource_v2.0

7/27/2019 eucaimp3

6/11

Change directory to where the dependency sources were extracted Your path will probably

be different

cd ~/Downloads/eucalyptus-src-deps/

Install Axis2

Extract and copy Axis2 No compilation here. Nice and easy.

tar -zxf axis2-1.4.tgz

mv axis2-1.4 $EUCALYPTUS/packages/

Install Axis2C

export APACHE_INCLUDES=/usr/include/httpd/

export APR_INCLUDES=/usr/include/apr-1/

export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0

tar zxf axis2c-src-1.6.0.tar.gz

cd axis2c-src-1.6.0

CFLAGS="-w" ./configure --prefix=${AXIS2C_HOME} --with-

apache2=$APACHE_INCLUDES --with-apr=$APR_INCLUDES --enable-multi-

thread=no

make

make install

Install Rampart/C

export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH

tar zxf rampartc-src-1.3.0-0euca2.tar.gz

cd rampartc-src-1.3.0

./configure --prefix=${AXIS2C_HOME} --enable-static=no --with-

axis2=${AXIS2C_HOME}/include/axis2-1.6.0

make

make install

Edit Rampart/C Configuration

Change the following in $AXIS2C_HOME/axis2.xml. In the 'inflow' section, change:

to

In the 'outflow' section, change:

to

7/27/2019 eucaimp3

7/11

Compiling Eucalyptus

Download Source Code

Fetch the latest source from launchpad

bzr branch lp:eucalyptus

Enter the source directory

cd eucalyptus

Compile

Set shell variables for Java environment:

export JAVA_HOME="/usr/lib/jvm/java-openjdk/"

export JAVA="$JAVA_HOME/jre/bin/java"

Compile Eucalyptus./configure --with-axis2c=/usr/lib64/axis2c/ --with-apache2-module-

dir=/usr/lib64/httpd/modules/

make

make install

Install Eucalyptus on Other Machines

Based heavily upon this wonderfulblog post.

Copy Installation to All Machines

Repeat for each machine.

rsync -a $EUCALYPTUS/ root@:$EUCALYPTUS/ ( being the IP

of the node you are copying to)

Set environment variables

These are needed pretty much everywhere. A good idea would be to place these in the

~/.bashrc files for root and eucalyptus users. You WILL have commands fail if you don't set

these.

export EUCALYPTUS=/opt/eucalyptus

export PATH=$PATH:$EUCALYPTUS/usr/sbin

Install Packages

Follow the "Installing Additional Packages" and "Installing Dependencies" instructions at the

top of this page to install needed packages onto the node

Configure

http://agrimmsreality.blogspot.com/2012/01/configuring-eucalyptus-3-devel.htmlhttp://agrimmsreality.blogspot.com/2012/01/configuring-eucalyptus-3-devel.htmlhttp://agrimmsreality.blogspot.com/2012/01/configuring-eucalyptus-3-devel.htmlhttp://agrimmsreality.blogspot.com/2012/01/configuring-eucalyptus-3-devel.html

7/27/2019 eucaimp3

8/11

Edit /opt/eucalyptus/etc/eucalyptus/eucalyptus.conf Find the following configuration

variables and set the values accordingly

EUCALYPTUS="/opt/eucalyptus"

HYPERVISOR="kvm"

USE_VIRTIO_DISK="1"

USE_VIRTIO_NET="1"INSTANCE_PATH="/opt/eucalyptus/instances"

VNET_BRIDGE="virbr0"

Register Startup Scripts

Create symbolic links to the daemons so that the `service` command will work with them

ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud

/etc/init.d/eucalyptus-cloud

ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc

ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

Edit Hosts

A common problem that occurs is that the euca_conf --initialize` command will fail. This

usually happens because the system hostname isn't in the /etc/hosts file.

Edit hosts file:

vim /etc/hosts

Add something like the following to the bottom of the file using your machine's IP address

and hostname:10.10.10.111 myhostname

Perform Inital Setup

euca_conf --setup

Start Cloud Controller (CLC), Storage Controller (SC) andWalrus

Start Service

euca_conf --initialize

service eucalyptus-cloud start

Get Credentials

mkdir euca-credentials

cd euca-credentials

euca_conf --get-credentials admin.zip

7/27/2019 eucaimp3

9/11

unzip admin.zip

source eucarc

If you get an error when you try to do this such as "index out of range", it means that the

eucalyptus services aren't fully operational yet. Be patient and eventually you can get your

credentials.

Register Walrus, CC, and SC

The component and storage flags are arbitrary names for the registered services. Change

to your liking. In our specific case in CS401, Walrus and the SC are running on the same

machine as the CLC, so the hostname is simply the static IP of the Morpheus server.

su eucalyptus

euca_conf --register-walrus --host --component walrus --

partition mycloud

euca_conf --register-sc --host --component storage --

partition mycloud

euca_conf --register-cluster --host --component cluster --

partition mycloud

It is important to note that the partition name for the Storage Controller and Cluster

Controller MUST BE THE SAME. If the names are different then your cloud will NOT

WORK.

Cluster Controller

Start Service

service eucalyptus-cc start

Node Controllers

Run on each NC

Install Packages

yum install eucalyptus-nc

Configure Ethernet Bridge

Enter the network configuration directory

cd /etc/sysconfig/network-scripts/

Open a new file called ifcfg-br0 in your text editor of choice

vim ifcfg-br0

7/27/2019 eucaimp3

10/11

Here is an example bridge configuration. Essentially, move your configuration from ifcfg-

eth0 to ifcfg-br0. Your ethernet device configuration will resemble this:

DEVICE="eth0"

TYPE="Ethernet"

HWADDR="00:18:8B:81:AE:E4"

BRIDGE="br0"

Your bridge configuration will resemble this:

DEVICE="br0"

TYPE="Bridge"

BOOTPROTO="static"

BROADCAST="10.15.255.255"

DNS1="10.13.1.25"

GATEWAY="10.15.1.1"

IPADDR="10.15.15.13"

NETMASK="255.255.0.0"ONBOOT="yes"

Restart the network service:service network restart

If all goes well your devices and bridge will come up.

Register Node with CC

From the CLC run:

euca_conf --register-nodes="nodeip1 nodeip2 etc"

Start Services

service libvirtd start

service eucalyptus-nc start

Verify Installation

Verify Registered Services on CLC

euca-describe-walruseseuca-describe-storage-controllers

euca-describe-clusters

Sample Output:

WALRUS walrus walrus 10.15.15.10

ENABLED {}

7/27/2019 eucaimp3

11/11

STORAGECONTROLLER storage storage 10.15.15.10

ENABLED {}

CLUSTER trinity trinity 10.15.15.12

ENABLED {}

Verify Cluster is Advertising Resourceseuca-describe-availability-zones verbose

Sample Output (that should be updated when the cluster actually advertises resources):

AVAILABILITYZONE trinity 10.15.15.12

arn:euca:eucalyptus:trinity:cluster:trinity/

AVAILABILITYZONE |- vm types free / max cpu ram disk

AVAILABILITYZONE |- m1.small 0002 / 0002 1 128 2

AVAILABILITYZONE |- c1.medium 0002 / 0002 1 256 5

AVAILABILITYZONE |- m1.large 0001 / 0001 2 512 10

AVAILABILITYZONE |- m1.xlarge 0000 / 0000 2 1024 20AVAILABILITYZONE |- c1.xlarge 0000 / 0000 4 2048 20