Convey Systems Administration Guide - conveysupport.com · Convey System Administration Guide v1.6 ii The Information in this document is provided for use with Convey Computer Corporation

i

Convey System Administration Guide

November 2010 Version 1.6 901-000002-000

Convey ComputerTM Corporation 2009-2010. All Rights Reserved.

1302 East Collins Richardson, TX 75081

Convey System Administration Guide v1.6 ii

The Information in this document is provided for use with Convey Computer Corporation (“Convey”) products. No license, express or implied, to any intellectual property associated with this document or such products is granted by this document. All products described in this document whose name is prefaced by “Convey” or “Convey enhanced“ (“Convey products”) are owned by Convey Computer Corporation (or those companies that have licensed technology to Convey) and are protected by patents, trade secrets, copyrights or other industrial property rights. The Convey products described in this document may still be in development. The final form of each product and release date thereof is at the sole and absolute discretion of Convey. Your purchase, license and/or use of Convey products shall be subject to Convey‟s then current sales terms and conditions.

Trademarks

The following are trademarks of Convey Computer Corporation in the United States and other countries:

Convey Computer

The Convey Logo

Convey HC-1

Trademarks of other companies

Red Hat and Red Hat Enterprise Linux are registered trademarks of Red Hat, Inc.

Fedora and RPM are trademarks of Red Hat, Inc.

SUSE and openSUSE are registered trademarks of Novell, Inc.

Linux is a registered trademark of Linus Torvald

Python is a registered trademark of the Python Software Foundation

GNU is a trademark of The Free Software Foundation

PERL is a trademark of the Yet Another Society DBA The Perl Foundation

Ubuntu and Canonical are registered trademarks of Canonical Ltd.

Debian is a registered trademark of Software in the Public Interest, Inc.

Convey System Administration Guide v1.6 iii

Revisions

Version Description

1.0 May 2008. Original printing.

1.1 August 2008. New system setup procedures.

1.2 May 2009. Reworked for Beta Release of HW, OS, and SW.

1.3 June 2009. Revised reboot instructions when coprocessor is hung. New scripts.

1.4 September 2009. Recovery DVD usage instructions. New SW repo conventions.

1.5 April 2010. New licensing mechanism. New DVDs for updating systems. New cross-development system install instructions.

1.6 November 2010. DoWebUpdate command for updating Convey servers. Web interface for adding/changing web access users/passwords.

Convey System Administration Guide v1.6 iv

Table of Contents

1 Overview ..................................................................................................................... 6 1.1 Intended Audience .................................................................................................. 6 1.2 Topics Covered ....................................................................................................... 6 1.3 Other Suggested Documents ................................................................................. 6

2 Installing Convey Software ......................................................................................... 7 2.1 Internet connection ................................................................................................. 8 2.2 Convey Software Repositories for Cross-Development Systems ......................... 8

2.2.1 Where to install the Convey .repo/.list files ..................................................... 8 2.2.2 Downloading and installing the Convey repository description files ............... 9

2.3 Convey Server Software ......................................................................................... 9 2.3.1 Convey O.S. Installation ................................................................................ 10 2.3.2 Installing Convey O.S. from a DVD ............................................................... 10 2.3.3 Convey Firmware Installation ........................................................................ 10 2.3.4 Convey Coprocessor Recipe Packages........................................................ 10 2.3.5 Updating Convey OS, firmware, and Recipe Packages ............................... 10 2.3.6 Installing other Convey Software products on a Convey Server .................. 11

2.4 Installing Convey Software on Cross-Development Systems.............................. 11 2.4.1 Compatible Linux® Distributions ................................................................... 11 2.4.2 SELinux Considerations ................................................................................ 12 2.4.3 Available Convey Software Development Packages .................................... 12 2.4.4 Identifying the Appropriate Installation Tool .................................................. 13 2.4.5 Installing Convey Software Packages ........................................................... 14

3 Updating Convey Software Packages ..................................................................... 15 3.1 Updating Convey Software Packages using a GUI ............................................. 16 3.2 Installing older versions of Convey Software Packages ...................................... 16 3.3 Version Numbers and Package Filenames .......................................................... 16 3.4 Manual Installation ................................................................................................ 16 3.5 Additional Installation Notes for Ubuntu/Debian and openSUSE ........................ 18

3.5.1 How to downgrade gcc and g++.................................................................... 18 3.6 Installing Convey Software without an Internet Connection ................................ 18

4 Booting and System Management ........................................................................... 20 4.1 Overview ............................................................................................................... 20 4.2 Powering Up the Convey Server .......................................................................... 20 4.3 First Power-Up and Boot ...................................................................................... 20

4.3.1 Intel Remote Management Module ............................................................... 20 4.3.2 First Boot ........................................................................................................ 20 4.3.3 Manual Configuration .................................................................................... 21

4.4 Boot Options ......................................................................................................... 21 4.4.1 enable/disable ................................................................................................ 22 4.4.2 caemode ........................................................................................................ 22 4.4.3 memory .......................................................................................................... 22 4.4.4 window ........................................................................................................... 22 4.4.5 pagesize ......................................................................................................... 22 4.4.6 interleave ....................................................................................................... 23

4.5 Coprocessor Modules ........................................................................................... 23 4.5.1 CNYMP Module ............................................................................................. 23

Convey System Administration Guide v1.6 v

4.5.2 CNYSYS Module ........................................................................................... 23 4.6 Kernel Configuration ............................................................................................. 24 4.7 Convey System Services ...................................................................................... 24

4.7.1 Convey Configuration Files ........................................................................... 25 4.8 Coprocessor Sanity Test Program ....................................................................... 25 4.9 When to Shutdown the Convey Server ................................................................ 26 4.10 Activating a Coprocessor Recipe ......................................................................... 26

5 Convey System Memory Management .................................................................... 27 5.1 Overview ............................................................................................................... 27

5.1.1 Intended Audience ......................................................................................... 27 5.1.2 Terminology ................................................................................................... 27

5.2 NUMA Memory Configuration ............................................................................... 27 5.2.1 Physical Memory Configuration ..................................................................... 28

5.3 Address Extension Feature .................................................................................. 28 5.3.1 Fully-mapped mode ....................................................................................... 28 5.3.2 Dynamic-mapped mode ................................................................................ 29

5.4 Memory Interleaving ............................................................................................. 29 5.4.1 Binary Interleave ............................................................................................ 30 5.4.2 31-31 Interleave ............................................................................................. 30 5.4.3 31-31 Interleave Page Allocation .................................................................. 30 5.4.4 31-31 Interleave Faults .................................................................................. 31

5.5 Variable Page Size Support ................................................................................. 31 5.5.1 Maximum Coprocessor Page Size ................................................................ 32 5.5.2 Linux HugeTLB Feature ................................................................................ 32

5.6 Memory Migration ................................................................................................. 32 6 Personality Management .......................................................................................... 34

6.1 Personality Signatures .......................................................................................... 34 6.2 Default Base Signature ......................................................................................... 34 6.3 System Default Signatures ................................................................................... 35 6.4 Convey Personality Management Script .............................................................. 35

7 Recovery DVDs ........................................................................................................ 37 7.1 How to use the Convey System Recovery DVD .................................................. 37

7.1.1 Preparation .................................................................................................... 37 7.1.2 Boot from DVD ............................................................................................... 37 7.1.3 Configure the Installation and Installing the OS ............................................ 38 7.1.4 System Configuration ........................................................................................... 38

7.2 Fixing the Convey SW Repository Files ............................................................... 39 7.3 Adding Convey Provided License Keys ............................................................... 40 7.4 Installing the Remaining Convey Software Products ........................................... 40 7.5 Checking the Management Processor ................................................................. 40 7.6 Updating the Server to the latest SW Versions .................................................... 41 7.7 Restoring a Convey Server without a DVD drive ................................................. 41

8 Customer Support Procedures ................................................................................. 42 9 Licensing ................................................................................................................... 43

9.1 Convey Licensed Software ................................................................................... 43 9.2 Old Licensing Scheme .......................................................................................... 43

Appendix A - Installing YUM on older SUSE™ Distributions ........................................... 44 Index . ................................................................................................................... 45

Convey System Administration Guide v1.6 6

1 Overview

This guide describes how to perform various tasks that are typically performed by a system administrator. These tasks require system administrator privileges.

Please note that this document contains clickable hyperlinks. If you are browsing this document on Convey‟s web site, or downloaded all of Convey‟s documents to the same directory, those clickable hyperlinks should work. If you only download a single document, clicking on the hyperlinks will fail.

1.1 Intended Audience

Administrators of a Convey hybrid-core server

Users of desktop Linux® systems (who have root access) used for cross-development of

applications for a Convey hybrid-core server

Administrators of customer provided Linux® systems used for cross-development of

applications for a Convey hybrid-core server

1.2 Topics Covered

How to update Convey server specific software, including:

Convey OS and patches

Convey server firmware

How to install and update other Convey software on a cross-development system or a Convey server, including the

Convey compilers, assembler, linker, and system libraries

Convey provided coprocessor personalities

Convey documentation

How to reboot a Convey server

How to check for updates for Convey‟s software packages

Personality Management

Recovering a Convey Server using the recovery DVDs

Customer Support Procedures

Licensing

1.3 Other Suggested Documents

Convey HC1 Service Guide – Describes how to install, upgrade, and troubleshoot the

HC1 hardware


2 Convey’s Support Website

Convey provides a technical support website for customers: http://www.conveysupport.com.

This website contains a variety of useful information for developers, system administrators, installers, etc.

A Convey provided web access user name and password are required to access the site, although anyone with such a username/password can create new web access usernames, by visiting http://www.conveysupport.com/protected/webaccess.html.

The most recent versions of all Convey documentation is provided, as well as older versions of most documents.

Support bulletins, FAQs, a bug reporting/tracking system, software repositories, and other resources are available.

All users of Convey servers or development software tools should have access to the Convey support website.

http://www.conveysupport.com/

http://www.conveysupport.com/protected/webaccess.html


3 Installing Convey Software

In general, Convey recommends using the Convey provided software repositories for most Convey software installations. These repositories are accessed using the standard Linux software installation tools (i.e. yum, zypper, apt-get), and require the system to be connected to

the internet. Users with a large number of Convey servers or cross-development systems may wish to create a local copy of the Convey software repositories, or download the individual packages and install them with the low level package installer provided by their Linux distribution. See Installing Convey Software without an Internet Connection for more details.

Some Convey software products require a license to be installed in order to use that product. As of the April 2010 release, all Convey software products may be installed on a compatible system before the license is installed. Convey servers are delivered with appropriate licenses already installed, based on the sales order for the server. Licenses for cross-development systems must be installed by a system administrator. Typically, Convey will email the contact person identified on a sales order for a cross-development system with instructions for obtaining the appropriate license(s).

3.1 Internet connection

Most of the directions below assume that a working internet connection exists. If a proxy server is required, the appropriate proxy configuration should be updated for the software installation tool of choice1. In addition, the wget command requires that the system‟s network settings correctly

specify any required proxy for http access to conveysupport.com.

If the system does not have a working internet connection, or you wish to install the Convey software from a Convey Software DVD, see the section titled Installing Convey Software without an Internet Connection.

3.2 Convey Software Repositories for Cross-Development Systems

On user provided cross development systems, the Convey software repository description files must be installed before other Convey software can be installed from those repositories. On Convey servers, the software repository description files are already installed.

3.2.1 Where to install the Convey .repo/.list files

On rpm based Linux distributions (Redhat, SuSE, Fedora, CentOS, Convey‟s OS),

Repository description filenames have a .repo suffix

All repository description files are located in either

/etc/yum.repos.d/ (for Redhat, CentOS, Convey‟s OS, Fedora, and older Suse/OpenSUSE

systems), or

/etc/zypp/repos.d/ (for newer OpenSUSE systems).

The Convey .repo files should be installed in whichever directory exists and already

contains other .repo files.

On Debian based Linux distributions (including Debian and Ubuntu),

1 For Convey Servers, see /etc/yum.conf


Repository description filenames have a .list suffix

The repository description files are located in /etc/apt/sources.list.d/, and the Convey

.list files should be installed there. This directory may be empty before installing the

Convey repository description files.

3.2.2 Downloading and installing the Convey repository description files

Once you know which directory to install Convey‟s repository description files in, you may download them from the Convey Support website by:

Logging into the system you want to install Convey‟s software products on, as the root

user

cd … (where … is the directory that contains all of your system‟s repository description

files, as described above)

wget http://conveysupport.com/repofiles/hc1-1/filelist

wget -i filelist

On an RPM based system, delete the unneeded .list files just downloaded, and on a

Debian based system, delete the unneeded .repo files just downloaded

See the wget man page for options required when an http proxy server must be used

If the wget utility is not already installed on your system, most Linux distributions provide it

in the vendor‟s software repository.

Alternatively, you can click on each repository description file in your browser, and copy/paste the contents of the file to the same name file on your Linux system.

After installing the Convey repository description files, each of those files must be edited, and the strings “username” and “password” must be replaced with the username and password provided for

your use when any Convey software product was purchased. E-mail [email protected] for the name of the contact at your site that can provide the appropriate username and password. If your site is using a local copy of the Convey repositories, it may not be password protected. In this case, you should delete the string “username:password@” from each repository description file.

3.3 Convey Server Software

If you are installing Convey‟s cross-development tools on a non-Convey system, please skip over this Convey Server Software information.

Convey servers come with the following items pre-installed (package names are in this font):

Many CentOS packages

Convey OS packages (Convey enhancements to CentOS) convey-kernel (or convey-kernel_beta), convey-platform_utils, convey-platform_base

Convey Firmware package convey-mp_fw

Convey Coprocessor recipe package(s) convey-recipe-release

Convey coprocessor base image package convey-aebase-image

mailto:[email protected]


Convey Personality packages(s) with an Application Engine FPGA bit image (these are optional licensed products) convey-aesp (where “sp” can be any Convey provided personality name)

as well as many of the Convey Software packages described later, that can be installed on both

Convey servers and non-Convey systems running a compatible Linux distribution.

Additional licensed packages may be installed with the standard yum package manager, from the

repositories on Convey‟s support website, or from a Convey Software DVD.

3.3.1 Convey O.S. Installation

A Convey server must have the Convey provided Linux® distribution installed. No other Linux

®

distributions may be installed on a Convey server. The Convey provided Linux distribution is pre-installed on all Convey servers. This distribution (and various updates) is also available on Convey‟s support website, http://conveysupport.com.

Convey does not recommend installing Convey‟s Linux distribution on non-Convey systems. Since Convey‟s Linux distribution is based on CentOS 5.3, CentOS may be installed on cross-development systems if you want the most similar Linux distribution on those systems.

3.3.2 Installing Convey O.S. from a DVD

If the Convey server includes the optional DVD drive, Convey‟s O.S. may be installed using a Convey supplied recovery DVD. You should never install Convey‟s O.S. from a DVD unless instructed to do so by the Convey Support group (typically required only when replacing the main disk in a Convey server). Please contact Convey if you believe you need to reinstall the O.S. on

a Convey server.

3.3.3 Convey Firmware Installation

A Convey server, in addition to the Convey provided Linux® distribution, requires the Convey

firmware package (convey-mp_fw) to be installed. This package is pre-installed on all Convey

servers. The Convey firmware package should not be installed on non-Convey systems.

3.3.4 Convey Coprocessor Recipe Packages

Recipe packages contain (or indirectly reference) “bit image files” that are loaded into some of the FPGAs that are part of the Convey coprocessor. A recipe is a Convey tested set of these images that are known to work well together. Typically, only the “released” recipe should be activated on a Convey server unless a Convey support representative recommends installing another recipe.

The “released” recipe is always in a package named convey-recipe-release.

Installing or updating a receipe does not activate that recipe. Any installed recipe may be activated with the pickRecipe or activaterecipe commands in /opt/convey/sbin.

Recipe packages should not be installed on non-Convey systems.

Note that when a recipe package is installed, it copies the relevant files to /opt/convey/fpgaimages, and then removes itself. The relevant files will still be present on the Convey server. This automatic removal allows older recipes to be installed after a newer recipe has been installed.

3.3.5 Updating Convey OS, firmware, and Recipe Packages

Updates to all Convey server software packages may be performed with the yum tool, if either

internet access or a locally mounted Convey DVD repository is available.

IMPORTANT NOTE: After updating or installing any Convey OS, firmware, or recipe packages,

the host system needs to be shutdown for at least two minutes, and then powered up again. Any

package that requires a shutdown will indicate so during installation.

http://conveysupport.com/


For example, to update every Convey package installed on a system, logon the system as root,

and enter the command:

yum update convey-\*

and answer “yes” when prompted.

Occasionally the yum database becomes out-of-date or corrupted, and yum won‟t be able to find or

update packages. If you see unexpected error messages while using yum, you can purge the yum

database with the command:

yum clean all

New packages, such as new recipe packages, or newly purchased personality packages, may be installed with the command:

yum install convey-xxx

where convey-xxx is the package name to be installed.

3.3.6 Installing other Convey Software products on a Convey Server

See the very next section, Installing Convey Software on Cross-Development Systems.

3.4 Installing Convey Software on Cross-Development Systems

Other Convey Software products, including Convey‟s software development products, may be installed on a Convey server or on any compatible user provided Linux

® system. When the

Convey compilers, assembler, linker, personality tools, and, optionally, the personality development kit are installed on a non-Convey system, they provide a cross-development and debug environment that permits developing and debugging code for the Convey server and coprocessor, using a software simulator. See the Convey Programmers Guide for more information.

Any other Convey products/packages that were purchased, such as the Convey Math Libraries, the Personality Development Kit, or a Convey Application Engine fpga image (for a particular personality), can be individually installed.

If you are installing or updatingConvey software products on a Convey server, you may skip the section titled Compatible Linux

® Distributions below.

The directions below assume that an appropriate Linux® distribution is already installed on an

x86-64 system. If a proxy server is required, the appropriate proxy configuration should be updated for the software installation tool of choice2, or you may use the manual installation instructions below. See the section titled Installing Convey Software without an Internet Connection for systems that do not have a working internet connection.

3.4.1 Compatible Linux® Distributions

The Convey software development products were developed and have been extensively tested on the 5.1, 5.2 and 5.3 releases of the CentOS and Red Hat 64-bit distributions. If you have no preference for any particular Linux distribution for your cross-development systems, Convey recommends you install either the CentOS or Red Hat distribution.

The Convey PDK and other Convey software development products have been installed and verified to work on an x86-64 system running the following 64-bit Linux

® distributions:

Red Hat® SUSE™ Fedora™ Core OpenSUSE™

2 For Convey Servers, see /etc/yum.conf


Desktop 5.1 Desktop 10 8 and 9 10.3 and 11.0

CentOS 5.2/5.3 Ubuntu® 8.0.4 Red Hat Enterprise Server 5.1-5.3

Debian

Different minor releases of these distributions are also likely to work. Newer major releases of these distributions are likely to work if no substantial incompatibilities with respect to libc, python

®,

and perl™ are introduced by the new release.

In addition to the base distribution, the “Software Development” super-package was typically installed (x86-64 versions of gcc and g++ are required). Most live distributions (Linux distributions

that boot off of a DVD or CD but are not installed on a hard disk) are likely to require additional packages that most DVD based distributions include automatically.

Other packages that are required, but typically included by default, are python®, and perl™. yum,

zypper, or apt-get is required if you want to install and update software using the Convey provided

repositories.

Other 64 bit Linux® distributions that have gcc 3.2 (or newer) and a corresponding g++ compiler,

GNU™ libc 2.5, a recent version of zypper, rpm or apt-get, and python 2.4 installed are likely to work well, but have not been tested by Convey Computers. Most up-to-date Linux

® distributions

with a 2.6.xx kernel will meet the above criteria, after any necessary optional packages are installed.

Convey software development products are available over the internet, as RPM™ and Debian®

packages. These packages may be downloaded individually, or installed using the provided yum and Debian repositories. These packages are also available on a CD by request (email your request to [email protected]).

3.4.2 SELinux Considerations

If SELinux is enabled on a cross-development system, please see the SELinux FAQ at www.conveysupport.com/protected/faqs for more information about configuring SELinux

(required in order for the Convey compilers to operate correctly).

3.4.3 Available Convey Software Development Packages

Users of Debian based distributions should note that any underscores in the package names below will be replaced with dashes in the Debian package name (i.e. convey-sp_per -> convey-sp-per).

Note that the full filename for a given package includes version and architecture information. The rpm packages have an architecture specification of x86_64, while the debian packages have an architecture specification of amd64. These two architecture specifications are equivalent. Convey software packages should only be installed on systems with x86-64 compatible processors.

The convey-1st package provides a few administrative scripts and adds /opt/convey/bin to

everyone‟s PATH

The convey-per-tools package provides the Convey personality database and some

personality management tools

Packages containing Convey compilers, support libraries, debugger, … (install all of these packages together)


http://www.conveysupport.com/protected/faqs


o convey-open64 (also installs the following)

o convey-utils64

o convey-utils64-opensource

o convey-platform_libs

Convey personality packages for software development

o convey-sp_per

o convey-dp_per

o convey-fap_per

o convey-none_per

o convey-pdk_per (only for use with the convey-pdk package)

Convey Personality Development Kit for developing a custom personality

o convey-pdk

Convey math libraries

o convey-cml

OpenMPI

o convey-openmpi

InfiniBand Drivers

o convey-ofed-basic, convey-ofed-hpc, convey-ofed-all (only install one of these)

Documentation not included in another package

o convey-manuals

3.4.4 Identifying the Appropriate Installation Tool

One of the following installation tools (or a compatible equivalent) is required to utilize the Convey repositories. If none of these tools are installed, either install the appropriate tool or skip to the section titled Manual Installation.

yum

zypper

apt-get

On all Redhat, Fedora, SUSE, and older openSUSE distributions, yum is the tool of choice. It

may be necessary to install yum on SUSE and openSUSE distributions (available on the

openSUSE installation media and from the openSUSE repositorty).

On openSUSE 10.3 and 11.x distributions, zypper is the tool of choice and should be installed by

default.

On Debian distributions (including Ubuntu), apt-get is the tool of choice.

On all other distributions, check to see if one of these tools is available, or can be installed from the repositories provided by that distribution. If none of these tools can be used, skip to the section titled Manual Installation.

Once you know which installation tool is available, follow the repository setup directions for that tool below.


3.4.5 Installing Convey Software Packages

Log in as root

The example commands below, for installing a package, use the yum command.

Substitute zypper or apt-get if appropriate for your Linux distribution. Remember to

change “_”s to “-“s in package names for Debian based distributions.

Install the convey-1st package first yum install convey-1st

Install any Convey provided licenses as instructed in the accompanying email. Email [email protected] for your site‟s contact person who can provide the appropriate licenses and instructions.

Install the Convey personality tools package yum install convey-per-tools

Install the Convey compilers, debugger, and other software development tools (install all 4 packages together) yum install convey-open64

Install any of the personality packages you need yum install convey-sp_per

yum install convey-none_per

…

Install any licensed packages you have a valid license for, such as yum install convey-cml

yum install convey-pdk

yum install convey-sp_ae

If you would prefer /opt/convey/bin to appear before /usr/bin in $PATH (and $path) on cross-

development systems, execute (as root) the script /opt/convey/sbin/preferoptconveybin, log

off your terminal, ssh, or telnet session and start a new terminal session. This script will automatically be executed when certain Convey server specific software packages are installed on a Convey server. The effect of this script can be undone by editing the files /etc/profiles.d/convey.sh and /etc/profiles.d/convey.csh, and moving /opt/convey/bin to

the end of the path.



4 Updating Convey Software Packages

4.1 Updating Convey Sofware on a Convey Server

On a Convey server, Convey provides a script to update most Convey SW packages (except those that are customer specific/proprietary), and ensure that compatible versions of FPGA recipes, AE personality firmware, management processor firmware, and other Convey software are installed, and the proper recipe is activated. This script may be used by:

Log in as root

yum update convey-support-tools

if this command complains that there is "No Match for argument", install the package with:

yum install convey-support-tools

/opt/convey/support/DoWebUpdate

This script updates all installed Convey provided software on your Convey Server, including various firmware packages, the firmware recipe, coprocessor personalities, development tools, the Convey OS and associated CentOS packages. This script ensures that the various firmware, recipe, operating system, and other packages are compatibile and up-to-date.

4.2 Updating Convey Software on non-Convey systems

All previously installed Convey software development packages and Convey server specific packages may be upgraded to the newest released version with the usual Linux update mechanism for your distribution. For Convey OS, this is:

yum update convey-\*

Note that yum‟s metadata cached on your system is typically reused for at least an hour, and that metadata may need to be reloaded from the Convey repositories if you are attempting to use a repository that has just been updated. For Convey servers, the command

yum clean metadata

will flush the existing metadata, and the next yum command will reload metadata from all the

repositories.

Also note that the database of installed software on most Linux distributions does not care where a package has been installed. Therefore, if you install a newer version of a package in a location other than the default, you may need to remove that version of the package before you can install it in the default location.

On Debian based distributions, the apt-get command is used to update Convey packages.

apt-get update (flushs old metadata)

apt-get upgrade convey-\*

On systems where zypper is available, packages may be updated with:

zypper update --type package convey-\*

Zypper may report multiple available versions for some packages. Please choose the newest (bigger version number) version of each package.


4.3 Updating Convey Software Packages using a GUI

After the Convey repositories have been setup, the standard installation tools provided with a Linux distribution, including graphical user interfaces to those tools (i.e. YaST, aptitude, Synaptics Package Manager), may be used to install, upgrade, and remove Convey supplied packages. Almost all Convey supplied packages have names that start with „convey-„, and are provided in both .rpm and .deb versions in corresponding yum and debian repositories. Customer proprietary packages developed by Convey typically do not have “convey-“ as part of the package name.

Note that the debian package names have dashes („-„) in their names wherever the rpm package names have underscores. This difference in package names is an artifact of the tool used to automatically convert rpm packages to debian packages.

4.4 Installing older versions of Convey Software Packages

Some of the native Linux installation tools support installing older versions of a package.

First, remove any installed versions of the package (except for kernel and AE personality image

packages). Then install the package specifying the desired version.

Installation Tool

Sample Syntax

zypper zypper install convey-1st=0.9.0

yum yum install convey-1st-0.9.0-0

apt-get apt-get install convey-1st=0.9.0

Note that the convey-kernel package and the personality packages with application engine images

(i.e. convey-sp_ae) may (and should) have multiple versions installed concurrently. Do not remove

any older (or newer) versions of these packages when installing a different version.

Note that some personality packages (convey-…_per) generally include all of the older versions of

that particular personality.

4.5 Version Numbers and Package Filenames

Throughout the discussion of package installation, package version numbers are depicted as w.x.y-z, always as part of a package filename, i.e. convey-1st-w.x.y-z.x86_64.rpm. The actual

packages on the website and installed on a system will typically have numeric version numbers, such as 1.1.1-1 or 1.2.1-2. w is the major version number, x is the minor version number, and y is the very minor version number. z is the so-called release number, and in Convey packages is typically an internal version number, and may not be strictly numeric.

The major version number (w) is changed whenever a major new release of a product occurs. The minor version number (x) is changed whenever minor functionality changes or some significant bug fixes are being released. The 3

rd version number (y) is changed when minor

changes are being released.

4.6 Manual Installation

These instructions assume either the rpm or dpkg command is available. On some distributions,

another installation program may be appropriate.

You will need the Convey assigned username and password for your site in order to access the packages on the Convey support website. No password is needed to explore the Convey Software DVD. In the URLs below, “release” may be replaced with one of the other release


stream names currently on the Convey support website, if you have an appropriate password for that release stream (i.e. beta).

All Convey created packages have a filename that starts with “convey-“.

Review the list of available packages in the following directories:

http://conveysupport.com/protected/install/hc1-1/convey_sdk/x86_64/RPMS/

http:// conveysupport.com/protected/install/ hc1-1/convey_sdk/updates/x86_64/RPMS/

http:// conveysupport.com/protected/install/ hc1-1/convey_platform_sdk/x86_64/RPMS/

http:// conveysupport.com/protected/install/ hc1-1/convey_platform_sdk/updates/x86_64/RPMS/

and for Convey servers only, the following directories also:

http:// conveysupport.com/protected/install/ hc1-1/convey_platform/x86_64/RPMS/

http:// conveysupport.com/protected/install/ hc1-1/convey_platform/updates/x86_64/RPMS/

http:// conveysupport.com/protected/install/ hc1-1/convey_centos/...

(in the convey_centos directory, please select the CentOS release # that matches your system)

Note that there may be multiple versions of the each package. Typically, you want to download/install the newest version of each package (the one with the largest version number).

The Convey Software DVD provides a similar structure.

To download and install one or more packages from Convey‟s website:

Login as root (typically with the su command)

cd dir

where dir is the directory you want to store the packages

Check which Convey packages are already installed, and remove any you want to upgrade or replace (except for kernel and application engine fpga image packages):

o On rpm based systems, use rpm –qa | grep convey

rpm –-erase package-name(s)

o On Debian based systems, use aptitude search convey-\*

apt-get remove package-name(s)

If aptitude is not available, use the graphical interface to the software installer

provided by your distribution to seach for packages with „convey‟ in their name

retrieve the packages you want to install

o wget http://username:[email protected]/protected/...

replacing „username‟ and „password‟ with the Convey provided username and password for web site access, or

o copy the packages from the Convey Software DVD

Install each package

o rpm –ihv convey-…-w.x.y-z.x86_64.rpm, or

o dpkg –i convey-…-w.x.y-z_amd64.deb (Debian based distributions)

The above steps can be repeated at any time to update the installed Convey software with a newer version of any package.

http://conveysupport.com/protected/install/hc1-1/convey_sdk/x86_64/RPMS/

http://conveycomputer.com/protected/install/release/convey_sdk/updates/x86_64/RPMS/

http://conveycomputer.com/protected/install/release/convey_platform_sdk/x86_64/RPMS/

http://conveycomputer.com/protected/install/release/convey_platform_sdk/updates/x86_64/RPMS/

http://conveycomputer.com/protected/install/release/convey_platform/x86_64/RPMS/

http://conveycomputer.com/protected/install/release/convey_platform/updates/x86_64/RPMS/

http://conveycomputer.com/protected/install/release/convey_centos/...


4.7 Additional Installation Notes for Ubuntu/Debian and openSUSE

Some distributions, including Ubuntu 8.0.4, openSUSE 10.3, and openSUSE 11.0, have a problem linking 32 bit C/C++ applications with the newer version of gcc and g++. In particular, when compiling the examples in the pdkSample directory that is part of the PDK personality, the linker will complain that it can‟t find the library stdc++. If you receive this error message, you need to downgrade to an older version of gcc and g++.

4.7.1 How to downgrade gcc and g++

First, see what versions of gcc and g++ are already installed:

ls /usr/bin/gcc* /usr/bin/g++*

If version 3.4 of gcc and g++ is not listed, please install the 3.4 version of those compilers from your Linux distributions CD or vendor repository.

After the 3.4 versions of gcc and g++ are available, execute the following commands logged in as root:

rm gcc g++

ln -s gcc-3.4 gcc

ln -s g++-3.4 g++

Any scripts or Makefiles that use „gcc‟ or „g++‟, without a version number, will now use the older 3.4 version of those products. The specific versions of gcc and g++ (i.e. gcc-4.1) can still be used directly, and the system can be restored to its original state by linking gcc and g++ to the newest install version of those products.

4.8 Installing Convey Software without an Internet Connection

If a reliable internet connection is not available on your system, you may install the Convey Software Products several different ways:

Install Convey Software products using a Convey provided Update DVD

o Email or call Convey support, and request a Convey Update DVD, and follow the directions that come with the update DVD, or

Make your own DVD or USB installation media:

o On another computer that has internet access, browse to http://conveysupport.com/protected/install/hc1-1, and for each directory of interest therein:

o Descend into directorties of interest, until you find the .rpm or .deb packages,

o right-click on each package whose name ends with „.rpm‟ for Red Hat /

SUSE based distributions, or „.deb‟ for Debian/Ubuntu based distributions,

and pick Save Target As (in Internet Explorer) or Save Link As (in Firefox/Netscape) to save each file to a local disk.

o You may also want to save the repository database files (Packages.gz for the debian repository, and the repodata directory for the yum repository) (you must preserve the entire existing directory structure to create a usable repository).

o You may also use the wget command (with the –r option) to download an

entire repository directory structure, and then copy it to a USB memory stick, a writable CD/DVD, or some other portable media.


o It may be easier to use a copy of the repositories if you create a separate CD or USB memory stick for each repository of interest, where the repodata or Packages.gz file is in the root directory of the CD or USB memory stick.

o Do not copy the convey-platform repository for use on a non-Convey system. The packages in this repository should only be installed on a Convey Hybrid-Core server.

o If the CD or USB memory stick includes the repository database files (Packages.gz and/or the repodata directory) then the software installation tools provided with your Linux distribution can probably treat the CD or USB memory stick as a repository (possibly after enabling the CD/USB media as a repository), and you may install the Convey packages like any other package, using yum, zypper, or apt-get. You might need to create the

repository description files by hand, to point to the correct subdirectory on the CD or USB memory stick.

o On Ubuntu, a USB memory stick can be enabled as a Software Source by adding a line such as deb file:/media/UsbName /

to /etc/apt/sources.list, where „UsbName‟ is the name the USB memory stick was mounted as in /media. The Software Sources tools in Ubuntu distributions can also add a CD as a Software source.

o If you cannot use the CD or USB memory stick as a Linux repository:

o Copy the packages (.repo or .deb files) from the Convey Software CD/DVD

(or other portable media containing the Convey packages) onto your Linux®

system (/tmp/rpms/ in the example below). Delete any Convey server

specific packages if you are installing on a non-Convey server, especially any Linux kernel packages.

o Login as root (typically with the su command), and enter the following

commands for Red Hat or SUSE based distributions: cd /tmp/rpms

rpm –ihv convey*

or these commands for Debian based distributions, such as Ubuntu:

cd /tmp/rpms

dkpg --install *.deb

o The above steps should be repeated whenever a new release of any Convey Software Product is required, except the command should be rpm –uhv package-file-name

or dpkg –list | grep convey # lists installed convey packages by

name dpkg --remove package-name # not the package-file-name dkpg --install package-file-name

o Each package-file-name to be installed must be spelled out completely, including the .rpm or .deb suffix, but a package-name does not include the

version number, the architecture, or the .rpm/.deb suffix


5 Booting and System Management

5.1 Overview

This chapter describes the process of booting the system, initialization of the Convey coprocessor, and management of the system.

5.2 Powering Up the Convey Server When the server is first plugged in, the blue Chassis Identity light will begin blinking. It will blink for about one minute. Once it has stopped blinking, you can push the front panel button to power up the Intel server. Do not power up the Intel server until the blue Chassis Identity light has stopped blinking.

5.3 First Power-Up and Boot

The Convey System comes pre-installed with Convey Linux, Convey software products, and basic system configuration.

5.3.1 Intel Remote Management Module

If your system is configured with the Intel Remote Management Module (RMM), then this is the recommended interface (console, keyboard, and mouse) for initial system power-on and configuration. The MAC address for the RMM will be on a label on the outside of the server.

The Intel website, intel.com, provides manuals and information about the RMM and configuration utilities. The psetup(1) utility that can be downloaded from the Intel website is useful in attaching to the RMM and assisting in the initial configuration. The psetup(1) utility identifies all RMMs on the local subnet and presents them based on their MAC address.

A link to the psetup utility is available on the Convey Support website, conveysupport.com. Click on the “Convey Hardware Help” tab on the top menu bar.

The RMM is initially configured with DHCP and the default user/password is „admin/password‟.

5.3.2 First Boot

The system is configured such that on the first boot at your site, the system enters a graphical configuration menu. This allows initial configuration of several aspects of the system. Some of the configuration tasks available are:

Change the password for the root user.

Add an initial user to the system (not root) and/or setup NIS.

Setup firewall rules.

Configure the network interface.

Configure date/time, time zone, and Network Time Protocol.

Install additional software packages via CDs.

If at some point later you would like to invoke this graphical configuration again, login as root and execute the following commands:

chkconfig firstboot on

touch /etc/reconfigSys


reboot

5.3.3 Manual Configuration

The following configuration updates are required at a minimum to place the system in general operation and connectivity to your network if you are not using the FirstBoot reconfiguration

interface just described in the previous section. Login as root and perform the following actions.

Select a password for super-user and update using the passwd(1) utility.

Select a host-name for the machine and update the HOSTNAME variable in the file

/etc/sysconfig/network.

Select an IP-ADDRESS for the system and update the IPADDR variable in the file

/etc/sysconfig/network-scripts/ifcfg-eth0.

Add a host-name and IP address entry to the file /etc/hosts.

If you are enabling a serial connection to the machine, add the following boot options to the kernel command line in the file /boot/grub/grub.conf.

o earlyprintk=serial,ttyS1,115200 console=ttyS1,115200

Reboot the system.

5.4 Boot Options

The Linux operating system is rich with boot options to control many aspects of system configuration. The goal for the Convey system is to provide as seamless as possible system initialization. Toward that end, a properly installed Convey system is designed with a set of defaults that allow the system to be fully configured and functional without any additional boot options. The system defaults are as follows:

The Convey coprocessor is fully enabled.

To the extent possible based on the installed physical memory on the Host server, all memory on the coprocessor is configured and accessible.

The memory interleave for the coprocessor defaults to 31-31 interleaving.

The page size for the coprocessor is set to the maximum available based on board configuration.

A 2 node NUMA configuration is established for memory domains by the kernel. Node 0 is the Host server and Node 1 is the Convey coprocessor.

Address extension mode is configured as fully-mapped.

The Convey boot options follow similar syntax to most standard Linux options. All Convey boot options are preceded by the keyword convey followed by a comma separated list (no spaces) of

keyword-value pairs. Note, keywords do not have to have an associated value. All keywords and values are lowercase. For example:

convey=keyword1=value1,keyword2,keyword3=value3

Multiple ‘convey‟ boot option tags are also permitted:

convey=keyword1=value1 convey=keyword2 convey=keyword3=value3

The following Convey boot options are provided to allow system administrators detailed control over the coprocessor configuration.


5.4.1 enable/disable

The enable/disable keywords control the discovery and initialization of the Convey

coprocessor. By default, the coprocessor is enabled. The disable option results in the

coprocessor not being initialized or available by the system. The memory on the coprocessor board is not configured. All other Convey boot options are ignored.

5.4.2 caemode

The caemode keyword controls the behavior of the Convey Address Extension mode. There are

currently 2 options available: fully-mapped (default) and dynamic-mapped. The caeinfo(1) command provides detailed information on the address extension configuration. Examples are:

convey=caemode=fully-mapped

convey=caemode=dynamic-mapped

5.4.3 memory

The memory keyword allows the administrator to restrict the amount of memory configured on the

coprocessor. By default, the system will configure all coprocessor memory up to limits that may be imposed by the physical memory configuration of the Host server. The option takes a single value that is the amount of memory to configure for the coprocessor. This can be expressed as a decimal value, hex value, or shorthand memory-suffix notation (k (kilobytes) = 1024; m (megabytes) = 1048576; g (gigabytes) = 1073741824). Examples are:

convey=memory=0x200000000

convey=memory=2g

convey=memory=1024m

convey=memory=0

The last example is a valid technique to configure the coprocessor with no local memory.

5.4.4 window

The window keyword allows the administrator to set the amount of Host memory allocated to the

memory window. The window size is constrained to a minimum of 4G and a maximum equal to the size of coprocessor memory. The window size must allow a minimum of 4G of free memory to be available on the Host when the configuration is complete. Currently, the window size must be a multiple of 1G bytes. The window size can be expressed as a decimal value, hex value, or memory-suffix notation. Examples are:

convey=window=0x200000000

convey=window=8g

convey=window=8192m

5.4.5 pagesize

The pagesize option allows the administrator to override the architectural maximum page size of

the coprocessor. This value will be ignored if it exceeds the architectural maximum page size for the installed board. Once the system is booted, the configured page size can be queried using the cnyinfo(1) command. The page size value can be expressed as a decimal value, hex

value, or memory-suffix notation. Examples are:

convey=pagesize=256k

convey=pagesize=1m

convey=pagesize=4096


5.4.6 interleave

The interleave option allows the administrator to configure specific interleave schemes for the

coprocessor. The coprocessor interleave can only be set at boot time and is configured during discovery and initialization. There are currently 2 types of interleave supported: binary and

3131. Examples are:

convey=interleave=binary

convey=interleave=3131

The coprocessor interleave can be queried using cnyinfo(1).

5.5 Coprocessor Modules

The initialization and configuration of the Convey coprocessor as a seamless part of the system is accomplished by two loadable Linux system modules. The CNYSYS module and the CNYMP module

are dynamically loaded at boot time and perform all discovery and initialization functions. The command lsmod(1) displays all currently loaded modules in the system including the Convey

modules. The following sections describe in detail the function of these modules.

5.5.1 CNYMP Module

The first Convey module to load during boot is CNYMP. This module provides out-of-band

communication with the management processor (MP) on the Convey coprocessor board. This communication is provided over a dedicated PCIE link with the MP. If the CNYMP module fails to

load and initialize, the system will boot but the coprocessor will be inaccessible. The CNYMP

module provides the following general features:

Discovery and initialization of the MP PCIE link

General I/O communication path with the MP

Supports remote login for field personnel

Transport path for dynamic personality loads and coprocessor firmware

Error handling and logging between the Host server and the MP

5.5.2 CNYSYS Module

The CNYSYS module loads immediately following the load of the CNYMP module. The CNYSYS

module depends on the CNYMP module for communication with the coprocessor management

processor (MP). If the CNYMP module fails to initialize then the CNYSYS module will also fail. Failure

of the CNYSYS module to initialize correctly results in the system booting without coprocessor

access. The main functions of this module are:

Discovery and initialization of the Convey coprocessor

Setting the coprocessor memory interleave

Configuration of coprocessor memory into the system

Configuration of the coprocessor interrupt path

Management of the personality cache on the MP

Handling coprocessor faults and traps

Coprocessor allocation policies (attach/detach)

Debug control (GDB save/restore context)


As just mentioned, one of the key functions of CNYSYS is configuration of coprocessor memory into

system. Coprocessor memory is configured into Linux and is accessible by all the standard memory management policies. For example, x-86 applications requiring large amounts of memory will spill over into coprocessor memory based on the NUMA system policies configured and not anything unique to the Convey system. The memory on the coprocessor board will be part of the memory statistics in /proc/meminfo.

From an application perspective, the CNYSYS module provides the allocation polices for acquiring

access to the coprocessor. The Convey Programmer’s Guide provides details of these policies and programming models. But it is important to note that the Convey coprocessor is modeled as an extension to the x-86 instruction set and not a separate stand-alone processor. For that reason, the coprocessor will not show up in the file /proc/cpuinfo as an independent processor.

5.6 Kernel Configuration

This section describes the basic kernel configuration for the Convey Linux operating system. The Convey Linux kernel was developed based on the Centos 5.4 kernel source base. Centos5.4 is a 2.6.18 released Linux kernel with extensions and fixes.

The overriding goal for the Convey Linux development was maintaining complete binary compatibility. This extends to middle-ware, system call interface, tools, drivers, and loadable modules. There are no changes to any system libraries or header files.

The kernel configuration has been tailored for the Intel Shoffner platform and its devices. There are several configuration selections made specifically for the Convey platform and its extensions. Some of the key kernel configuration options are:

Kernel crashdump support

64-bit Intel Processor (32-bit kernel is not supported)

No forced preemption (server platforms)

NUMA (multi-node Non-Uniform-Memory-Access; 8 node maximum)

Sparse Memory design

Maximum CPUs (8)

Timer Frequency (1000)

Kernel Profiling

The Convey Linux kernel has been fully qualified with the configuration described. Changes to this configuration, addition of drivers and/or modules, or specification of complex boot options may result in conflicts within the system and untested code paths.

5.7 Convey System Services

A service, as defined under Linux, is a shell script in the /etc/init.d directory. The Convey

platform supports a new service: convey. The convey service supports the full set of service

commands (start, stop, status, restart, reload). In general, the convey service is responsible for

the following events:

Setting of Convey specific sysctl(1) variables

Setting of the /proc/sys/kernel/printk level

Load and initialization of the CNYMP module

Load and initialization of the CNYSYS module


Configuration of the management processor

The utility, chkconfig(1), provides an easy to use interface to query the status of a service and

the run levels it is enabled for. For example, the following command,

chkconfig –list convey

will display the following line indicating the run levels that will execute this service.

convey 0:off 1:off 2:on 3:on 4:on 5:on 6:off

This indicates that the convey service is enabled for run levels 2, 3, 4, and 5. See the man page

for chkconfig(1) for a description of managing these run levels.

The service(8) utility provides an interface to execute service commands (stop, start, etc.)

against a specific service. The most frequent use of this command is to query the loaded state of the Convey modules.

service convey status

5.7.1 Convey Configuration Files

This section describes the Convey specific configuration files associated with the services just described. The Convey configuration files are maintained in the directory /etc/convey on Convey

installed systems. Each configuration file has a Convey installed version and may have a local site-specific version. The local site-specific configuration files can override or extend Convey configuration options.

The current configuration file set is:

modules.conf Defines the device names, permissions, owner, and group for the CNYSYS and CNYMP modules.

modules_local.conf Local site definitions that override the installed modules.conf.

mp.conf Configuration parameters for the Convey management processor. This includes hostname, Ethernet settings, console settings, and logging configuration.

mp_local.conf Local site definitions for management processor. This will typically include the IP/Mask for the management processor Ethernet.

sysctl.conf The format of this file is identical to /etc/sysctl.conf and contains Convey

specific sysctl variable settings.

sysctl_local.conf Local site sysctl variable settings.

yum.conf This file is referenced from /etc/yum.conf and contains Convey specific

yum(8) configurations. No local version of this file is defined.

5.8 Coprocessor Sanity Test Program

The program /opt/convey/sbin/cnydiag.aebase exercises basic functionality of the Convey

coprocessor, including memory access, coprocessor scalar instructions, internal datapaths in the coprocessor, and an application engine vector instruction. The coprocessor must be unattached,


and booted with “binary” memory interleave. If the server is booted with 31-31 memory interleave enabled, the vector test will fail, but the earlier tests should all pass.

5.9 When to Shutdown the Convey Server

If the coprocessor is attached by a process but appears to be hung (the application won‟t abort/exit), or when the Convey coprocessor is not attached (see the man page for cnyps) and

you cannot run the sanity test program successfully, please follow these steps:

Log onto the Convey server as root, and run /sbin/shutdown –r 1 rebooting to reset the coprocessor

After the host finishes rebooting, try running the sanity test program again. If it still fails, please contact Convey Customer Service.

5.10 Activating a Coprocessor Recipe

Installing a recipe package does not cause the coprocessor to use that recipe.

A Convey server can have many recipes installed, but only one is “active” at a time. To activate a particular recipe, Convey provides two tools:

/opt/convey/sbin/pickRecipe

o This script lists the available (installed) recipes, and lets the user select one to be activated.

/opt/convey/sbin/activerecipe recipe-filename

where recipe-filename is one of the “recipe...” files in /opt/convey/fpgaimages.

Important Note: After activating a recipe, you must shutdown the host system, wait for for two or more minutes, and then power up the host.


6 Convey System Memory Management

6.1 Overview

This chapter provides high level details about the Linux operating system physical and virtual memory management for the Convey Coprocessor. The goal behind these changes is to provide high memory bandwidth and performance for the coprocessor. There are four concepts that form the basis of Convey memory system that will be discussed.

NUMA Memory organization

Memory Interleaving

Variable Page Size Support

Inter-Node Memory Migration

The Convey Reference Manual will be referred to frequently for general description of the system organization and the hardware architecture.

6.1.1 Intended Audience

This chapter is intended for system administrators and application designers. The goal is to provide sufficient details about Convey memory management to enable development of high performance applications. Administrators need to understand how the operating system manages the Convey memory architecture and presents this to applications.

6.1.2 Terminology

The following terms are used to describe various components of the Convey system as it relates to memory management.

Host memory This refers to memory physically installed on the Intel x-86 Server.

Coprocessor memory

This refers to memory physically installed on the Convey coprocessor.

Memory window This refers to the physical memory on the Host server that is reserved/used to enable coherent access to memory on the coprocessor.

6.2 NUMA Memory Configuration

Physical memory is populated on the commodity Intel motherboard and on the Convey Coprocessor board thus forming a classic Non-Uniform-Memory-Architecture (NUMA). The degradation in memory latency and bandwidth when accessing remote memory (non-local to the processor) is significant and must be considered in any coprocessor application.

The Convey Linux kernel manages the physical memory of the system by using kernel nodes to separately configure the physical memory associated with the Intel motherboard and the Convey coprocessor. By convention, the coprocessor node will be the highest numbered node in the system. For most system configurations, node 0 will be the Intel board and node 1 will be the coprocessor board. This also implies the Linux kernel is configured with NUMA enabled and a maximum node count of 8.

All available Linux techniques and tools are supported for managing a multi-node NUMA configuration. The /sysfs file system describes each node and its attributes. For example, the

file /sys/devices/system/node/node1/meminfo provide all the memory statistics for the


coprocessor node. The file numastat in the same directory provides statistics on page allocations

with respect to the NUMA configuration. The numastat(1) utility is available from the numactl

package. This outputs a nicely formatted display of the numastat information for all nodes in the

system. Finally, the file /proc/meminfo provides memory statistics for the entire system (both

nodes).

6.2.1 Physical Memory Configuration

The previous section discussed the memory configuration from a logical Linux perspective. The physical implementation is different due to the architecture of the Convey coprocessor. Physical memory installed on the coprocessor board is actually implemented as a cache of memory on the Intel server board. This design is an artifact of the Intel Front Side Bus (FSB) and providing a single cache-coherent address space for the system. The implications of this design are as follows:

Because coprocessor memory is a cache, a portion of host memory (less than or equal to the size of memory on the coprocessor board) is configured as coprocessor memory.

The Linux kernel will automatically reserve and configure host memory equal to the size of coprocessor memory if available (default). No boot options are required.

The Linux kernel requires a minimum of 4 gigabytes of physical memory for the host server. The remainder of physical memory may be configured as coprocessor memory (memory window) based on coprocessor memory size.

If insufficient host memory is configured, the memory boot option is used, or the window

boot option is specified, the host memory configured for the coprocessor can be less than the physical memory size installed on the coprocessor (even zero).

6.3 Address Extension Feature

The initial release of the Convey system only supported 16G of coprocessor memory and required that 16G of Host memory be reserved for access to the coprocessor (one to one mapping; i.e. fully-mapped). The Shoffner X-86 motherboard supports a maximum of 128G of

physical memory (16 x 8G DIMMS) that would have limited the maximum amount of memory supported on the coprocessor given the previous architectural rules.

The Convey Address Extension feature was developed to enable support for maximum memory configurations of the Host Intel server and the Convey coprocessor (128G Host memory; 128G coprocessor memory) without requiring the one to one mapping of Host memory to coprocessor memory. A region of Host memory (known as the memory window) is still reserved to enable accesses to coprocessor memory but it is not required to be a one to one mapping. This new configuration capability is known as dynamic-mapped.

The architecture now supports a hardware mapping table between the Host memory window and coprocessor memory that enables mapping a region of the memory window onto any region of coprocessor memory. Memory window regions can be remapped dynamically as the X-86 references different regions of coprocessor memory. The remapping process is managed by the operating system in a manner similar to virtual memory page faults. The caeinfo(1) utility provides detailed information on the hardware and software management of the memory window.

The kernel boot option, caemode, allows the system administrator to select between the 2 modes: fully-mapped, dynamic-mapped. The following sections describe each in detail.

6.3.1 Fully-mapped mode

Fully-mapped mode is the system default and provides a full mapping of the Host memory window onto coprocessor memory. In this mode, each byte of coprocessor memory must be mapped by memory on the Host. The hardware mapping table is initialized at boot time to a one


to one mapping of the Host memory window onto coprocessor memory. The size of the memory window is configured by the operating system at boot time based on the following constraints:

Boot option convey=memory=### sets an upper limit on the size of coprocessor

memory configured.

Boot option convey=window=### sets an upper limit on the size of the Host memory

window.

A minimum of 4G of Host memory is reserved for the system.

A successful configuration must result in 3-4G of Host memory being free and allocatable for the kernel and applications.

The following example demonstrates the configuration rules for fully-mapped mode. A system with 64G of Host memory and 128G of coprocessor memory without any boot options will configure a memory window of approximately 58G and coprocessor memory of 58G. This results in 70G of coprocessor memory not being configured. The 6G of Host physical memory remains for the operating system, system data structures, and 3-4G of free memory (this assumes 2-3G consumed by the operating system).

6.3.2 Dynamic-mapped mode

Dynamic-mapped mode must be specified via the boot option caemode. The size of the memory window is configured at boot time based on the following constraints:

Boot option convey=window=### sets an upper limit on the size of the Host memory

window.

The Host memory window defaults to a size equal to 25% of the size of configured coprocessor memory.

A minimum of 4G of Host memory is reserved for the system.

A successful configuration must result in 3-4G of Host memory being free and allocatable for the kernel and applications.

Using the example from the prior section, a system with 64G of Host memory and 128G of coprocessor memory would configure a memory window of size 32G. The full 128G of coprocessor memory would be configured into the system and available. 32G of Host memory remain for the operating system, system data structures, and applications.

Determining the size to configure the Host memory window is completely application specific. If the application has limited X-86 access of its data in coprocessor memory, then smaller windows should provide good performance. As the application increases its X-86 accesses or the accesses are very random/distributed, memory window region swapping will occur. The size of the memory window will affect the amount of swapping in this case.

6.4 Memory Interleaving

This section describes the supported memory interleaving schemes on the Convey coprocessor. The coprocessor supports two types of interleaving; binary and 31-31 as described in XXX. The coprocessor memory interleave is set at boot time and currently defaults to 31-31 interleave if no boot option is specified. See the chapter 5.4.6 interleave for a detailed description of interleave

boot options.

The interleave schemes are independent of the type of DIMMs installed in the coprocessor (standard DIMMs or scatter-gather DIMMs). The interleave scheme affects which memory controllers, DIMMs, and banks are accessed based on memory address. The performance obtained by the interleave scheme is a function of the memory access pattern of the application and is not discussed in this section.


The type of interleave configured is determined at boot time and cannot be reconfigured after the coprocessor has been brought online. The kernel performs a coprocessor discovery and configuration process during boot. The coprocessor attributes are logged into the file /var/log/messages that includes the configured interleave type. The configured interleave can

also be determined using the cnyinfo(1) utility or displaying the file

/sys/devices/system/node/node1/interleave/summary (note, this assumes the coprocessor is

node 1).

6.4.1 Binary Interleave

The binary interleave scheme is designed to provide near peak performance for unity stride applications. All available memory will be configured; there are no interleave holes in the physical address space.

6.4.2 31-31 Interleave

The 31-31 interleave scheme was designed to meet the following system requirements:

Provide as high a bandwidth as possible for all memory access strides, with particular focus on power of two strides.

Keep each memory line (64 bytes) on a single memory controller. This is required to simplify the cache coherency protocol.

Maintain the interleave pattern across virtual memory page crossings. This helps large strides where only a few accesses are to each page.

All virtual addresses must map to unique physical addresses.

The scheme uses a two level hierarchical interleave approach. The Convey memory system using Scatter/Gather DIMMs has 1024 memory banks. The 1024 banks are divided into 32 groups of 32 banks each. The first interleave level selects one of 31 groups of banks. The second interleave level selects one of 31 banks within a group. Note that one of the groups of the 32 groups of banks is not used. Similarly, one bank within each group of banks is not used. The prime number (31) of banks and groups of banks is used to maximize the sustainable memory bandwidth for as many different strides as possible. This skipping of banks results in a 6% loss of available memory and a decrease in peak memory bandwidth of 6%. A coprocessor with 16 Gbytes of physical memory and 31-31 interleave configured, will have an available physical memory of 15.04 Gbytes (a loss of 960 Mbytes).

6.4.3 31-31 Interleave Page Allocation

The 31-31 scheme is a 2 level interleave. The first level has a required mapping function and the second level is a best effort. If a physical page of the required mapping cannot be found on the coprocessor for a specific virtual address, then one of several actions may take place:

If the memory policy for the virtual memory region allows, the memory allocator will attempt to allocate a page out of host memory (x86 Intel server board).

If host memory is also not available or the memory policy does not permit off node allocation, then the process is pended and the paging daemon is woken to free additional memory on the node.

Note, host memory does not have the mapping requirement for 31-31 interleave. The mapping function only comes into play for allocation of coprocessor memory.

The cny3131(1) utility provides detailed information about allocation patterns, color utilization,

page order usage, etc. The „p-miss‟ column when displaying primary color statistics indicates

that a page of the correct mapping was not available. This would trigger one of the above


actions. Allocation of a host memory page or starting of the paging daemon will result in performance degradation for the coprocessor application.

6.4.4 31-31 Interleave Faults

The Linux system call interface and general programming model gives any application almost complete control of its virtual address space. For example, the mmap(2) system call permits the

application to specify the exact virtual address at which to map a new object. This feature is used heavily by the dynamic library loader but rarely by other applications. Use of this capability can result in invalid physical-to-virtual mappings for a coprocessor application with 31-31 configured. For this reason, it is strongly advised not to specify the mapping virtual address to mmap(2).

The Linux kernel makes an extreme best effort to avoid and prevent invalid 31-31 mappings. But given the virtual address space control an application has, there is not a 100% guarantee. For this reason, the coprocessor has a detection algorithm in the TLB walker that will detect an invalid page mapping and generate a SIGBUS to the application. Detailed information is dumped to the

system console and system logs. This information should be captured and sent to Convey Customer Support.

6.5 Variable Page Size Support

This section discusses the Linux operating system support for Convey variable page size support. The goals for providing variable page size support are:

Reduce the number of TLB misses and page faults by the coprocessor

Increase the utilization of the coprocessor TLB cache

Allow the coprocessor TLB to cover all physical memory

The Convey Reference Manual (Chapter 4) describes the architectural details behind the

coprocessor variable page size support. The system supports page sizes 4K to 4M and every power of 2 page size in between. The kernel aggressively allocates the largest page possible during the following events:

Page faults (these can be copy-on-write)

File system reads

Reads from the swap area

Data migration

The allocation of contiguous memory (large pages) is applied only to processes that are currently attached to the coprocessor. Host X-86 processes cannot benefit from the large page allocation feature of the coprocessor.

The allocation of large pages is applied only to Convey attached processes and is independent of the node the memory is allocated from. Because the coprocessor can access Host and Coprocessor memory, large pages in either memory domain improves TLB performance. Note, that due to the memory activity in the Host domain, the ability to allocate the maximum large page is reduced.

The /proc pseudo-filesystem implements several files that provide detailed information about a

process. From a memory management point of view, the files maps, numa_maps, and smaps contain

data on the virtual address space of the process. The data is organized around each memory mapped region with the address space. There are man pages for proc(5) and numa_maps(5) that

cover the format of this information. Because of the importance of page placement and page size on application performance, a new /proc file has been added for Convey systems. The file pmaps

contains detailed data on physical memory placement and page size for the entire virtual address


space. This is extremely useful when tuning coprocessor applications. See the man page pmaps(5) for more detail.

6.5.1 Maximum Coprocessor Page Size

The current architecture of the coprocessor is designed to support a maximum page size of 4M. The architecture data of the coprocessor is acquired by the kernel at discovery time during boot. The maximum page size can also be controlled at boot time (boot option) or dynamically at run-time. The boot option, pagesize, defines the maximum page size to be used for the coprocessor up to the architectural limit. Also, the maximum page size of the coprocessor can be queried and set by the file /sys/devices/system/node/node1/pagesize (assuming node1 is the coprocessor).

The following command allows setting of the maximum page size to 16k bytes.

echo 16k > /sys/devices/system/node/node1/pagesize

This change takes effect immediately on the next memory allocation for an attached coprocessor application. This can be useful when diagnosing system problems or performance anomalies.

6.5.2 Linux HugeTLB Feature

The HugeTLB feature in Linux enables use of X86 2Mbyte pages by applications and is fully supported by the Convey Linux operating system. The coprocessor can access HugeTLB pages mapped into an application. However, because this feature is more restrictive than the coprocessor page size support and due to the complexity of maintaining 31-31 interleave mappings, coprocessor memory cannot be reserved into the HugeTLB memory pool. In addition, the HugeTLB feature does not currently have an API to specify the node from which to reserve memory for the pool. HugeTLB memory will always be allocated out of Host memory.

Applications being ported to use the Convey coprocessor that depend on the HugeTLB feature will execute correctly. The issue is one of performance. If the application port results in the coprocessor heavily accessing virtual memory that maps HugeTLB pages, changes will need to be made to allocate coprocessor memory or to be able to take advantage of page migration. This is covered in detail in the Convey Programmers Guide.

6.6 Memory Migration

This section describes memory migration within the Convey system architecture. The Convey system is configured as two NUMA nodes typically numbered 0 and 1. Memory migration allows the moving of the physical location of pages between nodes while the process is running. The virtual addresses the process sees do not change but the physical location of pages mapped to those virtual addresses do. The goal of memory migration is to reduce the latency of memory access by moving pages near to the processor accessing the memory. This is critical toward achieving maximum performance of the application.

It is important to note that page migration is a best effort by the kernel. All references to a page must be removable at the time of migrate or the page is dropped from the migration list. Examples of this are pages that are locked for I/O, shared pages that still have references from other processes, or dirty pages in the process of being cleaned. Other factors include lack of physical memory on the target node of the migration.

The Convey Programmers‟ Reference Guide describes the library API available for data management. The routine cny_migrate_data() provides a user interface for migrating virtual

regions within a process to/from the coprocessor. This is built on top of standard Linux system calls and is provided as an easy to use interface that is cognizant of the Convey coprocessor system architecture. The routine cny_memory_locale() assists in identifying the location of a

specified virtual region within the application. The information provided by cny_memory_locale()

may not be completely accurate if the memory has not been referenced following a migration.

When the routine cny_migrate_data() is applied to a memory region that is unpopulated (empty), the pages are not faulted in by this call. Instead, the memory policy for the region is updated with


the target node of the migration. Later, the pages will be faulted into memory of the target node thus avoiding a memory to memory copy. However, the cny_memory_locale() will still return empty until the pages are faulted into memory.


7 Personality Management

7.1 Personality Signatures

The Convey Coprocessor supports multiple personalities, and multiple versions of each personality. Although the user can exercise substantial control over which particular personality a given program will use, several tools are available for a system/site manager to guide users towards particular personalities and versions thereof. Note that there are multiple personality upgrading steps, only two of which are described here. A full discussion of personality upgrading is available in the Convey Programmers Guide. A complete signature identifies a particular version of a coprocessor personality. Signatures are composed of five numeric values, such as „2.1.1.1.0‟. The five fields (in order) are:

personality number

major version number

minor version number

hardware model number

canonical instruction set version number

The personality number indicates the type of instructions implemented in a particular personality, such as single precision vector, double precision vector, etc.

The major and minor version numbers indicate various updates to the personality, including bug fixes, performance improvements, and new instructions.

The hardware model number indicates which version of the Convey coprocessor this signature supports. Signatures that are identical except for the hardware model numbers are compatible, and an object file compiled for a particular signature can be executed using a compatible signature.

The fifth numeric value, the canonical instruction set version number, is reserved for future use. If specified as part of the signature, it must be zero.

7.2 Default Base Signature

After installing the convey-per-tools package for the first time (or anytime thereafter), the system administrator may update the file

/opt/convey/personalities/DefaultBasePersonality

This file contains a single default base signature for the system. This default base signature is used to fill in some missing parts of a user specified signature when compiling or assembling source code. If the user specifies a zero (or does not specify a value) for the personality number or the hardware model number, the corresponding value from the default base signature will be used. The major and minor version numbers in the default base signature are not used for anything. This „upgrading‟ of the user specified signature allows the system administrator to:

select which personality number to use when the user does not specify one, such as „single_precision‟,


select which hardware model number to use when the user does not specify one, typically the architecture version number of the Convey coprocessors installed on the Convey server this program will be run on, and

If the personality number in the default base signature is zero or an invalid personality number, the user must specify a valid personality number when compiling or assembling source code.

Convey recommends that the default value for the hardware model number be set to the value for the Convey coprocessor the program will be run on. Although signatures with different hardware model numbers are compatibile, the compilers use the hardware model number to optimize code for better performance.

Users of the Convey compilers and assembler will typically specify a personality number, and possibly a major and minor version numbers, but will not usually specify a hardware model number.

At the time this users guide was written, the default signature installed the first time the convey-1st package is installed was 2.1.1.1.0, which specifies the single precision personality number, and a hardware model number of 0 (the first version of the Convey coprocessor released. Future installations of Convey packages will not overwrite this file, even if the convey-per-tools package is uninstalled.

7.3 System Default Signatures

After installing the convey-per-tools package for the first time (or anytime thereafter), the system administrator may update the file

/opt/convey/personalities/SystemDefaults

This file contains zero or more signatures. The system administrator can modify this file to establish default signatures for each personality. These personality specific default signatures are used to „upgrade‟ a user supplied partial signature. When the user specifies a partial signature, that signature is upgraded (if possible) to match an actual installed personality. Typically, the major and/or minor version numbers are upgraded. See the upgrading discussion in the Convey Programming Guide for more information.

For example, if the SystemDefaults file contains 2.2.3.1.0, and the user specified “sp” as a

partial signature when compiling a routine (“sp” maps to “2” in the Convey Personality Database), then the partial signature will be upgraded to 2.2.3.x, where the value (x) for the hardware model number is obtained from the system‟s default base signature value (see above).

7.4 Convey Personality Management Script

After installing the convey-per-tools package for the first time (or anytime thereafter), the /opt/convey/sbin/cpm script may be used to manage personality signatures in the personality

database. The cpm script allows the system administrator to disable and enable particular

personalities. A disabled personality, although still installed on the system, cannot be used to compile, assemble, or execute coprocessor code. The cpm script can also be used to list installed

signatures, and get additional info about a particular signature.


Usage: /opt/convey/sbin/cpm action [personality list]

action What it does

list Lists the specified signatures, if installed.

$ cpm list 2.1.1.1.0 1.1.1.1.0 produces

Personality Compiler License Coproc Nickname(s)

Number Support Status Support

--------------- ------- ------------------ ------ --------------------------

1.1.1.1.0 Yes Permanent No double dp double_precision

double_vector dp_vector

2.1.1.1.0 Yes Permanent No single sp single_precision

single_vector sp_vector

If no signatures are specified, all signatures in the personality database will be listed.

A “compiler support” value of “yes” means the compiler can compile code for that signature. A “coproc support” value of “yes” means an executable requiring that personality may be executed on this system. The “license status” field shows if that personality has a valid license on this system. A valid license for a personality is required in order to exeute a program using that personality.

info More verbose than „list‟, the „readme‟ file for the specified signatures is also included.

enable The specified signatures are enabled.

disable The specified signatures are disabled. A disable signature cannot be used by the compiler.


8 Recovery DVDs

Convey provides a recovery DVD with each Convey Server shipped (1 set of DVDs per cluster or head node). Additional DVDs can be ordered through Convey‟s Support group.

This recovery DVD allows the system to be restored after a catastrophic disk or system failure, with a modest amount of user interaction.

WARNING: The System Recovery DVD will completely erase the systems main hard drive, and all data on that drive will be lost.

Note: the recovery process for non-head nodes (Convey server nodes WITHOUT a DVD drive) is described later, near the end of this chapter.

8.1 How to use the Convey System Recovery DVD

The Convey System Recovery DVD installs CentOS plus Convey platform extensions and enhancements.

In general, recovery / installation from the DVD involves the following steps:

System preparation including:

o Attaching a graphical (PC compatible) display to the VGA adapter, and attaching a USB keyboard /mouse, or

o Logging into the previously configured Intel RMM console for the host server

Booting off of the DVD.

Reviewing and possibly modifying the installation parameters.

Allowing the configured installation to install.

Removing the DVD and booting off of the installed disk.

Configuring the system via CentOS‟s post install configuration menus.

After the system is completely installed and running, the second DVD may be used to install various additional Convey products, including compilers, personalities, …

8.1.1 Preparation

Convey recommends that Convey OS installs be performed using the CentOS GUI installation program. The GUI installation program offers the full functionality. To use the graphical installation program, either:

attach a PC compatible graphical display, USB keboard, and USB mouse to appropriate ports located on the back panel of the Convey sever, or

Log into the previously configured Intel RMM host server console from a web browser

8.1.2 Boot from DVD

Install the Convey System Recovery DVD into the servers DVD drive. If necessary, configure the system BIOS to boot from the DVD and allow the system to boot. A successful DVD boot will result in a CentOS splash sceen being displayed on the console with the following text menu options:

- To install or upgrade in graphical mode, press the <ENTER> key.


- To install or upgrade in text mode, type: linux text <ENTER>

- Use the function keys listed below for more information.

[F1-Main] [F2-Options] [F3-General] [F4-Kernel] [F5-Rescue]

To abort the installation process at this time, reboot your machine and then eject the boot media DVD.

To continue with the installation, press enter and allow the system installer to load.

8.1.3 Configure the Installation and Installing the OS

After loading the Anaconda CentOS system installer, the CentOS welcome menu will be displayed; proceed through the following menus to configure the installation. Please note that network, firewall, system name, and a few other items will be configured after the installation, during the first hard disk boot:

1. Click Next to enter the disk partitioning menu.

2. Partition the disk as to your local requirements; a recommended partitioning is as follows:

a. 2G /boot partition

b. 64G Swap partition

c. The remainder of the disk allocated to the “/” partition.

3. Configure the time zone

4. Create a root password

5. Add additional packages; all recommended and required Convey base platform RPMs are already selected

6. Click next to arrive at the final installation menu; you may abort the installation at this

point by rebooting your machine and then ejecting the boot media.

7. Click next and wait for the installation to complete.

When the installation is complete the DVD will be ejected (remove the DVD from the drive) and the system will reboot. Upon a successful disk reboot the system will enter the CentOS “first boot” configuration GUI; proceed through these menus to configure the Convey Server according to your local requirements.

8.1.4 System Configuration

Additional system configuration occurs during the first, post install, boot. A GUI steps through the following menu options; complete details of the menu options can be found at http://www.centos.org/docs/5/. This is a summary of the configuration menus:

1. Cent OS welcome menu: Click Forward

2. Keyboard configuration menu: Select the appropriate keyboard type. Click Forward when complete.

3. Root Password menu: Create a root password. Click Forward when complete.

http://www.centos.org/docs/5/


4. Network configuration menu: The default configuration is DHCP. Click Change Network Configuration to

configure static IP addresses; a pop-up menu will be displayed that provides IP, DNS, /etc/hosts, and other configuration panels. Note if you click outside of the network configuration popup then the window manage will push the network pop up to the background; use the Alt+Tab keyboard short cut to bring the network configuration

menu to the foreground. Click on the popup X button at the upper right to exit the

network configuration. Click Forward when complete.

5. Firewall configuration menu: The firewall is disabled by default; configure as appropriate for your installation. Click Forward when complete.

6. SELinux configuration menu: SELinux is disabled by default; configure as appropriate for your installation. Click Forward when complete.

NOTE: if you enable SELinux, please see the SELinux FAQ at www.conveysupport.com/protected/faqs for more information about configuring SELinux (required in order for the Convey compilers to operate correctly).

7. Kdump configuration menu: kdump is enabled by default; Convey recommends that it remain configured with 128MB of reserved memory. Click Forward when complete.

8. Timezone configuration menu: Select the appropriate timezone. Click Forward when complete.

9. Date and Time configuration menu: Verify the current time and date; adjust as necessary. Use the NTP configuratioin tab to enable and configure NTP. Click Forward when complete.

10. Create User configuration menu: Define additional local users. Select the Use Network Login button to configure NIS.

Click Forward when complete.

11. Sound Card configuration menu: Click Forward to skip.

12. Additional CDs menu: Click Finish to complete the basic Convey / Centos server platform installation.

13. After several seconds, the CentOS graphical login menu is displayed.

8.2 Fixing the Convey SW Repository Files

You need to add the Convey provided username and password (for web access) to every Convey

SW Repository file in /etc/yum.repos.d.

Edit every file in /etc/yum.repos.d, whose name begins with “convey-“, and insert your company‟s Convey web access username and password into the URL entry. For example, if your site‟s username / password was abc / xyz, then a URL line that initially looked like

baseurl=http://conveysupport.com/protected/install/hc1-1/convey_sdk/x86_64/RPMS/

would look like this after editing:

http://www.conveysupport.com/protected/faqs


baseurl=http://abc:[email protected]/protected/install/hc1-1/convey_sdk/x86_64/RPMS/

8.3 Adding Convey Provided License Keys

After restoring the Convey OS, various optional products need to be installed, some of which require a Convey provided license.

You may request a copy of the license(s) from the Convey support group (email [email protected] from an email address associated with the organization that purchased the Convey server please).

8.4 Installing the Remaining Convey Software Products

See section Installing Convey Software on Cross-Development Systems. This section

applies to Convey servers as well as cross-development systems.

The Convey host system should now be fully configured and operational.

8.5 Checking the Management Processor

After following all the above steps, run the command: /opt/convey/sbin/mpinfo

If the Management Processor (MP) is communicating with the host system, something similar to the following will be appear:

# /opt/convey/sbin/mpinfo

Hostname : your hostname

Active recipe : recipe.beta-0.9.3-09_07_08

Next recipe : none

Active firmware : firmware-0.0-0.0.1124-1

Committed firmware : firmware-0.0-0.0.1124-1

Trial firmware : none

U-Boot : 1.3.0-rc3 (May 13 2009 - 16:03:38)

Configured eth0 : static 192.168.22.182/24:

Current eth0 : 192.168.22.182/24:192.168.22.5 up

Console baud : 115200

Xinetd : on

Clock Frequency : 150.000 MHz

If the management processor appears to not be communicating with the host system, the following steps should correct the problem:

Do a yum update convey-\* (as root) to update all of Convey‟s packages to the latest

available (and known to be consistent) versions. Alternatively, explicitly install the specific versions of the convey-kernel (or convey-kernel_beta), the convey-mp_fw, and the convey-uboot packages that you were using before the system was restored. If convey-mp_fw or convey-uboot was one of the packages updated by the above command, you must also shutdown the host as shown below

Shutdown the host (/sbin/shutdown -h now bye), wait at least two minutes, and

power up the host via the RMM or the host systems power button.

Try the mpinfo command again. If the MP is still not communicating with the host

system, please contact Convey support ([email protected]).




8.6 Updating the Server to the latest SW Versions

If you didn‟t do a yum update as described in the preceeding sub-section, you may wish to upgrade all the installed Convey SW to the latest released versions. The command

yum update convey-\* (logged in as root)

will upgrade all installed Convey SW packages, and may require a shutdown/reboot.

8.7 Restoring a Convey Server without a DVD drive

Contact the Convey support group ([email protected]) for detailed instructions.



9 Customer Support Procedures

Please use email or the toll free phone number for simple questions. The issue tracker web interface is ideal for reporting bugs and requesting enhancements to Convey products.

Email [email protected] : Convenient for quick simple questions.

Issue Tracker Web Interface This is the preferred contact method when additional information needs to be passed on to Convey, such as source files, screen dumps, etc. If you already have a Convey Issue Tracker username and password, browse to rt.conveysupport.com.

If you need an issue tracker username and password, browse to conveysupport.com/issuetracker.html and follow the directions there.

Call Convey‟s toll-free number 1-866-338-1768

Select the “Customer Support” extension (“2” when this document was written). If you are transferred to voicemail, please leave a message with your name, your company‟s name, your phone #, and a brief description of the problem. If you prefer to receive an e-mail response, please leave your e-mail address. This number will usually be answered 9am – 5pm Central Time (USA), if a customer service representative is available.

A customer service representative will typically acknowledge all email, voicemail, and new issues reported via the issue tracker within 24 hours, Monday-Friday (except holidays).

For extremely urgent issues, please send an email to [email protected] and call the toll-free #.


http://rt.conveysupport.com/

http://conveysupport.com/issuetracker.html



10 Licensing

10.1 Convey Licensed Software

Beginning with the April 2010 release, all of the Convey products may be installed on a Convey Server or customer provided cross-development system, but a license is required to use some of the features and capabilities provided by those packages.

For a Convey server, the required licenses are typically installed before the server is delivered.

For a cross-development system, and for an add-on order for new software for a Convey server, the necessary license(s) are typically e-mailed to the designated contact, along with instructions for installing those licenses.

The following are examples of what features and capabilities are licensed:

Using a Convey provided vector personality when executing on the coprocessor

Use of the Convey coprocessor simulator

Use of the Convey compiler suite

10.2 Old Licensing Scheme

Prior to April 2010, the packages containing the Convey personality tools, the FPGA personality images, the personality development kit, some of software development utilities, and the Convey Math Libraries were licensed. Those older packages can only be installed if a license key for that package is already present in the file /opt/convey/licenses. Usually, Convey Customer Support

would have emailed the license keys to each customer‟s designated contact person. Send e-mail to [email protected] for help with license keys.

The is no need to remove the /opt/convey/licenses file when installing products that use the

new licensing mechanism described above. You may wish to install older versions of some Convey packages someday, and the /opt/convey/licenses file may be required to install those

packages.



Appendix A - Installing YUM on older SUSE™ Distributions

Start up yast, then

in the Software part, pick Software Management, then

in the search bar, type yum (and click search), then

select the package named yum, then

click on accept (you may have to accept other packages due to dependencies)

On Suse distributions that support zypper (openSUSE 10.3 and later), please use zypper, not

yum.


Index . 31-31 interleave 29 apt-get 12 binary interleave 29 boot options 20 booting 19 configuration files 24 Convey software distribution CD 17 Convey System Services 23 customer support 41 default base signature 33 default signatures 34 distributions, Linux 10 First Boot 19 installation, manual 16 Installing Convey Software 7 installing convey software without an internet

connection 17 Intel Remote Management Module 19 kernel configuration 23 licensing 42 linux distributions 10

memory interleaving 28 memory migration 31 NUMA Memory Configuration 26 package version numbers 15 personalities 33 personality upgrading 33 Powering Up the Convey Server 19 Recovery DVDs 36 repository description file 7 RMM 19 SDK/Optional Products DVD 39 signature 33 software repository description files 7 system memory management 26 System Recovery DVD 36 updating Convey software packages 14 variable page size 30 wget 16 yum 12 zypper 12