1

ESI CFD Inc. 6767 Old Madison Pike, Ste. 600

Huntsville, AL 35806 Phone: (256) 713-4700 Fax: (256) 713-4799

Software Support: support.cfd@esi-group.comSoftware Sales: cfdinfo@esi-group.com

CFD Products V2009.2 Getting Started Guide

2

About ESI GROUP ESI Group is a pioneer and world-leading provider of digital simulation software for prototyping and manufacturing processes that take into account the physics of materials. ESI Group has developed an entire suite of coherent, industry-oriented software applications to realistically simulate a product's behavior during testing; to fine tune the manufacturing processes in synergy with the desired product performance; and to evaluate the environment's impact on product usage. ESI Group's product portfolio, which has been industrially validated and combined in multi-trade value chains, represents a unique collaborative virtual engineering solution, known as the Virtual Try-Out Space (VTOS), enabling a continuous improvement on the virtual prototype. By drastically reducing costs and development lead times, VTOS solutions offer major competitive advantages progressively eliminating the need for physical prototypes. About ESI CFD ESI CFD is a technology leader in the field of advanced computational fluid dynamics simulation software backed by more than 20 years of research based knowledge throughout a wide range of industries. ESI CFD’s broad range of products and services provide all of the necessary tools for advanced multiphysics analysis in a virtual prototype environment, significantly reducing time and expense through comprehensive up-front modeling and simulation. Key focus areas include aerospace, automotive, biomedical, fuel cells, MEMS, microfluidics, plasma and semiconductor.

3

Revision Information The information in this guide applies to all current ESI CFD products until superseded by a newer version of this guide. Published: May 2009 Disclaimers The documents and related know-how herein provided by ESI Group subject to contractual conditions are to remain confidential. The CLIENT shall not disclose the documentation and/or related know-how in whole or in part to any third party without the prior written permission of ESI Group. Copyright Information © copyright 1989-2009 ESI-Group, Inc. All rights reserved. This ESI Group documentation is the confidential and proprietary product of ESI-Group, Inc. Any unauthorized use, reproduction, or transfer of this manual is strictly prohibited. This documentation is subject to limited distribution and restricted disclosure. CFD-ACE™, CFD-ACE+™, CFD-CADalyzer™, CFD-VIEW™, CFD-GEOM™, SimManager™, CFD-VisCART™, CFD-Micromesh™ and CFD-FASTRAN™ are registered trademarks of ESI-Group. Portions of this product are owned by third party software vendors.

4

5

Table of Contents

CHAPTER 1: INTRODUCTION ...................................................................................... 9

CHAPTER 2: ESI CFD PORTAL.................................................................................. 11 2.1. Obtain Company Identification Number (COIN) ........................................................................................11 2.2. Register for a Portal Account.......................................................................................................................11 2.3. Receive Approval ........................................................................................................................................12 2.4. Account Renewal.........................................................................................................................................12 2.5. Portal Resources ..........................................................................................................................................13

CHAPTER 3: ESI CFD SOFTWARE ............................................................................ 15 3.1. Get to Know ESI CFD Software..................................................................................................................15

3.1.1. Software Release .............................................................................................................................15 3.1.2. Supported Platforms ........................................................................................................................15 3.1.3. Software Packages ...........................................................................................................................18

3.2. Obtain ESI CFD Software ...........................................................................................................................23 3.2.1. Which Platform? ..............................................................................................................................23 3.2.2. Which Software Package/Applications?..........................................................................................23 3.2.3. Request ESI CFD software on a DVD.............................................................................................24 3.2.4. Download ESI CFD software from portal .......................................................................................25 3.2.5. Download ESI CFD software from FTP site ...................................................................................27

3.3. Install ESI CFD Software – Windows.........................................................................................................29 3.3.1. Install from DVD.............................................................................................................................30 3.3.2. Install downloaded MSI packages ...................................................................................................31 3.3.3. Install downloaded ZIP packages ....................................................................................................32

3.4. Install ESI CFD Software – Linux/Unix......................................................................................................33 3.4.1. Install from DVD.............................................................................................................................34 3.4.2. Install downloaded EXE packages...................................................................................................35

CHAPTER 4: ESI CFD LICENSE ................................................................................. 38 4.1. Get to Know ESI Flexlm Licensing.............................................................................................................38

4.1.1. Node locked licenses .......................................................................................................................38 4.1.2. Floating licenses ..............................................................................................................................38

4.2. Obtaining a License .....................................................................................................................................39 4.3. Install License – Windows – Node Locked .................................................................................................40 4.4. Install License – Windows – Floating .........................................................................................................41 4.5. Install License – Linux – Node Locked.......................................................................................................43 4.6. Install License – Linux – Floating ...............................................................................................................44 4.7. Migration (Software and Licensing)............................................................................................................46 4.8. Troubleshooting and Tips ............................................................................................................................48

CHAPTER 5: ESI CFD PARALLEL.............................................................................. 50 5.1. CFD-ACE+ Parallel – Windows..................................................................................................................51 5.2. CFD-ACE+ Parallel – Linux .......................................................................................................................65 5.3. CFD-CADalyzer Parallel.............................................................................................................................76 5.4. CFD-FASTRAN Parallel – Windows..........................................................................................................77 5.5. CFD-FASTRAN Parallel – Linux ...............................................................................................................78 5.6. CFD-VisCART Parallel...............................................................................................................................79

6

CHAPTER 6: ESI CFD SUPPORT ............................................................................... 80

APPENDIX 1: INSTALLATION CHECKLIST ............................................................... 82

7

8

Chapter 1: Introduction Thank you for choosing ESI products for your CFD simulation needs. Typical "new" installations of ESI CFD products include the following steps:

1. Accessing ESI CFD Portal i. Receive Company Identification Number (COIN) ii. Register for an ESI CFD portal account

2. Software Installation and Configuration

i. Know which software/applications to install ii. Obtain ESI CFD software iii. Install software

3. License Installation and Configuration

i. Know which license is needed ii. Request license iii. Receive license iv. Install license v. Verify license/installation

A schematic of the process is shown in Figure 1.1. This document is designed to be a reference guide for all your installation needs. You can read it through or pick topics of interest using the PDF bookmarks.

Figure 1.1 Schematic of a "new" installation process

9

10

Chapter 2: ESI CFD Portal The ESI CFD portal has been designed for our customers and provides a wealth of information on ESI Group's CFD products. Apart from product downloads the portal also offers a treasure of information to increase user productivity. A customer login is necessary to access many areas of the portal. Therefore, as a new user, your first step is to get an ESI CFD portal account. This account is free for customers with a valid software maintenance contract. This chapter will take you through the process of obtaining an ESI CFD portal account and the resources you can expect at the portal.

Obtain Company Identification Number (COIN) Register for a Portal Account Receive Approval Account Renewal Portal Resources

2.1. Obtain Company Identification Number (COIN) A COIN is a 4 digit number (e.g. 0001) uniquely identifying a company or a department within a company and hence the products and services associated with it. ESI CFD portal users are tied to a Company Identification Number (COIN). You can find out your COIN in one of the following ways. Method 1: The COIN would be provided in the purchase acknowledgement/email sent by your ESI account manager immediately after the software purchase. If you were not directly involved in the purchase process, request this information from the appropriate purchase personnel in your company. Method 2: Send a request to support.cfd@esi-group.com with your company/department information and the name/email address of the person involved in the purchase process. 2.2. Register for a Portal Account Once you have the COIN, you can start the registration process at http://www.esi-cfd.com. Figure 2.1 shows a snapshot of the registration request you will see at the portal. You can choose the username and password you prefer. Your username will be visible to other users of the portal. Therefore, it is not recommended to use your e-mail address as the username. If you forget your password, please use the "Lost Password" link to receive a new password. Please note that customer support will not be able to provide you with a password. If you forget your username, please send an email to support (include the email-id that was used for registration) and we will be able to send you your username.

11

Figure 2.1 Registration for an ESI CFD portal account 2.3. Receive Approval Once you send the registration request, you will receive an email acknowledging the receipt of your request. You will confirm that you are the owner of the email address by clicking on the link in the acknowledgement email. After our administration team reviews and approves your request, you will receive a second email confirming your registration. This process is usually completed within 1-2 business days. This email will include details of your account and other information regarding software download and installation. You are now ready to login and start using the portal. 2.4. Account Renewal Your portal account is valid for the period of your maintenance contract and usually expires at the end of the term unless renewed. If you renewed your maintenance and still experience troubles logging in, send an email to support.cfd@esi-group.com and it will be corrected immediately.

12

2.5. Portal Resources Download Software* - for downloading the latest release and beta software versions for all supported platforms. Knowledge Base* - provides answers to some commonly asked questions and current passwords for ftp accounts. CFD Model Library - hosts over 120+ downloadable CFD models, all categorized by software tool, physics / features used, and industry area. CFD Paper Library – hosts over 350+ papers and presentations representing work done with ESI Group CFD products. Tips and Tricks - an archive of 120+ user tip topics guarantees the user to learn something new about the software. Community Forums* - allows users to interact with each other and discuss on topics such as software usage, software customization, engineering applications, and computer hardware/OS issues. Compile Usersubs* - using ESI Group’s "online" compilation capability, users can compile their user subroutines on our systems. This feature is handy if the compilation is infrequent and if the user doesn’t have a FORTRAN compiler. * indicates user should login to have access.

13

14

Chapter 3: ESI CFD Software This chapter describes available ESI CFD software products and how you can obtain them for installation and usage.

Get to Know ESI CFD Software Software Release Software Packages Supported Platforms

Obtain ESI CFD Software Install ESI CFD Software – Windows Install ESI CFD Software – Linux

3.1. Get to Know ESI CFD Software 3.1.1. Software Release New software releases are made available twice a year with the intent of bringing new developments quickly to the customers. The software version is indicated by YEAR.MR#. Here <MR#> indicate the major release number for the <YEAR>. For example, V2008.0 indicates the first major release for 2008 (available in late 2007 or early 2008). The following release V2008.2 indicates the second major release for 2008 (available in mid 2008). The current version is V2009.2. Each software application will also have a minor release number (eg. 2008.0.1) uniquely identifying the build date of that application. Beta versions with the latest fixes are also available in between releases and can be obtained from the portal or ftp site. 3.1.2. Supported Platforms Platforms indicate architecture related to Hardware and Operating System. ESI CFD software products are supported on a number of different platforms. Individual applications may not be supported on all platforms. Table 3.1 shows a list of currently supported platforms and their canonical names. Table 3.2 shows the current availability of individual applications on different platforms.

Use these tables to identify the canonical name of a platform, check current and future support for a particular platform or the availability of individual applications on a particular platform.

An up-to-date list of supported platforms for an application could also be found in the application’s release notes. The release notes can be found in the software download area by choosing the appropriate platform and application.

15

Table 3.1. Supported platforms for V2009.2

Platform Canonical Name

Windows 2000/XP on Intel/AMD (32-bit) pc-windows-nt5-x86

Windows XP/2003 on Intel/AMD (64-bit) pc-windows-nt5-x86_64

Linux RedHat Enterprise 5 on Intel/AMD (32-bit) pc-linux-rhe5-x86

Linux RedHat Enterprise 5 on Intel/AMD (64-bit) pc-linux-rhe5-x86_64

Linux SUSE 9 on SGI Altix (64-bit) sgi-linux-suse9-ia64

Linux SUSE 10 on Intel or AMD (32-bit) pc-linux-suse10-x86

Linux SUSE 10 on Intel or AMD (64-bit) pc-linux-suse10-x86_64

Linux RedHat Enterprise 4 on Intel/AMD (32-bit) - deprecated pc-linux-rhe4-x86

Linux RedHat Enterprise 4 on Intel/AMD (64-bit) - deprecated pc-linux-rhe4-x86_64

HP-UX 11.11+ on PA-RISC (64-bit) - deprecated hp-unix-hpux11.11-pa_risc

HP-UX 11.22+ on Itanium2 (64-bit) - deprecated hp-unix-hpux11.22-ia64

AIX 5.3 on IBM Powerpc (64-bit) ibm-unix-aix5.3-powerpc

Note:

1. The following platforms were discontinued in the 2009.0 release: IRIX 6.5 on SGI MIPS (64-bit) Solaris 8 on Sun SPARC (64-bit)

2. The platforms marked as deprecated will be discontinued in the 2010.0 release.

3. The following restrictions apply to Itanium2 platforms:

CFD-GEOM is not available due to lack of ACIS library support. CFD-ACE STS radiation and BEM electrostatics solver modules are not available. CFD-VisCART CAD interfaces are not available.

4. The following restrictions apply to IBM platforms:

Only CFD-FASTRAN-SOLVER packages are available.

16

Table 3.2. Availability of applications in different platforms for V2009.2

Application/Platform

pc-w

indo

ws-

nt5-

x86

pc

-win

dow

s-nt

5-x8

6_64

pc-li

nux-

rhe5

-x86

pc-li

nux-

rhe5

-x86

_64

sg

i-lin

ux-s

use9

-ia64

pc-li

nux-

suse

10-x

86

pc

-linu

x-su

se10

-x86

_64

pc

-linu

x-rh

e4-x

86

pc

-linu

x-rh

e4-x

86_6

4

hp-u

nix-

hpux

11.1

1-pa

_ris

c

hp-u

nix-

hpux

11.2

2-ia

64

ibm

-uni

x-ai

x5.3

-pow

erpc

CFD-ACE-GUI x x x x x x x x x x x

CFD-ACE-SOLVER x x x x x x x x x x x

CFD-CADalyzer x x

CFD-FASTRAN-GUI x x x x x x x x x x x

CFD-FASTRAN-SOLVER x x x x x x x x x x x x

CFD-GEOM x x x x x x x x x

CFD-TOPO-GUI x x x x x x x x x x x

CFD-TOPO-SOLVER x x x x

CFD-VIEW x x x x x x x x x x x

CFD-VisCART x x x x x x x x x x x

SimManager x x x x x x x x x x x

‘x’ denotes application available in this platform

17

3.1.3. Software Packages ESI CFD products are typically purchased as a combination of software packages and add-on modules. This modular approach allows you to choose a combination of applications and application features best suited for your needs, thus bringing immediate and perceptible benefits. Figure 3.1 shows a schematic of this approach.

Figure 3.1 Schematic of ESI CFD product configurations A software package is a collection of applications. An application has several modules and features within it. You will be installing multiple applications based on your software package(s). Modules and features within an application are activated through an appropriate license feature corresponding to the software package/application/application module/application feature. Most downloads are available at this level as applications. Individual applications may require other applications/utilities to be installed as well. Refer to section 3.2 to identify which applications and utilities should be installed. Example: Software Package: CFD-ACE+ Basepack Applications: CFD-GEOM (pre-processor application for mesh generation)

CFD-ACE-GUI (user interface application for model settings) CFD-ACE-SOLVER (application for physics based solver)

Flow Heat Transfer Application Modules/Features Turbulence Grid Deformation

CFD-VIEW (post-processor application to visualize results) SimManager (application to manage multiple simulations)

Add-on modules could be additional applications that are not part of a software package or additional features within applications. Additional features typically do not require additional installations. Table 3.3 shows a list of available software packages. Table 3.4 shows a current list of all available applications and a short description about the application.

Hint: Your purchase order will have a list of software packages and add-on modules with a code associated with each item. Codes of the form XXX – XX – ## indicate a software package and codes of the form XXX – X – ## indicate an application or an application feature.

+

SSooffttwwaarree PPaacckkaaggee AAdddd--oonn MMoodduulleess((XXXXXX –– XXXX –– ####))

License Module

Application Module/Feature (XXX – X – ##)

Application(XXX – X – ##)

Application(XXX – X – ##)

License Module

License Modules

Application Module/Feature

Application(XXX – X – ##)

(XXX – X – ##)

18

Table 3.3. Software packages available from ESI CFD

Product Category Software Package Included Applications

CFD GEOM/VIEW Package

ACE-CE-01 CFD-GEOM CFD-VIEW

CFD-ACE+ Basepack ACE-CT-01

CFD-ACE-GUI CFD-ACE-SOLVER CFD-GEOM CFD-VIEW

SimManager CFD-ACE+ Educational

Package ACE-CT-02

- Same as CFD-ACE+ Basepack -

CFD-ACE+ Basepack + CFD-FASTRAN Combo

Package ACE-CT-03

CFD-ACE-GUI CFD-ACE-SOLVER CFD-GEOM CFD-VIEW SimManager CFD-FASTRAN-GUI

CFD-FASTRAN-SOLVER

CFD-ACE+ Mphysics + CFD-FASTRAN Combo

Package ACE-CT-10

CFD-ACE-GUI CFD-ACE-SOLVER CFD-GEOM CFD-VIEW SimManager CFD-FASTRAN-GUI

CFD-FASTRAN-SOLVER CFD-ACE+ Mphysics +

ACE-CS-02 - Same as CFD-ACE+ Basepack -

CFD-ACE+ Fuel Cell ACE-CS-03 - Same as CFD-ACE+ Basepack -

CFD-ACE+ MEMS ACE-CS-04 - Same as CFD-ACE+ Basepack -

CFD-ACE+ Semi ACE-CS-05 - Same as CFD-ACE+ Basepack -

CFD-ACE+ Bio ACE-CS-06 - Same as CFD-ACE+ Basepack -

CFD-ACE+ Semi + Plasma ACE-CS-07 - Same as CFD-ACE+ Basepack -

CFD-ACE+

CFD-ACE+ Semi + Plasma ACE-CS-07 - Same as CFD-ACE+ Basepack -

Continued…

19

Product

Category Software Package Included Application

CFD-CADalyzer CFD-CADalyzer Package ALY-CT-01

CFD-CADalyzer CFD-ACE-SOLVER

CFD-FASTRAN Package FAS-CT-01

CFD-FASTRAN-GUI CFD-FASTRAN-SOLVER CFD-GEOM

CFD-VIEW CFD-FASTRAN CFD-FASTRAN Educational

Package FAS-CT-02

- Same as CFD-FASTRAN Package -

CFD-TOPO Package TOP-CT-01

CFD-TOPO-GUI CFD-TOPO-SOLVER CFD-GEOM

CFD-VIEW CFD-TOPO

CFD-TOPO Academic Package TOP-CT-02

CFD-TOPO-GUI CFD-TOPO-SOLVER CFD-GEOM

CFD-VIEW Exceptions

CFD-VisCART* CFD-VisCART VIS-E-01 CFD-VisCART

CFD-FASTRAN‡Stress Module

FAS-S-01 (with FAS-CT-01/FAS-CT-02)

CFD-FASTRAN-GUI CFD-FASTRAN-SOLVER CFD-ACE-SOLVER CFD-GEOM

CFD-VIEW Note: *CFD-VisCART shown here is a stand-alone application (not part of a package) and hence has a code of the form XXX – X – ##. Additional features in CFD-VisCART will also have codes of the same form. ‡Applicable only for CFD-FASTRAN Fluid-Structure Interaction module.

20

Table 3.4. Current application installation packages from ESI CFD

Availability Application Description W L

CFD-ACE-GUI CFD-ACE Graphical User Interface Requires CFD-ACE-SOLVER x x

CFD-ACE-SOLVER CFD-ACE multi-physics solver x x

CFD-FASTRAN-GUI CFD-FASTRAN Graphical User Interface Requires CFD-FASTRAN-SOLVER x x

CFD-FASTRAN-SOLVER CFD-FASTRAN computational solver x x

CFD-TOPO-GUI CFD-TOPO Graphical User Interface Requires CFD-TOPO-SOLVER x x

CFD-TOPO-SOLVER CFD-TOPO computational solver x x

CFD-CADalyzer CFD-CADalyzer Graphical Interface Requires CFD-ACE-SOLVER x

CFD-GEOM Geometry modeler and mesh generator x x

CFD-VisCART CFD-VisCART mesh generator x x

CFD-VIEW CFD-VIEW post-processor x x

SimManager Simulation management utility x x

ESI-Utils Run scripts and other utilities Required for all packages x x

ESI-Licensing ESI/Flexlm licensing package Required only on the license server machine x x

MPICH.NT Message Passing Interface for parallel processing Required for CFD-ACE+ parallel in Windows x

Install.com Installation script for Linux/Unix systems Required for first time Linux/Unix installation x

MD5 Check Sum Utility to verify file integrity after download or transfer x x

ESI CFD Package combining all ESI CFD applications x

W – Windows, L – Linux/Unix Systems

21

Note: Different modules within an application are activated through appropriate licensing. Users do not need to install additional applications to activate add-on modules. For example, in CFD-ACE+, additional modules such as Stress, VOF, Plasma, etc. can be activated by purchasing the appropriate license modules. However, the user does not need to install any additional applications other than CFD-ACE-GUI and CFD-ACE-SOLVER regardless of which modules were purchased. Additional features such as CAD readers for SAT, ProE, etc. can be activated through licensing and does not require additional installations. However, it should be noted that some CAD readers might require that the user has a valid license for the native CAD application. Refer to Appendix 3 to see if you require a native CAD application license to use the readers. Parallel processing on any available application is activated through licensing and usually does not require additional installations except for windows platforms. If you purchased parallel processing licenses for Windows platforms, you should also install MPICH v1.2.5 (a freely available 3rd party software and downloadable from the ESI CFD portal) for proper communication between different machines. CFD-FASTRAN based FSI simulations invoke STRESS module from CFD-ACE+ requiring the CFD-ACE-SOLVER application. If your purchase did not include any CFD-ACE+ software package, you should additionally install CFD-ACE-SOLVER application. ESI Utils contains wrapper scripts to run different applications. It is always recommended to run the applications from the UTILS directory as these scripts set certain necessary environment variables on the fly. This is automatically done by including the UTILS_xxxx.x/bin directory in the environment variable PATH.

22

3.2. Obtain ESI CFD Software 3.2.1. Which Platform? If you haven’t already chosen, your first step is to decide the platform(s) (OS/Hardware) on which you will install the software. For many users, installations will be done on a single machine or a group of machines operating under the same operating system. For users with a network license, the flexibility to install different applications across multiple platforms is available. A common example is to have pre/post processing applications on the user’s Windows workstation and the number crunching (solver) to be done through a Linux cluster. In some cases, a particular application may be supported only on certain platforms requiring the user to use multiple platforms for different applications. A common example would be for a user who uses CFD-CADalyzer that is available only on Windows and using a Linux workstation for other applications. 3.2.2. Which Software Package/Applications? The next step is to identify the applications that should be installed using one of the following methods: Option 1: Manual Selection

1. Your purchase invoice will have a list of software packages and add-on modules

2. Compare the package/module names on your invoice with the names of the software packages in Table 3.3 (column 2). Any other name in your purchase invoice is a license feature and does not require additional software

3. Find the required applications from column 3 of Table 3.3

4. From Table 3.4, find all additional application/utility that should also be installed. Typically, these

are ESI-Utils, ESI-License and MPICH applications. Option 2: From ESI CFD portal

1. Login to ESI CFD portal

2. Go to "Download Software" section

3. Identify your "Platform" and click on the "Download Files" link

4. Compare the package/module names on your invoice with the names of the software packages in the list of checkboxes available in the search tool. Any other name in your purchase invoice is a license feature and does not require additional software

5. Select any additional options that you purchased

6. The applications that need to be installed are listed and become available for download.

23

3.2.3. Request ESI CFD software on a DVD If you would like to receive a physical media of the purchased software packages, please contact your ESI sales representative or account manager. Please include the platform(s), software package(s) and software version(s) you are interested in. We need to verify a valid shipping address prior to processing the shipment.

24

3.2.4. Download ESI CFD software from portal You can download the latest "Release" version of each application as well as the current "Beta" version from the portal. The installation packages include all the executables and libraries needed to run the code and all the documentation related to the application. To download installation packages:

1. Login to ESI CFD portal

2. Go to "Download Software" section

3. Identify your "Platform" and click on the "Download Files" link

4. Compare the package/module names on your invoice with the names of the software packages in the list of checkboxes. Any other name in your purchase invoice is a license feature and does not require additional software

5. Select any additional options that you purchased and submit the query

6. The applications that need to be installed are listed and become available for download.

It is recommended to download the "Release" version unless you are informed by support to use "Beta" version based on specific bug fixes.

The following types of files are available at the portal: For Windows Platforms: .msi files: An InstallShield MSI package is available for each application. This installer includes the requested application (e.g. CFD-GEOM) and ESI-Utils. The ESI_CFD package includes all applications in a single installation package. .md5 files: An md5 checksum file is available for each application. Each md5 file contains a string that is unique to the downloaded package and can be used to verify the integrity of the downloaded file. The md5 checksum application for windows can also be downloaded from the website. It is also recommended to use this utility to verify file integrity if you are transferring the downloaded file to different machines/platforms. For Linux/Unix Platforms: .exe files: For each application, a .exe file is available. This file includes all necessary components for the requested application. You need to download ESI Utils separately. .md5 files: An md5 checksum file is available for each application. Each md5 file contains a string that is unique to the downloaded package and can be used to verify the integrity of the downloaded file. It is also recommended to use this utility to verify file integrity if you are transferring the downloaded file(s) to different machines/platforms.

25

Release Notes: The latest release notes can also be downloaded separately for each application. This PDF file will provide the up-to-date information regarding supported platforms, expected future support on platforms, new features added to the application and bugs fixed in the application.

26

3.2.5. Download ESI CFD software from FTP site After identifying the applications you need (see Section 3.2.2), you can download the current "Release" and "Beta" versions of the software from the FTP site. In addition to the current version, you can also download older versions (v2006 – v2009.0) of the software. The "Release" and "Beta" versions are available under separate account names as follows:

"Release" versions: software "Beta" versions: betasoft

It is recommended to download the "Release" version unless you are informed by support to use "Beta" based on specific bug fixes.

To download software from our FTP site, you need to have the correct username and password for the ftp site. This information can be found under "Knowledge Base" within the ESI CFD portal. These passwords are changed periodically. So please visit the portal to get the current password. Once you have the account information, you can download from the FTP site using your favorite FTP client program. An example session from a MS-DOS command window to download the latest release version for a Windows 32-bit platform is shown below. C:\>ftp ftp.esi-cfd.com Connected to ftp.esi-cfd.com. 220 (vsFTPd 2.0.1)

User (ftp.esi-cfd.com:(none)): software ("betasoft" for Beta versions) 331 Please specify the password.

Password: (Enter appropriate password obtained from portal) … 230 Login successful.

ftp> cd 2009.2 (Use "ls" to list available versions and "cd" to choose version) 250 Directory successfully changed.

ftp> cd pc-windows-nt5-x86 (Use "ls" to list platforms and "cd" to choose a platform) 250 Directory successfully changed.

ftp> bin (Use "bin" to ensure binary mode transfer of files) 200 Switching to Binary mode.

ftp> mget *.msi (Use "ls" to list available files and "mget" to download files) 200 Switching to Binary mode.

mget ACE_GUI_2009.2.pc-windows-nt5-x86.msi? n (Choose "y" for desired applications) …

mget LICENSES_10.8.pc-windows-nt5-x86.msi? y 200 PORT command successful. Consider using PASV. 150 Opening BINARY mode data connection for LICENSES_10.8.pc-windows-nt5-x86.msi (4047872 bytes). 226 File send OK. ftp: 4047872 bytes received in 17.10Seconds 236.75Kbytes/sec. …

ftp> quit 221 Goodbye. C:\>

27

The following types of files are available at the FTP site. For Windows Platforms: .msi files: An InstallShield MSI package is available for each application. This installer includes the requested application (e.g. CFD-GEOM) and also ESI-Utils. This is the recommended method of installation for new users. The ESI_CFD package includes all available applications in a single installation package. .zip files: A zip file is available for each application. These can be downloaded and extracted at the appropriate directories to install the application. The zip file does not include ESI Utils. So you need to download ESI Utils separately. The ESI_CFD package includes all applications including ESI Utils. .manifest files: A manifest file is available for each application and includes a list of the components in a package. .md5 files: An md5 checksum file is available for each application. Each md5 file contains a string that is unique to the downloaded package and can be used to verify the integrity of the downloaded file. The md5 checksum application for windows can also be downloaded from the website. It is also recommended to use this utility to verify file integrity if you are transferring the downloaded file to different machines/platforms. For Linux/Unix Platforms: .exe files: For each application, an exe file is available. This package includes all necessary components for the requested application. You need to download ESI Utils separately. .manifest files: A manifest file is available for each application and includes a list of the components in a package. .md5 files: An md5 checksum file is available for each application. Each md5 file contains a string that is unique to the downloaded package and can be used to verify the integrity of the downloaded file. It is also recommended to use this utility to verify file integrity if you are transferring the downloaded file to different machines/platforms. install.com: This script file will guide you through the installation process. If this is your first installation, it is necessary that you download install.com

28

3.3. Install ESI CFD Software – Windows In a typical installation, each application is extracted to the installation directory and the environment variables are set as shown schematically in Figure 3.2. Since the application directories have version numbers linked to it, multiple versions of an application can be installed side-by-side. Two environment variables: ESI_HOME and PATH are required for the proper functioning of the software. The following sections describe this process.

Directory Structure Environment Variables

Installation Directory (C:\Program Files\ESI_Software\) ESI_HOME: Installation Directory

(ESI_HOME: C:\Program Files\ESI_Software) Application_Version1 Directory (ACE_SOLVER_2009.2) .. Path: %ESI_HOME%\UTILS_Version\bin

Application_Version2 Directory(ACE_SOLVER_2009.0)

..

(Path: %ESI_HOME%\UTILS_2009.2\bin;…)

Figure 3.2 Typical directory structure and environment variables on a Windows platform

29

3.3.1. Install from DVD Request and obtain the correct DVD for your software package and platform (Section 3.2.3). Once you have the DVD follow these steps to install the desired applications.

1. Place the DVD into reader. Open Windows Explorer, navigate to the DVD drive. You will see two folders (one for the 32-bit version: pc-windows-nt5-x86 and one for the 64-bit version: pc-windows-nt5-x86_64). Choose the appropriate folder for your system. Double click the Setup.exe file. (note: A 32-bit version can be installed on both 32-bit and 64-bit systems whereas a 64-bit version can be installed only on a 64-bit system)

2. If the same version of the software had been previously installed, then the first panel shown will

give the option to: Modify, Repair, or Remove. Select Modify to install or uninstall certain components. Select Remove to uninstall the entire installation.

3. After a welcome panel, you will have the option to choose the installation directory. By default, it

will be C:\Program Files\ESI_Software.

4. The next panel allows you to either install all applications or choose from a list of software packages and components available for installation. Select all that is needed and click Next.

5. The installer will ask you to verify your choices and install the software.

In this method, the required environment variables are automatically set by the installer. Note that if the licensing package is being installed (required only on a license server), then your HostID information will be automatically generated and saved to a file called hostid.txt. This file will be located in the ESI CFD installation directory (typically C:\Program Files\ESI_Software). This is the file that is sent to support for license key generation. Chapter 4 on Licensing discusses this in detail. For all installations, a Python 2.5 dialogue box appears. This allows for the installation of Python 2.5. It is not required to run our software but is required if you are running Python outside the CFD-ACE environment. Select Yes if desired and follow the instructions on screen. If CFD-ACE-SOLVER has been installed, then an MPICH installation dialogue box will appear. This is required for parallel execution. Select Yes if you have purchased the parallel feature and then follow the instructions on screen. MPICH must be installed in the default location as administrator. Make sure MPIRegister.exe is executed during the process so the user/password information goes in the registry that MPICH uses. For CFD-CADalyzer there will be additional panels for CAD plug-ins. Select the plug-ins you purchased and follow the on screen instructions. The final step is to restart the machine. For some Windows operating systems this is required in order to set the environment variables. Select Yes.

You are now ready to request and install licenses required to run the software. See Chapter 4 for detailed information on licensing.

30

3.3.2. Install downloaded MSI packages MSI packages can be downloaded from the portal (Section 3.2.4) and the FTP site (Section 3.2.5). Once you have the downloaded packages, follow these steps to install the desired applications.

1. Navigate to the location of the downloaded MSI packages and double click on the MSI package corresponding to the application that should be installed.

2. If the same version of the software had been previously installed, then the first panel shown will

give the option to: Modify, Repair, or Remove. Select Remove to uninstall the application. Repeat step 1.

3. After a welcome panel, you will have the option to choose the installation directory. By default, it

will be C:\Program Files\ESI_Software.

4. The next panel allows you to either install all applications or choose from a list of software packages and components available for installation. Select all that is needed and click Next. If the MSI package is for a particular application, only that application will be available in the list.

5. The installer will ask you to verify your choices and install the software.

In this method, the required environment variables are automatically set by the installer. ESI Utils package is automatically installed with each application. So it is not necessary for you to install it separately. If you have purchased the parallel feature, you need to install MPICH separately. Locate the downloaded file mpich.nt.1.2.5.exe. Double click to start the installation process and follow the on-screen instructions on the screen. It is recommended to install MPICH in the default directory. This is typically "C:\Program Files\MPICH" on 32-bit machines and "C:\Program Files (x86)\MPICH" on 64-bit machines. After installation, make sure you run MpiRegister.exe to store the username/password information in the registry used by MPICH. MPIRegister.exe can be found within the MPICH installation directory. e.g. C:\Program Files\MPICH\mpd\bin.

You are now ready to request and install licenses required to start using the software. See Chapter 4 for detailed information on licensing.

31

3.3.3. Install downloaded ZIP packages ZIP packages can be downloaded from the FTP site (Section 3.2.5) and is the only option available for many older versions (2008.0 and earlier versions). Once you have the downloaded packages, follow these steps to install the desired applications.

1. Create an installation directory: C:\Program Files\ESI_Software

2. Using WinZip, extract the downloaded files to the installation directory. Make sure WinZip does not create a directory with the name App_2009.2.pc-windows-nt5-x86 in the process of extraction. The files would be extracted to a directory with the name App_2009.2 (e.g. the CFD-ACE-SOLVER application downloaded as ACE_SOLVER_2009.2.pc-windows-nt5-x86.zip would be extracted to the folder ACE_SOLVER_2009.2)

3. Repeat Step 2 for all applications. Make sure the ESI Utils application is also installed

4. The next step is to set the environment variables: ESI_HOME and PATH

5. Windows environment variables are set through "Control Panel" → System → Advanced →

Environment Variables → System Variables (Do not set in User Variables)

6. ESI_HOME must be set to the installation directory. Create a new environment variable Name: ESI_HOME Value: C:\Program Files\ESI_Software (Use the correct installation directory if different)

7. PATH should be set to include the UTILS_2009.2\bin directory. Edit the existing PATH variable by adding the following to the beginning of the current path. Name: PATH Value: %ESI_HOME%\UTILS_2009.2\bin;

You are now ready to request and install licenses required to run the software. See Chapter 4 for detailed information on licensing.

32

3.4. Install ESI CFD Software – Linux/Unix In a typical installation, each application is extracted to the installation directory and the environment variables are set as shown schematically in Figure 3.3. Since the application directories have version numbers linked to them, multiple versions of an application can be installed side-by-side. Two environment variables: ESI_HOME and PATH are required for the proper functioning of the software. The following sections describe this process.

Figure 3.3 Typical directory structure and environment variables on a Linux platform

Application_Version1 Directory (ACE_SOLVER_2009.2)

..

Directory Structure Environment Variables

Installation Directory ESI_HOME: Installation Directory (/usr/local/ESI_Software) (setenv ESI_HOME /usr/local/ESI_Software

export ESI_HOME=/usr/local/ESI_Software)

Path: $ESI_HOME/UTILS_Version/bin (setenv PATH $ESI_HOME/UTILS_2009.2/bin:$PATH

export PATH=$ESI_HOME/UTILS_2009.2/bin:$PATH) Application_Version2 Directory (ACE_SOLVER_2009.0) LD_LIBRARY_PATH: $ESI_HOME/UTILS_Version/lib

.. (setenv LD_LIBRARY_PATH $ESI_HOME/UTILS_2009.2/lib export LD_LIBRARY_PATH=$ESI_HOME/UTILS_2009.2/lib)

33

3.4.1. Install from DVD Request and obtain the correct DVD for your software package and platform (Section 3.2.3). The installation script (install.com) can be run from a locally attached DVD drive or from a remote drive. If you are running the script from a remote drive, the remote and local platforms do not have to be the same operating system. For example, you can install ESI software on a Solaris platform from a DVD that is mounted on an HP workstation. Once you have the DVD, the steps for installing the software on a UNIX platform from the DVD are:

1. Load and mount DVD You can mount the DVD on either the local machine or a remote platform. You may need to have root privileges to mount the DVD and should contact your system administrator for instructions on how to mount the DVD for your particular system. For the remainder of this instruction, we will assume that the DVD has been mounted to: the local mount point: /cdrom

(or) the remote mount point: /nfs/remotehost/cdrom

2. "cd" to the installation directory On the local machine change directories to the desired installation directory (you may need to create one if it does not exist): mkdir -p /usr/local/ESI_Software cd /usr/local/ESI_Software

3. Initiate installation process Execute the installation script located in the base directory of the DVD. For a locally mounted DVD you can use the command: /bin/csh –f /cdrom/install.com

(or) For a remotely mounted DVD you can use the command: /bin/csh –f /nfs/remotehost/cdrom/install.com

A sample session of install.com is shown at the end of this section.

4. Select the desired platform

5. Select desired and required applications

6. Select installation directory

7. Specify the network file system, if known

8. Configure user account Login as the end user and execute config_user_account from where it was installed. This file should have been created in your home directory when the install.com script was run. This file will set ESI_HOME, PATH, and LD_LIBRARY_PATH in the users startup file.

./config_user_account

You are now ready to request and install licenses required to run the software. See Chapter 4 for detailed information on licensing.

34

3.4.2. Install downloaded EXE packages EXE packages can be downloaded from the portal (Section 3.2.4) and the FTP site (Section 3.2.5). It is required to download the installation script "install.com". Once you have the downloaded packages, follow these steps to install the desired applications.

1. Create an installation directory mkdir –p /usr/local/ESI_Software

2. Move the downloaded packages to the installation directory mv *.exe /usr/local/ESI_Software mv install.com /usr/local/ESI_Software cd /usr/local/ESI_Software

3. Make the packages and "install.com" executable chmod +x *.exe chmod +x install.com

4. Run "install.com" ./install.com

A sample session of install.com is shown at the end of this section.

5. Select desired platform

6. Select desired and required applications

7. Select installation directory

8. Specify the network file system, if known

9. Configure user account Login as the end user and execute config_user_account from where it was installed. This file should have been created in your home directory when the install.com script was run. This file will set ESI_HOME, PATH, and LD_LIBRARY_PATH in the users startup file (.cshrc, .bashrc …). This should be repeated for each end user.

./config_user_account

You are now ready to request and install licenses required to run the software. See Chapter 4 for detailed information on licensing.

35

A sample session of install.com is shown below. [/home/support/ESI_Software]$ ./install.com ************************************************** ESI Software Installation ************************************************** Please consult the ESI Getting Started Guide for detailed instructions. ************************************************** Available platforms are: 1 hp-unix-hpux11.11-pa_risc HP-UX 11.11+ PA-RISC (64-bit) 2 hp-unix-hpux11.22-ia64 HP-UX 11.22 Itanium2 (64-bit) 3 ibm-unix-aix5.3-powerpc IBM AIX 5.3 PowerPC 4 pc-linux-rhe4-x86 Red Hat Enterprise Linux 4 x86 (32-bit) 5 pc-linux-rhe4-x86_64 Red Hat Enterprise Linux 4 x86 (64-bit) 6 pc-linux-rhe5-x86 Red Hat Enterprise Linux 5 x86 (32-bit) 7 pc-linux-rhe5-x86_64 Red Hat Enterprise Linux 5 x86 (64-bit) 8 sgi-linux-suse9-ia64 SUSE LINUX Enterprise Server 9 Itanium2 (64-bit) 9 sgi-unix-irix6.5-mips IRIX 6.5.11+ MIPS (64-bit) 10 sun-unix-solaris2.8-sparc SunOS 5.8+ SPARC (64-bit) 0 - abort installation Install for which platform (1-10) <4/pc-linux-rhe4-x86>? 4 Installing packages for the pc-linux-rhe4-x86 platform Checking for available ESI software products... ok Install CFD-ACE-SOLVER? [y/n] y (This list will include only packages that are found in the directory) Package(s) will be installed in directory: /home/support/ESI_Software Is this ok? [y/n] y -------------------------------------------------- Installing ACE_SOLVER_2009.2 Package -------------------------------------------------- Getting local operating system... Linux 2.6.9-78.0.5.EL Checking for GNU gzip program... ok Checking for valid package file... ok Extracting temporary installation file... ok Uncompressing installation file... ok Extracting gnutar executable... ok Unpacking installation file... ok Deleting temporary installation file... ok -------------------------------------------------- ACE_SOLVER_2009.2 Installation Complete -------------------------------------------------- Creating end-user configuration script (config_user_account)... Will end-users be using either AFS or NFS (network file systems) to access the software (if you don't know, answer No) [y/n]? n config_user_account script written to /home/support/ESI_Software ************************************************** ESI Software Installation Complete ************************************************** To finish the installation, you may need to perform the following steps: 1. Install the license server. 2. Request and install the license keys. 3. Configure user accounts by logging in as the end-user and executing config_user_account from the /home/support/ESI_Software directory. Contact support.cfd@esi-group-na.com if you have any questions.

36

37

Chapter 4: ESI CFD License This chapter describes the license requirements to execute ESI CFD products and provides instructions on how to request, obtain and install the licenses.

Get to know ESI flexlm licensing Obtaining a license Install license – Windows – Node locked Install license – Windows – Floating Install license – Linux – Node locked Install license – Linux – Floating Migration (Software and Licensing) Troubleshooting and Tips

4.1. Get to Know ESI Flexlm Licensing All ESI CFD software products are FLEXenabled applications and require licensing to function. As a system administrator or an end user, you will identify the type of license required, generate information required to obtain a license and install the license. This section discusses the different types of licenses. 4.1.1. Node locked licenses This type of license implies that the application can be run only on one machine or a set of authorized machines. They do not require a license server. Therefore, it is not necessary to install the ESI Licenses application. For most license features, unless the user had purchased additional processes, 2 simultaneous processes of any combination of applications supported by that particular license feature are allowed. Certain applications such as CFD-ACE-GUI and CFD-FASTRAN-GUI look for the presence of a valid license but do not ‘check out’ a license and hence do not count towards the 2 processes. Refer to your purchase invoice to find the actual number of processes allowed for your configuration. 4.1.2. Floating licenses This type of license requires a license server to manage and issue licenses to clients. The licenses are available to any machine (usually on a network) that can access the license server. Thus it is necessary to install the ESI Licenses application on the license server. Client machines that requests licenses to run ESI CFD applications are not required to have ESI Licenses application installed. License server and client machines can span multiple platforms (Windows/Linux/Unix). Currently Mac OS is not supported for use as a license server. Optionally, the license server can be a machine with only the ESI Licenses application without any other ESI CFD application. Optionally, redundant license server systems can also be set for hardware fail-safe operation and load balancing. Please contact support for additional details.

38

4.2. Obtaining a License To obtain a license file, follow these steps:

1. Install the license package If you haven’t already done so, install the ESI Licenses application using one of the methods described in the previous chapter (DVD/download from portal/download from FTP site). This is recommended even for node-locked installations because it provides access to certain tools that can be used to obtain information regarding the hardware and also to troubleshoot licensing issues. The application will be installed in the installation directory under the name LICENSES_10.8.

2. Generate HostID information

For Windows Platforms:

i. Run lmtools.exe (found in the LICENSES_10.8 folder). ii. Go to the "System Settings" tab. iii. Press the "Save Hostid Info to a File" button. This will give an option to write out a

hostid.txt file. iv. If installed from a DVD, then the hostid.txt file will be automatically written out and will be

located in the C:\Program Files\ESI_Software directory (or the chosen installation directory).

For Linux/Unix Platforms:

i. Run install_lic.sh (found in the LICENSES_10.8 directory). ii. It will return the hostname and lmhostid information.

3. Request license file

Send the information generated through the previous step (for Windows: Hostname and hostid.txt file, for Linux/Unix: Hostname and lmhostid) to your local ESI Group License or Sales Representative for generating a license. Please note that license generation is done at the corporate headquarters and therefore there may be a 1 to 2 day turnaround required. Please be sure to include your name, company, and COIN with your request.

If you are currently using an earlier version (2008.0 or 2008.2), please attach your V2008 license to the license request to help expedite the process.

If you are currently using 2009.0, you do not need a new license. Refer to section 4.7 to see if you need a new license.

39

4.3. Install License – Windows – Node Locked After receiving your hostid information and generating the keys, ESI Support group will return to you a license file. Once you have the license file, follow these steps to install the license.

1. Place the license file in the LICENSES_10.8 folder (The location is suggested for convenience not a requirement)

2. The next step is to create an environment variable: PAM_LMD_LICENSE_FILE

3. Windows environment variables are set through "Control Panel" → System → Advanced →

Environment Variables → System Variables (Do not set in User Variables)

4. PAM_LMD_LICENSE_FILE variable should be set to point to the license file location.

Create a new environment variable Name: PAM_LMD_LICENSE_FILE Value: C:\Program Files\ESI_Software\LICENSES_10.8\licensefile.dat Use the correct location and filename based on your installation.

5. Verify installation

To verify that you can launch an application, simply type the application name from a command window (preferably one of the licensed graphical applications such as CFD-GEOM or CFD-VIEW) C:\> CFD-GEOM If the application senses that the license environment is not properly set up, it will display a dialog box asking the user to specify the license server or the license file. Choose the license file to run the application. Note: There is a known problem with Flex9.5+ in that the location of the license server or license file is not saved to the registry. If after specifying the license server or file, the application asks the user again for the location, then you must run a utility added to address this problem. To run this utility, go to the UTILS_2009.2\bin directory and execute flexreg.exe. Upon execution, you can specify the license server or license file location. This information will be saved to the registry. However, it is a good practice to set the PAM_LMD_LICENSE_FILE environment variable rather than writing directly to the registry.

40

4.4. Install License – Windows – Floating After receiving your hostid information and generating the keys, ESI Support group will return to you a license file. Once you have the license file, follow these steps to install the license on the license server and provide license access to the clients.

1. Installation on the license server should be done with Administrative privileges. First, you will set the license server and then the client machine(s).

2. Place the license file in the LICENSES_10.8 folder of the license server.

3. Modify your license file to include the correct location of pam_lmd.exe. This change has to be

done on the VENDOR line of the file. Lines that start with # are comments and are inactive. So changes have to be made on uncommented lines. You can use your favorite text editor to implement these changes. To change the VENDOR line, identify the line similar to the 'original' line shown below and change to the 'modified' version as shown below: Original: VENDOR pam_lmd pam_lmd Modified: VENDOR pam_lmd "C:\Program Files\ESI_Software\LICENSES_10.8\pam_lmd.exe"

Use the correct location of pam_lmd.exe based on your installation.

Include the quotes as it has been found that some systems require it especially if the path has spaces in it.

4. The next step is to create an environment variable: PAM_LMD_LICENSE_FILE

5. Windows environment variables are set using the "Control Panel" → System → Advanced →

Environment Variables → System Variables (Do not set in User Variables)

6. The PAM_LMD_LICENSE_FILE variable should be set to point to the license file location or the license server as shown below. On the license server, you can choose to point to either one (i.e. license file vs license server). On the client machines, you will choose the server option. Option 1: Point to License File Create a new environment variable Name: PAM_LMD_LICENSE_FILE Value: C:\Program Files\ESI_Software\LICENSES_10.8\licensefile.dat Use the correct location and filename based on your installation. Option 2: Point to License Server Create a new environment variable Name: PAM_LMD_LICENSE_FILE Value: <port>@<hostname> The <port> and <hostname> information should match the values in the SERVER line of your license file. You can find the SERVER line in your license file in the following format: SERVER <hostname> <hostid> <port> SERVER licserver 010102c7b98c 7789

41

7. Start the flexlm license server Run lmtools.exe (can be found in the LICENSES_10.8 folder). Select "Service/License File" tab and activate "Configuration using Services" option. Select "Configure Services" tab and set the following parameters (use your actual installation path if it is different) Service Name: FLEXlm Service (This is the default name, but any name can be used) Path to the lmgrd.exe file: C:\Program Files\ESI_Software\LICENSES_10.8\lmgrd.exe Path to the license file: C:\Program Files\ESI_Software\LICENSES_10.8\license.dat Path to the debug file: C:\Program Files\ESI_Software\LICENSES_10.8\pamlicense.log (Note the log file does not exist yet but will be created when needed) Turn "Use Services" ON and Turn "Start Server at Power Up" ON. Press "Save Service" Select "Start/Stop/Reread" tab and click on "Start Server"

8. Configure client machines Before you can run your ESI Software, you need to configure the user accounts so that they can find the license server. This is to be done on all machines that will be accessing the license from the server. On the client machine, Create a new environment variable Name: PAM_LMD_LICENSE_FILE Value: <port>@<hostname> The <port> and <hostname> information should match the values in the SERVER line of your license file. You can find the SERVER line in your license file in the following format: SERVER <hostname> <hostid> <port> SERVER licserver 010102c7b98c 7789

9. Verify installation

To verify that you can launch an application, simply type the application name from a command window (preferably one of the licensed graphical applications such as CFD-GEOM or CFD-VIEW). You need to do this on the license server and all the client machines. C:\> CFD-GEOM If the application senses that the license environment is not properly set up, it will display a dialog box asking the user to specify the license server or the license file. Choose the license file/license server (e.g. 7789@licserver) to run the application. Note: There is a known problem with Flex9.5+ in that the location of the license server or license file is not saved to the registry. If after specifying the license server or file the application asks the user again for the location, then you must run a utility added to address this problem. To run this utility, go to the UTILS_2009.2\bin directory and execute flexreg.exe. Upon execution, you can specify the license server or license file location. This information will be saved to the registry. However, it is a good practice to set the PAM_LMD_LICENSE_FILE environment variable rather than writing directly to the registry.

42

4.5. Install License – Linux – Node Locked After receiving your hostid information and generating the keys, ESI Support group will return to you a license file. Once you have the license file, follow these steps to install the license.

1. Place the license file in the LICENSES_10.8 folder. Rename it to PAM_LICENSE

2. The next step is to set the license file. This is done as follows:

For any user account you must create or edit the .flexlmrc file in the user’s home directory. This file will specify the path to the license file. This can be done most simply by editing (using vi or some other text editor) the file ~/.flexlmrc In this file you should add the following line: PAM_LMD_LICENSE_FILE=path to license file If the license file is located in /usr/local/ESI_Software/LICENSES_10.8/licenses/PAM_LICENSE, then you would add: PAM_LMD_LICENSE_FILE=/usr/local/ESI_Software/LICENSES_10.8/licenses/PAM_LICENSE Alternately, PAM_LMD_LICENSE_FILE could also be set as an environment variable in the user’s startup files.

3. Verify installation

To verify that you can launch an application, simply type the application name from a UNIX/LINUX shell prompt (preferably one of the licensed graphical applications such as CFD-GEOM or CFD-VIEW). $ CFD-GEOM

43

4.6. Install License – Linux – Floating After receiving your hostid information and generating the keys, ESI Support group will return to you a license file. Once you have the license file, follow these steps to install the license on the license server and provide license access to the clients.

1. Installation on the license server should be done with Administrative privileges. First, you will set the license server and then the client machine(s).

2. Place the license file in the LICENSES_10.8/licenses folder of the license server. Rename the file

to PAM_LICENSE.temp

3. Modify your license file to include the correct location of pam_lmd.exe. This change has to be done on the VENDOR line of the file. Lines that start with # are comments and are inactive. So changes have to be made on uncommented lines. To change the VENDOR line, identify the line similar to the 'original' line shown below and change to the 'modified' version as shown below: Original: VENDOR pam_lmd pam_lmd Modified: VENDOR pam_lmd $PAMHOME

$PAMHOME will be used by the scripts to properly set the path to the daemon.

4. Start the flexlm license server

Run install_lic.sh (which can be found in the LICENSES_10.8 directory). Follow the instructions in the script and it will configure and install the license server. Provide the full path to PAMHOME. This is by default your LICENSES_10.8 directory. First a check is performed to see if this is the correct host. Then a PAM_LICENSE file will be created. (Alternately, you could have the license file named as PAM_LICENSE in step 2) Then the license daemon startup scripts are setup. Here you can choose any appropriate user account other than the root. The license daemon will be started and the license server should be up. Checkout the MANAGE_LIC.TXT file created for license management.

5. Configure client machines For any user account you must create or edit the .flexlmrc file in the user’s home directory. This file will specify the path to the license file. This can be done most simply by editing the file ~/.flexlmrc (using vi or some other text editor). In this file you should add the following line: PAM_LMD_LICENSE_FILE=port@host Where <port> and <host> can be found by looking in your license file for a line similar to: SERVER licserver 010102c7b98c 7789 For this license file you would therefore add: PAM_LMD_LICENSE_FILE=7789@licserver

44

6. Verify installation To verify that you can launch an application, simply type the application name from a UNIX/LINUX shell prompt (preferably one of the licensed graphical applications such as CFD-GEOM or CFD-VIEW). $ CFD-GEOM

45

4.7. Migration (Software and Licensing) New versions of ESI CFD software is released every 6 months with the objective of bringing new features to users quickly. This section describes what needs to be changed when you switch from one version to another. The current version is 2009.2. When you switch from one version to another, you will

1. Install new version of the software 2. Check if a new license is required

Software: To install a new version, follow the procedures as described in Chapter 2 to download the appropriate software and the procedures in Chapter 3 to install. Multiple versions of ESI CFD software could be installed side-by-side within the same installation directory as shown in Figure 4.1. Only major versions can be installed side-by-side. All other versions (updated release/Beta versions) should overwrite existing installations. If your installation requires manually changing the environment variables, always point your PATH to the latest UTILS/bin directory.

Figure 4.1 Multiple versions of ESI CFD software installed side-by-side When an application is run through the wrapper scripts from the UTILS directory, the latest available version of the application will be used. To choose a particular version, use the –runver option as shown in the example below: CFD-GEOM –runver 2008.0 To check the version of the application, use the –v option as shown in the example below. The version number will be displayed on the console window or written to a file in the current working directory. CFD-GEOM –v CFD-GEOM V2009.0.1.13 DTF V7.8.1 Build Date Fri 01/30/2009 10:49:41.91

46

License: New licenses are required when you switch to the first major release of any given year as shown in Figure 4.2 below. If you need a new license follow the procedures described earlier in this chapter to request, obtain and install a new license. Examples: If you are migrating to 2009.2 from 2009.0, you will not need a new license. If you are migrating to 2009.0 from 2008.2 or any other legacy version, you need to obtain a new license. If you are migrating to 2009.2 from any other legacy version, you need to obtain a new license.

Figure 4.2 New license requirements on migration to different versions. The version of the license you have can be found from your license file. There will be a separate section on 'License Information' as shown below indicating the version of the license you have. #---------- License Informations ---------- # Version : 2009.0 # Daemon : pam_lmd 10.8 pam_lmd # Installation : Maintenance (FMA) # Product : ESI #---------- Customer Informations ---------- Each license feature will also indicate the version of license you have. FEATURE CFD_ACEPACK_BASE pam_lmd 2009.0 31-mar-2009 2

V2008.0 V2008.2 V2009.0 V2009.2

License V2008 License V2009 License ….

Software ….

47

4.8. Troubleshooting and Tips ESI CFD suite of software uses FlexLM licensing software. It is possible for system changes, network problems, and other factors to cause your license not to work. For such situations, there is a license troubleshooting documentation that will assist you in finding the problem. There is also a diagnostic script provided with the software to output information that will help the support team in isolating your license related problem. There are two PDF documents that have been developed from experiences with license problems and are available in the Knowledge Base section of the portal. One is for Windows operating systems (Windows License Troubleshooter.pdf) and one for Linux/Unix systems (Linux License Troubleshooter.pdf). These documents cover topics such as: License file modifications LMTOOLS utility (Win) FlexLM version mismatch Future dated files Environment settings Path Settings Proper execution of the code Client-Server issues Windows service pack issues Available Licenses Graphics problems Advanced Troubleshooting Software Versions License Daemon License Log Files These PDF documents not only help you isolate the problem, they also give you solutions to correct the problem. On occasion, going through the PowerPoint may not allow you to identify the problem. For such situations, we have written a diagnostic script to help isolate the problem. This script is called installationChecker.exe on Windows systems and installationChecker on Linux/Unix systems. It is located in the UTILS_2009.2/bin directory. Once you run this script, it will create an installationchecker.log file. This file will contain information on the environment variables, FlexLM variables, operating system information, and software versions. If you have license problems, please use the appropriate troubleshooting document to identify and correct the problem. If that does not work, send us the installationChecker log file.

48

ESI CFD portal has a collection of usertips that have been gleaned through the experience of many of our users. You can do a search or browse for tips based on your application. License/Software installation related tips are usually found under the general tip category. Some tips related to commonly encountered software configuration issues are presented below. Using "startx" for Remote Display Custom Logo Display Windows XP Service Pack 2 - License Firewall Issues Windows XP Service Pack 2 - Webhelp Issues

49

Chapter 5: ESI CFD Parallel Parallel processing is supported by CFD-ACE+ and CFD-FASTRAN. This chapter provides details on how to setup your infrastructure for parallel processing of ESI CFD applications. For complete details on how to setup and run a parallel job, refer to the appropriate application’s user manuals.

CFD-ACE+ parallel – Windows CFD-ACE+ parallel – Linux CFD-CADalyzer parallel CFD-FASTRAN parallel – Windows CFD-FASTRAN parallel - Linux

Parallel Processing enables greater computing power through the aggregation of CPUs and memory. The basic concept is simple: if one machine takes T hours, N machines should take T/N hours. Similarly, if one machine requires X MB of RAM, each of the N machines should need only X/N MB of RAM. (Naturally, this is an idealization making these the asymptotic limits.) Since each individual machine has only modest requirements compared to super computers, they cost less and enable the user to perform large simulations on relatively inexpensive hardware. Large problems benefit most from parallel processing. Large can refer to just the number of cells (hundreds of thousands to tens of millions) and/or the computational complexity (detailed finite rate chemical kinetics and LES). Either way, spreading the computational load across multiple processors improves the turn-around time. This also decreases the pre-processor memory requirements thereby making more machines eligible since you are more likely to have many small memory machines than large memory ones. The communication path between processes in a parallel job is also important. Parallel processing of small problems usually creates too much communication relative to the computational loads. A multiple processor machine with shared memory will have the fastest communications. Distributed processors need a fast network (100 Megabit/sec or faster) to achieve practical speedup in turn-around time. Parallel processing involves a master process and one or more worker processes on a single/multiple processor machine and/or across multiple machines. As part of installation setup, you will typically perform the following steps to enable parallel processing:

1. Acquire and install parallel license features 2. Install applications across compute nodes to meet parallel processing requirements 3. Enable a communication environment to meet parallel processing requirements

In addition, you will also partition the grid into 'n' number of zones equal to the number of parallel processes being used. Nomenclature Node = any computer in the cluster Master = the “head node” or the one from which the user is logged in and launching the jobs there can be only one master node. Remote/Worker = any remote node that the computation may use during parallel execution

50

5.1. CFD-ACE+ Parallel – Windows The following installation checklist will help the users to install necessary packages and provide guidelines to check the connectivity between nodes in a cluster.

Verify that a PARALLEL license feature is available (required)

Verify that the required environment variables have been set on all nodes (required)

Verify that all nodes have access to the solver executable and they are the same version (required)

Verify that CFD-ACE-GUI knows how to find the cfd-ace-mpi command (required if launching

parallel runs from GUI)

Verify that MPICH 1.2.5 has been installed properly on all nodes of the cluster (required)

Register Username/Password on all machines with MPIRegister.exe (recommended)

Verify that every node has the same user account and password access with Administrator Privileges (required)

Verify every machine has access to the same shared directory with read/write privileges

(required)

Verify that UNC path is valid

Verify that the master node is in the host list (required)

Verify that test program 1 operates correctly (test)

Verify that test program 2 operates correctly (test)

Verify that test simulation operates correctly (test) The following pages describe each step in detail.

51

Verify that a PARALLEL license feature is available (required) Access to the parallel capability (and the number of nodes allowed) is controlled by ESI licensing. You will not be able to start a parallel job if you do not have this license feature installed on your license server or if you have already consumed all available parallel licenses. You will also have problems if you are trying to run on more nodes than are available in the license file. Verification

1. On the master node launch the license manager toolkit (in your ESI Software installation folder look for LICENSES_10.8/lmtools.exe)

2. Navigate to the “Server Status” tab and press the “Perform Status Inquiry” button.

3. You should see some output generated.

4. Look for a line like:

Users of CFD_PARALLEL: (Total of 16 licenses issued; Total of 4 licenses in use) This verifies that you have the CFD_PARALLEL feature. In the example above, the user has 16 parallel process licenses available. We can see that 4 of those licenses are in use (someone is probably running a 4 process parallel job). This user could start another job that uses 12 processes but not one that runs on 13 processes.

5. Note that if the solver successfully starts and does not find a parallel license or finds that all are in

use, and then it will write such an error message to the model output file. License Error: Unable to obtain license for feature: CFD_PARALLEL

52

Verify that the required environment variables have been set on all nodes (required) In order to access the solver executable, all the nodes in the cluster should have the required environment variables set. Verification

1. If all the nodes in the cluster are accessing the solver from the server node, then the ESI_HOME and PATH environment variable needs to be set and should point to path on the server node.

For e.g. ESI_HOME = \\servernode\ESI_Software

where servernode is the name of the server node PATH = %ESI_HOME%\UTILS_2009.2\bin

2. If all the nodes are accessing the solver locally, the ESI_HOME and PATH should point to path

locally on the node. For e.g. ESI_HOME = C:\Program Files\ESI_Software PATH = %ESI_HOME%\UTILS_2009.2\bin

53

Verify that all nodes have access to the solver executable and they are the same version (required) All the nodes in the cluster should have access to the CFD-ACE-SOLVER executable and the solver executable needs to be of the same version if solver is installed locally on each node. Verification The solver version can be verified by logging to the remote nodes and issuing the following command: C:\> CFD-ACE-SOLVER –v A CFD-ACE-SOLVER.version file is created which looks like: CFD-ACE-SOLVER V2009.0.14 Build:70 DTF V7.8.2 Build date 04/17/2009 18:50:25 Recommendation It is recommended to install the CFD-ACE+ package on the master node or a server machine. All the other nodes in the cluster can then access the solver from there. This will ensure that each node is seeing a solver of the same version. Make sure that ESI_HOME environment variable is pointing to the server node, where the solver executable is installed.

54

Verify that CFD-ACE-GUI knows how to find the cfd-ace-mpi command (required if launching parallel runs from GUI) While launching a parallel job from GUI, the GUI looks for the cfd-ace-mpi executable under the PATH variable. Verification Please make sure that PATH variable is set correctly. Your PATH variable should include: %ESI_HOME%\UTILS_20xx.x\bin In, CFD-ACE-GUI under Edit-> Preferences -> Tools -> Parallel Launch Command should be set to cfd-ace-mpi.exe

55

Verify that MPICH 1.2.5 has been installed properly on all nodes of the cluster (required) Parallel execution on windows requires that MPICH 1.2.5 be installed on ALL nodes of the cluster. Do not use any earlier or later version! You must install MPICH into the “C:\Program Files\MPICH” folder. If you use any other drive or folder then you will be required to set the MPICHDIR environment variable to point to the installation folder (e.g., MPICHDIR = D:\Programs\MPICH). Installing

1. You can install MPICH from the ESI CFD Windows installation DVD or by downloading from ESI CFD portal or directly from the MPICH website.

2. YOU MUST HAVE ADMINSTRATOR PRIVILEGES WHEN INSTALLLNG.

3. To install from ESI CFD Installation DVD:

Insert the DVD into your DVD Drive Use windows explorer to navigate to the DVD Drive and go into the MPICHNT folder Click on Setup.exe to launch the installer and follow the directions there.

4. To install from a downloaded package (from ESI CFD portal or MPICH website):

To download from ESI CFD website, go to http://www.esi-cfd.com/ To download from MPICH website, go to http://www.mcs.anl.gov/mpi/mpich/mpich-nt/ Follow the instructions to download and install mpich.nt.1.2.5.exe

Verifying Installation

1. You can verify the installation and the version number as follows: Start → Run → cmd.exe C:\> cd c:\Program Files\MPICH\mpd\bin\mpd.exe C:\Program Files\MPICH\mpd\bin\> mpd.exe –v Should return:

MPD - mpich daemon for Microsoft Windows, version 1.2.5 Jun 8 2003 Copyright 2002 Argonne National Lab

2. Also the mpd.exe service should be running on the machine. You can check this from the task

manager (Ctrl-Alt-Del → Task Manger → Processes and you should see a process mpd.exe running).

56

Register Username/Password on all machines with MPIRegister (recommended) It is recommended to register this username/password with MPIRegister so that you do not have to enter that information for every parallel run. Registering

1. Execute the MPIRegister.exe program (located in C:\Program Files\MPICH\mpd\bin folder)

2. Console windows will pop-up asking for your username and password and the information gets stored in the windows registry.

C:\Program Files (x86)\MPICH\mpd\bin>MPIRegister.exe account: <username> password: confirm: Do you want this action to be persistent (y/n)? y Password encrypted into the Registry.

57

Verify that every node has the same user account and password access with Administrator Privileges (required) For parallel processing to work properly every node in the cluster must have the same account (username) and password. If your cluster is set up with a domain controller then you can use a domain account to launch a parallel job. If you do not have a domain controller then you must set up user accounts on all the nodes individually with the same username and password on each node. Each user can have whatever password they choose, but they must use the same password on all the nodes. In other words, UserA-PasswordA must be the same on all the nodes and UserB-PasswordB must be the same on all the nodes. The account on the master node must have Administrative or Power User Privileges. The accounts on the worker nodes must have the same username/password but they need not have Administrative Privileges. Verification

1. You should be able to log into every node of the cluster with the same username and password combination. On the master node you must verify that you have Administrative or Power User Privileges by checking:

Start → Settings → Control Panel → Users and Passwords On the Users and Passwords window, you can set the same username and password if you have not done this before.

58

Verify every machine has access to the same shared directory with read/write privileges (required) All nodes in the cluster must be able to read/write data files to the same folder. In order to do this, you must setup a shared folder that all nodes have access to. The shared folder can be anywhere but for this example we will assume it is on the master node. The DTF file (and any other files needed for the simulation) should be placed in this shared folder to ensure that all nodes will have access to read and write the file. Setting up a shared folder

1. Launch windows explorer and navigate to the folder that you wish to share 2. Right-click on the folder and select 'Properties' 3. Go to the sharing tab and activate 'Share this Folder' 4. Change the share name if desired. 5. Ensure that the permissions are set so that everyone has ability to change and read files on that

share. Verifying that Sharing is working

1. Login to a remote node

2. Launch windows explorer and navigate to \\master\share (where master is the name of the master node and share is the name of the shared folder)

3. You should be able to see files in the shared folder.

4. You should be able to copy a file to the shared folder.

59

Verify that UNC Universal Working Path is valid While executing the cfd-ace-mpi script, the path for the working folder (where the DTF file and other associated files are located) needs to be specified. When specifying the working folder, use the complete UNC path (e.g. \\masternode\share). For multi-processor machine a path with drive letter (e.g. D:\Parallel) should work. Verification

1. Login to a remote node 2. Launch windows explorer and navigate to \\masternode\share or \\remotenode1\share etc.

Execution from Windows Command Prompt

1. If you are launching the cfd-ace-mpi script from a command prompt, you will have to use backward slashes to specify the working folder path.

For e.g., cfd-ace-mpi –model modelname.DTF –hosts modelname.hst –wd \\masternode\share -num 2

Execution from Cygwin Bash Shell or any other UNIX Shell

1. If you are launching the cfd-ace-mpi script, using any UNIX Shell (for e.g. Cygwin Bash Shell etc.), you will have to use forward slashes to specify the working folder path.

For e.g., cfd-ace-mpi –model modelname.DTF –hosts modelname.hst –wd //masternode/share -num 2

Execution from GUI

1. If you are using CFD-ACE-GUI to launch a parallel job, you need to specify the 'Universal Working Path' under Run → Parallel Run → Hosts → Remote and/or Local → Universal Working Path as shown below:

60

Verify that the master node is in the host list (required) MPICH implementation requires that the master node from where the parallel job is launched needs to be a part of the hosts list. In other words, the solver needs launched on the master node. Verification Make sure that the host list (modelname.hst) file that you create has the name of the master node in it.

61

Verify that test program 1 operates correctly (test) Before launching the cfd-ace-mpi script to start a parallel run, it is recommended to test the connectivity between the nodes. We provide two test programs to do that. In this section, you will use test program 1. In this program, the master node sends an integer number to the remote nodes in the cluster. This program allows the master nodes and the remote nodes to write a message in an output file. If these files are written properly, this indicates that all the remote nodes are properly connected to the master node and the user is ready to launch the cfd-ace-mpi script. Instructions to run the program

1. In the mpiConfig file, change the working folder path to the complete path of the executable sample.exe

2. Make sure that the folder where the executables are placed is shared. For e.g., if the executable

on your node node1 is located at: D:/Parallel_Test/test and test is the shared folder, the first line will read as: exe //node1/test/sample.exe. In the third and the consequent lines you can assign the hostname and the names of other machines in the cluster and also assign the number of processes to be launched on each node.

For e.g., Hosts node1 1 node2 1 node3 1

The above command means that the host node is node1 and one process is launched on it. The remote nodes in the cluster are node2 and node3 and one process is launched on each of them.

3. Make sure ESI_HOME has been set. 4. Make sure that MPICH has been installed properly. Check that the MPIRun is located at

“C:/Program Files/MPICH/mpd/bin/MPIRun”. 5. To launch the program you need to type the following at the command prompt:

"C:/Program Files/MPICH/mpd/bin/MPIRun" mpiConfig Output

1. A set of output files are created on the master node equal to the number of processes launched. In the above example 3 output files are created.

2. The master node writes the ROOT.out file and following message is written to it.

Number of processors = 3 I am the master node. I am sending the integer: 50

3. Remote nodes write SLAVE1.out and SLAVE2.out files and following message is written to it.

I am a remote node. I am receiving the integer. The integer received is: 50

62

Verify that test program 2 operates correctly (test) Before launching the cfd-ace-mpi script to start a parallel run, it is recommended to test the connectivity between the nodes. We provide two test programs to do that. In this section, you will use test program 2. When executed, the cfd-ace-mpi script launches CFD-ACE-SOLVER.exe process on the master and remote nodes. It also launches wmserver.exe. This process probes and writes messages to the output file. This test program, tests the communication between nodes in a cluster when two executables are launched on the master node. Instructions to run the program

1. In the mpiConfig file, change the working folder path to the complete path of the executables server.exe and worker.exe

For e.g., hosts node2 3 //aspc2/OldMachine/Mini-Project/test2/test2/worker.exe node1 1 //aspc2/OldMachine/Mini-Project/test2/test2/worker.exe node1 1 //aspc2/OldMachine/Mini-Project/test2/test2/server.exe

Here node1 is the master node and node2 is the remote node. Remote.exe is analogous to CFD-ACE-SOLVER.exe and is launched on the master node and the remote nodes. Server.exe is analogous to wmserver.exe and is launched on the host node. In the above example, 3 processes of worker.exe are launched on node2 and 1 process is launched on node1.

2. Make sure that the folder where the executables are placed is shared. 3. Make sure ESI_HOME has been set

4. Make sure that MPICH has been installed properly

5. To launch the program you need to type the following at the command prompt:

"C:/Program Files/MPICH/mpd/bin/MPIRun" mpiConfig Output

1. The output expected on the screen is:

3 sending counter=> 401 1 sending counter=> 201 0 sending counter=> 101 2 sending counter=> 301

4 receiving from proc # 3 counter => 401 4 receiving from proc # 2 counter => 301 4 receiving from proc # 0 counter => 101 4 receiving from proc # 1 counter => 201

2. First four lines are outputted from worker.exe as it sends a counter to the master node. 3. The next four lines are outputted from server.exe, which gathers the data from the processes and

prints them on the screen.

63

Verify that test simulation operates correctly (test) It is recommended to try to launch parallel simulation on a simple and small test case (Tutorial 1 - Backward Facing Step, lbstep.DTF) before attempting on your actual case. This will help you to identify any errors during the parallel simulation quickly. For detailed instructions regarding preparing your DTF file for parallel simulation, refer to your CFD-ACE+ user manual. Verification

1. If are able to generate the RSL and Output files for the sample test case then this indicates the parallel set-up is correct and you are ready to run your actual case in parallel

2. You can launch a parallel job either through executing the cfd-ace-mpi script from the command

prompt or through CFD-ACE-GUI. Instructions to Launch the Parallel Job

1. Open a command prompt window (MS DOS Window).

2. Change the folder to the working folder where lbstep.DTF file is residing.

3. Perform dtf_decompose to partition the grid into ‘n’ zones

4. Launch the parallel job by executing the following script: cfd-ace-mpi –model lbstep.DTF –wd \\masternode\share\ -num n where 'n' is the number of processors, you would like to run this case on.

Output

1. You should see a RSL and an OUT file created in the working folder.

64

5.2. CFD-ACE+ Parallel – Linux The following installation checklist will help the users to install necessary packages and provide guidelines to check the connectivity between nodes in a cluster.

Verify that a PARALLEL license feature is available (required)

Verify that the required environment variables have been set on all nodes (required)

Verify that all nodes have access to the solver executable and they are the same version (required)

Verify that CFD-ACE-GUI knows how to find the cfd-ace-mpi command (required if launching

parallel runs from GUI)

Verify that all the nodes in the cluster are NFS or SMB mounted for file sharing (required)

Verify that remote login with rsh is enabled on all the nodes of the cluster (required)

Verify that the master node is in the host list (required)

Verify that test simulation operates correctly (recommended) The following pages describe each step in detail.

65

Verify that a PARALLEL license feature is available (required) Access to the parallel capability (and the number of nodes allowed) is controlled by ESI licensing. You will not be able to start a parallel job if you do not have this license feature installed on your license server or if you have already consumed all available parallel licenses. You will also have problems if you are trying to run on more nodes than are available in the license file. Verification

1. From the master or remote node terminal, cd $ESI_HOME/LICENSES_10.8/flexnet/Linux_xxxx_2.3/ ./lmstat –a –c <port@server>

2. You should see some output generated.

3. Look for a line like:

Users of CFD_PARALLEL: (Total of 16 licenses issued; Total of 4 licenses in use) This verifies that you have the CFD_PARALLEL feature. In the example above, the user has 16 parallel process licenses available. We can see that 4 of those licenses are in use (someone is probably running a 4 process parallel job). This user could start another job that uses 12 processes but not one that runs on 13 processes.

4. Note that if the solver successfully starts and does not find a parallel license or finds that all are in

use, and then it will write such an error message to the model output file. License Error: Unable to obtain license for feature: CFD_PARALLEL

66

Verify that the required environment variables have been set on all nodes (required) In order to access the solver executable, all the nodes in the cluster should have the required environment variables set. Verification

1. If all the nodes in the cluster are accessing the solver from the server node, then the ESI_HOME and PATH environment variable needs to be set and should point to path on the server node.

For e.g.

setenv ESI_HOME /net/servernode/ESI_Software where servernode is the name of the server node

setenv PATH $ESI_HOME/UTILS_2009.2/bin:$PATH

2. If all the nodes are accessing the solver locally, the ESI_HOME and PATH should point to path locally on the node.

For e.g.

setenv ESI_HOME /usr/local/ESI_Software setenv PATH $ESI_HOME/UTILS_2009.2/bin

67

Verify that all nodes have access to the solver executable and they are the same version (required) All the nodes in the cluster should have access to the CFD-ACE-SOLVER executable and the solver executable needs to be of the same version if solver is installed locally on each node. Verification The solver version can be verified by logging to the remote nodes and issuing the following command: CFD-ACE-SOLVER –v A CFD-ACE-SOLVER.version file is created which looks like: CFD-ACE-SOLVER V2009.0.14 Build:70 DTF V7.8.2 Build date 04/17/2009 18:50:25 Recommendation It is recommended to install the CFD-ACE+ package on the master node or a server machine. All the other nodes in the cluster can then access the solver from there. This will ensure that each node is seeing a solver of the same version. Make sure that ESI_HOME environment variable is pointing to the server node, where the solver executable is installed.

68

Verify that CFD-ACE-GUI knows how to find the cfd-ace-mpi command (required if launching parallel runs from GUI) While launching a parallel job from GUI, the GUI looks for the cfd-ace-mpi executable under the PATH variable. Verification Please make sure that PATH variable is set correctly. Your PATH variable should include: $ESI_HOME/UTILS_20xx.x/bin In, CFD-ACE-GUI under Edit-> Preferences -> Tools -> Parallel Launch Command should be set to cfd-ace-mpi.exe

69

Verify that all the nodes in the cluster are NFS or SMB mounted for file sharing (required) For launching parallel jobs through cfd-ace-mpi on UNIX systems, a common file sharing system on all the nodes is required. This can most commonly be achieved if the file sharing is done through NFS or SMB mounting. NFS/SMB Mounting The Network File System (NFS) is a distributed file system that provides transparent access to remote disks. Instead of duplicating common directories such as /usr/local on every system, NFS provides a single copy of the directory that is shared by all systems on the network. To a host running NFS, remote file systems are indistinguishable from local ones. For the user, NFS means that he or she doesn't have to log into other systems to access files. Once NFS has been set up properly, users should be able to do all their work on their local system; remote files (data and executables) will appear to be local to their own system. To get more information on setting up NFS on your cluster, you can refer to the following web links: http://www.linux.org/docs/ldp/howto/NFS-HOWTO/intro.html http://nfs.sourceforge.net/nfs-howto/ar01s03.html http://www.softpanorama.org/Net/Application_layer/nfs.shtml http://support.microsoft.com/kb/324086 To get more information regarding setting up SMB, you can refer to the following web links: http://samba.anu.edu.au/cifs/docs/what-is-smb.html Verification There are several ways to check if your system is NFS/SMB mounted. On a UNIX shell, you can type the “df” command on each node of the cluster to get information on the file system on which each FILE resides. For example: [account@hosts]$ df –h Filesystem Size Used Avail Use% Mounted on /dev/hda1 5.8G 4.6G 904M 84% / /dev/hda3 29G 5.5G 22G 21% /wrk /dev/md0 374G 218G 138G 62% /home none 756M 0 756M 0% /dev/shm Under LINUX system, you can check the contents of /etc/mtab file on each node of the cluster and it lists the host node and the remote node names which are NFS mounted. Depending on which UNIX systems you are on, the name and location of this file can vary. Some commands and files are listed below to find file systems that are NFS, mounted. Description Solaris Non-Solaris Initial list of filesystems to export /etc/dfs/dfstab /etc/exports Command to export initial list shareall exportfs List of currently exported filesystems /etc/dfs/sharetab /etc/xtab Command to export one filesystem share exportfs List of local filesystems on server /etc/vfstab /etc/fstab

70

Verify that remote login with rsh/remsh/ssh is enabled on all the nodes of the cluster (required) Remote login using rsh/remsh/ssh should be enabled on all unix-based platforms to spawn the parallel processes and to transfer data across the nodes. The type of login used depends on the platform as shown below:

Platform Bit Remote Login RHE4 32/64 ssh RHE5 32/64 ssh SUSE10 32/64 ssh SUSE9 64 ssh HP-UX 64 remsh AIX 64 remsh All other unix-type platforms 32/64 rsh

Verification From the master node, try to rsh/remsh/ssh to other nodes in the cluster. Once this is complete, try to rsh/remsh/ssh to the master node from the other nodes. To 'ssh' from a command prompt, simply type: ssh username@remotenode Once you have ssh working, it is recommended that you setup passwordless ssh to avoid having to type in your password numerous times. To create passwordless ssh: 1. [master] $ ssh-keygen -t rsa 2. Press Enter when prompted for passphrase 3. [master] $ cat id_rsa.pub >>authorized_keys (for passwordless ssh from master to master) 4. [master] $ chmod 600 authorized_keys 5. [master] $ ssh user@worker 6. [worker] $ ssh user@master cat ~/.ssh/authorized_keys >> ~/.ssh/authorized_keys 7. [worker] $ chmod 600 authorized_keys 8. Now you should be able to login from master to worker without a password 9. Repeat steps [1-8] to enable passwordless ssh from worker to master To 'rsh' from a command prompt, simply type: rsh remotenode Make sure you are able to open/append to a file on a remote node through remote login using rsh. If you are not able to rsh to a remote node, most likely, the .rhosts file is not correct on the processors that would be used in the parallel job. Internally, cfd-ace-mpi script launches an rsh command (remote shell) to launch MPI on the machines listed in the hosts file. Login to the remote machine and check the .rhosts file. This file must be placed in the appropriate home directory on the remote processor. Furthermore, the file must have permissions set so that only the owning user may read the file. Permissions are changed using the chmod command.

71

Verify that the master node is in the host list (required) MPICH implementation requires that the master node from where the parallel job is launched needs to be a part of the hosts list. In other words, the solver needs launched on the master node. Verification Make sure that the host list (modelname.hst) file that you create has the name of the master node in it.

72

Verify that test simulation operates correctly (recommended) It is recommended to try to launch parallel simulation on a simple and small test case (Tutorial 1 - Backward Facing Step, lbstep.DTF) before attempting on your actual case. This will help you to identify any errors during the parallel simulation quickly. For detailed instructions regarding preparing your DTF file for parallel simulation, refer to your CFD-ACE+ user manual. Verification

1. If are able to generate the RSL and Output files for the sample test case then this indicates the parallel set-up is correct and you are ready to run your actual case in parallel

2. You can launch a parallel job either through executing the cfd-ace-mpi script from the command

prompt or through CFD-ACE-GUI. Instructions to Launch the Parallel Job

1. Open a terminal.

2. Change the folder to the working folder where lbstep.DTF file is residing.

3. Perform dtf_decompose to partition the grid into ‘n’ zones

4. Launch the parallel job by executing the following script: cfd-ace-mpi –model lbstep.DTF –wd \net\masternode\share\ -num n where 'n' is the number of processors, you would like to run this case on.

Output

1. You should see a RSL and an OUT file created in the working folder.

73

Options available with cfd-ace-mpi script cfd-ace-mpi is the script which launches CFD-ACE-SOLVER and other necessary processes to enable parallel processing. It initially checks that the minimum required arguments (what to run, where to run, and how many processes to run it on) are provided. Then it verifies that the hostfile (specified or assumed) exists and that remote access to the listed hosts is allowed. The next step checks that the DTF file can be accessed from all hosts. (Under MS-Windows, the utility does not currently verify remote execution capability nor remote access to the DTF file.) Domain decomposition occurs (unless specified otherwise) creating a new simulation in the DTF file. Finally, it runs the simulation. A complete list of the available options are provided below. cfd-ace-mpi -model <modelName> # model name -num <num> # number of processors to use (equals zones) -wd <path> # dir in a common file system [-hosts <file>] # file listing hosts - default: modelName.hst [-local] # run in the current dir on the current host only [-dts] # use DTS for DTF file access [-dtsName <n>] # use the named DTS server for DTF file access [-dtsContext <c>] # use the given DTS context for DTF file access [-pbs | -qsub] # -pbs: let PBS define -num, -wd, and -hosts # -qsub: submit the solver run to the default queue [-sim <sim_num>] # DTF sim number to use - default: 1 [-decomp <option>] # dtf_decompose options, e.g., -x|-y|-z|-cell_groups|-orig_topo [-nodecomp] # do not run dtf_decompose (assumes already done) [-cleave [-distributed [-supportFiles 'file1 file2']]] # -cleave: cleave the DTF file for concurrent file access # -distributed: work with distributed cleaved files # -supportFiles: extra files to copy out for distributed processing [-prePrepped] # do not decompose, cleave, etc (assumes already done) [-out <file>] # file to catch stdout [-job] # equivalent to -out modelName.job [-T] # run CFD-ACE-SOLVERT instead of CFD-ACE-SOLVER [-solver <solver>] # run the given solver - default: $ACE_SOLVER_HOME/bin/ CFD-ACE-SOLVER [-serverHost <serverHost>] # run the DTF and text I/O servers on the given host/machine. - default: first host [-useDtfServer] # use a DTF I/O server [-dtfIoServer <dtfIoServer>] # run the given DTF I/O server. - default: $ACE_SOLVER_ HOME/bin/DtfIoServer [-wmserver <wmServer>] # run the given write message server. - default: $ACE_S OLVER_HOME/bin/wmServer [-mpich|-nativempi|-hpmpi|-sgi-mpi|-topspin-mpi] # use MPICH or MPI provided by HP or a native version of MPI for message passing. HPMPI available for X86_64, Itanium, PA-RISC machines running Linux, or HP-UX. [-keepTmpFiles] # keep the scratch files created - default: delete them [-array <arrayname>] # SGI only: Array Services array to use [-verbose <level>] # increase the verbosity of the output (level : 1, 2, or 3) [-h|-help|-usage] # print out script usage [-v|-V|-version] # print out script version number

74

Domain decomposition for CFD-ACE+ parallel processing Before you run a model in parallel mode using CFD-ACE+, you need to perform domain decomposition, whereby you divide the grid/data among multiple processors. The number of zones is equated to the number of processes. In other words, each processor would solve for a single zone of the overall mesh. The parallel feature of CFD-ACE+ provides several methods and options for decomposition of the domain. In many cases, the manner of decomposition influences the parallel efficiency and speed-up. For better speed-ups it is important that the workload is balanced between processors. In other words, all the processors should get nearly equal number of cells. The user has to use his/her own intuition as to which decomposition would be most appropriate for his/her simulation before attempting to do the domain decomposition. dtf_decompose [-version] [-metis | -cell_groups | -orig_topo | -x | -y | -z | -wavefront] [-even] [-combined]

[-keepFF] [-w w1 w2...] [-file_out outFile.DTF] [-restart] inFile.DTF sim# num_procs where:

-version # prints version number -metis # decompose by METIS method (DEFAULT) -cell_groups # decompose by cell groups (ignores num_procs) -orig_topo # decompose along existing zones (ignores num_procs) -x # decompose along the X axis -y # decompose along the Y axis -z # decompose along the Z axis -wavefront # decompose using wavefront method -w w1 w2... # give weight factor for each process. # the total number of weight factors should be equal to num_procs -even # cause -x, -y, -z decomposition to be spatial balanced # in stead of workload balanced -combined # combine volumes and boundaries together with identical # condition in the same decomposed zone -kp # keep process inferface not combined if process interface # lies on the boundary face -keepFF # keep fluid-fluid interface inside of a zone -file_out outFile.DTF # alternate DTF file to place decomposed simulation # if not specified, decomposition will go into inFile.DTF -restart # force mapping of face data inFile.DTF # DTF file to be decomposed # decomposition will be placed here unless -file_out is used sim# # simulation number to decompose num_procs # decompose for num_proc processes (zones) # not needed for -cell_groups and -orig_topo options

75

5.3. CFD-CADalyzer Parallel CFD-CADalyzer uses CFD-ACE-SOLVER in the background for computations. Therefore the requirements for parallel setup is the same as CFD-ACE+. However, CFD-CADalyzer GUI has a different setup process than CFD-ACE+. Refer to CFD-CADalyzer’s user manual for more detailed instructions. A short version is provided below for reference. Configure hosts In this step you will define the hosts that are available for parallel run. This can be done in the GUI as shown below: Edit → Preferences → Hosts → New

Setup parallel run In this step, you will setup all the necessary parameters that are usually provided at the cfd-ace-mpi script. This panel could be reached from Tools → Parallel Solver Setup You should select the hosts that are to be used for a parallel run, the number of parallel processes on each host, the decomposition method and the working directory as shown below.

76

5.4. CFD-FASTRAN Parallel – Windows The following installation checklist will help the users to install necessary packages and provide guidelines to check the connectivity between nodes in a cluster. Refer to your CFD-FASTRAN user manual for complete instructions on how to setup and run parallel simulations.

Verify that a PARALLEL license feature is available (required)

Verify that all nodes have access to the solver executable and they are the same version (required)

Verify that CFD-FASTRAN-GUI knows how to find the CFD-FASTRAN-SOLVER command

(required if launching parallel runs from GUI)

Verify that every node has the same user account and password access with Administrator Privileges (required)

Verify every machine has access to the same shared directory with read/write privileges

(required)

Verify that the required environment variables (ESI_HOME, PATH and MDICEDIR) have been set on all nodes (required)

o ESI_HOME → Installation directory (C:\ESI_Software) o PATH → %ESI_HOME%\UTILS_2009.2\bin o MDICEDIR → %ESI_HOME%\FASTRAN_SOLVER_2009.2\data

Verify that cygwin is installed and passwordless SSH is setup (required)

Verify that the master node is in the host list (required)

77

5.5. CFD-FASTRAN Parallel – Linux The following installation checklist will help the users to install necessary packages and provide guidelines to check the connectivity between nodes in a cluster. Refer to your CFD-FASTRAN user manual for complete instructions on how to setup and run parallel simulations.

Verify that a PARALLEL license feature is available (required)

Verify that all nodes have access to the solver executable and they are the same version (required)

Verify that CFD-FASTRAN-GUI knows how to find the CFD-FASTRAN-SOLVER command

(required if launching parallel runs from GUI)

Verify that the required environment variables have been set on all nodes (required)

o ESI_HOME → Installation directory (/usr/local/ESI_Software) o PATH → $ESI_HOME/UTILS_2009.2/bin:$PATH

Verify that every node has the same user account and password access (required)

Verify every machine has access to the same shared directory with read/write privileges (required)

Verify that passwordless SSH is setup between all nodes (required)

Verify that the master node is in the host list (required)

78

5.6. CFD-VisCART Parallel CFD-VisCART parallel is built on HP-MPI in linux/unix platforms and MPICH in Windows platform. Before launching a parallel run, the user needs to make sure HP-MPI/MPICH is properly installed. Application and environment setting requirements for CFD-VisCART parallel are similar to CFD-ACE-SOLVER. Refer to sections 5.1-5.2 on how to setup the environment for parallel CFD-VisCART. The following section describes how to use CFD-VisCART in parallel. Usage Currently parallelization is only supported in batch mode. The user can setup/create the batch script in the regular manner and then use the following command to launch the parallel process: CFD-VisCART -np #processes –hosts <hostfile> –p batchscript.vbt If –hosts option is not specified, it will be assumed that all processes are to run on the local machine. Note here that the argument to specify batch file input is –p, not the regular –b as for serial runs. The suggested #processes is between 2 to 16. Higher number of processes may slow down mesh generation instead of speeding it up due to increased communication between process interfaces. An additional option is available with the batch command to save the mesh into DTF file: save_dtf [filename] <combine_zones(0/1)> where, 0: Save DTF file with decomposed zones in it. Process (or zonal) interfaces will be visible. 1: Save DTF file into one zone just like the serial run. This option needs a lot of extra memory in the first host since it needs to build some connectivities for the mesh. To submit a CFD-VisCART parallel run via a queuing system like PBS, the following launch command should be used in the PBS launch script: CFD-VisCART -np #processes –pbs –p batchscript.vbt

79

Chapter 6: ESI CFD Support If you have questions or issues related to installation and licensing, contact our support team. Email: support.cfd@esi-group.com Telephone : Huntsville : 1-256-713-4750 China :

France : Germany : India : Israel : Japan : Korea : Taiwan: To expedite the resolution of your issues, it is recommended that you send us the complete information including: Installation log Security settings on your system/network Detailed explanation of the steps you took trying to resolve the issue Logs from diagnostic/verbose/trace options relevant to an application, if available

80

81

Appendix 1: Installation Checklist The following checklist will assist you in getting started with ESI CFD software quickly and smoothly.

Software purchase

Receive COIN from your local ESI account manager

Register at ESI CFD portal

Receive registration approval

Get ESI CFD software (download from portal/request DVD)

Install ESI CFD software

Send Host ID information to ESI

Obtain license

Install license

Verify installation

Details for the first few steps would be available in the email sent by your local ESI account manager. A complete description of each step can be found in the Getting Started Guide available at the ESI CFD portal. If you have any additional questions, please contact our support team at support.cfd@esi-group.com or call your local ESI office. Our Huntsville office can be reached at (256) 713-4750

82