Managing SX.e and TWL with scripts and MARC

Jeremiah Curtis

AssumptionsYou should understand:

1. MARC -- What it is and generally what it does

2. Unix -- what it is and basic commands

3. Shell Scripts -- what they are and what they are used for

4. Basic understanding of TWL (for the TWL specific portions – you can nap during these portions if you are not

concerned with TWL)

Once upon a time…• … customers as well as NxTrend

personnel identified the need to have a standard set of scripts

• MARC would generate the standard set

• Everyone would be happy

The Shadow...• NxTrend support identified needs

that the MARC scripts did not address

• The product evolved, but MARC was not well suited to this change

• So, there became two sets of scripts– GFI for Go For It– MARC

Confusion Reigned...

• There was confusion – Which scripts should be used? – Which were getting installed?

An Alliance was Forged...

• The decision came that there should be only one set of scripts

• MARC should generate these scripts

• MARC should be flexible enough to handle future needs

Harmony was Restored...• All scripts will be generated by MARC

and are stored as templates• No longer hand editing scripts• Script issues will begin by regenerating

all scripts• New scripts are easily added

and existing ones easilyupdated

The End

The Changes

• The next section describes the current set of scripts

New Scripts and New Names

• Many new scripts have been added to MARC’s script base

• Many more scripts had a name change to standardize names and functions

The Progress Admin Server

• These scripts help manage the Progress Admin and Name Serversadmin.start

admin.shut

admin.query

nameserver.query

The Database• These scripts help manage the

database brokersdbbroker.startdbbroker.shutdbbroker.query

• These are database utilitiesdbstatstruncate.bi

The Application Broker

• These scripts help manage the Progress Application Brokersappbroker.start

appbroker.shut

appbroker.query

Parameter Files• The following are a new set of

parameter files for client connectionsclient.pfeditor.pfbatch.pfsingleuser.pfconnect.pf

The Environment

• These scripts help manage this particular environmentnxt.env

nxtall.start

nxtall.shut

Enterprise

• To start SX.enterprise, now use sxesxe

sxee

The Report Manager

• These are the Report Manager scriptsrptmgr (or rptmgr.start)

rptmgr.shut

Binary Dump and Load

• These scripts are for Dump and Loadseasybd sareabd

bddriver.p sareabddriver.p

bddumpall.p bddumparea.p

bdloadall.p bdloadarea.p

bdscript.p sareabdscript.p

After Imaging

• These scripts are for AIaimage.start

aimage.shut

aimage.backedup

aimage.list

aimage.archive

aimage.new

aistats

rollforward.ai

truncate.ai

Backups

• These are new standard backup scriptsbackup.data

backup.full

backup.online

backup.files

RxServer

• RxServer scripts are now generated by MARCrxsfax.sh

rxsprint.sh

emailout.sh

Other Scripts

• These don’t fit anywhere elserdclean

sassi.sh

shutuser

showuser

config_rpt

TWL

• TWL scripts have been mostly rewritten

• MARC generates all of the needed TWL scripts

• TWL scripts now follow a standard naming convention

Additional TWL Application Broker• These scripts help manage the Application

Broker that manages TWL connections to the SX.enterprise databasetwlentappbroker.starttwlentappbroker.shuttwlentappbroker.query

• TWL also uses the normal appbroker scripts which are generated from TWLMARC. These appbrokers are for connections to the TWL database

TWL RF Units

• These scripts are used to start the TWL application on RF unitstwlrf

twlentrf

twlentrf.pf

Additional TWL Environment

• Since TWL integrates with SX.enterprise, an additional environment is available using:twlsecondary.env

TWL End of Day• These scripts are used to run TWL End

of Day processing. These are usually called from the backup scripts

• The first script performs processing that requires a single-user connection to the DB while the second performs tasks using a normal multi-user connection to the DBtwleod1

twleod2

TWL Interfaces

• The following scripts are used to manage the TWL interfacestwlsend.start twlrcvpack.sh

twlsend.sh twlrcv.sh

twlsend.shut twlrcv.shut

twlsend.query twlrcv.query

The Templates

• The next section describes the new Script Generator in MARC

MARC Generate Scripts• MARC• Databases -> Configure ->

Generate Scripts• Select the database(s) to generate

scripts for• Select the scripts to generate

Script Template• All scripts generated by MARC

are stored as templates• These templates contain the

form of the final script• Critical portions of a template

contain tags• These tags get substituted with

real values to generate a script

MARC Generate Scripts• All of the templates reside in

subdirectories under /rd/marc/scripts/template

• All templates must end with a .tpl

• These subdirectories are:basedghpibmsco intchantaxwaretwl

• All of the templates in base are added to the list

• One system directory is included based on the selected machine type in Installations -> Configure (e.g. ibm)

• If optional products are setup in MARC, then those templates are also included in the list (e.g. twl)

• The custom directory is always checked first and overrides a standard template of the same name

MARC Generate Scripts

• Script templates may also be in /rd/marc/scripts/cust in the same subdirectories

• Copy the template from the directory under /rd/marc/scripts/template

Custom Scripts

Template Tags

• There are two types of template tags– System tags (defined by MARC)– Custom tags (defined by the user)

• Tags are names enclosed in ‘< >’– Example: <ENVNAME>

• The convention is to use tag names that are all capital letters

System Tags

• System tags are defined during the installation of MARC

• System tags contain values that MARC can get from the information in its own database

• System Tags can not be edited directly with the tag editor

Custom Tags• Users may use the Tag Editor to create

and edit tags.• Custom tags are created by users or

may be in some delivered scripts (e.g. rxsprint.sh)

• Values for the tags can be assigned in two ways:– By answering the prompt when MARC finds

an undefined tag– By using the tag editor in Databases ->

Configure -> Tag Editor

The Tag Editor

• The tag editor allows the user to:– Create new custom tags– Modify existing custom tags– Delete custom tags

Example Template• Here is a simple template

<SCRIPTNAME=ENV”helloworld.sh”>

<DESTDIR=BIN>

echo “Hello, world!”

• Top two lines are reserved for formatting• SCRIPTNAME is what the script will be

called• DESTDIR tells where the script will get

created

Script Naming

<SCRIPTNAME=ENV”helloworld.sh”>

• All scripts should be prefaced with the environment name

• Use ENV before the script name to substitute in the environment name in the MARC that is generating the script

Destinations

<DESTDIR=BIN>

• The destination directory can be a path• The destination can be one of these

predefined directoriesBINOPSYSRDEXEC

Building A Script• Here is a simple template

“helloworld.sh.tpl”

<SCRIPTNAME=ENV”helloworld.sh”>

<DESTDIR=BIN>

echo “Hello, world!”

• Should be created in the ../cust/base directory

• Generating script to /rd/bin/

Script Templates - Summary

• All scripts will be generated by MARC and are stored as templates

• Should no longer hand edit scripts• Script issues will begin by

regenerating all scripts• New scripts are easily added and

existing ones easily updated

MARC Enhancements

The next section describes some changes in Progress 9 and changes in MARC 10.1

First, Some Background

• With the advent of Progress 9, there are some new concepts to understand

• MARC 10.1 also contains many changes to accommodate the new version of Progress as well as a few enhancements

Storage Areas• The Progress database has changed to

incorporate a new concept of storage areas

• Storage areas are a logical grouping of tables

• Storage areas are also in their own files on disk and have extents

• Oh, and by the way, all Progress 9 databases are multi-volume and will have extents

Progress Admin and Name Servers

• MARC can now configure the ports used by the Progress Admin and Name Servers– Admin Server Port– Admin Server Admin Port– Name Server Port

• Allows for multiple installs of Progress– Live: /rd/dlc– Test: /test/rd/dlc

Installation Configure Screen

• New Environment Name• Online Backup Directory• AI Save Directory• Layout Changes

Database Configuration• With storage areas, MARC has the

ability to configure extents for the storage areas

• Each storage area can be configured to have a certain number of fixed extents plus the size for the fixed extents

AppServer Changes• The AppServer source code is not

on the Unix server anymore• Therefore the AppServer is not

compiled from MARC anymore• The AppServer is pushed from the

Staging Client

TWL Related Changes

• There are now two AppServer configurations– twlappsrv– twlsxappsrv

• TWL warehouse locations now only configure the name of the location

Backup Configuration• Backup configuration have been

completely rewritten• Configuring backups have changed

– They work– Tape Device– AI– Online Backup

• Script templates are copied for the type of backup needed

Shutuser

• The Disconnect User function has been completely rewritten

• MARC now calls the shutuser script– The script is generated by MARC– It works– There is a lookup available (F12) to

see users logged in

Dump and Load

• There are two types of dump and load– For an upgrade which included a Progress 9

upgrade, a dump and load is needed to put the data into the correct storage areas

– For maintenance, a dump and load is used to realign data and improves performance

– There are different functions for each type of dump and load

Maintenance Dump and Load

• Reworked in MARC 10.1– A storage area can be dumped and

loaded– The dump and load is based on

scripts proven to work

Script Generation

• The script generation module in MARC has been completely rewritten

• Script generation is now template-based• No hacking required• Script generation is logged

– scriptgen log– systemtag.lst

Questions

?

?

?

?

?

?

?

?

?

?

?

?