Bash on IBMi - schd.wsschd.ws/hosted_files/commons17/04/Bash on IBMi.pdf · – Best book I’ve found on Qshell

5/1/2017

© Agile Technology Architects, 2017New Berlin, WI 1

Bash on IBM i COMMON Annual Meeting

May 2017

Caicos 49:30am – 10:45 am

Tuesday

Presented by: Jim OberholtzerChief Technical ArchitectAgile Technology Architects

Agile Technology Architects

The Bash Shell

QSH on IBM i

Loading Bash onto your IBM i system (briefly)

Basics

Conclusions and Questions

Agenda

5/1/2017


IT Management Consulting, based on 30+ years of leadership, experience and performance

AGILE/SCRUM based application consulting services and education

Infrastructure Architecture consulting and Systems Management Services

Jim is an IBM World Wide Subject Matter Expert assisting IBM in the creation of the IBM i suite of certification exams. He has earned in excess of 40 IBM certifications



The Bash Shell

QSH on IBM i

Loading Bash onto your IBM i system (briefly)

Basics


Agenda

5/1/2017


What is a shell?

Some will say IBM i is an application

The LIC is the OS

Just as Windows is an app on top of DOS

Bash runs on top of Linux

Intel BIOS is the OS (LIC)

The Bash Shell

DOS

Windows

Linux

Bash Shell

IBM i

QCMD

Command prompt

GUI

Translate user commands into OS instructions– sort –n PhoneList >Phonelist.sorted

– useradd Myuser –p mY0assw0rD –g UserGroup• Linux user

– crtusrprf MYUSER password(mY0assw0rD) group(USERGROUP)

• IBM i user

• Passwords offer interesting questions about security– System Values come into play in a big way

The Bash Shell

5/1/2017


First shell called Bourne Shell– Steven Bourne shipped with UNIX version 7, 1979

csh Originally developed by Bill Joy at Berkeley – Known as csh “C shell”– Part of BSD (Berkley Software Distribution)

The Bash Shell

• *Remember, UNIX was a proprietary OS by Bell Labs• GNU was the open source follow on to BSD

Ksh was Korn shell by David Korn (7/1/1983) – Resembles C programming language

Bash “born” Jan 10, 1988, released in June 8, 1989– Bash means “Bourn-again shell”– Written by Brian Fox– Part of the GNU project (GNU’s not Unix*) – Shell for the GNU OS (Richard Stallman’s OS)

The Bash Shell

5/1/2017


“bashed” together features of sh, csh, and ksh. Portable Operating System Interface (POSIX) shellLinux came out in 1991

– Linux Torvalds– Uses GNU General Public License– Bash is default shell

2016 Build Conference announces a shocker– Microsoft added a Linux subsystem– Windows 10 supports Bash Binaries– Also supports other Ubuntu binaries– Canonical Ltd. working with Microsoft to make it so

The Bash Shell

For reference IBM’s PC‐DOS (MS‐DOS) came out in 1981System/38 was announce October 1978

Supports:– Here Documents (streaming input) – Regular expression (like Perl) – Associative Arrays

$ declare -A a # declare an associative array 'a' faking a bi-dimensional indexed array

$ i=1; j=2 # initialize some indices $ a[$i,$j]=5 # associate value "5" to key "$i,$j" (i.e. "1,2") $ echo ${a[$i,$j]} # print the stored value at key "$i,$j" 5

– Brace Expansion $ echo a{p,c,d,b}e ape ace ade abe

$ echo {a,b,c}{d,e,f} ad ae af bd be bf cd ce cf

– Redirection with “>” charactersort –n PhoneList >Phonelist.sorted

The Bash Shell

5/1/2017



The Bash Shell

QSH on IBM i

Loading Bash onto your IBM i system

Basics


Agenda

Unix like interface built into IBM i

Bourne like shell, with features of Korn Shell

Needed on IBM i:– 5770SS1 30 Qshell

– *5770SS1 33 Portable App Solutions Environment

– *5733SC1 *BASE IBM Portable Utilities for i

– *5733SC1 1 OpenSSH, OpenSSL, zlib

– *5733OPS *BASE Open Source Solutions

– OPS has 15 options, currently *base through 10 have anything

– Load all 15 options anyway, then PTFs will add function as it comes out

QSH on IBM i

* Optional but very highly recommended….

5/1/2017


Commands QSH and STRQSH are the same command

– Start the shell session from the command line

– QSH will use:• /etc .profile if it exists

• .profile if it’s in the users home directory

• More on these two things in a minute

– Environment Variables affect how things work• PATH

/usr/bin:.:/QOpenSys/usr/bin:/QOpenSys/QIBM/ProdData/SC1/OpenSSH/bin

• Needed for SSH command from IBM i into other systems, ie: HMC

QSH on IBM i

Environment Variable – QIBM_QSH_CMD_OUTPUT

– Determines what stdout is • None

• Stdout (c runtime terminal session, default)

• FILE=name

• FILEAPPEND=name

– ADDENVVAR command is used• Can be by job or system wide

QSH on IBM i

5/1/2017


Fkeys: Work as you would mostly expect

Put a number on the command line and press F14– The number of available lines will expand to that number so 10,

the F14 will give you 10 lines to enter commands on

Many shell commands can be put onto one – So: ls > grep –i ‘[a-z] [12]

• Will list all the files that begin with a character and have a one or two in it.

• There are two commands there ls (list) and grep (global regular expression printer)

• ls “pipes” it’s out put into grep

– Hence since you can stack many commands together you’ll need a longer line

QSH on IBM i

Consider this:cd /qibm/proddata/InstallationManager/eclipse/tools

imcl install com.ibm.websphere.WEBENAB.v85

-acceptLicense

-installationDirectory/QIBM/ProdData/WebSphere/AppServer/V85/EXPRESS

-repositories /QIBM/WAS/WASFIXPACKS/WAS/85511/FIXPACK

That line updates WAS 8.5 to fix pack 85511

QSH on IBM i

5/1/2017


Book: Qshell for iSeries– Published by MC Press

– Authored by Ted Holt and Fred Kulack

– Best book I’ve found on Qshell

– Much more available on internet• Most of the things you find will work with a small amount of effort

QSH on IBM i


The Bash Shell

QSH on IBM i


Basics


Agenda

5/1/2017


5733OPS looks something like this:

See wiki at:– IBM i Technology Updates/Open source Technologies

– Maintained by Kevin Adler

Loading Bash onto IBM i

5733OPS is found on the ESS site– My Entitled Systems Support

– Software Downloads

– Select 5770-SS1 IBM i • Open Hide/show

• Then Hide/Show on 5827:i7.2 B_GROUP1….

– You’ll find it as:F_MULTI_NLV_110_IBM_i_Open_Source_Solutions

Loading Bash onto IBM i

5/1/2017


Step one: Load 5733OPS– Load all 15 options

– Bash is in option 7• Loaded at: /QOpenSys/QIBM/ProdData/OPS/tools/bin

Step two: Load PTF group SF99225– This is a REQUIRED step

– Last update 1/19/17

– Load PTF SI63665 to get meaningful text on the options

• might be in the group by now, was not in early April

Loading BASH on IBM i

To get at BASH there are two ways– SSHD : MAC ssh command; Windows use Putty

– QSH

SSHD method– Obtain putty from chiark.greenend.org.uk

• Careful to get it from the real source to avoid clones

• Start the SSH server on the IBM i

• STRTCPSVR *SSHD*

• Connect with putty

• You’ll get

• “Yes” to store the RSA2 key

• Sign in IBM i profile


*Set the SSHD server to autostart or add it to the start up program

CHGTCPSVR SVRSPCVAL(*SSHD) AUTOSTART(*YES)

5/1/2017


QSH Method– QSH command

• Use: /QOpenSys/QIBM/ProdData/OPS/tools/bin/bash –version

• Use: /QOpenSys/QIBM/ProdData/OPS/tools/bin/bash


Set the path to get to BASH – PATH=/QOpenSys/QIBM/ProdData/OPS/tools/bin:$PATH

Set a link to the command:– ln -s /QOpenSys/QIBM/ProdData/OPS/tools/bin/bash

/QOpenSys/usr/bin/bash

– Now the bash command is in the delivered Path without any more fuss

– Simply type “bash” to start the Bash shell

Watch the prompt: Putty:

QSH

Basics

5/1/2017


A note about file systems– The root file system is not case sensitive

– Putty/SSH - Achieve the same thing


The Directory /QOpenSys is different– It IS case sensitive

– So Notice in directory /QOpenSys

– “cd agileprd” worked fine

– “cd AgilePrd” did not


5/1/2017



The Bash Shell

QSH on IBM i


Basics


Agenda

Let’s step back a second and do some review

– Three types of Files•Regular

•Executable

•Directories

– Regular files•Hold “stuff”

– Data, compressed files, scripts etc

•Can be executable but not generally

– Executable

Basics

5/1/2017


pwd Shows the current directoryWhen signifying a directory

– /QOpenSys/QIBM/ProdData/OPS/tools/bin – The “/” starts you from the root directory– Use the “forward” slash /– The “~” means home directory– In my case /home/agile (set on the user profile) – Or /home/myuserprf– So, ~/mygoodstuff/COMMON17 is:

– /home/agile/mygoodstuff/COMMON17

Basics

Remember where you are– Use pwd frequently

– If you rm –fR *.sh•All shell files from the current directory down will be removed

without any prompt

– If you rm –fR /*.sh all shell files on the system will be erased (pending authority of course)

Moral of story:– Do not sign on with *ALLOBJ

And– Check your directory often

Basics

5/1/2017


Basic Wildcards– ? Any single character– * Any string if characters– [set] Any character in set– [!set] Any characters not in set

So, a directory with the following list of files in it– bob, darlene, dave, ed, frank, and fred.– What will we get?

Basics

List reminder:– bob, darlene, dave, ed, frank, and fred

Basics

fr* frank, fred

*ed ed fred

b* bob

*e* darlene, dave, ed, fred,

*r* darlene, frank, fred

5/1/2017


Standard I/OUnix uses arbitrarily long sequences of characters

– Character not block

Everything on the system that produces data is treated as a file

– Includes hardware devices – Disk drives and terminals treated much the same– Unix programmers claim this is better

• Any one know why IBM i has to date never had a virus?– This is about to change with the advent of open source on IBM i.

– Windows is still the biggest target, followed by cell phones and Linux

– IBM i Open Source opens up the doors to the Database…….

Basics

Popluar Unix/Linux data filter utilities– cat Copy input to output (concatenate)– grep Search for strings in the input– Sort Sort lines in the input– cut Extract columns from the input– sed Perform editing operations on the input– tr Translate characters in the input– more Like cat, except page at a time

For a list of commands try:– cd /QOpenSys/bin– ls

• Not all the Linux commands are there. IE: less

Basics

5/1/2017


I/O redirection and pipeline– Take these examples:

cat <file1 > file2

cp file1 file2

grep '\<c.*h\>' /usr/share/dict/words

find words with a “c” and end in “h”

cut -c-8 test.txt

Extract 8 characters from the beginning of each line

– Pipelines move output to input as other commands:• To get a sorted list of all users on a Linux system:

Cut –d: -f1 < /etc/passwd | sort

Basics

Special Characters– There are a bunch of them:

• ~ home directory• # comment• $ variable expression• & background job• ( ) start/end a subshell• | pipe• [ ] start/stop character set

Some commands– Alias? What it sounds like

• IE: alias lf-’ls –lr’

Basics

5/1/2017


Output from ls:

Now lf:

Basics

Bash keywords

Basics

! esac select }case fi then [[do for until ]]done function whileelif if timeelse in {

5/1/2017


Variables– All variables are strings

•No datatypes in shell. Only strings•Variables are crated and assigned any time

– $ FILENAME=“StudentBean.java”– $ declare COST

– Input parameters to a shell script are indicated by number:

•&1, &2 , &3•The number of parameters is known by “$#”•The list of parameters is known by “$@

Security concern?– Moral of the story, security is more important than

ever

Basics

Basics

So:#!/bin/bash

#

# parm.sh: a positional parameter demonstration

printf "There are %d parameters(s) \n " "$#"

printf "The complete list is %s\n" "$@"

printf "The first parameter is %s\n" "$1"

printf "The second parameter is %s\n" "$2"

[root@RHE3 GPI]# ./parm.sh 1234 anyatall

There are 2 parameters(s)

The complete list is 1234, anyatall

The first parameter is 1234

The second parameter is anyatall

5/1/2017


Arrays– A list of values created with –a attribute

– $declare –a PRODUCTS• Creates an array of PRODUCTS

– $declare –a DEPT[0]=“Accounting” DEPT[1]=“Shipping”

– echo “${ DEPT [0] }”• Accounting

– echo “${ DEPT [1] }”• Shipping

Basics

Export– Makes the value of a variable globally available– Declare –x CVSROOT=“/home/cvs/cvsroot”– The –x makes the NAME exported– Now CVSROOT will deliver back “/home/cvs/cvsroot”– You can also use “export”– EXPORT $path – Notation: [-+]X

• -x sets the attribute• +x removes it

You need to think like the Linux geeks, “-” usually sets a named value pair

– So the ‘+’ would remove it

Basics

5/1/2017


Expressions– If command

– Followed by fi (end if)

– else

– Arithmetic Expressions:

let “SUM=5+5”

printf “%D” “$SUM”

10

Basics

If …then…else..if…then..fi…fi (this will look familiar) If [ conditional expression1 ] then

statement1 statement2

else if [ conditional expression2 ] then

statement3 fi

fi

Basics

5/1/2017


CASEprintf “%s -> ” “1 = delete, 2 = archive. Please choose one”

read REPLY

case “$REPLY in

1) rm “$TEMPFILE” ;;

2) mv “$TEMPFILE” “TEMPFILE.old” ;;

*) printf “%s\n” “$REPLY was not one of the choices” ;;

esac

Make really sure you are not ‘root’ and know what directory your in!!

Basics

While/Until Loopprintf “%s\n” “Enter the names of companies or type

quit”while true ; do

read –p “Company ?” COMPANYif [ “$COMPANY = “quit” ] then

breakelse if test –f “orders_$COMPANY.txt ; then

printf “%s\n” “There is an order on file”else

printf “%s\n” “There is no order on file”fi

done

Basics

5/1/2017


For Loops#!/bin/bashFor (( COUNTER=1; COUNTER<10; COUNTER++ ))

printf “the counter is now %d\n” “$COUNTER”Doneexit 0

Grouping Commandsls –l | {

while read FILE ; doecho “$FILE”

done}

– List into a file then process the results

Basics

FunctionsFunction my_function {

do my stuff…….

}

…………

my_function

………….

exit 0

Basics

• Variables declared inside a function are local to the function• Functions can be called recursively or in nested fashion

5/1/2017


A little math program:

Demonstrates – Variable assignment

– Conditional looping

– Math

Armed with that most anything is possible

Basics count=1echo $count

while [ $count ‐le 4 ]dolet count=$count+1echo $countdone

Change ownerships of objects:– In QSYS :

• CHGOWN OBJ('/qsys.lib/mylib.lib') NEWOWN(NEWOWN) SUBTREE(*YES)

– In IFS:• CHGOWN OBJ)'/mydirectrory/mysubdirectory')

NEWOWN(NEWOWN) SUBTREE(*YES)

– Only by a specific owner: (note it’s calling a shell command)

• QSH CMD('find <startdir> -user <OLDUSER> | xargs chown<NEWUSER>')

• This one takes some explanation– Find is piped into ‘xargs’

– The output from that is fed into ‘chown’ (chown is chgown in pase)

Basics

5/1/2017


SQL from Qshell? No problem:– Use the ‘db2’ utility

– Calls the DB2 CLI for you

– So:db2 “select * from qsys2.lib” | grep '\<q.*h\>' > more.txt

• Will list all the tables beginning with “q”, ending in “h” and pipe them into the command “more” (sadly “less” does not exist)

Db2 “SELECT * FROM TABLE (QSYS2.OBJECT_STATISTICS('MJATST ','*JRN') )” > mjatst.txt

• Pulls all the journals in library MJATST and puts it into a text file

– Let your imagination take you

Basics

DB2 from Bash is a bit more problematic– Really the DB2 command is PASE environment

– Db2sock project on bitbucket.org

– Driver is built: libdb400.a

– Needs system services

– Work around in progress• See

http://yips.idevcloud.com/wiki/index.php/Databases/SuperDriver

• And : https://bitbucket.org/litmis/db2sock

– Note: DB2 using this driver is not thread safe

– Should work from any of the Open Source languages

Basics

5/1/2017


Assuming you’ve set the path– Get around the db2 problems from BASH

– Use QSH, so from BASH

qsh -c ‘DB2 "create table jim.ptfout as (select PTF_PRODUCT_ID, PTF_RELEASE_LEVEL, PTF_LOADED_STATUS from qsys2.ptf_info) with data” ‘

– Creates a table in library “JIM” with the current PTF groups on the system

Basics

.profile – Sets defaults and your environment up

– Allows everyone to have unique setups based on how you use bash

– Requires a /home directory on IBM i • On user profile set home directory

• Usually /home/profilename

• Put your .bash…. Files in there

– Three types in order of use ( most generic to least ) :• .bash_profile

• .bash_login

• .profile

Basics

5/1/2017


Predefined Parameters– Bunches of them but here’s a partial list:

BASH The full path name of bash

HOSTTYPE The type of computer

HOME The name of your home directory

OSTYPE The Name of the OS

PPID The process ID of the shell’s parent process

PATH A colon-separated list of search paths to find a command to execute

Basics

OS=`uname ‐s`

case $OS in[Oo][Ss][4][o][o])alias ls='ls ‐F'PS1="\e[1m[ \u@\h:\w ] \n\t "PS2=">“

.profile

.profile

This will:– Set the keyboard behavior to match VIM– Tell IBM i that you want to use the BASH shell – Tell IBM i where to find the BASH shell

Basics

5/1/2017


Use .alias file

In home directory

Basics

Build your own functions– This one figures out what type of file it is and uses the

correct command to explode it…..

Basics

5/1/2017



The Bash Shell

QSH on IBM i

Basics



Agenda

BASH is another scripting language that is much like CL

Easy to learn

Available to IBM i via QSHELL and BASH

Now you can use Python instead of REXX

Calling shell scrips from CL is no real issueCHGVAR VAR(&CMD) VALUE(/tomcat5028/bin/catalina400.sh') SBMJOB CMD(STRQSH CMD(&CMD)) JOB(TOMCATSVR) +

JOBQ(RJSTOMCAT) CURLIB(*CRTDFT) INLLIBL(*NONE) +

CPYENVVAR(*YES) ALWMLTTHD(*YES)
