Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
Tips and Tricks in Unix Shell scripting DB2 Database Administration
Speaker: James SobieskiFourth Millennium Technologies
Session Code: D14 -May 26, 2016 8:00 AM – 9 AM| Platform: DB2 for Linux, UNIX and Windows
Photo by Steve from Austin, TX, USA
Agenda
• Scripting Uses
• Template Scripts
• Building Shell Functions
• Common Unix Commands used in scripts
• Awk usage
• Examples
2
Shell Scripting Uses
• Automating DBA Activities
• Scheduling Complex activities via Cron
• Developing a library of “How I Did Something” scripts
• If you are going to do something 2 or more times, script it!
• General Advice• Ensure Scripts are Documented
• Be Consistent – this encourages reuse
• Write re-usable code snippets
• “Borrow generously” from others’ scripts
• This presentation will deal with mainly Korn Shell techniques
• We will review many powerful Unix commands used in shells
3
Templates Script
• Recommend that you create a template script
• Include• Heading Comments
• Name, Description, Usage, Author, Create Date, Mod Date + Description
• Standard Functions
• Help or Usage
• Command Line Parsing
• Exit Cleanup
• DB2 Command post processing
• Common Debugging Code
• Standard Variables containing names of directories for TEMP files, outputs, scripts, etc
• Common scripting structures
4
Temporary Files
• Many scripts use temporary files
• Recommend putting filenames in a variable
• Include the PROGRAM name in the filename
• Include some clue to its use in the filename
• Use “$$” to include the scripts’ Process Id in the file name
TEMP_SQL_OUT=“$TEMP_DIR/$PROGNAME.sql_outfile.$$”
db2 –tvf $MySqlFile >$TEMP_SQL_OUT
RC=$?
Run_SQL_Post_Processing_Function $TEMP_SQL_OUT
rm $TEMP_SQL_OUT
5
Common Shell Variables :
• $0 - the program name, fully qualified
• $* - all the arguments passed on the invoking scripts
• $# - the number of arguments provided on the command line
• $n - “n” is an integer – each successive arg on command line
• $$ - the variable containing the Process Id of the script
6
Common Shell Variables :
• When referencing variables, one can • Test if variable is null and assign a default value. Use the :- construct
DBNAME=${1:-”Sample”}
• Test if the variable is null and print an error message and exitDBNAME=${1?”$usage”}
• Invoking a script with a dot “.” in front of it, runs the script in the caller’s shell. Note that all variables and changes made in the script remain in effect when it returns control. ./.profile
7
Building a Shell Script Funtion:
• Functions have two declaration syntaxes:• function <funcname> {
stmts}
• <funcname> ( parameter_vars) {stmts}
• Both allow parameters passed in• Parameters are provided the same way a script is provided
parms• Variables are global – available to calling script• Local variables can be declared
• <xxx>
• Inside function, $ variables are local
8
A Usage Function:
• Functions need to be defined prior to invoking them
• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }
if [ $# != 2 ]
then Usage
fi
9
Typical Template Variables:
• Variables in many Template Scripts
# Get start time in db2 format
START_TIME=$( date '+%Y-%-m-%d-%:H:%M:%S )
# Scripts, Temp and Output directories: SCRIPT_DIR=/dbamaint/scripts
TEMP_DIR=/TMP/script_temp
OUTPUTS_DIR=$HOME/work/outputs
10
A Usage Function:
• Functions need to be defined prior to invoking them
• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }
if [ $# != 2 ]
then Usage
fi
11
Processing Command Line Options:
• GETOPTS is a function that will search the command line for a character preceded by a minus or plus and allow you to process the next argument following that character:
while getopts i: c
do
case $c in
i) instnme=$OPTARG;;
\?) Usage
exit;;
esac
done
shift `expr $OPTIND - 1`
12
DB2 CLP Command
• The db2 command, a.k.a, Command Line Processor is powerful
• Commonly used options
• adfsf
13
Option Descriptiont Terminators in SQL/Statements so they can be multi-linedf Filename follows it - reads SQL/Statements from the file
m Returns the number of rows modified by the last SQL stmta Returns the SQLCA areas Stops the execution if an Error or Severe Error occursp Toggles prompting for input +p turns off promptc Toggles Autocommit. -c Auto commits after each Stmt. +c does notv Verbose - echos statements to std out as they are executed
The DB2 SQLCA
• When –a is provided on the db2 command the SQLCA is displayed after every statement executed
• Fields in the SQLCA include
14
Option DescriptionSQLCAIDSQLCABCSQLCODE
SQLERRMLSQLERRMC
SQLERRP
SQLERRD1SQLERRD2SQLERRD3SQLERRD4SQLERRD5SQLERRD6SQLWARNSQLSTATE
Post Processing DB2 SQLCA example
15
Function to run a DB2 SQL Statement
• Run_db2 is a function which will run a db2 command and set standard variables for subsequent interrogation
16
AWK Programs and common uses
• AWK is a programming language well suited for parsing, math and command processing
• Common uses include piping output into an Awk program to:• Select the Nth token from the command line:
grep “$( hostname –s ) db2nodes.cfg | awk ‘{ print $1}’
• Find the line number of a string in a fileawk ‘/James Sobieski/ {print NR }’ my_contacts
• db2 get dbm cfg | grep ‘DIAGPATH’ | awk -F"=" '{print $2}'
17
Common Simple GREP usage
• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file
• Common grep options
18
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common SORT command usage
• SORT is powerful tool to post-process output from a command or a file and deduplicate output
• Common sort options
19
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common HEAD unix command usage
• Head is powerful tool to pull lines of data from a file or stdoutstarting at the beginning of the file
• Common head options
20
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common TAIL unix command usage
• Tail is powerful tool to pull lines of data from a file or stdoutreading backwards through the file
• Common tail options
21
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common XARGS unix command usage
• XARGS is powerful tool which will execute a command substituting variables in positions of the command
• XARGS options
22
Option Description
Common Simple GREP usage
• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file
• Common grep options
23
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Example Snippets – Archiving the DB2 Diag log
• Archiving the db2 diag.log
24
Example Snippets – Post Processing db2pd -hadr
25
Example Snippets – Examining an Application Snapshot
• Snapu script:
26
Example Snippets – making list applications readable
• listu
27
Example Snippets – Displaying and Summarizing Tablespace status
• Show_ts.ksh
28
Example Snippets – Combining System Catalog Tables
• Show_tab_ix
29
Example Snippets – Counting SQL messages in a file
• show_all_sql_msgs.ksh
30
Example Snippets – using AWK to summarize vmstat
• vmstatt.ksh
• iostatt.ksh
31
Example Snippets – Capture Single SQL Execution Stats
• Stmt_execute.ksh
32
Example Snippets – Run an explain on SQL statement
• Jexpln.ksh
33
Example Snippets – Capture runaway Applheapsz
• Capture_applheapsz_too_large.ksh
34
Example Snippets – Add Partition to RP table
• Add_next_partition.ksh
35
Example Snippets – Convert Snapshot cmd output to script
• Gen_all_ss_variables.ksh
36
Example Snippets – Generate Views for Catalog Tables
• make_one_shortcat.ksh
37
Example Snippets – Generate Views for Catalog Tables
• make_all_shortcats.ksh
38
Example Snippets – Report on backups
• Report_all_backups_taken.ksh
39
Insert Name HereInsert Company Name HereInsert E-mail here
SessionTitle
Please fill out your session
evaluation before leaving!
Photo by Steve from Austin, TX, USA