View
3
Download
0
Category
Preview:
Citation preview
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
AGENDA
SAS ADMINISTRATORS USER GROUP
1st October 2014
199 Bishopsgate, London, EC2M 3TY
Time Topic Speaker
9.30 -10am Coffee and networking
10 -10.15am Introductions All
10.15 -10.45am Support Update Andrew Williams
10.45 -11.15am Tuning and Performance: Best practice with APM Steve Wallace
11.15 - 11.30am Coffee Break
11.30 – 12 noon Grid and stored processes best practices Peter Hobart
12 - 12.30pm Hints and Tips – sharing session All
12.30 - 12.45pm Feedback and discussion about future topics All
12.45 - 2pm Lunch and networking All
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SUPPORT UPDATE
GUIDANCE FOR SAS ADMINISTRATORS
Andrew Williams, SAS UK Customer Support, 1/10/14
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
AGENDA
• Creating / Managing Tracks
• Information on Applying Licenses
• Managing My Software Account
• Checking / Applying Hotfixes
• Services Restart Order
• Increased Logging
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
CREATING / MANAGING TRACKS
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
INFORMATION ON APPLYING LICENSES
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
MANAGING YOUR ACCOUNT
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
CHECKING / APPLYING HOTFIXES
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
USEFUL HOTFIX LINKS
• Hotfix Main Page
• http://ftp.sas.com/techsup/download/hotfix/hotfix.html
• Deployment Registry Troubleshooting
• http://support.sas.com/kb/38/551.html
• TSNEWS
• http://support.sas.com/techsup/news/tsnews.html
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SERVICES RESTART ORDER
SAS(R) 9.* INTELLIGENCE PLATFORM: SYSTEM ADMINISTRATION GUIDE
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
INCREASED LOGGING
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
INCREASED LOGGING
• Enabling Thread Dumps - http://support.sas.com/kb/40/127.html
• Enterprise Guide Debugging - http://support.sas.com/kb/17/730.html
• Changing Logging Level for Web Applications -
SAS(R) 9.* Intelligence Platform: Middle-Tier Administration Guide
• Object Spawner - http://support.sas.com/kb/34/893.html
• OLAP Server - http://support.sas.com/kb/36/728.html
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
QUESTIONS, PLEASE
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
SAS APM & PERFORMANCE
GUIDANCE FOR SAS ADMINISTRATORS
Steven Wallace, SAS UK Premium Support, 1/10/14
(with thanks to Alessio Tomasino)
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
AGENDA
• Brief recap on the SAS Audit & Performance Measurement package
• How it works, and what it produces
• Installation and other considerations of interest to SAS administrators
• How APM can inform investigations into performance & capacity planning
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
APM: WHAT IS IT?
A BRIEF REMINDER OF WHAT IT DOES & HOW
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
FUNCTIONALITY
• Enterprise BI Server Performance Reports
• Consumption of system resources (CPU, memory, some I/O) by SAS jobs
• SAS resources (various PROCs, SORT, OLAP cubes, table access, etc.)
• Also…
• Metadata Server Audit Reporting
• Report on modifications to users, groups, access rights & other metadata changes
that affect security of the SAS application, plus failed logins, etc.
• Enterprise BI Environment Status Reports
• Information on status of Web App & BI servers with alerts
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DESIGN (1)
• SAS servers use ARM logging to write
APM logs (via logconfig.apm.xml)
• SAS programs collect logs, parse the
data and write to SAS datasets
• Sample programs are provided to read
the SAS data and produce a number
of static HTML reports
• Dynamic audit reports are generated
with user-entered parameters
• The same datasets can be queried for
ad hoc analysis and reporting
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
DESIGN (2) OR TO PUT IT MORE SIMPLY…
APM Logs
SAS Server, e.g.
Workspace
Modified
logconfig.xml
ARM logging
SAS program
SAS datasets
Read logs,
parse data, &
store as SAS
datasets
SAS programs
Reports
Read those
datasets and
create static
HTML reports
(At execution) (batch, daily*) (batch / ad hoc)
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
GOOD PRACTICE
CONSIDERATIONS FOR SAS ADMINISTRATORS
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
GOOD PRACTICE BI SERVER STATUS REPORT
• Email integration for APM
notification in case of service
failure, e.g. notify NOC team
• Run during business hours at
intervals of 5 minutes
• Integration with third party
monitoring solutions is possible
• Stop before restarting services /
maintenance
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
GOOD PRACTICE METADATA AUDIT REPORTS
• Make sure administrative rights are
granted to the correct users
• Ensure the correct users are added
to the appropriate groups to prevent
unauthorized access
• Multiple authentication errors may
indicate unauthorized login attempt
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
GOOD PRACTICE BI SERVER REPORTS
• Recognize usage patterns and
trends
• Can help identify performance
bottlenecks
• Can help identify potential
performance improvements
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
“INSTALLATION” POINTS TO CONSIDER
• Request and download (free) package from support.sas.com
• Modify a script (.bat / .sh) to provide environment-specific details
• Run scripts to generate customized programs and log configuration files
• Import package (.spk) into metadata (for STP) and adjust some properties
• Schedule (customized) SAS batch programs for gathering, processing and
archiving the captured data
• APM adds no new SAS components (so SDW is not used)
• Server restarts are needed and config files will be modified
• Older versions required service restarts (to close log files)
• Installation can be done via SAS Premium Support / other engagement
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
APM USAGE GOOD PRACTICE
• Usual advice on changing “usermods” version of config files applies
• Can tailor “runDataArchive” script, e.g. to log basic execution information to
an external file (for debug)
• Scheduled job will move log files into the EBIAPM area, but…
• Need to delete these periodically to recoup space
• The ebiapm_archive_data program requires modification
• Change hard-coded date to something else, e.g. TODAY()-nn
• Can amend archive process to append to a historical table
• Default will create separate archive tables (named by cut-off date)
• May want to delete record types that aren’t of interest, to save space
• ArtifactUsageDetails space isn’t recouped by default (use COPY / SORT, etc.)
• Some (perm) tables are created during reporting, which also need pruning
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
RELEVANCE TO PERFORMANCE
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
PERFORMANCE A BIG SUBJECT…
Was OK before?“Performance is bad”
What changed?
Yes No
= Never OK= New work
• Volume of work
(data, users,…)?
• Indexing?
• Maintenance?
• Infrastructure?
• Etc. • Redesign
• Repair
• Restore
• Upgrade
• Etc.
• Design scope
adequate?
• Right tool for
this job?
• Expectations
realistic?
• Working as
designed?
• Capacity
sufficient?
• Expectations
realistic?
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
WHAT APM ISN’T
• APM information is not intended as a real-time investigative tool
• Logging is near real-time, but processing/reporting is typically overnight
• Logging is initiated within SAS applications, so cannot include potentially
relevant external factors, e.g. memory or disk contention from other work
• APM, therefore, has a valid, but limited remit:
• Can be useful to inform discussions on performance and capacity planning
• Can identify, e.g., changes in numbers of users, reports run, specific “heavy”
procedures and sorts, plus identifying the consumers of most resources
• Is not a substitute for system tools like Perfmon and SAR when attempting to
diagnose performance issues
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
APM DATA AD HOC ANALYSIS
• Consider where APM is installed if end users will require access
• May need to make a copy of data elsewhere, or create a R/O share
• There are a few key datasets that contain the relevant information
• Artifacts table
• ArtifactRelationships table
• ArtifactsPhysicalLocation table
• ArtifactUsageDetails – this is the main fact table
• There are, in addition, various audit-related tables
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
APM DATA INTRO TO FIELDS
Variable Length Description
Artifact $512 An object (e.g. table)
UsageID $512 Uniquely identifies the record
ParentUsage $512 (See below.)
CurrentUsage $512 Represent parent and child values as defined in the ARM standard.
HostName $100 The name of the server where activity occurred.
_loadTM 8 Timestamp showing when record was loaded into the dataset.
Datetime 8 Logged event start timestamp (datetime).
StopDt 8 Logged event end timestamp (datetime).
User (*) $127 Typically, in format account@domain.
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
APM DATA TYPES / SUBTYPES
• A variety of record types and
subtypes are available
• Volume of data is reduced
significantly if ACCESS type
records are deleted
• The derived stats are loaded
when data is added to the
datasets (MeasurementType =
“derived”)
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
APM DATA DERIVED INFORMATION
Variable Length Description
elapsedUserCPUTime 8 User CPU time for the event
elapsedSystemCPUTime 8 System CPU time for the event
elapsedTotalCPUTime 8 User + System CPU time
TotalIOCount 8 (Only for some types)
TotalMemoryUsage 8 (Only for some types)
Copyr i g ht © 2014, SAS Ins t i tu t e Inc . A l l r ights reser ve d .
QUESTIONS, PLEASE
Copyright © 2011, SAS Institute Inc. All rights reserved.
…and the SAS Grid
Stored Processes
34
Copyright © 2011, SAS Institute Inc. All rights reserved.
Stored Processes
Introduced with the SAS 9.x platform
Prompting Framework added in 9.2
Re-usable prompts
Dependent Prompts
Can run on
A stored process server (default)
A workspace server
35
Copyright © 2011, SAS Institute Inc. All rights reserved.
Stored Processes
Created from:
Enterprise Guide flows
DI Studio Jobs
Raw SAS code
Anything which can be turned into raw SAS code
Capabilities
Anything which SAS can do via code
36
Copyright © 2011, SAS Institute Inc. All rights reserved.
Stored Processes
Compared to macros
More user - friendly
No access to underlying code
Executed from a client (not a SAS session)
» SAS Enterprise Guide
» Microsoft Office (with SAS add-in)
» Web Browser
» JMP
» SAS Visual analytics
» PROC STP (SAS 9.3 onwards)
37
Copyright © 2011, SAS Institute Inc. All rights reserved.
Proc STP Syntax
PROC STP PROGRAM=“metadata-path-of-stored-process” <ODSOUT=STORE | REPLAY>;
INPUTDATA stored-process-data-file=member-name | “data-set-path”;
INPUTFILE stored-process-file<=local-fileref | “local-file-path”> ...;
INPUTPARAM parameter-name<=”parameter-value”><parameter-name<=”parameter-value”>>;
LIST< GROUP=level | (level1...leveln)>;
LOG FILE=local-fileref | local-file-path
OUTPUTDATA stored-process-data-file=member-name | “data-set-path”;
OUTPUTFILE stored-process-file<=local-fileref | “local-file-path”> ...;
OUTPUTPARAM parameter-name<=local-variable-name>;
38
Copyright © 2011, SAS Institute Inc. All rights reserved.
Proc STP Example
ods _all_ close;options metaserver = 'your-metadata-server'
metaport = your-metadata-server-port
metauser = 'your-userid'
metapass = 'your-password';
proc stp program='/Products/SAS Intelligence Platform/Samples/
Sample: Cholesterol by Sex and Age Group'
odsout=store;
run;
goptions device=png;
ods html path='your-output-directory' file='test.htm'
style=HTMLBlue;
proc document name=&_ODSDOC (read);
replay / levels=all;
run;
ods html close;
39
Copyright © 2011, SAS Institute Inc. All rights reserved.
Demo
Creating a stored process from code
Adding parameters
Running it from Microsoft Office
40
Copyright © 2011, SAS Institute Inc. All rights reserved.
OK, I cheated…it's a demo
I used a system data set
In the real world, need to consider libraries needed
Pre-assign
Embed in the Stored Process Code
If it's going to run on a workspace server, consider user's permissions to use the data
41
Copyright © 2011, SAS Institute Inc. All rights reserved.
More about _metauser
One of many macro variables available to stored processes
_metauser Specifies the login username that is used to connect to the metadata server.
_metaperson Specifies the Person metadata name that is associated with the _metauser
_program Name of the stored process. The value of _PROGRAM is frequently a path
_url URL of the Web server middle tier used to access the stored process.
More at http://support.sas.com/rnd/itech/doc9/dev_guide/stprocess/reserved.html
42
Copyright © 2011, SAS Institute Inc. All rights reserved.
Grid considerations
Job Priority
Latency
Force use of grid in Enterprise Guide
Allow parallel processing?
http://support.sas.com/resources/papers/proceedings14/SAS375-2014.pdf
43
Copyright © 2011, SAS Institute Inc. All rights reserved.
Sending jobs to the required queue
Method 9.3 9.4 Notes
Within a SAS server context, specify
"queue=" on the grid server metadata object
Y Y May need to create
multiple contexts
Create an ESUB script Y Y Allows complex
logic, requires
scripting ability
Use a Grid Options Set N Y Route user to a
queue based on
level and client used
44
Copyright © 2011, SAS Institute Inc. All rights reserved.
Questions?
Copyright © 2011, SAS Institute Inc. All rights reserved.
Thankyou
Recommended