Copyright © 2010, Scryer Analytics, LLC. All rights reserved. Optimizing SAS System Performance − A Platform Perspective Patrick McDonald Scryer Analytics,

Copyright © 2010, Scryer Analytics, LLC. All rights reserved.

Optimizing SAS System Performance− A Platform PerspectivePatrick McDonaldScryer Analytics, LLCJune 3, 2010


Presentation Overview

After this presentation you will know:• How your SAS code interacts with the hardware it runs

on.

• The different hardware configurations SAS may run on in your organization.

• How to help your IT organization diagnose and correct performance problems.

You probably won’t gain:• Any new SAS programming tips

• More than a very brief overview of efficient programming techniques


An Easy Question

What does this program do? Connects to DB2

SAS table of db2table

Disconnects from DB2

Creates x as previous y

Calculates Mean and N and outputs data

proc sql;connect to db2 (database=mydatabase);create table Table1 as select * from connection to db2 ( select * from db2table);disconnect from db2;quit;

data View1 / view=View1; set Table1;retain x;output;x=y; run;

proc summary data=View1 NWAY; var _numeric_;class c1 c2 c3; output out =p.mymeans mean= M N=COUNT;run;


What controls system performance?

Resources Relationships


Efficient Programming Practices

Writing Efficient Code Necessary Statements

Passes Through Data

Essential Read/Writes

Permanent SAS Data

Necessary Procedures

Sorting, Duplicates, Etc.

SAS Views

DBMS Optimization

Configuring/Tuning Options Buffer Allocation

Memory Allocation

Multithreading


Resource Model – CPU, RAM, I/O, & Disk


CPU

CPU What is a CPU?

• # of Sockets

• # of Chips

• # of Cores

• # of Co-processors

• Clock Speed

• Etc.

SPECfp

SPECint


RAM

Memory RAM

RAM per core

RAM per session

RAM for OS


I/O

I/O Types of Storage

• Network Attached Storage

• Local Disk

• Storage Area Network

The disk is the slowest part of the system ~10-60 MB/s read/write speeds

Throughput per session• 15-25 MB/s

• 50-75+ MB/s


A little more about storage

Storage Options HBAs

LUNS

RAID

Disks

Disk Speed

Disk Size

File Systems SAS User

Temporary Work Space

Permanent Data Storage

Utility (UTILLOC)


RAID Configurations in SAS Environments


Operating System Limitations

Windows (32 bit) Enterprise Edition (32 bit)

• ~2 GB of RAM practical limit

• 5 GB data set size practical limit (file cache contention)

Windows (x64) Enterprise Edition for x64

• Support issues (9.1)

• 5 GB data set size practical limit (file cache contention)


Operating System Limitations

Windows (Itanium) Enterprise Edition

(Itanium)• 10 GB data set size

practical limit (file cache contention)

Unix (64 bit) HPUX, Solaris, AIX etc.

• Limited by hardware only

• Access to additional memory

• No file cache contention issues


Architecture Limitations

Hardware Bottlenecks CPU (#, speed, etc.)

RAM

Backplane

Cache

Configuration/Tuning

Hyperthreading

SAN Bottlenecks I/O

• Host Bus Adaptors

• Ethernet (2 GB/s Ethernet)

• Paths to Disk

• Disks− RAID

− # of disks

− Disk Speed

− Disk Size

• Luns & File Systems


Redux: what does this program do?

Think like hardware?proc sql;

connect to db2 (database=mydatabase);create table Table1 as select * from connection to db2 ( select * from db2table);disconnect from db2;quit;




PROC SQL

What resources are used?proc sql;





Data Step






Proc Step






Typical BI/SAS Solution Architecture


BI Architecture

Web Server Loads CPU Intensive

Integer Calculations

Rack Servers

Pooled, Load Balanced

~ 100 concurrent sessions per core (CPU)

Small Text Files


BI Architecture

Application Server Loads CPU Intensive

Integer Calculations

Rack Servers

Pooled, Load balanced

~100 concurrent sessions per core (CPU).

Small Text Files


BI Architecture

SAS Metadata Server Memory Intensive

Metadata stored in memory for speed

Generally 2 CPU except for very large implementations

Metadata in RAM database


BI Architecture

SAS BI Servers CPU and or I/O Intensive

Heavy Floating Point (CPU)

Heavy I/O depending upon the number of sessions and volume of data

Heavy Memory (type of problem & number of concurrent sessions)

Large Volumes of Data


BI Architecture

SPD Server/RDBMS I/O Intensive

SAN Storage (75+ Mb/s sustained I/O throughput per session)

Large Volumes of Data


Questions


References

http://en.wikipedia.org/wiki/RAID

Optimizing SAS® Programs Course Notes

SGF 2009: How to maintain happy SAS users

SUGI 31: Solving SAS Performance Problems: Employing Host Based Tools


SIMPLICITYBEYOND

COMPLEXITY

Documents

Copyright © 2010, Scryer Analytics, LLC. All rights reserved. Optimizing SAS System Performance − A Platform Perspective Patrick McDonald Scryer Analytics,