59076 Chapter 8 Space

8/13/2019 59076 Chapter 8 Space

1/16

Teradata Architecture for IT and Developersby Tom Coffing and John Nolan

Coffing Data Warehousing. (c) 2013. Copying Prohibited.

Reprinted for Shyama Prasad Ghoshal, Accenture

[email protected]

Reprinted with permission as a subscription benefit of Skillport,

http://skillport.books24x7.com/

All rights reserved. Reproduction and/or distribution in whole or in part in electronic,paper orother forms without written permission is prohibited.
http://skillport.books24x7.com/

8/13/2019 59076 Chapter 8 Space

2/16

Chapter 8: Space

Overview

"It's kind of fun to do the impossible."- Walt Disney

When your System Arrives, there is only User named DBC

When Teradata first arrives, there is only one user called DBC. This DBC name comes from Teradata's first system called

the DBC 1012 back in 1988. The DBC stood for Database Computer and 10 to the 12 thpower is a Terabyte, thus DBC1012. Every system ever built by Teradata has a supreme user at the top of the hierarchy called DBC.

USER DBC

User DBC has always been at the top of the Teradata hierarchy. The employee that logs on with the DBC loginis the most powerful person in the company. They can essentially do anything to the Teradata databasebecause user DBC contains all Teradata Database software components and all system tables.

Before you define application users and databases, you should first use the CREATE USER statement to createa special administrative user to complete these tasks. Then keep the DBC password in a safe and hidden place.

Create an administrative user, and then logon as that user to protect sensitive data in DBC. In addition, change and securethe DBC password.

First Assignment is to create another User just under DBC

eradata Architecture for IT and Developers

Reprinted for OET7P/913461, Accenture Coffing Data Warehousing, Coffing Publishing (c) 2013, Copying Prohibited

Page 2 / 16

8/13/2019 59076 Chapter 8 Space

3/16

You can name the user anything you want. The standard is actually SYSDBA, but whom better than TeraTom to watch aTeradata system? Don't give your new user all the space. DBC needs at least 15% of the total system space for theTransient Journal and other functions that run automatically under the DBC user.

USER DBC

n System user DBC contains all Teradata Database software components and all system tables.

n Before you define application users and databases, you should first use the CREATE USER statement to create aspecial administrative user to complete these tasks.

n The amount of space for the administrative user is allocated from DBC's current PERM space. DBC becomes theowner of your administrative user and of all users and databases you subsequently create.

n Be sure to leave enough space in DBC to accommodate the growth of system tables and logs.

n You can name the user anything you would like. We have called the user SYSDBA. Create the administrative user, andthen logon as that user to protect sensitive data in DBC. In addition, change and secure the DBC password.

n To ensure perm space is from the administrative user, logon as that user to add other users and databases.

Perm and Spool Space



Page 3 / 16

8/13/2019 59076 Chapter 8 Space

4/16

Space has only to do with space on the data warehouse disks. Each AMP controls its own disk farm (each AMP isattached to four physical disks) and about 60% of each disk will be used for tables, and that is called PERM space. Theother 40% (Spool) is work space for user queries and answer sets. No AMP can get into another AMP's disk.

Perm Space is for Permanent Tables

PERM Space is where an AMP keeps its tables. That is what you need to understand. You will also find out that PERMspace also houses Secondary Indexes, Join Indexes and Permanent Journals. Just remember that PERM is for the tablesand indexes! Notice the different sizes of tables. This is because tables are different sizes.

Spool Space is work space that builds a User's Answer Sets



Page 4 / 16

8/13/2019 59076 Chapter 8 Space

5/16

Spool space is used by each AMP in order to build the answer set for the user.

Spool Space is in an AMP's Memory and on its Disk

AMPs have memory called File System Generating Cache (FSG) used for processing.

Users are Assigned Spool Space Limits



Page 5 / 16

8/13/2019 59076 Chapter 8 Space

6/16

Every Useris assigned SpoolSpace so they can submit SQL and retrieve answer sets.

The Spoolin the database Marketing is 10 GB so each user defaultsto 10 GBof Spool.

Any User in Marketing can run queries, but are aborted if they go over the 10 GBlimit.

All 3users in Marketing can query simultaneously and use 30 GBof Spool in total.

Three users in Sales defaulted to the max (5 GB) but the internwas assigned less.

The final user in DBC was given 100 GB of Spool because they are brilliant.

Spool is assigned to users, and the only way you are aborted is if you go over your spool limit. Marketing isn't restricted intotal to 10 GB of spool, but it is the individual max. Just like a speed limit of 60 MPH on the highway. Each car can go 60MPH and each User in Marketing can query at a max speed limit of 10 GB of spool.

What is the Purpose of Spool Limits?

There are two reasons for Spool Limits:

1. If a user makes a mistake and runs a query that could take weeks to run, it will abort as soon as the user goes overtheir allotted spool limit.

2. It keeps individual users from hogging the system.

Spool is assigned to users, and the only way a user is aborted is if they go over their spool limit. Marketing, Sales, andDBC have unlimited spool, but the max for each individual user is 10 GB in Marketing, 5 GB in Sales, and our power useris at 100 GB.

Why did my query Abort and say "Out of Spool"?

How is it possible that I ran out of spool?



Page 6 / 16

8/13/2019 59076 Chapter 8 Space

7/16

You ran out of spool because your query used over your limitof 10 MBsof spool.

It is also possible that you have logged onto multiple machines or ran multiple queries, and the combinationwentover 10 MBs of spool.

It is also very likely that the data you were working with was NOTevenly distributed (skewed), and this is a majorcause of Spool errors.

Spool is assigned to users and the only way a user is aborted is if they go over their spool limit. No user has ever failedbecause they are in Marketing and Marketing has only 10 GB of spool. It doesn't work that way. Thousands of users inMarketing could run queries simultaneously because Marketing has unlimited amounts of spool, but each user in Marketingis limited to 10 GBs individually.

How can Skewed Data cause me to run "Out of Spool"?

Each User's Spool limit is actually done per AMP, so if you are assigned 10 GBsof spool, and the system has10 AMPs,you are really assigned 1 GBof Spool per AMP!

If data is skewed,and you exceed your 1 GB limit on any AMP, you are "out of spool".

Spool is assigned to every user, but since Teradata is a parallel processing system, each AMP is only concerned withthem. Each AMP processes its portion of the data in parallel. Because of this philosophy your Spool Space (10 GB) isdivided among the total AMPs in the system. If you have 10 GBs of Spool and there are 10 AMPs you get 1 GB per AMP. Ifyou go over 1 GB on any AMP you are aborted and "Out of Spool".

Why did my Join cause me to run "Out of Spool"?

You might not have put in a Join Condition.SELECT First_Name, Last_Name, Department_Name

FROM Employee_Table as E

INNER JOIN

Department_Table as D

You might have Aliased the table and then fullyqualified with the real table name. SELECT First_Name, Last_Name, Department_Name

FROM Employee_Table as E

INNER JOIN

Department_Table as D

ON Employee_Table.Dept_No = D.Dept_No ;

There might be skewed data on one of thetables.



Page 7 / 16

8/13/2019 59076 Chapter 8 Space

8/16

Finding out how much Space you have

You can use this view in the DBC Data Dictionary to see your space. Notice that there are no quotes around the keywordUSER. This is because Teradata knows your login and the keyword USER means you.

Space per AMP on all tables in a Database shows Skew

What does my system look like when it first arrives?

A Lot of NULLs on a table on an Outer Join.SELECT e.*, d.* from Employee_Table as E

LEFT OUTER JOINDepartment_Table as D

ON E.Dept_No= D.Dept_No ;



Page 8 / 16

8/13/2019 59076 Chapter 8 Space

9/16

All Teradata systems start with one USER called DBC.

The first Teradata machine ever built came out in 1988 and it was called the DBC 1012. The DBC portion stood for

Database Computer. The 1012 was named because 10 to the 12

th

power is equal to a Terabyte. So, the DBC 1012 was aDatabase Computer designed to process Terabytes of data. So, every system starts with one USER called DBC and DBCowns all the PERM Space in the system.

DBC owns all the PERM Space in the system on day one

When the system is new and arrives at your company DBCis the only USER. DBCcounts up all the diskspaceattached to eachAMPand considers that PERMspace owned by DBC.

DBC owns all the disk space on day one of your system's arrival. DBC will then begin to allocate space to other databasesor users. Think of PERM space like money. If DBC has 10 Terabytes of space it is like having 10 dollars. If you give away5 dollars, you only have 5 dollars left. Spool space is more like a speed limit.

DBC's First Assignment is Spool Space



Page 9 / 16

8/13/2019 59076 Chapter 8 Space

10/16

DBC will create a database called Spool_Reserve(any name will do), but it will reserve between 20% to 40%for Spool. What really happens is that DBC creates Spool_Reserve to claim PERMSpace, but never places a

table in the database.

When a database is given PERM Space, and no object is created in that database, it is used for Spool. Spool is unusedPERM!

DBC's 2nd Assignment is to CREATE Users and Databases

DBC's 2ndassignment will be to create some USERs or DATABASEs and the hierarchy begins. If a USER or DATABASEis assigned PERM space, it can CREATE tables.

The Teradata Hierarchy Begins



Page 10 / 16

8/13/2019 59076 Chapter 8 Space

11/16

Notice in example 1 that DBC owns 10 TB of PERM space. Notice that after DBC created Spool_Reserve (4 TB), USERRetail (2 TB) and USER Financial (2 TB) that DBC now only owns only 2 TB of PERM space.

The Teradata Hierarchy Continues

USER Retail and USER Financial now create the databases and users desired.

Differences between PERM and SPOOL



Page 11 / 16

8/13/2019 59076 Chapter 8 Space

12/16

There are 1,000 users in Retail. Since Retail has 10 GB of spool, that means that every user gets 10 GB of spool. That isthe maximum limit for Retail. What it does NOT mean is that Retail is limited to only 10 GB of spool in total. Every user

could logon and run a 9 GB query taking up Terabytes of Spool, and nobody would run out of spool. Spool is system wideand calculated on an individual level only.

Databases, Users, and Views

For security purposes, the Retail tables will be kept in their own database called Retail_Tbls (in this example). The general

Retail User Population will NOT have access directly to these tables. A Database called Retail_Views houses the viewsthat access the tables. So, the DBA will create Access Rights that allow the views to read the tables and the Users toSELECT from the views.

What are Similarities between a DATABASE and a USER?

A Database or a User can be assigned PERM Space

If the Database Marketingis assigned 10 GB of PERM, that means it can hold up to 10 GB of Permanent Tables.



Page 12 / 16

8/13/2019 59076 Chapter 8 Space

13/16

What is the Difference between a DATABASE and a USER?

A USER has a login and password and therefore canrun queries.

Objects that take up PERM Space

Permanent Space (Perm space) is the maximum amount of storage assigned to a user or database for holding:

n Table Rows

n Fallback Tables

n Secondary Index Subtables

n Stored Procedures

n User Defined Functions (UDFs)

n Permanent Journals

Viewsand Macrosdo NOTtake up any Perm Space!

A Series of Quizzes on Adding and Subtracting Space

If the User Mariais assigned 10 GB of PERM, that means she can hold up to 10 GB of Permanent Tables.

A Database or a User can be assigned Spool Space

If the Database Marketingis assigned 10 GB of Spool, that means all users under marketing can each run 10 GBqueries.

If the User Mariais assigned 10 GB of Spool, that means she can run up to 10 GB queries, and any user created underMaria will default to 10 GB queries.

Marketing has 10 GB of Perm and Spool. Sales has 5 GB Perm and Spool.

Marketing then Creates Stan and gives him 1 GB Perm and 10 GB Spool.

Sales then Creates Mary and gives her 1 GB Perm and 5 GB Spool.



Page 13 / 16

8/13/2019 59076 Chapter 8 Space

14/16

Answer 1 to Quiz on Space

Space Transfer Quiz

Stan has just been transferred to Sales.

Marketing has 10 GB of Perm and Spool. Sales has 5 GB Perm and Spool.

Marketing then Creates Stan and gives him 1 GB Perm and 10 GB Spool.

Sales then Creates Mary and gives her 1 GB Perm and 5 GB Spool.

If a USER is dropped, their PERM Space goes up to the immediate parent.

If a USER is transferred, (GIVE Statement) they take their space with them.

After the transfer, how much Perm / Spool is in:

Marketing _______________ ______________



Page 14 / 16

8/13/2019 59076 Chapter 8 Space

15/16

Answer to Space Transfer Quiz

Stan has just been transferred to Sales.

Drop Space Quiz

What happensNOWif Stan is Dropped?

Answers to Drop Space Quiz

Sales _______________ ______________

Stan _______________ ______________

If a USER is dropped, their PERM Space goes up to their immediate parent.


After the transfer, how much Perm / Spool is in:

Marketing 9 GB Perm 10 GB Spool

Sales 4 GB Perm 5 GB Spool

Stan 1 GB Perm 10 GB Spool



After the drop, how much Perm / Spool is in:

Marketing ___________ ___________

Sales ___________ ___________

Stan ___________ ___________





Page 15 / 16

8/13/2019 59076 Chapter 8 Space

16/16

What happensNOWif Stan is Dropped?

After the drop how much Perm / Spool is in:

Marketing 9 GB Perm 10 GB Spool

Sales 5 GB Perm 5 GB Spool

Stan dropped (0) dropped (0)


Page 16 / 16

Documents

59076 Chapter 8 Space