Dr. Uwe Röhm School of Information Technologies INFO2120 – INFO2820 – COMP5138 Database Systems Week 8: Database Application Development (Kifer/Bernstein/Lewis

Dr. Uwe RöhmSchool of Information Technologies

INFO2120 – INFO2820 – COMP5138Database Systems Week 8: Database Application Development(Kifer/Bernstein/Lewis – Chapter 8; Ramakrishnan/Gehrke – Chapter 6; Ullman/Widom – Chapter 9)

INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-2

Outline

Database Application Architectures

Client-side DB Application Development Call-level Database APIs: PHP/PDO and JDBC Database Application Design Principles

Server-side DB Application Development Stored Procedures

Based on slides from Kifer/Bernstein/Lewis (2006) “Database Systems”and from Ramakrishnan/Gehrke (2003) “Database Management Systems”,

and also including material from Fekete and Röhm.

Database Applications



Data-intensive Systems

Three types of functionality:

The system architecture determines whether these three components reside on a single system (1-tier) or whether they are distributed across several tiers

Presentation Logic- Input – keyboard/mouse- Output – monitor/printer

Processing Logic- Business rules - I/O processing

Data Management(Storage Logic)

- data storage and retrieval

GUI Interface

Procedures, functions,programs

DBMS activities


Possible System Architectures

1-Tier Architectures: Centralised Systems

2-Tier Architectures: Client-Server Systems

3-Tier Architectures Client - Server - Middleware Internet Applications Web Databases


Centralized System

Presentation Services - displays forms, handles flow of information to/from screen

Application Services - implements user request, interacts with DBMS Transactional properties automatic (isolation is trivial) or not required

(this is not really an enterprise) DBMS runs within the user process Examples:

Access; any application with an integrated DB (e.g. SQLite) – from smartphones to PCs

presentation application

services services DBMS

user module

centralized system

API


Client/Server Model of TPS

DBMS

database servermachine

presentation applicationservices services

presentation applicationservices services

• •

•

client machines

communication /network


Three-Tiered Model of TPS

DBMS

database servermachine

presentationserver •

• •

client machines

communication (IPC or network)

presentationserver

applicationserver

application / webserver machine

Presentation Tier Middle Tier Data ManagementTier


Interactive vs. Non-Interactive SQL

Interactive SQL: SQL statements input from terminal; DBMS outputs to screen Inadequate for most uses

It may be necessary to process the data before output Amount of data returned not known in advance SQL has very limited expressive power (not Turing-complete)

Non-interactive SQL: SQL statements are included in an application program written in a host language, like C, Java, COBOL Nowadays also: as embedded in dynamic webpages

Client-side vs. Server-side application development Server-side: Stored Procedures and Triggers


Outline


Client-side DB Application Development Call-level Database APIs: PHP/PDO Call-level Database API for Java: JDBC Database Application Design Principles


Based on slides from Kifer/Bernstein/Lewis (2006) “Database Systems”and from Ramakrishnan/Gehrke (2003) “Database Management Systems”,

and also including material from Fekete and Röhm.


SQL in Application Code

SQL commands can be called from within a host language (e.g., C++ or Java) program. SQL statements can refer to host variables (including special

variables used to return status). Must include a statement to connect to the right database.

Two main integration approaches: Statement-level interface (SLI)

Embed SQL in the host language (Embedded SQL in C, SQLJ) Application program is a mixture of host language statements and SQL

statements and directives

Call-level interface (CLI) Create special API to call SQL commands (JDBC, ODBC, PHP, …) SQL statements are passed as arguments to host language (library)

procedures / APIs


DBMS

JDBC, ODBC, PDO, …

NativeInterface

CLI

Call-level Interfaces and Database APIs

Rather than modify compiler, add library with database calls (API) Special standardized interface: procedures/objects Pass SQL strings from language,present result sets in language-friendly way Supposedly DBMS-neutral

a “driver” executes the calls and translates them into DBMS-specific code database can be across a network

Several Variants SQL/CLI: “SQL Call-Level-Interface”

Part of the SQL-92 standard; “The assembler under the APIs”

ODBC: “Open DataBase Connectivity” Side-branch of early version of SQL/CLI Enhanced to: OLE/db, and further ADO.NET

JDBC: “Java DataBase Connectivity” Java standard

PDO Persistency standard for PHP Data Objects

PDO – PHP Data Objects


PHP

PHP is a scripting language for dynamic websites PHP – original recursive acronym for "PHP: Hypertext Preprocessor” embedded into HTML Indicated by <?php PHP-code ?>

There are several different approacheson how to connect in PHP scripts to databases Vendor-specific database extensions

e.g. pgsql (PostgreSQL) or pci8 (Oracle)

=> Outdated!

Some abstraction layers on top (typically for PHP 5.1 onwards) e.g. PDO (“PHP Data Objects”) Generic DB library also via PEAR (PHP Extension&Application Repository)


PHP 101

A dynamically-typed scripting language Embedded in normal HTML page Offers the usual programming constructs:

Variable Condition statements Loops Input/output

Example (example.php):<html><head><title>PHP Test</title></head><body> <h1>This is a PHP test</h1> Today is <?php echo "a just normal day" ?>, the <?php echo date("F j, Y") ?>.</body></html>


PHP 101: Variables in PHP

Must begin with $ Dynamically typed – it is OK to not declare a type for a variable. But you give a variable a value that belongs to a “class,” in which case,

methods of that class are available to it.

String Variables: PHP solves a very important problem for languages that commonly construct

strings as values: How do I tell whether a substring needs to be interpreted as a variable and replaced

by its value?

PHP solution: Double quotes means replace; single quotes means don’t.

$100 = ”one hundred dollars”;

$sue = ’You owe me $100.’;

$joe = ”You owe me $100.”; Value of $sue is ’You owe me $100’,

while the value of $joe is ’You owe me one hundred dollars’.


PHP 101: Array Variables in PHP

Two kinds: numeric and associative. Numeric arrays are ordinary indexed 0,1,…

Example: $a = array("Paul", "George", "John", "Ringo");

Then $a[0] is "Paul", $a[1] is "George", and so on.

Elements of an associative array $a are pairs x => y, where x is a key string and y is any value.

If x => y is an element of $a, then $a[x] is y. Example: $a = array("bass" => "Paul", "guitar" => "George",

"guitar2"=>"John", "drums" => "Ringo"); Then $a[‘bass’] is "Paul", $a[‘drums’] is "Ringo", and so on.


PDO – PHP Data Objects

Introduced since PHP 5.1 (in 2005) Object-oriented extension to PHP for database programming

that provides a database abstraction layer Generic driver model to connect to different database engines

via the same API Significant improvement over the previous proprietary APIs

URL: http://www.php.net/manual/en/intro.pdo.php



PDO Example<?php function printClassList ($unit_of_study, $user, $pwd) { try { /* connect to the database */ $conn=new PDO('pgsql:host=localhost:port=5432:dbname=unidb", $user, $pwd);

/* prepare a dynamic query */ $stmt = $conn->prepare('SELECT name FROM Student NATURAL JOIN Enrolled WHERE uosCode = :uos'); $stmt->bindValue( ':uos', $unit_of_study, PDO::PARAM_STR, 8 ); /* execute the query and loop through the resultset */ $results = $stmt->execute(); while ( $row = $results->fetch() ) { print " student: ", $row['name']; }

/* clean up */ $stmt->closeCursor(); $conn = null; } catch (PDOException $sqle) { /* error handling */ print "SQL exception : ", $sqle->getMessage(); } } ?>


Core Problems with SQL Interfaces

(1) Establishing a database connection

(2) Static vs. Dynamic SQL

(3) Mapping of domain types to data types of host Concept of host variable How to treat NULL values?

(4) Impedance Mismatch: SQL operates on sets of tuples Host languages like C do not support a set-of-records abstraction,

but only a one-value-at-a-time semantic Solution: Cursor Concept

Iteration mechanism (loop) for processing a set of tuples

(5) Error handling


(1): PDO Run-Time Architecture

DBMS

PHP code PDO

MySQL

driver

PostgreSQL

driver

Oracledriver

MySQLdatabase

PostgreSQLdatabase

Oracledatabase

PDO is DBMS independent PDO functions are generic PDO allows to connect to specific driver

Using parameters of PDO constructor Even to different databases from the same program

Database drivers are loaded and used at run-time

. . .


PDO Connections

Session with a data source started by creating a PDO object: $conn = new PDO( DSN, $userid, $passwd [,$params] );

Data Source Name (DSN) of the form <driver>:<connectionParameter1>;<connectionParameter2>;…

For example with PostgreSQL:$conn = new PDO(

"pgsql:host=postgres.it.usyd.edu.au;dbname=unidb",$user,$pw);

driver connectionParameters

Details: http://www.php.net/manual/en/pdo.construct.php

db login

PDO Connection Drivers

Driver support for variety of DBMSs MySQL (prefix: mysql) PostgreSQL (prefix: pgsql) Oracle (prefix: oci) IBM DB2 (prefix: ibm) SQL Server (prefix: sqlsrv) sqlite (prefix: sqlite) … DSN syntax and additional DB parameters vary for each driver Check manuals: http://www.php.net/manual/en/pdo.drivers.php

Example for Oracle: $conn = new PDO( "oci:dbname=oracle10g.it.usyd.edu.au:1521/ORCL", $user, $pwd );INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-23

Note:drivers need to be installed first as part of the PHP server's configuration…


PDO Connection Example

<?php try { /* connect to the database */ $conn = new PDO('pgsql:host=localhost:port=5432:dbname=unidb", $user, $pw);

/* query database */ $stmt = $conn->query('SELECT name FROM Student WHERE studID=4711');

… Do Actual Work ….

/* clean up */ $stmt->closeCursor(); $conn = null; } /* error handling */ catch (PDOException $sqle) { print "SQL exception : ", $sqle->getMessage(); }?>

PDO Objects


PDO

__construct(…)query()

prepare()beginTransaction(

)commit()rollBack()

…

prepare(stmt)

query(stmt)

PDOStatement

string $queryString

bindValue()bindParam()bindColumn()

execute()fetch()

fetchColumn()fetchAll()

nextRowset()closeCursor()errorCode()

…

PDOException

array $errorInfo

getMessage()getPrevious()

getCode()getFile()getLine()

…

PDO Class Interface Start SQL statements

query() for static SQL, or prepare() for parameterized SQL queries exec() for immediately executing some SQL; returns num rows

Transaction control beginTransaction() starts a database transaction (otherwise: autocommit)

commit() successfully finishes current transaction rollBack() aborts current transaction inTransaction() checks whether there's an active transaction

Sets/gets connection parameters (often driver specific) getAttribute(…) setAttribute(…)

Error Handling errorCode() errorInfo()


[cf. http://www.php.net/manual/en/class.pdo.php]

Side Note on DB Connections

Establishing a database connection takes some time… Network communication, memory allocation, dbs authorization

So do this only once in your program … but not for individual SQL queries

Modern, multi-threaded applications will typically want to have a pool of connections that are re-used Might be handled by your runtime library

(that's what happens in PHP) But for, e.g., Java programs better be mindful of connection costs!



(2) Executing SQL Statements

Three different ways of executing SQL statements: PDOStatement PDO::query(sql) semi-static SQL statements PDOStatement PDO::prepare(sql) parameterized SQL statements num_rows PDO::exec(sql) immediately run SQL command

PDOStatement class:Precompiled, parameterized SQL statements: Structure is fixed after call to PDO::prepare() Values of parameters are determined at run-time Fetch and store routines are executed when

PDOStatement::execute() is executed to communicate argument values with DBMS

PDOStatement::execute() can be invoked multiple times with different values of in parameters

Each invocation uses same query execution plan


PDOStatement with Semi-static SQL

Simplest way to execute some static SQL query:<?php try { /* connect to the database */

…

/* query database */ $stmt = $conn->query('SELECT name FROM Student WHERE studID=4711');

$name = $stmt->fetchColumn(); /* just fetch the single return value */

print $name;

/* clean up */ $stmt->closeCursor();

} /* error handling */ catch (PDOException $sqle) { print "SQL exception : ", $sqle->getMessage(); }?>

This is 'semi-static' because one could construct the SQL string during runtime. Warning: DON'T DO THIS! Use parameterized queries instead! (cf. SQL Injection problem later)


Static vs. Dynamic SQL

SQL constructs in an application can take two forms: Standard SQL statements (static embedded SQL):

Useful when SQL portion of program is known at compile time Only available with Embedded SQL in compiled language…

Directives (dynamic SQL): Useful when SQL portion of program not known at compile time. Application constructs SQL statements at run time as values of host language variables that are manipulated by directives.

Problem is: PHP is not a compiled language;So everything in PHP/PDO is by definition dynamic SQL…

Still: Try to avoid constructing SQL strings in the program from user input, rather use fixed query structures with parameters (parameterized queries)


$query = "SELECT E.studId FROM EnrolledEnrolled E WHERE E.uosCode = ? AND E.semester = ?";

$stmt = $conn->prepare ( $query );• Prepares the statement• Creates a prepared statement object, $stmt, containing the prepared statement• PlaceholdersPlaceholders (?) mark positions of in in parameters; special API is provided to plug the actual values in positions indicated by the ??’s

placeholders

Approach 2: Preparing and Executing a parameterized Query


var $uos_code, $semester;………$stmt->bindValue(1, $uos_code); // set value of first inin parameter $stmt->bindValue(2, $semester); // set value of second inin parameter

$stmt->execute ();• Evaluates parameters bound with setParameter() only now• Executes the query• Associates a result set with the same PDOStatement

while ( $row = $stmt->fetch ( ) ) { // advance the cursor $j = $row['studId']; // fetch output int-value …process output value…}

Preparing & Executing a Query (cont’d)


(3) Host Variables

Data transfer between DBMS and application Mapping of SQL domain types to data types of host language PHP PDO:

Host variables are normal mixed PHP variables that are dynamically typed and accessed during runtime:$studid = 12345;$stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=?");$stmt->bindValue(1, $studid);

Note: in statement-level APIs such as ESQL/C: Host variables must be declared before usage

EXEC SQL BEGIN DECLARE SECTION; int studid = 12345; char sname[21];

EXEC SQL END DECLARE SECTION;

Variables shared by

host and SQL

PDO: Parameterized Queries

Two Approaches for specifying query parameters:

1. Anonymous Placeholders $studid = 12345;$stmt = $conn->prepare(

"SELECT name FROM Student WHERE sid=?");$stmt->bindValue(1, $studid);

2. Named Placeholders$studid = 12345;$stmt = $conn->prepare(

"SELECT name FROM Student WHERE sid=:s");$stmt->bindValue(':s', $studid);


PDO: Binding Host Variables

Two Approaches for binding host variables as input params: PDOStatement::bindValue() binds value of host variable at call PDOStatement::bindParam() binds host variable by reference

Example$studid = 12345;

$stmt = $conn->prepare( "SELECT name FROM Student WHERE sid=:s");

$stmt->bindParam(':s', $studid);

$studid = 56789; $stmt->execute();


PDO: Typing Host Variables

Host variables can be dynamically typed$stmt = $conn->prepare(

"SELECT name FROM Student WHERE sid=:s");$stmt->bindValue(':s', 12345);

or type-safe with (optional) third type parameterPDO::PARAM_INT represents an SQL INTEGERPDO::PARAM_STR represents a SQL CHAR or VARCHARPDO::PARAM_BOOL represents a booleanPDO::PARAM_LOB represents a SQL large object data typePDO::PARAM_NULL represents SQL NULL

Example:$studid = 12345;$stmt = $conn->prepare(

"SELECT name FROM Student WHERE sid=:s");$stmt->bindValue(':s', $studid, PDO::PARAM_INT);


PDO: Binding Output Variables For binding output parameters:

PDOStatement::bindColumn() binds a output column to a PHP varPDOStatement::fetch(PDO::FETCH_BOUND) fetches values into vars

Can also be strongly typed during bindColumn() call Example:

$sql = "SELECT name,gender,address FROM Student WHERE sid=4711";

$stmt= $conn->prepare($sql);$stmt->execute();

/* option 1: bind by column number */

$stmt->bindColumn(1, $name, PDO::PARAM_STR);

$stmt->bindColumn(2, $gender, PDO::PARAM_STR );

/* option 2: bind by column name */

$stmt->bindColumn('address', $addr);

$row = $stmt->fetch(PDO::FETCH_BOUND);

print $name, '\t',$gender, '\t',$addr, '\n';INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm) 08-37


Preparing & Executing Dynamic Updates

$sql="INSERT INTO Student VALUES(?,?,?,?)";

$pstmt = $conn->prepare($sql);

$pstmt.bindValue(1, $sid, PDO::PARAM_INT);

$pstmt.bindValue(2, $sname, PDO::PARAM_STR);

$pstmt.bindValue(3, $birthdate, PDO::PARAM_STR);

$pstmt.bindValue(4, $country, PDO::PARAM_STR);

/* execute with latest values from host variables */

$pstmt.execute();

$numRows1 = $pstmt.rowCount();

/* execute again with dynamically bound values */

$pstmt.execute( array(1234,'Obama',NULL,'USA') );

$numRows2 = $pstmt.rowCount();


(4) Buffer Mismatch Problem(also: Impedance Mismatch)

SELECTcursor

Base table

Result set(or pointers to it)application

Problem: SQL deals with tables (of arbitrary size); host language program deals with fixed size buffers How is the application to allocate storage for the result of a SELECT

statement? Solution: Cursor concept

Fetch a single row at a time


Mapping of Sets: Cursor Concept

Result set – set of rows produced by a SELECT statement Cursor – pointer to a row in the result set. Cursor operations:

Declaration Open – execute SELECT to determine result set and initialize pointer Fetch – advance pointer and retrieve next row (JDBC: next() call) Close – deallocate cursor

Cursor in PDO – via PDOStatement

Cursor concept with PHP/PDO:$stmt = $conn->prepare("SELECT title,name,address FROM Emp");$stmt->execute();while ( $row = $stmt->fetch() ) {

$data = $row[0] . "\t" . $row[1] . "\t" . $row[2] . "\n";

print $data;

}$stmt->closeCursor();

PHP language natively supports arrays; good for small results$stmt->execute();$resultset = $stmt->fetchAll();

foreach ( $resultset as $row ) {

print_r($row);

}just be mindful that this can be VERY memory hungry for large results


You can address result columns either by name or position

PDOStatement::fetch()

mixed PDOStatement::fetch ( [ int $fetch_style [, int $cursor_orientation = PDO::FETCH_ORI_NEXT [, int $cursor_offset = 0 ]]] )where

$fetch_styleControls how new result row will be returned to caller

PDO::FETCH_ASSOC as an associative array PDO::FETCH_NUM as numerically-index array, starting at 0 PDO::FETCH_BOTH both of above (DEFAULT) PDO::FETCH_BOUND fetch in bound output column variables …

$cursor_orientationWhether it is a scrollable cursor, or not (DEFAULT)

$cursor_offsetfor a scrollable cursor, the absolute row number to fetch first


NULL Handling in PDO

Remember: Null values mean neither 0 nor empty string Hence special indication of unknown values needed. In PHP this is quite natural, as PHP supports NULL:

$stmt = $conn->query("SELECT gender FROM Student …");$row = $stmt->fetch(); if ( is_null($row['gender']) ){ /* null value */ }else{ /* no null value */}

Other languages require a special indicator variable. Eg. C: EXEC SQL select gender into :gender:indicator from Student where sid=4711;if ( indicator == -1 ){ /* null value */ }else{ /* no null value */}


PHP: isset() vs. empty() vs. is_null() isset(var)

Returns TRUE if var exists and is not NULL, otherwise returns FALSE.

[http://php.net/manual/en/function.isset.php]

empty(var) Returns FALSE if var

exists and has a non-empty, non-zero value, otherwise TRUE.

[http://php.net/manual/en/function.empty.php]


is_null(var)

Returns TRUEif var === NULL,otherwise FALSE

[http://php.net/manual/en/function.is-null.php]

http://techtalk.virendrachandak.com/php-isset-vs-empty-vs-is_null/

NULL Handling in PDO (cont'd)

In PDO, the NULL behaviour can be further configured PDO connection attribute PDO::ATTR_ORACLE_NULLS

(available with all drivers, not just Oracle): PDO::NULL_NATURAL no conversion. PDO::NULL_EMPTY_STRING empty string is converted to NULL. PDO::NULL_TO_STRING NULL is converted to an empty string.



(5) Error Handling

Multitude of potential problems No database connection or connection timeout Wrong login or missing privileges SQL syntax errors Empty results NULL values …

Hence always check database return values, Provide error handling code, resp. exception handlers Gracefully react to errors or empty results or NULL values NEVER show database errors to end users

Not only bad user experience, but huge security risk…


You should avoid this!

Also cf. error #... Of http://www.sans.org/top25-software-errors/

Error Handling with PDO

Two mechanism:

1. Explicitly testing for error codes after each statement Both PDO and PDOStatement objects provide error status functions: errorCode() fetches the SQLSTATE of last statement errorInfo() fetches extended error information of last stmt.

2. Error handling via normal exception mechanism of PHP This has to be configured on a connection (PDO) object via PDO::setAttribute()

PDO::ATTR_ERRMODE: Error reporting. PDO::ERRMODE_SILENT: Just set error codes. PDO::ERRMODE_WARNING: Raise E_WARNING. PDO::ERRMODE_EXCEPTION: Throw exceptions.

Example:try { …} catch ( PDOException $ex ) { print ex.getMessage();}


SQLSTATE

a five characters alphanumeric identifier defined in SQL-92 Two characters error class value Followed by a three characters sub-class value

Examples: 00000 successful completion Class 01 indicates a warning

eg. 01004 Warning: string data, right truncation or 01007 Warning: privilege not granted

Class 02: no data error (SQLSTATE: 02000) Class 08: connection error

eg. 08001 Error: unable to establish SQL connection

…

List of available SQLSTATEs:http://docstore.mik.ua/orelly/java-ent/jenut/ch08_06.htm


Exception Handling with PDO

Class PDOException PDOException::getMessage() returns exception message PDOException::getCode() returns the exception code …

Example: 1. Configure to have thrown exceptions on SQL errrors$dbh->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);

2. Catch-Try block around PDO statements:try { …} catch ( PDOException $ex ) { print ex.getMessage();}



Cf. Example in PDO<?php function printClassList ($unit_of_study, $user, $pwd) { try { /* connect to the database */ $conn=new PDO('pgsql:host=localhost:port=5432:dbname=unidb", $user, $pwd);

/* prepare a dynamic query */ $stmt = $conn->prepare('SELECT name FROM Student NATURAL JOIN Enrolled WHERE uosCode = :uos'); $stmt->bindParam( ':uos', $unit_of_study, PDO::PARAM_STR, 8 ); /* execute the query and loop through the resultset */ $results = $stmt->execute(); while ( $row = $results->fetch() ) { print " student: ", $row['name']; }

/* clean up */ $stmt->closeCursor(); $conn = null; } catch (PDOException $sqle) { /* error handling */ print "SQL exception : ", $sqle->getMessage(); } } ?>

Host variableconcept

cursor concept

error handling

Time for a Break…

INFO2120/INFO2820/COMP5138 "Database Systems" - 2013 (U. Röhm)

JDBCJava Database Connectivity

The following part is meant as background reading for students doing the assignment in Java/JDBC – such as Postgraduate students from COMP5138…



JDBC - “Java Database Connectivity”

JDBC is a Java API for communicating with database systems supporting SQL

JDBC supports a variety of features for querying and updating data, and for retrieving query results

JDBC also supports metadata retrieval, such as querying about relations present in the database and the names and types of relation attributes

Model for communicating with the database: Open a connection Create a “statement” object Execute queries using the Statement object to send queries and

fetch results Exception mechanism to handle errors


JDBC Exampleimport java.sql.*;public void printLecturerName ( String unit_of_study, String user, String pwd){ try {/* connect to the database */ Class.forName ("org.postgresql.Driver"); Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd);

/* prepare the dynamic query */ PreparedStatement stmt = conn.prepareStatement( “select name from Student natural join Enrolled where uosCode=?”); stmt.setString(1, unit_of_study);

/* execute the query and loop through the resultset */ ResultSet rset = stmt.executeQuery(); while ( rset.next() ) { System.out.println(“ student: “ + rset.getString(1)); }

/* clean up */ stmt.close(); conn.close(); } catch (SQLException sqle) { /* error handling */ System.out.println("SQLException : " + sqle); }}


(1): JDBC Run-Time Architecture

DBMS

applicationdriver

manager

PostgreSQL

driver

SQLServer

driver

Oracledriver

PostgreSQLdatabase

SQLServerdatabase

Oracledatabase

JDBC is DBMS independent JDBC functions are generic DriverManager allows to connect to specific driver

Even to different databases from the same program

Database drivers are loaded and used at run-time JDBC was one of the first APIs giving this flexibility and a lot of effort was put into

making this as flexible as possible also during runtime. Hence one indirection more than with PHP/PDO and also more effort to include legacy (non-Java) drivers.

. . .


JDBC Architecture

Four architectural components: Application

(initiates and terminates connections, submits SQL statements) Driver manager

(loads JDBC driver during runtime) Note: This part is not explicitly present with PHP/PDO as with PHP, the

drivers have to be pre-configured as part of the PHP configuration

Driver (connects to data source, transmits requests and returns/translates results and error codes)

Data source (processes SQL statements)


JDBC Driver Management

Steps to submit a database query: Load the JDBC driver (during runtime as part of the program) Connect to the data source Execute SQL statements

All drivers are managed by the DriverManager class

Loading a JDBC driver (variants): Class.forName(driver_class_name)

For example for PostgreSQL: Class.forName(“org.postgresql.Driver”); or example for Oracle: Class.forName(“oracle.jdbc.driver.OracleDriver”);

When starting the Java application:-Djdbc.drivers=org.posgresql


JDBC Connections

A session with a data source is started through the creation of a Connection object Via the DriverManager:DriverManager.getConnection(DB_URL,userid,passwd);

Database URL of the form jdbc:<subprotocol>:<connectionParameters>

For example with PostgreSQL:Connection conn = DriverManager.getConnection("jdbc:postgresql://localhost:5432/unidb",user,pwd);

subprotocol connectionParameters


Example JDBC Code

import java.sql.*;

public static void JDBCexample( String user, String pwd )

{

try {

Class.forName ("org.postgresql.Driver");

Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd);

Statement stmt = conn.createStatement();

… Do Actual Work ….

stmt.close();

conn.close();

}

catch (SQLException sqle) {

System.out.println("SQLException : " + sqle); }

}


Connection Class Interface

Sets isolation level for the current connection. public int getTransactionIsolation() and

void setTransactionIsolation(int level)

Specifies whether transactions in this connection are read-only public boolean getReadOnly() and

void setReadOnly(boolean b)

If autocommit is set, then each SQL statement is considered its own transaction. Otherwise, a transaction is committed using commit(), or aborted using rollback(). public boolean getAutoCommit() and

void setAutoCommit(boolean b)

Checks whether connection is still open. public boolean isClosed()


(2) Executing SQL Statements

Three different ways of executing SQL statements: Statement (both static and dynamic SQL statements) PreparedStatement (semi-static SQL statements) CallableStatement (stored procedures)

PreparedStatement class:Precompiled, parameterized SQL statements: Structure is fixed Values of parameters are determined at run-time Fetch and store routines are executed at client when EXECUTE is

executed to communicate argument values with DBMS EXECUTE can be invoked multiple times with different values of in

parameters Each invocation uses same query execution plan


String query = “SELECT E.studId FROM EnrolledEnrolled E” +“WHERE E.uosCode = ? AND E.semester = ?”;

PreparedStatement ps = con.prepareStatement ( query );• Prepares the statement• Creates a prepared statement object, ps, containing the prepared statement• PlaceholdersPlaceholders (?) mark positions of in in parameters; special API is provided to plug the actual values in positions indicated by the ??’s

placeholders

Preparing and Executing a Query


String uos_code, semester;………ps.setString(1, uos_code); // set value of first inin parameter ps.setString(2, semester); // set value of second inin parameter

ResultSet res = ps.executeQuery ( );• Creates a result set object, res• Executes the query• Stores the result set produced by execution in res

while ( res.next ( ) ) { // advance the cursor j = res.getInt (“studId”); // fetch output int-value …process output value…}

Preparing & Executing a Query (cont’d)


(3) Host Variables

Data transfer between DBMS and application Mapping of SQL domain types to data types of host language JDBC:

Host variables are normal Java variables that are accessed using specific, strongly-typed functions.

Example:int studid = 12345;Statement stmt = con.Statement( “SELECT name FROM Student WHERE sid=?”);stmt.setInt(1, studid);


Preparing & Executing Dynamic Updates

String sql=“INSERT INTO Student VALUES(?,?,?,?)”;

PreparedStatment pstmt=con.prepareStatement(sql);

pstmt.clearParameters();

pstmt.setInt(1,sid);

pstmt.setString(2,sname);

pstmt.setDate(3, new java.sql.Date(birthdate));

pstmt.setString(4, country);

// we know that no rows are returned, thus we use executeUpdate()

int numRows = pstmt.executeUpdate();

Note: PreparedStatement.executeUpdate only returns the number of affected records


(4) JDBC: ResultSets

PreparedStatement.executeQuery returns data, encapsulated in a ResultSet object (a cursor)

ResultSet rs=pstmt.executeQuery(sql);// rs is now a cursorwhile (rs.next()) { // process the data}

rs.close()

A ResultSet is a very powerful cursor: previous(): moves one row back absolute(int num): moves to the row with the specified number relative (int num): moves forward or backward first() and last() wasNull() dealing with NULL values


Matching Java and SQL Types

getTimestamp()java.sql.TimeStampTIMESTAMP

getTime()java.sql.TimeTIME

getDate()java.sql.DateDATE

getFloat()DoubleREAL

getInt()IntegerINTEGER

getDouble()DoubleFLOAT

getDouble()DoubleDOUBLE

getString()StringVARCHAR

getString()StringCHAR

getBoolean()BooleanBIT

ResultSet get method

Java classSQL Type

NULL Handling in JDBC

Remember: Null values mean neither 0 nor empty string Hence special indication of unknown values needed JDBC:

wasNull() call for individual columns on ResultSet

Embedded SQL in C etc.: null-indicator variable Example:

EXEC SQL select name into :sname:indicator from Student where sid=:studid;

if ( indicator == -1 ){ /* null value */ }else{ /* no null value */}



(5) JDBC Error Handling:Exceptions and Warnings

Most of java.sql can throw and SQLException if an error occurs. SQLWarning is a subclass of SQLException; not as severe (they are not

thrown and their existence has to be explicitly tested)

try { stmt=con.createStatement(); warning=con.getWarnings(); while(warning != null) { // handle SQLWarnings; warning = warning.getNextWarning(); } con.clearWarnings(); stmt.executeUpdate(queryString); warning = con.getWarnings(); …} //end trycatch( SQLException SQLe) { // handle the exception}


Cf. Example in JDBCimport java.sql.*;public void printLecturerName ( String unit_of_study, String user, String pwd){ try {/* connect to the database */ Class.forName ("org.postgresql.Driver"); Connection conn = DriverManager.getConnection( "jdbc:postgresql://localhost:5432/unidb",user,pwd);

/* prepare the dynamic query */ PreparedStatement stmt = conn.prepareStatement( “select name from Student natural join Enrolled where uosCode=?”); stmt.setString(1, unit_of_study);

/* execute the query and loop through the resultset */ ResultSet rset = stmt.executeQuery(); while ( rset.next() ) { System.out.println(“ student: “ + rset.getString(1)); }

/* clean up */ stmt.close(); conn.close(); } catch (SQLException sqle) { /* error handling */ System.out.println("SQLException : " + sqle); }}

Host variable concept

cursor concept

error handling


JDBC: Access to Database Metadata

The class DatabaseMetaData provides information about database relations

Has functions for getting all tables, all columns of the table, primary keys etc.

E.g. to print column names and types of a relation

DatabaseMetaData dbmd = conn.getMetaData( );

ResultSet rs = dbmd.getColumns( null, “UNI-DB”, “Student”, “%” ); //Arguments: catalog, schema-pattern, table-pattern, column-pattern // Returns: 1 row for each column, with several attributes such as // COLUMN_NAME, TYPE_NAME, etc.

while ( rs.next( ) ) { System.out.println( rs.getString(“COLUMN_NAME”) ,

rs.getString(“TYPE_NAME”); }

There are also functions for getting information such as Foreign key references in the schema Database limits like maximum row size, maximum no. of connections, etc


This Week’s Agenda


Client-side DB Application Development Call-level Database APIs: PDO and JDBC Database Programming Design Principles



Design Principles for DB Applications

For larger project, the correct ‘layering’ of an app is crucial Presentation layer Business logic Data access layer Data management

General Design Principles: Separate Data Access Layer and the remaining application logic

Dynamic web-languages such as PHP are very tempting in this respect, but horrible to maintain, extend or simply keep secure!

Rather: all database access logic should be in its own dedicated data access object and data source wrapping module

Do proper error handling don’t expose internal database error messages

Validate any user input; use dynamic SQL with parameter parsing Secure your code against SQL injection attacks

cf. Model-Viewer-Control (MVC) principle




SQL Code Injection Vulnerability

SQL-Injectionto infiltrate a SQL database with own SQL commands. Can be used to execute SQL statements with elevated privileges or

to impersonate another user.

Without direct database connection (e.g. web application) Injecting SQL via un-checked user input. Exploiting buffer overflows.

Oracle standard packages have many buffer overflows.

Output on attacker’s screen.

With a direct database connection SQL Injection in built-in or user-defined procedures. Buffer overflows in built-in or user-defined procedures.

Risk when a procedure is not defined with the AUTHID CURRENT_USER keyword (executes with the privileges of the owner


Hacking a Web Database

Web-applications often construct a SQL-statement from separate strings. If a web-application does not thoroughly check the user’s input, in general

every database on every operating system is vulnerable.

Example: Consider the following SQL query in PHP$result=$conn->query('SELECT * FROM users WHERE username="'.$_POST['username'].'"'); The query selects all rows from the users table where the username is equal to

the one put in the query string. Problem: quotes in $_POST['username'] not escaped & the string not validated Consider what would happen if we supply:

" OR 1 OR username = " (a double-quote, followed by a textual " OR 1 OR username = " followed by another double-quote)….

Also, another line of SQL code can be added by adding a quote and a semicolon to the end so that the line…

Many more problems possible…


Protecting a Web Database Be careful to check all parameters which can end up in such

SQL statements! Never trust user provided data!

Use dynamic SQL statements with explicit, type-checked parameters (bindValue() and bindParam() functions).

Restrict the privileges of the user/role of the web application E.g. with Oracle: Revoke EXECUTE privilege on Oracle standard

packages when not needed. Specially for the PUBLIC role.

Patch, patch, patch ;-) Also: NEVER directly return database error messages

Not very user-friendly AND it gives attackers hints


This Week’s Agenda


Client-side DB Application Development Database-APIs: PDO and JDBC Database Application Design Principles



Stored Procedures

Run application logic within the database server Included as schema element (stored in DBMS) Invoked by the application

Advantages: Central code-base for all applications Improved maintainability Additional abstraction layer

(programmers do not need to know the schema) Reduced data transfer Less long-held locks DBMS-centric security and consistent logging/auditing (important!)

Note: although named procedures, can also be functions


Stored Procedures

Call P P

Call P

In/out arguments

Application (client)

P

Intermediate results

DBMS (server)

Network connection

Network connection

table

table

Regular procedure

Stored procedure


SQL/PSM

Stored Procedures not only have full access to SQL All major database systems provide extensions of SQL to a

simple, general purpose language SQL:1999 Standard: SQL/PSM PostgreSQL: PL/pgSQL Oracle: PL/SQL (syntax differs!!!)

Extensions Local variables, loops, if-then-else conditions

Example: CREATE PROCEDURE ShowNumberOfEnrolments SELECT uosCode, COUNT(*) FROM Enrolled GROUP BY uosCode

Calling Stored Procedures: CALL statement Example: CALL ShowNumberOfEnrolments();


Procedure Declarations

Procedure Declarations (with SQL/SPM)CREATE PROCEDURE name ( parameter1,…, parameterN ) local variable declarations procedure code;

Stored Procedures can have parameters of a valid SQL type (parameter types must match) three different modes

IN arguments to procedure OUT return values INOUT combination of IN and OUT

CREATE PROCEDURE CountEnrolments( IN uos VARCHAR ) SELECT COUNT(*) FROM Enrolled WHERE uosCode = uos;

CALL CountEnrolments (‘INFO2120’);


PostgreSQL: PL/pgSQL

Extents SQL by programming language contructs Only knows functions! CREATE FUNCTION name RETURNS ... AS... Compound statements: BEGIN … END; SQL variables: DECLARE section

variable-name sql-type; Assignments: variable := expression; IF statement: IF condition THEN … ELSE … END IF; Loop statements: FOR var IN range (WHILE cond )

LOOP … END LOOP; Return values: RETURN expression; Call statement: CALL procedure(parameters); Transactions: COMMIT; ROLLBACK;

(cf. http://www.postgresql.org/docs/8.4/static/plpgsql.ht

ml)


Tip: CREATE OR REPLACE to avoid

‘name-already-used’ PL/pgSQL Example

PL/pgSQL procedure declarationCREATE OR REPLACE FUNCTION name ( parameter1, …, parameterN ) RETURNS sqlType

AS $$

DECLAREvariable sqlType;…

BEGIN

…

END;

$$ LANGUAGE plpgsql;

where parameterX is declared as (IN is default):[IN|OUT|IN OUT] name sqlType

optional

Tip: final delimiter must match the one

used after AS

(cf. http://www.postgresql.org/docs/8.4/static/plpgsql-structure.html)


PostgreSQL PL/pgSQL Example

CREATE OR REPLACE FUNCTION RateStudent (studId INTEGER, uos VARCHAR) RETURNS CHAR AS $$DECLARE

grade CHAR; marks INTEGER;

BEGIN SELECT SUM(marks) INTO marks

FROM Assessment WHERE sid=$1 AND uosCode=$2;IF ( marks>84 ) THEN grade := ‘HD’;ELSIF ( marks>74 ) THEN grade := ‘D’;ELSIF ( marks>64 ) THEN grade := ‘CR’;ELSIF ( marks>50 ) THEN grade := ‘P’;ELSE grade := ‘F’;END IF;RAISE NOTICE 'Final grade is: %s', grade;RETURN grade;

END;

$$ LANGUAGE plpgsql;


Calling Stored Procedures from Clients

Embedded SQLEXEC SQL BEGIN DECLARE SECTIONchar courseId(8);EXEC SQL END DECLARE SECTIONEXEC SQL CALL CountEnrolments(:courseId);

JDBC:CallableStatement cstmt = conn.prepareCall( “{call CountEnrolments(?)}”);

cstmt.setString(1,courseId);cstmt.executeUpdate();

SQLJ#sql Iterator studnum(int count)#sql studnum = {CALL CountEnrolments(:courseId)}while ( studnum.next() ) { … }


Calling Stored Procedures from PDO

Calling a Stored Procedure with parameters:(here: first IN, second an INOUT parameter) var $empname; var $empid = 42; $cstmt = $conn->prepare("CALL HighestPaidEmp(?,?)"); $cstmt->bindParam(1, $empid); $cstmt->bindParam(2, $empname, PDO::PARAM_STR|PDO::PARAM_INPUT_OUTPUT, 20); $cstmt->execute(); print $empname;

The syntax for calling stored Functions is as follows:$stproc_stmt = $conn->prepare("?=CALL

funcname(?,?,?)");

The first ? refers to the return value of the function and is also to be registered as an PDO::PARAM_INPUT_OUTPUT parameter.

Specify as INOUT parameter with bitwise-or of type and inout flag

Out strings require a max length


Calling Stored Procedures from JDBC

Calling a Stored Procedure with parameters:(here: first IN, second an OUT parameter)

CallableStatement cstmt = conn.prepareCall( “{call HighestPaidEmp(?,?)}”);

cstmt.setInt(1, empid);cstmt.registeroutParameter(2, Types.VARCHAR);cstmt.executeUpdate();String empname = cstmt.getString(2);

The syntax for calling stored Functions is as follows:CallableStatement stproc_stmt = conn.prepareCall

("{ ? = call _funcname(?,?,?)}");

The first ? refers to the return value of the function and is also to be registered as an OUT parameter.


Externally Defined Stored Procedures

Stored Procedures can also be defined using external code in a programming language

Example: SQL/PSMCREATE PROCEDURE RankStudents ( IN number INT )LANGUAGE JAVA EXTERNAL NAME ‘file:///c:/storedProcs/rank.jar’

Oracle PL/SQL Example:CREATE PROCEDURE RankStudents (number IN INT )IS LANGUAGE JAVANAME ‘file:///c:/storedProcs/rank.jar’


Stored Procedure Engine in Oracle

Pre-9i: Always interpreted execution Since 9i: also compiled native execution


Latest From Stored Procedures

Virtual machines now ‘integrated’ with DBMS E.g. Java with Oracle .Net CLR with IBM, Oracle, and SQL Server PostgreSQL: Supports several scripting languages such as perl etc.

MySQL: Working on Stored procedures in V5… alpha today

But degree of integration differs heavily Oracle DBMS and Java VM: Two different processes

Bad for performance because of context switches and data copying

Similar with .Net integration in DB2 SQL Server 2005 & 2008: CLR tightly integrated into DBMS

Should give better performance, but let’s see first…

PostgreSQL: C-code dynamically linked to code But potential security thread…


CLR Integration in SQL Server 2008

Problem: CLR and database are two different runtime environments Both provide memory / thread management and synchronization

Goals: Reliability, Scalability, Security, Performance

Also: UDTs, streaming functions, UDAs

SQL SERVER

SQL Server OS(memory, threads, synchronization)

CLR


DBMS Comparison

DBMS Internal Stored Procedures

External Stored Procedures

C Java .NET CLR

IBM DB2 SQL/PSM yes yes yes

Oracle PL/SQL yes yes yes

SQLServer T-Sql yes J# yes

Sybase T-Sql (yes) yes no

PostgreSQL PL/pgSQL; PL/Tcl; PL/Perl; PL/Python

yes no no

MySQL since version 5; SQL/PSM syntax

no no no


Lessons Learned

Same core issues for any db client-side development Data and type conversion: Host Variables NULL value semantic: Indicator variables and testing methods Impedance Mismatch: Cursor Concept Dynamic versus static SQL

Database APIs You should in particular be able to write small PHP or JDBC programs

DB Application Design Principles DAO Pattern; Error Handling; protection against SQL Injection

Server-side database programming How to use stored procedures to run code inside a DBMS

e.g. with PostgreSQL's pl/pgsql or with Oracle’s PL/SQL

Modern database engines provide virtual machine environments to run external code near the data


References

Kifer/Bernstein/Lewis (2nd edition) Chapter 8

Ramakrishnan/Gehrke (3rd edition - the ‘Cow’ book) Chapter 6

Ullman/Widom (3rd edition of ‘First Course in Database Systems’) Chapter 9 (covers Stored Procedures, ESQL, CLI, JDBC and PHP)

Research Papers and Presentations: Acheson, et al.: “Hosting the .NET Runtime in Microsoft SQL Server”.

SIGMOD 2004. E.M. Fayo: “Advanced SQL Injection in Oracle Databases”, Powerpoint

presentation, February 2005.Database Documentation: PHP PDO extensions: http://www.php.net/manual/en/book.pdo.php The PostgreSQL Global Development Group: “PostgreSQL 8.2.4 Documentation”, 2009. Oracle Corporation: “Oracle 10.1 Database Concepts”,2003. MySQL website: http://www.mysql.com

http://www.mysql.com/


Next Lecture (after Easter Break)

Transaction Management Transaction Concept Serializability SQL Commands to Control Transactions

Readings: Kifer/Bernstein/Lewis book, Chapter 18 or alternatively (if you prefer those books):

Ramakrishnan/Gehrke (Cow book), Chapter 16 Ullman/Widom, Chapter 6.6 onwards

Documents

Dr. Uwe Röhm School of Information Technologies INFO2120 – INFO2820 – COMP5138 Database Systems Week 8: Database Application Development (Kifer/Bernstein/Lewis