Introduction to PostgreSQL and JDBC

PostgreSQL - Introduction

● A free and open source object-relational database engine.

● Initially developed at UC Berkeley.

● ANSI-SQL:2008 compliance and support for a wide range of relational database features.

● Supports an array of operating systems,

● e.g: Linux, UNIX (AIX, BSD, HP-UX, SGI IRIX, Mac OS X, Solaris, Tru64), and Windows.

● Native programing support for main stream programming languages,e.g: C, C++, Java, C#, Python, Perl, etc.

PostgresSQL - Where to get help

● Documentation is comprehensive.

– http://www.postgresql.org/docs/9.2/interactive/ index.html● Mailing lists and IRC channel.

– http://www.postgresql.org/list/

– http://www.postgresql.org/community/irc/

Accessing the CS department PostgresSQL instance

● PostgreSQL version is 9.6.2● URL - faure.cs.colostate.edu Port - 5432● Username - CS login, Password - CSU ID.● Can be accessed using different tools

– PSQL– pgAdmin3

● Please note that certain administrative functionalities may not be granted.

pgAdmin3

● A graphical tool similar to MySQL Workbench, SQL workbench, etc.

● Supports database administration, data manipulation, query visulazation and editing, query debugging, etc.

● Should be installed separately.

● http://www.pgadmin.org/

pgAdmin3

pgAdmin3

PostgresSQL Commands

● Accessing the default database

– psql -h faure -p 5432 -U loginname -d loginname● Example

– psql -h faure -p 5432 -U jdoe -d jdoe● Accessing a different database

– psql -h faure -p 5432 -U jdoe -d mydb● Accessing a database from outside CS network

– psql -h faure.cs.colostate.edu -p 5432 -U jdoe -d mydb

PSQL

● PSQL - a command line interactive client.

● Allows to execute SQL commands as well as non-SQL commands.

● Non-SQL commands(a.k.a. meta commands) start with a ‘\’.

– \h - help for SQL commands

– \q - quit

– \? - administrative options● More Info :

http://www.postgresql.org/docs/current/interactive/app-psql.html

Executing SQL commands in PSQL

● Two main approaches

– Executing command directly in the psql console

– Storing commands in a file and use it as the input for psql console.

● Running commands from a file

● command(after logging into psql console)

– \i /path/to/file

– example● \i ~/srcipt.sql

Working with databases

● Show the list of databases(in psql)

– \list

● Creating a database

– Using psql console

● CREATE DATABASE [options] dbname

– Using createdb command line tool

● createdb -h faure -p 5432 -U pallapua mydb● Remove a database

– Using psql console

● DROP DATABASE dbname

– Using dropdb command line tool

● dropdb -h faure -p 5432 -U pallapua mydb

Working with databases

● Updating a database

– Updates the attributes of the database such as owner, name, security configurations, etc.

– Using ALTER DATABASE command

– Examples:● ALTER DATABASE name RENAME TO new name● ALTER DATABASE name OWNER TO new owner

Working with tables

● List the set of tables of the current database

– \dt● creating a table

– Using CREATE TABLE command

– Example :

– CREATE TABLE students(id varchar(10),

f name varchar(50),

l name varchar(50),

dob date,

gpa real);● Deleting a table

– Using DROP TABLE command

– Example :● DROP TABLE students;

Working with tables

● Creating a table with a primary key constraint with a single attribute

CREATE TABLE distributors (

did integer,

name varchar(40),

PRIMARY KEY(did)

);

Working with tables

● Creating a table with a primary key constraint with multiple attributes

CREATE TABLE films (

code char(5),

title varchar(40),

did integer,

date_prod date,

kind varchar(10),

len interval hour to minute,

CONSTRAINT code_title PRIMARY KEY(code,title)

);

Working with tables

● Creating a table with unique constraint

CREATE TABLE films (

code char(5) PRIMARY KEY,

title varchar(40),

did integer,

date_prod date,

kind varchar(10),

len interval hour to minute,

CONSTRAINT production UNIQUE(date_prod)

);

Working with tables

● Creating a table with a column constraint

CREATE TABLE distributors (

did integer CHECK (did > 100),

name varchar(40)

);

Working with tables

● Describing a table using \d+ table name

Working with tables

● Inserting rows into a table

– Using INSERT INTO command

– Examples :● INSERT INTO students values(‘123456789’, ‘John’,

‘Doe’, ‘1990-10-10’, 3.7);● INSERT INTO students(f name, l name, dob, gpa, id)

values(‘John’, ‘Doe’, ‘1990-10-10’, 3.7, ‘123456789’);● INSERT INTO students(id, f name, l name)

values(‘123456789’, ‘John’, ‘Doe’);

Working with tables

● Modifying an existing table

– Using ALTER TABLE command

– Examples :● ALTER TABLE students RENAME TO csu students;● ALTER TABLE students RENAME COLUMN f name TO first name;● ALTER TABLE students ADD COLUMN zipcode varchar(15);● ALTER TABLE students DROP COLUMN zipcode integer● ALTER TABLE students ALTER COLUMN zipcode TYPE integer● ALTER TABLE students ADD PRIMARY KEY (id);● ALTER TABLE students ADD CONSTRAINT id l name key

UNIQUE (id, l name)

EXPLAIN command

● Provides information about execution plans.

● Scan mode(plain scan, indexing), join algorithms, cost, buffer usage, etc.

● Cost is provided mainly in terms of number of disk page fetches.

● Adding the extra parameter ANALYZE will make the query executed for real.

● Parameter VERBOSE will provide detailed information

● Results can be obtained in different formats like text, json, yaml, xml, etc

EXPLAIN command

● A simple EXPLAIN command.

EXPLAIN SELECT * FROM foo;

QUERY PLAN---------------------------------------------------------

Seq Scan on foo (cost=0.00..155.00 rows=10000 width=4) (1 row)

EXPLAIN command

● A simple EXPLAIN command with results formatted as a JSON document.

EXPLAIN (FORMAT JSON) SELECT * FROM foo;

QUERY PLAN----------------------------------------[ + + "Plan": + "Node Type": "Seq Scan", + "Relation Name": "foo", + "Alias": "foo", + "Startup Cost": 0.00, + "Total Cost": 155.00, + "Plan Rows": 10000, + "Plan Width": 4 + + +](1 row)

EXPLAIN command

● A simple EXPLAIN command.

EXPLAIN SELECT * FROM tenk1;

QUERY PLAN-------------------------------------------------------------

Seq Scan on tenk1 (cost=0.00..458.00 rows=10000 width=244)

EXPLAIN command

● A simple EXPLAIN command with a filter condition.

EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 7000;

QUERY PLAN------------------------------------------------------------

Seq Scan on tenk1 (cost=0.00..483.00 rows=7033 width=244) Filter: (unique1 < 7000)

EXPLAIN command

● A simple EXPLAIN command with a filter condition with low selectivity.

EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 100;

QUERY PLAN---------------------------------------------------------------

Bitmap Heap Scan on tenk1 (cost=2.37..232.35 rows=106 width=244)

Recheck Cond: (unique1 < 100)

-> Bitmap Index Scan on tenk1_unique1 (cost=0.00..2.37 rows=106 width=0)

Index Cond: (unique1 < 100)

a

● A simple EXPLAIN command with a filter condition with low selectivity.

● EXPLAIN SELECT * FROM tenk1 WHERE unique1 < 3;

QUERY PLAN----------------------------------------------------------------

Index Scan using tenk1_unique1 on tenk1 (cost=0.00..10.00 rows=2 width=244)

Index Cond: (unique1 < 3)

Database Application

Development with JDBC

Embedded SQL

● Embedding SQL commands directly inside a host programming language.

● An example code for declaring the variables to be used in the query.

EXEC SQL BEGIN DECLARE SECTION char c_sname[20];

long c_sid;

short c_rating;

float c_age;

EXEC SQL END DECLARE SECTION● An example code for executing an insert statement

EXEC SQL

INSERT INTO Sailors VALUES(:c_name, :c_id,

:c_rating, :c_age);

Embedded SQL

● Preprocessed by a DB engine specific preprocessor before compiling.

● Type conversions are handled by the preprocessor.

● Generated executables are coupled with the DB engine.

● Better performance, detecting errors at the compile time.

JDBC/ODBC

● Integrates SQL with general purpose programming languages.

● Provides an API(application programming interface) for developers to access databases.

● Provides DBMS independent programming model at source level as well as at executable level.

● Corresponding database engine driver is loaded and linked at the runtime.

● Our focus is on JDBC

Main components of the JDBC architecture

● Application

● Driver Manager

● Drivers

● Data Source

Different types of JDBC drivers

● Type I - Bridges

Translates JDBC function calls into function calls of another

API, e.g.:- JDBC-ODBC bridge● Type II - Native API driver

Uses a client side API provided by the DB engine, e.g.:-

Oracle OCI driver● Type III - MiddleWare Driver

Communicates with a middle-ware server that translates

JDBC requests into DBMS specific method invocations● Type IV - Database Protocol Driver

A database driver implementation that converts JDBC calls directly into a vendor-specific database protocol.

JDBC Driver Management

● Extremely simplified JDK 1.4 onwards

● The only requirement is to include the driver in the classpath.

● Driver libraries can be downloaded from corresponding vendors.

e.g.:- For PostgreSQL - http://jdbc.postgresql.org● Appropriate driver is selected based on the connection URL

jdbc:<subprotocol>:<otherparameters>

e.g.:-

jdbc:postgresql://faure.cs.colostate.edu:5432/pallapua

JDBC - Creating a connection

PostgreSQL Connection Properties

● A list of String key-value pairs that define the behavior of a connection.

– user - The database user on whose behalf the connection is being made.

– password - Password of the user

– ssl - whether to use transport level security or not. Requires setting up a trust store.

– logUnclosedConnections - Prints a log statement for each unclosed connection at the end. Useful for developers to detect connection leaks.

● Refer https://jdbc.postgresql.org/documentation/92/connect.html for the complete list.

More on JDBC Connections

● Each server can support a limited number of concurrent connections at a given time.

● Each connection needs to be closed after being used.

conn.close();● Establishing a connection is expensive. So connections should be

reused.

● Default JDBC implementation does not support connection pooling.

● In real world applications, a connection pooling library such as

Apache DBCP should be used.

(http://commons.apache.org/proper/commons-dbcp/)

Executing SQL Statements

● There are three different ways of creating SQL statements;

● Statements, PreparedStatements and CallableStatements.

● With Statements, entire SQL command is passed as a parameter.

Statement stmt = conn.createStatement();

ResultSet resultSet = stmt.executeQuery("SELECT * " + "FROM students WHERE gpa > 3.5");

Executing SQL Statements

● PreparedStatements allow parameterized statements.

PreparedStatement prepStmt = conn.prepareStatement(

"SELECT * " + "FROM students WHERE gpa > ?");

prepStmt.setInt(1, 2);

ResultSet resultSet = prepStmt.executeQuery();● Parameters are sanitized. More resistant against SQL injection

attacks.

● Preferred over Statement type.

● CallableStatements are used with Stored Procedures.

ResultSet

● Provides the set-of-abstract abstraction for regular programming languages.

● Inspired by Cursors in embedded SQL.

● Can iterate through the resulting records from the query.

ResultSet resultSet = prepStmt.executeQuery();

while(resultSet.next())

{

System.out.println(resultSet.getString(1) + ","

+ resultSet.getString(2) +","

+ resultSet.getString(3) + ","

+ resultSet.getFloat(5));

}

ResultSet types

● TYPE FORWARD ONLY - Not scrollable. The set of rows contained will depend on the DB engine implementation. This is the default behavior.

● TYPE SCROLL INSENSITIVE - Scrollable. Not sensitive to concurrent updates taking place at the DB.

● TYPE SCROLL SENSITIVE - Scrollable. Sensitive to concurrent Updates.

Store Procedures

● Allows certain portions of the application logic to be executed in the address space of the database system.

● Reduces the communication cost between DB system and the client application.

● Stored procedures can be shared among multiple clients.

● It’s not necessary to expose the DB Schema to the outside.

● DB system resources can become a bottleneck.

● Analogous to function invocations with input parameters and returned values.

An Example Store Procedure

CREATE PROCEDURE AddInventory (

IN book_isbn CHAR(10),

IN addedQty INTEGER)

UPDATE Books

SET qty_in_stock = qty_in_stock + addedQty

WHERE book_isbn = isbn

Invoking Stored Procedures with JDBC

CallableStatement callableStmt =

conn.prepareCall("AddInventory");

ResultSet rs = callableStmt.executeQuery();