Upload
lmjoshi
View
224
Download
0
Embed Size (px)
Citation preview
8/8/2019 Mysql Tips
1/48
Interesting Things to Know about MySQLMike Chirico ([email protected])Copyright (c) 2004 (GNU Free Documentation License)Last Updated: Sun Jan 21 09:44:21 EST 2007
The latest version of this document can be found at:http://souptonuts.sourceforge.net/readme_mysql.htm
TIP 1:
Find out who is doing what, and kill the process if needed.This example kills Id 657.
mysql> show processlist;show processlist;+-----+------+-----------+---------+---------+-------+-------+--------------
----+ Id User Host db Command Time State Info
+-----+------+-----------+---------+---------+-------+-------+------------------+
657 prog localhost weather Sleep 28619 NULL 782 prog localhost weather Sleep 853 NULL 785 prog localhost NULL Query 0 NULL show processl
ist +-----+------+-----------+---------+---------+-------+-------+--------------
----+3 rows in set (0.00 sec)
mysql>kill 657
Or, from the command line, to kill process 782
[root@third-fl-71 mysql]# mysqladmin processlist+-----+------+-----------+---------+---------+------+-------+---------------
---+ Id User Host db Command Time State Info+-----+------+-----------+---------+---------+------+-------+---------------
---+ 782 prog localhost weather Sleep 2676
785 prog localhost Sleep 1823 793 root localhost Query 0 show processli
st +-----+------+-----------+---------+---------+------+-------+---------------
---+[root@third-fl-71 mysql]#
[root@third-fl-71 mysql]# mysqladmin kill 782
Note, the following can also be helpful
mysql> show status;or
mysql> show status\G
8/8/2019 Mysql Tips
2/48
alsomysql> show innodb status;
ormysql> show table status like '%';
The above gives you create time and other information.
TIP 2:
Clean up binary log files. For a default install they may be in
/usr/local/var/or
/var/lib/mysql/
with names ending in -bin.000001,-bin.000002,.. The followingcommand may help find out where the logs are located.
mysql> show variables like '%home%';+---------------------------+-----------------+ Variable_name Value +---------------------------+-----------------+ bdb_home /var/lib/mysql/ innodb_data_home_dir innodb_log_group_home_dir ./ +---------------------------+-----------------+
mysql> reset master;reset master;Query OK, 0 rows affected (0.02 sec)
See (Tip 24:) details working with binary log files and (Tip 25:) explainshow to setup logging. (Tip 37:) shows have to setup MASTER and SLAVEreplication.
TIP 3:
Can the order of the columns in a create statement make a difference? YES
create table t (a int,b int,timeUpdate timestamp,timeEnter timestamp );
The first timestamp will always be the "automatically generated" time. Soif the record is updated, or inserted, this time gets changed. If theorder is changed, "timeEnter" is before "timeUpdate", then, "timeEnter"would get updated. First timestamp column updates automatically.
Note, in the table above timeEnter will only get updated if passed a nullvalue.
8/8/2019 Mysql Tips
3/48
insert into t (a,b,timeEnter) values (1,2,NULL);
Hints: Need mm-dd-yyyy hh:mm:ss format?
select a,b,DATE_FORMAT(timeUpdate,'%m-%d-%Y %T'),DATE_FORMAT(timeEnter,'%m-%d-%Y %T') from t;
+------+------+---------------------------------------+-------------------
-------------------+ a b DATE_FORMAT(timeUpdate,'%m-%d-%Y %T') DATE_FORMAT(timeEn
ter,'%m-%d-%Y %T') +------+------+---------------------------------------+-------------------
-------------------+ 3 2 04-15-2004 19:14:36 04-15-2004 19:15:0
7 3 2 04-15-2004 19:14:39 04-15-2004 19:15:0
7 5 5 00-00-0000 00:00:00 04-15-2004 19:15:5
3 1 2 00-00-0000 00:00:00 04-15-2004 19:20:1
5 +------+------+---------------------------------------+--------------------------------------+
4 rows in set (0.00 sec)
TIP 4:
Connect, create table and select with Perl (Linux). First the DBI module isneeded, which
can be installed from the system prompt as follows:
# perl -MCPAN -e shellcpan> install DBIcpan> install DBD::mysql
The following is an example program:
#! /usr/bin/perl -w# Copyright (GPL) Mike Chirico [email protected]## Program does the following:# o connects to mysql# o creates perlTest if it doesn't exist# o inserts records# o selects and displays records## This program assumes DBI## perl -MCPAN -e shell# cpan> install DBI# cpan> install DBD::mysql####
#
use strict;
8/8/2019 Mysql Tips
4/48
use DBI;
# You will need to change the following:# o database# o user# o passwordmy $database="yourdatabase";
my $user="user1";my $passwd="hidden";my $count = 0;my $tblcreate= "CREATE TABLE IF NOT EXISTS perlTest (pkey int(11) NOT NULL auto_increment,a int,b int,c int,timeEnter timestamp(14),PRIMARY KEY (pkey)
) ";
my $insert= "insert into perlTest (a,b,c)values (1,2,3),(4,5,6),(7,8,9)";
my $select="select a,b,c from perlTest ";
my $dsn = "DBI:mysql:host=localhost;database=${database}";my $dbh = DBI->connect ($dsn, $user, $passwd)
or die "Cannot connect to server\n";
my $s = $dbh->prepare($tblcreate);$s->execute();$s = $dbh->prepare($insert);$s->execute();
$s = $dbh->prepare($select);$s->execute();
while(my @val = $s->fetchrow_array()){
print " $val[0] $val[1] $val[2]\n";++$count;
}$s->finish();
$dbh->disconnect ( );
exit (0);
8/8/2019 Mysql Tips
5/48
TIP 5:
Remove duplicate entries. Assume the following table and data.
CREATE TABLE IF NOT EXISTS dupTest (pkey int(11) NOT NULL auto_increment,a int,
b int,c int,timeEnter timestamp(14),PRIMARY KEY (pkey)
);
insert into dupTest (a,b,c) values (1,2,3),(1,2,3),(1,5,4),(1,6,4);
mysql> select * from dupTest;select * from dupTest;+------+------+------+------+---------------------+ pkey a b c timeEnter +------+------+------+------+---------------------+ 1 1 2 3 2004-04-16 10:55:35 2 1 2 3 2004-04-16 10:55:35 3 1 5 4 2004-04-16 10:55:35 4 1 6 4 2004-04-16 10:55:35 +------+------+------+------+---------------------+4 rows in set (0.00 sec)
mysql>
Note, the first two rows contains duplicates in columns a and b. It contains
other duplicates; but, leaves the other duplicates alone.
mysql> ALTER IGNORE TABLE dupTest ADD UNIQUE INDEX(a,b);
mysql> select * from dupTest;select * from dupTest;+------+------+------+------+---------------------+ pkey a b c timeEnter +------+------+------+------+---------------------+ 1 1 2 3 2004-04-16 11:11:42 3 1 5 4 2004-04-16 11:11:42 4 1 6 4 2004-04-16 11:11:42 +------+------+------+------+---------------------+3 rows in set (0.00 sec)
TIP 6:
Show status information on a table. Note, if the database was startedwith --safe-show-database or --skip-show-database some of these commandsmay not work. Note the "\G" option may provide a nicer format.
Show the create statement:
8/8/2019 Mysql Tips
6/48
8/8/2019 Mysql Tips
7/48
pkey int(11) NOT NULL auto_increment,a int,b varchar(12),timeEnter timestamp(14),PRIMARY KEY (pkey)
) type=MyISAM;
CREATE TABLE log_summary (pkey int(11) NOT NULL auto_increment,a int,b varchar(12),timeEnter timestamp(14),PRIMARY KEY (pkey)
) type=MERGE UNION(log_01,log_02) INSERT_METHOD=LAST;
mysql> insert into log_01 (a,b) values (1,'log1');mysql> insert into log_02 (a,b) values (1,'log2');
mysql> select * from log_summary;
select * from log_summary;+------+------+------+---------------------+ pkey a b timeEnter +------+------+------+---------------------+ 1 1 log1 2004-04-16 11:59:55 1 1 log2 2004-04-16 12:00:08 +------+------+------+---------------------+2 rows in set (0.00 sec)
Reference:http://dev.mysql.com/doc/mysql/en/MERGE.html
TIP 9:
Updating foreign keys in a multiuser environment. Using LAST_INSERT_ID().Also see (TIP 29) after reading this.
The LAST_INSERT_ID() is unique to the login session. This allows updatingof foreign keys.
CREATE TABLE keytest (pkey int(11) NOT NULL auto_increment,ptext text,ptype int(11),timeEnter timestamp(14),PRIMARY KEY (pkey));
CREATE TABLE foreignkeytest (pkey int(11) NOT NULL auto_increment,pkeykeytest int(11) NOT NULL,ptext text,ptype int(11),timeEnter timestamp(14),PRIMARY KEY (pkey));
mysql> insert into keytest(ptext,ptype) values ('one',1);
8/8/2019 Mysql Tips
8/48
mysql> select LAST_INSERT_ID() from keytest;+------------------+ last_insert_id() +------------------+ 1 +------------------+1 row in set (0.03 sec)
mysql> insert into foreignkeytest (ptext,pkeykeytest) values ('one',LAST_INSERT_ID());
Note: If your session didn't update any records, LAST_INSERT_ID() will bezero. Never
assume LAST_INSERT_ID()+1 will be the next record. If another session inserts a record,
this value may be taken. You are assured that this value will be unique to the "session".
**SPECIAL NOTE: MySQL 4.1.2. supports UUID.
mysql> select uuid();+--------------------------------------+ uuid() +--------------------------------------+ 167c1afe-0a0f-1027-891e-0004e222b485 +--------------------------------------+1 row in set (0.00 sec)
"A UUID is designed as a number that is globally unique in space and time.
Two calls to UUID() are expected to generate two different values, even if
these calls are performed on two separate computers that are notconnected to each other."
So in the future, UUID() could be used as a better primary key. The advantage
being tables from a one server could be up-loaded to a second server without
worrying about duplicate keys.
TIP 10:
Monitor port 3306:
tcpdump -i eth0 -nN -vvv -xX -s 1500 port 3306 and host not 192.168.1.102
The -s is length of each packet. This monitors all traffic on port 3306 excluding
the good client 192.168.1.102. The out will be in the following format:
[root@third-fl-71 root]# tcpdump -i eth0 -nN -vvv -xX -s 1500 port 3306 and host not 192.168.1.102
tcpdump: listening on eth013:05:01.857705 192.168.1.103.4131 > 192.168.1.71.mysql: S [tcp sum ok] 2542149285:2542149285(0) win 16384
8/8/2019 Mysql Tips
9/48
0x0000 4500 0030 0b39 4000 8006 6b90 c0a8 0167 [email protected] c0a8 0147 1023 0cea 9786 1ea5 0000 0000 ...G.#..........0x0020 7002 4000 ebe7 0000 0204 05b4 0101 0402 [email protected]:05:04.863552 192.168.1.103.4131 > 192.168.1.71.mysql: S [tcp sum ok] 2542149285:2542149285(0) win 16384
8/8/2019 Mysql Tips
10/48
// PRIMARY KEY (pkey)//// );//// insert into exams (name,exam,score) values ('Bob',1,75);// insert into exams (name,exam,score) values ('Bob',2,77);// insert into exams (name,exam,score) values ('Bob',3,78);
// insert into exams (name,exam,score) values ('Bob',4,80);//// insert into exams (name,exam,score) values ('Sue',1,90);// insert into exams (name,exam,score) values ('Sue',2,97);// insert into exams (name,exam,score) values ('Sue',3,98);// insert into exams (name,exam,score) values ('Sue',4,99);//
import java.sql.*;
public class Select{
public static void main (String[ ] args){Connection conn = null;String url = "jdbc:mysql://localhost/";String userName = "username1";String password = "password1";
try{Class.forName ("com.mysql.jdbc.Driver").newInstance ( );conn = DriverManager.getConnection (url, userName, password);// System.out.println ("Connected");
Statement s = conn.createStatement ( );s.executeQuery ("SELECT pkey,name,exam,score FROM test.exams");ResultSet rs = s.getResultSet ( );int count = 0;
while (rs.next ( )) // loop through rows of result set{int pkey = rs.getInt (1);String name = rs.getString(2);int exam = rs.getInt(3);int score = rs.getInt(4);++count;System.out.println (count + ",inum: " + pkey + ",name: " +name + ",exam: " + exam + ",score: " + score );
}rs.close ( ); // close result sets.close ( ); // close statementSystem.out.println (count + " rows were returned");
}catch (Exception e){System.err.println ("Cannot connect to server"+e);}
finally
{if (conn != null){
8/8/2019 Mysql Tips
11/48
try{conn.close ( );
// System.out.println ("Disconnected"); /* for debugging */}
catch (Exception e) { /* ignore close errors */ }}
}}
}
TIP 13:
Print defaults for the current client connection
$ my_print_defaults client mysql--port=3306--socket=/tmp/mysql.sock--no-auto-rehash
Note, for client connections, you can put setting inthe user's .my.cnf file. This file is located in theirhome directory.
~/.my.cnf
Example file that stores the user and password plus
ssl settings that will connect to the remotehosts big.squeezel.com.
[client]user=adminpassword=pass32rDhost=big.squeezel.comssl-ca=/etc/mysql/cacert.pemssl-cert=/etc/mysql/laptopFOO-cert.pemssl-key=/etc/mysql/laptopFOO-key.pem
Reference tip 38 on setting up ssl connections.
TIP 14:
Quick Status:
mysql> \s--------------/usr/local/bin/mysql Ver 14.3 Distrib 4.1.1-alpha, for pc-linux (i686)
Connection id: 642
Current database:Current user: prog@localhostSSL: Not in use
8/8/2019 Mysql Tips
12/48
Current pager: stdoutUsing outfile: ''Using delimiter: ;Server version: 4.1.1-alpha-logProtocol version: 10Connection: Localhost via UNIX socketClient characterset: latin1_swedish_ci
Server characterset: latin1_swedish_ciUNIX socket: /tmp/mysql.sockUptime: 1 day 15 hours 24 min 38 sec
Threads: 1 Questions: 4896 Slow queries: 0Opens: 14 Flush tables: 1 Open tables: 8 Queries per second avg: 0.035--------------
TIP 15:
"error: 1016: Can't open file:" If it's from an orphaned file, not in the databasebut on disk, then, the disk file may need to be deleted.
myisamchk can help with damaged files. It's best to stop the database.
# su -# mysqladmin shutdown
# cd /usr/local/var/database# myisamchk *
# /etc/init.d/mysql restart
TIP 16:
Finding records that do not match between two tables.
CREATE TABLE bookreport (b_id int(11) NOT NULL auto_increment,s_id int(11) NOT NULL,report varchar(50),PRIMARY KEY (b_id)
);
CREATE TABLE student (s_id int(11) NOT NULL auto_increment,name varchar(15),PRIMARY KEY (s_id)
);
insert into student (name) values ('bob');insert into bookreport (s_id,report)values ( last_insert_id(),'A Death in the Family');
insert into student (name) values ('sue');insert into bookreport (s_id,report)
8/8/2019 Mysql Tips
13/48
values ( last_insert_id(),'Go Tell It On the Mountain');
insert into student (name) values ('doug');insert into bookreport (s_id,report)values ( last_insert_id(),'The Red Badge of Courage');
insert into student (name) values ('tom');
To find the sudents where are missing reports:
select s.name from student sleft outer join bookreport b on s.s_id = b.s_id
where b.s_id is null;
+------+ name +------+ tom
+------+1 row in set (0.00 sec)
Ok, next suppose there is an orphan record inin bookreport. First delete a matching recordin student:
delete from student where s_id in (select max(s_id) from bookreport);
Now, how to find which one is orphaned:
select * from bookreport b left outer join
student s on b.s_id=s.s_id where s.s_id is null;
+------+------+--------------------------+------+------+ b_id s_id report s_id name +------+------+--------------------------+------+------+ 4 4 The Red Badge of Courage NULL NULL +------+------+--------------------------+------+------+1 row in set (0.00 sec)
To clean things up (Note in 4.1 you can't do subquery onsame table in a delete so it has to be done in 2 steps):
select @t_sid:=b.s_id from bookreport b left outer joinstudent s on b.s_id=s.s_id where s.s_id is null;
delete from student where s_id=@t_sid;
But, functions do work in delete. For instance thefollowing is possible:
delete from student where s_id=max(s_id);
It just a problem when joining the table where the
delete will occur with another table. Anotheroption is two create a second temp table andlocking the first one.
8/8/2019 Mysql Tips
14/48
TIP 17:
Getting a random roll of the dice:
CREATE TABLE dice (d_id int(11) NOT NULL auto_increment,roll int,PRIMARY KEY (d_id)
);
insert into dice (roll) values (1);insert into dice (roll) values (2);insert into dice (roll) values (3);insert into dice (roll) values (4);insert into dice (roll) values (5);insert into dice (roll) values (6);
select roll from dice order by rand() limit 1;
TIP 18:
Creating and using your own password file.This assumes MySQL has been configured with SSL support [ ./configure --wit
h-openssl ]
CREATE TABLE password (
p_id int(11) NOT NULL auto_increment,id varchar(20),valid enum('y','n'),password BLOB,timeEnter timestamp,PRIMARY KEY (p_id)
);
insert into password (id,password)values ('bob',des_encrypt('secret','somekey'));
insert into password (id,password)values ('tom',des_encrypt('password','somekey'));
insert into password (id,password)values ('kate',des_encrypt('desmark','somekey'));
insert into password (id,password)values ('tim',des_encrypt('tim','somekey'));
insert into password (id,password)values ('sue',des_encrypt('SUM','somekey'));
insert into password (id,password)values ('john',des_encrypt('dotgo86','somekey'));
insert into password (id)values ('randal');
mysql> select id,des_decrypt(password,'somekey') from password;
+--------+---------------------------------+
8/8/2019 Mysql Tips
15/48
id des_decrypt(password,'somekey') +--------+---------------------------------+ bob secret tom password kate desmark tim tim sue SUM
john dotgo86 randal NULL +--------+---------------------------------+7 rows in set (0.00 sec)
Note the bad passwords in the file ('secret','password', andpassword is the same as the id.
The following update statement will fill in thevalue for valid:
update password set valid =COALESCE(concat(substring('y',1,1-sign(abs(find_in_set( lcase( des_decrypt(password,'somekey')),concat('password,secret,simple,',lcase(id),',test'))))
),
substring('n',1,sign(abs(find_in_set( lcase( des_decrypt(password,'somekey')),concat('password,secret,simple,',lcase(id),',test'))))
)),'n');
Which gives the following:
mysql> select id,valid from password;select id,valid from password;+--------+-------+ id valid +--------+-------+ bob n tom n kate y tim n sue y john y randal n +--------+-------+7 rows in set (0.00 sec)
To understand it, try taking the following select apart:
selectCOALESCE(concat(
8/8/2019 Mysql Tips
16/48
substring('y',1,1-sign(abs(find_in_set( lcase( des_decrypt(password,'somekey')),concat('password,secret,simple,',lcase(id),',test'))))
),
substring('n',1,
sign(abs(find_in_set( lcase( des_decrypt(password,'somekey')),concat('password,secret,simple,',lcase(id),',test'))))
)),'n')as validfrom password;
TIP 19:
Order in the count:
create table a (a varchar(10));
insert into a values ('a'),('a'),('b'),('c');
Note
select a,count(a) as count from a group by a order by count;
+------+-------+ a count
+------+-------+ b 1 c 1 a 2 +------+-------+3 rows in set (0.00 sec)
butSee above name the function ------v
select a,count(a) from a group by a order by count(a);ERROR 1111 (HY000): Invalid use of group function
TIP 20:
Installation: configured for encryption, C API, and user defined functions.
./configure --with-openssl --enable-thread-safe-client --with-mysqld-ldflags=-rdynamic
The --with-openssl is very helpful for creating your ownpassword file. Also, if doing C API, having thread safecalls "could" come in handly...it's what I use.
See (TIP 27) for user defined functions.
8/8/2019 Mysql Tips
17/48
Complete Steps:
shell> groupadd mysqlshell> useradd -g mysql mysqlshell> cd mysql-VERSIONshell> ./configure --with-openssl --enable-thread-safe-client --with-mysqld
-ldflags=-rdynamic
shell> makeshell> make installshell> cp support-files/my-medium.cnf /etc/my.cnfshell> cd /usr/local/mysqlshell> bin/mysql_install_db --user=mysqlshell> chown -R root .shell> chown -R mysql varshell> chgrp -R mysql .shell> bin/mysqld_safe --user=mysql &
See (TIP 25) for configuring the log-bin and log files in /etc/my.cnf
Installing mysql so that it will startup automatically.This also enables it to be restarted, "/etc/init/mysql restart".
cp ./support-files/mysql.server /etc/init.d/mysqlcd /etc/rc3.dln -s ../init.d/mysql S85mysqlln -s ../init.d/mysql K85mysqlcd /etc/rc5.dln -s ../init.d/mysql S85mysqlln -s ../init.d/mysql K85mysqlcd ../init.dchmod 755 mysql
Deleting any blank users or passwords, and creating a validuser is shown below.
STEP 1:
First, connect to the mysql databasewith the connect command:
mysql> connect mysql;
STEP 2:
Next, delete all blank accounts and/orglobal host listing:
mysql> DELETE FROM user WHERE User = '';mysql> DELETE FROM db WHERE Host = '%';
STEP 3:
Delete any accounts with blank passwords:
mysql> DELETE FROM user where password='';
STEP 4:
Create a valid admin account. The example
8/8/2019 Mysql Tips
18/48
here creates admin1.
mysql> GRANT ALL PRIVILEGES ON *.* TO admin1@localhostIDENTIFIED BY 's3cr3tpass45' WITH GRANT OPTION;
or if you want the account to get access fromany host.
mysql> GRANT ALL PRIVILEGES ON *.* TO admin1@"%"IDENTIFIED BY 's3cr3tpass45' WITH GRANT OPTION;
STEP 5:
Restart the server "/etc/init.d/mysql restart"
TIP 21:
Getting 1 matching row from an OUTER join
CREATE TABLE parent (pkey int(11) NOT NULL auto_increment,pname varchar(5),timeEnter timestamp,PRIMARY KEY (pkey)
);
CREATE TABLE child (ckey int(11) NOT NULL auto_increment,pkey int(11) NOT NULL,
cname varchar(5),timeEnter timestamp,PRIMARY KEY (ckey)
);
insert into parent(pname) values ('A');select @a_lid:=last_insert_id();insert into child(pkey,cname) values (@a_lid,'a1');insert into child(pkey,cname) values (@a_lid,'a2');insert into child(pkey,cname) values (@a_lid,'a3');insert into parent(pname) values ('B');select @a_lid:=last_insert_id();insert into child(pkey,cname) values (@a_lid,'b1');insert into child(pkey,cname) values (@a_lid,'b2');insert into parent(pname) values ('C');
mysql> select p.*,c.cname,count(c.pkey) as numberfrom parent p left outer join child c on p.pkey=c.pkey
where c.pkey is not null group by c.pkey;
+------+-------+---------------------+-------+--------+ pkey pname timeEnter cname number +------+-------+---------------------+-------+--------+
5 A 2004-04-28 09:56:59 a1 3 6 B 2004-04-28 09:56:59 b1 2 +------+-------+---------------------+-------+--------+
8/8/2019 Mysql Tips
19/48
2 rows in set (0.01 sec)
For comparison, here is a listing of all the matching data:
mysql> select * from parent p left outer join child c on p.pkey=c.pkey where c.pkey is not null;
select * from parent p left outer join child c on p.pkey=c.pkey where c.pk
ey is not null;+------+-------+---------------------+------+------+-------+---------------
------+ pkey pname timeEnter ckey pkey cname timeEnter+------+-------+---------------------+------+------+-------+---------------
------+ 5 A 2004-04-28 09:56:59 7 5 a1 2004-04-28 09:
56:59 5 A 2004-04-28 09:56:59 8 5 a2 2004-04-28 09:
56:59 5 A 2004-04-28 09:56:59 9 5 a3 2004-04-28 09:
56:59 6 B 2004-04-28 09:56:59 10 6 b1 2004-04-28 09:56:59
6 B 2004-04-28 09:56:59 11 6 b2 2004-04-28 09:56:59
+------+-------+---------------------+------+------+-------+---------------------+
5 rows in set (0.00 sec)
TIP 22:
Getting a virtual row count.
Assume the following table:
CREATE TABLE student (s_id int(11) NOT NULL auto_increment,name varchar(15),PRIMARY KEY (s_id)
);
Add some rows and delete (just to convice you this works in all cases)
insert into student (name) values ('tom');insert into student (name) values ('bob');insert into student (name) values ('sue');insert into student (name) values ('mike');insert into student (name) values ('joe');insert into student (name) values ('zoe');insert into student (name) values ('harpo');
delete from student where name = 'bob';
Now, note mc is the row count...independent of s_id;
select a.name,sum(1) as mcfrom student a, student bwhere a.s_id
8/8/2019 Mysql Tips
20/48
group by a.s_id, a.name order by mc;+-------+------+ name mc +-------+------+ harpo 1 zoe 2 joe 3
mike 4 sue 5 tom 6 +-------+------+6 rows in set (0.00 sec)
mysql>
*Note: SEE ( TIP 27 ), using the sequence function.
TIP 23:
Computing running and sliding aggregates. Assume the following tableand data:
CREATE TABLE dice (d_id int(11) NOT NULL auto_increment,roll int,PRIMARY KEY (d_id)
);
insert into dice (roll) values (1);insert into dice (roll) values (2);
insert into dice (roll) values (3);insert into dice (roll) values (4);insert into dice (roll) values (5);insert into dice (roll) values (6);
The following gives a running sum:
mysql> select a.roll, sum(b.roll) from dice a, dice bwhere b.d_id
8/8/2019 Mysql Tips
21/48
where b.d_id show binlog events;show binlog events;+------------------------+-----+------------+-----------+--------------+-----
---------------------------------------+ Log_name Pos Event_type Server_id Orig_log_pos Info
+------------------------+-----+------------+-----------+--------------+-----
---------------------------------------+ third-fl-71-bin.000001 4 Start 1 4 Serv
er ver: 4.1.1-alpha-log, Binlog ver: 3 third-fl-71-bin.000001 79 Query 1 79 use
`test`; create table j2 (a int) third-fl-71-bin.000001 137 Query 1 137 use
`test`; insert into j2 (a) values (3) third-fl-71-bin.000001 201 Query 1 201 use
`test`; insert into j2 (a) values (9) third-fl-71-bin.000001 265 Query 1 265 use
`test`; delete from j2 where a=3 +------------------------+-----+------------+-----------+--------------+-----
---------------------------------------+5 rows in set (0.00 sec)
Note that the logfiles by default are located in "/usr/local/var" (see TIP 25:) or execute "mysql> show variables" and look
for the entry under the datadir. It's possible to query specific logfiles ifthey exist:
mysql> show binlog events in 'third-fl-71-bin.000001';
It's also possible to specify the start position and limit the number of reco
rds as follows:
mysql> show binlog events from 201 limit 2;
8/8/2019 Mysql Tips
22/48
show binlog events from 201 limit 2;+------------------------+-----+------------+-----------+--------------+----
---------------------------------------+ Log_name Pos Event_type Server_id Orig_log_pos Inf
o +------------------------+-----+------------+-----------+--------------+----
---------------------------------------+
third-fl-71-bin.000001 201 Query 1 201 use`test`; insert into j2 (a) values (9)
third-fl-71-bin.000001 265 Query 1 265 use`test`; delete from j2 where a=3
+------------------------+-----+------------+-----------+--------------+-------------------------------------------+
2 rows in set (0.00 sec)
Or, from the command line:
$ mysqlbinlog
Another option is writing a C api. For a complete example of this (selectbinlog.c) download the following:
http://prdownloads.sourceforge.net/souptonuts/mysql_select-0.0.1.tar.gz?download
or take a peek at
http://souptonuts.sourceforge.net/code/selectbinlog.c.html
This sample "selectbinlog.c" selects only the "Query" events. The code is pretty simple.
FYI: If you do a lot of tracking, you may want to write the information to aBerkeley DB.
Contrary to the name Berkeley DB is not a database but a hash, or there is anoption forb-tree format. MySQL can use Berkeley DB for the underlying table structure.
It's very fast,and you won't get logs of your logs. If you're using Linux, Berkeley DB is a
lready installedon your system. Ok, so how does one use Berkeley DB? Samples can be found at
the followinglink. Look for berkeleydb_0.x.x.tar.gz at the following link
http://sourceforge.net/project/showfiles.php?group_id=79066&package_id=80379
The newer version of Berkeley DB also supports encryption. Berkeley DB is open source
and can be downloaded from the following:
http://www.sleepycat.com/
NOTE: MySQL uses Berkeley DB if configured for it
./configure --with-berkeley-db
And tables are created with the "ENGINE" or "TYPE"option:
8/8/2019 Mysql Tips
23/48
CREATE TABLE b (i INT) ENGINE = BDB;or
CREATE TABLE b (i INT) TYPE = BDB;
TIP 25:
Setting the binary log file. This can be configured in /etc/my.cnf
# The MySQL server[mysqld]port = 3306socket = /tmp/mysql.socklog-bin = /usr/local/var/mysqlLOGb.loglog = /usr/local/var/mysqlLOG.log
or for the default hostname just log-bin
[mysqld]port = 3306socket = /tmp/mysql.socklog-bin
IMPORTANT: The default my.cnf has a "log-bin" entry. Make sure it is not listed twice.
It's also possible to set the text log file. Note both "log-bin" and "log" are set here:
[mysqld]
port = 3306socket = /tmp/mysql.socklog-bin = /usr/local/var/mysqlLOGb.loglog = /usr/local/var/mysqlLOG.log
Note, (see TIP 24:) the binary log file works with the "show binlog events command" and shows, updates,
creates, deletes, and drops, but no select statements. In contrast, the textlog-file "/usr/local/var/mysqlLOG.log"shows the user, and the select statement, all of the above, but it's in a tex
t file. I believe the textformat is going away with version 5.0.
The following command "DOES NOT" give information about these log files:
mysql> show logs;
This is only for Berkeley DB type databases before commit on this type of table. It's not what you want!
TIP 26:
Free articles and tutorials on normalization.
a) http://dev.mysql.com/tech-resources/articles/intro-to-normalization.html
8/8/2019 Mysql Tips
24/48
8/8/2019 Mysql Tips
25/48
+-----------------------+ 5 +-----------------------+1 row in set (0.00 sec)
mysql> select sequence(3.4);select sequence(3.4);
+---------------+ sequence(3.4) +---------------+ 4 +---------------+1 row in set (0.00 sec)
mysql> create table junk (a varchar(1));mysql> insert into junk (a) values ('a'),('b'),('c'),('d'),('e');
mysql> select sequence(),a from junk;
select sequence(),a from junk;+------------+------+ sequence() a +------------+------+ 1 a 2 b 3 c 4 d 5 e +------------+------+5 rows in set (0.00 sec)
An example of an aggregate function:
mysql> create table cost (inum int, cost double);create table cost (inum int, cost double);Query OK, 0 rows affected (0.00 sec)
mysql> insert into cost (inum,cost) values (3,4.5);insert into cost (inum,cost) values (3,4.5);Query OK, 1 row affected (0.00 sec)
mysql> insert into cost (inum,cost) values (10,21.3);insert into cost (inum,cost) values (10,21.3);Query OK, 1 row affected (0.00 sec)
mysql> select avgcost(inum,cost) from cost;select avgcost(inum,cost) from cost;+--------------------+ avgcost(inum,cost) +--------------------+ 17.4231 +--------------------+1 row in set (0.00 sec)
OK, HOW TO COMPILE:
For best results mysql-4.1.2-alpha or above configured with the following option:
8/8/2019 Mysql Tips
26/48
--with-mysqld-ldflags=-rdynamic
The full line of my configure is as follows since I use openssl as well.
STEP A:
./configure --with-openssl --enable-thread-safe-client --with-mysqld-ldf
lags=-rdynamic
makemake install/etc/init.d/mysql restart
(Note: if "/etc/init.d/mysql" does not exist see (TIP 20:) )
STEP B:
Under "mysql-4.1.2-alpha/sql" issue the following command.
# make udf_example.so
STEP C:
As root copy the file to a directory where mysql can find it.
# cp udf_example.so /usr/lib/udf_example.so
STEP D:
Load mysql. You may want to look at udf_example.cc, since it as instructions
as well. From here issue the following commands:
mysql> CREATE FUNCTION metaphon RETURNS STRING SONAME "udf_example.so";
mysql> CREATE FUNCTION myfunc_double RETURNS REAL SONAME "udf_example.so";
mysql> CREATE FUNCTION myfunc_int RETURNS INTEGER SONAME "udf_example.so";
mysql> CREATE FUNCTION sequence RETURNS INTEGER SONAME "udf_example.so";
mysql> CREATE FUNCTION lookup RETURNS STRING SONAME "udf_example.so";
mysql> CREATE FUNCTION reverse_lookup RETURNS STRING SONAME "udf_example.so";
mysql> CREATE AGGREGATE FUNCTION avgcost RETURNS REAL SONAME "udf_example.so";
And the functions can be deleted by the following:
mysql> DROP FUNCTION metaphon;mysql> DROP FUNCTION myfunc_double;mysql> DROP FUNCTION myfunc_int;mysql> DROP FUNCTION lookup;mysql> DROP FUNCTION reverse_lookup;mysql> DROP FUNCTION avgcost;
That's it. Just run the functions above. These can serve as templates for writing
8/8/2019 Mysql Tips
27/48
your own.
TIP 28:
Loading Data into Tables from Text Files.
Assume you have the following table.
CREATE TABLE loadtest (pkey int(11) NOT NULL auto_increment,name varchar(20),exam int,score int,timeEnter timestamp(14),PRIMARY KEY (pkey));
And you have the following formatted text file as shownbelow with the unix "tail" command:
$ tail /tmp/out.txt'name22999990',2,94'name22999991',3,93'name22999992',0,91'name22999993',1,93'name22999994',2,90'name22999995',3,93'name22999996',0,93'name22999997',1,89'name22999998',2,85
'name22999999',3,88
NOTE: loadtest contains the "pkey" and "timeEnter" fields which are notpresent in the "/tmp/out.txt" file. Therefore, to successfully loadthe specific fields issue the following:
mysql> load data infile '/tmp/out.txt' into table loadtestfields terminated by ',' (name,exam,score);
TIP 29:
Referential Integrity with InnoDB tables.
STEP 1 (First create the tables as InnoDB)
CREATE TABLE agents (akey INTEGER NOT NULL auto_increment PRIMARY KEY,ln varchar(30),fn varchar(25),phone varchar(20),timeEnter timestamp(14))ENGINE = InnoDB;
CREATE TABLE clients (ckey INTEGER NOT NULL auto_increment PRIMARY KEY,
8/8/2019 Mysql Tips
28/48
f_akey INTEGER NOT NULL,ln varchar(30),fn varchar(25),phone varchar(20),FOREIGN KEY (f_akey) REFERENCES agents(akey))ENGINE = InnoDB;
STEP 2 (Insert entries -- successful way).
mysql> insert into agents (ln,fn,phone) values ('Anderson','Bob','215-782-2134');
mysql> select @ckey:=last_insert_id();
mysql> insert into clients (f_akey,ln,fn,phone)values (@ckey,'Chirico','Abby','215-782-2353');
myslq> insert into clients (f_akey,ln,fn,phone)
values (@ckey,'Payne','Zoe','215-782-2352');
The "last_insert_id()" must be assigned to a variable, because the client entries
for the two client keys have the same agent. After the first insert into the client
table "last_insert_id()" is incremented, reflecting the new add to the client table.
STEP 3 (Try to insert a client record without a matching agent -- unsuccessful way)
mysql> insert into agents (ln,fn,phone) values ('Splat','Sporkey','215-782-9987');
Above is ok
myslq> insert into clients (f_akey,ln,fn,phone)values (last_insert_id(),'Landis','Susan','215-782-5222
');
Above Ok for the first record, but, below last_insert_id() has been
incremented and the insert will be incorrect. And probably fail,if there is no matching "akey" in agents.
myslq> insert into clients (f_akey,ln,fn,phone)values (last_insert_id(),'Landis','Brian','215-782-5222
');
SPECIAL NOTE (The "clients" table must be dropped before the "agents" table)
TIP 30:
Is it Possible to do Incremental Backups? And is it possible to restore
changes for only one user? See TIP 24 and TIP 25 for setting up andusing "binlog" and the text log. NOTE: "mysql> show logs;" IS NOT a statuson these log files -- it's only for Berkeley DB.
8/8/2019 Mysql Tips
29/48
From the shell prompt "mysqlbinlog" will list all the changes. Note yourbinary log file may be in a different directory, and have a different namedepending on the setting in /etc/my.cnf
$ mysqlbinlog /usr/local/var/mysqlLOGbin.000001
#040616 8:41:23 server id 1 log_pos 17465 Query thread_id=384SET TIMESTAMP=1087389683;CREATE TABLE exams (
pkey int(11) NOT NULL auto_increment,name varchar(15) default NULL,exam int(11) default NULL,score int(11) default NULL,PRIMARY KEY (pkey)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;# at 17761#040616 8:41:50 server id 1 log_pos 17761 Intvar
SET INSERT_ID=1;# at 17789#040616 8:41:50 server id 1 log_pos 17789 Query thread_id=384SET TIMESTAMP=1087389710;insert into exams (name,exam,score) values ('Bob',1,75);# at 17879#040616 8:41:50 server id 1 log_pos 17879 Intvar....[SNIPPED]# at 18615#040616 8:41:50 server id 1 log_pos 18615 Query thread_id=384SET TIMESTAMP=1087389710;insert into exams (name,exam,score) values ('Sue',4,99);# at 18705
#040616 8:52:35 server id 1 log_pos 18705 IntvarSET INSERT_ID=23000010;# at 18733# LOAD DATA INFILE '/tmp/out.txt' INTO TABLE `loadtest` FIELDS TERMINATED
BY ','..[SNIPPED]# file_id: 1 block_len: 200
Note the thread_id "384", session "040616", TIMESTAMP vales, and actual sqlstatements. With grep, awk, or sed it's possible to weed out unwanted querie
s.This assumes you have a base, or a "full restore" starting point.
Compare above to the actual commands listed here:
CREATE TABLE exams (pkey int(11) NOT NULL auto_increment,name varchar(15) default NULL,exam int(11) default NULL,score int(11) default NULL,PRIMARY KEY (pkey)
) ENGINE=MyISAM;
insert into exams (name,exam,score) values ('Bob',1,75);insert into exams (name,exam,score) values ('Bob',2,77);insert into exams (name,exam,score) values ('Bob',3,78);
insert into exams (name,exam,score) values ('Bob',4,80);
insert into exams (name,exam,score) values ('Sue',1,90);
8/8/2019 Mysql Tips
30/48
insert into exams (name,exam,score) values ('Sue',2,97);insert into exams (name,exam,score) values ('Sue',3,98);insert into exams (name,exam,score) values ('Sue',4,99);
mysql> load data infile '/tmp/out.txt' into table examsfields terminated by ',' (name,exam,score);
TIP 31:
Getting XML and HTML output.
Assume you have the table "exams" in the database "test".Then, the following will give you XML output if executedfrom the shell prompt with the "-X" option. For html outputuse the "-H" option.
$ mysql -X -e "select * from exams" test
1Bob175
2
Bob277
3Bob378
4Bob480
5Sue190
8/8/2019 Mysql Tips
31/48
Or, (-N is depreciated so use --skip-column-names ) for no heading:
$ mysql --skip-column-names -e "select * from exams" test
+---+------+------+------+ 1 Bob 1 75
2 Bob 2 77 3 Bob 3 78 4 Bob 4 80 5 Sue 1 90 6 Sue 2 97 7 Sue 3 98 8 Sue 4 99 +---+------+------+------+
And, add the "-s" silent option to get the following:
$ mysql --skip-column-names -s -e "select * from exams" test
1 Bob 1 752 Bob 2 773 Bob 3 784 Bob 4 805 Sue 1 906 Sue 2 977 Sue 3 988 Sue 4 99
To prevent being prompted for a password. Create "/home/chirico/.my.cnf"
with password and user. An example is shown below. Unless a useris entered "piggy" it will default to "chirico", this user.
[client]user=piggypassword=p1ggyp1ssw6r5
Use this method. It will not show up when doing "ps -aux"
TIP 32:
Views? Can you do a "create view" in MySQL? No. Not until version 5.
TIP 33:
Spatial Extensions - MySQL 4.1 supports spatial extensions.
Getting the X value of a point.
mysql> SELECT X(GeomFromText('Point(5.02 7.9)'));
+------------------------------------+ X(GeomFromText('Point(5.02 7.9)'))
+------------------------------------+ 5.02 +------------------------------------+
8/8/2019 Mysql Tips
32/48
1 row in set (0.00 sec)
Note that a comma separates points, and the x,y values are separated by spaces.
Getting the length of a line.
(1 1)/
//
(0 0)
mysql> SELECT GLength(GeomFromText('LineString(0 0,1 1)'));
+----------------------------------------------+ GLength(GeomFromText('LineString(0 0,1 1)')) +----------------------------------------------+ 1.4142135623731
+----------------------------------------------+
Getting the total length of all the lines making a step.
(2 1) ---- (2 2)
(1 0) ---- (1 1)
(0 0)
mysql> SELECT GLength(GeomFromText('LineString(0 0,1 0,1 1,1 2,2 2)'));
+----------------------------------------------------------+ GLength(GeomFromText('LineString(0 0,1 0,1 1,1 2,2 2)')) +----------------------------------------------------------+ 4 +----------------------------------------------------------+
Or the perimiter of the following triangle:
(1 1)/ \
/ \(0 0) /________ \ (2 0)
mysql> SELECT GLength(GeomFromText('LineString(0 0,1 1,2 0,0 0)'));
+------------------------------------------------------+ GLength(GeomFromText('LineString(0 0,1 1,2 0,0 0)')) +------------------------------------------------------+ 4.8284271247462 +------------------------------------------------------+
Note the following select statement is a 5x5 square withleft bottom point at 0 0.
8/8/2019 Mysql Tips
33/48
(5 0) (5 5)-----------
-----------(0 0) (0 5)
mysql> SELECT Area(GeomFromText('Polygon((0 0,0 5,5 5,5 0,0 0))'));
+------------------------------------------------------+ Area(GeomFromText('Polygon((0 0,0 5,5 5,5 0,0 0))')) +------------------------------------------------------+ 25 +------------------------------------------------------+1 row in set (0.00 sec)
The area of the inner square "9" minus the area of the outer square "25" is"16".
5 0 ------------------ 5 5 4 1 4 4 ---------- ---------- 1 1 1 4
0 0 ------------------ 0 5
mysql> SELECT Area(GeomFromText('Polygon((0 0,0 5,5 5,5 0,0 0),(1 1,14,4 4,4 1,1 1))'));
+----------------------------------------------------------------------------+
Area(GeomFromText('Polygon((0 0,0 5,5 5,5 0,0 0),(1 1,1 4,4 4,4 1,11))'))
+----------------------------------------------------------------------------+
16
+----------------------------------------------------------------------------+
1 row in set (0.00 sec)
Reference:http://dev.mysql.com/doc/mysql/en/Spatial_extensions_in_MySQL.htmlhttp://mysql.planetmirror.com/doc/mysql/en/Populating_spatial_columns.ht
mlhttp://www.ctch.net/mysql/manual_Spatial_extensions_in_MySQL.html
TIP 34:
8/8/2019 Mysql Tips
34/48
Creating Tables for Spatial Extensions.
mysql> CREATE TABLE spat (p POINT, g POLYGON, l LINESTRING, geo GEOMETRY );
Note, GEOMETRY is general and will take the following values:
- POINT- POLYGON- LINESTRING- MULTIPOINT- MULTILINESTRING- MULTIPOLYGON- GEOMETRYCOLLECTION
Inserting values.
mysql> INSERT INTO spat (p) VALUES (GeomFromText('POINT(1 2)'));
Displaying values:
mysql> select X(p),Y(p) from spat;
+------+------+ X(p) Y(p) +------+------+ 1 2 +------+------+
Note with "geo" it's possible to insert combinations
mysql> insert into spat(geo) values
(GeomFromText('MULTIPOLYGON(((0 0,10 0,10 10,0 10,0 0)),((55,7 5,7 7,5 7, 5 5)))'));
Now, to see what's in "geo", do the following:
mysql> select AsText(geo) from spat;
+-------------------------------------------------------------------+
AsText(geo)
+-------------------------------------------------------------------+
POINT(1 2)
MULTIPOLYGON(((0 0,10 0,10 10,0 10,0 0)),((5 5,7 5,7 7,5 7,5 5)))
+-------------------------------------------------------------------+
2 rows in set (0.02 sec)
TIP 35:
Working with Spatial Relationship Functions.
... example will follow
8/8/2019 Mysql Tips
35/48
TIP 36:
Difficult Column Names, Like `DATE` -- use backtick.
If using "date" as a column name, enclose it in backticks ` as follows:
CREATE TABLE IF NOT EXISTS stocks (pkey int(11) NOT NULL auto_increment,`date` date,ticker varchar(5),open float,high float,low float,close float,volume int,timeEnter timestamp(14),
PRIMARY KEY (pkey)
) ;
To get 1 day old data reference:http://biz.swcp.com/stocks/#Daily update
Statement to load data:
mysql> load data infile '/tmp/sp500hst.txt' into table stocksfields terminated by ',' (date,ticker,open,high,low,close,vo
lume);
TIP 37:
Setting up MASTER and SLAVE replication. The master will need to have binary
logging turned on.
MASTER (These steps are performed on the mysql installation that you have
designated as the MASTER). Binary logging needs to be turned on.Under the [mysqld] section in the my.cnf file add log-bin. You c
anassign this value to a file. If you assign it to a file, make su
rethe directory exists. My master IP address is 192.168.1.12 withhostname big.squeezel.com.
Step 1. Turn on binary logging. Create the necessary directories.
$ mkdir -p /var/mysql
/etc/my.cnf
[mysqld]
log-bin=/var/mysql/mysqlLOGM.log
Step 2. Restart mysql.
8/8/2019 Mysql Tips
36/48
$ /etc/init.d/mysqld restart
Step 3. Create an account on the master that has access to the tables
that will be replicated.
mysql> GRANT ALL PRIVILEGES ON *.* TO admin1@"%"IDENTIFIED BY 's3cr3tpass45' WITH GRANT OPTION;
Step 4. Login to mysql and run the "show master status" command.
mysql> show master status;+------------------+----------+--------------+--------
----------+ File Position Binlog_Do_DB Binlog_
Ignore_DB +------------------+----------+--------------+--------
----------+
mysqlLOGb.000021 270 +------------------+----------+--------------+--------
----------+1 row in set (0.00 sec)
Note the name of the file. In my case it is "mysqlLOGb.000021" and the
position. In my case the position is 270. This position will update with
changes to you database. You want a quit master to setup master slave
replication.
SLAVE (These steps are performed on the mysql installation that you havedesignated as the SLAVE). Binary logging does not have to be tur
nedon; however, it doesn't hurt. If you want all updates logged on
theslave, including the ones from the master. Then, use log-slave-u
pdates.
Step 1. Turn on binary logging, including the log-slave-updates. Create
necessary directories, and assign a unique server-id.
$ mkdir -p /var/mysql
The following changes are made in the my.cnf file. By default this
is located in /etc/my.cnf
/etc/my.cnf
[mysqld]log-bin=/var/mysql/mysqlLOGS30.loglog-slave-updates
server-id=30
Step 2. From mysql, in the slave, put in the IP of the
8/8/2019 Mysql Tips
37/48
Master or the hostname.
mysql> CHANGE MASTER TO MASTER_HOST='big.squeezel.com',
MASTER_USER='admin1',MASTER_PASSWORD='s3cr3tpass45',MASTER_LOG_FILE='mysqlLOGb.000021',
MASTER_LOG_POS=270;START SLAVE;show slave status;
This will only log future changes from the point that you implemented thechange above. If you want to get all past changes, execute the following co
mmand.
mysql> LOAD DATA FROM MASTER;
Problems:
1. Make sure the slave can get access to the master. The account must
allow outside access. Note the grant command above admin1@"%".
2. Sometimes the slave will not catch the changes. You must stop the
slave.
mysql> stop slave;
Then, run the CHANGE MASTER command above with the correct filename and
position.
References:http://dev.mysql.com/doc/refman/5.0/en/load-data-from-master.html
TIP 38:
Setting up ssl connections.
Step 1. Create your own CA.
This example is done on Fedora Core 6 as root. Note, I'm using password: "password123", and
filling in the request for Country Name "US", State "Pennsylvania" respectively. However, the
extra challenge password is not filled out. Instead, I have just hit return on this option.
It's very important to record the responses to the questions. Do it inan emacs session, since
you'll need this information when signing and generating certificates.
[root@livingroom ~]# cd[root@livingroom ~]# mkdir -p /root/certs/mysql/CA
8/8/2019 Mysql Tips
38/48
[root@livingroom CA]# /etc/pki/tls/misc/CA -newca/etc/pki/tls/misc/CA -newcaCA certificate filename (or enter to create)
Making CA certificate ...Generating a 1024 bit RSA private key
.........................++++++
...............................++++++writing new private key to '../../CA/private/./cakey.pem'Enter PEM pass phrase:password123
Verifying - Enter PEM pass phrase:password123
-----You are about to be asked to enter information that will
be incorporatedinto your certificate request.What you are about to enter is what is called a Distingui
shed Name or a DN. There are quite a few fields but you can leave some blankFor some fields there will be a default value,If you enter '.', the field will be left blank.-----Country Name (2 letter code) [GB]:USUSState or Province Name (full name) [Berkshire]:Pennsylvan
iaPennsylvaniaLocality Name (eg, city) [Newbury]:Elkins ParkElkins ParkOrganization Name (eg, company) [My Company Ltd]:chirico_
Widgetschirico_WidgetsOrganizational Unit Name (eg, section) []:chirico_Widgetschirico_WidgetsCommon Name (eg, your name or your server's hostname) []:
livingroom.squeezel.comlivingroom.squeezel.comEmail Address []:[email protected]@livingroom.mchirico.org
Please enter the following 'extra' attributesto be sent with your certificate requestA challenge password []:
An optional company name []:
Using configuration from /etc/pki/tls/openssl.cnfEnter pass phrase for ../../CA/private/./cakey.pem:passwo
rd123
Check that the request matches the signatureSignature okCertificate Details:
Serial Number: 0 (0x0)Validity
Not Before: Jan 16 16:21:25 2007 GMTNot After : Jan 15 16:21:25 2010 GMT
Subject:
8/8/2019 Mysql Tips
39/48
countryName = USstateOrProvinceName = PennsylvaniaorganizationName = chirico_WidgetsorganizationalUnitName = chirico_WidgetscommonName = livingroom.squeez
el.comemailAddress = chirico@livingroo
m.mchirico.orgX509v3 extensions:
X509v3 Basic Constraints:CA:FALSE
Netscape Comment:OpenSSL Generated Certificate
X509v3 Subject Key Identifier:95:33:77:BA:B9:7A:EE:4C:B0:0C:49:1F:56:93
:ED:CA:AE:19:9C:49X509v3 Authority Key Identifier:
keyid:95:33:77:BA:B9:7A:EE:4C:B0:0C:49:1F:56:93:ED:CA:AE:19:9C:49
Certificate is to be certified until Jan 15 16:21:25 2010GMT (1095 days)
Write out database with 1 new entriesData Base Updated[root@livingroom CA]#
Step 2. Adjust CA expiration, if desired.
Change the number of days the certificate is good for. Note above
, this certificate is goodfor 1095 days. Which is fine, but if you needed to extend this, y
ou can do it with the followingcommand.
[root@livingroom mysql]# openssl x509 -in ../CA/cacert.pem-days 3650 -out cacert.pem -signkey ../CA/private/cakey.pem
Getting Private keyEnter pass phrase for ../CA/private/cakey.pem:password123
Step 3. Create Server Certificates.
Here I'm generating two certificates. One for the computer livingroom, and one for the
computer big.squeezel.com. Generate a unique certificate for each of your servers.
# mkdir server_certs# openssl req -new -nodes \
-subj '/CN=livingroom.squeezel.com/O=chirico_Widgets/C=US/ST=Pennsylvania/L=Elkins Park/[email protected]'\
-keyout livingroomFOO-key.pem -out livingroomFOO-req.pem
-days 3650
# openssl req -new -nodes \
8/8/2019 Mysql Tips
40/48
-subj '/CN=big.squeezel.com/O=chirico_Widgets/C=US/ST=Pennsylvania/L=Elkins Park/[email protected]'\
-keyout bigFOO-key.pem -out bigFOO-req.pem -days 3650
Step 4. Sign the certificates. You'll be asked for the password during this step.
# openssl ca -days 3650 -out livingroomFOO-cert.pem -infileslivingroomFOO-req.pem
# openssl ca -days 3650 -out bigFOO-cert.pem -infiles bigFOO-req.pem
Step 5. Create a directory for the certs "/etc/mysql" and copy the files.
# mkdir -p /etc/mysql# cp /root/certs/CA/cacert.pem /etc/mysql/.# cp /root/certs/mysql/server_certs/*.pem /etc/mysql/.
Step 6. Edit mysql my.cnf file. In may case it is "/etc/my.cnf"
[mysqld]ssl-ca=/etc/mysql/cacert.pemssl-cert=/etc/mysql/livingroomFOO-cert.pemssl-key=/etc/mysql/livingroomFOO-key.pem
Step 7. Restart mysql
# /etc/init.d/mysqld restart
Step 8. Create a user that requires ssl. This is done in mysql. Do this for each client that
will need to connect.
mysql> grant all privileges on *.* to 'admin'@'localhost' identified by 'pass32rD' require ssl;
mysql> grant all privileges on *.* to 'admin'@'big.squeezel.com' identified by 'pass32rD' require ssl;
Step 9. Create the user's ".my.cnf" file. For example, for user chirico,the following file is created. Note how
the two servers differ.
This is done of server livingroom for user chirico.
[chirico@livingroom ~]$ cat /home/chirico/.my.cnf[client]user=admin
password=pass32rDssl-ca=/etc/mysql/cacert.pemssl-cert=/etc/mysql/livingroomFOO-cert.pe
8/8/2019 Mysql Tips
41/48
mssl-key=/etc/mysql/livingroomFOO-key.pem
[chirico@big ~]$ cat /home/zchirico/.my.cnf[client]user=admin
password=pass32rDssl-ca=/etc/mysql/cacert.pemssl-cert=/etc/mysql/bigFOO-cert.pemssl-key=/etc/mysql/bigFOO-key.pem
Step 10. Connect to the server.
[chirico@big ~]$ mysql --host=livingroom.squeezel.com
If you can a value for the Ssl_cipher after you connected, then, you
r connection is secure.
mysql> SHOW STATUS LIKE 'Ssl_cipher';+---------------+--------------------+ Variable_name Value +---------------+--------------------+ Ssl_cipher DHE-RSA-AES256-SHA +---------------+--------------------+1 row in set (0.00 sec)
Notes: If you're looking for ssl C API examples, download the link below
and reference the example capi/stmt_exec_ssl.c
http://www.kitebird.com/mysql-book/sampdb-3ed/sampdb.tar.gz
TIP 39:
mysqlshow - A command run from the command prompt that will quickly displaydatabase, table and
column information.
[chirico@big mysql]$ mysqlshow+--------------------+ Databases +--------------------+ information_schema admin mysql test +--------------------+
mysql_config - Displays compile time options. Inaddition to what is also shown below, it can
be used to show socket, version and cflags information.
8/8/2019 Mysql Tips
42/48
[chirico@big mysql]$ mysql_config --libs-L/usr/lib64/mysql -lmysqlclient -lz -lcrypt -lnsl -lm -L/usr/
lib64 -lssl -lcrypto
[chirico@big mysql]$ mysql_config --include-I/usr/include/mysql
TIP 40:
Queries through ssh. The server may have port 3306 closed to the outside and or
access is only granted for local accounts. Given these constraints from a remote
client, say a laptop, it is possible to perform queries and have the results returned
to the laptop provided the laptop has ssh access to the server.
For the purpose of these examples the client computer is laptop.squeezel.com and the database computer
is big.squeezel.com
[chirico@laptop ~]$ ssh big.squeezel.com 'mysql -u user1 --password=s3cr3tpass45 --database=database1 -e "create table t1 (a int)"'
Below the heading is removed -N and columns are tab separated.
[chirico@laptop ~]$ ssh big.squeezel.com 'mysql -N -B -u user1 --password=s3cr3tpass45 --database=database1 -e "select * from t2"'
A remote mysqldump is shown below.
[chirico@laptop ~]$ ssh big.squeezel.com 'mysqldump -u user1 --password=s3cr3tpass45 --database=database1 t1' > localfile
A perl example is shown below. Note the where clause below and the use of \\\" to escape quotes.
(To download this example: wget http://downloads.sourceforge.net/souptonuts/perl_sshEx1.pl)
#!/usr/bin/perl# Created by [email protected]# Updated: Sat Jan 20 10:06:41 EST 2007# perl_sshEx1.pl## Download: http://downloads.sourceforge.net/souptonuts/perl_sshEx1.p
l## Shared keys must be setup on target server.### Put in Your Values Here.my $user="user1";my $password="s3cr3tpass45";
my $database="database1";my $remote_server="big.squeezel.com";#
8/8/2019 Mysql Tips
43/48
my $SSH_CMD="ssh $remote_server 'mysql -N -B -u $user --database=$database --password=$password ";
## MySQL Command - Use \\\ for quotes. See belowmy $MySQL_CMD=" -e \"select * from email where name = \\\"Chirico\\\"
\"'";#
##
open my $SCMD, "$SSH_CMD $MySQL_CMD" or die "Could not ssh\n";
while(){
print $_;}# Files are closed automatically in Perl on exit, but# you can close if you like.close $SSH_CMD;
LONGWINDED TIPS:
LONGWINDED TIP 1: (May want to skip, since it's a long example)
SQL Characteristic Functions: Do it without "if", "case", or "GROUP_CONCAT".
Yes, there is use for this..."if" statements sometimes cause problems
when used in combination.
The simple secret, and it's also why they work in almost all databases, is
8/8/2019 Mysql Tips
44/48
thefollowing functions:
o sign (x) returns -1,0, +1 for values x < 0, x = 0, x > 0 respectivelyo abs( sign( x) ) returns 0 if x = 0 else, 1 if x > 0 or x < 0o 1-abs( sign( x) ) complement of the above, since this returns 1 only
if x = 0
Quick example: sign(-1) = -1, abs( sign(-1) ) = 1, 1-abs( sign(-1) )= 0
Data for full example:
CREATE TABLE exams (pkey int(11) NOT NULL auto_increment,name varchar(15),exam int,score int,PRIMARY KEY (pkey)
);
insert into exams (name,exam,score) values ('Bob',1,75);insert into exams (name,exam,score) values ('Bob',2,77);insert into exams (name,exam,score) values ('Bob',3,78);insert into exams (name,exam,score) values ('Bob',4,80);
insert into exams (name,exam,score) values ('Sue',1,90);insert into exams (name,exam,score) values ('Sue',2,97);insert into exams (name,exam,score) values ('Sue',3,98);insert into exams (name,exam,score) values ('Sue',4,99);
mysql> select * from exams;
+------+------+------+-------+ pkey name exam score +------+------+------+-------+ 1 Bob 1 75 2 Bob 2 77 3 Bob 3 78 4 Bob 4 80 5 Sue 1 90 6 Sue 2 97 7 Sue 3 98 8 Sue 4 99 +------+------+------+-------+8 rows in set (0.00 sec)
mysql> select name,sum(score*(1-abs(sign(exam-1)))) as exam1,sum(score*(1-abs(sign(exam-2)))) as exam2,sum(score*(1-abs(sign(exam-3)))) as exam3,sum(score*(1-abs(sign(exam-4)))) as exam4from exams group by name;
+------+-------+-------+-------+-------+ name exam1 exam2 exam3 exam4 +------+-------+-------+-------+-------+ Bob 75 77 78 80
Sue 90 97 98 99 +------+-------+-------+-------+-------+2 rows in set (0.00 sec)
8/8/2019 Mysql Tips
45/48
Note, the above pivot table was created with one select statement
You may think IF's would be cleaner. NO, WATCH OUT!Look the following gives INCORRECT RESULTS!
mysql> select name,
if(exam=1,score,null) as exam1,if(exam=2,score,null) as exam2,if(exam=3,score,null) as exam3,if(exam=4,score,null) as exam4from exams group by name;
+------+-------+-------+-------+-------+ name exam1 exam2 exam3 exam4 +------+-------+-------+-------+-------+ Bob 75 NULL NULL NULL Sue 90 NULL NULL NULL +------+-------+-------+-------+-------+
2 rows in set (0.00 sec)
Note the above gives indeterminate results.
Paul DuBois [ paul at snake.net ] showed me the correct way toperform this select statement. According to him
When you include a GROUP BY clause in a query, the only values youcan select are the grouped columns or summary values calculatedfrom the groups. If you display additional columns, they're nottied to the grouped columns and the values displayed for them areindeterminate.
If you rewrite the query like this, you get the correct result:
select name,sum(if(exam=1,score,null)) as exam1,sum(if(exam=2,score,null)) as exam2,sum(if(exam=3,score,null)) as exam3,sum(if(exam=4,score,null)) as exam4from exams group by name;
+------+-------+-------+-------+-------+ name exam1 exam2 exam3 exam4 +------+-------+-------+-------+-------+ Bob 75 77 78 80 Sue 90 97 98 99 +------+-------+-------+-------+-------+
mysql> select name,sum(score*(1-abs(sign(exam-1)))) as exam1,sum(score*(1-abs(sign(exam-2)))) as exam2,sum(score*(1-abs(sign(exam-3)))) as exam3,sum(score*(1-abs(sign(exam-4)))) as exam4,sum(score*(1-abs(sign(exam- 2)))) - sum(score*(1-abs(sign(exam-
1)))) as delta_1_2,sum(score*(1-abs(sign(exam- 3)))) - sum(score*(1-abs(sign(exam-
2)))) as delta_2_3,sum(score*(1-abs(sign(exam- 4)))) - sum(score*(1-abs(sign(exam-
3)))) as delta_3_4
8/8/2019 Mysql Tips
46/48
from exams group by name;+------+-------+-------+-------+-------+-----------+-----------+-----------
+ name exam1 exam2 exam3 exam4 delta_1_2 delta_2_3 delta_3_4
+------+-------+-------+-------+-------+-----------+-----------+-----------
+
Bob 75 77 78 80 2 1 2
Sue 90 97 98 99 7 1 1
+------+-------+-------+-------+-------+-----------+-----------+-----------+
2 rows in set (0.00 sec)
Above delta_1_2 shows the difference between the first and second exams, with the numbers
being positive because both Bob and Sue improved their score with each exam. Calculating
the deltas here shows it's possible to compare two rows, not columns whichis easily donewith the standard SQL statements but rows in the original table.
mysql>select name,sum(score*(1-abs(sign(exam-1)))) as exam1,sum(score*(1-abs(sign(exam-2)))) as exam2,sum(score*(1-abs(sign(exam-3)))) as exam3,sum(score*(1-abs(sign(exam-4)))) as exam4,sum(score*(1-abs(sign(exam- 2)))) - sum(score*(1-abs(sign(exam- 1)))) a
s delta_1_2,sum(score*(1-abs(sign(exam- 3)))) - sum(score*(1-abs(sign(exam- 2)))) a
s delta_2_3,
sum(score*(1-abs(sign(exam- 4)))) - sum(score*(1-abs(sign(exam- 3)))) as delta_3_4,
sum(score*(1-abs(sign(exam- 2)))) - sum(score*(1-abs(sign(exam- 1))))+
sum(score*(1-abs(sign(exam- 3)))) - sum(score*(1-abs(sign(exam- 2))))+
sum(score*(1-abs(sign(exam- 4)))) - sum(score*(1-abs(sign(exam- 3))))as TotalIncPoints
from exams group by name;
+------+-------+-------+-------+-------+-----------+-----------+-----------+----------------+
name exam1 exam2 exam3 exam4 delta_1_2 delta_2_3 delta_3_4 TotalIncPoints
+------+-------+-------+-------+-------+-----------+-----------+-----------+----------------+
Bob 75 77 78 80 2 1 2 5
Sue 90 97 98 99 7 1 1 9
+------+-------+-------+-------+-------+-----------+-----------+-----------+----------------+
2 rows in set (0.00 sec)
TotalIncPoints shows the sum of the deltas.
select name,
8/8/2019 Mysql Tips
47/48
sum(score*(1-abs(sign(exam-1)))) as exam1,sum(score*(1-abs(sign(exam-2)))) as exam2,sum(score*(1-abs(sign(exam-3)))) as exam3,sum(score*(1-abs(sign(exam-4)))) as exam4,sum(score*(1-abs(sign(exam- 2)))) - sum(score*(1-abs(sign(exam- 1)))) a
s delta_1_2,sum(score*(1-abs(sign(exam- 3)))) - sum(score*(1-abs(sign(exam- 2)))) a
s delta_2_3,sum(score*(1-abs(sign(exam- 4)))) - sum(score*(1-abs(sign(exam- 3)))) a
s delta_3_4,
sum(score*(1-abs(sign(exam- 2)))) - sum(score*(1-abs(sign(exam- 1))))+
sum(score*(1-abs(sign(exam- 3)))) - sum(score*(1-abs(sign(exam- 2))))+
sum(score*(1-abs(sign(exam- 4)))) - sum(score*(1-abs(sign(exam- 3))))as TotalIncPoints,
(sum(score*(1-abs(sign(exam-1)))) +sum(score*(1-abs(sign(exam-2)))) +sum(score*(1-abs(sign(exam-3)))) +sum(score*(1-abs(sign(exam-4)))))/4 as AVG
from exams group by name;
+------+-------+-------+-------+-------+-----------+-----------+-----------+----------------+-------+
name exam1 exam2 exam3 exam4 delta_1_2 delta_2_3 delta_3_4 TotalIncPoints AVG
+------+-------+-------+-------+-------+-----------+-----------+-----------+----------------+-------+
Bob 75 77 78 80 2 1 2 5 77.50
Sue 90 97 98 99 7 1 1 9 96.00
+------+-------+-------+-------+-------+-----------+-----------+-----------+----------------+-------+
2 rows in set (0.00 sec)
It's possible to combine Total Increasing Point TotalIncPoints with AVG. In fact, it's possible to combine
all of the example cuts of the data into one SQL statement, which providesadditional options for displaying
data on your page
select name,sum(score*(1-abs(sign(exam-1)))) as exam1,sum(score*(1-abs(sign(exam-2)))) as exam2,sum(score*(1-abs(sign(exam-3)))) as exam3,sum(score*(1-abs(sign(exam-4)))) as exam4,
(sum(score*(1-abs(sign(exam-1)))) +sum(score*(1-abs(sign(exam-2)))))/2 as AVG1_2,
(sum(score*(1-abs(sign(exam-2)))) +sum(score*(1-abs(sign(exam-3)))))/2 as AVG2_3,
(sum(score*(1-abs(sign(exam-3)))) +sum(score*(1-abs(sign(exam-4)))))/2 as AVG3_4,
8/8/2019 Mysql Tips
48/48
(sum(score*(1-abs(sign(exam-1)))) +sum(score*(1-abs(sign(exam-2)))) +sum(score*(1-abs(sign(exam-3)))) +sum(score*(1-abs(sign(exam-4)))))/4 as AVG
from exams group by name;
+------+-------+-------+-------+-------+--------+--------+--------+-------+ name exam1 exam2 exam3 exam4 AVG1_2 AVG2_3 AVG3_4 AVG +------+-------+-------+-------+-------+--------+--------+--------+-------+ Bob 75 77 78 80 76.00 77.50 79.00 77.50 Sue 90 97 98 99 93.50 97.50 98.50 96.00 +------+-------+-------+-------+-------+--------+--------+--------+-------+2 rows in set (0.00 sec)
Exam scores are listing along with moving averages...again it's all with one
select statement.
REFERENCES:
Good article on Spatial Extensions with MySQLhttp://dev.mysql.com/tech-resources/articles/4.1/gis-with-mysql.html
Good article on "Cross tabulations" or de-normalizing data to show stats:http://dev.mysql.com/tech-resources/articles/wizard/print_version.html
MySQL performance tuning tips:http://www.linux-mag.com/cgi-bin/printer.pl?issue=2001-06&article=mysql
See (TIP 26:) for links on normalization.
Berkeley DB sample programs:Look for berkeleydb_0.0.24.tar.gz or the latest version at:http://sourceforge.net/project/showfiles.php?group_id=79066&package_id=80379
RECOMMENDED READING:
"MySQL: The definitive guide to using, programming, and administering MySQL4.1 and 5.0, Third Edition",
Paul DuBois, Sams, Pub Date: March 08, 2005. ISBN: 978-0-672-32673-8
"MySQL Cookbook", Paul DuBois, O'Reilly, ISBN: 0-596-00145-2
"High Performance MySQL", Derek J. Balling, Jeremy Zawodny, ISBN: 0-596-00306-4
"Database Design for Mere Mortals", Michael J. Hernandez, Addison-Wesley
"Securing MySQL: step-by-step",Artur Maj (See link below)http://www.securityfocus.com/infocus/1726