Remote Execution for Oracle SQL

https://method5.github.io

2018-03-15 1

Summary1. Parallel remote execution SQL extension2. Easily run SQL, PL/SQL, and shell scripts3. Advanced features in a simple syntax:

select *from table(m5('select * from dual'));

4. Complements existing automation tools5. Open-source, secure, robust implementation6. More resources - https://method5.github.io7. Find, fix, and prevent problems everywhere

2018-03-15 2

About Jon Heller1. Oracle developer or DBA for 15 years2. DBA at Ventech Solutions in Urbandale, Iowa3. Stack Overflow top user in Oracle and PL/SQL4. github.com/jonheller15. BS and MCS in Computer Science, NCSU6. Certifications: PL/SQL, DBA, SQL Expert, SQL Tuning7. hjon@ventechsolutions.com

2018-03-15 3

Oracle Automation Gap

1. Groundhog Day DBA

2. Many simple tasks do not scale

3. Only obvious, pre-defined tasks are automated

4. Every environment has unique challenges

5. SQL and PL/SQL are great but per-database

2018-03-15 4

Current Tools - Not Good Enough

2018-03-15 5

1. Won't transform your processes

2. Slow, complex, or insecure

3. IDE, plugin, website, files, agents

4. Often expensive, closed source

5. Pre-defined tasks only

6. None of them are relational

Ideal Solution1. Treat everything as one database2. New dynamic SQL - control what and where3. New SQL syntax would be perfect:

SELECT *FROM DBA_USERSWHERE PROFILE = 'DEFAULT'

TARGETS ('DEV', 'QA')

4. We can get surprisingly close to that

2018-03-15 6

A Robust Solution – Method5

1. Not your typical home-made script

2. In production since 2014 for one of the largest healthcare data centers in the world

3. >400 databases, >1PB data

4. >10 million runs, >1800 tests, open source

5. Security has always been a priority

2018-03-15 7

Interface - Function or Procedureselect *

from table(m5('select * from dual', 'dev,qa'));

begin

m5_proc(

p_code => 'begin null; end;',

p_targets => 'dev,qa',

p_table_name => 'test_data',

p_table_exists_action => 'drop',

p_asynchronous => true,

p_run_as_sys => false

);

end;

/

2018-03-15 8

WHAT WHERE

WHATWHERE

HOW

Advantages

1. Performance - fast, parallel, asynchronous

2. Interface - plain SQL and PL/SQL

3. Relational - save, share, and join

4. Administration - for most users - none

5. Security - nothing to worry about

6. Exceptions and Metadata - handled

2018-03-15 9

Parameters

1. P_CODE - what to run

2. P_TARGETS - where to run it

3. P_TABLE_NAME - where to save it

4. P_TABLE_EXISTS_ACTION - if it already exists

5. P_ASYCHRONOUS - return or wait for all rows

6. P_RUN_AS_SYS - run with SYSDBA privilege

2018-03-15 10

Other Features1. Tables - data, _META, _ERR2. Views - M5_RESULTS, M5_METADATA, M5_ERRORS3. M5_ links - M5_* created in your schema4. Global Data Dictionary - Common tables refreshed

nightly (M5_DBA_USERS, M5_V$PARAMETER, etc.)5. Admin Email - Summary of daily issues6. Version Star - Use "**" for version differences7. Examples - Many pre-built, complex examples

2018-03-15 11

Simple Example

2018-03-15 12

More Simple Examples1. select * from table(m5('select ** from dba_data_files'));

2. select * from table(m5('#!/bin/ksh

df -h|grep Mounted;df -h|grep /tmp;'));

3. select * from m5_dba_users where username = 'SOME_USER';

4. select * from m5_v$parameter where name='ddl_lock_timeout';

5. begin

m5_proc('alter system set ddl_lock_timeout = 0', 'dev');

end;

/

select * from m5_results;

select * from m5_metadata;

select * from m5_errors;

2018-03-15 13

Example: Compare Everything Everywhere

2018-03-15 14

Example: Global ASH

2018-03-15 15

Example: ASM Forecast

2018-03-15 16

Example: Space Treemap

2018-03-15 17

Technologies1. Dynamic, templated SQL and PL/SQL

2. Database Links3. PL/SQL Lexer4. DBMS_SCHEDULER, DBMS_PIPES 5. Table-driven configuration6. Oracle Data Cartridge

2018-03-15 18

Install and Administer1. Requirements2. One DBA needed for setup and administration3. Download open source code, follow install_method5.md

and administer_method5.md4. Everything lives inside the database5. How much time will it take?6. Create GitHub issue or send email if problems

Any DBA can try it out in less than one hour

2018-03-15 19

Why Method5 is Safe1. Public database links, password sharing, direct logons2. Auditing - M5_AUDIT and database audit trail3. Multi-Step Authentication - Oracle and OS username4. Intrusion Detection - Invalid access or config changes email admin5. Shell Script and SYS - Commands from master are encrypted, remote DBA cannot break into it6. Open Source - Not just security through obscurity

2018-03-15 20

Minimum Privileges

1 - Only slightly configurable (strongly not recommended)2 - Only slightly configurable (not recommended)3 - No need to worry about4 - Highly configurable

2018-03-15 21

Method5 User

Master 1) High 3) Low

Remote 2) Medium to High 4) None

User Configuration

1. M5_USER - username (oracle and OS), email, is admin, default targets

2. M5_ROLE - name, targets, sys, shell, links, sandbox

3. M5_USER_ROLE - username, role

4. M5_USER_PRIV - role, privilege

2018-03-15 22

Security Query Examples1. m5_proc('#!/bin/ksh'||chr(10)||'crontab -l', '%');

2. m5_proc('#!/bin/ksh. /export/home/oracle/set_localASM.shcat $ORACLE_HOME/network/admin/sqlnet.ora', '%');

3. m5_proc('select ** from dba_profiles', %');

4. m5_proc('select ** from dba_audit_trail where returncode in (1017,2800)', %');

5. select database_name, value from m5_v$parameter where name= 'sec_case_sensitive_logon';

6. select * from m5_dba_role_privs where granted_role = 'DBA';

2018-03-15 23

More Information1. https://method5.github.io2. Download code, user guide, examples, roadmap,

presentation, and more3. Email the creator:

jon@jonheller.org orjon.heller@ventechsolutions.com

4. Create an issue on the GitHub repository:https://github.com/method5/method5

2018-03-15 24

Your New Mission:Automate Everything

1. Not just faster

2. Find it, fix it, and prevent it everywhere

3. Always check all databases

4. Be proactive (preventive maintenance)

5. Treat the whole data center as one database

6. Run a query today to save time tomorrow

2018-03-15 25