View
237
Download
1
Category
Preview:
Citation preview
7/31/2019 Etl Application
1/65
ETL APPLICATION
A.SRADHANAND
DEVELOPED
BY
M.Sc. (Comp. Sc.)
4602-525
7/31/2019 Etl Application
2/65
CONTENTS
ABSTRACT
SYSTEM ANALYSIS
SYSTEM DESIGN
SCREENS
TESTING
IMPLEMENTATIONFEATURES
CONCLUSION
BIBLIOGRAPHY
INTRODUCTION
ORGANIZATION PROFILE
7/31/2019 Etl Application
3/65
INTRODUCTION
Now a days there are so many databases are availablelike ORACLE, SQL SERVER, MS-ACCESS, DBASE etc.,
Some organizations are using some type of database
and some are using another.
The different databases are provides different
types of securities. Some provides high security
(ORACLE, SQL SERVER), some provides low security
(MS-ACCESS etc.,).
7/31/2019 Etl Application
4/65
ABSTRACT
The project entitles Extractor Transformer Loader
deals with automation of all departments. Presently the
transactions at Head of the department of SPIL
Technologies are carried in different database applications
(ORACLE, MS-ACCESS, DBASE).
Departments that play a major role at SPIL
Technologies are:
Extractor
Transformer
Loader
7/31/2019 Etl Application
5/65
In simple, using ETL Application any developer with
minimal knowledge can easily do the following things
Connect to multiple databases easily
Create databases online
Modify databases online
Execute SQL statements online Extract the database, Transform the required
database and load that database into destination
user
7/31/2019 Etl Application
6/65
ORGANIZATION PROFILE
SPIL Software Limited provides a wide range of value-added
Information Technology (IT) services. We specialize in
providing Contract Programming services, On Site Software
Development services and Permanent Staffing in the
following areas:
Internet / Web
Client / Server
Mainframe / Middleware
UNIX / Databases Telecommunication / Networking And other
platforms
7/31/2019 Etl Application
7/65
EXISTING SYSTEM
In the source and destination different databases
are present. In that department (at destination), the data
are collected from the different databases manually from
different departments and later are fed into differentdatabases respectively. From the corresponding databases
create reports and make hardcopies. These processes are
consuming more time.
7/31/2019 Etl Application
8/65
PROPOSED SYSTEM
To avoid this problem, they are proposed todevelop one application in such a way that
It connect the databases automatically according
to user requirements, converting the source database intodestination database (ORACLE) and creates softcopies
in destination user.
7/31/2019 Etl Application
9/65
FEASIBILITY ANALYSIS
In this, I analyzing the feasibility according to
the following points
Operational Feasibility
Technical Feasibility
Economical Feasibility
7/31/2019 Etl Application
10/65
Data stores in their respective database
Generating the soft copies Transformation is completed in less amount of
time
System will not slow down the performance in any
area
Operational Feasibility
7/31/2019 Etl Application
11/65
Technical Feasibility
Any no. of users can use the system in any time
There are technical guarantees like accuracy,
reliability and data security
7/31/2019 Etl Application
12/65
Economical Feasibility
The proposed system is developed using
Java2.0(JDK) Software, which is available free of costat www.java.sun.com.
http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/http://www.java.sun.com/7/31/2019 Etl Application
13/65
ENVIRONMENT SPECIFICATION
Hardware:
Processor : Pentium III
Clock Speed : 500 MHz
RAM : 64 MB
Hard Disk : 4.3 GB
7/31/2019 Etl Application
14/65
Software:
Operating System : Windows NT/Windows 98/Windows 95/ XP
Back End : 1.Source : DBASE, MS-Access, Oracle
2.Destination : OracleCompiler :Javac
Language :Java
GUI :Swing
Data Connectivity :JDBCODBC Driver
7/31/2019 Etl Application
15/65
SYSTEM DESIGN
Tables
Data Flow Diagrams
Flow Charts
7/31/2019 Etl Application
16/65
TABLES
Table name: Branch_info
Field Name Data Type Description
BRANCH_CODE VARCHAR2 Branch codeBRANCH_NAME VARCHAR2 Branch name
ADD1 VARCHAR2 Address1
ADD2 VARCHAR2 Address2
CITY VARCHAR2 City name
STATE VARCHAR2 State
PINCODE NUMBER Pin code numberPHONE NUMBER Phone number
FAX NUMBER Fax number
EMAIL VARCHAR2 Email id
7/31/2019 Etl Application
17/65
Table name: Branch_db
Field Name Data Type Description
BRANCH_CODE VARCHAR2 Branch code
DSN_NAME VARCHAR2 Data Source Name
DB_TYPE VARCHAR2 Database Type
DB_NAME VARCHAR2 Table names
Table name: Zonal_db
Field Name Data Type Description
D_BRANCH_CODE VARCHAR2 Branch code
D_DSN_NAME VARCHAR2 Data Source Name
D_DB_TYPE VARCHAR2 Database Type
D_DB_NAME VARCHAR2 Table names
NO_OF_FIELDS NUMBER Number of Fields in Table
NO_OF_RECORDS NUMBER Number of Records
7/31/2019 Etl Application
18/65
DATA FLOW DIAGRAMS
7/31/2019 Etl Application
19/65
7/31/2019 Etl Application
20/65
FIRST LEVEL DFD:
1) Request Branch for Retrieving/removing/adding
Branch information
2) Retrieving/removing/adding Branch information
3) Retrieving/removing/adding Database information
4) Receiving respected manipulation information
5) Request ETL Wizard for transformation
6) Retrieving the Branch information7) Retrieving the Branch database
8) Retrieving/storing branch and table information after
transformation/loader is occurring.
9) Storing structure and records of transformed database
10) Receiving ETL information11) Request Help for more information about application
12) Received required response
13) Request to close application
14) Response for closing application
7/31/2019 Etl Application
21/65
7/31/2019 Etl Application
22/65
SECOND LEVEL DFD1 :
1) Requesting for add branch
2) Adding branch information
3) Adding branch database
4) Response for add branch
5) Requesting for remove branch
6) Removing requested branch information
7) Removing requested branch database8) Response for remove branch
9) Requesting for view branch
10) Retrieving requested branch information
11) Retrieving requested branch database12) Response for view branch
13) Requesting for view all branches
14) Retrieving all branches information
15) Retrieving all branches database
16) Response for view all branches
7/31/2019 Etl Application
23/65
7/31/2019 Etl Application
24/65
SECOND LEVEL DFD2 :
1) Requesting for ETL Wizard2) Extracting the branch information
3) Extracting the branch database
4) Extracting information passed to transformer
5) Storing branch and table information6) Storing table structure and records
7) Informing loader that transform is completed
and invoking the load module
8) Retrieving table records
9) Shows the request response to user
7/31/2019 Etl Application
25/65
7/31/2019 Etl Application
26/65
SECOND LEVEL DFD3 :
1) Requesting author information
2) Displays author information
3) Requesting shortcut information
4) Displays shortcut information
7/31/2019 Etl Application
27/65
FLOW CHARTS
7/31/2019 Etl Application
28/65
7/31/2019 Etl Application
29/65
7/31/2019 Etl Application
30/65
7/31/2019 Etl Application
31/65
7/31/2019 Etl Application
32/65
7/31/2019 Etl Application
33/65
7/31/2019 Etl Application
34/65
7/31/2019 Etl Application
35/65
7/31/2019 Etl Application
36/65
7/31/2019 Etl Application
37/65
7/31/2019 Etl Application
38/65
7/31/2019 Etl Application
39/65
FUNCTIONAL DESCRIPTOIN
WITH SCREENS
General Functions:
Branch
ETL Wizard Queries
Help
Exit
7/31/2019 Etl Application
40/65
The various activities of the ETL Application are dealt as
different functions in the system. The main function is
performing ETL Wizard.
7/31/2019 Etl Application
41/65
I) Branch:This function is related to branches manipulation.
In this function (Menu) we are having again different
functionalities (Menu items). They are
1)Add Branch
2)Remove Branch
3)View Branch
4)View Branches
7/31/2019 Etl Application
42/65
7/31/2019 Etl Application
43/65
a. Add Branch:
For the department if any new branch is added then
by using this function we can add the database of new branch
into their respective databases.
7/31/2019 Etl Application
44/65
b. Remove Branch:
If any branch is removed from the department then by
using the function Remove Branch we can remove the
particular branch database.
7/31/2019 Etl Application
45/65
c. View Branch:
If any body (department related persons) wants to see
the particular database of the branch then we can see the
database of the particular branch by using the function ViewBranch.
7/31/2019 Etl Application
46/65
d. View All Branches:
If any body (department related persons) wants to see the all
databases of the branches then we can see the databases of
the all branches by using the function View All Branches.
7/31/2019 Etl Application
47/65
II) ETL Wizard
1) Start Wizard:
Whenever we are choosing this function, it will display
the screen called Select Branch, in which user has to
choose the branch code and depending on the branch code
it will automatically displays the branch name, DSN name,
all databases which are in the particular branch and type
of the database. After choosing the branch code user has to
select the database then click on Submit then it will
displays the screen called Transaction Center. If userselecting or clicking the button called Abort then the
process is aborted and goes to the main menu.
7/31/2019 Etl Application
48/65
7/31/2019 Etl Application
49/65
In the Transaction Center, it will display the
database structure and the related database recordssimultaneously. In this there are two buttons are there. They
are Transformation required and Transformation not
required. If the user clicking on the button Transformation
required then it will transform the data of selecting databaseinto the destination database (Oracle) and display the message
Transaction is completed. If the user selecting or clicking the
button Transaction is not required then it will aborting the
process and goes to the main menu.
7/31/2019 Etl Application
50/65
7/31/2019 Etl Application
51/65
After transformation is completed then another screen is
displays. The screen name is Loading. Through this screen
user can store the transformed data into destination user
(Oracle user) and also it will show the transformed data. In this
also there are two buttons. If we clicking on the View then the
database is loading into the destination user and shows theTransformation Loading records on different screen. Whenever
user is closing the widow it will automatically goes to the main
menu. If we select Abort then it will abort the process and
goes to the main menu.
7/31/2019 Etl Application
52/65
7/31/2019 Etl Application
53/65
III) .Queries:
Whenever user wants to create a query then he can
choose the Queries. In this there is one function is available.That function (Menu item) is Query.
Whenever the user choosing or selecting the Query then
it will display the separate window called Query Form.Through this screen user can check the databases that are in the
destination with the attribute names and related query response
information at a time.
Whenever the user is closing the window then it will
automatically goes to the main menu.
7/31/2019 Etl Application
54/65
7/31/2019 Etl Application
55/65
IV. Help:
In this function (Menu) we are having two different
functions (Menu item). They are
1) Shortcuts
2) About Me
7/31/2019 Etl Application
56/65
a. Shortcuts:
This will display the shortcuts for different menu items,
which are using in this application.
7/31/2019 Etl Application
57/65
b. About Me:
This will display the programmer and designer name
7/31/2019 Etl Application
58/65
V. Exit:
In this function (Menu) we are having one function
(Menu item) called Exit. Whenever we are choosing thisfunction the application is closed with the message called
Thankyou to using this Application.
7/31/2019 Etl Application
59/65
TESTING OF SYSTEM DEVELOPED FOR
SPIL TECHONOLOGIES
The system developed for SPIL Technologies was
thoroughly tested for any errors all through the development
phases. First, after the development of each screen the relevant
test data is used as input to test the screen. This way all the
screens were tested and at the same time cross transactions
(data transformation without losing any data or not) and also
checked for valid postings.
After the screen wise testing is completed the system istested with some sample data as a finally run of the free (the test
data is used as input is enclose in the report format).
Finally, the system was cleared for implementation.
7/31/2019 Etl Application
60/65
Implementations of Developed System at SPIL
Technologies
Present system was developed for and at SPIL
Technologies, Ameerpet, Hyderabad where the actual
implementation of the system is performed. In the view above,
the system developed is delivered to SPIL Technologies, after
thoroughly testing with the sample data. The system is checkedfor coding errors, runtime errors, data validations and postings
The equipment installation and hardware and software
check out are properly taken care. As SPIL Technologies is
already using the computers no new equipment need be
installed. Only the system developed has to be installed on the
available equipment. The client already fulfills the hardware
and software requirements by the time the order placement was
finalized.
7/31/2019 Etl Application
61/65
The end users, management and user personnel who will
be directly or indirectly involved in the usage of the system at
SPIL Technologies are trained for a short duration whichconcludes the implementation.
7/31/2019 Etl Application
62/65
FEATURES:
Manipulating the branches
Conversion wizard
Shortcuts
Saving Output results
Multiple database connections
Displays database schema and its tuples
7/31/2019 Etl Application
63/65
CONCLUSION
The main aim of this project is to automaticallyextracting source databases (Oracle, MS-Access, SQL Server)
and transformed those databases to Oracle database and
loading into destination user.
I am in the process of enhancing this project to
generalization. i.e., Converting any source database to any other
destination database.
7/31/2019 Etl Application
64/65
BIBLIOGRAPHY
1. Modern System Analysis And Design : Jeffrey A. Hoffer
2. Software Engineering Concepts : Richard Fairley
3. Complete Reference Java 2.0 : Patrick Naughton
4. Thinking in Java : Bruce Eckel
7/31/2019 Etl Application
65/65
THANK YOU
Recommended