Assignment

Requirements

Tools to be used: SSIS for package , MySQL for Database, SQL Server for scheduling job

Use the given MySQLscript.txt script to create the necessary database and tables and use MySQLData.sql script to populate the PayrollDB database ( only PayrollDb) . Based on this data model please create the following packages to perform the ETL tasks.

SSIS Package 1:

Create a package “pkg_InsertTestData” to create new employee data based on the sample given below. The package should create 1000 new employees during every execution.

INSERT INTO `payrolldb`.`employee` (`empid`, `empname`, `email`, `password`, `salary`) values (100 +,'E100','E100@xyz.com','E100',100);

INSERT INTO `payrolldb`.`empdeptmapping` (`empid`, `deptid`) Values (100,6)

Note:

1. Use this MYSQL function UNIX_TIMESTAMP() to generate a new employeeid and replace “100” in above statements and insert. Add 1 second delay between each insert.

2. For empdeptmapping, if the newly generated EmpId is even then DeptID should be “1” else “2”.

Schedule this package as a job to run every one minute.

SSIS Package 2:

Create another package “pkg_Employee_Upsert” to move the newly created employee/empdeptmapping/department records from “Payrolldb” to “payrollstaging” database. Perform following calculation for DA and add it to the destination table along with other values.

DA Calculattion:

DA= Salary*10% if DePtID=1

DA= Salary*12.5% if DePtID=2

Else It should be Salary*0.5%

Schedule this package as a job to run once in every five minute.

SSIS Package 3:

Create another package “pkg_Employee_XML_update” to create an xml for each employee newly upserted into “Payrollstaging”.Employee and update it back to Employe.“XMLValue” column.

The XML format is given below:

<Employee empid=”Empid001”>

<name>name1</name>

<email>someone@gmail.com</email>

<password>^&Y&nfk’`</password>

<salary>50,000</salary>

<NewDa>5000</NewDa >

<departments>

<department id=”deptid1”>department_name1</department>

<department id=”deptid2”>department_name2</department>

</departments>

</Employee>

Note:

1. Each employee XML should have all the department details based on the department mapping table.

2. Validate the XML before inserting into the table. Insert it only if the XML is valid.3. If the XML is valid create it in “valid” directory If it is invalid then create the XML file in “Invalid”

directory. Employee ID should be the filename

Schedule this package as a job to run once in every five minute.