Workforce_Management_System

CMPE 273 – Team Project Report TEAM 3

Team Project

Workforce Management System

Department of Computer Engineering

San José State University

CMPE 273: Enterprise Distributed Systems

TEAM – 3

Name: Ashok Kumar Mishra Student ID: 009991514 Name: Nishant Sharma Student ID: 010028226 Name: Shruthi Narayanan Student ID: 009984494 Name: Akanksha Singh Student ID: 010030839 Name: Mahesh Dhamgunde Student ID: 010019165 Name: Deep Tuteja Student ID: 010119941

May 6th, 2015


Contribution of Team members 1. Ashok Kumar Mishra - In initial phase, I worked on requirement analysis. I analyzed all

functionalities and worked on flow of activities. With help of this, I worked on navigation

of wireframe with Shruthi. I worked on CRUD of categories and sub categories with

Nishant. I worked on Redis caching with Shruthi. I worked on error handling and

validations. I handled Building module as well.

2. Nishant Sharma - I worked on creation of API documentation after requirement analysis. I

decided different error codes and error messages for all possible errors. I worked on

CRUD of categories and sub categories with Ashok. I worked on integration, testing and

removed errors after integration with Akansha. I implemented Report module. I also

worked on Request caching. I also worked on performance testing using JMeter.

3. Shruthi Narayanan - I worked on Wireframe design of different windows in our application.

I decided navigation from one page to other. I worked on List of users and edit user

functionality. I worked on basic UI design. I populated DB with mock data. I worked on

normalization and de normalization of DB with Mahesh. I handled Guard module. I

worked on Redis caching with Ashok.

4. Akanksha Singh - I worked on DB design with Deep and Mahesh. We designed DB schema

and normalized - de normalized according to initial requirements. I created header and

changed view of header according to user logged in. I worked on integration, testing and

removed errors after integration with Nishant. I implemented connection pooling. I also

implemented Alert module.

5. Mahesh Dhamgunde - I worked on DB design with Deep and Akansha. We designed DB

schema and normalized - de normalized according to initial requirements. I handled Client

module. I worked on integration of different modules and perform test-cases. I worked

with Deep to analyze performance of application.

6. Deep Tuteja - I worked on DB design with Akansha and Mahesh. We designed DB schema

and normalized - de normalized according to initial requirements. I worked on preventing

SQL injection. I handled Billing module. I also worked on SQL cache as well. I also worked

on integration of code. I worked on creation of report of our application and analyzed

performance of application with with Mahesh.


Design

i. Client

To design our client, we have used HTML and angular in .ejs pages. We have designed single

page application. Which means our page will not reload for each user request. Either you want

to submit any data or you want to get any data, our web page will be dynamically updated

based on the O/P of previous step.

HTML is great for static web pages. But it cannot handle when we try to use it for dynamic data.

Angular has been developed to overcome HTML’s this problem. Angular JS is developed for

dynamic UI. It has its own MVC architecture.

Following are some of the advantages of Angular JS.

Angular JS supports single page application.

Data injection into page is very simple.

Angular JS handles dependencies very well.

ii. Server

We have used “Express Framework of node.js” to implement business logic in our application.

And we have also used “Rabbit Message Queue” as Message Oriented Middleware.

Node JavaScript

“node.js” is server side JavaScript. Its basic idea is “Event Driven asynchronous callbacks”. It is

basically an single thread application. Whenever there is any request from client, it will assign

that operation to a thread from thread pool available to it. It will also assign one call function to

that task which will be called when task assigned to that particular thread is over by main

thread and appropriate response will be send to client.

Advantages of node.js

Asynchronous, so server does not get blocked. And it remains available to handle client

request.

Non-blocking I/O.

Rabbit Message Queue

We have used rabbit mq as a message oriented middleware. It is an open source message

broker that implements Advanced Messaging Queue Protocol. It uses queue structure in

between client and server to exchange messages. Client puts messages on a queue. Servers

which are listening on that queue will consume that message, and will provide necessary data


back to that client through a temporary queue. Following are some of the advantages of Rabbit

MQ.

Robust messaging for messages

Very easy to integrate in code and easy to use.

Runs on all major operating systems.

Open source and commercially supported.

iii. DB design

We have used Stored Procedures to perform any operation on database. Following are the

advantages of SPs.

Stored Procedure can help us in achieving a consistent implementation of logic across

various applications. We can club, code and test commonly performed tasks under single

SP.

SPs can also help enhance performance of our application. Many tasks are implemented in

SP as series of SQL statements. So using only single request to DB server we can execute

multiple SQL statements and get desired output.

We can also write business logic in SPs. And execute next query based on o/p of previous

query. So instead of coming back to our application for business logic, we can implement

that logic in SPs itself and all tasks are done in single DB server call only.

Stored procedure also eliminates the need for all users to know the details of tables in

database.

Following is the DB structure and their connection with other tables.



Different modules

i. Client

Following functionalities were implemented as a part of client module.

Add a client

Edit a client


Search for client based on various criteria

Delete a client


Client dashboard


Client details

ii. Building

Following functionality are implemented as a part of a building module.


Add a building for registered clients

Edit a building

Delete a building


Get details of each building

iii. Guard

Following functionality are implemented as a part of a guard module.


Add a Guard

Edit any specific Guard

Assign duties to a Guard


Search for a Guard. We can have any combinations of following criteria to search a Guard.

Get Duty Details of any Guard


Delete a Guard

Home screen when guard logged in


Guard can modify his details

iv. Alert

Following functionality are implemented as a part of a Alert module.


Create any Alert by Admin

List all Alerts

List Alerts by Building , Client


Delete any Alert

Search for an alert on basis of following criteria


Alert page when client is logged in


Client can search for alerts

Alert page when Guard is logged in


Search by Guard on basis of Building

Search by Guard


v. Report

Following functionality are implemented as a part of a Report module.

List of all Reports

Get Details of one Report


Submit Report

Search Report


Client Search Report

vi. Billing

Following functionality are implemented as a part of a billing module.

Submit a Bill. Following details will be required at time of submission


List all bills and bills for specific client


Update a bill

Billing page for Client



Role Management In our application user can have any of following roles.

Admin

Admin have the highest privileges in each module as shown below.

o Client

Add client

Search Client

Edit Client

Delete Client

View client details

o Guard

Create a guard

Assign him his weekly duties

Edit any guard

Search for guard

Delete any guard

View all guards details

o Building

Add building

Delete building

View building details

Edit building

o Alerts

Create alerts

Search alerts by client and building

Search alerts

o Report

List all Reports

Search Report

View single Report

o Bill

Submit bill

List all bills and bills by client

Update a bill

Client

Client has following privileges in each module.

o Client


View and modify his details

Client dashboard

o Alerts

View submitted alerts

Search alerts

o Report

List all Report for his account

Search Report

o Bill

View all his bills

Guard

Guard has access to following modules only.

o Guard

View his details

Modify his details

o Alert

View alerts based on building

o Report

Submit Report

Search Reports

List his Reports


Performance Enhancement Technique To measure performance in each of following criteria, we have simultaneously send request for

1, 10, 100, 250 and 500 users. We have plot graph after adding each of the following techniques

one by one.

We applied following enhancement methods on Report Module one by one. On other modules

also we have applied all enhancement methods. During taking below snapshot, there were

5000 guards, 5000 building and 12,000 alerts present in our database. Average data that was

being fetched during following Test cases was around 1.4 MB.

i. Base Graph

Following is the performance graph of very basic application. During taking following O/P, there

wasn’t any technique was used explained below.

o 1 User

o 10 Users


o 100 Users

o 250 Users


o 500 Users

ii. De-normalization

De-normalization is method to allow duplicate data in our database so that when we try to

fetch that data back, we will not have to apply any joins on db tables and all required data is

already present in single db table. This will greatly improve application performance because of

the use of simple queries and data being fetched from single table.

While designing our DB tables, we used above principle. And we can see the improvement in

performance of our application as shown in below graph.


o 1 User

o 10 Users

o 100 Users


o 250 Users

o 500 Users


iii. SQL caching

SQL database tables are stored in hard disks. So whenever we want to fetch any data from a

table we will have to perform hard disk I/O, which is very time consuming operation. But SQL

cache eliminates this I/O operation. Whenever we fetch data from any table first time SQL

server will not only returns the desired data but it will also store that data in SQL cache as well

which usually resides in RAM. So whenever we fetch any data from that table again, instead of

going to hard disk to fetch data, SQL server will read it from SQL cache and will directly returns

the desired data.

SQL servers always caches data. To measure the performance first we can disable the SQL

cache. Measure the performance and than measure the performance with SQL cache enabled.

In our application we have used SQL cache and performance has been improved due to this

which is shown below.

o 1 User


o 10 Users

o 100 Users


o 250 Users

o 500 Users


iv. Connection Pooling

Opening and Closing database connections is an expensive and time consuming process. If we

can derive any mechanism that can minimize this time then our application’s performance will

improve for sure. Connection pool is answer to this problem.

A connection pool is a cache of database connections maintained so that database connections

can be reused when future request to the database required. Every time there is an need to

access database, the pool is searched for any free db connection. Once it is found, we can use it

to access database. Once our use is over we need to put that connection back to connection

pool to be used again.

What will happen if all the connections in connection pool are busy? Either we can drop that

request and user will again have to send request to access database. Or we can implement wait

queue which will be used to store user request for the time any database connection becomes

free. By using this method success% will increase.

We have used both of the above methods in our application. And performance has been

increased for above mentioned reasons. JMETER graphs are shown below.

o 1 User


o 10 Users

o 100 Users


o 250 Users

v. Data Caching

Data caching is a mechanism to store query and its output, so that we will not have to access

database if the o/p of our desired query is already present in cache. This mechanism will save

our time in creating new database connection and fetching data from the database server

which is the most time taken by in any application.


We used “Redis” server in our application to store query and its result as an additional cache

before database server. It stores data in key-value pair. Here database query will be the key and

result from the database will be value.

If we do not find any query’s o/p in data cache then we will fetch the data from database and

stored it in data cache to be used later when same dataset may be required.

Performance of our application was improved after introducing data cache in our application.

Graphs are shown below.

o 1 User

o 10 Users


o 100 Users

o 250 Users


vi. Request Caching

Request caching is mainly used to provide a mechanism to application to scale faster and

perform faster. This caching mechanism is applicable only to idempotent requests, because

only such requests will yield same results when executed multiple times. This means only GET

requests can be cached and not POST.

In this mechanism when we get ant http GET request we will search our cache for same

request. If we found one, we will fetch response for that request from cache and response time

for that request will be very less as we will not have to go to application server to fetch the

data.

We have implemented Request caching in our application, And performance has been

improved because of the above reasons. JMETER graph for it are as shown below.

o 1 User


o 10 Users

o 100 Users


o 250 Users

o 500 Users


vii. Paging

Paging is a memory management technique. In this memory is divided into fixed sized pages.

Whenever user wants any data instead of fetching all rows from that table, our application will

fetch only predefined number of rows. We have given user an option to provide number of

rows he wants to see in single page.

So by this technique even though number of database access will be increased, but single

database access will be much faster and more practical.

o 1 User


o 10 Users


o 100 Users

o 250 Users


o 500 Users


viii. Overall Performance

I have integrated all JMeter readings in below table. In this we can see how performance is

getting enhanced after applying each technology.

Load Type Users

Response Time ( In milliseconds) Average Time for each Request Test 1 Test 2 Test 3

Base

1 362 393 343 366

10 929 953 912 931.33

100 11738 12032 12403 12057.66

250 30082 29910 29310 29767.33

500 58790 59030 62030 59950

De-Normalize

1 311 308 392 337

10 925 942 1023 963.33

100 11698 11878 12342 11972.66

250 29169 29390 28332 28963.66

500 58422 57034 58203 57886.33

SQL Cache

1 301 322 403 342

10 898 932 1023 951

100 11734 11376 12123 11744.33

250 28343 28290 29143 28592

500 55137 56225 54005 55122.33

Connection Pool

1 322 323 343 329.33

10 831 841 839 837

100 11179 11340 11430 11316.33

250 27968 27834 28102 27968

500 55894 56340 55943 56059

Data Cache

1 100 84 92 92

10 130 140 124 131.33

100 3619 3232 3354 3401.66

250 9234 9103 9420 9252.33

500 18580 17902 17730 18070.66

Response Cache

1 68 59 43 56.66

10 71 79 67 72.33

100 2876 2778 2654 2769.33

250 7851 7920 8010 7927

500 16074 15832 15783 15896.33

SQL Paging

1 30 24 32 28.66

10 23 25 22 23.33


100 2221 2403 2389 2337.66

250 7050 6940 7121 7037

500 12460 12093 12392 12315

Graph illustration of above results.

Graph with respect to response time


Graph with respect to % improvement over base project


Observations and lessons learnt

1. While applying connection pooling, we learnt that a lot of time can be saved from connection

pooling reducing significant overhead of server. This improves performance of application. This

reduces response time and results in faster application.

2. De-normalization helps in getting better performance in this case. We learnt efficient DB

handling from this.

3. By preventing SQL injection, we learnt how to stop malicious attacks on application. No one

should be able to access any account without having valid access.

4. Redis caching helped in improving the performance of system. Data is accessed once when

application starts. After that cached key value pairs are accessed. As DB is not accessed every

time, execution time is reduced.

5. We learnt asynchronous behavior of node while developing this application.

6. During implementing SQL caching and Request caching, we learnt that even though at small

scale these methods does not give any impact but while working under heavy traffic the

improvement in performance in very significant.

7. While developing all functionalities of application, we learnt why proper modularization of

code is important. This removes redundancy and also helps in making changes easy to apply.

Documents

Workforce_Management_System