Diploma thesis 2001 Distributed ICS Distributed ICSsecurity.hsr.ch/theses/DA_2001_DistributedICS.pdf · Diploma thesis 2001 Distributed ICS Distributed ICS AUTHOR(S): Martin Seelhofer

Diploma thesis 2001

Dis

trib

ute

d IC

S D

istr

ibu

ted

ICS

AUTHOR(S): Martin Seelhofer I98a, ZHW, Winterthur Fabio Soldati I98a, ZHW, Winterthur

SUPERVISORS: Dr. Andreas Steffen ZHW, Winterthur Andreas Hess SunGard AG, Zurich

DATE: 28-Oct-2001

T R A D I N G A N D R I S K S Y S T E M S

on behalf of

Abstract

The main goal of our diploma thesis consisted of sketching and developing a client-server architecture which extends an existing component-framework with a distribution mechanism for log-messages. The already available framework was developed by ‘SunGard Trading and Risk Systems’ in Zurich. It serves the purpose to simplify the integration of different systems and their data while offering the possibility to read some data out of an arbitrary source, to transform it according to user-defined rules and to finally write it into an arbitrary target. For this kind of process, the term ‘Dataflow’ is used.

After having made ourselves familiar with the topic and having defined the first Dataflow examples, we started to arrange the management document, which contains the way we planned to proceed, the responsibilities and the time management. Thereafter, we set up the catalog containing the detailed requirements in cooperation with Andreas Hess, our supervisor on SunGard’s part. Since, after some work of persuasion by a SunGard engineer, we decided to follow international guidelines in the way we structured our documents, we overstretched our schedule a bit on these. However, we were able to catch up with our schedule in the first part of the implementation phase.

As the analysis phase revealed, the existing system posed a serious problem which had to be solved early in the design phase: most of the generated log data could not be uniquely associated with a Dataflow. Our system solves the problem with an unusual but at the same time very simple approach: It executes each Dataflow inside a separate operating system process.

While having been forced to use Java as the programming language and target platform, we were allowed to freely specify the design of our system and to choose the network technologies to be used. A comparison of the most important technologies being applicable resulted in the decision to use RMI as the communication protocol. Although at first sight, a multicast technology seemed to be appropriate, the degree of complexity of such a solution would have been out of all proportion to the gain in bandwidth.

One of the main requirements to our solution consisted of possibly keeping the coupling to the existing framework very tiny in order to allow it to be further developed as independently as possible. We succeeded in narrowing down the necessary adjustments to the existing system to 22 characters of source code!

Summarizing the project, we didn’t encounter any serious difficulties. Once more, the choice of a system as loosely coupled as possible turned out to become a big advantage. This was specifically true for the allocation of the modules and their integration and the testability of the individual components. However, the pleasant co-operation with our supervisors and the motivating atmosphere at SunGard Trading and Risk Systems might have also contributed to the smooth course of action.

Winterthur, 27-October-2001

Martin Seelhofer Fabio Soldati

Zusammenfassung

Aufgabe unserer Diplomarbeit war es, eine Client-/Server-Architektur zu entwerfen und umzusetzen, welche ein bestehendes Komponenten-Framework um einen Verteilmechanismus für die anfallenden Log-Daten erweitert. Das bestehende Framework wurde von der Firma ‚SunGard Trading and Risk Systems’ in Zürich entworfen und dient dem Zweck, die Integration unterschiedlicher Systeme und deren Daten zu vereinfachen. Es bietet die Möglichkeit, Daten aus einer beliebigen Quelle zu lesen, diese nach eigenen Regeln zu transformieren und danach in ein beliebiges Ziel zu schreiben, wobei für solche Ablaufsmuster der Begriff ‚Dataflow’ verwendet wird.

Nachdem wir uns in die Thematik eingelesen und die ersten Dataflow-Beispiele selbst definiert hatten, wagten wir uns an das Zusammenstellen eines Management Dokuments heran, welches unser Vorgehen, die Verantwortlichkeiten und das Zeitmanagement beinhaltet. Danach erstellten wir in Zusammenarbeit mit Andreas Hess, unserem Betreuer seitens des Auftraggebers, den detaillierten Anforderungskatalog. Da wir – nach einiger Überzeugungsarbeit durch einen SunGard Mitarbeiter – beschlossen hatten, unsere Dokumente an internationalen Standards zu orientieren, brauchten wir hierfür etwas mehr Zeit als geplant. Die verlorene Zeit konnten wir dann im ersten Teil der Implementierungsphase aber wieder gut machen.

Wie sich in der Analysephase zeigte, besass das bestehende System ein schwerwiegendes Problem, welches früh in der Design-Phase gelöst werden musste: die meisten der generierten Log-Daten konnten nicht eindeutig einem Dataflow zugeordnet werden. Unser System löst das Problem mit einem ungewöhnlichen zugleich aber sehr simplen Ansatz: Es führt jeden Dataflow in einem separaten Betriebssystemprozess aus.

Während Java als Programmiersprache und Zielplattform als Rahmenbedingung vorgegeben war, wurde der System-Entwurf und die Wahl der zu verwendenden Netzwerk-Technologien uns selbst überlassen. Nach einem Vergleich der wichtigsten fünf in Frage kommenden Technologien, haben wir uns für den Einsatz von RMI als Kommunikations-Protokoll entschieden. Auf den ersten Blick schien sich zwar der Einsatz einer Multicast-fähigen Technologie geradezu aufzudrängen. Der Komplexitätsgrad einer solchen Lösung hätte allerdings nicht im Verhältnis zum Bandbreiten-Gewinn gestanden.

Eine der Hauptanforderungen an unsere Lösung bestand darin, die Kopplung an das bestehende Framework so gering wie möglich zu halten, damit dieses so unabhängig wie möglich weiterentwickelt werden kann. Es ist uns dabei gelungen, die am bestehenden System nötigen Anpassungen auf ganze 22 Zeichen Quellcode zu begrenzen!

Zum Ablauf der Diplomarbeit kann gesagt werden, dass wir auf keine grösseren Schwierigkeiten gestossen sind. Die Wahl eines möglichst lose gekoppelten Systems stellte sich einmal mehr als äusserst ideal heraus. Dies traf sowohl auf die Modulaufteilung und –Integration wie auch auf die Testbarkeit der einzelnen Komponenten zu. Es dürften aber auch die angenehme Zusammenarbeit mit unseren Betreuern und die motivierende Atmosphäre bei SunGard das Ihre dazu beigetragen haben.

Winterthur, 27. Oktober 2001


Aufgabenstellung

Eine Schnittstellen- und Integrations-Applikation ist verantwortlich für die Manipulation und den Transport von Daten unterschiedlicher Systeme. Diese Applikation wurde als „Adapter Framework“ von der Firma SunGard Trading and Risk Systems in 100 % Java 2 geschrieben. Sie hat zur Absicht die Integration unterschiedlicher Systeme und deren Daten zu vereinfachen. Dabei erlaubt dieses „Toolkit“ Datenflüsse zwischen verschiedenen Systemen zu erstellen. Die ausgeschriebene Diplomarbeit soll dieses Framework erweitern. Hierbei stellt ein JAVA Component Server das zentrale Objekt eines verteilten Systems dar. Der JAVA Component Server wird zur Entkopplung eines lokalen Systems benötigt, um koordiniert und konsistent auf Objekte zuzugreifen. Dabei verwaltet und koordiniert er sämtliche Zugriffe und Manipulationen auf den konsistenten Daten und Objekten. In diesem System greifen unabhängige und verteilte Softwarekomponenten auf gemeinsam genutzte JAVA Components zu. Diese Java Components sind weitgehend eigenständige Objekte, welche unabhängig voneinander konfigurierbar und ausführbar sind. Zur Laufzeit werden Log-Daten produziert, welche die Zustände und Vorgänge dieser Objekte beschreiben. Diese Daten sind momentan an die direkte Umgebung gebunden. Zum heutigen Zeitpunkt können diese Objekte remote gestartet und gestoppt werden. Zur Laufzeit werden Log-Daten produziert, welche wir als Auditfunktionalität auch auf dem Remote Client zugänglich machen möchten. In der vorgeschlagenen Diplomarbeit soll ein Framework erarbeitet werden, welches uns erlaubt verteilt, und unabhängig auf diese Log-Daten zuzugreifen.

Ziele: • Im Zentrum dieses Frameworks steht die Konzeption und Umsetzung eines verteilten Systems,

welches diese Auditfunktionalität anbietet. • Ein auf Java basierender Serverdienst hat die Aufgabe, diese Log-Daten transparent zu verteilen. • Jeder Audit-Client muss zur Laufzeit eine direkte Ansicht auf die Log-Daten der gewünschten

Objekte erhalten. Dabei müssen sich Clients am Server registrieren und autorisieren können. • Ein sogenannter Audit-Client hat also die Möglichkeit, sich diesem Serverdienst anzumelden und

wird mit den Laufzeitinformationen der gewählten Objekte vom Server bedient. • Analyse

o In einem ersten theoretischen Analyse-Teil, soll die Ausgangslage erfasst und die vorhandene Umgebung kennengelernt werden. Dabei wird der Rahmen festgelegt, die Aufgabe zerlegt sowie analysiert und das Grobkonzept zur Aufgabenstellung verfeinert (detaillierte Zielsetzung und Anforderungsspezifikation).

o Zur Aufwand- und Zeitschätzung soll ein Zeitplan erstellt werden. Die Architektur und das System-Design soll unter Berücksichtigung unterschiedlicher Technologien als Konzept anhand der ausgearbeiteten Anforderungen erarbeitet werden.

o Die verwendbaren Technologien und Methoden müssen miteinander verglichen, diskutiert und bewertet werden.

• Implementation o Basierend auf den Erkenntnissen der Erarbeitung von Zielsetzung und

Anforderungsspezifikation erfolgt eine Umsetzung des Frameworks. Die Lösung soll direkt in die vorhandenen Komponenten integriert werden und damit den gewählten Lösungsvorschlag aufzeigen und demonstrieren.

Winterthur, 6. September 2001

Dr. Andreas Steffen

Preface

Why in English? This diploma thesis was entirely written in English to conform to the internal guidelines of ‘SunGard Trading and Risk Systems’.

Document structure We tried to let our documents conform to international standards in the area of software development. The standards applied are indicated in the following figure which shows the documents we produced. Additionally, the standards are indicated on the second page of each document:

ProjectManagement

Plan

getting

started

SoftwareRequirementsSpecification

analyzing

the problem

SoftwareDesign

Description

designing a

solution

Installationguide &User‘sManual

releasing

a prototype

Diplomathesis

Diplomathesis

Diplomathesis

Diplomathesis

IEEE 1058 IEEE 830 IEEE 1016

Who should read what document?

Document Audience

Project Management Plan Project Managers, Supervisors & Examiners

Software Requirements Specification Customer, Project Managers, Supervisors & Examiners

Software Design Description Software Engineers, Supervisors & Examiners

Installation Manual & User’s Guide End Users, Supervisors & Examiners

CD-Rom At the end of this diploma thesis, you’ll find a CD containing the following items:

! Java-Class files needed to run the client application ! Java-Class files needed to run the server side components ! Java-sources for the above ! Setup script to install a pre-configured system for demonstration and test purposes ! JavaDoc-style documentation of the source code in HTML-format ! A running version (minimally adjusted) of SunGard’s existing ICS, checked out of

the repository October 10, 2001. ! All the relevant documents (project management, requirements, this document, …) ! A cool Autostart-Menu

Preface Diploma thesis

A look into the future We hope that our product will be widely used and will help the people of SunGard (Switzerland) to gain acceptance within their company.

To do Nothing, because everything is perfect, already ☺

Thanks At this point, we would like to say thanks to all the people who have supported us. These include but are not limited to:

! Stephanie Mutter & Tina Moosberger

! Dr. Andreas Steffen, ZHW

! Andreas Hess, SunGard

! Chris Hauser, SunGard

! Roman Bestler, SunGard

! Thomas Frank, SunGard

! Stephan Lamprecht, I98a ZHW (Thanx for helping us out with enough supply of RedBull)

Winterthur, 27-Oct.-2001


Project Management

Plan

Dis

trib

ute

d IC

S D

istr

ibu

ted

ICS

AUTHOR(S) Martin Seelhofer I98a, ZHW, Winterthur Fabio Soldati I98a, ZHW, Winterthur

SUPERVISORS Dr. Andreas Steffen ZHW, Winterthur Andreas Hess SunGard AG, Zürich

DATE 27-Oct-2001 VERSION 1.4 FILE Project Plan 1.4.doc

Version history:

Version Description Authors Date

1.0 First Draft F. Soldati, M. Seelhofer 14-Sep-2001

1.1 Added Risk Management aspects F. Soldati, M. Seelhofer 18-Sep-2001

1.2 Adjusted Project Organization M. Seelhofer 20-Sep-2001

1.3 Added Implementation plan M. Seelhofer, F. Soldati, 26-Oct-2001

1.4 Final version M. Seelhofer, F. Soldati, 27-Oct-2001

This document conforms to the international IEEE standard 1058.1, “Standard for Software Project Management Plans”.

Table of contents

Chapter 1 Introduction 1 1.1 Purpose 1 1.2 Overview 1 1.3 Scope of Supply 1 1.4 Project Plan 2 1.5 The Personal Situation 2

Chapter 2 Project Organization 3 2.1 Development Model 3 2.2 Organization 3 2.3 Responsibilities 3 2.4 Administration of Files, Documents 4

2.4.1 Prototypes 4 2.4.2 Documents 4 2.4.3 Directory Structure 4

2.5 Versioning 4

Chapter 3 Management Methods 5 3.1 Team-Management 5 3.2 Assumptions, Dependencies and Restrictions 5 3.3 Risk-Management 5 3.4 Project-Monitoring and -Control 5

Chapter 4 Technical Aspects 6 4.1 Tools and Technologies 6

Chapter 5 Time management and costs 7 5.1 Packages of Work 7 5.2 Dependencies 7 5.3 Costs and Resources 7

5.3.1 Project Schedule 7 5.4 Milestones 8 5.5 The first Story 8 5.6 The second Story 9 5.7 The final Story 9 5.8 Time Management 10

Project Management Plan Chapter 1: Introduction

Page 1 of 10

Chapter 1 Introduction

1.1 Purpose

This is the main management document for the project Distributed ICS. It contains a lot of management information like responsibilities, descriptions of the methodologies used for development, list of rules used to work together in the project-team, risk-management and the project-schedule. Additionally, applications used for the development-phase as well as needed and used resources are listed.

Since this document’s purpose is to provide an overview of the project from a manager’s point of view, it does not contain any technical details. That information can be found in the documents ‘Software Requirements Specification’ and ‘System Design Description’, respectively.

1.2 Overview

To get a basic idea of what this project is all about, you can read the document ‘Software Requirements Specification’.

The project team consists of Fabio Soldati and Martin Seelhofer, both students of the University of Applied Sciences of Zurich in Winterthur (ZHW). The project will result in the two students’ diploma thesis.

Dr. Andreas Steffen, a lecturer at the ZHW, and Andreas Hess, a technical consultant at SunGard Trading and Risk Systems AG in Zurich, are the supervisors of this project.

All the software is written in Java. The development environments JBuilder 5.0 by Borland and Together 5.02 by TogetherSoft are used for that purpose. Drawings are created using Microsoft Visio 2000 or PowerPoint 97 / 2000. Text documents are written with Microsoft Word 97 / 2000. The project schedule and some other documents with tabular representation of data were created using Microsoft Excel 97 / 2000.

1.3 Scope of Supply

The following items belong to the scope of supply:

• This Project Management Plan

• Software Requirements Specification

• Software Design Description

• System prototype (built around ICS-classes checked-out on 10. of October [beta version])

• Java Source code

• HTML document of the source code (created with JavaDoc)

• CD containing all documents, the software and the source code

Project Management Plan Chapter 1: Introduction

Page 2 of 10

1.4 Project Plan

This project plan was created at the beginning of the project and discussed with the supervisors. It will be adjusted, as changes are needed. However, if extensive changes must be made, all involved persons should be consulted.

Any member of the project team is allowed to apply changes. In order not to confuse versions, changed documents must be acknowledged by the whole team. Acknowledged versions should get a new version number.

1.5 The Personal Situation

Team members: abbreviation

Martin Seelhofer [email protected] MS

Fabio Soldati [email protected] FS

Supervisors:

Andreas Hess [email protected] AH

Dr. Andreas Steffen [email protected] AS

Project Management Plan Chapter 2: Project Organization

Page 3 of 10

Chapter 2 Project Organization

2.1 Development Model

For this project, we chose to generally follow the spiral model by Barry Boehm:

“Basically, the idea is evolutionary development, using the waterfall model for each step; it's intended to help manage risks. Don't define in detail the entire system at first. The developers should only define the highest priority features. Define and implement those, then get feedback from users/customers (such feedback distinguishes "evolutionary" from "incremental" development). With this knowledge, they should then go back to define and implement more features in smaller chunks”

taken from the website: http://www.faqs.org/faqs/software-eng/part1/section-4.html

However, some features of the Extreme Programming (XP) methodology by Kent Beck will be used. These include but are not limited to: coding Unit Tests before coding the corresponding classes, defining Milestones in terms of ‘Stories’ (modules) and ‘Tasks’ (short units of work to be accomplished within a day or two), daily stand up meetings etc.

2.2 Organization

The project team consists of two members (F. Soldati and M. Seelhofer) and two supervisors (A. Steffen and A. Hess). Please refer to section 1.5, The Personal Situation, for details.

2.3 Responsibilities

Generally, all members of the team are equally responsible for all the tasks. This is specifically true for the phases of analysis and system design and for NOT changing interfaces without consulting all other members during the implementation phase. However, in the implementation phase, each member will have the sole responsibility for the functionality of his own modules (modules developed by him).

Each team member should create backups of his code himself. No additional mechanisms for assuring quality will be considered.

The responsibility to back up documents not belonging to a specific member is assigned to Martin Seelhofer. In turn, Fabio Soldati will be responsible for removing deprecated document versions from the project- and moving it to a special archive-folder.

Project Management Plan Chapter 2: Project Organization

Page 4 of 10

2.4 Administration of Files, Documents

2.4.1 Prototypes

The source code of each prototype (by the term ‘prototype’ we mean the result of a milestone of the implementation phase) will be put into a separate folder in the project-directory. This folder will carry the date and the version number in its name.

2.4.2 Documents

All the documents reside on the folder ‘Diplomarbeit-ZHW2001’ on SunGard AG’s share ‘\\zusirius\scratch\’. Since this folder is NOT backed-up automatically, manual backups are made every day. (See chapter 2.3, Responsibilities)

Further on, during the implementation phase, a backup on CD is made every week.

2.4.3 Directory Structure

The root directory Java Runtime Environment Project directory of distributed ICS Batch flies for controlling the programs Binaries for distributed ICS Class files from distributed ICS Class files from ICS Images and icon for the ICS Manager Resources (Icons and messagebundles) for the Audit Client Some test files for checking the functionality Third party jar files Source files Source code distributed ICS Source code old ICS

2.5 Versioning

The version numbers consist of 3 numbers, which are separated by a dot. While software modules may use all the 3 numbers, documents only make use of the first two:

The last number identifies the number of revisions inside the same version. Formatting documents, changing sentences or correcting the spelling are examples. An example for software modules is the rewrite of a method (e.g. for performance reasons) which still provides the same functionality.

The second number is used to keep track of important steps, like changing a document’s structure or adding new information / functionality.

The first number identifies the main version. It must be increased after reaching a new level of functionality is implemented. Zero indicates that the document / module is not yet ready to be released.

0.0.0

Project Management Plan Chapter 3: Management Methods

Page 5 of 10

Chapter 3 Management Methods

3.1 Team-Management

The workload will be split up in equally sized parts for each of the team members. By composing those parts with care, the team members will be able to develop independently. For a discussion of responsibilities, see chapter 2.3.

3.2 Assumptions, Dependencies and Restrictions

While this project will entirely be implemented in Java, no additional restrictions are applied. This means, that no specific technology needs to be used as long as the requirement specifications do not prescribe it (there exists e.g. no restriction about the network communication technology to be used). However, even in those cases, we may be forced to make some compromises during the implementation phase. While this can even result in minor differences from the requirements, it is required to discuss them with the supervisors and to well document them.

3.3 Risk-Management

The following table lists some risks, which could affect this project negatively and the (estimated) possibilities for them to take effect:

Risk Cause Result Possibility Solution

Loss of data Hardware failure Delay 5% Backup regularly

Unforeseen inability of a team member to work on this project

Sickness, accident, criminal action

Severe Delay 5% None (keeping the team members happy?)

The system design phase takes too long

Too complex procedures Delay or reduction of functionality

30% Loosening requirements

Prototypes take too much time to implement

Technology too complex Delay 20% Choose simple and widely used technologies

Too much functionality Misinterpretation of requirements

Delay 10% Regular consultation of requirement specs

Insufficient performance Insufficient network conditions, heavy server load

Loss of quality, not meeting requirements

10% Evaluation, tests

Implementation of unnecessary functions

Requirement specifications too extensive

Delay 5% Working overtime

3.4 Project-Monitoring and -Control

Review sessions will be called-up as needed. Usually, one longer session with the team members and both supervisors will be held per week. Additionally, several shorter sessions will be held on which the team members and Andreas Hess will participate. The goal of the sessions will be to discuss open questions, exchange information and monitor the project schedule.

Every participant of a review session will be responsible for writing protocols himself.

Project Management Plan Chapter 4: Technical Aspects

Page 6 of 10

Chapter 4 Technical Aspects

4.1 Tools and Technologies

This system is developed by making use of object oriented technology. For this purpose, the following tools are used:

Tool Purpose Java Development Kit 1.3.1, by SUN • Programming Language, Runtime Environment

Together 5.02, by TogetherSoft • Object Oriented Design • Coding • Diagrams (Sequence diagrams, collaboration

diagrams, class diagrams)

JBuilder 5.0, by Inprise • Browsing source code of existing system (ICS)

Project Management Plan Chapter 5: Time management and costs

Page 7 of 10

Chapter 5 Time management and costs

5.1 Packages of Work

In this project, one package of work is referred to as ‘story’. These stories will be defined in terms of the goals of milestones and prototypes. They will be discussed thoroughly every week and adjusted as needed.

5.2 Dependencies

The chronological dependencies between stories are depicted in the project schedule.

5.3 Costs and Resources

This project is carried out as a diploma thesis. Therefore, no cost estimate is made. Instead, the available budget consists of a total of 7 weeks of time per team member.

5.3.1 Project Schedule

06.0

9.20

01

07.0

9.20

01

10.0

9.20

01

11.0

9.20

01

12.0

9.20

01

13.0

9.20

01

14.0

9.20

01

17.0

9.20

01

18.0

9.20

01

19.0

9.20

01

20.0

9.20

01

21.0

9.20

01

24.0

9.20

01

25.0

9.20

01

26.0

9.20

01

27.0

9.20

01

28.0

9.20

01

01.1

0.20

01

02.1

0.20

01

03.1

0.20

01

04.1

0.20

01

05.1

0.20

01

08.1

0.20

01

09.1

0.20

01

10.1

0.20

01

11.1

0.20

01

12.1

0.20

01

15.1

0.20

01

16.1

0.20

01

17.1

0.20

01

18.1

0.20

01

19.1

0.20

01

22.1

0.20

01

23.1

0.20

01

24.1

0.20

01

25.1

0.20

01

26.1

0.20

01

5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6 2 3 4 5 6

Do Fr Mo Di Mi Do Fr Mo Di Mi Do Fr Mo Di Mi Do Fr Mo Di Mi Do Fr Mo Di Mi Do Fr Mo Di Mi Do Fr Mo Di Mi Do Fr

Analyzing the projectUnderstanding the goals of this projectfinding / choosing Literature and TechnologyProject-ScheduleRequirement specificationsUnderstanding ICS (code, package structure)Some Design-StudiesRough draft MLS •Adjusting Project-ScheduleSome test-implementations for demonstration and documentation purposesDefining a solution for the "ID-problem"Final draft MLS •

Software DesignOverview of MLS-server's functionalityRough structure of the MLS-serverBreak structure into modulesDefine Milestones (Implementation)Design paper •

ImplementationFirst Story* •Second Story* •Final Story* •Functional testsTest-Installation •

DocumentationDocumentationFinal Draft •

* see Chapter 'Implementation Plan'

Milestone •

Figure 1: Schedule


Page 8 of 10

5.4 Milestones

The milestones to achieve in this project are referred to as stories, according to the Extreme Programming method developed by Kent Beck. As he suggested, we split up each story into tasks of only a few person-days of work. Our estimations are rounded to a quarter of a day. To be able to verify our estimations, the effectively used amount of work is listed, too.

Since the project schedule lets us work on the implementation for exactly 3 weeks each, we decided to set up one story per week. Therefore, we have about 5 days per person to achieve the goals of each story. However, software development always turns out to be more time-consuming than anticipated and there are usually many other things to do instead of writing code – discussions with the supervisors, answering email, solving unforeseen problems or making private telephone calls. As a consequence, we decided to plan only about 60% of the available time to be used for implementing tasks.

5.5 The first Story

The first story leads us to a first prototype with yet limited functionality. It will already be possible to start, stop and abort DataFlows (each running in its own virtual machine) and to receive log-messages printed to standard out on the client side. But there will not yet be a graphical user interface to do this, everything must be done through command line.

The specific tasks in this story are:

Task Resp.1 Est.2 Eff.3

Develop the process manager, which starts a separate process for each DataFlow. No monitoring will be integrated, but a client must have the possibility to attach to a running DataFlow.

MS 1 2

Develop first approach of the filter functionality as described in the document “Requirements Specification”.

FS/MS 1.5 1.5

Extend the class MessageLogging with the functionality needed to be able to send the log-messages which DataFlows do produce to a client.

FS/MS 0.5 0.25

Implement the message-distribution module. In its first version it won’t do buffering, it simply will block. However, it should already filter the messages according to the user’s settings.

FS 1 0.5

Refactor the existing remote interface into a console client able to allow filters specified as command line parameters.

FS/MS 0.5 1

Total 4.5 5.25

1 The person who will be responsible for this task and will program it 2 How much time to implement does the responsible person estimate this task to take (if both team

members work on a task, the accumulated time is estimated!) 3 How much time to implement did this task actually take (don’t be surprised here: according to

http://www.ration.com, the average software project overshoots its schedule by 222%)


Page 9 of 10

5.6 The second Story

The second story ends up in a second prototype, which will already contain a simple GUI. This GUI of the audit-client module will allow starting, stopping and aborting DataFlows as well as displaying the log-messages received. Additionally, the user management functions will be integrated, a client will have to log-in before doing anything on the server.



Implement a GUI, which provides commands for starting, stopping and aborting DataFlows and is able to display log-messages received by the server. It will not yet be possible to set filters during run-time; they must still be specified with a startup parameter.

FS/MS 2 2.5

Develop the user database functionality (defining / removing users). This includes an interface to ask for validity of a given user / password-pair.

MS 1 0.75

Generate the log-in mechanism. This contains a login-dialog on the client side and an access-control class on the server side which interfaces with the user database.

MS 1 0.75

Extend the distribution module with a buffering mechanism. FS 0.5 0.25

Total 4.5 4.25

5.7 The final Story

The last story ends up in the final prototype of the Distributed ICS. It will contain the fully functional GUI and therefore a multi-document-interface. Finally, it will be possible to change the applied filter during run-time.

On the server side, the final prototype will monitor the message distribution mechanism and keep track of a DataFlow’s execution by writing special messages into the ‘meta-log’-database (which will be a file).



Review and clean up the code FS/MS 1.5 2

Extend the filter functionality (more generalized / on-the-fly changing / etc)

FS 1.5 1

Add monitoring to the buffer mechanism of the distribution module.

MS 0.5 0.25

Extend the user-DB to be able to set administrator’s permissions MS 0.5 0.25

Introduce the meta-log FS/MS 0.5 0.25

Make the GUI (even) more user friendly F/M 1 2.5

Total 5.5 6.25


Page 10 of 10

5.8 Time Management

We will maintain a time sheet where we write down the hours we work on this project. You may find these documents (Microsoft Excel format) on the CD. The following table lists a summary of the total times of the different topics:

Topic Result [hours]

Understanding ICS 27:17 Requirement Analysis 34:17 Requirement Specification 50:53 Software Design 19:37 Implementation 269:11 Test 54:55 Documentation 120:30 Miscellaneous 27:28 Total 604:10

Table 1 Time Management

Software Requirements Specification

Dis

trib

ute

d IC

S D

istr

ibu

ted

ICS



DATE 27-Oct-2001 VERSION 1.5 FILE Requirements Specification 1.4.doc

Version history:


1.0 First Draft F. Soldati, M. Seelhofer 14-Sep-2001

1.1 Concretized requirements F. Soldati, M. Seelhofer 17-Sep-2001

1.2 Adjusted Use Cases F. Soldati, M. Seelhofer 18-Sep-2001

1.3 Revised document F. Soldati, M. Seelhofer 20-Sep-2001

1.4 Adjusted structure to IEEE 830 M. Seelhofer 21-Sep-2001

1.5 Marked fulfilled requirements F. Soldati, M. Seelhofer 27-Oct-2001

This document conforms to the international IEEE standard 830, “Recommended Practice for Software Requirements Specifications”.

Table of contents

Chapter 1 Introduction 1 1.1 Purpose 1 1.2 Scope 1 1.3 Definitions, Acronyms and Abbreviations 1 1.4 References 2 1.5 Overview 3

Chapter 2 General Description 4 2.1 Product Perspective 4

2.1.1 What is ICS (Inter Connection Server)? 4

2.1.2 The starting point 5

2.2 Product Function 5 2.3 User Characteristics 6 2.4 General Considerations 6 2.5 Assumptions and Dependencies 6

Chapter 3 Specific Requirements 7 3.1 Functional Requirements 8

3.1.1 Inputs 8

3.1.2 Operations 8

3.1.3 Outputs 9

3.1.4 Error Handling 10

3.2 External Interface Requirements 10 3.2.1 User Interfaces 10

3.2.2 Hardware Interfaces 11

3.2.3 Software Interfaces 11

3.2.4 Communication Interfaces 11

3.3 Performance Requirements 12 3.3.1 System Load Factors 12

3.3.2 Database Factors 12

3.3.3 Response Factors 12

3.3.4 Resource Utilization 12

3.4 Design Constraints 13 3.4.1 Hardware Environmental Requirements 13

3.4.2 Software Environmental Requirements 13

3.5 Quality Requirements 13 3.5.1 Reliability 13

Software Requirements Specification Table of Contents

3.5.2 Availability 13

3.5.3 Security 14

3.5.4 Maintainability 14

3.5.5 Portability 14

3.5.6 Standards 14

Chapter 4 Use Cases 15 4.1 Use cases for audit-clients 15

4.1.1 Login 15

4.1.2 Logout 15

4.1.3 Start a dataflow 16

4.1.4 Stop a dataflow 16

4.1.5 Abort a dataflow 16

4.1.6 Receive list of available dataflows 17

4.1.7 Get status of dataflows 17

4.1.8 Subscribe “Dataflow Log-Messages” 18

4.1.9 Unsubscribe “Dataflow Log-Messages” 18

4.1.10 Receive log message 19

4.1.11 Set Filter 19

4.2 Use Case Diagram 20

Appendix A Glossary 21

Page 1 of 21


1.1 Purpose

This document defines the requirements to the piece of software produced in the project Distributed ICS. The resulting prototype will be used in conjunction with SunGard’s main integration tool, the Inter Connection Server, which is used to connect arbitrary data storage systems.

The software to be developed extends ICS’ functionality with log-message distribution. The goal is to be able to view log-messages produced by the ICS-internals through a graphical user interface started on a remote host.

1.2 Scope

This project will result in a prototype of the client-/server software ‘Distributed ICS’, which will be built around the existing Inter Connection Server (ICS). Care has to be taken to make only minimal adjustments to the existing code.

1.3 Definitions, Acronyms and Abbreviations

In order to entirely understand this document, some expressions must be explained in detail. The following table lists some of the most essential ones:

Expression Meaning Administrator A person using the audit-client to monitor open connections of other users. Auditing Ability to watch the ICS-server produce messages as they are generated. Audit-Client A remote interface (console or graphical) to do the auditing. Authentication The process of identifying a user and granting access to him. A username and

password must be supplied in some kind of log-in procedure. Dataflow A data-conversion process. Usually done by reading from a certain source

(e.g. from a file or database), then transforming the data (e.g. doing some calculations) and afterwards writing the results into some destination (e.g. into a file or database):

source destinationtransformation

Software Requirements Specification Chapter 1: Introduction

Page 2 of 21

Example: A text file consisting of records containing numbers in the first column and the text “US-$” in the second column must be converted to the currency CHF. This is done with a transformation, which calculates the amount of CHF by multiplying the amount of US-$ with the fixed number 1.72. The result must be written into an MS-SQL-database:

Text-file

MS-SQL-DB

12.5 ; US-$ 21.5 ; CHF

1 US-$ 1.72 CHF

Filtering An audit-client receives only those messages, which meet a certain criteria.

Only attribute values can be used as criteria. Some examples are shown here: Examples of filter-criteria: Criteria Meaning

Component=“FilePeer:in” Show messages produced by FilePeer components (direction: input) only

Method=”write” Show all messages produced by ‘write’-methods (any component)

Type=”Error” Show error messages only

Filters are generally not adjustable (pre-defined)!

ICS Stands for Inter Connection Server, SunGard’s middleware used for integration pourposes at customer sites.

Message An object produced by the message-log-system. Currently, ICS supplies information about the Dataflow, component, method, type (notification level) and the message text. A timestamp will be introduced in the extended message logging system: Examples of log-messages: Dataflow Component Method Type Text [Timestamp]

TestFlow FilePeer:out write Message Record [35] successfully written 20.09.01 16:04:12

TestFlow FilePeer:in read Error … 20.09.01 16:04:27

Meta-Logging Writing all important or exceptional activities noticed by the system (logins,

errors, warnings, …) into a separate log-file. Standard user A person using the audit-client to view log messages produced by Dataflows.

1.4 References

The following documents are part of the same bundle:

• Project Management Plan

• Software Design Description

Software Requirements Specification Chapter 1: Introduction

Page 3 of 21

1.5 Overview

In the next chapter, you’ll find an introductory description of what this project is about. It describes the goal of it, the function range of the existing component framework (the Inter Connection Server a.k.a. ICS) and the idea behind the extension of it (which is subject to this project).

Chapter 3 lists all the requirements in detail. Chapter 4 shows the most important use-cases and the corresponding use-case diagrams. Appendix A contains a Glossary.

Software Requirements Specification Chapter 2: General Description

Page 4 of 21

Chapter 2 General Description

2.1 Product Perspective

The main goal of this project is to develop a client/server architecture, which allows users to remotely view log-messages produced by the Inter Connection Server. In this component framework, processes known as ‘Dataflows’ generate log-messages. These messages should be delivered to the auditing clients. The system should also contain a simple user management database and an access control facility.

2.1.1 What is ICS (Inter Connection Server)?

To develop the Distributed ICS it is important to understand what ICS is and what it does. ICS is a software toolkit, an “adapter framework,” written 100% in Java 2. The intent of the adapter framework is to simplify integration among different systems that would otherwise find it difficult to share data.

It is common in large organizations to have requirements to connect many different systems so that they can pass information between one another. This typically requires a dedicated “point-to-point” data link, and customized programming to allow the systems to “talk” to one another.

Because most of these companies have a very heterogeneous system, the ICS must be very flexible in configuration.

ICS addresses this problem by creating an ”application plumbing” solution, which automates much of the custom coding that is required for system integration. It accomplishes this by providing developers with a set of standard components, which can be used to link systems-to-systems or systems-to-middleware. The structure of such a link is referred to as ‘Dataflow’ and can be stored onto disk by specifying a name for it.

ICS currently provides a set of standard components for data-access:

• JDBC (MSSQL, Sybase, Oracle, DB2, etc.) • Flat Files • IBM MQ Series • Transformation • Filtering • Routing • Generic Error Handling • Dynamic Message Logging System (MLS)


Page 5 of 21

2.1.2 The starting point

A basic message-logging system does already exist in the current version of ICS. It is realized with a single module collecting all the messages from the different components/objects attached to the defined “Dataflows”. These messages are currently written to standard out and – optionally – into any kind of data storage for which appropriate Peers (data access objects) exist. In the current version, users are already able to control Dataflows (start, stop and abort) through a simple remote interface:

controls Dataflow A controls Dataflow B

User A User BControl Clients

- Control Dataflows- Poll for Status

ICS Server

11.01.2001 13:40 debug11.01.2001 13:41 info11.01.2001 13:4211.01.2001 13:43 debug11.01.2001 13:44 debug11.01.2001 13:45 warn

Figure 1 Current logging system

2.2 Product Function

The existing message logging system must be extended to provide additional functionality: Clients residing on remote computers must be able to view the log-messages produced inside ICS. Usually, only log-messages produced by Dataflows started by the same user should be displayed. But in order to supply administrators with a more powerful tool, these must be able to view any log-messages produced.

A crucial requirement is to leave the core ICS-classes untouched in order to keep the already established and widely used functionality running and not to introduce bugs in the currently stable component-framework.


Page 6 of 21

The following figure shows an overview of the message logging system to be achieved:

controls Dataflow A

controls Dataflow B

User AUser B

Audit Clients

- Control Dataflows- Control Logging- Poll for Status

Management Server


controls all Dataflows

Adm inistrator



Figure 2 Extended logging system

2.3 User Characteristics

In this project, we will distinguish standard users from administrators. While a standard user will be considered to have no or only poor technical background (business people, no offence), an administrator will be considered being familiar with at least the basic concepts of the Inter Connection Server. This distinction will become apparent on some functions of the resulting prototype.

2.4 General Considerations

Since this project is carried out inside the framework of a diploma thesis, the main goal is only to fulfill all the must-have requirements and not specifically to integrate as many of the optional features as possible. However, some of those will still be considered achievable.

The software- as well as the coding- and documentation-language to be used will be (American) English.

2.5 Assumptions and Dependencies

Since this project will be developed around the existing ICS, the programming language and environment to be used will be Java 2 or the Java Development Kit 1.3.1, respectively.

Since Java can generally be used in a platform-independent way, the extension to the existing ICS (the subject of this project) is not allowed to use platform specific features.

Software Requirements Specification Chapter 3: Specific Requirements

Page 7 of 21

Chapter 3 Specific Requirements

The MLS Project is divided into three subsystems:

• a management server, which is responsible for handling connections to remote audit-clients and for requests to start, stop and abort Dataflows.

• a log-data distribution module, which is responsible for collecting all log messages produced by all the Dataflows and to distribute these messages to the appropriate audit clients. It exposes a (user-defined-) filter-mechanism to deliver messages of the requested types only.

• an audit client, which allows users to display log-messages produced by Dataflows remotely. This can be done through a nice and easy-to-use GUI or alternatively through a simple command line interface.

User m anagem ent& adm ission control

Dataflowsm anagem ent

System m onitoring

Audit-Client 1 Audit-Client 2 Audit-Client n...

Log-data distribution m odule

Management server

Log configuration /Filters

Figure 3 Subsystem overview

Note: Requirements of must-have nature are marked with the letter ‘M’ on the right side, optional ones are marked with an ‘O’. Requirements which were met are marked with the symbol ‘!’ while others are marked with the symbol ‘"’.

Examples:

Start Dataflows M !

Keyed passwords O "


Page 8 of 21

3.1 Functional Requirements

3.1.1 Inputs

Requirements: • The input to the system consists of the log-messages produced by ICS. (see section 1.3 for details on the message format)

M !

3.1.2 Operations

Authorization

Description: Because the ICS runs within a network, the ICS management system needs some kind of authorization mechanism. Users can send commands (starting / stopping / aborting Dataflows etc.) to the ICS management system, but not before successful login and not after logout.

Requirements: • Add/remove users to/from the database (locally) M !

• Assign administrator’s permissions to an existing user (locally) M !

• Revoke administrator’s permissions from an existing user (locally) M !

• Add/remove users to/from the database (remotely) O "

• Assign administrator’s permissions to an existing user (remotely) O "

• Revoke administrator’s permissions from an existing user (remotely)

O "

• Encrypted passwords O !

• Log-in by specifying a user name and password M !

• Log-out M !

Dataflow management Description: The Dataflow management module has to communicate with the clients,

which have the possibility to start, stop and abort Dataflows and to connect to Dataflows, which they previously started.

Requirements: • Start Dataflows M !

• Stop running Dataflows M !

• Abort running Dataflows M !

• Connect to running Dataflows, started by the same user M !


Page 9 of 21

3.1.3 Outputs

Filtering messages Description: The management server must provide server-side filters and must therefore be

able to send only those log-messages to the client, which fit the filter criteria.

The filter function must be designed in a way that allows users to view messages of specific type only. E.g. receive error messages only, receive messages produced by ICS-components (such as FilePeers) only or receive messages produced by specific methods (e.g. put-methods) only. To describe it in a more general way, it must be possible to filter the log-messages by a certain attribute-value. Currently, the attributes of the log-messages are:

Dataflow the name of the Dataflow which produced the message

Component the ICS-component which produced the message

Method the ICS-component’s method which produced the message

Type the notification level (‘message’, ‘trace’, ‘debug’, ‘system message’, ‘warning’ and ‘error’)

Text the message’s text (the actual message)

A timestamp must additionally be introduced.

Attribute values are e.g. “FilePeer:in” (component name), “debug” (type) or “Record [136] successfully written” (message).

Requirements: • Supply pre-defined filter-patterns (as described in chapter 1.3.) for log-message attribute values except the text-attribute.

M !

• Extend the filter-mechanism to handle the text-attribute, too. For this purpose, supply a ‘*’ to be used to let users define their own filter-patterns. Like that, a user can write e.g. “Record [*]*” which means that any messages starting with the text “Record [“ then containing any kind of string followed by a “]” should be retrieved.

O !

Configuration of the log-data distribution module

Description: Since the ICS management server must provide (server-side) filters, it must therefore be able to send only those log-messages to the client, which fit the filter criteria.

Requirements: • Per-Dataflow filters. A user must be able to turn on different filters for different Dataflows / Dataflow-views.

M !

• Server-side filters (filters must act on the server-side and prevent messages not meeting the criteria from being delivered)

M !

• Save/Load filter settings on the client side O !

Keep part of history Description: A nice extension of the audit-functionality would be to keep some messages

persistent for a while. This would allow identifying where problems occurred during crashed or erroneous Dataflows. However, this functionality is entirely optional.

Requirements: • Request and view the last 10’000 messages produced. O "


Page 10 of 21

3.1.4 Error Handling

Description: The Distributed ICS server must be able to deal with some exceptional situations. This includes a mechanism to respond to broken connections, report low system resources by sending out warnings and send emails to administrators when serious problems occur.

Requirements: • Keep log of every important or exceptional activity inside the management server (login attempts, broken connections etc.) inside a so-called ‘meta-log’

M !

• Possibility to view meta-log messages (text file) M !

• Inform administrators about errors by email O "

• Send warnings to audit-clients when problems are likely to occur shortly

O !

• 3-time reconnection-mechanism (without user’s interaction). If a connection between an audit-client and the Dataflow, sending log-messages to it, breaks, the system must 3 times try to reconnect while the execution of the corresponding Dataflow does NOT stop.

O "

3.2 External Interface Requirements

3.2.1 User Interfaces

General functions

Description: The user interface must provide login functionality. Therefore, a user must be able to specify a user-id and a password. This pair is sent to the management server, which accepts / rejects the login request.

Additionally, the (already implemented) functions for starting, stopping and aborting Dataflows must be available through the remote interface.

Requirements: • Required to send username and password to the server before starting to work with Dataflows

M !

• Possibility to start, stop and abort a dataflow M !

• Possibility to choose a filter (before starting a dataflow), which is sent to the server. The filter must be applied on server-side.

M !

• Possibility to choose a filter while a Dataflow is running M !

• Possibility to store/load user-defined filter-patterns O !

• Clients can start multiple Dataflows concurrently M !

Console interface Description: A subset of the above functions must be accessible through a command line

interface.

Requirements: • Possibility to start, stop and abort Dataflows M !

• Possibility to specify a filter before connecting to or starting a flow M !


Page 11 of 21

GUI (Graphical User Interface)

Description: A nice and easy-to-use graphical user interface must be implemented.

Requirements: • Possibility to start, stop and abort a dataflow M !

• GUI follows the „Java Look & Feel” design guidelines M !

• Possibility to choose a filter. The filter must be sent to the server and applied on the message logging system

M !

• Possibility to store/load user-defined filter-patterns O !

• Messages from different Dataflows are displayed in separate sub-windows of the GUI

M !

• By default, messages are sorted in chronological order M !

• Possibility to sort messages in ascending/descending order according to each column (= attributes of Java-class Message)

O "

• Possibility to show some statistics about the message traffic O "

• Possibility to find a message according to a user-defined string O "

• The following items must be displayed on the client side: messages (with all their attributes) / the monitored Dataflow’s status

M !

• The following items must be made available (displayed on request) on the client side: list of available Dataflows / list of all running Dataflows (administrators) / list of running Dataflows, started by the same user (standard users)

M !

3.2.2 Hardware Interfaces

No hardware interface must be taken into account.

3.2.3 Software Interfaces

No specific API (application programming interface) must be developed, since as long as the project is implemented in plain Java, it is a very easy task to extend its functionality.

3.2.4 Communication Interfaces

Apart from the remote user interface, no additional communication interface must be considered.


Page 12 of 21

3.3 Performance Requirements

3.3.1 System Load Factors

Requirements: • Handle at most 5 concurrent clients M !

3.3.2 Database Factors

Description: To provide the authorization function, simple user management functionality is needed, which must be realized with a small database containing name, password and permission entries. These entries are used to distinguish normal users from administrators.

Requirements: • The system shall provide a simple user management database (may even be a file)

M !

• Add/remove users to/from the database (locally) M !

• Assign administrator’s permissions to an existing user (locally) M !

• Revoke administrator’s permissions from an existing user (locally) M !

• Add/remove users to/from the database (remotely) O "

• Assign administrator’s permissions to an existing user (remotely) O "

• Revoke administrator’s permissions from an existing user (remotely)

O "

3.3.3 Response Factors

Requirements: • Under normal conditions (no network congestion, no network error, no software crash of any kind, no broken connections, etc.): guarantee delivery of the first log-message to the client in less than 1 second after creation. Under heavy load (especially when messages are produced in bursts # e.g. trace-messages), messages may be delivered in more than 1 second. This is usually the case, if the production rate becomes greater than the network speed (available bandwidth).

M !

3.3.4 Resource Utilization

No disk utilization or memory constraints must be met.


Page 13 of 21

3.4 Design Constraints

3.4.1 Hardware Environmental Requirements

Description: This system will be running on a TCP/IP-based network. While the main purpose of it will be to run inside a local area network, there may be the request to use it through e.g. an ISDN-connection in the future (will NOT be considered in this version). Therefore, already in this version, not more bandwidth than needed should be consumed.

Requirements: • Run on an TCP/IP-based network M !

• Not more bandwidth than needed should be consumed. But in order not to make the system unnecessarily complex and difficult to implement, data compression and the use of multicast technology is NOT required.

M !

3.4.2 Software Environmental Requirements

Description: Since this system is thought of as an extension to the already well functioning and widely used Inter Connection Server, special care must be taken to keep the system (existing ICS and the extended Message Logging module) as understandable, extendable and maintainable as possible.

Requirements: • The core components of the Inter Connection Server must be left untouched.

M !

• The chosen class-hierarchy / package-structure must be designed in a way that leaves the door open for future changes in the message-transport technology (log-messages from the server to the audit-clients).

M !

3.5 Quality Requirements

3.5.1 Reliability

Requirements: • Guaranteed message delivery under normal conditions (no network error, no software crash of any kind, no broken connections, no unusual server load etc.).

M !

• Guaranteed order of message-arrival M !

3.5.2 Availability

See section 3.1.2, Operations.


Page 14 of 21

3.5.3 Security

Requirements: • The user management database shall use encryption before storing passwords.

O !

3.5.4 Maintainability

See section 3.4.2, Software Environmental Requirements.

3.5.5 Portability

No additional precautions will be considered, since Java applications are generally platform-independent.

3.5.6 Standards

Requirements: • The written Java-code must be documented using the JavaDoc-convention

M !

• Unit tests must be implemented using the testing framework ‘JUnit’ O !

Software Requirements Specification Chapter 4: Use Cases

Page 15 of 21

Chapter 4 Use Cases

4.1 Use cases for audit-clients

4.1.1 Login

Use Case Name / Number: Login Trigger: User Further concerned triggers: Purpose / Objective: Every user has to authenticate himself General description: The client has to authenticate himself by sending a

username/password combination to the ICS manager. Category (importance): Primary General order of events: 1. Login dialog with “Ok/Cancel” button

2. send user/pwd to ICS manager 3. ICS manager checks user/pwd 4. ICS manager accepts/reject user 5. connection established

Alternative 2: Errors: - User not exists in list

- Wrong password Pre-condition: Not connected Post-condition: Connected Remarks: Different privileges for Admins/Users need special care. Open Questions: Security of the transmission (RC5, SHA)

Automatic server discovery ?

4.1.2 Logout

Use Case Name / Number: Logout Trigger: User Further concerned triggers: When the client terminates Purpose / Objective: The user has to logout after a session General description: The client disconnects himself from the ICS manager.

Additional communication with the server can only be archived by a new login process.

Category (importance): Primary General order of events: 1. Client sends logout command to the ICS manager

2. ICS manager checks in user management for the user id 3. User will be removed from user management list

Alternative 2: Errors: - User not exists Pre-condition: Connected Post-condition: Disconnected Remarks: Open Questions:


Page 16 of 21

4.1.3 Start a dataflow

Use Case Name / Number: Start a dataflow Trigger: User Further concerned triggers: Purpose / Objective: The user should be able to start an arbitrary dataflow. General description: Category (importance): Primary General order of events: 1. Client orders to start a dataflow to the ICS manager. Alternative 2: Errors: - Start dataflow failed

- Not connected / not authorized Pre-condition: Connected, Name of dataflow is known Post-condition: Dataflow starts running Remarks: Open Questions:

4.1.4 Stop a dataflow

Use Case Name / Number: Stop a dataflow Trigger: User Further concerned triggers: Purpose / Objective: The user should be able to stop ‘his’ dataflow. General description: Stop means a regular shutdown. Category (importance): Primary General order of events: 1. Client orders to stop ‘his’ dataflow to the ICS manager. Alternative 2: Errors: - Stop dataflow failed / Dataflow is already stopped

- Not connected / not authorized Pre-condition: Connected, Dataflow is running Post-condition: Dataflow stops running Remarks: Open Questions:

4.1.5 Abort a dataflow

Use Case Name / Number: Abort a dataflow Trigger: User Further concerned triggers: Purpose / Objective: The user should be able to abort ‘his’ dataflow. General description: Abort means that every thread will be killed immediately Category (importance): Primary General order of events: 1. Client orders to abort ‘his’ dataflow to the ICS manager. Alternative 2: Errors: - Abort dataflow failed / Dataflow is already stopped

- Not connected / not authorized Pre-condition: Connected, Dataflow is running Post-condition: Dataflow aborts running Remarks: Open Questions:


Page 17 of 21

4.1.6 Receive list of available dataflows

Use Case Name / Number: Receive list of available dataflows Trigger: User Further concerned triggers: Polling from the client. (Because we only use one-way

communication between the client and the ICS-manager, the client must ask the server for new states periodically)

Purpose / Objective: The user should be able to view a list of all available dataflows. General description: Category (importance): Primary General order of events: 1. Client asks the ICS manager for the list of available

dataflows 2. ICS manager checks if the user is logged-in correctly. 3. ICS manager sends this list to the client

Alternative 2: Errors: - User not exists

- Not connected / not authorized Pre-condition: Connected Post-condition: Remarks: Because the list of available dataflows is statically, this list must

only be received on startup. Open Questions:

4.1.7 Get status of dataflows

Use Case Name / Number: Get status of dataflows Trigger: User Further concerned triggers: Purpose / Objective: The user should be able to view the status of his dataflow. General description: I.e. the user wants know if ‘his’ dataflow is running, stopped, … Category (importance): Primary General order of events: 1. Client asks ICS manager for a status report. Alternative 2: Errors: - Not connected / not authorized Pre-condition: Connected Post-condition: Remarks: The administrator will get a list of all dataflows. Open Questions:


Page 18 of 21

4.1.8 Subscribe “Dataflow Log-Messages”

Use Case Name / Number: Subscribe “Dataflow Log-Messages” Trigger: User Further concerned triggers: Purpose / Objective: The user has the possibility to subscribe dataflow log-messages. General description: To view all the logging data the user-client must have one (or

more if he has administrator’s permissions) personal log-message writer. This writer is connected to the singleton message logging class in an ICS process.

Category (importance): Primary General order of events: 1. Client informs the ICS manager about the wanted dataflow

2. Server creates on the current dataflow process a new log-message writer with a link to the asking client

Alternative 2: Errors: - Dataflow not exists

- Client is not allowed to view this data Pre-condition: Connected

Running Dataflow Post-condition: New log-message writer Remarks: Open Questions:

4.1.9 Unsubscribe “Dataflow Log-Messages”

Use Case Name / Number: Unsubscribe “Dataflow Log-Messages” Trigger: User Further concerned triggers: On closing the client Purpose / Objective: The user has the possibility to unsubscribe dataflow log-

messages. General description: Category (importance): Primary General order of events: 1. Client asks the ICS manager for unsubscribing messages

2. Server removes the log-message writer. Alternative 2: Errors: - Dataflow not exists

- Not subscribed Pre-condition: Connected

Running Dataflow Subscribed on the current dataflow

Post-condition: Log-message writer removed Remarks: Open Questions:


Page 19 of 21

4.1.10 Receive log message

Use Case Name / Number: Receive log message Trigger: Logging Module Further concerned triggers: Purpose / Objective: On new messages from the ICS, the Logging Module has to

distribute all messages to its corresponding clients. General description: Category (importance): Primary General order of events: 1. Logging Module gets all clients who are subscribed to the

actual dataflow. 2. Logging Module sends the message to all this clients

Alternative 2: Errors: Pre-condition: New messages from the MessageLogging Class Post-condition: Message processed Remarks: Open Questions:

4.1.11 Set Filter

Use Case Name / Number: Set Filter Trigger: User Further concerned triggers: Purpose / Objective: The user is able to set different filters on a dataflow General description: e.g. only error messages will be displayed or just messages of a

certain component Category (importance): Primary General order of events: 1. Client sets a new filter. IMS manager must inform the

corresponding log-message writer about that. Alternative 2: Errors: - Illegal Filter Format Pre-condition: Connected

Running Dataflow Subscribed on the current dataflow

Post-condition: New Filter Remarks: Open Questions:


Page 20 of 21

4.2 Use Case Diagram

User

UseCase1

Login

UseCase1

Logout

UseCase1

Start dataflow

UseCase1

Stop dataflowUseCase1

Abort dataflow

UseCase1

Get status of a dataflow

UseCase1

Receive list of dataflows

UseCase1

Subscribe"Dataflow Log-Messages"

UseCase1

Unsubscribe"Dataflow Log-Messages"

UseCase1

Set filter

UseCase1

Receive m essage

Figure 4 Use Case Diagram

Page 21 of 21

Appendix A Glossary

Access-Control An Audit-Client gains access to the server through a login procedure.

Auditing A user should be able to view messages produced by the ICS as they are generated.

Authentication A user must identify himself by a username/ password dialog.

Compression The MLS may have to support data compression depending on the amount of data.

Configuration Client:

Congestion-Control The server may monitor the load of the network/server. Console A command-line interface should be supplied. Error-Recovery The server should be able to handle broken connections. Error-Reporting The MLS should be able to send error-reports by email. Filtering The Client has the possibility to filter messages (e.g. only

Error-messages may be displayed) GUI A functional and easy-to-use GUI should be supplied. History It should be possible to view old log-data. Meta-Logging All important or exceptional activities (each logins,

errors, warnings, …) must be written into a separate log-file. In this document, this mechanism is referred to as ‘Meta-Logging’.

Multicast To avoid potential bottlenecks (like high network traffic), IP-multicast should be considered.

Performance Through the use of multithreading the highest possible performance should be achieved.

Persistence The client should be able to partially store received data. Privacy It should be possible to restrict a user’s view to a certain

level. (e.g. messages produced by his own dataflow) (Soft-)Real-Time Messages produced by the ICS should be delivered to the

audit-client with only little delay. Scalability In this first approach a maximum number of 5 clients

will be served. Special care has to be taken while designing the MLS to be able to easily increase this number in future versions.

User-Management A simple user-management for all audit-clients must be maintained.

View A user should have the possibility to change the view of the data.

Software Design Description



DATE 28-Oct-2001 VERSION 1.5 FILE Software Design Description 1.5.doc

Dis

trib

ute

d IC

S

Version history: Version Description Authors Date 1.0 First Draft F. Soldati, M. Seelhofer 20-Sep-2001 1.1 Added Port Allocation / Flow of data F. Soldati 24-Sep-2001 1.2 Added package descriptions F. Soldati, M. Seelhofer 17-Okt-2001 1.3 Adjusted doc-structure to IEEE 1016 F. Soldati, M. Seelhofer 23-Okt-2001 1.4 Additional sections in chapter 7 F. Soldati, M. Seelhofer 25-Okt-2001 1.5 Final version F. Soldati, M.Seelhofer 27-Okt-2001

This document conforms to the international IEEE standard 1016, “Recommended Practice for Software Design Descriptions”.

Table of contents

Chapter 1 Introduction 1 1.1 Purpose 2 1.2 Scope 2 1.3 Definitions, Acronyms and Abbreviations 3 1.4 References 4 1.5 Overview 4

Chapter 2 System Overview 5

Chapter 3 Design Considerations 6 3.1 Assumptions and Dependencies 7 3.2 General Constraints 7 3.3 Goals and Guidelines 8 3.4 Development Methods 8

3.4.1 eXtreme Programming 9

Chapter 4 Decomposition Description 10 4.1 Server 12

4.1.1 The „ID problem“ of ICS-Dataflows 12 4.1.2 Module decomposition 14

4.2 Dataflow Process 14 4.2.1 Module decomposition 14

4.3 Audit-Client 15 4.3.1 Module decomposition 15

Chapter 5 Dependency Description 16 5.1 Intermodule Dependencies 17

5.1.1 Flow of data 17 5.1.2 Port allocation 19 5.1.3 Distributing Log-messages 20

5.2 Interprocess Dependencies 21 5.3 Data Dependencies 21

5.3.1 Connection Information 21 5.3.2 Dataflow identification 22

Chapter 6 Interface Description 23 6.1 Network architecture 24

6.1.1 CORBA (Common Object Request Broker Architecture) 24 6.1.2 RMI (Remote Message Invocation) 25 6.1.3 JMS (Java Message Service) 25 6.1.4 Sockets 26 6.1.5 JSDT (Java Sharing Data Toolkit) 26 6.1.6 Conclusion 27

Software Design Description Table of Contents

6.2 Multicast? 27 6.3 Defining RMI-interfaces 28

6.3.1 What is RMI 28 6.3.2 How to use RMI 29 6.3.3 RMI object identification 30

6.4 The interfaces used 31 6.4.1 The interface ‘IManagementServer’ 31 6.4.2 The interface ‘IProcessManager’ 34 6.4.3 The interface ‘IDataFlowManager’ 35 6.4.4 The interface ‘ILogMessageReceiver’ 35

Chapter 7 Detailed System Design 36 7.1 Package structure 37 7.2 The package ‘audit’ 38

7.2.1 Model-View-Controller (MVC) 39 7.2.2 Menu items with attached Java Actions 40 7.2.3 Multi document support 41 7.2.4 Treeview 42 7.2.5 Internationalization (i18n) 43 7.2.6 Configuration 45

7.3 The package ‘logging’ 46 7.3.1 The class ‘DistributedMessageLogging’ 46 7.3.2 The class ‘LogListenerModule’ 48 7.3.3 Filtering 49

7.4 The package ‘management’ 50 7.4.1 The class ‚ManagementServer’ 51 7.4.2 The class ‚UserManager’ 52 7.4.3 The class ‚ConnectionManager’ 53 7.4.4 The class ‚ProcessManager’ 53 7.4.5 The class ‘DataFlowManager’ 55

7.5 The package ‘test’ 56 7.6 The package ‘util’ 57 7.7 Performance issues 58

Appendix A Bibliography 60

Appendix B Index 61

Page 1 of 62


1.1 Purpose 2

1.2 Scope 2

1.3 Definitions, Acronyms and Abbreviations 3

1.4 References 4

1.5 Overview 4

Software Design Description Chapter 1: Introduction

Page 2 of 62

1.1 Purpose

The purpose of this document is to show how the project Distributed ICS (or D-ICS in its short form) is structured. While it contains low-level details of each module later in the document, the main (and maybe the most frequently read) part of it contains a structural overview of all subsystems, interfaces and data. Therefore, system engineers as well as project managers or even end-users might be interested in reading this document. To know what we are going to describe, let’s recall what this system is all about: The resulting piece of software, the Distributed-ICS will be used in conjunction with SunGard’s main integration tool, the Inter Connection Server, which is used to connect arbitrary data storage systems. D-ICS extends ICS’ functionality with log-message distribution. With a graphical user interface, it is possible to remotely view log-messages produced by the ICS-internals. Note: This document is written in general conformance with the IEEE Standard 1016, IEEE

Recommended Practice for Software Design Descriptions. One of the major features of this standard is the presentation of the system's design from different perspectives or views: While a manager may be mainly interested in the overall design and solutions to certain problems, a software engineer must be provided with detailed information about how a specific feature is implemented in order to be able to easily maintain and adjust the system. However, we have chosen not to include the system-user’s point of view into this document. Since we expect it to be asked for much more frequently than the rest, we extracted it into a separate document, the “installation guide & user’s manual”.

In contrast to the “Software Requirements Specification” (which is primarily written for the customer and the user and tells what the system must do), most of this document contains information for software professionals, such as system designers, testers, reviewers and the product support team (will there be any?). It contains a detailed description of how the system is implemented.

1.2 Scope

The project’s result is a CD with the following contents:

• Java-Class files needed to run the client application

• Java-Class files needed to run the server side components

• Java-sources for the above

• Setup script to install a pre-configured system for demonstration and test purposes

• JavaDoc-style documentation of the source code in HTML-format

• A running version (minimally adjusted) of SunGard’s existing ICS, checked out of the repository October 10, 2001.

• All the relevant documents (project management, requirements, this document, …)

• A cool Autostart-Menu


Page 3 of 62

1.3 Definitions, Acronyms and Abbreviations

Expression Meaning API Application Programming Interface. Basically a library of functions (or

classes in an OO-environment). Client An application or object using the functionality of a server. Dataflow* A data-conversion process. Usually done by reading from a certain source,

then transforming the data and afterwards writing the results into some destination. Not to be mixed up with the ‘flow of data’, the way data is passed throughout the network!

Dataflow Process A process in which a Dataflow is running. Filter A collection of different criteria used to select the log-messages of interest

to a user. IPC Stands for Inter Process Communication. Will be done by using the RMI

mechanism of Java (high-level point of view) or network sockets (low-level point of view), respectively.

JVM Stands for Java Virtual Machine, which is the application (the interpreter) to run java programs. Runs with all its Java-Objects and Threads inside one process. May be executed several times independently.

Message digest The result of the computation of a value uniquely identifying a document. Also known as the term ‘hash value’.

Message level* The type of a message: Message, Info, Debug, Trace, Error. Sometimes also referred to as ‘Message type’.

Port A TCP-port. An integer used to distinguish running network applications. Polling In the context of this document, the term ‘polling’ has nothing to do with

politics. It is used for a process of calling a certain method regularly and thereby testing for some condition.

Process An independently running computing operation, sometimes also called a running software application.

Reference Java’s counterpart to pointers. Think of it as a ‘safe and convenient pointer’ which needs no indirection (no *-characters as in e.g. C/C++) to access the object pointed to.

RMI Java’s Remote Method Invocation-mechanism which is essentially the Java-version of Remote Procedure Calls.

Server Definition 1: An application or object providing some functionality which may be accessed over a network.

Definition 2: A computer on which a server application (according to Definition 1) is running.

Server-Process The process in which the main server-side functionality is running. Singleton class A class which is instantiated exactly once inside each process.

* For further information refer to the document ‘Software Requirements Specification’.


Page 4 of 62

1.4 References

For readers not familiar with this project, we recommend reading the following documents:

• Software Requirements Specification

• Project Management Plan Distributed ICS

1.5 Overview

In the following chapter, you’ll find an introductory description of how the system is implemented. It describes its design in a rather rough manner and does not include details since it is primarily thought of to give you an overview. Chapter 3 contains information about our assumptions and the dependencies on other pieces of software like the already existing Inter Connection Server (ICS) developed at SunGard AG (Switzerland). It does also list the goals (of the development process, not the project), some guidelines and the development methods used. The following chapters contain more details about the system’s design such as a ‘Decomposition Description’ (Chapter 4), a ‘Dependency Description’ (Chapter 5) and an ‘Interface Description’ (Chapter 6) while still providing not too many low-level details of the system’s internals. The low-level details follow in chapter 7, in the ‘Detailed System Design’-section, which also contains a description of the package structure and some details on code-level.

Page 5 of 62

Chapter 2 System Overview

This chapter demonstrates how we broke down the system into parts and gives you a basic idea of how those parts interact. The point of view is a (very) high-level one:

Dataflow B

Log-ModuleAdm in

Log-ModuleB

Dataflow A

Log-ModuleAdm in

Log-ModuleA

Managem ent Server

Processm anagement

Userm anangem ent

Log configuration

Monitor activities

Audit ClientAdm inistrator

15:09 debug open all15:10 trace successf15:11 error write file15:12 error close file15:13 debug open all15:15 trace successf

controls Dataflow B

AuditClient B

15:09 debug open all15:10 trace successf

controls Dataflow A

AuditClient A

15:09 debug open all15:10 trace successf15:11 error write file15:12 error close file

Server

Server Process

Dataflow Process Dataflow Process


Figure 1: System’s Overview

Subsystem Description

Server Process

Controls all the server side components. Creates and monitors Dataflow processes. Provides a simple user database system to distinguish standard users (less privileges) from administrators (more privileges).

Serv

er-s

ide

Dataflow Process

Listens to commands sent by the server process. Able to send log-messages to a (remote) client. Does buffering to reduce the slow-down factor of low-bandwidth networks.

Clie

nt-s

ide

Audit-Client

Remote interface to control Dataflows and receive/display log messages. Differs between standard users and administrators. Provides a graphical user interface (GUI).

Page 6 of 62

Chapter 3 Design Considerations

3.1 Assumptions and Dependencies 7

3.2 General Constraints 7

3.3 Goals and Guidelines 8

3.4 Development Methods 8

Software Design Description Chapter 3: Design Considerations

Page 7 of 62

As a draft version of this document was written before the development phase, we tried to put some rules concerning the way we were going to act during that phase into it. Therefore, the information found in this chapter describes some of the issues which needed to be addressed or resolved before we attempted to devise a design solution and deals with what we wanted to achieve in a project manager’s point of view.

3.1 Assumptions and Dependencies

Since this system is thought of as an extension to SunGard’s existing component framework, the Inter Connection Server (ICS), it is somewhat dependent on ICS not to change too excessively. However, ICS is already in wide use and will be further developed independently in the future. Therefore, a main issue was (and still is) to provide a way to easily adjust a new version of the ICS to work together with the log-message distribution mechanism of D-ICS. The solution consists of a change of only two lines of code. This change relates to the Java class MessageLogging inside the package ICS.PD.Util. The method

public static synchronized MessageLogging getInstance() { return MessageLogging.getInstance(); }

must simply be changed to

public static synchronized DistributedMessageLogging getInstance() { return DistributedMessageLogging.getInstance(); }

in order to work with D-ICS. Additionally, the package where the class DistributedMessageLogging can be found must be imported:

import dics.logging.*; Please refer to chapter 7.3.1, The class ‘DistributedMessageLogging’ for details on the class DistributedMessageLogging. Note: As described above, this system is an extension to the existing ICS and therefore is not

able to run without it!

3.2 General Constraints

While this system will mainly be used on systems running Microsoft Windows, it should be able to execute on computers running Linux as well. Since it is developed using the programming language Java and its corresponding development kit (in version 1.3.1_01), it virtually is executable on any operating system for which a JVM exists.


Page 8 of 62

However, since problems may arise out of the use of different JVM-versions (they usually differ in the way serialization is applied to objects), we put the Java run-time environment (JRE) corresponding to the used JDK-version onto the project-CD. Note: A short (probably insufficient) test with SuSE Linux in version 7.2 showed no sign of

trouble. However, since the ability to run on a Linux-system wasn’t a requirement of must-have nature, we didn’t spend too much time on this.

3.3 Goals and Guidelines

Our goal was to achieve all the requirements marked with the letter ‘M’ (for ‘Must-have’) in the document ‘Software Requirements Specification’ and to probably implement some of the optional ones. Additionally, we set up the following guidelines:

Guideline Reason Readability > Performance Since the system will be used by different users and maintained

by different programmers than the ones who have implemented it, writing code which is easy to understand has greater precedence than finding best performing solutions.

Asynchronous > Synchronous To decouple the components of the system as good as possible, asynchronous operation modes should be chosen where applicable. However, decisions should not be made without keeping an eye on complexity…

Internals > User Interface An internally well designed system makes it easier to apply future changes. Therefore, priority lies mainly on the work behind the scenes and not on the user interface.

3.4 Development Methods

“Listening, Testing, Coding, Designing. That's all there is to software. Anyone who tells you different is selling something.”

Kent Beck Our customer (SunGard AG in Zurich, Switzerland) uses a controversial development method called ‘Extreme Programming (XP)’. This is the name Kent Beck has given to a lightweight development process that he has been evolving over the years.


Page 9 of 62

3.4.1 eXtreme Programming

XP is basically a very lightweight combination of practices to create a team that can rapidly produce extremely reliable, efficient and well-factored software. We are going to discuss some of its concepts, now:

Concepts we decided to adopt from XP

Concept Description Code the unit test first Creating unit tests before coding them helps a developer to

consider what really needs to be done. The time it takes to write a unit test and the code to make it pass is about the same as just to code it up straight away. But if you already have the unit tests, you don't need to create them after the code possibly saving you a lot of time.

Make frequent small releases The idea behind this is to give the customer as much feedback of a system’s functionality as possible. Since almost no system is ‘compatible with the customer’s needs’ in its first version, this concept allows to adjust it in a very early development state instead of possibly being forced to rewrite a large part of the system.

Integrate often Developers should integrate and release their code every few hours. This avoids diverging or fragmented development efforts. Comment: We tried to integrate our code at the end of

every day, which resulted in a running version (almost) every day of the development phase.

Concepts we decided NOT to or only partially adopt from XP

Concept Description Pair programming XP says, that “2 people working at a single computer will

add as much functionality as two working separately except that it will be much higher in quality”. Comment: We decided to only partially use this concept,

since writing simple code can easily be done by one person alone…

Stand up meetings A stand up meeting every day is used to communicate problems and their solutions to the whole development team. Comment: For a team of only two software engineers, we

spotted no clear advantage, since they can communicate as needed all the time.

Note: Refer to http://www.extremeprogramming.org for more details about XP.

Page 10 of 62

Chapter 4 Decomposition Description

4.1 Server 12

4.2 Dataflow Process 14

4.3 Audit-Client 15

Software Design Description Chapter 4: Decomposition Description

Page 11 of 62

As mentioned in the ‘System Overview’-section (Chapter 2), the system consists of three subsystems which in turn consist of several smaller modules. These will shortly be described. If you are looking for a more detailed discussion of the modules, you can refer to the ‘Detailed System Design’-section (Chapter 7). The following figure shows the subsystems’ decomposition into modules and their interaction. It also illustrates the flow of data and the directions of the communication paths throughout the network (indicated with arrowheads):

Processmanagement

Audit Client A

Audit C lient B

Audit Administrator

! Start/Stop Dataflow/Process! Add/Remove Logging Module" Configure Logging Module (filters)

W rite Log Messages

! Start / Stop Dataflows! set Log Filters

Producer

Consumer

setFilter

monitorBuffer

connectAudit-Client

Report Status

MessageLogging(Dataflow A)

ICS(Dataflow A)

LogMessage-W riter Admin

LogMessage-W riter A

Process A

LogMessage-W riter Admin

MessageLogging(Dataflow B)

LogMessage-W riter B

ICS(Dataflow B)

Process B

Log-datadistribution m odule

Buffer

Audit C lients

Usermanangement

Log configuration

M anagem entServer

Client-side

Monitor activities

Server-side

Figure 2: Main design overview Note: A client may open multiple windows on the same Dataflow which results in multiple

Log-Message Writers sending messages to the same client, one for each window.


Page 12 of 62

4.1 Server

The server-side subsystem processes requests sent using the remote user interface. It is also responsible for controlling the access-privileges of such requests, managing a simple user database and dealing with ICS-Dataflows. Additionally, it monitors the connections from clients and writes some log-messages (this is the Meta-Log function) about what is going on.

4.1.1 The „ID problem“ of ICS-Dataflows

In ICS, in the version we use, every component of a Dataflow writes its data to the same MessageLogging object which is realized using the design-pattern singleton:

MessageLogging(singleton)

DataFlow 1

DataFlow 2

Figure 3: Logging of messages to the singleton object ‘MessageLogging’

One of the goals of Distributed ICS is to distribute log-messages generated by ICS-components to all the auditing clients. To be able to do this, the originating Dataflow must be identifiable. Unfortunately, ICS’ design does not allow doing this directly. Although its components do their work in a very reliable manner, they do not know to which Dataflow they belong or in other words, for which Dataflow they carry out their work. It is therefore not possible to retrieve the Dataflow-name for a generated log-message without reasonable effort. As you can see, this “ID-problem” had to be solved very early in the development process since this was a highly critical problem which – if a solution could not have been developed without excessively stretching the project’s schedule – could even have spawned the end of this project! Fortunately, we were able to find some solutions to retrieve the Dataflow-name corresponding to a log-message generated by an ICS-component:

Solution 1 – get the name from the ICS component manager When an ICS-component sends a message to the logging object, it sends a reference to itself (the this-pointer) as well. Therefore, it is possible to uniquely identify every component sending log-messages. Since a Dataflow-object contains a list of references to all attached components, a solution to the “ID-problem” would be to compare this list with the reference to the message-originator.


Page 13 of 62

However, this solution would mean to deeply bind Distributed ICS to the existing system. Future changes would take more effort and the software would be harder to maintain. Therefore, this solution lands in the dustbin.

Solution 2 – run every dataflow in an own process A second (and better) solution is to run every Dataflow in an own process (in terms of operating system processes! See chapter 1.3 Definitions, Acronyms and Abbreviations) or in other words to run each Dataflow in its own JVM. With this solution, every (running) Dataflow has its own separate message-logging object which makes Dataflow-name-detection a very simple task.

MessageLogging1(singleton)

DataFlow 1

DataFlow 2

MessageLogging2(singleton)

Process 1

Process 2

Figure 4: Running dataflows in own processes

The advantage of this solution lies in its simplicity: (almost) nothing of the existing system has to be adjusted. Instead, an additional Dataflow-Management module can be built on top of it. A further plus point of this solution is that it increases the reliability of the whole system. When one Dataflow blocks or crashes, it doesn’t affect the other running Dataflows. However, this solution does also have a disadvantage which must be mentioned. Every JVM consumes a significant amount of memory (About 6 Mbytes of system memory without having generated any log-messages). According to the people of SunGard, normally 5-10 and exceptionally up to 100 Dataflows are configured on a system out of which only a few usually run concurrently. While working on this project, a memory stick of 256 Mbytes cost CHF 84. Since the clients of SunGard AG usually operate in the area of Financial Services, the memory issue really shouldn’t be much of a problem!


Page 14 of 62

4.1.2 Module decomposition

Module Description

Management Server The main server module. Started through command line. Controls everything on the server side.

Process Manager Used by the Management Server to control Dataflow processes. Each ICS-Dataflow runs in a separate process space (in terms of operating system processes). We chose this approach to circumvent the ‘id-problem’ described in the previous chapter.

User Manager Used by the Management Server to access a simple user database and to do some authentication.

Connection Manager Used by the Management Server to monitor active connections from client applications (the so called ‘Audit-Clients’).

Meta-Logging Specialized log-message mechanism to keep track of D-ICS specific activities.

System Monitoring Used to react to unusual system behavior such as ‘dead’ Dataflow processes.

4.2 Dataflow Process

This subsystem is responsible for Dataflow-Management and message distribution. It also contains the interface-implementation for IPC between the server- and a Dataflow-process.


Module Description

Dataflow Manager The main module inside a Dataflow process. Started through command line. Loads and runs a Dataflow. Listens to commands sent by the Process Manager (IPC).

Distributed-Message-Logging Derived from the Message-Logging-module of the original ICS. Extends it with the distribution mechanism.

Log-Message Writer The log-message distribution module. Does filtering and buffering. Sends messages over the network.


Page 15 of 62

4.3 Audit-Client

This subsystem contains the client application to access the server side-functionality. Therefore, it provides an easy-to-use graphical user interface as well as the functions needed to send commands to the server and receive log-messages by a Dataflow-process.


Module Description

Audit Client Controller The main client module. Started through command line. Loads and displays the GUI.

Management Proxy Interface module used to access the server’s functionality. Used to centralize technology-dependent source-code.

Log-Message Receiver Listens for log-messages. Contacted by the Log-Message Writer on the server’s side (Dataflow Management).

Page 16 of 62

Chapter 5 Dependency Description

5.1 Intermodule Dependencies 17

5.2 Interprocess Dependencies 21

5.3 Data Dependencies 21

Software Design Description Chapter 5: Dependency Description

Page 17 of 62

5.1 Intermodule Dependencies

5.1.1 Flow of data

If the system is running in normal operation mode (delivers only log-messages of type ‘Message’), one can expect little output of log messages. But if you view debug or trace messages, the Dataflows may produce a huge amount of output and therefore a lot of network traffic. If all this traffic would be controlled by routing it through a central module inside the server-process, a dangerous bottleneck would result. To reduce this risk, we decided to let the distribution modules send their log-messages directly to their clients. In the figure below, the flow of data is visualized and the orientation of each connection is indicated. We organized the flow of data as simple as possible. This resulted in only one bi-directional path, the one between the Management Server and the Dataflow-Management module:

controls & listensfor Dataflow A

controls & listensfor Dataflow B

controls & listensfor all Dataflows

User A User B Administrator

Audit Clients

ServerW rite Log-Messages

! Control Dataflows! Control Logging! Poll for Status

! Submit port when ready! Control Dataflows" Control Logging

Dataflow A

Legend:

Control traffic

Log traffic(large amountof data)

LogMessage-W riter

Dataflow B

LogMessage-W riter

a.)

b.)

c.)

Figure 5: Flow of data All the paths (type a, b and c) have exactly one source and one destination and are therefore unicast connections. They are described in more detail on the following page.


Page 18 of 62

Path a): Audit Client →→→→ Management Server

Audit Client

- Control Dataflows- Control Logging

- Status query by polling

Server

ManagementServer

A unidirectional connection. The client can control the Dataflows and the logging behavior. To get status information from the server, the client polls for the active state in a predefined interval. The server uses this polling procedure as a keep-alive mechanism. If a client doesn’t poll for the current state longer than a certain amount of time, the server (the Connection Manager to be more specific) supposes that the connection to the client has broken and removes it from its connection list.

Path b): Management Server ↔↔↔↔ Dataflow Process

Server

- submit Port when readyDataflow

LogMessage-W riter- Control Dataflows

- Control Logging

ProcessManager

The Process Manager (a module inside the Management Server) is responsible to start the Dataflow process. When such a process is up, it sends the port number where it is listening for commands to the Process Manager. Now knowing this port, the Process Manager is able to send commands such as stop and abort the Dataflow process and change logging properties such as the filter settings.

Path c): Dataflow Process →→→→ Audit Client

Audit Client

write Log-Messages

Server

Dataflow

LogMessage-W riter

This unidirectional connection is very simple. The Logging Module just forwards all messages to its client.


Page 19 of 62

5.1.2 Port allocation

Since every component of our system wanting to receive commands or data (Management server, Audit clients, Log-message Writers) needs its own port listener (RMI-server object) but may still run on the same computer, the ports must be assigned carefully. We decided to do this in the following manner:

Out:

IP / Port [In]:Audit Client 1

a.)

b.)

LogMessage-W riter 1

Out:

Audit-Client 1

RMI-registry Port:1100 .. xxxx(dynam ic)

In:

M anagm entServer

RMI-registry Port:1099(static)

In: Out:

c.)

d.)

IP / Port [In]:Audit Client 1

IP / Port [In]:Dataflowproc.

e.)

Dataflowprocess

RMI-registryPort:1100 .. xxxx(dynam ic)

In: Out:

Client

Server

Figure 6: Port allocation

a) The Management Server (and the process manager) binds its listener to the default

RMI-port (1099) b) The Audit-Client must try to get a free port which has to be higher than the default

communications port (>1099). Once a port has been allocated, it is used to receive log-messages from the logging module. The port and the IP-address of the client (even when used on the same computer) is sent to the Management Server, which passes this data to the log-message Writer modules.

c) The Dataflow Manager (inside the Dataflow-process) acts similar to the audit client. It tries to get a free port (>1099). It does also send its IP-address and port number to the process manager running inside the same server-process as the Management Server.

d) Every Log-message Writer inside a Dataflow-process is contacted by the Management Server (actually the process manager) which informs it about the IP and port of the corresponding Audit-Client.

e) The Log-message Writer now can send log messages to the clients. Note: For more information on RMI, refer to chapter 6.3, Defining RMI-interfaces.


Page 20 of 62

5.1.3 Distributing Log-messages

As you can read in earlier chapters, we introduced the module Log-Message Writer (sometimes also called Logging Module) to distribute the log-messages to the correct clients. Apart from dealing with connection issues (e.g. broken connections) these modules also support a buffer to decouple the execution of a Dataflow and the generation of its log-messages from the network. Additionally, a Log-Message Writer does only deliver messages meeting a certain criteria – the filter-criteria – which prevents unwanted incoming messages (a user might e.g. only be interested in error messages) from being placed in the buffer. Note: A client may open multiple windows on the same Dataflow which results in multiple

Log-Message Writers sending messages to the same client, one for each window. After being properly initialized, a Log-Message Writer can be attached to the Message Logging Object. It is by the way possible to attach more than one module to the Message Logging Object. This opens a very flexible way to distribute the log-messages.

Message Logging

LogMessage-W riter 1

LogMessage-W riter n

LogMessage-W riter 2 ...

Audit Client 1 Audit Client nAudit Client 2

Producer

Consumer

setFilter

monitorBuffer

connectAudit-Client

Buffer

Figure 7: Remote Logging

Note: In contrast to the module name Log-Message Writer which was chosen to reflect the

module’s role inside the whole system, the main Java class implementing this module’s functions is called LogListenerModule, since in the eye of the Message Logging object, it ‘listens’ for messages. See chapter 7.3.2, The class ‘LogListenerModule’ for details.


Page 21 of 62

5.2 Interprocess Dependencies

As we described in section 4.1.1, The „ID problem“ of ICS-Dataflows, the easiest way to circumvent the “ID-problem” is to run each Dataflow in its own process. That causes our system to need some kind of process management functionality. The responsible module must be able to create a new process running a separate Java Virtual Machine. It must also provide control access to the Dataflow-process (starting, stopping, aborting and setting logging-properties such as filter settings) and maintain a list with the state of all Dataflows:

Process Management

Dataflow A running

Dataflow B ready

Dataflow C -

Dataflow D stopping

Process

Dataflow A

running

Process

Dataflow D

stopping

Process

Dataflow B

ready

Figure 8: Process Management

5.3 Data Dependencies

5.3.1 Connection Information

Apart from the server’s name (or IP-address) and its port, no additional information is needed to establish a control-connection between an Audit-Client and the Management Server. In contrast, the Audit-Client must somehow tell the Management Server (which itself forwards the information to the Dataflow Manager) on what port it is listening for log-messages to be able to receive any. This information is encapsulated inside a ‘Connection Info’-structure and includes the following items:

Item Description

Owner The user name which is working with the Audit-Client to which this Connection Info-structure belongs.

IP-address The IP-address of the computer on which the Audit-Client is running.


Page 22 of 62

Service id The object identification of the object which receives log-messages (a ‘listener’). Is different for every listening object.

Port The port where the Audit-Client listens for incoming connection requests for the message receiving objects. Is the same for every listener of an Audit-Client.

Filter Container The object containing various settings for the filter criteria applied inside a Dataflow process to decide which messages to deliver to the Audit-Client.

Type The type of the listening module. Used to differentiate between normal log-message listeners and control-message listeners.

5.3.2 Dataflow identification

ICS allows to specify user groups to hierarchically organize Dataflows:

Root node User group ‘Marketing’ User subgroup ‘Group Miller’ Dataflow ‘FlowA’ User group ‘Sales’ Dataflow ‘FlowS’ Dataflow ‘LongFlow’ Dataflow ‘FlowWithErrors’

Figure 9: Hierarchically organized Dataflows This feature called for a way to uniquely identify a Dataflow, since it is possible to create different Dataflows with the same name (pretty much like it is possible to have files with the same name in different folders). After a short discussion with Andreas Hess (of SunGard AG), we decided to use the following naming convention:

In analogy to path names of modern file-systems, a name uniquely identifying a Dataflow consists of all its user group names separated by a certain string. In contrast to file-systems which use forward- (/) or backslashes (\), two double points (::) are used as separator.

The fully qualified names of the Dataflows in the above figure would be:

(group) (group) (group) Marketing::Group Miller:: FlowA (group) Sales::FlowS LongFlow FlowWithErrors

Figure 10: Fully qualified Dataflow names

Page 23 of 62

Chapter 6 Interface Description

6.1 Network architecture 24

6.2 Multicast? 27

6.3 Defining RMI-interfaces 28

6.4 The interfaces used 31

Software Design Description Chapter 6: Interface Description

Page 24 of 62

This chapter deals with internal interfaces, only. For information about the user interface, please refer to the separate document ‚Installation guide & user manual’.

6.1 Network architecture

This section lists possible technologies for the network architecture of our system, while concentrating on the most popular ones. At the end, a conclusion section can be found wherein we compare these technologies. Note: You may miss one of the oldest technologies in this area: RPC (Remote Call

Procedure), probably the grandfather of every modern network communication protocol. However, since it’s a technology usually used for ANSI-C applications, it wouldn’t make much sense to use it in our system. Today, there are many technologies available for Java programs which are much more comfortable.

6.1.1 CORBA (Common Object Request Broker Architecture)

Description: CORBA allows one to call functions or class methods located at a remote (or local) server. Because CORBA uses an IDL (Interface Description Language) the client and the server may be written in different programming languages.

HowTo: 1. Define an Interface according to the IDL guidelines of CORBA 2. Generate skeleton, stub and a Example Servant with a CORBA

IDL-compiler 3. Modify the Example Servant 4. Compile the Client and the Server with an arbitrary Compiler (C++,

Java, Smalltalk, …) Communication: Synchronous / Asynchronous Lookup: ORB (Object Request Broker) Used Protocols: GIOP (General Inter-ORB Protocol) / IIOP (Internet IOP)

over an arbitrary standard layer 3 protocol Multicast: Supported License: Free Special Requirements: IDL-Compiler Pros.: Probably the most flexible communication architecture. Supported by

all the common programming languages. Cons.: Rather complex to implement


Page 25 of 62

6.1.2 RMI (Remote Message Invocation)

Description: RMI allows one to call methods of Java classes located at a remote (or local) system. A client object does this by calling the method as if it would be located on the local system, once a reference to the remote object has been gained.

HowTo: 1. Define a Java interface 2. Generate skeleton and stubs with the RMI compiler 3. Implement the interface on the server side

Communication: Synchronous Lookup: RMI Registry (Naming service which must be started on server-side) Used Protocols: JRMP (Java Remote Method Protocol)

Note: Sun does not appear to use this term any longer. They seem to simply refer to “RMI transport protocol”.

Multicast: Not supported License: Free Special Requirements: rmic (RMI Compiler) Pros.: Easy to implement. Widely available. Quasi standard for Java

applications. Easy to guarantee at-most-once semantics. Cons.: Compatible with Java applications only. No multicast

6.1.3 JMS (Java Message Service)

Description: The Java Message Service is a Java API that allows applications to create, send, receive, and read messages. A messaging system is a peer-to-peer facility: a messaging client can send messages to, and receive messages from, any other client. Each client connects to a messaging agent that provides facilities for creating, sending, and receiving messages.

HowTo: 1. Server must start the Enterprise server 2. Server must implement a message producer 3. Client must implement a message consumer

Communication: Asynchronous Lookup: Enterprise server Used Protocols: IP Multicast: Supported License: J2EE (There are rumors about some freely available implementations of

the enterprise server) Special Requirements: Enterprise server Pros.: Allows publish-subscribe architecture. Widely used. Cons.: Enterprise server must be installed


Page 26 of 62

6.1.4 Sockets

Description: This network communication architecture is based on system level calls. It allows to transport binary data which must be interpreted by the application.

HowTo: Server-Socket listens on a port for requests from the clients. Client Socket is able to send and receive messages.

Communication: Synchronous / Asynchronous Lookup: Must be implemented by hand Used Protocols: IP, UDP, TCP Multicast: Possible to implement (Java even provides Multicast Sockets) License: Free, because you have to write the code by yourself or to steal it. Special Requirements: - Pros.: Very slim and flexible because its based on system level. Cons.: A lot of work. Costly to maintain.

6.1.5 JSDT (Java Sharing Data Toolkit)

Description: JSDT was introduced by Sun to fill the gap between RMI and JMS. It is an asynchronous technology (in contrast to RMI) and it doesn’t need the Enterprise server as JMS does. JSDT uses sessions and channels. A Session is a collection of related Clients which can exchange data via defined communication paths. Channels are a specific instance of a potentially multi-party communication path between two or more Clients within a given Session. (Similar to the path- and channel-concept of ATM).

HowTo: 1. Server must start the jsdt.registry 2. Server or a client must create a new session with a new channel 3. Clients must add a ChannelConsumer 4. Server may write new messages to the channel now

Communication: Asynchronous Lookup: jsdt.registry Used Protocols: Sockets, HTTP, LRMP (Light Weight Reliable Multicast) Multicast: Supported License: Free Special Requirements: > Java Version 1.1 Pros.: Allows publish-subscribe architecture (multicast). Runs without the

Enterprise server. Almost protocol independent. Cons.: Not widely used.


Page 27 of 62

6.1.6 Conclusion

To choose the most useful communication architecture, we had to be sure of what we really needed our service to do. First we had to decide if we would really need multicast. The answer was: no! (see chapter 6.2). On the other hand we had to take special care about the environment, wherein our system should be able to run:

Protocol: almost every LAN is running TCP/IP today. Therefore all of the described technologies from the above chapters can be used.

Programming language: because the ICS is already programmed and running in Java, we’re on a safe side, if we use a Java architecture, too.

Additionally the architecture should be easy to maintain and possibly be based on freely available tools. Sockets and CORBA are rather complex and would be too time-consuming to implement for this application. JMS needs an extra Enterprise Server (and therefore we would have to deal with licensing) and JSDT is not widely used. With these arguments in mind, we decided to use the popular RMI. It fulfils all our criteria and is well supported by Sun Microsystems. At last, we were bringing some experience of our own with this architecture, which presumably contributed most to give the outbalance and showed to be a big advantage during the development phase.

6.2 Multicast?

The document ‘Software Requirement Specification’ demands that several clients can view the logging output of a single Dataflow. Our first thoughts were to use multicast to reduce the network traffic. But after spending some thoughts on it, we realized that multicast would be very complex to implement and generate a lot of group management traffic. The following example demonstrates this problem:

Logging System

Client A

views errormessages

Client B

views error, warn &debug messages

Client C

views error & w arnmessages

produces error,warn & debug

messages

Figure 11: Multicast groups


Page 28 of 62

For the sample configuration shown in the figure above, the following multicast groups would have to be created:

Published Subscribers

Group Message type Client A Client B Client C

1 error X X X

2 warn X X

3 debug X

Table 1: Multicast group management

This example is very simple; it only needs three multicast groups. But if you allow arbitrary filter configurations (as we do in our system), the amount of multicast groups rises to almost infinity and would generate too much network traffic. We decided not to use multicast!

6.3 Defining RMI-interfaces

Our highly asynchronous and distributed system required a lot of internal interfaces to be introduced. Since we decided to use Java’s easy to use RMI-mechanism for the system’s modules to communicate, we are going to describe this technology in the next chapter before discussing the actual interfaces themselves in the following chapter. In order not to pointlessly duplicate information published in hundreds of books and thousands of websites, we will (try to) be brief. Note: In the context of RMI, a ‘server-object’ is located at the end of a communication path

whereas a ‘client-object’ is situated at the start of it. However, since our system uses these terms also in a more abstract manner, they have another meaning in its discussion. A ‘server-side object’ in the system’s context stands for an object residing inside one of the server-side processes (the process with the Management Server or the Dataflow-processes) whereas a ‘client-side object’ stands for an object residing inside an Audit-Client.

We try to avoid this inconsistency by speaking of local- and remote- instead of server- or client-objects in the context of RMI. ‘local’ means the origin of an RMI-connection (its starting-point) while ‘remote’ stands for the destination (its end-point). Any questions?

6.3.1 What is RMI

The Java Remote Method Invocation mechanism allows one to call object methods located at a remote (or local) system. A client object does this by calling the method as if it would be located on the local system, once a reference to the remote object has been gained.


Page 29 of 62

Actually, the client does not call the remote object itself, but a stub object located on the local system. The stub object serializes all arguments to the method (! if using objects of custom types as arguments, these types MUST implement the interface Serializable!!!) and passes them on through the network to a skeleton object residing on the server system. Then, the method of the server-side object itself is executed, the return value is passed back to the skeleton object, serialized and sent back through the network where the stub object passes the return value back to the client. If something goes wrong, a java.rmi.RemoteException is raised.

Server-object

Client-object

Server-Skeleton

Client-Stub

Network

Invoke

Return

Invoke

Return

Figure 12: calling methods using RMI

Note: In our system, we use RMI not only to let computers within a network communicate

with each other. We use it to communicate across process-boundaries on the same computer as well!

6.3.2 How to use RMI

The following steps must be taken to write server- and client-side RMI-classes:

Step 1: Define a remote interface! IMyInterface.java

interface IMyInterface extends java.rmi.remote {…void doSomething() throws java.rmi.RemoteException…

}

Step 2: Implement the interface! MyServer.java

class MyServer extends UnicastRemoteObjectimplements IMyInterface {

…Naming.rebind(servicename, object);

…}

Step 3: Compile! IMyInterface.class MyServer.class

Step 4: Create Stub and skeleton (proxy) with ‘rmic’rmic MyServer.class! MyServer_stub.class MyServer_skel.class

Step 1: Write client! MyClient.class

class MyClient … {…

IMyInterface myobject = (IMyInterface)Naming.lookup(“rmi://servername:1099/” + servicename, object);

…}

Step 2: Compile! MyClient.class

server-side / remote client-side / local

Step 5: Start the registry and register objects:rmiregistry / java MyServer

Starting the registry can also be done inMyServer.java with the following line of code

java.rmi.registry.LocateRegistry.createRegistry(1099);

Step 3: Start the client:java MyClient

Figure 13: necessary steps to develop local and remote classes using RMI


Page 30 of 62

From the programmer’s point of view, there is only one difference between calling a method of a local and calling a method of a remote object. When working with remote objects, the programmer must first check if this object is available on the server to which a connection is to be established. This is done by using method lookup()of class Naming (in package java.rmi) As you can see in the example below, the reference which is returned is of the type of the remote-interface and not of the implementing class itself. Listing 6.3.1: how to retrieve a reference to a remote object using RMI 1 try { 2 IMyInterface rObject = (IMyInterface)Naming.lookup( 3 "rmi://" + servername + ":1099/" + servicename); 4 } catch (Exception e) { 5 /* handle Exceptions */ 6 }

On the remote (the server’s) side, the programmer must write a class, which is able to create an instance of the object to be accessed remotely. After instantiation, the object must be registered at (or: bound to) the RMI-registry where it is associated with the supplied service name (see the next chapter). Additionally, the class may start the RMI-registry (which is usually started by typing rmiregistry into the console) itself as shown in the following example. Listing 6.3.2: how to start the RMI-registry programmatically 1 try { 2 IMyInterface myObject = new MyServer(); 3 /* the next line must be placed BEFORE Naming.rebind()!!! */ 4 LocateRegistry.createRegistry(1099); /* standard RMI-port */ 5 Naming.rebind(servicename, myObject); 6 } catch (Exception e) { 7 /* handle Exceptions */ 8 }

Note: While it might make sense (and usually is the method of choice) to start only one

common registry for all server-processes, we chose to start a separate registry for each of them and therefore to let every process listen on another TCP-port. While the former method would also have been possible to consider, the latter makes the system less protocol-dependent. The only drawback of this solution is the lost memory due to multiple instances of the registry. See chapter 5.1.2, Port allocation, for details on the chosen ports.

6.3.3 RMI object identification

Objects must be associated with a unique service name when they are bound to the RMI-registry. This allows the registry to distinguish multiple instances of the same class. The service names can be freely chosen, but when a reference to such an object is queried, this is done through a Universal Resource Indicator (URI) which must be of the following form:


Page 31 of 62

rmi://ksy008.zhwin.ch/LogMessageListener_3

Protocol(always ‚rmi‘)

server name(hostname or IP)

service name(usually contains the class-name)

Figure 14: RMI-object identification

Note: To distinguish objects of the same class, we introduced static class attributes which are increased with every instantiation. These numbers are appended to the class names in the objects’ constructors which generates a unique URI for every object.

6.4 The interfaces used

We introduced the following RMI-interfaces:

Interface Description

IManagementServer Defines the management server’s functions available to Audit-Clients. Exposes methods out of various areas: user-, connection-management and dataflow-control.

IProcessManager Defines the functions available to access the Process Manager. Used by Dataflow-processes to exchange information with the Process Manager.

IDataFlowManager Defines the functions available to access a Dataflow-process. Used by the Dataflow-process controller to pass on commands received by the management server which in turn received the command by an Audit-Client.

ILogMessageReceiver Exposes a method to pass over a log-message. Every object receiving log-messages from a Dataflow-process should implement it.

6.4.1 The interface ‘IManagementServer’

This interface is used by the client to pass along commands to the server. These commands include logging in, starting Dataflows, connecting to Dataflows as well as pausing, stopping and aborting Dataflows. Additionally, it is used to adjust the filter settings of the log-message distributing modules. The following sequence diagrams show how some commands are handled.


Page 32 of 62

Logging in:

AuditClient ManagementServer

return session ID

login(user, password, ip)

UserManager

check access(user,password)

return access level

ConnectionManager

start session(user, ip)

return session ID

clie

nt-s

ide

serv

er-s

ide

Figure 15: sequence diagram: logging in

Starting a Dataflow:

ManagementServer ProcessManager

check session

start DataFlow(user, dataflow, ...)

AuditClient

start DataFlow(session, dataflow, ...)

ConnectionManager

access granted / denied

see process management...

return

DataFlowManagerexecute

return

clie

nt-s

ide

serv

er-s

ide

Figure 16: sequence diagram: start dataflow


Page 33 of 62

Connecting to a Dataflow:


check session

connect to DataFlow(dataflow, ...)

AuditClient

connect to DataFlow(session, dataflow, ...)

ConnectionManager



return

DataFlowManager

add user

return

check access(dataflow, privileges)


return

clie

nt-s

ide

serv

er-s

ide

Figure 17: sequence diagram: connecting to a Dataflow

Stopping and aborting a Dataflow:


check session

stop/abort DataFlow(dataflow, ...)

AuditClient

stop/abort DataFlow(session, dataflow, ...)

ConnectionManager


return return




DataFlowManager

stop/abort

return

clie

nt-s

ide

serv

er-s

ide

Figure 18: sequence diagram: Stop/abort dataflow


Page 34 of 62

Setting a filter:


check session

set filter(dataflow, ...)

AuditClient

set filter(session, dataflow, ...)

ConnectionManager



return

DataFlowManager

set filter

return



return

clie

nt-s

ide

serv

er-s

ide

Figure 19: sequence diagram: set filter

6.4.2 The interface ‘IProcessManager’

This interface is used to cross process-boundaries. It provides access to the Process Manager from outside the server-process. When a Dataflow-process becomes ready, its manager retrieves a reference to the interface by querying the registry running on port 1099 (not shown in the diagram below) and notifies the Process Manager. Afterwards, it runs the corresponding Dataflow-object and waits for its end. The Process Manager may send additional commands while the Dataflow Manager is waiting.


Page 35 of 62

ProcessManager

DataFlowManagerexecute (command line)

signal readyness / tell port

get waiting connections

DataFlowload

start()

control calls [connect/stop/abort/pause/setFilter]

end

signal end

serv

er-p

roce

ss

data

flow

-pro

cess

Figure 20: sequence diagram: interaction between the Process Manager and a Dataflow-process

6.4.3 The interface ‘IDataFlowManager’

While IProcessManager is used by a Dataflow Manager to access the Process Manager from outside the server-process, this interface is used for the opposite direction. It simply allows the Process Manager to pass on the calls received by the Management Server to the Dataflow Manager controlling the correct Dataflow-process.

6.4.4 The interface ‘ILogMessageReceiver’

ILogMessageReceiver is a very simple interface which must be implemented by modules listening for log-messages. It defines only the method writeMessage() which is called by a Log-Message Writer to deliver a log-message.

Page 36 of 62

Chapter 7 Detailed System Design

7.1 Package structure 37

7.2 The package ‘audit’ 38

7.3 The package ‘logging’ 46

7.4 The package ‘management’ 50

7.5 The package ‘test’ 56

7.6 The package ‘util’ 57

7.7 Performance issues 58

Software Design Description Chapter 7: Detailed System Design

Page 37 of 62

This chapter contains detailed information about the software design of the Distributed ICS. We chose to follow the package structure (in terms of Java-packages) in our discussion of the system’s details.

7.1 Package structure

This system consists of a large number of different Java-class-files. Therefore, we had to decide how we wanted to organize them in order to keep the whole system maintainable. We chose the following package structure:

The main package (contains no source code) Classes concerning the client side only Contains user interface classes Controller classes (GUI-related event handlers, listener classes) Classes which represent dialog windows Global definitions (from the client applications point of view) Classes dealing with the tree view of dataflow objects Classes representing different tree node types Helper classes Classes dealing with everything about logging The buffer implementation Message distribution classes Classes providing Filter functionality Classes concerning the functionality of the Management server Configuration classes (currently not used) Project-specific exception definitions Monitoring classes Classes dealing with Dataflow processes Classes representing Dataflow process-states User database classes (Currently only file access available) GUI-classes for the user database functionality Project wide test-classes Utility classes (RMI helper classes, timer, etc.)


Page 38 of 62

7.2 The package ‘audit’

This package has its focus on the GUI of the Audit-Client. Although the GUI took a big part of our time and includes a lot of code, we try to discuss just the most important and interesting parts here. To get a general overview of the GUI, the figure below shows a screenshot of the application’s main window which is divided into the following parts:

• a menu, which allows to control the application.

• a toolbar that provides shortcut access to the most important commands.

• a tree view on the left side, which lists all Dataflows available on the server in a hierarchical manner.

• a tabbed pane on the bottom, which contains buttons to control the selected dataflows.

• a multi-document area in the center, where the log message windows are managed.

• a statusbar on the bottom, which informs about the current status.

MainFrame

LogMessageFrames

DataflowControlTabs

TreeView

StatusBar

Figure 21: Screenshot of the AuditClient


Page 39 of 62

7.2.1 Model-View-Controller (MVC)

Where it was possible and reasonable we divided the classes strictly according to the well known MVC concept. Every view is managed by a separate controller (see Figure 22). The AuditClientCtrl controls the main application. In the startup phase it creates the MainFrameCtrl which instantiates the MainFrame and makes it visible. The two other controller classes, DataFlowControlCtrl and LogMessageCtrl, will be created dynamically if a user wants to control a Dataflow or get messages from it. The class diagram below illustrates that every controller tab has its own DataFlowControlCtrl and every internal logging frame a LogMessageCtrl class. To keep these two controller classes as small as possible, we extracted the two listener classes which are responsible for the RMI connections and calls.

AuditC lientCtrl

-mfCtrl: MainFram eCtrl-dfcCtrl: Vector-lmfCtrl: Vector

+AuditClientCtrl()

M ainFram eCtrl

-acctrl: AuditClientCtrl

+MainFram eCtrl(acctrl: AuditClientCtrl)

M ainFram e

-mfctrl: MainFrameCtrl

+MainFram e(m fctrl: MainFram eCtrl)

1 1

11DataFlowControlCtrl


+DataFlowControlCtrl(acctrl AuditClientCtrl)

DataFlowControl

-dfctrl: DataFlowControlCtrl

+DataFlowControl(dfctrl: DataFlowControlCtrl)

1

1

1

DataFlowControlListener


+DataFlowControlListener(acctrl AuditClientCtrl)

LogM essageCtrl


+LogMessageCtrl(acctrl AuditClientCtrl)

LogM essageFram e

-lmctrl: LogMessageCtrl

+LogMessageFrame(lm ctrl: LogMessageCtrl)

1

1

LogM essageListener


+LogMessgeListener(acctrl AuditClientCtrl)

1 1

1 1

1

1

0..*

0..*

Figure 22: Class diagram about the MVC concept


Page 40 of 62

7.2.2 Menu items with attached Java Actions

The following figure shows a situation where a menu command and a toolbar button each should fire the same function-call and each should be disabled when no connection to the server exists:

Action: new...

Figure 23: Actions and GUI control elements To deal with the function-calls, we simply assigned a reference to the same command string (a static class attribute of the controller class) to each of the corresponding control elements. This allowed us to use the same if-then-else branch for each of the group members when doing the reference comparison to identify the action’s source (inside the action handling method actionPerformed). Listing 7.2.1: Identifying an action 1 public void actionPerformed(ActionEvent e) { 2 ... 3 [else] if (e.getActionCommand() == THIS_COMMAND) { 4 /* call whatever method needed */ 5 } 6 }

Notes: Line 3: The second part of the comparison is usually a static class attribute of type String

which is the command string attached to the control elements (usually done in their constructor).

To achieve easy enabling/disabling of corresponding control elements, we created a Java Action for every such control element group and added it to each of the group members. After having done this, enabling or disabling the group became fairly easy and consisted of simply enabling/disabling the attached Action. Listing 7.2.2: How to create two items with the same action attached 1 JMenuItem newWindowItem = new JMenuItem("new..."); 2 newWindowItem.setAction(MainFrameCtrl.windowNewAction); 3 4 JButton newWindowButton = new JButton(new ImageIcon("new.gif"); 5 newWindowButton.setAction(MainFrameCtrl.windowNewAction); To further ease the use of Java Actions, we implemented our own action class (class MyAction).


Page 41 of 62

Listing 7.2.3: Enabling/disabling a control element group by using class MyAction 1 public class MainFrameCtrl { 2 3 ... 4 5 /// Action for the Window-New control elements 6 public static final MyAction windowNewAction = new MyAction(); 7 8 /** Modifies all items belonging to the "windowNew"-Action. */ 9 public void enableWinNew(boolean enable) { 10 windowNewAction.setEnabled(enable); 11 } 12 13 ... 14 15 }

7.2.3 Multi document support

To provide a Multi-Document-Interface (MDI), we had to look for a way to display and handle child windows. We decided to use the JInternalFrame class with which it is possible to display a JFrame-like window within another window. The code for using internal frames is similar in many ways to the code for using regular Swing frames. The following code shows a common way to write own implementations of internal frames and how to instantiate them: Listing 7.2.4: How to write a class derived from JInternalFrame create a new internal frame 1 public class MyInternalFrame extends JInternalFrame { 2 static int openFrameCount = 0; 3 4 public MyInternalFrame() { 5 super("Document #" + (++openFrameCount), 6 true, //resizable 7 true, //closable 8 true, //maximizable 9 true);//iconifiable 10 11 //...Create the GUI and put it in the window... 12 } 13 } Listing 7.2.5: How to create a new instance of an internal frame 1 protected void createFrame() { 2 MyInternalFrame frame = new MyInternalFrame(); 3 frame.setVisible(true); 4 desktop.add(frame); 5 try { 6 frame.setSelected(true); 7 } catch (java.beans.PropertyVetoException e) {} 8 }


Page 42 of 62

7.2.4 Treeview

In chapter 5.3.2: Dataflow identification, we discussed the naming of Dataflows. A fully qualified Dataflow name is of the form:

[groupX]::[groupX.Y]::…::[dataflow] ( e.g: Sales::Financial::US2CHF )

To supply the user not simply with a one dimensional table listing all these illegible strings, we decided to create a tree view similar to the one provided by the ICS Configuration Manager. The Java API provides such a view with the class JTree.

Tree view basics While you can read the details of how to create and use class JTree in the Java API documentation, we want to supply you with a short summary of how we used it at this point. When using a tree view, the first thing to deal with is its underlying model. A JTree has a reference to an interface of type TreeModel attached to it which contains all the information visualized by the tree view. The default implementation of this interface, the DefaultTreeModel, uses nodes (through references to another interface, the TreeNode) to maintain the tree-like structure. Java again provides a default implementation for nodes, the class DefaultMutableTreeNode. To incorporate user-defined behavior (or simply additional information), it is a common solution to write an abstract class which is derived from DefaultMutableTreeNode and builds the starting point for the different types of user-defined tree nodes. For example, the node hierarchy used in this system looks like this:

RootNode

+RootNode()

GroupNode

+GroupNode(groupnam e: String)

DataFlowNode

-controlled: boolean+DataFlowNode(dataflow: String)+getControlledState():boolean+setControlledState(b: boolean)

INodeextends DefaultMutableTreeNode

-name: String+INode(nam e: String)+getName():String+getFullNam e():String+getIcon():ImageIcon

Figure 24: Node organization for the Dataflow tree view


Page 43 of 62

Customizing a tree view Our intention was to create a tree view which allows to identify a currently monitored Dataflow with one glance of an eye. Since the default-way a JTree draws itself didn’t quite meet our needs, we decided to write our own tree-cell renderer. Usually, you do this by writing an inner class (inside the tree view class which extends class JTree) which is derived from class DefaultTreeCellRenderer. However, we didn’t follow this procedure. After finding out, that class DefaultTreeCellRenderer is simply derived from JLabel and doesn’t implement very spectacular methods of its own, we decided to write our cell renderer without deriving it from DefaultTreeCellRenderer but extending JLabel directly. This forced us to gather some more information about the way the DefaultTreeCellRenderer is implemented. We achieved this by simply decompiling it and looking at the code. But, hey: Don’t tell anyone! Note: Did you ever run into the situation of being doomed to understand a system but having

no documentation? Where the only thing you had access to was the Java source code and really had some problems understanding it? Where you didn’t like the programming style (e.g. no or unsatisfactory indentation)? If true, try the following: compile it, then decompile it using one of the freely available tools and have another look at it. Better now?

You can find a really sophisticated tool, the DJ Java Decompiler, at: http://members.fortunecity.com/neshkov/dj.html.

7.2.5 Internationalization (i18n)

The strange name ‘i18n’ comes from the lazy Java developers. They are too lazy to type in all the 20 characters of the term internationalization, every time it occurs:

“Internationalization is the process of designing an application so that it can be adapted to various languages and regions without engineering changes. Sometimes the term internationalization is abbreviated as i18n, because there are 18 letters between the first ‘i’ and the last ‘n’”

taken from the website: http://java.sun.com/docs/books/tutorial/i18n/intro/index.html

The reason why we decided to use Internationalization was actually to get source code files, which are simpler to maintain. The Java Internationalization functionality stores all the strings which are to be displayed in a separate text-file, a so-called ‘ResourceBundle’. This allows you to edit all strings in one central file. Additionally, you can easily translate the contents of such a file into other languages and thereby create a different language version of your application with almost no effort. When using I18N, Java first tries to find the correct localized ResourceBundle. If a specific string is not found, the basic ResourceBundle is used instead. The localized versions of the ResourceBundles carry abbreviations of the language and country names in their filenames. You can see some examples in the following illustration:


Page 44 of 62

Basic Resource Bundle:MessageBundle.properties

German Resource Bundle:MessageBundle_de_DE.properties

Language specific Resource Bundles

US-English Resource Bundle:MessageBundle_en_US.properties ...

Figure 25: Resource Bundle dependency

The following listing demonstrates how to read a string out of a ResourceBundle: Listing 7.2.6: Internationalization example 1 public class I18NSample { 2 3 static public void main(String[] args) { 4 ResourceBundle messages; 5 6 // sets a language 7 Locale currentLocale = new Locale("en", "US"); 8 9 messages = ResourceBundle.getBundle("MessageBundle", 10 currentLocale); 11 System.out.println(messages.getString("hello")); 12 System.out.println(messages.getString("world")); 13 } 14 } Since ResourceBundles are loaded like Property files, a ResourceBundle must follow the naming pattern [bundle-name].properties, e.g. ‘MessageBundle.properties’. Here’s an example: Listing 7.2.7: Contents of an example ResourceBundle ‘MessageBundle.properties’ 1 hello = Hello 2 world = World While the above internationalization example uses English as its language setting, it is an easy task to support other languages lik German. You must simply change the locale-settings (see Listing 7.2.6: Line 7) to Locale("de", "DE") and create the corresponding ResourceBundle: Listing 7.2.8: MessageBundle_de_DE.properties 1 hello = Hallo 2 world = Welt


Page 45 of 62

7.2.6 Configuration

The configuration contains all variable values for the application, such as window placement and size or the language settings. Usually the configuration should manage itself. Manual adjustments are only necessary if you want to reset the actual settings. We implemented the configuration load and store mechanism using the Properties helper class of the Java API. This class manages its attributes and values inside a hash-table which is quite simple but does also have an unappealing disadvantage: Naturally, a hash-table sorts its data according to the used hash-values (which are not the attribute names themselves, but the result of a calculation over them). Therefore, every time you store a hash-table to a file which you previously arranged, you will lose its order. Note: The configuration management could (should?) be replaced with a ‘more sophisticated’

version which uses an XML-file, sometimes. Listing 7.2.9: the (sorted version) of the configuration file ‘AuditClient.ini’ 1 #Ini-File for the ICS Audit-Client 2 #Mon Oct 22 19:04:06 CEST 2001 3 4 I18N_country=DE 5 I18N_language=de 6 7 LOGIN_user=doejohn 8 LOGIN_server0=localhost 9 LOGIN_server1=160.131.20.10 10 11 WINDOW_x=4 12 WINDOW_y=5 13 WINDOW_width=1017 14 WINDOW_height=707 15 WINDOW_divider=438 16 WINDOW_hDivider=215 17 WINDOW_vDivider=477 18 19 LOGWINDOW_height=346 20 LOGWINDOW_width=544 21 LOGWINDOW_colarray0=79 22 LOGWINDOW_colarray1=15 23 LOGWINDOW_colarray2=101 24 LOGWINDOW_colarray3=102 25 LOGWINDOW_colarray4=100 26 LOGWINDOW_colarray5=99 27 LOGWINDOW_path=E\:\\Da\\config 28 29 CONTROL_dataflow0=Marketing\:\:Group Miller\:\:FlowA 30 CONTROL_dataflow1=FlowWithErrors 31 CONTROL_dataflow2=LongFlow 32 33 FILTER_path=E\:\\da\\src\\classes


Page 46 of 62

7.3 The package ‘logging’

The existing ICS uses the singleton class MessageLogging for collecting all log messages. Therefore, every component wanting to write a message to the log-message subsystem must get the instance of this class first.

M essageLogging

-instance : MessageLogging = null

#MessageLogging()+getInstance : MessageLogging+writeMessage(Message m sg)()+...

0..*

1

Figure 26: Class diagram: MessageLogging

Listing 7.3.1: Code snippet of the MessageLogging class (untouched) 1 public class MessageLogging{ 2 3 /** makes sure that only one instance of this class exists */ 4 public static synchronized MessageLogging getInstance() { 5 if (instance == null) { // instance is a class attribute 6 instance = new MessageLogging(); 7 } 8 return instance; 9 } 10 11 /** handles the message (e.g write it to a log file) */ 12 public writeMessage(Message msg) { 13 ... 14 } 15 } Note: To be able to run D-ICS, you must apply changes to the class MessageLogging as

described in chapter 3.1, Assumptions and Dependencies and the following chapter.

7.3.1 The class ‘DistributedMessageLogging’

For the extension with log-message distribution, we tried to touch the existing code as little as possible. To achieve this, we introduced a new class called DistributedMessageLogging which is derived from the existing MessageLogging class. This way, we only had to pass on getInstance()-method calls to the class MessageLogging to our own class (see Listing 7.2.6:).


Page 47 of 62

M essageLogging

-instance : MessageLogging = null

#MessageLogging()+getInstance : MessageLogging+writeMessage(Message msg)()+...

DistributedM essageLogging

-instance : DistributedMessageLogging = null-userList: Vector#DistrbutedMessageLogging()+getInstance : DistributedMessageLogging+writeMessage(Message msg)()+addUser(conInfo ConnectionInfo)+removeUser(conInfo ConnectionInfo)+setFilter(conInfo ConnectionInfo)-distributeToUsers(Message msg)

0..*

1

Figure 27: Class diagram ‘DistributedMessageLogging’

Listing 7.3.2: Changes to the MessageLogging class 1 /** pass on the call to the DistributedMessageLogging class */ 2 public static synchronized DistributedMessageLogging getInstance() { 3 return DistributedMessageLogging.getInstance() 4 } As you can see in the figure above, the DistributedMessageLogging class has some new methods which allow to add and remove users and manage the filter settings. The method addUser() creates a new LogListenerModule which is responsible for sending messages of the ICS-components attached to the Dataflow (remember: in our solution, every Dataflow runs in its own JVM) over the network. With the setFilter() method, a user may change a previously chosen filter (e.g. just error messages). The DistributedMessageLogging class maintains the filter-settings indirectly by managing a hash table associating Users with their ConnectionInfo-structures which contain the filter settings. When a new message comes from an ICS-component, the writeMessage() method first calls the corresponding method of its super-class (MessageLogging) and then distributes the message to all active log-message modules (of type LogListenerModule). Listing 7.3.3: Code snippet from the DistributedMessageLogging class 1 public class DistributedMessageLogging extends MessageLogging { 2 3 /// maintain a list with all user who want messages 4 private static Hashtable users = new Hashtable(); 5 6 ... 7 8 /** writes the message to the super method and distributes 9 it to all users from the hashtable */ 10 public void writeMessage(Message msg){ 11 super.writeMessage(msg); 12 distributeToUsers(msg) 13 } 14 15 /** distribute the message to all users from the hashtable */ 16 private void distributeToUsers(MessageEx msg) { 17 for (Enumeration e = users.keys(); e.hasMoreElements(); ) { 18 // gets the next element 19 ConnectionInfor next = (ConnectionInfo)e.nextElement(); 20 // gets the LogListenerModule from the hashtable 21 (LogListenerModule)users.get(next).writeMessage(msg); 22 } 23 }


Page 48 of 62

7.3.2 The class ‘LogListenerModule’

The LogListenerModule is responsible to send a message to its corresponding Audit-Client. When you create a new instance of this class, it tries to get a connection to the client. The client’s address is given in the ConnectionInfo-structure. When the method sendMessage() is called, it first checks if the given message meets the current filter criteria. If this is the case, the message is put into the fifo-buffer. The buffering mechanism is realized inside the helper class FifoBuffer. This mechanism is just for decoupling the LogListenerModule from the network. In the situation of heavy network traffic, this prevents the occurrence of an immediate slow-down effect. A message can reside in the buffer for a while and be processed later (by a separate thread for each buffer). The following figure shows that the buffer is fully separated from the LogListenerModule. The only requirement to a class which needs to do buffering is to implement the IBuffer interface and therefore provide a process() method which will be called by the buffer’s reading thread to finally deliver a message.

LogM essageM odule

-coninfo: ConnectionInfo

+LogMessageModule(coninfo ConnectionInfo)+sendMessage(m sg: Message)+process(obj: Object): void

IBuffer

+process(obj: Object)

FifoBuffer

-linkedList: LinkedList = null-size: int

+FifoBuffer(size: int)+put(obj: Object): void+get(): Object+dispose(): void

ReaderThread

-fifoBuffer: FifoBuffer

+ReaderThread(fifoBuffer: FifoBuffer)+run() : void

Buffer

ILogMessageReceiver

+writeMessage(m sg: Message)

Rem ote

Figure 28: Class diagram: LogListenerModule with FifoBuffer

The FifoBuffer class is implemented using a simple version of the well-known Producer/Consumer pattern. The consumer runs in its own thread which waits for new messages from the buffer. If a new message is retrieved, the consumer sends it to the attached IBuffer interface and therefore back to the LogListenerModule which owns the buffer object. Note: The separation of the LogListenerModule and the Buffer allowed us to implement

synchronous message delivery (without the Buffer) as a first approach (in the first milestone) and later easily extend it with the buffer mechanism to make it asynchronous.


Page 49 of 62

7.3.3 Filtering

A user must be able to set a filter for every view he opens on a specific Dataflow. To get a very flexible solution we introduced the class FilterContainer which maintains a list (a vector) of filter objects. These are derived from an abstract base class, the AbstractFilter class. It is required that every concrete filter object (every derived class) must implement the passed() method defined by the class AbstractFilter. This method is used to check if a message meets the encapsulated filter criteria. The FilterContainer class itself does also provide a method passed(). It contains a simple loop to walk through all filter objects stored in the list and ask them if a message meets their criteria (by calling their passed() method). If a filter object refuses to accept a message, its passed() method-call returns ‘false’. This causes the container to itself return ‘false’ which indicates that a message should be thrown away.

FilterContainer

-filters: Vecor = new Vector()

+FilterContainer()+add(filter: AbstractFilter):void+clear():void+passed(msg:Message):boolean

AbstractFilter

+AbstractFilter()+passed(M essage msg):boolean

1 0..*

M essageLevelF ilter

-enabledLevels: Vector

+MessageLevelFilter(enabledLevels: Vector)+passed(Message msg):boolean

Com pareFilter

-pattern: String-attribute: int

+CompareFilter(attribute:int, pattern:String)+passed(Message msg):boolean

Figure 29: Class diagram: Filter

Listing 7.3.4: Code snippet from the FilterContainer class 1 public class FilterContainer { 2 3 /// list with all filters 4 private Vector filters = new Vector(); 5 6 /** Constructor */ 7 public FilterContainer() { 8 } 9 10 /** 11 * Cycles through all filters and checks if a message is valid. 12 */ 13 public boolean passed(MessageEx msg) { 14 for (Iterator i = filters.iterator(); i.hasNext(); ) { 15 if (((AbstractFilter)i.next()).passed(msg) == false) { 16 return false; 17 } 18 } 19 return true; 20 } 21 }


Page 50 of 62

The chosen design (classes FilterContainer and AbstractFilter) allows to implement very complex filters. In the current version, we integrated a MessageLevelFilter class which filters out messages of a certain message level (e.g. just ‘Error’ or ‘Info’ messages) and a CompareFilter class which allows to search for strings. The latter also supports regular expressions: If a user is looking for messages from the input and output peers only, he may set the filter pattern to “(input)|(output)peer” and activate the regular expression support.

7.4 The package ‘management’

The classes in this package collaborate in the following way (simplified):

Dataflow ProcessDataflow ProcessDataflow Process

server : ManagementServer

userManager : UserManager

connectionManager : ConnectionManager

processManager : ProcessManager

DataFlowManager

Server Process

Dataflow Processes

verify

handle session

start/connect to/stop/abort dataflow

start/connect to/ stop/abort dataflow

Rem

ote

ac

cess

Figure 30: collaboration diagram of package ‘management’ The classes shown in the above figure will be discussed in the following chapters.


Page 51 of 62

7.4.1 The class ‚ManagementServer’

This is the core class of the package. On one hand, it is the access point (the RMI-server object) for connections initiated by Audit-Clients. On the other hand, it is the controller of every server-side activity while delegating a lot of work to helper classes. The following (somewhat incomplete) class diagram shows how these helper classes are connected to the class ManagementServer:

M anagem entServer

-userManager: UserManager-connectionManager: ConnectionManager-processManager: ProcessManager

+Managem entServer(props: Properties)+login(...)+changePassword(...)+ping(...)+getStatus(...)+setFilter(...)...

UserM anager

-db: IUserDB

+UserManager(provider: String)+login(user: String, pwd: byte[])+isAdm in(user)+changePassword(user: String, oldp: byte[], newp: byte[])

ConnectionM anager

-sessions: Hashtable

+ConnectionManager()+checkSession(ci: ConnectionInfo)+ping(ci: ConnectionInfo)

1

SessionID

-id: int-type: int-owner: String

+SessionID(user: String, type: int)+getID(): int+getType(): int+getOwner(): int

0..*

ProcessM anager

+SERVER_PORT: int-runningDataFlows: Hashtable-cm: ComponentManager+ProcessManager(props: Properties)...

IManagementServerextends java.rm i.Remote

-userManager: UserManager-connectionManager: ConnectionManager-processManager: ProcessManager

+login(...)+changePassword(...)+ping(...)+getStatus(...)+setFilter(...)...

Figure 31: Class diagram of classes working with class ManagementServer As you can see in the diagram above, the class ManagementServer implements the interface IManagementServer. This interface not only defines all the methods accessible by the Audit-Client but also makes it possible for instances of implementing classes to register themselves in the RMI registry. As discussed in chapter 5.1.2: Port allocation, the port where the registry is running is always 1099 for the process where the ManagementServer-object is running. While the class UserManager is used to monitor a user’s login-attempt and to get his/her access level (admin or user), the class ConnectionManager monitors connections originating from Audit-Clients and generates unique ids (instances of class SessionID) for them. Finally, class ProcessManager is responsible for creating and monitoring Dataflow-processes and to do the Inter-Process-Communication (IPC) with them. Note: The class ManagementServer (its main-method) must be run to start the whole

server.


Page 52 of 62

7.4.2 The class ‚UserManager’

The class UserManager, as its name indicates, deals with user management issues. It is responsible for querying the user database in the first place but in fact is introduced only to decouple class ManagementServer from the database functionality needed. The ManagementServer uses UserManager’s login()-method to determine if a user is allowed to send commands to the server and to get the access level (admin or user). This prevents calls from unauthorized clients from being handled and results in some kind of a protection mechanism. To be as database-provider-independent as possible, we generated an interface which defines only those methods a database provider must expose. In our first approach, we integrated a database provider which simply uses a file to store user information:

UserDBFile

-filename: String-users: HashMap+UserDBFile(zipit: boolean)+getUsers(): Vector+exists(user: String)+addUser(user: String)+rem oveUser(user: String)+checkAccess(user: String, pwd: byte[])+changePwd(user: String, oldp: byte[], newp:byte[])+getAccessLevel(user: String): int+changeAccessLevel(user: String, level: int)

UserM anager

-db: IUserDB+UserManager(provider: String)+login(user: String, pwd: byte[])+isAdm in(user)+changePassword(user: String, oldp: byte[], newp: byte[])

UserInfo+nam e: String+pwd: byte[]+accesslevel: int+UserInfo(nam e: String, pwd: byte[], level: int)

IUserDB

+getUsers(): Vector+exists(user: String)+addUser(user: String)+removeUser(user: String)+checkAccess(user: String, pwd: byte[])+changePwd(user: String, oldp: byte[], newp: byte[])+getAccessLevel(user: String): int+changeAccessLevel(user: String, level: int)

10..*

Figure 32: class diagram of package ‘dics.management.user’

The class UserDBFile uses a file (the file ‘user.db’ in the root directory of this software is currently hard-coded #) to store user information. This is accomplished by directly ‘serializing’ the hashmap of UserInfo-structures to the file using Java’s ObjectOutputStream. To avoid saving passwords in plain text and to (supposedly) challenge ourselves, we decided to only work with values generated with the Secure Hash Algorithm (SHA) out of the passwords in plain text. That’s the reason why you see the type byte[] anywhere our Java-code deals with passwords. The following code-excerpt demonstrates how SHA may be used to obtain hash values out of plain text passwords: Listing 7.4.1: How to calculate hash values using SHA in Java 1 MessageDigest md = MessageDigest.getInstance("SHA"); 2 byte[] sign = md.digest(pwd.getBytes("UTF-16 "); To do the calculation, a byte array must be passed to the digest()-method of class MessageDigest. A String can be transformed to a byte array using method getBytes().


Page 53 of 62

So much for the challenge…

7.4.3 The class ‚ConnectionManager’

This system maintains a lot of network connections. Some of them are used to control the system’s behavior while others are used to distribute log-messages to users (see chapter 0, The class ‘LogListenerModule’). The class which is monitoring the connections originating from Audit-Clients is class ConnectionManager. This class manages sessions, handles server-side connection timeouts and maintains a list of connections waiting for a specific Dataflow to be started. These waiting connections are passed over to ProcessManager by the ManagementServer as soon as a Dataflow is started.

ConnectionM anager

-sessions: Hashtable-timestam ps: Hashtable

+ConnectionManager()+startSession(ci: ConnectionInfo)+checkSession(ci: ConnectionInfo)+refreshSession(ci: ConnectionInfo)+ping(ci: ConnectionInfo)+rem ove(ci: ConnectionInfo)+getConnections()+getW aitingConnections()+addW aitingConnection(...)+rem oveW aitingConnection(...)

1

SessionID

-id: int-type: int-owner: String

+SessionID(user: String, type: int)+getID(): int+getType(): int+getOwner(): int

0..*

ConnectionInfo

-server: String-port: int-service: String-owner: String-filter: FilterContainer+ConnectionInfo(...)+getPort(): int+getIP(): String+setFilter(flt: FilterContainer)+getFilter(): FilterContainer+getOwner(): int

0..*

1

Figure 33: class diagram of ‘ConnectionManager’ and some related classes

Class SessionID contains the kind of information usually encountered in the area of session management. Here you find the name of the user starting the session (the owner), the type of the session and a unique identification number (the ‘id’). This class is to be considered ‘another convenience class’ since the data it encapsulates and its functionality could also have been implemented with another approach. However, session management is realized inside ConnectionManager by a hash table associating ConnectionInfo-objects with SessionID-objects. A method is provided to test the validity of a given ConnectionInfo-object (method checkSession()). The class ConnectionInfo has various purposes. First, it encapsulates information about where (which means on what port number) Audit-Clients are listening for log-messages. To go on, it constructs and exposes a unique identification for each such log-message listening object as described in chapters 5.1.2, Port allocation and 6.3.3, RMI object identification.

7.4.4 The class ‚ProcessManager’

This class resides in the sub-package process of package dics.management and collaborates with some other classes in the following way (simplified):


Page 54 of 62

pc : ProcessControllerpc : ProcessControllerpc : ProcessController

Dataflow ProcessDataflow ProcessDataflow Process

processManager : ProcessManager

pc : ProcessController

DataFlowManager

Server Process

Dataflow Processes

send control commands

send control commands & monitor process

ManagementServer

Figure 34: collaboration diagram of package ‘dics.management.process’

The Process Manager, the single instance of class ProcessManager, creates an object to start and monitor each Dataflow-process. This object is of type ProcessController, which is a small inner class of ProcessManager. It is derived from Thread and therefore returns immediately after instantiation. This way, the Process Manager is not blocked and may simply call the controller thread’s start message to get it running. Otherwise, the Process Manager would be blocked for several seconds, since a Dataflow-process may take a while to start up (a whole new JVM is started). The following class diagram shows some more details about class ProcessManager:

ProcessM anager

+SERVER_PORT: int-runningDataFlows: Hashtable-cm: ComponentManager+ProcessManager(props: Properties)...

IProcessManagerextends java.rm i.Remote

+processReady(dataflow: String, port: int)+processEnd(dataflow)

1

ProcessController

-dataflow: String-process: Process-owner: String

+ProcessController(...)+run()

0..*

ProcessInfo

-dataflow: String-port: int-status: AbstractStatus-dfm : IDataFlowManager

+ProcessInfo(...)+lookupDataFlowManager()

IDataFlowManagerextends java.rm i.Remote

+addUser(ci: ConnectionInfo)+removeUser(ci: ConnectionInfo)+setFilter(ci: ConnectionInfo)+stop(ci: ConnectionInfo)+abort(ci: ConnectionInfo)

Figure 35: simplified class diagram of class ‘ProcessManager’ and its collaborators


Page 55 of 62

As you can see in the diagram above, some information about a Dataflow-process is encapsulated inside an instance of class ProcessInfo which is created in a process controller’s constructor. Once the controller thread is started, a new Dataflow-process is created through a call to Runtime.getRuntime().exec(). The creation of a Dataflow-process is an asynchronous operation, since no mechanism exists in Java which would make it possible to be notified when such a process becomes ready. Therefore, we had to implement our own synchronization-method: If a Dataflow-process is ready, it notifies its controller by sending it a message. This is done by calling the method processReady() of the ProcessManager-object (which is also of type IProcessManager) obtained with a lookup in the RMI-registry on port 1099. You can read more about this in the next chapter. Note: All the running instances of ProcessInfo / ProcessController (and therewith all the

running Dataflow-processes) are stored in a hashtable. If an Audit-Client wants to start a Dataflow, the ProcessManager first tries to look it up in this table. If it points out to be already running, the client’s command is blocked, since the same Dataflow is not allowed to be started more than once. The reason for this lies in the unpredictability of the results while concurrently accessing the same data source or destination.

7.4.5 The class ‘DataFlowManager’

Class DataFlowManager is a wrapper around the original ICS-Dataflow-class and provides the functionality needed to control a Dataflow-process out of the server-process. It is the access point for Inter Process Communication (IPC).

DataFlowM anager

+SERVER_PORT: int-runningDataFlows: Hashtable-cm: ComponentManager+DataFlowManager(props: Properties)+runDataFlow()+stop(ci: ConnectionInfo)+abort(ci: ConnectionInfo)+pause(ci: ConnectionInfo)+join(ci: ConnectionInfo)+setFilter(ci: ConnectionInfo)+addUser(ci: ConnectionInfo)+rem oveUser(ci: ConnectionInfo)

IProcessManagerextends java.rm i.Remote

+processReady(dataflow: String, port: int)+processEnd(dataflow)

IDataFlowManagerextends java.rm i.Remote

+addUser(ci: ConnectionInfo)+removeUser(ci: ConnectionInfo)+setFilter(ci: ConnectionInfo)+stop(ci: ConnectionInfo)+abort(ci: ConnectionInfo)

IDataFlow

+start()+stop()+abort()+pause()+join()

Figure 36: simplified class diagram of class ‘DataFlowManager’ and its collaborators As you can read in chapters 5.1.1, Flow of data and 7.4.4, The class ‚ProcessManager’, the connection between a DataFlowManager and the ProcessManager is of bi-directional nature. This means that on the one hand, a DataFlowManager can send messages to (call methods of) the ProcessManager but on the other hand, the ProcessManager can also send messages to a DataFlowManager.


Page 56 of 62

Let’s review the sequence diagram of chapter 6.4.2, The interface ‘IProcessManager’, which shows some details about the lifetime of an instance of DataFlowManager and how some method calls are made:

ProcessManager

DataFlowManagerexecute (command line)

signal readyness / tell port

get waiting connections

DataFlowload

start()

control calls [connect/stop/abort/pause/setFilter]

end

signal end

serv

er-p

roce

ss

data

flow

-pro

cess

Figure 37: sequence diagram: lifetime of an instance of class ‘DataFlowManager’

As you can see in the above figure, a Dataflow Manager informs the Process Manager that it has been properly loaded. After having actually started the Dataflow itself, a communications port is opened to which commands may be sent by the Process Manager (grey lines in the diagram).

7.5 The package ‘test’

This package contains only one file, the class AllTests. This is simply a helper class which contains a method used by the testing framework JUnit. The mentioned method constructs a test-suite and returns it to the framework. JUnit is a simple but widely used framework to write repeatable tests. Its purpose is to get immediate feedback on written code and is usually used where the programming style "code a little, test a little, code a little, test a little, …" is practiced. We found an interesting article on the web, which very accurately describes this programming style in only one sentence:

“The style here is to write a few lines of code, then a test that should run, or even better, to write a test that won't run, then write the code that will make it run.”

Taken from the article ‘Test Infected: Programmers love writing tests’ found at http://junit.sourceforge.net/doc/testinfected/testing.htm

For more information on JUnit, please refer to http://www.junit.org or http://junit.sourceforge.net.


Page 57 of 62

7.6 The package ‘util’

This package contains utility classes out of various areas. Here’s a list of them and their purposes:

Class Purpose

DataFlowNaming Contains functions to convert Dataflow names to their fully qualified representation. See chapter 5.3.2, Dataflow identification, for more information on this.

Encryption Contains a method to calculate hash values out of the bytes of strings. Used to prevent passwords from being transported over the network in plain text. See chapter 0,

The class ‚UserManager’, for more information.

HashtableWithVector Helper class to associate a vector of objects with another object. Based on class Hashtable.

RMIConnection Helper class to asynchronously open a connection to an RMI-server object. Interface IRMIConnection must be implemented by classes using it. See chapter 6.1, Defining RMI-interfaces, for details about RMI.

IRMIConnection Contains the method signalConnected() which hands over a reference to the remote object to a client object using RMIConnection.

RMIHelper Provides methods all around RMI: start a registry, bind objects to a registry and look up objects in a registry.

RollingList A linked list which is limited in its size. If the maximum allowable size is reached, the oldest elements will be thrown away.

SimpleTimer Provides a simple timer mechanism, as its name suggests.

TimeMeasuring Offers the functionality of a stop watch.


Page 58 of 62

7.7 Performance issues

Our first version of the Audit-Client (GUI) had a significant performance issue when a lot of messages (e.g. trace) had to be written to log windows. When started on the same computer, the processor usage for the GUI reached almost 100%, while the corresponding Dataflow-process demanded just about 2%! Therefore we had to make some measurements to localize the bottleneck. At first sight, the ‘refresh mechanism’ of the log window looked suspicious. To verify this, we measured the execution time of a simple Dataflow with 10’000 entries. We ran the same test various times and with some slight differences. First, we ran it once with the log window in maximized and then in minimized state. Afterwards, we disabled the printMessage() method once on client-side and then on server-side. Finally, we chose different log-message levels.. The following table contains the results:

Log Window maximized Message, Debug, Trace 2381s Message 268s - 12s Log Window minimized Message, Debug, Trace 170s Message 30s - 12s printMessage disabled (client-side) Message, Debug, Trace 39s Message 28s - 12s printMessage disabled (server-side) Message, Debug, Trace 20s Message 14s - 12s

Table 1: Performance measurements

Note: All tests were carried out in the same environment: A computer with a Pentium II-

processor and with 256 MByte of RAM. These tests were sufficient to prove the ‘refresh mechanism’ to be the sinner: When the log-window was maximized, it took more than 12 times the execution-time as with a minimized window! The time difference between the ‘client-side-‘ and ‘server-side tests’ was much smaller. A glance at our code disclosed the exact location of the problem: Every time a new message was added to the table, its renderer refreshed the model and repainted the actual view. This pointed out to be a very time-consuming way of updating the view.


Page 59 of 62

To cure this behavior, we decided to decouple the arrival of a log-message from its visual representation and thereby decreased the refresh rate of the view to about half a second. We introduced a thread, which controls the refresh-rate of the table. The following measurements show that we were on the right track: The execution time now was almost 15 times smaller than before and the processor usage declined to a value of around 50%:

before Tuning: after Tuning: Benefit: Log Window maximized Message, Debug, Trace 2381s 147s 1520% Message 268s 29s 824% - 12s 10s 20% Log Window minimized Message, Debug, Trace 170s 140s 21% Message 30s 25s 20% - 12s 10s 20% printMessage disabled (client-side) Message, Debug, Trace 39s 36s 21% Message 28s 24s 17% - 12s 10s 20% printMessage disabled (server-side) Message, Debug, Trace 20s 18s 11% Message 14s 13s 8% - 12s 10s 20%

Table 2: Performance measurements after tuning

Note: Our decision made sense, anyway, because it’s impossible for human eyes to follow an

actively scrolling text at a rate of much more than 10 lines per second.

Page 60 of 62

Appendix A Bibliography [DOC1] “Refactoring - Improving the Design of Existing Code” 2000

Martin Fowler, Addison Wesley ISBN 0-201-48567-2

[DOC2] “Design Patterns – Elements of Reusable OO-Software” 2000 Erich Gamma et al., Addison Wesley ISBN 0-201-63361-2

[DOC3] “Planning eXtreme Programming” 2001 Kent Beck / Martin Fowler, Addison Wesley ISBN 0-201-71091-9

[DOC4] “Java Network Programming” 2000 Elliotte Rusty Harold, O’Reilly & Associates ISBN 1-56592-870-9

[DOC5] “Client/Server Programming with Java and CORBA” 1998 Robert Orfali / Dan Harkey, Wiley & Sons ISBN 0-471-24578-X

[URL1] http://java.sun.com/products/java-media/jsdt/

information on the ‘Java Shared Data Toolkit’-library [URL2] http://java.sun.com/products/jms/

information on the ‘Java Message Service’-API [URL3] http://java.sun.com/j2se/1.3/docs/api/index.html

Java API-documentation of JDK 1.3 [URL4] http://www.cit.gu.edu.au/teaching/CIT3190/documentation/sdd.html

info on IEEE 1016 – Recommended Practice for Software Design Descriptions

[URL5] http://www.cs.uidaho.edu/~cs480/SDDGL.htm info on IEEE 1016 – Recommended Practice for Software Design Descriptions

[URL6] http://www.dur.ac.uk/~dcs8s00/phases/require.html info on IEEE 830 - Recommended Practice for Software Requirements Specifications

Page 61 of 62

Appendix B Index

A administrator................................................5 ANSI-C......................................................24 API...............................................................3 asynchronous .............................................28 Audit client ................................................19 Audit Client ...............................................18 Audit Client Controller..............................15 Audit-Client.........................................15, 19

B bottleneck ..................................................17

C CD ...............................................................2 class

AbstractFilter.........................................49 ConnectionManager ..............................53 DataFlowManager .................................55 DataFlowNaming ..................................57 DistributedMessageLogging..............7, 46 Encryption .............................................57 FifoBuffer ..............................................48 FilterContainer.......................................49 HashtableWithVector ............................57 IRMIConnection....................................57 LogListenerModule ...............................48 ManagementServer................................51 ProcessController ..................................54 ProcessInfo ............................................55 ProcessManager.....................................54 RMIConnection .....................................57 RMIHelper.............................................57 RollingList.............................................57 SimpleTimer ..........................................57 TimeMeasuring......................................57 UserManager ...................................52, 57

Client ...........................................................3 client-object ...............................................28 Configuration.............................................45 Connection Information.............................21 Connection Manager .................................14 ConnectionManager ..................................53 CORBA .....................................................24

D Dataflow ......................................... 3, 12, 21

fully qualified name.............................. 22 identification......................................... 22 process .................................................. 13 Process .......................................... 3, 5, 18

Dataflow Manager .............................. 14, 19 DataFlowManager .................................... 55 Dataflow-process ................................ 21, 58 distributed ................................................. 28 Distributed ICS ........................................... 2 DistributedMessageLogging................. 7, 46 Distributed-Message-Logging .................. 14

E Extreme Programming................................ 8

F Filter............................................................ 3 Filter Container......................................... 22 Filtering .................................................... 49 flow of data......................................... 11, 17

G GUI ........................................................... 38 guidelines.................................................... 8

I ICS ....................See Inter Connection Server IDataFlowManager ................................... 35 ID-problem ......................................... 12, 21 IEEE 1016................................................... 2 ILogMessageReceiver .............................. 35 IManagementServer.................................. 31 Inter Connection Server.......................... 2, 7 interface .................................................... 24

RMI....................................................... 31 Internationalization................................... 43 IP-address ................................................. 21 IPC.............................................................. 3 IProcessManager....................................... 34

Software Design Description Appendix B: Index

Page 62 of 62

J Java..............................................................7 Java Actions...............................................40 JInternalFrame...........................................41 JMS............................................................25 JRE ..............................................................8 JSDT..........................................................26 JTree ..........................................................42 JVM.............................................................3

L Linux ...........................................................7 LogListenerModule .............................20, 48 Log-Message Receiver ..............................15 Log-message Writer ..................................19 Log-Message Writer............................14, 20 log-messages..............................................17

M Management Proxy....................................15 Management Server.............................14, 19 Management Server...................................18 ManagementServer....................................51 Message digest.............................................3 Message level ..............................................3 MessageLogging....................................7, 12 Meta-Logging ............................................14 Microsoft Windows.....................................7 Model-View-Controller.............................39 Multicast ..............................................23, 27 multicast groups.........................................28

N network architecture ..................................24

O Owner ........................................................21

P package

audit .......................................................38 logging...................................................46 management...........................................50

test......................................................... 56 util......................................................... 57

Package structure ...................................... 37 Pair programming ....................................... 9 Performance........................................ 36, 58 Polling......................................................... 3 Port........................................................ 3, 22 process ...................................................... 13 Process ........................................................ 3 Process Manager....................................... 14 ProcessManager........................................ 53 Project-plan................................................. 4

R Reference .................................................... 3 RMI................................................. 3, 25, 28

object identification .............................. 30 RPC........................................................... 24

S Server.......................................................... 3

Process ................................................ 3, 5 server-object ............................................. 28 Service identification................................ 22 Singleton..................................................... 3 Sockets...................................................... 26 Software Requirements Specification..... 2, 4 Stand up meeting ........................................ 9 subsystems ................................................ 11 System Monitoring ................................... 14 system overview ......................................... 5

T Treeview ................................................... 42 Type .......................................................... 22

U unidirectional ............................................ 18 unit test ....................................................... 9 User Manager ........................................... 14 UserManager ............................................ 52

X XP .......................See Extreme Programming

Installation guide & User's manual

Dis

trib

ute

d IC

S D

istr

ibu

ted

ICS



DATE 23-Oct-2001 VERSION 1.2 FILE Installation & User Manual 1.2.doc

Version history:


1.0 First Draft F. Soldati, M. Seelhofer 23-Oct-2001

1.1 Added Installation guide F. Soldati, M. Seelhofer 26-Oct-2001

1.2 Reformatting M. Seelhofer 27-Oct-2001

Table of contents

Chapter 1 Installation 1 1.1 System Requirements 1 1.2 Installation instructions 1

1.2.1 Classpath 2

Chapter 2 User’s manual 3 2.1 Overview 3 2.2 Management Server 3 2.3 Console client 4 2.4 GUI client 5 2.5 Startup and login 5

2.5.1 Main window 5

2.5.2 Control the dataflows 6

2.5.3 Open a log-message window 6

2.5.4 Log-Message Window 7

2.5.5 Setting a filter 7

2.6 User Management 9

Installation guide & User's manual Chapter 1: Installation

Page 1 of 9

Chapter 1 Installation

1.1 System Requirements

As the distributed ICS uses Java 2, your system should fulfill the following criteria, which are taken from the Java 2 system requirements specification:

“The Java 2 run-time environment (JRE) is intended for use on the Windows 95, Windows 98, Windows NT 4.0 with Service Pack 5, Windows 2000; Linux kernel v 2.2.12 and glibc v2.1.2-11 or later; Solaris 2.6, Solaris 7, and Solaris 8 operating environments.”

A Pentium 166MHz or faster processor with at least 64 megabytes of physical RAM is required to run the graphically based Audit Client. You should have at least 60 megabytes of free disk space.

Note: The test-installation on the supplemented CD is fully Windows-oriented. You’ll have to install the system manually and write your own start-scripts if you want to test it on a system not running one of the operating systems in the Windows9x/NT-family.

1.2 Installation instructions

The installation is quite simple. Just start the install batch file from the root directory of the CD: install.bat c:\program files\dics

This script will copy all needed files to the directory specified. The copied directory structure is:

The root directory Java Runtime Environment Binaries for JRE Libraries for JRE Project directory of distributed ICS Batch flies for controlling the programs Additional batch files (not often used) Binaries for distributed ICS Class files from distributed ICS Class files from ICS Images and icon for the ICS Manager Resources (Icons and messagebundles) for the Audit Client Some test files for checking the functionality Third party jar files

Figure 1: directory structure

The batch file also installs some test files. These are located in the ‘test’-directory and include some databases (in plain ASCII) and a simple ICS configuration file. The current configuration uses these files for testing purposes. So if you like to use your own files you have to adapt the batch files (See chapter 2.2 for the syntax).

Installation guide & User's manual Chapter 1: Installation

Page 2 of 9

After installation, you can start the programs through the batch-files in ‘dics\Project\Batch’:

Batch-File Purpose

ManagementServer.bat Starts the management server with the configuration saved in the test directory.

StartClient.bat Starts the GUI client. UserAdministration.bat Starts the User Manager.

1.2.1 Classpath

The ‘classpath’-environment variable is often a source of errors and confusion when using Java. Therefore, make sure its settings are correct and that all the Jar-files are included.

In our case, the Jar-files are located in the directory ‘dics\project\bin\thirdparty’. They all have to be included in order for the system to work!

You can use the config.bat script form the batch directory to set the system environment variables.

Installation guide & User's manual Chapter 2: User’s manual

Page 3 of 9

Chapter 2 User’s manual

2.1 Overview

The distributed ICS system is using a usual client-/server-architecture (see Figure 2) and provides two kinds of applications:

• the Management Server (or simply: the Server) controls all dataflows.

• the Audit Client may start, stop, abort and pause a dataflow. Additionally, it can be used to monitor log-messages of a Dataflow running on the server. Apart from the easy-to-use graphical user interface, there exists a rudimentary console client.

controls Dataflow A

controls Dataflow B

User AUser B

Audit Clients

- Control Dataflows- Control Logging- Poll for Status

Managem ent Server



Adm inistrator



Figure 2: Overview

In the following chapters you will find a description of these applications.

2.2 Management Server

To be able to control the Dataflows you must first start the Management Server. This can be done through console input or by using a script (batch-file). The Management Server can be started using the following format:

java [classname] [configfile]

Example: java dics.management.ManagementServer test\newconfig.ics

Attributes: [classname] the application class (dics.management.ManagementServer). [configfile] the configuration file with the ICS dataflows. This file should be

created with the ICS Manager. For testing purposes you may use the newconfig.ics configuration file which is located in the test directory.


Page 4 of 9

Note: The Management Server must have access to the file ‘user.db‘ (see chapter 2.6 ‘User

Management’). This file has to be in the working directory.

2.3 Console client

The console client allows you to control a Dataflow over the system console. The command line input expects one of the following formats:

java [classname] [server] [user] [password] [command]

java [classname] [server] [user] [password] [command] [dataflowName]

java [classname] [server] [user] [password] [command] [dataflowName] [level]

Examples: java dics.audit.AuditClient localhost johndoe "" start LongFlow

java dics.audit.AuditClient localhost johndoe "" connect LongFlow debug

Attributes: [classname] the application class (dics.audit.AuditClient).

[server] the ip or hostname of the management server.

[user] the user name.

[password] the password.

[command] can be one of the following: start - starts a dataflow * stop - stops a dataflow * connect - allows to listen to log-messages from a dataflow * list - prints a list with all dataflows

[dataflowName] the name of the Dataflow.

[level] can be one of the following: message - prints only messages of type message. warning - prints only messages of type warning. error - prints only messages of type error. trace - prints only messages of type trace. debug - prints only messages of type debug.

* Works only if the attribute [DataflowName] is set.


Page 5 of 9

2.4 GUI client

The GUI client is much more powerful and user friendly than his console pendant. The following chapters describe the most important functions of the GUI.

2.5 Startup and login

On startup, a login dialog appears. Here you must enter the hostname or IP of the computer on which the management server is running, your username and your password. With the change button, you may change your password.

Here you may change the password

Figure 3: Login dialog

2.5.1 Main window

After a successful login procedure the main window will come in to view:

Selects thedataflows

Controls the dataflows

Figure 4: Main window


Page 6 of 9

2.5.2 Control the dataflows

To control a dataflow you must add a new tabbed pane for it on the bottom. This can be done by double-clicking a dataflow directly in the tree or by using the Select dataflow dialog which can be reached through menu control - dataflow. Having done this, you can control the dataflow directly with the buttons on the tabbed pane:

start pause

restart

stop

abort

Figure 5: Controlling a dataflow (left: stopped / right: running)

2.5.3 Open a log-message window

To open a new log-message window you can select the ‘new...’-item from the window menu or the load… item to load the settings from a file. In the window settings dialog you can also load a configuration-file and of course save the settings.

If you want messages from one dataflow only, you must select the dataflow name and load a filter by pressing the filter settings button (see chapter 2.5.5). You can also listen to more flows by adding new rows.

If you don’t choose a filter, you will only see messages of type ‘message‘.

Deletes a row

Adds a new row

Opens the filter settings dialog

Selects a dataflow

Saves the settings

Loads settingsClears all

Figure 6: Log window settings


Page 7 of 9

2.5.4 Log-Message Window

When everything is okay, a new internal window appears, which displays all the new messages according to your settings. Figure 7 shows a window with some messages:

Figure 7: Log-Message Window

You have the possibility to save this data to a CSV file (Comma Separated Values) by using the ‘save as...’ item from the file menu. CSV is a common file type, which is supported by many other applications (e.g. by Microsoft Excel).

You can also load a previously saved CSV file into the Audit-Client. Select a valid CSV file in the dialog which pops up after selecting the ‘open...’ item from the file menu and a new Window with the messages will appear

2.5.5 Setting a filter

You can select the „filter settings“ item in the menu window to apply any kind of filter criteria. The appearing dialog resembles the window settings dialog. You can use it to alter your settings. You may also save them to a file or open an existing one, as you can see in the screenshot below.

The dialog allows you to choose the message levels you would like to receive. Additionally, you may specify a string which is used to look for log-messages containing it. You may even use regular expressions to specify a more complex criteria (see the examples at the end of this chapter).


Page 8 of 9

Message level settings

Sets the message type

Enables regular expressions

Defines a filter pattern

Figure 8: Filter settings

Regular Expressions

Regular expressions allow you to apply very powerful filters. The following table lists the most useful expressions you can use inside a regular expression: See also http://jakarta.apache.org/regexp/apidocs/org/apache/regexp/RE.html

Predefined Clauses . Matches any character other than newline \w Matches a "word" character (alphanumeric plus "_") \W Matches a non-word character \s Matches a whitespace character \S Matches a non-whitespace character \d Matches a digit character \D Matches a non-digit character Boundary Matchers ^ Matches only at the beginning of a line $ Matches only at the end of a line \b Matches only at a word boundary \B Matches only at a non-word boundary Boundary Matchers A* Matches A 0 or more times (greedy) A+ Matches A 1 or more times (greedy) A? Matches A 1 or 0 times (greedy) A{n} Matches A exactly n times (greedy) A{n,} Matches A at least n times (greedy) A{n,m} Matches A at least n but not more than m times (greedy) Boundary Matchers AB Matches A followed by B A|B Matches either A or B (A) Used for subexpression grouping


Page 9 of 9

Examples without regular expressions:

MessageType: Father Pattern: FilePeer → Lists only messages originating from the ‘FilePeer’-ICS-component.

MessageType: Message Pattern: Data → Lists only messages that start with the word ‘Data’.

Example with regular expressions:

MessageType: Father Pattern: (Input)|(Output)Peer → Lists only messages from ‘Input’- and ‘OutputPeers’

2.6 User Management

The GUI for the User Management is almost self explaining. On startup the application checks for the file ‘user.db’ in the current working directory. If it doesn’t exist a new (and empty) file will be created. Afterwards, you may add and remove users as you like (see Figure 9).

Figure 9: User Management