Max OngUniversity of Sheffield, UK.

AHM 2004 Session 2.3: Workflow Composition, Wednesday 1st September 2004, 4pm.

Workflow Advisor in DAMEWorkflow Advisor in DAME

Abstract. There are an increasingly large number of web services available today. New services are constantly being introduced, each performing different tasks ranging from simple data transfer operations to highly complex engineering and mathematical processes. There is an emerging need for the ability to dynamically discover how available services, resources and data could be utilised not only to process a task, but to achieve the desired outcome in the most suitable manner. This paper investigates the emerging need for workflow advice to aid decision support on the Grid and proposes a workflow advice system that incorporates a reasoning system designed to perform such a task. The case for a Workflow Advisor system is explored within the context of advanced aero engine fault diagnostics in the UK e-Science Grid project, DAME.

• UK E-Science pilot Grid project, funded by EPSRC• Develop a Grid enabled diagnostic system• Demonstrate this on the Rolls-Royce aero engine diagnostics

problem• Increasing amount of engine data being collected

• New aero engines are being instrumented with engine monitoring units possessing significantly greater capability to record and analyse data• Each engine on a civil airliner could generate about 1Gbyte of data per flight.• Rolls-Royce currently has over 50,000 engines in service with total operations of around 10M flying hours per month.

• The key objectives are to:• reduce delays and cost of ownership for the aircraft.• provide the infrastructure to manage the large amounts of data,• perform compute-intensive analysis and modelling to identify abnormal events that have occurred• identify the potential faults that require maintenance to prevent failures and aircraft downtime

Distributed Aircraft Maintenance Environment (DAME)

Intelligent Workflows

New Web Services are constantly being introduced every day, each performing different tasks ranging from simple data transfer operations to highly complex engineering and mathematical processes.

Open standards have enabled inter-operability between resources available across the virtual Grid computing environment.

However, there is an emerging need for the ability to dynamically discover how these available services, resources and data could be best utilised and integrated not only to process a task, but to achieve the desired outcome in the most suitable manner.

Example scenario where Workflow Advisor is used

Key challenges for Workflow Advisor:

•To provide the best advice on constructing complex workflows using the available Grid-enabled resources to complete a desired task

•To provide the underlying knowledge capture and knowledge-based techniques for search and reasoning across complex workflow information to support that advice.

Workflow composition is crucial

• Selection and composition of individual services into a set of workflow processes is crucial to perform a task correctly

• There may exist more than one way to perform a task. It would be ideal to be able to explore every possibility and compare all end results.

• However, given the time-constraints, human-resource factors and costs involved in a real-world application, it is highly advantageous to have an intelligent system that can:

• Provide user knowledge of how available services can be utilised to achieve a desired objective beforehand

• Reduce the time taken to complete a job

• Advise on how best to utilise an available resource

•This is particularly important when:

• User is unfamiliar with available services and resources but needs to perform a task

• A particular service or resource is scarce or expensive

• A particular process is time-consuming or not readily available

TR AData X TR B

AnalyseData

VisualiseResult

Straightforward Workflow Process with known transformations

AnalysisA

Data X

AnalysisB

FindMatch

RetrieveEventHistory

CompareResults

AnalysisC

ProposeAction

RetrieveMetadata

VisualiseResults

Knowledge

No novelty detected.Try something else.

Knowledge

No novelty detected.Try something else.

Knowledge

Analysis C showed significantnovelty in the data.

Knowledge

Novelty confirmed

Knowledge

Novelty can be spottedusing visualisation tool.

Complex Workflow Processes with multiple transformations - use CBR knowledge-based search techniques

Intelligent Workflow Advice in DAME

• Make use of “intelligence” to guide users through workflows in complex systems.

• In DAME sophisticated, but generic, CBR tools are used to yield a ranked analysis of relevant case histories followed by a rule-base which guides the aircraft maintenance engineer through a set of actions.

• Capturing domain knowledge and evaluation of workflow results is the key to creating suitable new workflows for new tasks.

• The service-oriented, collaborative Workflow Advisor developed in DAME allows new forms of advice and guidance to be offered to users.

• Workflow Advisor facilitates personalisation of workflow composition by individual users as well as allows collaboration between users in the problem-solving process.

Interaction between Workflow Advisor, workflow management and provenance in DAME

• Information can be analysed by a multitude of tools

• Not only the collaboration of engine experts may be of value, but also the capturing of wide ranging knowledge and experience within the aircraft maintenance environment

DAME Workflow Advisor Implementation

• Demonstrator for aircraft engine “birdstrike” scenario in DAME portal

• Service oriented architecture, accessible via Web browser over Internet

• Closely coupled relationship between worfklow advice, provenance and workflow management

• Workflow “Cases” captured in provenance data in DAME

• In Workflow Advisor, CBR technology is used to perform knowledge-based search across that data

• New workflows to be composed by adapting previous successful workflow cases and executed for current task via DAME Workflow Manager

• Understanding of services available in the Grid environment is required

• On-line DAME Grid PortalOn-line DAME Grid Portal• Arrival of aircrafts at airportArrival of aircrafts at airport• Maintenance Analyst WindowMaintenance Analyst Window

• DAME diagnostic workbenchDAME diagnostic workbench• Downloadable vibration dataDownloadable vibration data• Vibration AnalysisVibration Analysis• Engine data pattern matchingEngine data pattern matching• Engine model simulationEngine model simulation• CBR Maintenance AnalysisCBR Maintenance Analysis

Introduction to CBR

• CBR is a mature, low-risk subfield of A.I.

• Primary knowledge source is

• a memory of stored cases recording specific prior episodes

• not generalised rules

• New solutions generated by adapting relevant cases from memory to suit new situations

Retrieve

Propose Solution

Adapt Justify

Criticize

Evaluate

Store

Case-Based Reasoning (CBR)

Case Representation

What is a case?

“A case is a contextualised piece of knowledge representing an experience that teaches a lesson fundamental to achieving the goals of the reasoner.” (Kolodner, 1993)

Cases link together knowledge that belongs together:

Information concerning the fault

Response to the fault

Effects of those responses

Features & Functions

• CBR Search and Rank

• Enhanced nearest neighbour algorithm (kNN)

• Induction

• User-variable weightings

• Service Oriented Architecture

• Accessability

• Web browser-based access to CBR

• Web GUI functions for both search & authoring

• Dynamic

• Live on-line authoring environment

• Search across latest updated cases

• Flexible implementation

• Can be deployed as multiple services on Grid nodes

• Data to application or application to data

• Can also be run as batch job on local/Grid machine

Case Databases

Casebase Manager

Case-Based ReasoningCase-Based ReasoningEngine (API)Engine (API)

Grid/Web Service Interface

Grid/Web Service Clients(Web Browser)

Sheffield Univ CBR Technology

CaseRepresentation

Case Retrieval

Case Retainment Case Revision

Case Reuse

• Results of CBR search & rankResults of CBR search & rank

• View a case and it’s attachmentsView a case and it’s attachments

Sheffield University CBR Technology

Further Work

• Full integration with DAME Workflow Manager to compose and execute workflows

• Multiple service nodes for distributed CBR processing

• Mobile Grid SupportCBR

EngineCBR

Service Node

CBR BrokerService

CBREngine

CBR Service Node

CBREngine

CBR Service Node

CBREngine

CBR Service Node

Grid Computing Environment

Decision Support Services

Summary

• Emerging requirement for intelligent workflow advice in the increasingly complex Grid environment

• The key challenge is to provide best practice advice on constructing complex workflows to achieve a desired task

• Case-Based Reasoning (CBR) is used as the underlying technology for knowledge-based search and reasoning in the DAME Workflow Advisor

Visit the DAME Project

• Further information

• Live software demonstrations

• Exhibition times:

Wednesday 1st, 18:00-19:00 (EPSRC booth)

Thursday 2nd, 10:00-12:00 (EPSRC booth)

Thursday 13:00-18:00 (White Rose Grid booth-22)

Friday 3rd, 12:30-14:30 (EPSRC booth)