Poor-definition, Uncertainty and Human Factors - A Case for Interactive

Evolutionary Problem Reformulation?

I. C. Parmee

Advanced Computational Technologies, Exeter, UK. iparmee@ad-comtech.co.uk

Setting the Scene•Ill-definition, uncertainty and multiple objectives - primary characteristics of real-world decision-making processes.

• During initial stages little knowledge of problem at hand may be available.

• Primary task - improve problem definition in terms of variables, constraint and quantitative and qualitative objectives.

• Problem space can develop with information gained

• Dynamical process - optimisation plays a secondary role following establishment of well-defined problem domain.

• Presentation speculates upon role of evolutionary computing, complementary computational intelligence techniques and interactive systems that support problem definition

Conceptual Design

• Main area of interest - evolutionary engineering design particularly higher levels of design process

• Conceptualisation represents highly complex human-centred activity supported by basic machine-based models of the problem domain.

• Search across ill-defined space of possible solutions - fuzzy objective functions, vague concepts of structure of final solution.

• Solutions / partial solutions explored and assessed with regard to constraints and objectives considered relevant at that time.

• Heuristics, approximation and experimentation - major role

• Flexibility evident in establishment of domain bounds, objectives and constraints.

• Design environment evolves with the solutions as designer gains understanding of functional requirements and possible structures. •Simple human / computer-based models - largely qualitative in nature -utilised to establish initial direction.

• Decision-making environment characterised by uncertainty in terms of lack of available data and a poorly defined initial specification.

• Discovery and accumulation of knowledge appertaining to problem definition and objective preferences prevalent in highly dynamical human / machine-based process.

…

Quote from Goel:

…………...problem formulation and reformulation are integral parts of creative design. Designers’ understanding of a problem typically evolves during creative design processing. This evolution of problem understanding may lead to (possibly radical) changes in the problem and solution representations. [….] in creative design, knowledge needed to address a problem typically is not available in a form directly applicable to the problem. Instead, at least some of the needed knowledge has to be acquired from other knowledge sources, by analogical transfer from a different problem for example. […] creativity in design may occur in degrees, where the degree of creativity may depend upon the extent of problem and solution reformulation and the transfer of knowledge from different knowledge sources to the design problem.

Goel A. K., Design, Analogy and Creativity. IEEE Expert, Intelligent Systems and their Applications, 12(3). (1997) 62 – 70)

Changing Objectives During Decision-making • Discovery and knowledge accumulation aspects common across decision-making.

• Exploration will likely result in re-formulation of the problem domain through iterative search and analysis of identified solutions.

• For illustrative purposes consider a job-related relocation to a new city and the daunting problem of finding a family home:

• Initial investigation - identifying appropriate districts based upon criteria relating to:

• quality of local schools; •safety / security issues; • proximity to places of work, transport, highway networks, shopping and leisure facilities etc.;• average price and type of housing and overall environment.

• Other criteria relate directly to the ideal property e.g.:

• maximum cost;• number of bedrooms;• garden, garage, parking etc.

• Several criteria would be considered hard constraints (i.e. maximum cost) in the first instance.

• Decision-making team is the family - each probably rate the relative importance of the above criteria in a slightly different manner - opinions of each member will carry a varying degree of influence.

• Likely that initially there is a pretty clear vision of what the ideal property will look like and the preferred location.

Information Gathering

• Initial information gathering provides quantitative and qualitative data relating to location from wide variety of sources some reliable and some based upon hearsay.

•Gradually overall picture is established - results in elimination of some options and inclusion of new possibilities.

• New possible locations discovered during explorative trips to those already identified.

• Possible change in preferences relating to property type, style etc as new options arise

Concept of Compromise and Problem Re-definition

• As property details are gathered - likely apparent that ideal solution is hard to find - concept of compromise becomes a reality.

• Hard constraints may soften • objective preferences will constantly be discussed and re-defined in the light of accumulated knowledge regarding districts and property availability within them.

• Particular characteristics of areas initially thought unsuitable may suddenly appear attractive.

• Search concentration may shift with discovery that such areas have suitable properties within the pre-set price range.

• Initial hard constraint on max. price may soften as close to ideal properties in favoured locations become available. Other compromises are explored to accommodate increased costs.

• Process becomes uncertain mix of subjective / objective decisions as goal-posts move, objectives rapidly change in nature and external pressures (time constraints?) begin to take precedence.

• Quite probable that chosen home differs significantly from the one first envisaged e.g.

• Location is ideal - guest bedroom is sacrificed, garden is minute but the second car has to go.

• The period town-house has become a modern detached but the budget is intact.

• Route to work may be longer but property close to ideal at a good price in an up-and-coming neighbourhood has been found.

Problem Commonalities

• Although a seemingly simple problem overall search process is highly complex - uncertainty, compromise and problem re-definition inherent features.

• Although differing from commercial and industrial decision-making scenarios analogies are apparent.

• Much can be learnt much from everyday decision-making scenarios and this knowledge can be utilised when designing interactive evolutionary search environments that can support complex decision-making processes.

Knowledge Generation and Extraction

• Machine-based search and exploration environment that provides problem information to the designer / decision-making team is required

•. Processing of such information and discussion results in recognition of similarities with other problem areas and discovery of alternative approaches.

• Major characteristic of population-based search is the generation of much possibly relevant information most of which is discarded.

• Development of interactive systems supports capture of such information and utilisation in re-formulation of problem through application and integration of experiential knowledge.

• Can this knowledge be embedded in further evolutionary search relating to the re-defined problem?

• Re-definition of objectives / objective preferences important aspect of evolution of the problem space.

• Primary role of evolutionary machine-based search and exploration processes can be generation of information.

• Moves utilisation of EC away from application over set number of generations to a continuous exploratory process where changes to objectives, variable ranges and constraint based upon information generated.

• Results in a moving, evolving problem space where primary task is design of an optimal problem space

• Theme has been central to much previous work leading to establishment of an interactive evolutionary design system (IEDS) that supports relatively continuous, iterative user / evolutionary search process

Earlier Work

• Theme has been central to previous work - development of EC strategies relating to the higher levels of the design process has related to:• identification of high performance regions of complex

conceptual design space (vmCOGAs)

• identification of optimal alternative system configurations through utilisation of dual-agent strategies for search across mixed discrete / continuous decision hierarchies.

• Other work relates to the concurrent satisfaction of both quantitative and qualitative criteria through the integration of fuzzy rule bases with evolutionary search.

The Evolutionary Interactive Design System

• Requirement for system that supports on-line extraction of information that supports easily implemented change

• Investigation of various techniques that can be combined within an overall architecture.

• Satisfaction of multiple objectives (i.e.> 10) major requirement

• Objectives must be very flexible re preferences / weightings to allow exploration of problem domain - supports better understanding of complex interactions between variable space and objective space.

Decision-maker / Designer

Information GatheringProcessesCOGAsTaguchi

etc

Co-evolutionary / Stand-alone Multi-objective Processes

Linguistic Preferences / Objective Weighting

Components of the Interactive Evolutionary Design System

Interactive Evolutionary Design System

Scenario (A)Evolution

Machine-Based Agents

Rule-BasedPreferences

External Agents (Design Team)

On-line Database

Information gathering processes

Scenario (C)Evolution

Scenario (B)Evolution

Two modes of operation:

Mode 1: Much uncertainty re problem domain; coarse model of system under design; little knowledge of relative importance of objectives / constraints; prime variables or variable ranges.

Requirement: exploration of initial design space to gather information re above.

Method: introduce either single evolution relating to one objective or multiple evolutions each relating to differing objective

Extract optimal information during evolutionary process

Information Gathering via Cluster-oriented Genetic Algorithms (COGAs)

Developed to:

• Rapidly decompose complex conceptual design space into regions of high performance

• Support extraction of relevant design information from such regions through good solution cover.

• Provide a greater understanding of multi-objective interaction

• Indicate best direction during early stages of design

How?

• Highly explorative GA / GAs

• Solutions extracted and passed through Adaptive Filter

• Better solutions pass into Final Clustering Set - defines HP regions

Design Environment• Preliminary design of military air frames with BAE

• Uncertain requirements and fuzzy objectives - long gestation periods between initial design brief and realisation of the product.

changes in operational requirements and technolo-gical advances - responsive, highly flexible strategy required - design change and compromise inherent features

Design exploration leading to innovative and creative activity must be supported.

CAPS (Computer Aided Design Studies), a BAE suite of preliminary design models utilised to support airframe design.

MiniCAPS - much abridged version of CAPS used for experimentation purposes - retains major characteristics of overall requirements.

9 input variables, eleven outputs relating to a range of objectives.

Application of COGA to Preliminary Airframe Design

1 2

3 4

Figures 1 to 4 show the effect of increasing the filter threshold setting. Low settings of figure 1 result in a large cluster of medium fitness solutions increasing the filter setting results in the identification of the two disjoint clusters of figure 4.

COGA applied to differing internal geometries of turbine

cooling hole problem

High-performance regions relating to various objectives

Lines define boundaries of the high performance regions for each objective - shaded area defines common region containing HP solutions that satisfy more than one objective. (a) Common region containing high performance solutions for Ferry Range and Turn Rate identified but Specific Excess Power(SEP)cannot be satisfied.; (b)Relaxing filter threshold for SEP allows lower fitness SEP solutions through, boundary moves towards feasible region; (c) Further relaxation results in the identification of a feasible region for all objectives.

a b c

Mode 2:

Having established better understanding of design domain in terms of:

• Relative sensitivity of objectives to each variable and any variable redundancy

• Appropriate variable ranges

• Degree of conflict between objectives, objective redundancy, indication of objective satisfaction difficulties

• Solution distribution, design space characteristics

Designer can modify design space, set objective preferences and establish more definitive multi-objective GA-based search. Process still continuous with on-line variation of design space, design scenarios and objective preferences.

• Simple linguistic rules facilitate direct preference manipulation by the designer e.g:

relation intended meaning is equally important < is less important << is much less important > is more important >> is much more important

• Ranked preferences relating to multi-objectives can be introduced and altered during an evolutionary run.

• Designer only required to answer a minimal set of straightforward questions

• Preferences transformed into numerical objective weightings

Rule-based Objective Preferences

Co-evolutionary Multi-objective Satisfaction How?

• Concurrent GA processes each optimise one objective

• Fitness measure for individuals within each GA is adjusted by comparing distance between solutions of one objective with those of others

• Penalty relating to the degree of diversity of variables of each objective process is imposed taking into consideration a generational constraint map

• Initial convergence upon individual objectives leads to overall convergence of all processes upon a single compromise design region.

• On-line sensitivity analysis utilising Taguchi ensures relative importance of a parameter is taken into account .

Range Constraint Maps

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100Generation

Ran

ge

Co

ns

trai

nt

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100Generation

Ran

ge

Co

ns

train

t

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100Generation

Ran

ge

Co

ns

train

t

0

0.2

0.4

0.6

0.8

1

0 20 40 60 80 100Generation

Ran

ge

Co

ns

train

t

(a) (b)

(c) (d)Initially map allows each GA to produce solutions based on own objective As run progresses the map, through inflicted penalties, reduces variable diversity to draw all concurrent GA searches from separate objectives towards a single compromise region. Maps include a linear decrease in range constraint and a range constraint reduction based on a sine curve.

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0 25 50 75 100 125 150

SE P 1

S0

S1

P AR E T O

Uncertain Data and Approximate Results

• Aim is to support better understanding of objective interaction / conflict through graphical representation rather than accurately defining regions of the Pareto frontier.

• Technique supports generation of information where variables and objectives and can vary as problem knowledge expands.

• Approach takes into consideration uncertainties and ill-definition inherent in preliminary design models and degree of initial understanding of problem domain.

•Approach considered more viable than utilisation of more sophisticated techniques that identify optimal non-dominated solutions that lie upon the true Pareto frontier at this stage.

• Notion of ‘rubbish in, rubbish out’ must be taken into consideration.

Scenario (A)Evolution

Machine-Based Agents

Rule-BasedPreferences

External Agents (Design Team)

On-line Database

Information gathering processes

Scenario (C)Evolution

Scenario (B)Evolution

Preferences and Co-evolutionary MOGA combined(a) Ferry Range is much more importantb) All objectives are of equal importance(c) Ferry Range is much less important

A B

C

Agents for Scenario / Dynamical Constraint Satisfaction

• Designer likely to have several ideal scenarios such as: ‘I would like objective A to be greater than 0.6 and objective C to be less than 83.56; objectives B, D, E should be maximised; variable 2 should have a value of between 128.0 and 164.5; a value graeter than o.32 is prefered for variable 7 ’

• Incremental Agent operates as follows:

1 Use designer’s original preferences for both objectives and scenarios and run optimisation process

2 If some scenarios are not fulfilled, agent suggests increase in their importance of these scenarios

3 If some scenarios still not fulfilled even when classed as ‘most important’ agent suggests change to variable ranges in scenario.

4 If some scenarios still not fulfilled agent reports to designer and asks for assistance.

Agent Co-operation

• Consider a system with several agents each trying to optimise a single objective

• Each agent is aware of the quality of its own solution

• If agent 1 solution is inferior and contradicting to others, agent 1 should compromise and accept worse solution to benefit group as a whole

• If agents can’t decide, user is consulted.

• If user resolves conflict agents remember decision for next time

• Inter agent polling based upon objective and scenario preferences utilised to resolve conflicts

Closed preference / scenario loop

Summary

• Real-world multi-objective decision-making processes where problem domain develops with information gained

• EC can support such processes through highly interactive systems

• generated information provides problem insights supporting problem reformulation.

• Initial framework briefly outlined - relatively seamless development of problem space where the decision-maker’s knowledge becomes embedded within an iterative human / evolutionary computational process ultimate goal

• Concept moves away from identification of non-dominated solutions and generation of an n-dimensional Pareto frontier.

• Inherent uncertainties and human-centred aspects of complex decision-making environments renders such approaches less viable - utility well-founded in more well-defined problem areas.

• Moves away from identification of solutions through short-term application of evolutionary search techniques.

• Continuous, dynamic explorative process - search and exploration capabilities of iterative designer / evolutionary systems.

• Concept could best utilise processing capabilities of present and future computing technology during complex human / machine-based decision-making activities.

•Further research - much modified structure where agent technologies play a major and, to some extent, autonomous role to ensure appropriate communication and information processing capabilities.