Upload
baldwin-gordon
View
215
Download
1
Tags:
Embed Size (px)
Citation preview
1
Heuristic Programming in Business Solutions
ISECON’02
Kurt F. Lauckner
Department of Computer Science
Eastern Michigan University
2
What is “Soft Computing”
Often called Heuristic programmingIncludes:
Expert Systems and Fuzzy LogicNeural NetworksEvolutionary Systems
Genetic AlgorithmsGenetic ProgrammingOther Evolutionary Structures
3
Expert SystemsExpert systems are also called Rule-based systems.
Expert’s expertise is built into the program through a collection of rules.
The desired program functions at the same level as the human expert.
The rules are typically of the form:If (some condition) then (some action)Example: If (gas near empty AND going on long trip) then
(stop at gas station AND fill the gas tank AND check the oil).
EXCON: An expert system used by DEC
4
Expert Systems
Two major parts of an expert system:The knowledge base: The collection of rules that
make up the expert system.The inference engine: A program that uses the rules
by making several passes over them. On each pass, the inference engine looks for all rules whose
condition is satisfied (if part).It then takes the action (then part) and makes another pass over all
the rules looking for matching condition.This goes on until no rules’ conditions are matched.The results are all those action parts left.
5
Expert SystemsInference engines can pass through the rules in
different directions:Forward chaining: Going from a rule’s condition
to a rule’s action and using the action as a new condition.
Backward chaining: Goes in the other direction.Example: Medical doctors use both.
Forward chaining: Going to the doctor with symptoms (stomach pain). The doctor will come up with a diagnosis (ulcer).
Backward chaining: The doctor asks if patient has been eating green apples knowing green apples cause stomach aches.
6
Expert System -- Case StudyBusiness Resource Software, Inc. (BRS) has a
series of expert systems products that are a Knowledge Base of Business Experts. Their clientele include:
American ExpressAT&TBack-In-A-FlashBoeingCentury 21Dun & BradstreetE.I.DuPontEastman KodakErnst & YoungFederal ExpressHarvard UniversityIBM Corp.
J.C.PennyMcDonaldsMotorolaNew York LifeOlympusPfizerRoadway ExpressShell OilTexas InstrumentsToyotaWells Fargo BankXerox
7
BSR -- Expert System Products Plan Write Expert Edition --
Evaluates business idea and documents business plan with an Expert Assistant.
Plan Write for Marketing-- Produces a complete, fully documented marketing plan with defined objectives.
Plan Write for Hi-Tech Marketing -- A special version designed to assist the high technology industries to produce a complete, fully documented marketing plan with defined objectives and tactical lists for implementation.
Plan Write for Pricing -- Helps you to set the optimal price on any product or service.
Insight for Sales Strategy --Analyzes complex sales, identifies roadblocks, rates the potential for closing the sale, provides a pipeline of all forecasted sales.
Plan Write for Sales -- Develops and documents a plan for selling including strategy and tactics.
http://www.business-strategy-brs.com/customers.asp
8
Fuzzy Logic
Fuzzy logic comprises fuzzy sets and approximate reasoning
A fuzzy “fact” is any assertion or piece of information, and can have a “degree of truth”, usually a value between 0 and 1
Fuzziness: “A type of imprecision which is associated with ... classes in which there is no sharp transition from membership to non-membership” - Zadeh (1970)
9
Linguistic Variables
Linguistic variable: “a variable whose values are words or sentences in a natural or artificial language.” - Zadeh
Linguistic variables translate ordinary language into logical or numerical statements
Imprecision of linguistic variables makes them useful for reasoning
10
Linguistic VariablesIn fuzzy set theory we can translate the body temperatures of patients into linguistic values with an appropriate degree of membership µ.
With the linguistic variable fever, fuzzy membership functions can be defined: --low --normal --raised --strong_fever
1
0
µ(x)
x 96°F 98°F 100°F 102°F 104°F 106°F 108°F 110°F
strong_feverraisednormallow
11
Fuzzy Logic -- Case Study
Economic red flags in risk analysis and audit planning
Touche Ross [1974], Coopers and Lybrand [1977], and Price Waterhouse [1985] released a list of red flags for auditors
Deshmukh and Romine describe a fuzzy number based spreadsheet approach in assessing the risk of management fraud using red flags http://www.swcollege.com/acct/jac/jac12/jac12_article1.html
12
Fuzzy Logic -- Case Study
Uses FuziCalc® which provides a spreadsheet based environment that incorporates fuzzy numbers in the calculations
Define fuzzy sets: Strong internal control Weak internal control
Define that the internal control is effective if it detects or prevents 90% of the errors or irregularities
80% effective control may have a membership of 0.9 in a fuzzy set of "strong internal control" and a membership of 0.1 in the fuzzy set "weak internal control”
These concepts are useful in measuring and quantifying red flags
13
Fuzzy Logic -- Case Study
“The Loebbecke, Eining, and Willingham [1989] model stipulates that the probability of management fraud is a function of three factors: condition, attitude, and motivation.
“An effort must be made to aggregate red flags for each factor and then combine these three factors to determine the probability of management fraud.
“There are no statistical formulas to do this. “The fuzzy number based spreadsheet can help accomplish this
by mathematically manipulating fuzzy numbers. “If each red flag is considered equally important then the beliefs
for all red flags are equally weighed and the belief for the Condition factor can be calculated”
14
Neural Networks
Neuron: Basic building-block of the brain.There are several specialized types, but all have
the same basic structure:The basic structure of an animal neuron.
16
What NNs are and why they are useful
Connection weights store the informationWeight values usually determined by learning
procedureEach PE acts independently of all othersEach PE relies only on local informationConnection pattern provides redundancy and
facilitates fault toleranceThe most common NN uses backpropagation
17
Processing Element
Propagation Function(Linear Combiner)
Activation Function
Xn
X3
X2
X1
Yj
Example:q = f(X,W)
Example (Sigmoid):
Wj 1
Wj 2
Wj 3
Wj n
Yj = f(X,W)
qj = xiwj ii=1
n
∑
qj
Yj =1
1+e−qj
18
Neural Network -- Case Study
Real Estate Example--Home price estimatorThe 17 inputs include:
Number of bedrooms, bathrooms, & other rooms.Garage type (attached, detached or none)Patio (yes or no)Air conditioningSpecialized windows (high R factor, security)Area code
One hidden layer with 8 nodes
19
Real Estate Case Study (Con’t)
Using 100 different homesWith 17 inputs and selling price of the homeUsed EasyNN a Windows under $40 neural
network program shell Trained network for several hoursResults of homes not known to the neural
network were estimated to within 2% of actual pricing
20
LVQ or Kohonen NetworksLVQ (Learning Vector Quantizer) neural
networks are often called Kohonen networksProbably second only to backpropagation in
number of applicationsNo rigorous mathematical derivation Seem to be more biologically oriented than
most paradigmsReduce dimensionality of inputsLVQ-I Makes use of unsupervised learning
Network learns from input patterns alone
21
Self-Organizing Feature Maps
Primary features of input cause corresponding local responses in the output PE field.
Are non-linear mappings of input space onto the output PE space (field).The mapping is referred to as a self-
organizing feature mapsExtensions of LVQ-I resemble the way
certain areas of the brain are organized
22
General Comments about LVQ-I
Main accomplishments of LVQ-I networks is they cluster input data patterns into classes, thus reducing dimensionality of data (e.g., automatically parameterizes the data)
Input vectors are usually normalizedWeight vector normalization is done for all
weights connected to a given output PE from all input PEs
23
Kohonen Neural Network -- Case StudyMarketing: A campaign is being run to increase sales of a product. It is intended to do a targeted mailshot to individuals most likely to be interested in the product. There is information available from a consumer survey which includes information about current purchasers of the product. An unsupervised net is trained using information from the consumer survey. The groups formed are examined and all individuals in the groups rich in existing purchasers are used as targets for the mailshot.
Data Cleaning: Data being extracted from a database is being analysed but unfortunately the data is too inconsistent for the analysis technique. An unsupervised net is used to create groups and the analysis is based on the individual groups instead of the entire database.http://www.neusciences.com/Technologies/nn_unsupervised.htm
24
Evolution--Darwin
“…if variations useful to any organic being do occur, assuredly individuals thus characterized will have the best chance of being preserved in the struggle for life; and from the strong principle of inheritance they will tend to produce offspring similarly characterized. This principle of preservation, I have called, for the sake of brevity, Natural Selection.”
--C. Darwin, 1859
25
Evolution--Darwin
Reproduction of individuals in the population Variation that affects the likelihood of
survival of individualsHeredity in reproduction (e.g.,like begets like) Competition caused by finite resources.
Translating Darwin’s statement, the four essential preconditions for the occurrence of evolution by natural selection are:
--Banzhaf
26
Evolutionary Algorithms
Populations of solutions Innovation operations (e.g., mutation) Conservation operations (e.g.,crossover) Quality differentials (e.g.,graded fitness
function, richness of differences) Selection (e.g., differences in population
are used to select fittest.
Basic Ingredients of EAs:
27
Genetic Algorithm--Holland Uses binary genome of fixed length
Crossover most common and usually of the is one-point crossover variety
Mutation used very sparingly Classifier systems used to select fittest consist of
three parts: Rule-based programming language Simulated market economy Genetic algorithm
1 0 0 1 1 1 0 1 0
1 0 0 1 1 1 0 1 0 1 0 0 1 1 1 1 1 1
1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 0 1 0Parents Offspring
28
Genetic Algorithm -- Case Study
Improve model for tactical asset allocation
Improve model for international equity strategies
Improvements of up to 82% in cumulative portfolio value as compared with passive benchmark model
29
Evolutionary Programming
Stochastic decision making: GP uses pseudo-random numbers to mimic the randomness of natural evolution.
Program structures: GP assembles variable length program structures from basic units called functions and terminals.
Genetic operators: GP transforms the initial programs in the population using genetic operators (e.g. crossover, mutation, reproduction)
Simulated evolution of a population by means of fitness-based selection.
Essential Common Themes:
30
Primitives of Genetic Programs
Terminal set: terminals are the inputs to the GP program (named by fact they are terminals of a tree structure)
31
GP Primitives (Con’t)
Function set: these are the statements, operators, and functions available to the GP system.Boolean functions ( AND, NOT, OR, XOR etc.)Arithmetic functions (plus, minus, multiply etc.)Variable assignment functions ( x :=3)Indexed memory functionsConditional statements (IF, THEN, ELSE, CASE)Control transfer statements (GOTO, CALL, JMP)Loop statements (WHILE…DO, REPEAT…)Subroutines (for example a robot: turn left, stop…)
32
Executable Program StructuresFunctions and terminals must be assembled into
some structure that can be executed and evaluated.Two methods of initializing tree structure:
Full method of tree initializationPick functions as nodes until maximum depth reachedPick terminals (leaves) of the tree from terminal set
Grow method of tree initializationPick randomly from functions and terminal setsIf function selected continue with that branchIf terminal selected the branch ends
33
Execution of Structure
In the tree structure use the pre-order traversal to test fitness of all individuals of the population
Apply selection algorithmUse crossover, mutation and reproduction to
create new populationContinue until termination criterion met or
maximum number of generations reached
34
Genetic Programming -- Case Study
GP paradigm tool STROGANOFF (1990s)Used in predicting USD/SF currency exchange
ratesUsed in predicting movements of the Dow Jones
industrial indexUsed in predicting movements of the Nikkei225
index of the Tokyo Stock ExchangeOther applications: derivative markets, banking
and finance, marketing