1

SISTEMAS INTELIGENTES

João Miguel da Costa Sousa

Technical University of Lisbon, Instituto Superior TécnicoDep. of Mechanical Engineering, Center of Intelligent Systems/IDMEC

1049-001 Lisboa, Portugal, E- mail: jmsousa@ist.utl.pt,http://www.dem.ist.utl.pt/~jsousa

Goals

To recognize computational approaches tointelligence.

To understand the motivation for using computationalintelligence systems.

To master the basic design methodology forcomputational intelligence systems.

To use intelligent systems for solving problems inengineering (scientific) problems.

To understand the motivation for using artificialintelligence systems.

2

2

João M. C. Sousa February 19, 2009 3

Computational Intelligence

Neural, fuzzy, evolutionary and hybrid systems (IEEE CIS)

[1] van Eck et al. (2006)

network architecture

classificationdata

optimization

decision making

modeling

simulation

system

4

Engineering Systems

Intelligent Systems and Engineering Systems

Intelligent Systems can help to address modeling,design and management of Engineering Systems.

3

Program

1. Introduction to Intelligent Systems

Intelligent Systems and Artificial Intelligence.Characteristics of Intelligent Systems.

2. Fuzzy Systems: Basic Concepts

Fuzzy operators. Fuzzy relations. Fuzzy inference.Types of fuzzy systems.

3. Neural Networks

Adaptive networks. Supervised learning in neuralnetworks. Neuro-fuzzy systems.

5

Program

4. Inteligent Modeling and Decision

Neural modeling. Fuzzy modeling. Decision theory.Intelligent decision. Fuzzy decision theory.

5. Intelligent Control

Fuzzy control. Model-based fuzzy control. ModelPredictive control. Branch-and-bound and GeneticAlgorithms applied to control.

6. Applications

Pattern recognition. Classification. Optical characterrecognition. Optimization of logistic processes.Supply chains. Medical applications. Many others!

6

4

Main bibliography

J.-S. Jang, C.-T. Sun and E. Mizutani. Neuro-Fuzzy and SoftComputing: A Computational Approach to Learning andMachine Intelligence. Prentice Hall, New Jersey, 1997.

Fakhreddine O. Karray and Clarence De Silva. Soft Computingand Intelligent Systems Design. Addison Wesley, 2004.

J.M.C. Sousa. Class Sheets of Intelligent Systems, 2010.

J.M.C. Sousa and U. Kaymak. Fuzzy Decision Making in Modelingand Control. World Scientific Series in Robotics and IntelligentSystems, vol. 27, Dec. 2002.

7

Other bibliography

Michael Negnevitsky. Artificial Intelligence: A Guide toIntelligent Systems. Addison-Wesley, Pearson Education, 2002.Andries P. Engelbrecht. Computational Intelligence: AnIntroduction. John Wiley, Chichester, 2002.G. Klir and T. Folger. Fuzzy Sets Uncertainty and Information.Prentice Hall, 1988.S. Haykin. Neural Networks - A Comprehensive Foundation.Prentice Hall, 1999.R. Babuska. Fuzzy Modeling for Control. Kluwer AcademicPublishers, 1998.J. Kennedy, R. C. Eberhart and Y. Shi. Swarm Intelligence.Morgan Kaufmann Publishers, 2002.Marco Dorigo and Thomas Stützle. Ant Colony Optimization. TheMIT Press. July 2004.

8

5

Evaluation

Final exam (50%) and Project (50%)

Assignments can be an alternative to final exam:1. Fuzzy set theory

2. Fuzzy modeling and clustering

3. Neural networks

4. Papers/literature

Matlab to be used in assignments and Project, whenappropriate

9

Intelligence

Definition: ability to learn, understand, applyknowledge, or think abstractly, especially in relation tonew or trying situations (Longman Dictionary)

Properties:

understanding (awareness)

acting (conclusions)

reasoning

thinking

10

6

What is Artificial Intelligence?

Artificial Intelligence (AI) is the study of agents thatexist in an environment and perceive and act

AI is the art of making computers do smart things

AI is a programming style, where programs operate ondata according to rules to accomplish goals

AI is the activity of providing such machines ascomputers with behavior that would be regarded asintelligent if it were observed by humans

Branch of computer science that is concerned with theautomation of intelligent behavior

11

Why use intelligent systems?

Automation of repetitive tasks

Augmenting limited information processing capabilityof humans

Easy interaction with machines

Understanding human brain and intelligence

Find out limits of (human) intelligence

12

7

House of intelligence?

Headintelligence connected to mind as opposed to body

Brain

Bodybody is a shell that houses mind and hence intelligence

How to reconcile the separation between the body andthe mind?

13

Models of brain

Mechanistic (Newtonian age)

Complicated (electronic) circuitry (age of electricity)

Large network of simple elements (age of neuro-science)

connectionist (elements serve a common goal)

“tools” for selfish genes

14

8

Mechanistic view

Brain is a mechanicalmachinery that performsoperations

Ideas are results of theseoperations

Energy for operations isprovided by water

Separation of ideas ofthings from thingsthemselves

15

Logic

Statement is either true or not

Statement and its complement can not be true at thesame time

Statements are represented by symbols

Correct thinking is the process of finding correctconclusions given correct statements

Basis of symbolic AI

16

9

Symbolic AI

Logician’s approach to intelligence

Precise mathematical/logical problem formulation andits solution

Sensitive to representation (and hence to errors in therepresentation)

Not robust

User provides the solution or class of solutions

Powerful paradigm for symbol manipulation

17

Connectionist AI

Inspired by the anatomic structure of the brain

Massive parallelization

System goals specified precisely

Robust in the face of errors in weight factors

Robust to loss of connections or connection elements

System is adapted to the problem through learning

18

10

Evolutionary AI

Based on a broader definition of intelligence

Inspired by natural evolution

Basic elements can be simple, complex, homogeneous,heterogeneous, etc.

Basic elements follow own goals that need notcorrespond to system goals

Elements that contribute to system goals survivebetter

19

Evolutionary AI

Intelligence is an emergent behavior (ant intelligence)

Interaction amongst elements is more important thanelements themselves

Highly adaptive to fast changes in the environmentthrough evolution

Also at the basis of social models of intelligence usingmultiple agents

Solutions are grown or evolved, rather than specifiedexplicitly

20

11

Soft Computing (SC)

Main premise is to deal with uncertainty andimprecision in the environment“Soft computing is an emerging approach tocomputing which parallels the remarkable ability ofthe human mind to reason and to learn in anenvironment of uncertainty and imprecision” (Lotfi A.Zadeh, 1992)

Extensive numeric computation as opposed tosymbolic manipulation only

21

Soft Computing

Collection of methodologies, to exploit tolerance forimprecision, uncertainty and partial truth to achievetractability, robustness and low cost solution

The methodologies in SC are complementary ratherthan competitive

In many cases a problem can be solved mosteffectively by using combinations of SC techniques

Link: World Federation on Soft Computing

http://www.softcomputing.org/

22

12

Soft computing constituents

A consortium of several paradigms

Closely related to machine learning

23

Methodology Strength

Neural networks Learning and adaptation

Fuzzy set theory Knowledge representationusing fuzzy if-then rules

Evolutionary algorithmsand bio-inspired agents

Systematic random search(optimization)

Conventional AI Symbolic manipulation

Historical developments

Symbolic AICybernetics (1947)

Artificial intelligence (1956)

LISP programming language(1960)

Knowledge engineering andexpert systems (mid 1970’s)

Neural networksMcCulloch-Pitts neuron model(1943)

Perceptron (1957)

Adaline and Madaline (1960’s)

Backpropagation algorithm(1974)

Cognitron and neocognitron(1975)

Self organizing map (1980)

Hopfield net (1982)

Boltzmann machine (1983)

Backpropagation boom (1986)24

13

Historical developments

Fuzzy systems

Fuzzy sets (1965)

Fuzzy controller (1974)

Fuzzy c-means clustering(1974)

Fuzzy modelling - TSKmodel (1985)

ANFIS (1991)

CANFIS (1994)

Other methodologies

Genetic algorithm(1970’s)

Artificial life (1980’s)

Immune modelling(1980’s)

Genetic programming(1990’s)

Bio-inspired algorithms:ACO, PSO, etc. (1990’s)

25

Characteristics of SC

Human expertise, e.g. fuzzy if-then rules

Biologically inspired computing models

New optimization techniquese.g. evolutionary search or artificial colonies of insectsfor non-gradient based optimization

Numerical computation

New application domains, extends the range of fieldswithin which AI is applied: e.g. non-linear regression

26

14

Characteristics of SC

Model-free learning: explicit model structure notalways given

Intensive computation

Fault tolerance: deleting neurons or rules degradesperformance gracefully

Goal driven characteristics

Real world applications: handling of uncertainty andimprecision, adaptability

27

Neural networks

Inspired by biological nervous systems

A lot of active research in brain modeling

Intelligence arises out of co-ordinated actions of manycomputational elements (neurons)

28

15

Neural networks

Biological neurons areconnected together bysynapses

Synapses can modifytheir strength

Weight factors modelsynapses that modifytheir strength

29

Neural networks

Data representation in the form of weight factors

Implicit representation (data are not stored anywhereexplicitly)

Distributed storage

Many types of neural networksfeed-forward neural networks

self-organizing maps

recurrent networks

radial basis function networks

General function approximators

30

16

Fuzzy set theory

In between connectionist systems and symbolic AI

Systematic calculus to deal with imprecise, incompleteand vague information

Natural interface to deal with fuzziness in naturallanguage

Numerical computations performed by usingmembership functions that represent linguistic labels

31

Fuzzy set theory

Essentially a rule based system

Conclusions are drawn by the inference system, giventhe knowledge in the rule base

Some types of fuzzy systems are equivalent to radialbasis function networks

Sets a link between numeric computations andsymbolic representation

32

17

Evolutionary computation

Inspired by evolution of biological systems

Evolution of “better” individuals in a society withcompetition

Competition can be for limited resources or through“survival of the fittest”

Related to heuristically informed search techniqueswithin symbolic AI

Requires a mechanism for selecting successfulindividuals

33

Evolutionary computation

Several forms of evolutionary computation:

genetic algorithms

genetic programming

evolutionary strategies

artificial life

Other randomized search approaches:

simulated annealing

random search

34

18

Biologically inspired algorithms

Artificial Life algorithms: swarm, ants, wasps, beesArtificial ant colonies: maybe the most used methodfrom the artificial life algorithms.Introduced by Marco Dorigo (1992), has been wellreceived by academic world and it is starting to beused in industrial applications.Applications: Traveling Salesman Problem (TSP),Vehicle Routing, Quadratic Assignment Problem,Internet Routing, Logistic Scheduling, clustering anddata mining problems.

35

Ant Colony Optimization (ACO)

36Nest

Food Source

F

N

F

N

F

N

L R L R

19

ACO in the TSP problem

37

Goals of intelligent systems

To recognize computational approaches tointelligence.

To understand the motivation for using intelligentsystems.

To master the basic design methodology for intelligentsystems.

To use intelligent systems for solving problems in thedomain of informatics and engineering (connectionalso to other areas of science).

To understand the motivation for using artificialintelligence systems.

38