Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
1
Artificial Intelligence: An IntroductionArtificial Intelligence: An Introduction
Byoung-Tak Zhang
School of Computer Science and EngineeringSeoul National University
E-mail: [email protected]://bi.snu.ac.kr/
2(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Can Machines Think?Can Machines Think?
The Turing TestComputing Machinery and Intelligence [Turing, 1950]
2
3(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Chess PlayingChess Playing
Garry Kasparov and Deep Blue. © 1997
4(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
화성탐사화성탐사 로봇로봇 소저너소저너
3
5(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
자연자연 언어언어 처리처리
다의성 (Polysemy)4 I keep the money in the bank.4 I walk along the bank of the river.
중의성 (Ambiguity)4Time flies like an arrow.4 I saw a man with a telescope.
다양성 (Diversity)4She sold him a book for five dollars.4He bought a book for five dollars from her.
관련지식4어휘적지식, 문법적지식, 상황, 문맥지식
6(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
전문가전문가 시스템시스템
4
7(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
웹웹 정보검색정보검색
Text Data
Classification System
Information Filtering System
questionuser profile
feedback
answer
DB
LocationDate
DB Record
DB Template Filling& InformationExtraction System
Information FilteringInformation Extraction
filtered data
Text Classification
Preprocessing and Indexing
8(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
History of AIHistory of AIEarly enthusiasm (1950’s & 1960’s)4 Turing test (1950)4 1956 Dartmouth conference4 Emphasize on intelligent general problem solving
Emphasis on knowledge (1970’s)4 Domain specific knowledge4 DENDRAL, MYCIN
AI became an industry (late 1970’s & 1980’s)4 Knowledge-based systems or expert systems4 Wide applications in various domains
Searching for alternative paradigms (late 1980’s - early 1990’s)4 AI’s Winter: limitations of symbolic/logical approaches4 New paradigms: neural networks, fuzzy logic, genetic algorithms, artificial life
Resurge of AI (mid 1990’s – present)4 Internet, Information retrieval, data mining, bioinformatics4 Intelligent agents, autonomous robots
Recent trends:4 Probabilistic computation4 Biological basis of intelligence4 Brain research, cognitive science
5
9(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Artificial Intelligence (AI)Artificial Intelligence (AI)
Symbolic AI Rule-Based Systems
Connectionist AI Neural Networks
Evolutionary AI Genetic Algorithms
Molecular AI: DNA Computing
10(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Research Areas and ApproachesResearch Areas and Approaches
ArtificialIntelligence
Research
Rationalism (Logical)Empiricism (Statistical)Connectionism (Neural)Evolutionary (Genetic)Biological (Molecular)
Paradigm
Application
Intelligent AgentsInformation RetrievalElectronic CommerceData MiningBioinformaticsNatural Language Proc.Expert Systems
Learning AlgorithmsInference MechanismsKnowledge RepresentationIntelligent System Architecture
6
11(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
지능형지능형 에이전트에이전트
Autonomous Agents
Biological Agents Robotic Agents Computational Agents
Software Agents Artificial Life Agents
EntertainmentAgents
Task-specificAgents
Viruses
12(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Applications of Intelligent Agents Applications of Intelligent Agents (1)(1)
E-mail Agents4Beyond Mail, Lotus Notes, Maxims
Scheduling Agents4ContactFinder
Desktop Agents4Office 2000 Help, Open Sesame
Web-Browsing Assistants4WebWatcher, Letizia
Information Filtering Agents4Amalthaea, Jester, InfoFinders, Remembrance agent,
PHOAKS, SiteSeer
7
13(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Applications of Intelligent Agents Applications of Intelligent Agents (2)(2)
News-service Agents4NewsHound, GroupLens, FireFly, Fab, ReferralWeb,
NewTComparison Shopping Agents4Mysimon, BargainFinder, Bazzar, Shopbor, Fido
Brokering Agents4PersonalLogic, Barnes, Kasbah, Jango, Yenta
Auction Agents4AuctionBot, AuctionWeb
Negotiation Agents4DataDetector, T@T
14(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Computers Meet BiosciencesComputers Meet Biosciences
Bioinformation Technology(BIT)
BT IT
8
15(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
AI in Life SciencesAI in Life Sciences
Structure analysisStructure analysis4 Protein structure comparison4 Protein structure prediction 4 RNA structure modeling
Pathway analysisPathway analysis4Metabolic pathway4 Regulatory networks
Sequence analysisSequence analysis4 Sequence alignment4 Structure and function prediction4 Gene finding
Expression analysisExpression analysis4 Gene expression analysis4 Gene clustering
16(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
“Owing to this struggle for life, any variation, however slight and from whatever cause proceeding, if it be in any degree profitable to an individual of any species, in its infinitely complex relations to other organic beings and to external nature, will tend to the preservation of that individual, and will generally be inherited by its offspring.”
Origin of Species “Charles Darwin (1859)”
Evolutionary ComputationEvolutionary Computation: : Nature as ComputerNature as Computer
9
17(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Genetic AlgorithmsGenetic Algorithms
solutions
1100101010
1011101110
0011011001
1100110001
110010 1110
101110 1110
110010 1010
crossovercrossover
mutationmutation
00110
101110 1010
10011
00110 10010
evaluationevaluation
1100101110
1011101010
0011001001
solutions
fitnesscomputation
roulettewheel
selectionselectionnew population
encodingchromosomes
18(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Hot Water Flashing Nozzle (1)Hot Water Flashing Nozzle (1)
Start
Hot water entering Steam and droplet at exit
At throat: Mach 1 and onset of flashing
Application Example 1Application Example 1
Hans-Paul Schwefelperformed the original experiments
10
19(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Concrete Shell RoofConcrete Shell Roof
under own and outer load (snow and wind)
Height 1.34m
Spherical shapeOptimal shape
Half span 5.00m
Savings : 36% shell thickness
27% armation
max min
Orthogonal bending strength
→ϕm
Application Example 3Application Example 3
20(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Cooperating Robots (3)Cooperating Robots (3)
Cooperating Autonomous Robots
Application Example 13Application Example 13
11
21(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
CoCo--evolving Soccer evolving Soccer SoftbotsSoftbots (1)(1)
At RoboCup there are two "leagues": the "real" robot league and the "virtual" softbot leagueHow do you do this with GP?4GP breeding strategies: homogeneous and heterogeneous4Decision of the basic set of function with which to evolve players4Creation of an evaluation environment for our GP individuals
Application Example 14Application Example 14
CoCo--evolvingevolvingSoccer Soccer SoftbotsSoftbotsWith Genetic With Genetic ProgrammingProgramming
22(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
CoCo--evolving Soccer evolving Soccer SoftbotsSoftbots (2)(2)
Initial Random Population
Application Example 14Application Example 14
12
23(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
CoCo--evolving Soccer evolving Soccer SoftbotsSoftbots (3)(3)
Kiddie Soccer
Application Example 14Application Example 14
24(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
CoCo--evolving Soccer evolving Soccer SoftbotsSoftbots (4)(4)
Learning to Block the Goal
Application Example 14Application Example 14
13
25(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
CoCo--evolving Soccer evolving Soccer SoftbotsSoftbots (5)(5)
Becoming Territorial
Application Example 14Application Example 14
26(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
10001000--Pentium BeowulfPentium Beowulf--Style Style Cluster Computer for Parallel GPCluster Computer for Parallel GP
14
27(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
컴퓨터와컴퓨터와 생명체의생명체의 계산계산 속도와속도와 기억기억용량의용량의 비교비교((MoravecMoravec, 1988), 1988)
28(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Von NeumannVon Neumann’’s s The Computer The Computer and the Brain (1958)and the Brain (1958)
John von Neumann (1903-1957)
15
29(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
The Computer and the BrainThe Computer and the Brain
- 10 billion neurons(100 trillion synapses)
- Distributed processing- Nonlinear processing- Parallel processing- Carbon-based (wet)
- Less than 1 million processors (10 –9 sec each, neuron: 10-3 sec)
- Central processing- Arithmetic operation (linearity) - Sequential processing- Silicon-based (dry)
30(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
From Biological Neurons to From Biological Neurons to Artificial NeuronsArtificial Neurons
16
31(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Activation FunctionScaling Function
Output Comparison
Information Propagation
Error Backpropagation
Input x1
Input x2
Input x3
Output
Input Layer Hidden Layer Output Layer
Weights
Activation Function
Multilayer Multilayer PerceptronPerceptron (MLP)(MLP)
∑∈
−≡outputsk
kkd otwE 2)(21)( r
iiiii w
Ewwww∂∂
−=ΔΔ+← η ,
x )(xfo =
32(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Application Example:Application Example:Autonomous Land Vehicle (ALV)Autonomous Land Vehicle (ALV)
NN learns to steer an autonomous vehicle.960 input units, 4 hidden units, 30 output units Driving at speeds up to 70 miles per hour
Weight valuesfor one of the hidden units
Image of aforward -mountedcamera
ALVINN System
17
33(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Neural Nets for Face Recognition
960 x 3 x 4 network is trained on gray-level images of faces to predict whether a person is looking to their left, right, ahead, or up.
34(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Humans and ComputersHumans and Computers
Current Computers
What Kind of Computers?
Human Computers
The Entire Problem Space
18
35(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
What is What is the information the information processing principleprocessing principle underlying underlying
human intelligence?human intelligence?
36(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Mind, Brain, Cell, MoleculeMind, Brain, Cell, Molecule
Brain
Cell
Molecule
MindMind
1011 cells
1010 mol.
∞ memory
19
37(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Molecular Basis of Learning and Memory Molecular Basis of Learning and Memory in the Brainin the Brain
38(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Principles of Information Processing in the Principles of Information Processing in the BrainBrain
The Principle of Uncertainty4Precision vs. prediction
The Principle of Nonseparability “UN-IBM”4Processor vs. memory
The Principle of Infinity4Limited matter vs. unbounded memory
The Principle of “Big Numbers Count”4Hyperinteraction of 1011 neurons (or > 1017 molecules)
The Principle of “Matter Matters”4Material basis of “consciousness” [Zhang, 2005]
20
39(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Unconventional ComputingUnconventional Computing
Quantum Computing4Atoms4Superposition, quantum entanglements
Chemical Computing 4Chemicals4Reaction-diffusion computing
Molecular Computing4Molecules4“Self-organizing hardware”
40(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Molecular Computing: Molecular Computing: BioMoleculesBioMolecules as Computeras Computer
011001101010001 ATGCTCGAAGCT
21
41(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
DNA Molecules for Information DNA Molecules for Information Storage and ProcessingStorage and Processing
DNAmemory strands
a t c g g t c a t ag c a c t
0 0 0
a t c g g t c a t a
1 0 1t a g c c c g t g a
Writing: make DNA sequences
Reading: hybridization and readout
42(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
BioBio--Lab ProcedureLab Procedure
1. Sequence Design and Synthesis1. Sequence Design and Synthesis
2. Hybridization2. Hybridization
3. Ligation3. Ligation
4. Learning (Affinity Separation)4. Learning (Affinity Separation)
5. Classification5. Classification
Initial Version Space
cs four
staffee
?status?dept
five
faculty
?floor
Primer
Sticky End
cs faculty four
cs faculty five
cs faculty ?
cs staff four
cs ? ?
ee faculty ?
ee ? ?
? ? ?
...
...
...
...
22
43(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Molecular Computers vs. Silicon ComputersMolecular Computers vs. Silicon Computers
DeterministicProbabilisticReproducibility
Very highUltrahighDensity
HighLowReliability
Ultra-fast (nanosec)Fast (millisec)SpeedSequentialMassively parallelParallelismFixed (synchronous)Amorphous (asynchronous)Configuration
CommunicationMediumProcessing
2D switching3D collisionSolid (dry)Liquid (wet) or Gaseous (dry)HardwiredBallistic
Silicon ComputersMolecular Computers
44(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Molecular Information ProcessingMolecular Information Processing
MP4.avi
23
45(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/x8 x9
x12
x1x2
x3
x4
x5
x6
x7x10
x11
x13
x14
x15
y= 1
x15=0
x14=0
x13=0
x12=1
x11=0
x10=1
x9=0
x8=0
x7 =0
x6=0
x5=0
x4=1
x3=0
x2=0
x1=1
y= 0
x15=0
x14=1
x13=0
x12=0
x11=0
x10=0
x9=1
x8=0
x7 =0
x6=0
x5=0
x4=0
x3=1
x2=1
x1=0
y=1
x15=0
x14=0
x13=1
x12=0
x11=0
x10=0
x9=0
x8=1
x7 =0
x6=1
x5=0
x4=0
x3=1
x2=0
x1=0
4 examples
x4 x10 y=1x1
x4 x12 y=1x1
x10 x12 y=1x4
x3 x9 y=0x2
x3 x14 y=0x2
x9 x14 y=0x3
x6 x8 y=1x3
x6 x13 y=1x3
x8 x13 y=1x6
1
2
3
1
2
3
y=1
x15=1
x14=0
x13=0
x12=0
x11=1
x10=0
x9=0
x8=1
x7 =0
x6=0
x5=0
x4=0
x3=0
x2=0
x1=04
x11 x15 y=0x84
Round 1Round 2Round 3
46(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Benchmark Problem: Digit ImagesBenchmark Problem: Digit Images
• 8x8=64 bit images (made from 64x64 scanned gray images)• Training set: 3823 images• Test set: 1797 images
24
47(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Pattern Information ProcessingPattern Information Processing
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
1 7 13 19 25 31 37 43 49 55 61 67 73 79 85 91 97
epoch
Cla
ssifi
catio
n ra
te
Order 1Order 2
48(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Effect of MemoryEffect of Memory--Chunk SizeChunk Size
0 2 4 6 8 100
1
2
3
4
5
6
7
8
9
Corruption intensity
Ave
rage
err
or (
# of
mis
mat
ches
)
Cardinality 2Cardinality 3Cardinality 4
25
49(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Effect of the Error Tolerance LevelEffect of the Error Tolerance Level
0 2 4 6 8 100
2
4
6
8
10
12
14
Corruption intensity
Ave
rage
err
or (
# of
mis
mat
ches
)
tau 0tau 1tau 2tau 3tau 4
0 2 4 6 8 100
2
4
6
8
10
12
14
Corruption intensity
Ave
rage
err
or (
# of
mis
mat
ches
)
tau 0tau 1tau 2tau 3tau 4
0 2 4 6 8 100
2
4
6
8
10
12
14
Corruption intensity
Ave
rage
err
or (
# of
mis
mat
ches
)
tau 0tau 1tau 2tau 3tau 4
0 2 4 6 8 100
2
4
6
8
10
12
14
Corruption intensity
Ave
rage
err
or (
# of
mis
mat
ches
)tau 0tau 1tau 2tau 3tau 4
50(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Biological ApplicationBiological Application
120 samples from60 leukemia patients
Diagnosis
[Cheok et al., Nature Genetics, 2003]
Gene expression data
Training with 6-fold validation
Class: ALL/AML
&
26
51(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
DaDa VinciVinci’’s Dream of Flying Machiness Dream of Flying Machines
52(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Piston Engine
Engines of FlightEngines of Flight
Jet Engine Rocket Engine
27
53(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
TuringTuring’’s Dream of Intelligent Machiness Dream of Intelligent Machines
Alan Turing(1912-1954)
Computing Machinery and Intelligence (1950)
54(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Computers and IntelligenceComputers and Intelligence
28
55(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Future Technology EnablersFuture Technology Enablers
Source: Motorola, Inc, 2000Now +2 +4 +6 +8 +10 +12
Full motion mobile video/office
Metal gates, Hi-k/metal oxides, Lo-k with Cu, SOI
Pervasive voice recognition, “smart”transportation
Vertical/3D CMOS, Micro-wireless nets, Integrated optics
Smart lab-on-chip, plastic/printed ICs, self-assembly
Quantum computer, molecular electronics
Bio-electric computers
Wearable communications, wireless remote medicine, ‘hardware over internet’ !
1e6-1e7 x lower power for lifetime batteries
True neural computing
56(c) 2000-2006 SNU CSE Biointelligence Lab, http://bi.snu.ac.kr/
Artificial Intelligence (AI)Artificial Intelligence (AI)
Symbolic AI Rule-Based Systems
Connectionist AI Neural Networks
Evolutionary AI Genetic Algorithms
Molecular AI: DNA Computing