Cognitive Model Comparisons:The Road to Artificial General Intelligence?

Christian Lebiere (cl@cmu.edu)Cleotilde Gonzalez (coty@cmu.edu)

Carnegie Mellon UniversityWalter Warwick (wwarwick@alionscience.com)

Alion Science & Technology

Challenges in AI & Cognitive Science

• Both fields have similar history of challenge problems despite compatible ends but different means

• Artificial Intelligence: maximize task performance– Started with ambitious but poorly defined test (Turing Test)– Evolved narrow, precise, overspecialized challenges (Chess)– Recently attempted broader tests (Robocup, Grand Challenge)

• Cognitive Science: fit human capabilities (design guide)– Started with ambitious, ill-defined capacities list (Newell Test)– Organized a series of complex task comparisons (AMBR, HEM)– Is taking on broader but integrated challenges (DSF?)

3/7/09 Artificial General Intelligence Conference 2

Cognitive Challenge Pitfalls

• Challenge is fundamentally about the task, not cognition– Too much task analysis and KE, too little cognitive theory

• Task is too narrow; too much data available– Reduces to data fitting – favors parameterization over principle

• Task is too specialized (typical cognitive psychology)– Single cognitive aspect – misses generality, integration

• Lack of common simulation environment– Each framework/theory only tackles what they do well

• Lack of comparable human data– Emphasizes functionality – loses cognitive constraints

3/7/09 Artificial General Intelligence Conference 3

Desirable Challenge Attributes

• Lightweight– Limit integration overhead and task analysis/knowledge eng.

• Fast– Rapid model development and collection of monte carlo runs

• Open-ended and dynamic– Less parameterization, generalization to emergent behavior

• Simple and tractable– Direct relation from cognitive mechanisms to behavioral data

• Integrated– Toward integrated agent capturing architectural interactions

3/7/09 Artificial General Intelligence Conference 4

DSF Challenge Comparison

• Dynamic Stocks and Flows – Instance of Dynamic Decision Making– Control a dynamic system given unexpected environmental fluctuations– Simple version of real-world situations (financial, ecological, technical, game)

3/7/09 Artificial General Intelligence Conference 5

• Integrated tasks– Anticipate events– Control system

• Cognitive functions– Sequence learning - PC– Action selection - BG

• Implementation– VB on Windows – Text socket protocol

Generalization Scenarios

• Humans learn to control system over time for simple functions– Highly variable but quantifiable performance over learning process– Complexity of task scalable along a number of cognitive dimensions

3/7/09 Artificial General Intelligence Conference 6

• Environmental i/o– Complex sequences– Stochastic noise– Multiple variables

• System dynamics– Feedback delay– Non-linear effects– Real-time control

• Multi-agent system– Other controllers– Payoff manipulations

DSF Comparison Schedule

• Official announcement expected March 15• Task environment with socket connection for model,

data and documentation available on web site• Symposium April 1st at BRIMS conference (Sundance)• Model submission by May 15• Best entries invited to symposium at European

cognitive modeling conference (travel supported)• Email DSFChallenge@gmail.com to be added to

distribution list for official announcements/updates

3/7/09 Artificial General Intelligence Conference 7