1
Slow Intelligence Systems
Session and Panel
2
Panelists
• Erland Jungert• Francesco Colace• Tiansi Dong• Shi-Kuo Chang (Moderator)
3
Outline
• Motivation• Introduction to SIS• Application: Ontological Filters• Application: Topic/Trend Detection• Discussion
4
Motivation: Common Characteristics ofNew Generation Information Systems
• Connected• Multiple sourced• Knowledge-based• Personalized• Hybrid
5
Smarter Planet• We are all now connected - economically,
technically and socially. Our planet is becoming smarter via integration of information scattered in many different data sources: from the sensors, on the web, in our personal devices, in documents and in databases, or hidden within application programs. Often we need to get information from several of these sources to complete a task. Examples include healthcare, science, the business world and our personal lives. (Quoted from Josephine M. Cheng, IBM Fellow and Vice President of IBM Research)
6
(courtesy of IBM)
7
Hybrid Intelligence• While processor speed and storage capacity
have grown remarkably, the geometric growth in user communities, online computer usage, and the availability of data is in some ways is even more remarkable. Hybrid Intelligence offers great opportunities we have to harness this data availability to build systems of immense potential. While today s large scale systems are evolutionarily based on the distributed computing technologies envisioned in the 70 s and 80 s, sheer scaling has led to many unanticipated challenges. (quoted from Alfred Z. Spector, Vice President, Research and Special Initiatives, Google, USA)
8
Hybrid IntelligenceUsers and computers doing more than either could
individually (quoted from Alfred Z. Spector, Google).
9
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time.
• A slow intelligence system is a system that (i) solves
problems by trying different solutions, (ii) is context-
aware to adapt to different situations and to propagate
knowledge, and (iii) may not perform well in the
short run but continuously learns to improve its
performance over time.
10
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time
through a process involving • Enumeration
11
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time
through a process involving • Enumeration• Propagation
12
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time
through a process involving • Enumeration• Propagation• Adaptation
13
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time
through a process involving • Enumeration• Propagation• Adaptation• Elimination
14
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time
through a process involving • Enumeration• Propagation• Adaptation• Elimination
• Concentration
15
Slow Intelligence Systems• Slow Intelligence Systems are general-
purpose systems characterized by being able to improve performance over time
through a process involving • Enumeration• Propagation• Adaptation• Elimination
• Concentration• Slow Decision Cycle to complement Fast
Decision Cycle
16
Slow Intelligence Systems
• A SIS continuously learns, searches for new solutions and propagates and
shares its experience with other peers.
• From the structural point of view, a SIS is a system with multiple decision
cycles such that actions of slow decision cycle(s) may override actions of quick decision cycle(s), resulting in
poorer performance in the short run but better performance in the long-
run.
17
SIS Basic Building Block (BBB)
18
Advanced Building Block (ABB)
19
SIS is a component-based systembuilt from BBBs and ABBs
20
The SIS Testbed for Healthcare Systems
21
Ontological Filters for Slow Intelligence Systems
Shi-Kuo Chang, Emilio Zegarra, Francesco Colace and Massimo De Santo
22
Production of personalized or custom-tailored goods or services to meet consumers' diverse and changing needs
SIS Application to Product Configuration
23
Ontological Filter and Slow Intelligence System
Figure 6 - Ontological Filter and the Slow Intelligent System
24
A Scenario
• A customer would like to buy a Personal Computer in order to play videogames and surf on the internet.
• He knows that he needs an operating system, a web browser and an antivirus package.
• In particular, the user prefers a Microsoft Windows operating system. He lives in the United States and prefers to have a desktop. He also prefers low cost components.
25
Ontological Transform for Product Configurator
26
Building Topic/Trend Detection System based on Slow Intelligence
Chia-Chun Shih & Ting-Chun PengInstitute for Information Industry
Taipei, Taiwan
27
• An online trend detection system requires careful resource allocation and automatic algorithm adaptation to process huge size of heterogeneous data.
• This research adopts Slow Intelligence, which provides a framework for systems with insufficient computing resources to gradually adapt to environments, to response the challenges.
• Four Slow Intelligence subsystems are proposed, and each subsystem targets a challenge in designing online topic/trend detection systems.
28
Introduction • Topic Detection and Tracking (TDT)
– Initiated by DARPA at 1996– discover the topical structure in
unsegmented streams of news reporting as it appears across multiple media
– Tasks:• Topic Detection• Topic Tracking• First Story Detection• Story Segmentation• Link Detection
29
Topic/Trend Detection System
• Objective– Detect current hot topics and to predict future hot
topics based on data collected from Social Media
• Three components– Crawler & Extractor: Collect data and extract
information from Social Media– Topic Extractor: Detect hot topics from a set of text
documents– Trend Detector: Detect trends (future hot topics)
based on currently available data
Crawler & Extractor
Topic Extractor
Trend Detector
SocialMedia
Current Hot topics
Future Hot topics
30
Topic/Trend Detection System
• Crawler & Extractor
(cont’d)
Web dataDB
WebCrawler
HTMLdocuments
InformationExtractor
* Extract articles and metadata (title, author, content, etc) from semi-structured web content
User’sKeywords of
Interests
Topic Extractor
Social Media
Textdocuments
Crawler & Extractor
31
Topic/Trend Detection System
• Topic Extractor
(cont’d)
Web dataDB
Topic WordExtraction
Topic WordClustering
Hot topicextraction
Currenttopics
CurrentHot topics
Topic Extractor
• Apply TF-IDF scheme to generate Top-N topic words for each document
• Apply clustering algorithm to cluster topic words into topic groups. The topic groups are treated as “topics” • Apply aging theory to
find hot topics
32
Topic/Trend Detection System
• Trend Detector
(cont’d)
Trend Detector
Currenttopics
Trend EstimationAlgorithms
Topic Trend(Future Hot Topics)
33
T/TD System with Slow Intelligence
• Four complexities of designing online topic/trend detection systems
• 1. It is unlikely to collect all web data based on limited amount of computing
.resources The system needs to develop data collection strategies which can concentrate limited resources on collecting important web data.
34
T/TD System with Slow Intelligence
• 2. Many computation methods are available for estimating trends. If parameter settings are also taken into account, there are too many combinations to choose. Furthermore, Internet is a changing environment, which means current best solution may not perform well in the future. The system needs to automatically (or at least quasi-automatically) find best solution from many alternatives in a changing environment.
(cont’d)
35
T/TD System with Slow Intelligence
• 3. The crawler needs to revisit websites to collect up-to-date data in hourly or daily intervals. Each site has different amount of to-be-update data and different policy to restrict frequent access, which are unknown beforehand. The system needs to find feasible data collection schedule based on past experience.
(cont’d)
36
T/TD System with Slow Intelligence
• 4. Any changes in web pages may disrupt Extractors. It needs automatic repair mechanism for Extractors if many websites are being monitored. The repair mechanism needs to detect errors of Extractors, find alternatives, and choose the best solution from alternatives to fix the disrupted Extractors.
(cont’d)
Crawler &
Extractor
37
T/TD System with Slow Intelligence
1. SIS to help restrict the range of data collection
(cont’d)
Knowledge of data
Knowledge of algorithm
38
T/TD System with Slow Intelligence
2. SIS to help select and adapt trend detection algorithms
(cont’d)
39
T/TD System with Slow Intelligence
3. SIS to help scheduling Crawler
(cont’d)
40
T/TD System with Slow Intelligence
4. SIS to help adapt Extractors
(cont’d)
41
Enumerator Adaptor Eliminator Concentrator
Slow Intelligence System Building Blocks
Crawler & Extractor Topic Extractror Trend Detector
Topic/Trend Detection System
SIS system for scheduling Crawlers
SIS system for Selecting Trend Estimation MethodSIS System for
Focused Crawling
SIS system for adapting extractors
Enumerator Adaptor Eliminator Concentrator
Slow Intelligence System Building Blocks
Crawler & Extractor Topic Extractror Trend Detector
Topic/Trend Detection System
SIS system for scheduling Crawlers
SIS system for Selecting Trend Estimation MethodSIS System for
Focused Crawling
SIS system for adapting extractors
42
Discussion• There are a large number of
intelligent systems, quasi-intelligent systems and semi-intelligent systems that are "slow". Distributed intelligence systems, multiple agents systems and emergency management systems are mostly slow intelligence systems that exhibit the characteristics of multiple decision cycles.
43
Discussion (continued)
• Since time is relative, "slow" intelligence systems for some can also be "fast" for others.
• A slow intelligence system can evolve
into a fast intelligence system.
• A framework for knowledge-based software engineering.
Q&A
The End