Conceptualising Search as a Set of Cognitive ProsthesesElaine Toms
Professor of Information Innovation & Management
The University of Sheffield
Basic assumption of information retrieval?
A User enters a search query, and the System responds with one or more relevant information objects (e.g., snippets, documents)
Presentation to Search Solutions, November 25, 2020
2
The Reality..
The system only knows what the user enters into the search box
The system selectively knows about the user’s environment, albeit in a limited fashion, e.g., previous search queries, documents clicked on, scrolling, personal location,…
The system may know more about the user from 3rd party linked data (e.g., Twitter, Facebook, work-based systems)
But…
Presentation to Search Solutions, November 25, 2020
3
OrientationOverviewEvidence
ExplanationInstructionsDefinition
DistractionTime
ObjectiveMotivation
DomainWorkplace CultureManagerial factorsOrganizational factors
DateFormat
LanguageGenre
AuthenticityTopic
Domain
Environment
Task
Situ
atio
n
User System + Content
Outcome
Interface
KnowledgeExperience MotivationPhysical abilityCognitive abilityLearning styleMental Model
Source: Adapted from Toms (2011). Task-based information searching and retrieval in Ruthven & Kelly (eds.) Interactive Information Seeking, Behaviour and Retrieval, p. 49
The system does not know which of these variables will influence how the user interprets relevance
Scan object
Select object
Examine Results
Query
Stop
Behaviour
Blending Behavioural Actions with Cognitive Functions
Chaining
Comparing
Extracting
Verifying
Evaluating
Constructing
Recognisingcues
Integrating
Scanning
Reading
Cognition
Synthesizing
How do we support cognitive functions?Presentation to Search Solutions, November 25, 2020
5
Source: unpublished manuscript © E. Toms 2020
… with Cognitive ‘Prostheses’!
Definition: An electronic computational device that extends the capability of human cognition or sense perception (https://en.oxforddictionaries.com/definition/us/cognitive_prosthesis)
Something that amplifies human cognition and perception and leverages human intellectual capacities (www.lpi.usra.edu/publications/reports/CB-1089/ford.pdf
fundamentally different from the Turing Test ambition – it doesn't set out to imitate human abilities, but to extend them.
“Shift from making artificial super humans who replace us to making superhumanly intelligent artifacts that can amplify and support our own cognitive abilities.” http://www.lpi.usra.edu/publications/reports/CB-1089/ford.pdf
Not physical prostheses that aid physical actions or brain implants to support/augment cognitive activity
✗http://www.trbimg.com/img-53bdedb6/turbine/la-sci-sn-pentagon-neural-prosthetic-memory-20140709
https://encrypted-tbn3.gstatic.com/images?q=tbn:ANd9GcRJMotwObuTzvxyPZ1JCKCyixX5S7DwEVB1KrYQHS7Powf59nf-
Presentation to Search Solutions, November 25, 2020
6
Cognitive Prostheses for Search?
• Augment human cognitive function, and not just support behavioural actions;
• Assist how one decides among information objects and gains insights from those objects
• Facilitate all information processes and analyses from filtering and comparing to extracting, sensing, and scanning, etc.
A toolbox of cognitive prostheses!
Presentation to Search Solutions, November 25, 2020
7
Behaviour
Scan object
Select object
Examine Results
Query
Stop
Cognitive Prostheses that support Behavioural action
Presentation to Search Solutions, November 25, 2020
8
Query suggestionsSpell checkerBoolean (native or by fill-in form)
Filter by date, format, publisher, etc.
Document format, e.g., headers, paragraph, list
Highlighted query wordsSnippetURLEntity Cards or summaries
Source: unpublished manuscript © E. Toms 2020
Behaviour
Scan object
Select object
Examine Results
Query
Stop
Chaining
Comparing
Extracting
Verifying
Evaluating
Constructing
Recognisingcues
Integrating
Scanning
Reading
Cognition
Synthesizing
Cognitive Prostheses that support Cognitive function?
Presentation to Search Solutions, November 25, 2020
9
What do we provide to facilitate and support these activities?
Source: unpublished manuscript © E. Toms 2020
Task process(Kulthau)
Information collection
Focus formulation
Presentation
Functions(Prefontaine et al)
Existing tools
Connection
Organization
Stimulation
Divergence
SuggestionOrganization
ConnectionCollaboration
Connection
Definition & Explanation
Collaboration
Discrimination
Suggestion
Task Organizers
Bookmarks
Notes
Hyper LinksMenus
Tagging
Auto-complete
Refine SearchRelevance Ranking
Types of Information
(Vakkari)Task initiation
Prefocus exploration
Topic selection
Simplification
Stimulation
Definition & Explanation
Perspectives
Connection
Idea Generation
GeneralBackground/Theoretical Information
Faceted Background Information
Specific Information
Sharing Options
Search History
Recommender Systems
Mouse Overs
Visualization
Simplification
Search Box
Dictionary
Source: Adapted from: Gilbert et al
Sufficient?
Presentation to Search Solutions, November 25, 2020
10
Cognitive Activities used in Information Processing
Interaction Patterns Annotating Assigning Comparing Composing/Decomposing Drilling Filtering Linking / Unlinking Selecting Storing / Retrieving Translating
Tasks/Sub-Tasks Where, i.e., location How to Scope Current status Condition Trend Expertise Stats Compare Relation
Presentation to Search Solutions, November 25, 2020
11
Sedig et al. 2012 Toms et al
Interface
Presentation to Search Solutions, November 25, 2020
12
System
User Classic Hackneyed View
Source: unpublished manuscript © E. Toms 2020
Interface
Presentation to Search Solutions, November 25, 2020
13
System
User & user’s context
Closer to Today’s Reality
Source: unpublished manuscript © E. Toms 2020
Questions/Issues/Directions
Presentation to Search Solutions, November 25, 2020
14 Cognitive functions/activities
Which ones should be facilitated? Is there a common set that is cross domain? Which ones are unique to a
particular domain? [Analogical to the differences across typical office applications]
Possible approaches: Common integrated interface for ‘knowledge work’ – think Microsoft
Office ribbons on steroids A 'dashboard’ that includes Search + eDiscovery + Text Analytics + Data
Analytics – an integrated tool – a swiss-army knife – for information access, retrieval and use
Need new thinking about what IR R&D needs to achieve and also new models and frameworks for how we think about the role of search in real-world tasks
Ultimately, where does/should the Human Stop and the Machine Start?
.
Two ExamplesP4 (Research Scientist) Impacts of tidal energy
extraction in the Minas Passage in the Bay of Fundy.
Sedimentation patterns and fish behavior in the area of a tidal turbine.
Collecting data from the Integrated Science Data Management system (ISDM) and new satellite technology
Historical and new data will be used to validate models
Compare with other areas (e.g., Avon River and the Petticodiac)
P5 (Coastal Zone Advisor) Best practices in coastal
setbacks Started by looking at
coastal setback policies in other areas.
Collect examples and critically evaluating whether or not these policies will apply
Assessing if results in the literature are effective in practice.
Concerned about the source
And who to contactPresentation to Search Solutions, November 25, 2020
15
Take away thoughts
Knowledge work is here to stay, but technological innovation in KM has not kept up with rest of tech development
AI may steam roller the KM world in the same way that Web search proved to be a ‘disruptive technology’ changing forever the way we searched for information, but the onus is on us to direct it
Need for the information behaviour community to step back from the needs and uses studies for greater granularity of what actually occurs within a particular context or information use environment; developers only know what they understand and requirements need specifics
Unlocking the value in information resources may be the next big differentiator for organisations but it does mean organisations have the tools to help them achieve it
Workplace might be anywhere, anyplace, anytime Humans make decisions; technologies make recommendations
Presentation to Search Solutions, November 25, 2020
16