© 2001 - 2007 Franz J. Kurfess Usability and Knowledge 1 CPE/CSC 580: Knowledge Management Dr. Franz J. Kurfess Computer Science Department Cal Poly

© 2001 - 2007 Franz J. Kurfess Usability and Knowledge 1

CPE/CSC 580: Knowledge Management

CPE/CSC 580: Knowledge Management

Dr. Franz J. Kurfess

Computer Science Department

Cal Poly


Course OverviewCourse Overview Introduction Knowledge Processing

Knowledge Acquisition, Representation and Manipulation

Usability and Knowledge Effective Use

Knowledge Organization Classification, Categorization Ontologies, Taxonomies, Thesauri

Knowledge Retrieval Information Retrieval Knowledge Navigation

Knowledge Presentation Knowledge Visualization

Knowledge Exchange Knowledge Capture, Transfer,

and Distribution Knowledge Management

Techniques Topic Maps, Agents

Knowledge Management Tools Ontology Development Reasoning

Knowledge Management in Organizations Content Management Systems Knowledge Sharing


Overview Usability of Knowledge

Overview Usability of Knowledge

Introduction usability of tools and systems

vs. usability of knowledge Usability Evaluations Usability Frameworks

Usability Considerations for Knowledge Management

Important Concepts and Terms

Chapter Summary


LogisticsLogistics Tablet PCs ConfTool installed at http://wiki.csc.calpoly.edu/conftool/ Term Project

Milestone Week 2 should be on the team Web page (see http://www.csc.calpoly.edu/~fkurfess/Courses/581/S07/Project/Team-Web-Pages.shtml)

Assignments A1 (KM Tools) due Thu, April 19

Paper and Presentation topic proposal due “Week 3” please submit via

Blackboard and ConfTool (title, abstract, topic, keywords - no outline, bibliography)


Usability EvaluationsUsability Evaluationsformative evaluation

done at different stages of development influences the design of the system as it is being

developed relies on quick feedback from users

or other ways to obtain feedback

summative evaluation assesses the quality of a finished product no influence during design and development users can evaluate the actual product


Four evaluation paradigmsFour evaluation paradigms

‘quick and dirty’usability testingfield studiespredictive evaluation


Quick and dirtyQuick and dirty

‘quick & dirty’ evaluation describes a common practice designers informally get feedback from users or

consultants to confirm that their ideas are in-line with users’ needs and are liked.

Quick & dirty evaluations can be done any timeThe emphasis is on fast input to the design process

rather than carefully documented findings.


Usability TestingUsability Testing recording the performance of typical users

on typical tasks in controlled settings field observations may also be used

users are watched recorded on video their activities are logged

mouse movements, key presses

evaluation calculation of performance times identification of errors explanation why the users did what they did

user satisfaction questionnaires and interviews are used to elicit the opinions of users


Field StudiesField Studiesdone in natural settingsto understand what users do naturally and how

technology impacts themin product design field studies can be used to

- identify opportunities for new technology- determine design requirements - decide how best to introduce new technology- evaluate technology in use


Predictive EvaluationPredictive Evaluationexperts apply their knowledge of typical users to

predict usability problems often guided by heuristics

another approach involves theoretical modelsusers need not be presentrelatively quick & inexpensive


DECIDE: A framework to guide

evaluation

DECIDE: A framework to guide

evaluationDetermine the goals the evaluation addresses.Explore the specific questions to be answered.Choose the evaluation paradigm and techniques to

answer the questions.Identify the practical issues.Decide how to deal with the ethical issues.Evaluate, interpret and present the data.


Determine the GoalsDetermine the Goals

high-level goals of the evaluation stakeholders

in the overall system for the specific evaluation

selection of the usability evaluation paradigm probably influenced by the goals

examples of goals Identify the best presentation method for knowledge. Check to ensure that different access methods are consistent. Investigate how technology affects the usage of knowledge. Improve the usability of an existing product, work flow, or

common practice.


Explore the QuestionsExplore the Questionsquestions to be asked during usability evaluations

hypothesis (“research question”) questions can be used to clarify aspects of the goals may include questions used during interactions with

evaluation participants

examples for questions related to knowledge Why do users need/want to know this? How do users act on this knowledge? What happens if they do their task without this

knowledge? What is the source of this knowledge? Who is responsible

for veracity, maintenance, access control?


Choose the Evaluation Paradigm & TechniquesChoose the Evaluation Paradigm & Techniques

Four evaluation paradigms ‘quick and dirty’ usability testing field studies predictive evaluation

Techniques observing users asking users about their

opinions asking experts about their

opinions testing the performance of

users modeling the task

performance of users


Identify Practical Issues Identify Practical Issues selection of users

types of users size of participant pool

budget scheduleevaluators

internal/externalfacilities and equipment

usability lab recording equipment


Decide on Ethical IssuesDecide on Ethical Issuesinformed consent formparticipants have a right to

know the goals of the study what will happen to the findings privacy of personal information not to be quoted without their agreement leave when they wish be treated politely


Evaluate, Interpret and Present Data

Evaluate, Interpret and Present Data

may depend on the paradigm and techniques usedevaluation aspects

Reliability: can the study be replicated? Validity: is it measuring what you thought? Biases: is the process creating biases? Scope: can the findings be generalized? Ecological validity: is the environment of the

study influencing it e.g. Hawthorn effect


Usability Considerations for KM

Usability Considerations for KM

knowledge-intensive tasks and activities acquisition, organization, manipulation, retrieval,

presentation of knowledge

knowledge-centric interaction methods text-based, visual, auditory

tools for knowledge management generic categories, specific tools as examples

usability measures for knowledge-intensive tasks qualitative / quantitative subjective / objective


Activity: Knowledge Usability in Student

Research

Activity: Knowledge Usability in Student

ResearchScenario: A student (team) needs to do investigate a topic, e.g. to prepare a research paper, or to work on a project.

Task: Identify activities, methods, tools, and usability measures for this scenario.

Deliverable: A document created with a tool of your choice that presents the knowledge your team collected about knowledge usability to be posted on Blackboard AI Discussion Board


Activity Worksheet :Knowledge Usability Activity Worksheet :Knowledge Usability

Scenario description: describe the sample scenario that serves as the basis for your

investigation

Tasks and Activities: what are the critical tasks and activities related to dealing with

knowledge

Interaction Methods: how doe you interact with the computer to deal with the knowledge

KM Tools: what are the tools you’re using

Usability Measures how do you measure the usability of the tools and methods


Activity Worksheet:Knowledge Usability Activity Worksheet:Knowledge Usability

Scenario description:

Tasks and Activities:

Interaction Methods:

KM Tools:

Usability Measures


Activity Lecture Preparation:

Knowledge Usability

Activity Lecture Preparation:

Knowledge Usability Scenario description: an instructor needs to prepare a lecture for a Computer Science class the material will be presented to class in a face-to-face session with computer-based presentation tools

Tasks and Activities: determination of the topic

based on course catalog description, extended course outline, textbook, related courses identification of essential concepts and terms acquisition of knowledge about concepts and terms

possibly recursive until the desired level of detail is reached creation of a framework for the arrangement of the concepts

relationship between concepts, in particular dependencies presentation of the knowledge

sequence, method (natural language in spoken or written form, grapic, demo, simulation)

Interaction Methods: search for related material (documents) organization of knowledge (hierarchical and graph-based frameworks)

KM Tools: search (Google, Google Scholar, CiteSeer, IEEE Digital Library, PolyCat, Spotlight) knowledge collection utility (Google Notebook, Zotero, Scrapbook, DevonThink) knowledge organization tools (outliner, concept mapping, ontology editor)

Usability Measures ratio of useful/irrelevant documents (precision, recall) effort to perform an activity or task (time, basic actions, cost, utilization of resources) user satisfaction “pleasantness”


Key pointsKey pointsAn evaluation paradigm is an approach that is

influenced by particular theories and philosophies. Five categories of techniques were identified:

observing users, asking users, asking experts, user testing, modeling users.

An evaluation framework like DECIDE offers some guidance for the practical aspects of usability evaluations


Activity: DECIDE Framework

Activity: DECIDE Framework

apply the DECIDE framework to your team project in this class specifically to the

usability of knowledge distinguish between

usability of the tool or system, and usability of the knowledge the system deals with

Determine the goals the evaluation addresses.

Explore the specific questions to be answered.

Choose the evaluation paradigm and techniques to answer the questions.

Identify the practical issues. Decide how to deal with the

ethical issues. Evaluate, interpret and present

the data.





Using Knowledge Using Knowledge What do we want when we look for information or

knowledge? answers, not documents!

current retrieval systems identify documents that may or may not contain the answer irrelevant documents partial answers multiple answers

inconsistent, wrong context, …


Current Usage of Retrieval SystemsCurrent Usage of Retrieval Systems

tools to identify potentially relevant documentsformulation of questions as unnatural queries

either simplistic sets of keywords, or complex expressions

ranking of retrieved documents according to obscure criteria re-formulation of queries to influence ranking

mostly batch processing submit query wait view result


Better Usage of Retrieval Systems

Better Usage of Retrieval Systems

provide answers to questionsfind the right information fastanalyze information, combine it into easily digestible

formatssummarize longer documents, sets of related

documentsrelate it to decisions to be made


Pre-TestPre-Test


MotivationMotivation


ObjectivesObjectives


Knowledge Usage in Context

Knowledge Usage in Context

terminology dissemination, utilization, diffusion, technology transfer

knowledge dissemination and utilization traditional model conceptual and instrumental use related terms

knowledge use as learning processknowledge life span

usefulness and accessibility of knowledge

knowledge life cycle activities dealing with knowledge throughout its useful life


What is Knowledge Utilization?

What is Knowledge Utilization?

no clear definitionoften used interchangeably with

knowledge dissemination knowledge transfer knowledge usage

usually assume two aspects distribution of knowledge, information, or products incorporation of conceptual or instrumental use of

knowledge into relevant activities


Knowledge Dissemination and Utilization

Knowledge Dissemination and Utilization

due to a number of factors, existing knowledge is not used effectively driven by the dissemination side (researchers), rather than

the knowledge use side (practitioners) finding practical applications of knowledge is often left to potential

users

lack of coordinated knowledge utilization activities ad hoc dissemination models, very few attempts at systematic

approaches to utilization

cumbersome accessibility finding and accessing knowledge has been the domain of

specialists (librarians, consultants)


Dissemination TypesDissemination Typesspread

one-way diffusion or distribution of knowledge and information

choice users actively seek and acquire knowledge from established or

alternative sources users learn about their options

exchange interactions between people multi-directional flow of knowledge and information

implementation technical assistance, training, interpersonal activities

[NCDDR 1996]


Extension Model of Knowledge Use

Extension Model of Knowledge Use

rational, linear conception of the process of knowledge utilization knowledge is packaged and moved from one place to

another based on the assumption that knowledge can be arranged into

definable, useable units that can be transmitted easily

“getting the word out” based on the hope that potential users will hear about it,

and be willing and capable to utilize it does not reflect the use of knowledge in many situations

[NCDDR 1996]


Complex Model of Knowledge Use

Complex Model of Knowledge Use

the process usually is not rational nor linear complex

multiple sources, multiple media and paths of delivery interdependencies between individual knowledge items context may be critical

transactional may involve transactions between source (expert) and user

dependent on the background of potential users pre-existing knowledge, beliefs, experiences

the user is involved in the usage process problem-solver constructor of a personal knowledge base

[NCDDR 1996]


Knowledge UsageKnowledge Usageconceptual use

changes in levels of knowledge, understanding, or attitude

instrumental use changes in behavior and practice

strategic use manipulation of knowledge to attain specific goals

power, profit, political gain

[NCDDR 1996]


Knowledge Usage MetaphorsKnowledge Usage Metaphors“tabula rasa”

the learner’s mind is an empty slate upon which people “in the know” impress knowledge

learner as a sponge soaking up knowledge, largely without filtering or

processing

brain as a computer processes information in a systematic fashion as it is

received from outside sources


Knowledge Use as Learning Process

Knowledge Use as Learning Process

role of knowledge dynamic set of understandings influenced by its originators

and its users

role of the learner actively filters and shapes knowledge

integration into existing knowledge

constructs models of the the environment explanations to make sense of the world

pre-existing (mis-)understandings may have to be changed they result in discrepancies of the mental model

[NCDDR 1996]


Dimensions of Knowledge Utilization

Dimensions of Knowledge Utilization

dissemination source originator of information initiator of dissemination

content new knowledge supporting information

dissemination medium packaging and transmission of knowledge

user person or organization to receive and apply the knowledge

[NCDDR 1996]


Knowledge Life SpanKnowledge Life Span

[Kaplan 1997]


Knowledge Life CycleKnowledge Life Cycle

[Kaplan 1997]


Utilization of Knowledge Assets

Utilization of Knowledge Assets

[Konno 2000]


Knowledge Usage IssuesKnowledge Usage Issuesselection of knowledgecomposition of knowledgemerging of knowledge itemsmodification of knowledgemodification of system aspectspreservation of consistencyuser motivation


Knowledge Usage TemplateKnowledge Usage Template

[Skyrme 1999]


KM Benefits TreeKM Benefits Tree

[Skyrme 1999]


Technology vs. UsageTechnology vs. Usage

Today’s Technology Centered Systems

User and Usage Centered

Query

Task Context

Organize

ThinkSelect

Evaluate

Output

Collaborate

BrowseTransfer into Search

Gather

Store

Create

Plan

[Griffin 2000]


Carnegie Mellon University: Digital Video Libraries

• speech, image and natural language technologies integration

Univ of Michigan: Intelligent Agent Architectures

• software agents; resource federation; artificial service market economies; educational impact

Stanford Univ: Uniform Access• interoperability; protocols &

standards; distributed object architectures; interface design for distributed information retrieval

full-content search and retrieval of video segments

general access, extensibility for heterogeneous distributed resources

new DL cross-disciplinary capabilities, intellectual perspectives and linkages

Project/Research Focus research goal

Digital Libraries Initiative (DLI)Phase 1 Projects


[Griffin 2000]


Project/Research Focus

Univ of California, Santa Barbara: Geographic Information Systems

• spatially-indexed data; content-based retrieval; image-compression; metadata

Univ of Illinois: Intelligent Search and the Net

• large-scale information retrieval across knowledge domains; semantic search; SGML; user/usage studies

Univ of California, Berkeley: Media Integration and Access

• new models of “documents”; natural language processing; content-based image retrieval; innovative interface design

resources for geosciences research and education communities

new models and services for multi-media information management in a networked world

semantic retrieval across the net; alternatives for publishers of scientific journals

research goal



[Griffin 2000]


Computer & Communications CompaniesDigital Equipment CorpXerox CorpXerox PARCIntel CorpApple CorporationBellcoreEastman Kodak CoIBMLockheedInterconnect Tech CorpEnterprise Integration (EIT)BellcoreIntervalMicrosoft CorpBell Atlantic Network ServicesAT&THewlett PackardUnited TechnologiesSoftquadBRS/DatawareSpyglassHitachi

Professional SocietiesAmerican Math Society (AMA)ACMIEEEAmerican Institute of Aeronautics and Astronautics (AIAA)American Physical SocietyAmerican Institute of PhysicsNCGIAAssociation of Research Libraries

Government Agencies and LabsDMA/CIOU S NavyUSGSNASA/ARCRes Agcy of CaliforniaSan Diego Assn of Govts

LibrariesProject Site Univ LibsUSGS LibraryLibrary of CongressCalifornia State LibrarySonoma County LibrarySt. Louis Public LibraryNew York Public Libs

Other UniversitiesSUNY BuffaloUniv of MaineUniv of ArizonaOpen University, U.K.Univ of WisconsinUniv of ColoradoMITCornell Univ

Publishers/Content ProvidersElsevier Science GroupEncyclopedia BritannicaMcGraw-Hill PublishersDialog Information ServicesO'ReillyWAIS IncQED CommunicationsJohn Wiley & SonsU.S. News & World ReportM&T PublishingTribune CompanyUMI

-- DLI Lead Institutions --Carnegie Mellon University of California, Berkeley University of IllinoisStanford University University of Michigan Univ of California, Santa Barbara

Other/Non-ProfitsCNRIEnvironmental Systems Res InstMellon FoundationKellogg FoundationGetty Foundation

Primary & Secondary SchoolsProject-local comm schoolsFairfax County Public SchoolsWinchester-Thurston SchoolAnn Arbor Public SchoolsStuyvesant High School, NYCShasta County Ofc of Edu

Flow of Resources, Technologies, Knowledge, Intellectual Products

International OrgsERCIM

DLI Collaboration and Partnering

DLI Collaboration and Partnering

[Griffin 2000]


PharmaceuticalDesign

StructuralBiology

ChemicalDynamics

Computer Speed in Billion of Operations per Second

1970 1980 1990 2000

0.1

1

10

100

1000

Airfoil

48 Hour Weather

2D PlasmaModeling

72 HourWeather

3D Plasma

Modeling

Vehicle Signature

Climate ModelingFluid TurbulenceHuman Genome

Ocean CirculationQuantum ChromdynamicsSemiconductor ModelingSuperconductor ModelingViscous Fluid DynamicsVision and Cognition

Estimate of HiggsBoson Mass

Grand Challenge Requirements

Grand Challenge Requirements

[Griffin 2000]


Traffic Requirements for Bandwidth

100

101

102

103

104

105

106

107

108

109

1010

Steady Bursty

CompositeImaging

InteractiveVisualization

VideoTeleconf

TextFile

Transfer

CollaborationTechnology

DistributedComputing

ImageTransfer

Multi-MediaDatabaseAccess

Multi-MediaMail

Electronic Mail

CharacterData Transfer

Bandwidth Peak Rate

Application RequirementsApplication Requirements

[Griffin 2000]


Computing Capability (FLOPS)

Network Capability(bandwidth)

Two Dimensional Thinking,Early 1990s...

Two Dimensional Thinking,Early 1990s...

[Griffin 2000]


Computing (flops)Digital content

Communications

(bandwidth, connectivity)

Digital Libraries technologytrajectory: intellectualaccess to globally distributed information

less more

Three Dimensional Thinking,

Mid-90s...

Three Dimensional Thinking,

Mid-90s...

[Griffin 2000]


Today: Context and Structure

Today: Context and Structure

[Griffin 2000]


Data 01001001100011111100

Information

Knowledge

Understanding The universe is expanding!

cont

ext

anal

ysis

stru

ctur

e

Infe

renc

e

Evolution of UnderstandingEvolution of Understanding

[Griffin 2000]


LifeSciences

2000

PhysicalSciencesEngineer

ing

Life Sciences

Information

Sciences

PhysicalSciencesEngineer

ing

2010

Information

Sciences

Social Sciences,Humanities

Social Sciences,Humanities

A Vision of Disciplinarity in 2010

A Vision of Disciplinarity in 2010

[Griffin 2000]


FY 1999 Competition Data ~50 proposals requesting $25M ~30 countries Formal Program with UK/JISC (Circular 15/98)

International Digital Libraries Collaborative

Research Program

International Digital Libraries Collaborative

Research Program

[Griffin 2000]


http://www.euromktg.com/globstats/

April 1999

English 107.2M 56.5%non-English 82.7M 43.5% European 54.9M 30.0%By end of year 2000

English 160Mnon-English 167M

Languages and the Internet

Languages and the Internet

[Griffin 2000]


Stanford InfoBus: CORBA distributed object technology

ICIC IC

PM

PM

IS

LS LS LSPM

PM

ISIS

IPS

PM

IPS

PM: Protocol Machine

LS: Library Service

IC: Interface Client

IS: Information Source

IPS: Information Processing Service

* objects, collections, services, platforms….

KM InfrastructureKM Infrastructure

[Griffin 2000]


Traditional KM Stress: Service Selection, Organization, Structure

for Access Centralization, Standards Physical objects & standard genres

Contemporary Technological Capabilities (e.g. WWW) Stress: Flexibility, Openness Rapid Evolution Decentralization (geographic,

administrative) Digital objects, old + new genres

Design Space for KM Systems & Beyond

Merging Intellectual Perspectives

Merging Intellectual Perspectives

[Griffin 2000]


Audio Level

Key Words

Word Relevance

Camera Motion

Scene Changes

Histogram Scene Analysis

© Carnegie Mellon University 2/96

Application of Integrated Technologies

Application of Integrated Technologies

[Griffin 2000]

© 2001 - 2007 Franz J. Kurfess Usability and Knowledge 68UC Berkeley Digital Library Testbed

Building Large Collections of Diverse

Information

Building Large Collections of Diverse

InformationType Examples June 96 Dec 96

Documents articles, EIRs,water reports

40,900pp.

20 GB 96,600pp

48GB

Images DWRwildflowersCorelHabitatsTotal

14,8382,90522,000039,743 238GB

15,5067,43728,10115852,000 306GB

Aerialphotos

Suisun MarchSac-SJ Delta

5000img

3.4GB 500 img 3.4GB

SensorData

Delta fish flow 30days

.02MB 30days .02MB

GIS Data dams, fish,watersheds,etc.

various 50MB various 52MB

DOQs SF Bay Area 102 img 5GB 102 Img 5GB

Digital LineGraphs

SF BayNorth Coast

100MB100MB

Total 268GB 363GB

[Griffin 2000]


1965

1975

1985

1995

2000

2010ARPANET Internet KnowledgeNet

PROTOCOLS IP FTP HTTP CORBA Semantic Agents

SERVICES Distributed Files

GlobalHypermedia

DistributedObjects

GlobalSemantics

FUNCTIONProof ofConcept

Access Analysis

UNITS Packets Files Links Objects Concepts

??

??

??

New ConceptualizationsNew Conceptualizations

[Griffin 2000]


GoalsGoalsgather information and build collections

conversion of existing collections creation of new collections from scratch integration of collections

create new communities communication evaluation collaboration

make technology disappear from our awareness and experience

[Griffin 2000]


Usage AspectsUsage Aspectscollection and evaluation of usage information

single items vs. sets of items individuals vs. groups temporal relationships info-space relationships

examples relevance feedback, user profiles, citation analysis, hypertext

links, collaborative filtering

problems technical aspects, quantity of data privacy


Post-TestPost-Test


ReferencesReferences [Griffin 2000] Stephen M. Griffin, Digital Libraries Initiative, National

Science Foundation, www.dli2.nsf.gov, 2000 [Kaplan 1997] R.M. Kaplan, Knowledge Management - Access to an

Untapped Resource, October 1997. [Konno 2000] Noburu Konno, Knowledge Strategy and “Ba” - The Practice

of Knowledge-Based Management. Column, Inc., 2000. [NCDDR 1996] A Review of the Literature on Dissemination and

Knowledge Utilization. National Center for the Dissemination of Disability Research (NCDDR), www.ncddr.org, 1996

[Skyrme 1999].David Skyrme. Knowledge Usage Template. David Skyrme Associates, www.skyrme.com, 1999.

[Gil 2000]


Important Concepts and Terms

Important Concepts and Terms natural language processing

neural network predicate logic propositional logic rational agent rationality Turing test

agent automated reasoning belief network cognitive science computer science hidden Markov model intelligence knowledge representation linguistics Lisp logic machine learning microworlds


Summary Knowledge UsageSummary Knowledge Usage
