Taxonomy Book Camp SharePoint IA 11-17-10

Earley & Associates, Inc. | Classification: PUBLIC USE Copyright © 2010 Earley & Associates, Inc. All Rights Reserved.

Information Architecture and Taxonomy Management in SharePoint

Taxonomy Boot Camp, November 16th, 2010Jeff Carr - Senior Information Architect & Search Consultant

Paul Wlodarczyk - Director, Solutions Consulting

SharePoint is very easy to implementbadly

3Copyright © 2010 Earley & Associates, Inc. All Rights Reserved.

Typical SharePoint Projects

3

“Hey! We got SharePoint! It has got blogs, wikis, workspaces, team sites, and search—let us have all of that. We don't need anyone to help us. It is easy to set

up, and we’ll just learn as we use it. We only need a site or two to store the documents in. If the users want in, we’ll give them some sites to play with.”

“Hey! We have 20 sites now. Lots of content. Not sure what we are doing. Not sure how it all connects together. We think we know how to manage it, though

we don’t know how big it will get. And we also can’t control how big it gets because we are not entirely sure who is using it and why.”

then, a couple months later…

Source: Managing and Implementing Microsoft® SharePoint® 2010 Projects - O’Reilly Media


• It’s commonplace with SharePoint to start with the technology first and push off the gathering and documentation of requirements until later, if at all. Adopted by IT followed by the provisioning of a few sites as business users become

aware of its existence (easy to deploy).

Mass proliferation of sites, lists and libraries and an assortment of individuals and groups start to turn on various bits of functionality resulting in a deployment that is haphazard and confusing.

• SharePoint has been specifically designed to remove management of the information environment away from IT and into the hands of business users. Site management is (oftentimes) dropped into the lap of a single or small group of

uninformed individuals that are unaware of best practices in areas like content management, information architecture, taxonomy and metadata

Information governance becomes crucial since many organizations lack standard ways of managing content.

The Technology-Centric Approach

Where is the information architecture?

Requirements

RequirementsResearchResearch

Use Cases & PersonasUse Cases & Personas

Site Map & NavigationSite Map & Navigation

WireframesWireframes

TaxonomyTaxonomy

Content Modeling /Metadata


Prototyping /Testing

Prototyping /Testing

The IA ProcessFor SharePoint

TaxonomyTaxonomy




• The success of SharePoint in any environment will be measured by your user’s ability to easily find information

• The technology will process the inputs that we provide whether they make sense or not (garbage-in/garbage-out)

• Information Architecture… Establishes the foundation for Findability - but Findability is not an attribute of technology,

it is a set of standards and processes that are applied to organizational information

Involves modeling content in a way that captures both “is-ness” and “about-ness” How we describe our content - information lifecycles, retention, metadata and taxonomy

Understanding our users and their needs - Roles, responsibilities, tasks and activities required to support the pursuit business goals and processes

• Requires that we leverage features and functionality to support both

What Goes In Must Come Out…


Importance of Defining Standard Terminology

High potential for confusion: • Safe Work Procedure• Safe Operating Procedure• HSEOP (HSE Operating

Procedure)• HSE Manual


Core IA Architectural Concepts

Site Collectio

n

Collection of sitesPrimary source of global navigation

Sites & Sub-Sites

Container for lists/libraries

source of “quick launch” navigation

Lists &Librarie

s

Basic unit of storage, collection of documents or

items


• Content Type - A reusable collection of settings that define the behavior and properties for a specific type of information. Comprised of a collection of metadata attributes, information management policies, workflow and standard templates.

• Site Column - Metadata attribute (also known as a field) that can be assigned to one or more content type definitions, lists or document libraries. Used to help ensure consistent application of metadata across content in SharePoint. Date, Single or Multiline Text, People, Choice, Lookup (taxonomy)

• Site List - A tabular structure of items presented in a row (content) and column (metadata) format. Some examples: Contact, Task, Calendar

Custom List - Taxonomy or controlled vocabulary used to populate dropdown menu (defined as Choice or Lookup column types).

Core IA Architectural Concepts

Taxonomy


• Steps involved in surfacing controlled lists of terms for tagging:

Enabling Tagging

1. Site List 2. Site Column 3. Content Type

Tagging in the user interface (when adding new or editing an existing Policy)


• Metadata can only be tagged and stored as flat controlled vocabulary – no hierarchy possible

Limitations of Site Lists

Policy Type

• Education• Environment

al• Health• Information

Information Policy Sub-Type

• Archiving Information• Intellectual Property• Privacy &

eCommunications• Records Management

Policy Type

• Education• Education Sub-Types

• Environmental• Environmental Sub-

Types• Health

• Health Sub-Types• Information

• Archiving Information• Intellectual Property • Privacy &

eCommunications• Records Management

Possible Possible Not Possible


Site Boundaries & Inheritance

Problem: Constructs are specific to the site collection in which

they were created

GlobalContent

Types, Site Columns and

Site Lists

LocalContent


Site Lists

Solution: Requires manual or custom development for

replication/syncing of constructs across Site Collections

Global

Global

Global

Global

Global

Global

Global

Global

Global

Global


Navigational Limitations

Problem: Fragmented UX when navigating between different site

collections

Navigation is “naturally” only

within a Site Collection

Solution: Requires custom development for the creation of a consistent experience across the

environment

• Specific to a Site Collection• Based largely on Sites and Sub Sites• Quick launch shows “current site” elements• Top-level navigation shows sub sites and peers

NavNav

Nav

Nav

NavNav

Nav

Nav

Nav

Nav


• Problem - Search is often installed and simple OOTB configurations ignored

• Full-text indexing along with the document title, short snippet and ten results per page become the common default experience

• Frequently filled with redundant, outdated or irrelevant content (clear reflection of the information that has been uploaded into the system) Inconsistencies in how information is enriched will result in a poor search experience

• Ref: http://www.earley.com/blog/enterprise-search-why-we-cant-just-get-google

Search Experience

• Best Bets• Search thesaurus • Authoritative sites • Search scopes• Managed properties


• In addition to site collections, content types, site columns, lists, libraries and views…

• Managed Metadata - A hierarchical collection of predefined centrally managed terms that are applied by publishers as metadata attributes for content items. Managed Term - A predefined word or phrase created and managed by a user with

appropriate permissions and often organized into a hierarchy (controlled vocabularies, taxonomic in nature).

Enterprise Keywords - A non-hierarchical word or phrase that has been added to the keyword set directly by a system user (uncontrolled vocabularies, folksonomic in nature).

• Term Store - A database that is used to house both Managed Terms and Managed Keywords. Groups - From a taxonomy perspective, a group is a flat list or hierarchical collection of

related attributes comprised of one or more Term Sets.

Term Set - A flat list or hierarchical collection of related Terms that belong to a Group.

Term - A word or phrase that can be applied by publishers and system users as metadata to content.

Core Architectural Concepts


Term Store Management Tool

Group

Term Set

Terms

Term Attributes

Centralized Managementof Metadata


• Auto-Suggest - Display of taxonomy terms as a user types characters into a Managed Metadata field.

Tagging: Auto-Suggest

Preferred Term

Term Hierarchy

Term Definition

Synonym


• Hierarchy - Display of taxonomy terms in a popup window that provides the ability to browse through the defined hierarchy.

Tagging: Browsing the Hierarchy

Preferred Term

Term Definition

Synonyms


Content Type Hub

Centralized Management of Content Types

GlobalContent


Site Lists

Corpora

teGlo

bal

Global

Global

Global

Global

Global

Global

Global

Create and manage global content types in a single location and push them out to

subscribing site collections

Global


• In addition, we now have the Refinement Panel and document previews…

Search Enhancement

Metadata

DocumentPreviews Scope

Presentation


• Physically oriented and architectural constructs bound by site collections

• A lack of cross site collection synchronization of fundamental IA building blocks content types, metadata, taxonomy and navigation

• Metadata and taxonomy is simplistic Inability to create and manage taxonomic relationships between terms (no hierarchy,

associations, synonyms defined separately as part of the thesaurus file)

• OOTB search reflects all limitations (inability to easily surface and leverage metadata)

Overall Shortcomings

• Physically oriented and architectural constructs such as navigation are bound by site collections

• Improved metadata and taxonomy, but still basic application Ability to define synonyms, but applied to the tagging process (search thesaurus is still

separate)

Inability to create and manage taxonomic complex relationships between terms (associative)

Earley & Associates, Inc. | Classification: PUBLIC USE Copyright © 2010 Earley & Associates, Inc. All Rights Reserved.

Taxonomy in SharePoint Search and Metadata

Paul Wlodarczyk

Director Solutions Consulting

Earley & Associates

22


• Integration of Taxonomy with SharePoint • Vendor Landscape

Tagging and Auto-classification Search User Experience and Search Relevance

• How to Decide

Agenda

23

24Copyright © 2010 Earley & Associates, Inc. All Rights Reserved. 24

• Tagging and Taxonomy: Out of the box: just flat lists. Can be

extended to hierarchical controlled vocabularies with third party extensions

Search: No faceted search out of the box, but advanced search can behave in a faceted way using metadata.

Tagging and Taxonomy: Hierarchical Term Stores, Suggested Terms, Definitions.

Search: Search refinement (facets) based on metadata

• Third Party Solutions Enterprise Taxonomy and Metadata Management Content Classification Custom Search Applications Search

Taxonomy Management, Classification, and Search in SharePoint


Taxonomy Integration with SharePoint

There are several points of integration for taxonomy in SharePoint:

1. Content Metadata: Taxonomy as source of terms for metadata

2. Search Configuration: Thesaurus and Best Bets can be derived from taxonomy

3. Content Index: Leverage taxonomic relationships for classification rules, modify relevancy ranking in the search index

4. Search User Experience: Create a custom search application that uses taxonomy for driving facets, navigation, related searches, suggested searches, etc.

25

1

2

3

4


Taxonomy Integration with SharePoint

There are several points of integration for taxonomy in SharePoint:

1. Content Metadata: Taxonomy as source of terms for metadata

2. Search Configuration: Thesaurus and Best Bets can be derived from taxonomy

3. Content Index: Leverage taxonomic relationships for classification rules, modify relevancy ranking in the search index

4. Search User Experience: Create a custom search application that uses taxonomy for driving facets, navigation, related searches, suggested searches, etc.

5. Third Party Search Engine: replace SharePoint Search with another platform that can consume taxonomy

26

1

2

3

4

3rd Party Query Engine

3rd Party Index Engine

5


Third Party Application Scope

Auto-classification

Tagging

• Applications vary in depth, breadth, and complexity Tagging plug-ins Search UX plug-ins Taxonomy Management

Suites Classifiers Search engines with user

experience toolkits

27

Search


TAXONOMY AND CONTENT CLASSIFICATION

28


• Manual Tagging Users can generate keywords for specific SharePoint columns, either

as free text, or chosen from controlled vocabularies defined during configuration of the column.

• Taxonomy-driven Manual Tagging Taxonomy / term store is source of preferred terms for tagging

metadata. In SharePoint 2010, third-party tools integrate with the term store. In SharePoint 2007, third-party add-ons provide a hierarchical user experience for manual tagging, based upon a taxonomy.

• Taxonomy-driven Auto-classification Third party classification engine uses taxonomy and/or other methods

to inform a rules-based classification of documents. Metadata are generated, usually as a flat list of terms in a keyword column that can be manually revised, often with a hierarchical view of the vocabulary.

Approaches to SharePoint Classification


• Centrally manage an enterprise taxonomy

• Use enterprise taxonomy as source of preferred terms / controlled vocabularies (and synch with 2010 Term Store)

• Auto-classification using the taxonomy as a source of SharePoint metadata

• Make up for term store shortcomings in 2007: Manage SharePoint Metadata Map controlled vocabularies to SharePoint columns Display hierarchy in tagging user interface Persisting the taxonomic relationships in the metadata

Key Capabilities for Classification in SharePoint

30


Manual Tagging Example: WordMap

31

Metadata columns defined by SharePoint Admin – Department, Product, Locations

Controlled vocabulary from taxonomy mapped to columns



32

Creating new columns and associating them with controlled vocabularies


Hierarchy is preserved in the metadata (can be viewed on

hover) and is available for search

Controlled vocabulary presented for selection, then

shown in view



Auto-classification Example: Smartlogic

34

Documents can be auto-classified in SharePoint based upon policies



35

Documents can be auto-classified using a manual trigger



36

Auto-classify results shown in Edit Properties dialog

Can be manually edited using Add or Remove



37

Tagging interface enables multiple terms to be selected

from a taxonomy-driven CV to edit the automatically applied

terms


Auto-classification Example: Concept Searching

Metadata columns defined by SharePoint Admin – “Agricultural”

Metadata results from auto-classification of concepts that are related to preferred

terms in the taxonomy


• Concept Searching Taxonomy Management and Auto-Classification

Full integration with SharePoint and MS Office Applications

Classifier is concept-based: finds concepts then maps them to the preferred terms in the taxonomy

• SchemaLogic SharePoint Metadata Management – MetaPoint


Integration with Term Store for manual metadata tagging

Suggests tags in MS Word

No auto-classification solution

• Smartlogic Taxonomy Management and Auto-Classification


Classifier is rules-based: Rules are derived from taxonomic relationships, and preferred / non-preferred / related terms

Tagging and Classification Applications for SharePoint


TAXONOMY AND SEARCH

40


• Search Relevance / Indexing Integrate a taxonomy-driven classifier with the

indexing process for SharePoint or Third Party search (Google, FAST, Attivio, etc.)

Use taxonomy as source for preferred terms / equivalence terms in search

• Search User Experience Use taxonomic relationships to drive navigation (e.g.

tree browse) and faceted search or tag clouds Use taxonomic relationships to suggest related

searches

Key capabilities for Taxonomy Integration with Search


Search User Experience: Smartlogic

42

• Related terms in Smartlogic Semaphore. The user searched for “rights” in the SharePoint search box. Smartlogic shows related terms on the right for “rights” from the taxonomy. Search term highlighting is native MOSS functionality, showing search terms, not taxonomy terms.


Search User Experience: Smartlogic

43

• Faceted search in Smartlogic Semaphore. The user refined their search for “rights” by selecting “Employment Rights” in related terms.

• A facet is added for “Employment Rights”; the suggestion box changes to show Related Categories. Multiple facets will be shown if the user drills down, and facets can be removed by clicking the X on the facet


Search User Experience: BA Insight

• “Refine your search” for faceted search and tag clouds

• Facets can be hierarchical based upon taxonomy (e.g. Client, Practice, Matter), and can show calculated ranges (e.g. dates) and document metadata (e.g. Author).



45

• Document preview - User interaction with the preview affects relevancy rankings.



46

• Search term highlighting in the preview

• Key concepts are search terms used like folksonomic tags

• Key concepts can be used as facets in the full view – so can quickly find the most relevant pages

• Pages can be saved to a “research notebook”


• In the Taxonomy Browse view using the SharePoint search engine, the user can narrow the search to a sub-tree within the taxonomy.

• Here they will search for the concept “water quality” within the “Environment” sub-tree of preferred terms (and their clues).

Search User Experience: Concept Searching

47


• Search results pages can be enhanced using taxonomy-driven facets for search refinement (right) and related searches (left).

• The search results can be filtered on individual terms or the concept, and search terms are highlighted in the extracts.

Search User Experience: Concept Searching

48


• FAST can create custom search applications for SharePoint

Search User Experience: FAST

49


• FAST facets can include graphical representations (e.g. date sliders, pies, maps).

Search User Experience: FAST

50


• Attivio can provide faceted search and custom applications atop SharePoint.

Search User Experience: Attivio

51


• Attivio custom “active dashboards” can integrate structured and unstructured data into rich business intelligence applications..

Search User Experience: Attivio

52


• With so many options, it can be daunting to know where to begin

• Our advice: Choose Search technology first… Recall and relevance drives business results User experience drives adoption

… Then decide on classification tools …

… Then decide on taxonomy management. Only required if taxonomy is consumed outside of SharePoint

and Search, or for large / complex / volatile taxonomies, or to support auto-classification

How to decide?

53


How much do you need?

54

SharePoint OOB

Basic Faceted Search

Taxonomy-DrivenFaceted Search

Advanced Faceted Search

Use advanced search for filtering on metadata fields. Navigation is based on site architecture / folder

structure

Expose facets on main search screen, basic tree view (SharePoint 2010)

Auto-classification; Related search suggestions, broader/narrower, thesaurus

relates search terms to preferred terms, tree browse (BA Insight, Concept Searching, Smartlogic)

Facets span content types, fully-flexible UX,

deep indexing, relate to structured data (FAST,

Attivio)

Faceted Search in SharePoint


• Organizations are really good at creating information and a well planned and intelligently constructed foundation is the basis for successful information architectures and high quality user experiences

• A taxonomy by itself lacks value - it becomes powerful when it’s applied to content and surfaced through information access mechanisms like search and navigation

• Don’t skip IA process just because SharePoint is easy to implement

• Master OOTB features first and ensure it’s configured properly to meet your needs

• Keep your eye on consistency across sites, site collections through governance (must be enforced)

• SharePoint itself is not intended to be an enterprise taxonomy management tool.

Summary


Contact

Jeff Carr

Senior Information Architect & Search ConsultantEmail: [email protected]: @siftonparkLinkedIn: linkedin.com/in/siftonpark

Paul Wlodarczyk

Director, Solutions ConsultingEmail: [email protected]: @ twitcontentguyLinkedIn: linkedin.com/in/paulw