6
A Perfect Storm for Intranet Search: How One Company Navigates J. Gregory Moxness, Moderator Missile Systems, Raytheon Corporation. [email protected] Kevin J. Lynch, Corresponding Author Raytheon Corporation. [email protected] Breanna Anderson SchemaLogic Corporation. [email protected] Beth Loring Design and Usability Center, Bentley College. [email protected] Christine Connors Raytheon Corporation. [email protected] Jule Zacher Verity Corporation. [email protected] Manya Kapikian Raytheon Corporation. [email protected] A “perfect storm” has formed in the intranet search space, where information is doubling yearly, no usable metadata exists, and search users’ expectations continue to increase. New information repository types, collaboration methods, and security protocols complicate the design of a comprehensive solution. Over the past two years, the moderator and participants collaborated to lead the successful design, development, and deployment of intranet search and browse capabilities for a large manufacturing organization. Significant lessons learned inform researchers and implementers of behind-the-firewall intranet search capabilities, and have implications

A perfect storm for intranet search: How one company navigates

Embed Size (px)

Citation preview

Page 1: A perfect storm for intranet search: How one company navigates

A Perfect Storm for Intranet Search: How One CompanyNavigates

J. Gregory Moxness, ModeratorMissile Systems, Raytheon Corporation. [email protected]

Kevin J. Lynch, Corresponding AuthorRaytheon Corporation. [email protected]

Breanna AndersonSchemaLogic Corporation. [email protected]

Beth LoringDesign and Usability Center, Bentley College. [email protected]

Christine ConnorsRaytheon Corporation. [email protected]

Jule ZacherVerity Corporation. [email protected]

Manya KapikianRaytheon Corporation. [email protected]

A “perfect storm” has formed in the intranet search space, where information isdoubling yearly, no usable metadata exists, and search users’ expectations continue toincrease. New information repository types, collaboration methods, and securityprotocols complicate the design of a comprehensive solution. Over the past two years,the moderator and participants collaborated to lead the successful design,development, and deployment of intranet search and browse capabilities for a largemanufacturing organization. Significant lessons learned inform researchers andimplementers of behind-the-firewall intranet search capabilities, and have implications

Page 2: A perfect storm for intranet search: How one company navigates

for information retrieval, recommendation systems, and designs for next-generationdigital workspaces. We attribute success to a focus on user-centered design, vendorselection and participation, and the diversity of backgrounds of the team responsiblefor design and deployment. Panel members consist of representative stakeholders ofall development groups who participated. Still, significant obstacles to continuedsuccess remain, and panel members have differing perspectives on how best toproceed.

Background

In June of 2003, a Raytheon intranet search and browse survey was administered to a randomset of Raytheon employees and had 199 respondents. Only 26% of respondents indicated thethen-current capabilities were “Good” or “Excellent.” In November of 2004, 3 months after a newdeployment of enterprise search capabilities to 60,000 users, 69% of respondents to the samesurvey indicated the new system was “Good” or “Excellent.” This turnaround was part of a $2.3Meffort that involved 3 internal company groups and 3 external vendor companies. Internalknowledge representation, content management, and intelligent search teams participatedclosely with external vendors across the project life cycle.

Project

Raytheon, like other large companies, manufactures products with long life cycles and employsan aging baby boom population headed for retirement. In this environment, knowledge transferhas become more important than ever. A significant amount of knowledge has been transferredover the years into the systems and processes that design and develop products; however,much rich tacit knowledge remains only with Raytheon’s most senior people. Knowledge transferefforts exist, such as a strong mentor/mentee program, and processes to capture andcommunicate lessons learned across the company. Often, though, lessons learned become“lessons accumulated” due to sheer size of the company and necessary knowledgecompartmentalization.

The Information and Knowledge Management program was funded in 2004 for $2.3M to beginto develop a multi-pronged system-based approach to this problem. A “knowledgerepresentation mirror” concept was developed using a metadata registry to keep a consistent,manageable representation of taxonomies and metadata, and to evolve into an ontologicalregistry for the company. The knowledge representation team developed and deployed over 20taxonomies and metadata schemas. The content management team employed the metadata

Page 3: A perfect storm for intranet search: How one company navigates

schemas for content publishers. The intelligent search team used metadata schemas andtaxonomies for information retrieval and navigation in enterprise search.

Search, classification, and recommendation engines -- combined with a content managementsystem and a metadata registry -- form the core systems supporting publishing, classifying,storing, and retrieval of document and web site information for the enterprise search application.Many lessons have been learned in the deployment of taxonomies for enterprise search. Webelieve the most significant problems were uncovered and addressed before deploymentbecause of the user-centered design approach we followed. The Bentley College UsabilityDesign and Testing Center provided state-of-the-art usability facilities.

While the project has initially been successful, many challenges remain. The recommendationengine currently captures objects transactions in the search engine, consisting almost entirely ofunstructured information. Thousands of structured repositories behind hundreds of applicationsstore related valuable information that could be leveraged in intelligent systems integrationscenarios. However, the recommendation, search, and classification engines are currentlyunable to “touch” these systems. A metadata registry approach is being taken to attempt asemantic integration of these systems.

Since enterprise search is “read-only,” we are extracting the models for a range of system types,and using the models to integrate the systems into our existing classification and navigationschemes. The set of recommended object types within enterprise search can then be expanded.Most importantly, we can continue our user-centered design on specific high-value opportunitiesin an “extensible search” environment we are deploying in 2005.

Format

Four-minute presentations will be given by 4 speakers on one or more of the issues outlinedabove. Then, brief insights of the other panel participants based on their involvement with theproject will be given, lasting no more than 10 minutes in total. A 90-second Flash demonstrationof the deployed system will also be shown. The panel will then posit the actions that should betaken given the obstacles still faced, and will briefly discuss differing positions. Audienceinvolvement will be encouraged during this time as part of each proposed action.

Proposed audience discussion items include:

How far can recommender systems take us in solving knowledge transfer problems?Is enterprise search the right place to start? If not, what is the best opportunity?

Page 4: A perfect storm for intranet search: How one company navigates

What about personalization? Especially in portal environments? Has this failed?What do we now know that can take us to a next-generation digital workspace?Can we expect semantic integration technology to keep apace of our problem?

Position Statements

Kevin J. Lynch

“Omniscient Search” is not here yet. Simple answers for search don’t exist behind a company’sfirewall. Information exists in so many systems and formats, with widely varying ability to(retroactively!) determine what authors’ intents were, or, going forward, who the correct audienceshould be for a given document or web site for a given query. This team understood thecomplexity and difficulty of the problem when it began, but had to determine what to focus on tobe successful in its funded time frame. In 2005, it is a certainty the average quality of searchresults will decrease if we only maintain our newly deployed system, and what is now required isa complete rethinking of how we can continue to maintain users’ confidence, and evolve from asimple search to “one-stop shopping” for search needs in the corporation. Making enterprisesearch an easy and consistently reliable starting point for a much wider variety of searchesacross the company is one of our biggest current challenges. How do we add capabilities andstill keep it “Google-simple?” I believe the Amazon-type recommender systems have their placein a solution. I believe portals will do more to hurt our current efforts than help, regardless of theirability to deliver “personalized” results.

J. Gregory Moxness

We believe that search is one of the most powerful tools for Knowledge Management. As thereare only two ways to get to information: 1) you can navigate to it if what you are looking for, knowit exists, and know where it might be found, 2) otherwise you need search.

Setting the context, I view the Everything from the point of view of physics. That is the Universe isa system and Everything is a process with inputs, outputs, resources (space-time, mass, andcharge) and feedback. Theoretically, physics also says everything can be represented asinformation. It also suggests that (by string theory) it is contained in 11 dimensional space.Whether this is indeed the case, I have used these notions to describe what I will call"information space". Each "dimension" of information space is a navigable / searchablestructure (aka taxonomy). In business terms, we represent processes, products (processinput/output), resources (space, time, people, $, ...), and feedback (business rules, constraints,

Page 5: A perfect storm for intranet search: How one company navigates

requirements) as taxonomies where intersecting nodes represent navigation and/or searchresults.

This idea would suggest that knowledge representation could be accomplished in asystems-theoretic structure using these basic objects. This would also imply that, in an idealworld, information and knowledge could be navigated or searched by both human and machineusing the same organizing principles suggested above. Of course, the typical user would needan advanced visualization for navigation of this new "information space". This project is thebeginning of understanding how to design a Knowledge Representation Architecture to enablethat kind of explicit knowledge search and navigation.

Beth Loring

Did the end users weather the storm? We think so, although there were several moments when itseemed the boat would capsize! This is because users often say they want certain features, butwhen they see the features implemented, they change their mind. For example, the earlyresearch indicated that users wanted to be able to drill down into repositories in complex ways.This made sense to the design team, which was made up knowledge architects and metadataspecialists, because they are expert searchers. But when the first sketch of the UI was shown tousers -- with its 5 different tabs for searching, browsing and viewing the results -- the end usersalmost jumped ship! After numerous participatory design sessions with end users, the UI wasdown to 3 tabs, the first being a simple Search box with various optional filters. And ultimately,this is what will be used in real life, when busy people are doing their jobs.

Could we have avoided the near-capsizing of the boat? Possibly. I believe that if a usabilityspecialist had been involved earlier in the project, the initial user research might have been donedifferently. If it were me, I would have validated and prioritized the search features up front byconducting focus groups and contextual interviews in the workplace. This would have reducedthe number of design iterations ... and reduced the money we spent on life preservers.

Christine Connors

Diversity of skill and thought is highly prized and much touted by the Fortune 500. Contracts,products and prestige are earned and lost based on a team's ability to perform. Navigation andsearch of a team's assets cannot be based on one version of the "truth;" nor should it bereflected in a fun-house mirror. Expert skills should be leveraged to devise, maintain and publish

Page 6: A perfect storm for intranet search: How one company navigates

schemata to enable teams to quickly find the data they need, in their own terms, regardless ofthe sub-culture that encoded it.

Jule Zacher

Companies everywhere want to improve the quality of search results without making the processcomplex - either for the user or the content owners. The question "what does the searcher know,that they are not telling us?," bundled with an understood set of search tools, can allow for thedevelopment of a successful search application. Far too often, neither of these are consideredand the vendor is brought into the project in the last stages. This late arrival adds frustration toboth the client and the vendor as the breadth of the search software has seldom beeninvestigated and often the application design plans for functionality run counter to the innerworkings of the software. The Raytheon search project addressed both understanding the usersand the search tools in advance of development resulting in an application that provided a bigleap forward for their users. The next step is to remember search development never ends andto budget accordingly.

The search tools available are often complex to describe without a working application using theclient's data. Due to time constraints, Raytheon's initial focus groups were conducted with onlywire frames depicting the search screens. This resulted in user confusion, many revisions of thewire frames, and some more complex features being dropped. A functioning application forsome the user assessment would have improved the design process.