This article was downloaded by: [University of Toronto Libraries]On: 20 December 2014, At: 05:24Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Cartography and Geographic Information SystemsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/tcag19

Focus Groups for Design Improvement in DynamicCartographyMark Monmonier & Myke GluckPublished online: 14 Mar 2013.

To cite this article: Mark Monmonier & Myke Gluck (1994) Focus Groups for Design Improvement in Dynamic Cartography,Cartography and Geographic Information Systems, 21:1, 37-47

To link to this article: http://dx.doi.org/10.1559/152304094782563948

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Focus Groups for Design Improvement in DynamicCartography

Mark Monmonier and Myke Gluck

ABSTRACT. Four focus group interviews provided an evaluation of the concepts of the graphic narrative andthe stage-and-play metaphor for dynamic cartography. The 26 information, cartographic, and computer spe-cialists who participated in the interviews provided a range of opinions on graphic scripts and the dynamicintegration of maps and statistical graphs. Respondents in each session first viewed a graphic script designedto explore the correlation between two spatial distributions and then discussed the script's informativeness,coherence, merit, and deficiencies. Respondents next viewed and discussed a two-part demonstration of portionsof a time-series script and of user-control enhancements for the correlation script. Participants found the graphicnarrative engaging and informative, were able to discern patterns in the data without identifying false patterns,and contributed a variety of suggestions and criticisms useful in refining both the prototype scripts and thetheory of narrative graphics. As a design improvement methodology, focus groups should prove to be useful inaddressing a broad range of cartographic problems.

KEY WORDS: design improvement, dynamic cartography, focus groups, graphic script, interface design,multimedia, narrative graphics, user needs

Introduction

This article describes an effort to evaluate and refinedynamic maps. The evaluation focuses on one of twoprototype graphic scripts described (Monrnonier 19922)

in a special issue of this journal published in late 1992. Thescript demonstrated a stage-and-play strategy for narrativecartography, which is part of the larger concept called atlastouring. Although various'articles published over the pastseveral years (Monmonier 1989; 1990a,b; 19922,b) have dis-cussed the potential of atlas touring, the two prototypegraphic scripts were the first attempt to make operationala multimedia strategy for integrating maps and graphs andfor extending dynamic cartography beyond interactivegraphics, computer animation, and temporally sequencedchoropleth maps. The present paper describes a strategyuseful in improving design concepts developed during pro-to typing by systematically soliciting the opinions and re-actions of typical users. The research described hereinrepresents a complementary collaboration between the co-authors: Mark Monrnonier, the author of the graphic scripts,and Myke Gluck, an expert in human-computer interaction(HCI). We collected data using focus group interviewing,a qualitative research method appropriate for exploratoryevaluations of interface effectiveness, and we worked to-gether in developing the focus group protocol and analyz-ing the results. The cartographic specialist (Monmonier)devised additional software for exploring the integration ofatlas touring concepts and interactive graphics, and the HOspecialist (Gluck) conducted the focus group interviews.

Mark Monmonier is a professor of geography at Syracuse Uni-versity, Syracuse, New York 13244-1160. Myke Gluck is anassistant professor in the School of Library and Information Stud-ies at Florida State University, Tallahassee, Florida 32J0~.

Our article has three parts. The first section discusses theuses and limitations of focus group evaluation, explains ourfocus group protocol and its rationale, describes our strat-egies for recruiting and conducting focus groups, and sum-marizes the backgrounds of participants. The second sectionexamines, condenses, and interprets participants' com-ments; and the final section relates these results to boththe prototype software and further implementations of theatlas touring concept. Our conclusions also address the wideruse of group interviews for evaluating user interfaces andgeographic software.

Focus Group MethodFocus groups are a low-cost, efficient qualitative methodfor investigation and design improvement (see note 1). Theycan be used to gain insight into the perceptions and reac-tions of different populations, and they are effective at manydifferent stages of needs assessment and product devel-opment. Although marketing and advertising research usefocus groups extensively, the cartographic literature haslargely ignored the utility of focus group interviews iri eval-uating alternative designs (Monmonier and Johnson 1991).

Focus Group InterviewingFocus groups consist of 1- to 2-hr discussions among fiveto ten individuals led by a trained moderator or facilitator.For a particular study, investigators often conduct severalsessions with different participants. These sessions are heldin a relaxed but private environment conducive to the freeexchange of ideas on a specific topic or theme. The facili-tator uses a precisely designed protocol consisting of meet-ing guidelines and a list of questions to lead the discussionand to keep it focussed. The facilitator does not read theprotocol verbatim but guides the discussion with purpose-

Cartography and Geographic Information Systems, Vol. 21, No.1, 1994, pp. 37-47

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

ful informality; brief exploration of significant detours ger-mane to the topic is permitted. Protocols usually includefrom four to eight key stimulus questions that address theprocess or product targeted for the group discussion. Thefacilitator actively uses additional questions called probesto encourage others to express differing views, to supportdirect communication among participants, and to furtherstimulate participants to present examples, supporting evi-dence, .or the logic behind their responses.

Focus groups were developed as an alternative to tradi-tional individual interview methods that used closed-endedquestions. These earlier methods limited the range of re-sponses and were seen as unduly influenced by the inter-viewer. Dissatisfaction with closed-form interviews led firstto nondirected interviews in which the interviewees, notthe interviewer, directed the interview. These techniqueswere refined to form the more systematic method of focusgroups (see Merton, Fiske, and Kendall 1956). Advertisingand marketing researchers have found focus groups a cost-effective means of better understanding consumer motivesand reactions. For example, film producers attempt to in-crease profits by filming several endings to a movie, usingfocus groups to evaluate these alternative endings, and re-leasing a film that provides the greatest audience satisfac-tion (Vichas 1983). Focus groups are especially appropriatewhen researchers need to identify the range of user reac-tions, preferences, likes, and dislikes (see note 2).

Focus groups assume that people form their opinions inisolation and that interaction with others can modify theseattitudes and perceptions. Group sessions encourage par-ticipants to reveal their points of view, emotions, and rea-soning. Although group discussion is easy for some peopleand hard for others, focus groups often have an effect sim-ilar to the candid interaction of strangers on a long train orbus ride; we might say more on a topic to a stranger thanto a person who we know well. Selection of participantssimilar in status or experience so that group members per-ceive themselves as equals also contributes to a free andopen exchange of views. Occasionally, researchers use theresults from one focus group to modify the protocol for thenext. More frequently, however, all groups for a particularstudy use the same protocol.

Classic focus groups consist of strangers with commondemographic attributes. Because groups of strangers arehard to form, social scientists often use groups in whichsome participants may know each other (McClure, Ryan,and Lauterbach 1992). Analysis indicates that acquaintancerelationships do not significantly distort the otherwise per-missive environment. An atmosphere of equality of statusis more critical than a group of complete strangers. Krueger(1988, p. 45) lists six limitations of focus groups:

1. The interviewer has less control than in an individualinterview. This frequently leads to inefficient repeti-tion and discussion of irrelevant issues. However, anexperienced facilitator can minimize these problems.

2. Data are difficult to analyze. Because subjects maycontradict themselves as the discussion proceeds, in-vestigators must be careful not to treat statements outof context.

38

3. Obtaining consistent, useful results requires the useof trained facilitators.

4. Groups with similar demographics discussing the sametopic often yield quite different opinions.

5. Recruiting participants and scheduling a mutuallyconvenient meeting can be difficult.

6. Finding a place and setting conducive to a free andopen discussion can be difficult.

Analyses of focus group data vary from very detailedanalyses to more cursory reviews of the major strengths,weaknesses, opportunities, and threats regarding the prod-uct, process, or idea under study. Generally, the ability tosee trends and patterns in the sessions addressing the ma-jor issues is the prime advantage of focus groups. Focusgroups are usually executed quickly and at lower cost thanmore extensive qualitative or quantitative methods. Focusgroups provide comments and suggestions that are oftenimplemented quickly. Researchers look for consensus amongparticipants as well as a range and diversity of responses.In the mode of exploratory data analysis, focus group an-alysts look for predictable information as well as for outliersthat might help them understand the processes at work.

The process of analysis must be deliberate and verifiable(Krueger 1988, p. 111). Although the trends in the data areoften qualitative, these findings should be clear to all re-searchers. That is, someone else looking at the same notesor transcripts should reach the same conclusions. To min-imize bias, analysts should consider the actual words usedby the participants; consider the context in which the wordsare stated; note shifts in opinions, if any, as the discussionproceeds; weigh statements based on experience moreheavily than those that are hypothetical; and emphasize thebig ideas that address the strengths, weaknesses, oppor-tunities, and threats associated with the product, process,or concept driving the research.

Reports on focus group research vary from presentationof raw data, to descriptive summaries of the data, to asummary of the meaning of the data (Krueger 1988, pp.120-121). Interpretations or expressions of meaning areusually part of a focus group report. Results often includesta tements such as "the prevalent feeling was ... :', "sev-eral participants strongly felt .... ", or "most participantsagreed that .... " Such reports tend to emphasize the bigideas and major issues. Most reports include direct quotesfrom participants (maintaining confidentiality) to clarify theparticipants' ideas, to get the reader's attention, and to pro-vide a more personal level of understanding. Reports areoften oral or 5-15 pages, and they include an executivesummary of "themajor findings and results (usually the lastto be written); a statement of the problem, questions, andmethods; a results section; discussion of the limitations ofthe study and alternative interpretations; conclusions andrecommendations; and appendices containing a copy of thefacilitator's protocol and the demographic survey used toverify the similarity of the participants.

Focus Group ProtocolOur goal was to evaluate the concept of graphic narrativesand to understand better users' reactions to dynamic spatial

Cartography and Geographic Information Systems

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

and statistical displays. The protocol included a generalintroduction to the goals of atlas touring, the required con-sent notice advising participants of their right to leave atany point, the directions to the facilitator for conductingthe demos, a closing "thank you" with an invitation tosubmit additional comments in writing, and, of course, alist of discussion issues and probes. The protocol, as ap-proved by the Syracuse University Human Subjects ReviewBoard, follows:

Distribute bio-questionnaire.Before we start, I have a short statement I must read:Your participation in this presentation of computer dis-played spatial information and the following discussionis completely voluntary. Your participation in this activ-ity will be kept confidential, and your name will notappear in any of the results or conclusions from thissession. The session will be tape-recorded, but only theresearchers will hear the tape to help refresh our mem-ory. The tape wiII be destroyed when its review is com-plete. You may cease participation in this activity at anytime for any reason without prejudice. This activity hasbeen approved by the Syracuse University Human Sub-jects Review Board.And now, I'd like you to take a minute or two to com-plete the short biographical questionnaire I handed out.Allow 2 minutes.Let me have the bio sheets, please. Thank you.Collect bio-questionnaire.OK? The purpose of inviting you here today is to helpus better understand how dynamic graphics, carto-graphic animation, and computer-human interactionmight lead to more informative, more efficient, and moreinsightful means of analyzing geographic data. We areinterested in particular in the possible value of a narra-tive approach to data analysis, as distinct from the morecommon "navigational" approach characteristic of GIS(geographic information systems) software and on-linelibrary information systems. In a navigational system,the user basically calls the shots within limits imposedby the software and the database. Under an interactiveapproach, the system controls the display and tells astory. This story, or script, can be a sequence of. maps,graphs, and text that address a particular query, or re-lationship, or perhaps provide a quick overview of a da-tabase. Rather than try to describe this in words, though,I am going to show you one complete script, and thenportions of a second script that demonstrate some ad-ditional dynamic graphic features of a narrative systemfor geographic data.And then, after you watch the demo for about 10 min-utes while I talk, we are going to address a few questionsto get your reactions, opinions, ideas, and suggestions.Because the demo runs at its own pace, I'd like you tohold your comments until I'm finished. But then, pleasefeel free to give me your honest opinions.Now we're going to look at a 10-minute demo that ad-dresses the correlation between two variables. The firstof these is a so-called dependent variable, whose spatialpattern we want to explain. And the second is the so-called independent variable, whose pattern we 'examineto explain and account for the first variable. But let'swatch it.Run and narrate Atlas Touring demo 1. When done, moveaway from system. Turn up lights (if dimmed).

Cartography and Geographic Information Systems

For the first or a new issue:OK? Now I want this group to address some specificissues. This is a focus group, so we need to focus onspecific topics in a pre-arranged sequence. So please tryto focus on the issue at hand. There will be an oppor-tunity later to address points you'd like to make that wedidn't get to. OK? Now the first of these issues is ...ORThat's good. Let's move on to another issue, which is ...

PROBESCan you give me a specific example?Why do you feel that way?What makes you see it that way?Does anyone feel differently?How do you see it?

Issue 1: LET'S START BY THINKING ABOUT THE IN-FORMATIVENESS OF THE GRAPHIC SCRIPT YOU ]USTSAW.Did you find this graphic sequence informative? Do youthink you have a better understanding now of the twospatial distributions and how they vary throughout theU.S. -at the state level, at least? What did the graphicscript tell you?

PROBEWas the sequence of graphic phrases, or scenes,appropriate?

PROBEDid you find the graphic script interesting? Did it makeyou want to explore the relationship further?

PROBE

Issue 2: LET'S THINK ABOUT THE COHERENCE OFTHE GRAPHIC SCRIPT.Did the progression of graphic phrases seem logical? Werethere any times when the transition seemed too abrupt?

PROBEDid the maps and graphs fit together?

PROBEWas there an appropriate or helpful use of color?

PROBEWas the text useful or necessary? Were the words,phrases, labels, and sentences helpful?

PROBEDid the text fit well with the maps and graphs, and didthe label at the bottom of the screen help you understandthe point (or purpose) of the graphic sequence?

PROBEWas it ever difficult to read the text while watching thescreen?

PROBE

Issue 3: LET'S THINK ABOUT SOME GOOD POINTSAND BAD POINTS OF THE GRAPHIC SCRIPT.What did you find most useful about what you saw?

PROBEDid anything seem useless? Or, was anything particu-larly confusing, or distracting, or frustrating?

PROBENow we're going to look at two relatively short graphicphrases selected from a much longer graphic script con-cerned with spatial-temporal data, and then I am goingto show you a short, interactive demo that illustratessome possible modifications of the narrative approach.

Run and narrate AT DEMO 2 (parts A and B). When done,move away from system. Turn up lights (if dimmed).

39

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

Issue 4: LET'S THINK ABOUT WHAT I JUSTDEMONSTRATED.What do you like most about what I just showed you?

PROBEWhat do you like least about what I just showed you?

PROBEIs it important to be able to control the pace and directionof the script?

PROBE

Issue 5:LET'STHINKABOUTHOW A GRAPHICSCRIPTMIGHT BE CUSTOMIZED TO THE INTERESTSANDNEEDS OF A SPECIFICUSER.How do you think any of what I showed you might becustomized to serve you or someone elsemore efficiently?

PROBEWhat kinds of preferences do you think might be use-fully-stored in a user profile?

PROBEIssue 6 (optional): LET'S THINK ABOUT HOW THISAPPROACH (THE NARRATIVEAPPROACH) MIGHTFIT INTO AN INTERACTIVE SYSTEM FOR GEO-GRAPHICALANALYSIS.Well, would it fit into an interactive system?

PROBE

Well, thank you for your help ....Because you might have some other ideas or sugges-tions, I am going to hand out a brief follow-up surveyon which I'd like you to jot down anything else thatoccurs to you that you think might be useful. There's astamped self-addressed envelope too. You might fill thisout later today, here, or at home. But please drop it inthe mail within the next day or so.And again, thank you.

A separate script was developed for the facilitator's soft-ware demonstrations. In order to have consistent demon-strations for the four groups, each of two softwaredemonstration sets was presented without participant in-terruption. After the basic introduction and consent state-ments, participants completed a short experience andeducation survey, used to verify the appropriateness ofparticipants and to identify relevant differences within andbetween groups.

The protocol included six issues for discussion: five forall focus groups and an optional sixth issue to be discussedif time permitted. Participants discussed the first three is-sues after the first software demonstration and the remain-ing two (or three) issues after the second demonstration.

The informativeness of the content of the correlation scriptwas the first issue. We sought a sense of what group mem-bers walked away with from the script, their sense of theappropriateness of the phrases and scenes, and their in-terest in further exploration of the relationship between thevariables presented. The second issue addressed the co-herence of the graphic scripts. This involved participants'assessments of the logic of the graphic sequences, the useof color, the visibility and role of text, and the integrationof maps, graphs, and text. The third issue pursued whatparticipants felt were the good and bad points of the cor-relation script, what they saw as useful, and what they sawas useless, confusing, distracting, or frustrating. After dis-cussion of this issue, the groups watched the second dem-

40

onstration set, which included a short time-series script andseveral user-control features involving the correlation script.

Groups then discussed the fourth issue, which probedthe good and bad points of the second demonstration set.This issue assessed attitudes toward user control and in-teractive systems. Issue five addressed how participantsmight like to see custornization or individualization of the"canned" demonstrations they just watched, and the rolea "user profile" might play in a larger system presentinggraphic scripts with user controls. The sixth issue, whichnot all groups discussed, investigated participants' viewson the role of the narrative approach and its fit with inter-active systems for geographical analysis.

As discussion of the various aspects of each of the fiveor six issues proceeded, the facilitator followed up partici-pants' comments with probes (Dervin and Nilan 1986). Aset of five probes was used to elicit additional comments:

Can you give me a specific example?Why do you feel that way?What makes you see it that way?Does anyone feel differently?How do you see it?

Not all probes are used with all comments; selection anduse of the probes are completely at the discretion of thefacilitator during the focus group. A trained and experi-enced facilitator makes the probes seem natural and variesthem to stimulate discussion without appearing stilted orrigid. Use of probes enhances the environment for sup-porting free expression, and permits digressions from theletter of an issue to explore the spirit of the issue. Theprobes also encourage all members of the group to partic-ipate, thereby limiting anyone member's ability to domi-nate the focus group.

The protocol also included sample phrasing for transi-tions between major issues. To begin the first issue, forinstance, the facilitator might say

OK? Now I want this group to address some specificissues. This is a focus group, so we need to focus onspecifictopics in a pre-arranged sequence. So try to focuson the issue at hand. There will be an opportunity laterto address points you'd like to make that we didn't getto. OK?Now the first of these issues is ....

To move to the next major issue, he or she might interject"That's good. Let's move on to another issue, which is .... "

The protocol called for repeated statements of thanks tothe members for coming and participating. The facilitatorclosed each focus group by handing out and discussing ashort follow-up survey that participants were encouragedto fill out and return in a stamped, pre-addressed envelope.This survey sought comments on either the content or con-duct of the focus group sessions.

Participants and SitesFocus groups held during the summer of 1992 at four dif-ferent sites collected data from 26 participants: five at theSchool of Information Studies (1ST)at Syracuse University,in Syracuse, New York; six at the National Geographic So-

Cartography and Geographic Information Systems

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

ciety (NGS) in Washington, D.C.; eight in the NationalMapping Division of the u.s. Geological Survey (USGS) inReston, Virginia; and seven at the International BusinessMachines Corporation (IBM) research and developmentcenter in Kingston, New York. Each session was held in asmall, comfortable conference room, with a Macintoshcomputer for the demonstrations and two tape recordersfor recording the discussion.

The five 1ST participants were advanced graduate stu-dents and faculty. All but one were members of library orinformation science associations. All used computers forelectronic mail and word processing, and most also usedspreadsheet and statistical packages. Experience in this fieldranged from 2 yr to over 10 yr. Most had 2-3 college sta-tistics courses. Two used Apple Macintosh computers, andthree used IDMor clone desktop computers. The three fe-males and two males ranged in age from 33 to 50 (see note3).

The six NGS participants were junior and senior carto-graphic researchers and GIS analysts. Most were membersof ACSM, ASPRS, or NACIS. Most used computers forword processing and mapping, and several also usedspreadsheet or image processing software. Experience inthe field ranged from less than 2 yr to more than 10 yr,with most having 6-10 yr experience. One had no collegestatistics course, but most had taken several such courses.Most used IDM or clone desktop computers, but two usedMacintoshes. The four females and two males ranged inage from 21 to 38.

The eight USGS participants were all geographers or car-tographers. Less than half reported membership in profes-sional associations, but those that did listed AAG, ASCM,NACIS, or URISA. Most used computers for electronic mail,word processing, mapping, or geographical analysis. Mosthad more than 10 yr experience, and the remainder had 6-10 yr experience. Only one had no college-level statisticscourse, while the others had taken at least two such courses.All used IDM or clone desktop machines, but several alsoused Macintoshes. The five men and three women rangedin age from 32 to 45.

The seven IDMparticipants were mostly application pro-grammers with varying levels of experience. Most reportedno professional memberships, but those who did listedACSM, AGU, IEEE, or URISA. Most used computers forword processing and electronic mail, and some also usedGIS or CAD software. Half had no formal college course instatistics, but a few had 2-3 such courses. Most used IDMdesktop or RISC 6000 machines. The six males and onefemale ranged in age from 24 to 60.

Carefully selected participants allow focus group inter-views to canvass a range of interested users or potentialcustomers and to obtain the views of various stakeholdergroups with interests in the concept, design, or product(see note 4). In our study, we sought to understand theviews of experts in the design and use of information ingeneral (IST), experts in the design of cartographic prod-ucts (NGS) and geographic analysis (USGS), and expertsin the development of software systems related to carto-graphic analysis (IDM). This goal, as well as personal con-tacts, led to the selection of the four institutions. Although

Cartography and Geographic Information Systems

we might have surveyed novice users, environmental an-alysts, public administrators, and other groups of stake-holders, the groups selected were most appropriate forevaluating the concept and potential of graphic narratives.

Demonstration SoftwareThe focus group sessions were divided into two parts, eachpreceded by a software demonstration. The first demon-stration addressed bivariate correlation, and the seconddemonstration consisted of two segments, one illustratingthe dynamic portrayal of spatial-temporal data and the othershowing how key introductory graphic phrases from thefirst demonstration might be made interactive.

The first demonstration consisted of the correlation scriptdescribed by Monmonier (1992a, pp. 254-255). It illustratesbasic elements of the stage-and-play metaphor used in bothprototypes, and its running time (approximately 11 min) isabout half that of the other script (approximately 19 min).Time was important because the focus groups were con-ducted in workplace settings where few people would vol-unteer for a lengthy session.

The second software demonstration illustrated howgraphic phrases can display spatial-temporal data and howa graphic phrase for exploring a distribution can be whollyor partly under user control. Its first part consisted of twoexcerpts from the second prototype script, the historicalscript, which examines spatial-temporal patterns in theAmerican daily newspaper industry (Monmonier 1992, pp.255-256). The first excerpt (from act two, scene two) is adynamic map showing rate of change for a 5-yr period thatmoves forward from 1900 to 1990, 1 yr at a time; it runsabout half a minute. The second excerpt, condensed fromthe third act of the historical script, shows the southwest-ward movement (between 1900 and 1990) of the centers ofmass for daily newspaper firms (in magenta) and for pop-ulation (in green); this segment runs approximately 2 min.

The second part of the second focus group demonstra-tion compares automatic and interactive approaches to de-scribing a distribution numerically and geographically. Theseapproaches offer an alternative to the initial scenes of thecorrelation script's first act. To provide a more graphicallyprecise representation of gaps, clusters, and the entire dis-tribution, as well as to accommodate larger numbers of datapoints, the univariate scatterplot in Figure 1 replaces thevertical bar graph used in the correlation script (Monmonier1992a, figure 1). A small open dot represents each state'sdata value. A text block above the graph identifies the cur-rent value represented by a movable breakpoint. The colorred highlights the part of the graph representing valuesless than the breakpoint value; red also highlights all mappolygons linked to dots within this interval.

A pull-down Preferences menu offers four user-con-trolled options: Manual, Drive, Automatic, and Regions.Manual mode is fully interactive: The mouse controls theposition of breakpoint, and the user explores the distribu-tion by moving the mouse left and right. In contrast, drivemode is partly automatic; the breakpoint moves automati-cally across the univariate scatterplot, but the user controlsits speed and direction by manipulating a marker in a ver-

41

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

Figure 1. Sample screen for the interactive portion of the secondfocus group demonstration.

Figure 2. Lower left-hand portion of the drive-mode screen, show-ing elements of the vertical drive column used to control the di-rection and speed of the breakpoint. Marker in the green zone callsfor a moderateforward (that is, to the right in Figure 1) movementof the breakpoint. Moving the marker upward would increase thespeed, whereas moving it downward slightly would reduce thespeed.

CoherenceThere was overwhelming consensus that the correlationscript was logical, coherent, and well suited to presentingbivariate geographic relationships. But despite a logical se-quence of scenes, most people felt overwhelmed with in-formation. Although the maps, text, and graphics workedwell individually, integrating them posed problems whentwo or more graphics shared the screen. Several viewersobjected to the need in act one to watch a map and histo-gram simultaneously. A typical viewer remarked, "I foundmyself watching the map more than the graphs. When the

Informa tivenessThe correlation script was informative insofar as partici-pants did not see a strong correlation where none existed.Viewers anticipating a clear, positive relationship betweenthe female labor-foree-participation rate and the female per-centage of locally elected officials were sometimes frus-trated or disappointed by the lack of a blatantly obviousvisual correlation. "I kept looking for some kind of pattern,and I didn't really see it," a typical participant remarked.But several people commented that the lack of a clear pat-tern was quite compatible with the moderately low Pearsoncorrelation coefficient (r = 0.31) and the map of residualspresented in the script's final act. Most important, no par-ticipants described a false or nonexistent pattern or rela-tionship. As one viewer noted, "I think it achieved itspurpose, which is that there is not a correlation. That'swhat I came away with." Many subjects wanted to slowdown the presentation or to stop and reverse it to viewindividual states and regional patterns more closely. "Thereis no time to think about what's occurring because the pre-sentation goes on to the next segment so rapidly."

In general, the correlation script was engaging as well asinformative. Many of the participants commented aboutspecific places and noted regional patterns. As one viewerobserved, "I saw a pattern [for the dependent variable] inthe East Coast and New England, but the Southwest seemedto have a lower correlation-it presented some parts [of thecountry] better than others." Despite complaints about thescript's speed and certain graphic effects, such as its harshblinking symbols, most participants agreed that scriptedgraphic sequences were both interesting and useful. "As adidactic or teaching tool," one noted, "I think it is verygood, but as a research tool I don't think so. With a goodregression chart, a trained statistician would probably findmost of this unnecessary."

process or choose another option until the second full sweepis finished. Regions mode is also automatic; the systemaddresses each of the nine census divisions in turn by dis-playing its name to the right of the map and temporarilyhighlighting its polygons and dots with a red fill.

Principal FindingsThis section summarizes reactions of participants to keyissues addressed in the focus group protocol. These find-ings reflect a consensus among participants across the fourfocus groups as well as minority opinions offering inter-esting or useful insights.

."~ 0 0

14%I••••

tical drive column to the right of the scatterplot. As Figure2 illustrates, the drive column is divided into three color-coded zones. Placing the marker in the green, "forward"zone at the top makes the breakpoint move forward, to theright. Placing the marker in the yellow, "reverse" zone atthe bottom makes the breakpoint move backward, towardto the left, and moving the marker into the red, "freeze"zone in the center halts the breakpoint in its current posi-tion. In drive mode, the mouse simulates a joystick; push-ing the mouse away from the user moves the marker upwardin the drive column, whereas pulling the mouse towardthe user moves the marker downward. The farther themarker is from the red, freeze zone, the more rapidly thebreakpoint travels along the univariate scatterplot. In au-tomatic mode, the third option, the breakpoint moves at aconstant rate, sweeps over and back across the scatterplottwice, and stops at the far left after its second sweep. Theuser cannot stop the breakpoint and cannot repeat the

42 Cartography and Geographic Information Systems

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

map went to different shades, I more or less forgot aboutthe graph."

The facilitator's preliminary briefing and running com-mentary made the presentation comprehensible. As oneparticipant remarked, "It was all logical, but without thenarrative it would have been difficult .... If 1 had to readthe description and watch what was going on, I would havehad a hard time." Viewers familiar with regression werepleased that the concluding scene summarized the scriptwith an analysis of residuals.

Because the weak bivariate relationship ran counter toviewer expectations, a preliminary interpretation would haveminimized confusion or uncertainty. As one participant ob-served, "It did not seem to have a lot of impact, whereasif it had been said right up front 'These are the two varia-bles and we are going to prove there's no correlation,' andthen as you showed all this data, then I could look for that.... " But had the bivariate correlation been stronger andmore blatantly obvious, interpreting the script's maps andgraphs before showing them would have been less useful.

Act two's spectrally encoded bivariate map, or cross map,puzzled several of the participants. Some people did notreadily understanding its purpose, and others were both-ered by its color scheme. Nonetheless, the groups acknowl-edged that, once explained, the cross map was a usefulpart of the overall narrative.

Good and Bad Points of Demonstration OneIn addition to the points already discussed, group memberscommented critically about various aspects of the correla-tion script. Most participants approved the use of signaturehues for differentiating dependent and independent vari-ables and the use of color for highlighting noteworthy partsof the display. "I think it's good to use different colors fordifferent variables," one viewer commented. "I think theuse of color and the shading definitely brought the pointhome," another said. Participants also applauded the useof maps to present regional information and the use ofgraphs to describe univariate distributions and their statis-tical correlation. One noted that "It was great to have thehistogram there because you got that level of understand-ing of the data as well as the spatial distribution." Yet thehistogram would have been more useful, many believed,had viewers been able to scan it at leisure. The scatterplotdrew fewer comments. "When I saw the scatterplot," onesatisfied viewer said, "I felt a lot more comfortable." Incommenting on the automatically brushed scatterplot at thebeginning of act two, another participant suggested thathighlighting all points (and corresponding polygons) belowor to the right of the brush might have been more revealingthan highlighting only points instantaneously within thebrush. The alternagraphics sequence at the end of act onealso drew few comments; viewers seemed to appreciatethat the map's throbbing shift between red and blue re-flected a weak correlation.

Aside from the presentation's fast pace and lack of usercontrol, the most consistently negative response focusedon the blinking symbols employed in the introductory scenesof act one for both a category-by-category canvass of thechoropleth map and a regional canvass of the map and

Cartography and Geographic Information Systems

histogram. Members of all four groups objected to flashingpolygons or bars: "1 find the flashing really offensive." "Theflashing was so distracting, especially while studying thehistogram." Although more effective use of the computer'scolor look-up table might have provided smoother, less harshblinking, some viewers opposed any display that simulta-neously blinked more than one symboL Despite these neg-ative reactions, a few participants thought canvass-by-category flashing contributed to a rapid comprehension ofthe choropleth map's categories and symbols.

Good and Bad Points of Demonstration TwoFollowing the second, two-part demonstration, partici-pants were most eager to talk about the user control offeredin the second part of the demonstration. Although viewersexpressed a general interest in the various modes for ex-ploring a univariate distribution cartographically and sta-tistically, almost all favored the manual mode over theautomatic and drive modes: "I did not like any of the au-tomatic stuff-too noisy and too fast .... " "The interactiveability was nice where you could participate more, to go atyour own pace for as long as you like." "I liked the partwhere you could play with it yourself." "Assuming an in-teractive mode were available during a canned demo, itwould be useful to just to hit the escape button, freeze thedisplay, and then go interactive." Drive mode was leastpreferred. "The drive seemed degraded; I wanted morecontrol at my fingertips." But automatic mode had somesupport: "I liked the overall package, and the options onthe amount of controL You could sit back and let it run.""It's like a 35-mm camera. It's nice to go manual, but some-times it's nice to have automatic."

The region mode seemed useful but drew little comment.Substitution of a univariate scatterplot for the histogrambelow the map also seemed generally acceptable, althoughone viewer objected, "I would like to have seen the his-togram there instead of the [univariate scatterplot]."

In discussing the dynamic maps examining the news-paper industry in the first part of the demonstration, par-ticipants expressed .a general need for some control overthe pace and direction of graphic scripts. A few also wantedto be able to alter category breaks on the choropleth mapsand to isolate groups of states for separate analysis. "Con-trolling the speed is important but the ability to mix andmatch variables is important too." Several viewers wouldhave liked to stop the action at key years, such as 1929 or1941.

The spatial-temporal maps in the first part of the dem-onstration drew a variety of critical comments, positive andnegative. The dynamic map showing the percentage changein number of newspaper firms moved too rapidly for manyviewers. "We were in 1945 before I was even sure of whatI was looking at," one noted. Another objected, "It washard to get because it seemed that lots of places turned redall of a sudden, and it was hard to catch what was causingit-like economics or what?" A few participants were trou-bled by the use of red to represent gains and blue to rep-resent losses. Others had difficulty grasping the map'stemporally variable percentage rate of change. One partici-pant wondered, for instance, why a map showing change

43

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

over a S-yr time span was moving ahead 1 yr at a time. Incontrast, viewers seemed favorably impressed by the dy-namic centrographic map presenting a horse race betweenpopulation and daily newspapers. As one viewer stated,"1 liked it when the two variables were chasing each other."Another commented on the logical sequence of animatedgraphics: "1 liked the way you showed the population cen-ter move separately and then showed the newspaper cen-ter. It was nice to see how they proceeded across the maptogether in time, not just how they laid there together. Youcould see how one moved faster than the other." One viewersuggested that a progressive zoom from the U.S. map tothe more detailed map centered on Illinois would be clearerthan the brief, abrupt display of a locator inset map.

CustomizingIn discussing customization, the participants offered a va-riety of suggestions for improving graphic scripts, althoughnot all of these recommendations could be implementedthrough a user profile or "preferences" file.

Typical candidates for customization included the speedor pace of the script, the number of repetitions for specificgraphic phrases, the number of categories and classificationschemes for choropleth maps, the regional divisions em-ployed in canvass-by-region sequences, the time span forthe percentage rate of change portrayed on a dynamic spa-tial-temporal map, and the color schemes used for signa-ture hues and bivariate maps. User profiles might alsocontain preferences for or against specific kinds of displaysor graphic phrases, such as histograms, univariate scatter-plots, and cross maps. In addition, graphic scripts mightconveniently be customized by highlighting or providingcomparatively detailed treatment of specific regions, states,places, variables, or periods of time.

Several people recognized the need to adjust the presen-tation to the knowledge level of the audience. New usersmight elect a user profile designed for the typical "novice,""intermediate," or "expert," and after using the system foran hour or so make whatever changes seem worthwhile.Most participants felt that the more experienced the user,the less appropriate a narrative approach. Very experiencedusers would probably prefer a tool kit of graphic phraseswith which to construct their own presentation scripts bymixing and matching graphic functions. In contrast, lessexperienced users might appreciate a slow, carefully pacedtutorial explaining some of the more complex, less familiargraphic phrases. Tutorial scripts that verify understandingby asking a series of questions about the display, the data,or sample patterns or relationships might be useful.

Another helpful customization is a mechanism for re-cording the analysis that had been done and the conclu-sions that were reached. A "print" or "save" facility mightprovide a "travel history" record so a user might easilyrecall scripts and phrases that had previously been viewed.If the analysis had not been completed in one session, thisinformation would be useful if the user wanted to continueexploring the data at a later time. Also useful is an anno-tation process that allows the viewer to attach interpreta-tions, reminder notes, and observations about particularly

44

revealing aspects of what he or she had seen. One 'partici-pant suggested an automatic means for monitoring usageto track options viewers really employed when given theopportunity to select various graphic phrases and composetheir own narratives.

Participants also mentioned a number of other enhance-ments. Several viewers saw the need for audio within thenarrative software to replace the voice-over provided by themoderator during focus group sessions. If a speech syn-thesizer were used, the gender, volume, and pitch of theoral narrative could be customized to the user's prefer-ences. Viewers might also be allowed to select either a terseor a comparatively detailed commentary. One person wanteda "fuzziness" control with which the user could vary thesensitivity of a display or analysis. At a low-fuzziness set-ting a dynamic spatial-temporal map might show all stateswith an absolute rate of change exceeding, say, 3 percent,whereas at a higher fuzziness threshold the display mightshow only those states where the change exceeded 10percent.

Implications for DesignWhat do these findings suggest to authors of graphic scriptsand designers of scripting software? Some messages fromour focus groups are obvious, for instance, the value ofuser control over the script's pace and direction. Equallyclear is the need to avoid harsh blinking symbols, dynamicdisplays with simultaneous multiple visual foci, and per-plexing cross maps with counterintuitive color schemes.Fully authored graphic scripts frozen on a noninteractivemedium such as videotape require careful attention to du-ration, speed, and visual complexity. Other messages fromthe focus groups are more subtle. Many objections andfrustrations, we believe, can be avoided by carefully pre-paring viewers for what they are about to see and by tellingthem what to look for.

Several participants maintained that graphic scripts - butnot necessarily graphic phrases-are inherently more use-ful as tools of presentation than as tools of analysis. Weshare this view. Analysis, after all, is too personal, toospontaneous, and too seat-of-the-pants idiosyncratic to del-egate to fully authored graphic scripts similar in design tothe two atlas touring prototypes. Other participants madethe equally important point that graphic scripts can be valu-able didactic devices. This second observation, with whichwe also agree, helps clarify the graphic script's role inanalysis. In essence, scripting can promote analysis in atleast two important ways, by introducing and summarizingunfamiliar datasets and by explaining and illustrating un-familiar methods of graphical analysis. Because exploratorydata analysis (EDA) seems destined for an increasingly im-portant role in geographic research, graphic scripts can beuseful vehicles for indoctrinating analysts in both canonicaland alternative approaches to the use of EDA tool kits.Moreover, analysts who compose scripts for scientific peersor legislative leaders can promote thoroughness and co-herence by previewing and editing their presentations.

We think it useful to distinguish between open scripts,with which the user can interact, and closed scripts, which

Cartography and Geographic Information Systems

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

brook no interference. The correlation script shown to focusgroup participants clearly is a closed script. If the revisedscript is also closed, the sequence of maps and graphicswould be largely unchanged, although the presentationwould progress at a noticeably slower pace and a voice-over, moderator, or block of highly legible text would an-nounce beforehand the purpose and interpretation of eachscene. Moreover, scenes that had simultaneously flashingsymbols would instead merely highlight relevant elements~nd then pause for viewer inspection. Other pauses wouldoccur at the end of each scene as well as at places withinscenes where the viewer might want to ponder a dynamicsequence or examine a static display.

Uncertainty about the appropriate duration of these pausessuggests the need for a modest yet essential degree ofopenness: Unless captured on a totally closed medium suchas videotape, the script ought not advance to the next sceneor graphic phrase until the viewer or moderator presses a"resume" key. This minimal openness avoids confrontingviewers with new information until they have been able todigest what they have just seen, read, or heard. A "rerun"key that repeats the immediately preceding sequence wouldprovide even greater openness.

How much additional 'openness is necessary and useful?Resume and rerun keys are comparatively straightforwardoptions that the viewer should readily comprehend. In con-trast, a "freeze" key is largely useless without a "backup"key because the viewer who considers a pattern worth pon-dering is unlikely to react instantly to halt the forwardprogress of the script. A better strategy might be a thirdoption that makes interactive a display that moments be-fore had been automatic. For example, the viewer who hasjust watched the system automatically brush a scatterplotor sweep a single class break across the distribution graphbelow a two-category univariate map should be invited tomanipulate the brush or class break and to explore the datadirectly.

Although not every scene or graphic phrase in the pro-totype scripts is readily explored in "manual" mode, manyscreens can be redesigned to provide slider bars the usercan grasp and manipulate. With a quantile choropleth map,for instance, the viewer might use a slider bar to controlthe number of categories. With a four-category cross map,the viewer could either use a slider bar to highlight specificcategories or use a mouse or track ball to vary the classifi-cation's bivariate breakpoint. For the map of residuals linkedto a scatterplot in the final act of the correlation script, apair of slider bars could let the viewer (1) highlight positiveresiduals, negative residuals, or both, and (2) vary the min·imum magnitude of the residuals highlighted. For the dy-namic spatial temporal map shown at the beginning of thesecond focus group demonstration, a viewer might userone slider bar to vary the length of the period for calculatingthe rate of change and another slider bar to move the timeperiod forward or backward in time.

However open or interactive the script, customizationcan avoid confusion and promote efficiency. If the corre·lation script were to be rewritten, it would best emerge asa software package with not one but several scripts. At leastone version should be customized for viewers with little or

Cartography and Geographic Information Systems

no experience with statistical maps, whereas another ver-sion should accommodate viewers well versed in spatialanalysis and statistical graphics. Ideally, the neophyte's scriptwould include optional tutorial segments to introduce theviewer to the use and interpretation of scatterplot brushing,bivariate cross maps, and residuals from regression. In con-trast, the expert script would allow the viewer to excludeoutliers from the regression analysis and substitute lowesssmoothing (Cleveland 1979) for least-squares linear regres-sion. Other informative enhancements for experienced an-alysts are spatial statistics such as the Moran coefficient toassess the effect of spatial autocorrelation (Griffith 1987)and geographic templates to explore the roles of spatialtrends and regional patterns on bivariate correlation (Mon-monier 1990b).

Because some software enhancements are interdepen-dent, the next most appropriate level of openness is a fullyinteractive system providing menus with which the analystcan select variables, edit and view the data in a spread-sheet, customize the display, and activate specific graphicphrases and analysis functions. At this more advanced level,scripting can affect the design and use of an interactivesystem in three ways. First, the layout and organization ofthe menus can reflect a narrative. For example, the menubar at the top of the screen could present the correlationscript's three acts as a group of adjacent pull-down menus,with act one on the left and act three on the right. Eachpull-down menu could list its act's text blocks and graphicscripts in a narrative sequence from top to bottom so thata viewer could recreate the script in its intended, originalorder as well as jump back freely to a particular segment.Second, a separate command window could present thescript's text blocks and graphic phrases as a list of shortdescriptors arranged in narrative order. In its fully auto-mated, "player piano" mode, the system would advancethrough this list from top to bottom, highlighting each de-scriptor in the command window while displaying thecorresponding text block or graphic sequence in a largerview window. In contrast, a user electing the system's im-provisational mode could scroll up or down the commandwindow and select for viewing individual descriptors orportions of the script. The system's menus could, of course,offer a richer, more flexible set of views and functions.Third, background scripts might screen the data for poten-tially interesting and meaningful patterns that a viewer oughtnot overlook. Besides providing a tool kit for. exploratoryanalysis, the system can actively help the user identify in-triguing questions or plausible solutions.

A script introduced in fully automatic mode might beespecially useful to the user who wants a brief overview ofa new dataset or quick instruction in a new method ofanalysis. Whether viewed automatically or freely, the nar-rative list of views in the command window could dem-onstrate how an expert would use the system's graphicphrases and analysis functions to address bivariate corre-lation or a spatial-temporal distribution. A library of suchcanonical scripts for a variety of specific analytical and dis-play objectives should be especially useful for exploratorydata analysis, which lacks the recipes available to moremathematically oriented forms of quantitative analysis.

45

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

Moreover, in the same way that experienced cooks userecipes to communicate with peers and apprentices, ex-perienced graphic analysts who submit canonical scripts toa review panel for critical evaluation and validation mightuse graphical recipes to reach a mass audience of beginnersand professionals.

A final salient observation from the focus groups is theneed for a commentary or voice-over. By providing an ef-ficient and effective link with familiar facts and relevantconcepts, natural language establishes a context that makesdynamic graphics meaningful. Yet because text blocks areefficient principally for the lone user who wants to reviewa complex description of the data, spoken text tends to bemore effective than visual text. Although a single viewermight conveniently read a screen of text and then hit theresume key when finished, text blocks are awkward formultiple viewers differing in both experience and readingspeed. Moreover, because some information is best im-parted while the user is watching the display, written textcan distract the viewer whereas spoken text can interpret,inform, and reinforce. But even though a watch-and-listenstrategy might seem more efficient than a listen-then-watchstrategy, both approaches can be combined, the first to alertthe viewer to the goals of the dynamic display that followsand the second to promote an accurate interpretation. Be-cause most users will not have a moderator present, syn-thesized or recorded speech will play an important role inthe development and implementation of graphic scripts.

Concluding RemarksParticipants contributed numerous insights, direct and in-direct, useful in refining the prototype scripts and enhanc-ing the theory of graphic narratives. These results dem-onstrate that focus groups are a relevant data-collectionstrategy for improving the design of maps, dynamic or static.Although largely subjective in administration and interpre-tation, group interviews can play an important role inadapting any atlas, electronic or print, to users' needs andlimitations. After all, design itself is a subjective, largelywholistic process; and focus groups appropriately avoid thelimitations of inherently narrow subject-testing strategiesmore suited to evaluating hypotheses than to refining com-plex presentations. Narrative graphics and other new formsof cartographic expression warrant an exploratory phase ofpragmatic development concerned with adapting and per-fecting a graphic presentation incrementally, to the pointwhere "Does it work?" becomes a meaningful question.Wholistic or Gestalt reactions obtained with focus groupscan contribute far more to this development phase thancontrolled testing of individual features.

NOTES

1. For more complete descriptions of the focus groupprocess, see the texts by Kreuger (1988) and Morgan(1988).

2. Another qualitative method we considered was "talk-along activities" in which subjects actually perform tasksusing a product such as a software prototype (Ericssonand Simon 1980). Recording the subjects' verbalizationsas they work provides a base for analyzing reactions and

46

opinions. This method seemed more appropriate for laterevaluation, after another design cycle responding to whatwe learned from the focus groups. It would be too sen-sitive to the particular idiosyncrasies of the implemen-tation, and, therefore, not as good as focus groups forthe deep evaluation of the conceptual basis for graphicnarratives that we sought. We anticipate using the talk-along technique at a later stage to assess the usability ofthe scripts themselves.

3. Some specific details about the participants are omittedor generalized to protect the anonymity and confiden-tiality of the individuals.

4. Although our focus group samples do not meet the cus-tomary requirements for rigorous statistical testing, thisis not a serious drawback. Focus group data are quali-tative, not quantitative, and our goal was increased in-sight, not parameter estimation. Moreover, the largersamples required for confirmatory statistics would havebeen difficult to recruit. Although these shortcomingslimit our ability to generalize, they do not preclude anincreased understanding of the issues.

ACKNOWLEDGMENTSThe support of National Science Foundation (grant no. SES-90-22845) is gratefully acknowledged.

REFERENCES

Becker, Richard A., and William S. Cleveland. 1987. Brushing Scat-terplots. Technometrics, vol. 29, pp. 127-142.

Cleveland, William S. 1979. Robust Locally Weighted Regressionand Smoothing Scatterplots. Journal of the American Statistical As-sociation, vol. 74, pp. 829-836.

Dervin, B., and M. Nilan. 1986, Information Needs and Uses. An-nual Review of Information Science and Technology, vol. 21, pp. 3-35.

Ericsson, K., and H. A. Simon. 1980. Verbal Reports as Data. Psy-chology Review, vol. 87, no. 3, pp. 215-251.

Griffith, Daniel A. 1987. Spatial Autocorrelation: A Primer. Washing-ton, D.C.: Association of American Geographers.

Krueger, Richard A. 1987. Focus Groups: A Practical Guidefor AppliedResearch. Beverly Hills, California: Sage.

McClure, Charles R., Joe Ryan, and Diana Lauterbach. 1992. TheRole of Public Libraries in the Use of Intemet/NREN InformationServices: Preliminary Findings. In Proceedings of the American So-ciety of Information Science, Midyear. Meeting, Albuquerque, NewMexico, May 27-30, pp. 46-61.

Merton, Robert K., M. Fiske, and P. L. Kendall. 1956. The FocusedInterview. Glencoe, illinois: Free Press.

Monmonier, Mark. 1989. Graphic Scripts for the Sequenced Vi-sualization of Geographic Data. In Proceedings of GIS/LIS '89, Or-lando, Florida, November 26-30, pp. 381-389.

Monmonier, Mark. 1990a. Atlas Touring: Concepts and DevelopmentStrategies for a Geographic Visualization Support System. CASE Cen-ter Technical Report 9011, June, 1990. New York State Center forAdvanced Technology in Computer Applications and SoftwareEngineering. 60 pp.

Monmonier, Mark. 1990b. Strategies for the Interactive Explorationof Geographic Correlation. In Proceedings of the 4th InternationalSymposium on Spatial Data Handling, Zurich, Switzerland, 1990, pp.512-521.

Monmonier, Mark. 1992a. Authoring Graphic Scripts: Experiencesand Principles. Cartography and Geographic Information Systems,vol. 19, no. 4, pp. 247-260, 272.

Cartography and Geographic Information Systems

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014

Monmonier, Mark. 1992b. Time and Motion as Strategic Variablesin the Analysis and Communication of Correlation. In Proceed-ings of the 5th International Symposium on Spatial Data Handling,Charleston, South Carolina, pp. 72-81.

Monmonier, Mark, and Branden B. Johnson. 1991. Using Quali-tative Data Gathering Techniques to Improve the Design of En-

Cartography and Geographic Information Sy~tems

vironmental Maps. In Proceedings of the 15th InternationalCartographic Conference, Bournemouth, U.K., vol. 1, pp_ 364-373.

Morgan, D.L. 1988. Focus Groups as Qualitative Research. NewburyPark, California: Sage.

Vichas, Robert P. 1983. 11 Ways Focus Groups Produce Profit-Making Ideas. Marketing Times, vol. 30, no. 2, pp. 17-18.

47

Dow

nloa

ded

by [

Uni

vers

ity o

f T

oron

to L

ibra

ries

] at

05:

24 2

0 D

ecem

ber

2014