View
213
Download
0
Category
Preview:
Citation preview
Running head: VIRTUALLY THERE 1
Virtually There:
An Evaluation of the Usability of American Art Museums’ Digital Collections
Carolyn English
17:610:553:90
School of Communication and InformationRutgers University
April 27, 2015
VIRTUALLY THERE 2
Abstract
American art museums display some of the most beautiful works of art on their own walls, but
how do their virtual collections look? In this paper, the usability of four museums’ digital
collections—The Art Institute of Chicago, Museum of Fine Arts, Boston, National Gallery
of Art, and Philadelphia Museum of Art—were reviewed, rated and compared to reveal some
of the most user-friendly features of current digital collections. Specific elements of the user
experience in this case relate to searching and browsing capabilities, metadata, images, social
media, and help resources. Both qualitative and quantitative methods were employed to paint the
most detailed picture of each collection. Efficient faceted searching and detailed images were
some of the author’s favorite features, while poor navigation through search results and a lack of
help resources were some of the most troublesome areas.
VIRTUALLY THERE 3
Virtually There:
An Evaluation of the Usability of American Art Museum Digital Collections
American art museums display some of the most beautiful works of art on their own
walls, but how do their virtual collections look? Museums of the future will still be revered and
visited for the masterpieces on display, but they will also be accessed virtually for detailed
information and images of their entire collections, both on and off display. These virtual
museums will need to satisfy a range of users—from travelers planning visits to those who wish
to reflect on their visits to artists to students and educators. It is critical that they provide a user
experience that satisfies people with a variety of abilities and intentions while also promoting the
museum’s reputation through good design and an efficient interface.
The author was inspired by the Smithsonian Institute’s impressive 3D scanning project
(http://3d.si.edu/), which will provide virtual visitors with as close to an actual physical
experience as possible by intricately capturing its artifacts using 3D technology. Art museums
likely have a long way to go before taking on a project like this, but they face the same future
with the need to develop and promote a strong online presence. It is this prospect that calls for a
review of the current usability of art museums’ digital collections.
In this study, four traditional American art museums’ digital collections were explored—
The Art Institute of Chicago, Museum of Fine Arts, Boston, National Gallery of Art, and
Philadelphia Museum of Art. Several specific factors relating to the user experience were
described, quantified, analyzed and compared to determine strong and weak user experience
features. The author walked through each website examining the overall design, like color
scheme and interface, searching and browsing navigation, metadata, images, social media
presence, and help resources. Certain elements, like number of advanced search fields, were
VIRTUALLY THERE 4
measured, while other features, like quality of images, were subjectively quantified on a 5-point
scale to provide a clear comparison of the databases. Commentary is also included to paint a
picture of the usability of these four digital collections.
Literature Review
There has not been extensive research completed on this exact topic, but there are many
discussions surrounding the concepts of usability, user experience, digital libraries and cultural
heritage collections, social media, and visual resource management.
Before discussing the specifics of the user experience, we should fully understand the
need for it in the art museums of today and especially the future. In a study of use policies on art
museum libraries, Esther Roth-Katz (2012) stated that the benefits are two-fold, “Not only is
there a belief that the public is entitled to access to resources, but that increasing awareness of
these resources can be mobilized as a marketing tool for the museum itself.” But while such
access is expected, “Art museums must consider the implications of allowing open access to
valuable and fragile resources” (Roth-Katz, 2012). Such precious information has never been so
freely available, and museums must adjust to the new infrastructure. Unfortunately, there has not
been much formal research completed on the matter. Roth-Katz calls for more, which should
ultimately benefit both the institutions and the users of their websites, so the author would like to
contribute to the remote access conversation.
It is also important to understand who the users are. Pallas and Economides (2008)
described the diversity of virtual museum visitors in their study of the evaluation of art museum
websites in which they developed a framework to do so, “A wide number of users, from pre-
schoolers to art historians and researchers, may visit digital museums.” Kris Wetterlund’s
research from 2008 focused on bringing the art museum to the classroom. The development of
VIRTUALLY THERE 5
their digital collections has been a welcome resource to educators. “Before the existence of
digital images of works of art from art museums, teachers who wanted to use works of art in the
classroom arranged field trips to the museum or were restricted to slides, overhead
transparencies, or printed reproductions” (Wetterlund, 2008). So if educators are utilizing their
collections, so might the students for assignments or for independent use should they be inspired
by their education. Other users may include people planning a trip or revisiting a work of art,
time period, or style of art from a previous visit.
Now users have been established, what they wish to do online also determines the
requirements of the digital collection. A recent study exploring web searching behavior for
cultural heritage institution visitors (Skov and Ingwersen, 2014) described four motivational
categories of visitors to remote access visitors to the Science Museum in London in 1998:
Gathering information for planning an upcoming visit to the physical museum;
Self-motivated research for specific content information;
Assigned research for specific content information; and
Engage in casual browsing.
Again, the finding is that a digital collection must be comfortable for a novice user to browse,
but rich enough to benefit a serious student or professor in the field.
This growth of virtual visitors might make one question the consequences to physical
visitation. Wetterlund (2008) calms museum professionals’ worries by citing a study performed
by the Minneapolis Institute of Arts that was funded by the Institute of Museum and Library
Services, which “found that 78% of online visitors to their Web site had been to the real museum
as well.” So if anything, a strong virtual presence only encourages visitation or enhances the
overall experience by extending the access beyond the buildings’ walls. “Another study found
VIRTUALLY THERE 6
that most visitors to the museum Web sites are scholars and teachers” (Wetterlund, 2008). Since
many of the visits seem to have an educational focus, the need for a digital collection similar to a
library is likely the experience these users are used to and would expect from a similar
institution, because a museum is quite comparable to a library of art.
Interestingly, just like information literacy for librarians, museum literacy remains an
issue when utilizing their digital collections, especially when it comes to K-12 instruction. This
tells us that any supplemental information, particularly in a narrative form, would greatly
enhance the experience for those who are not well-versed in technical art terms or have a pre-
existing knowledge of the works. Wetterlund (2008) explained that museums have posted more
educational resources on their websites. Seamlessly including supplementary information within
the search database would enhance the user experience for both novices and experts. The need
for museum or information literacy doesn’t just apply to users but museum professionals too.
Paul Marty (2006) conducted interviews with 21 museum professionals to determine what skills
were necessary to enhance their work. He discovered that “there is a growing sense among
museum professionals that the skills taught in Library and Information Science (LIS) programs
are increasingly relevant to their everyday needs” (Marty, 2006). This type of expertise would
directly contribute to the type of information available in digital collections and their design,
which is critical, since “the emergence of this new role has coincided with the growing belief
that museum information resources should be as readily available as information resources are in
most libraries” (Marty, 2006). This is why a review of art museums’ digital collections from the
perspective of a budding information professional is relevant. But while this is simply a review
of the current state of affairs, future direction should also be considered. Collections will only
continue to improve.
VIRTUALLY THERE 7
Eventually, as technology is developed and more readily accessible by a range of
budgets, museums will likely follow the Smithsonian in providing even more interactive and
realistic virtual visit. Patel et al. (2005) described a new project called ARCO, or Augmented
Representation of Cultural Objects:
It is notable that although many museums have now established an online presence on the Internet, currently this presence is almost invariably a 2D one; that is associated Web sites comprise 2D images and textual descriptions. ARCO [Augmented Representation of Cultural Objects] on the other hand recognizes that objects are 3D in nature, that they have a front and back, top and bottom, mass and volume. ARCO seeks to enhance the awareness and experience of cultural objects by providing technologies for creating 3D digital surrogates of artefacts and allowing user to interact with them.
The trend is catching on, so museums must be prepared to update their digital collections
to become even more dynamic and user-friendly.
So, when user experience is mentioned, what exactly does it mean? Hariri and Norouzi
(2011) performed an extensive literature review to determine specific criteria for evaluating
digital libraries, which helped direct the author’s checkpoints. What is interesting is that they
admit that specific identification of such information is lacking. At the same time, “the quality of
user interface design has improved dramatically” (Hariri & Norouzi, 2011). They listed 22
specific factors in order of usage in evaluations, of which 11 matched the purposes of this study
(highlighted in bold font): navigation, searching, design, guidance, error management,
presentation, learnability, user control, consistency, language, feedback, ease of use, match
between system and the real world, customization, user support, user workload, interaction,
compatibility, visibility of system status, user experience, flexibility, and accessibility. Xie
(2006) also provided a detailed list of usability. Not only is there the idea of usability in general,
but interface usability pertains specifically to search and browse, navigation, help features, view
VIRTUALLY THERE 8
and output, and accessibility (Xie, 2006). Clearly, there are many factors that overlap and
contribute to the overall experience.
Such factors also require some context within the concept of usability. Even evaluation
itself requires context, as it can pertain to technical or conceptual ideas. The question could be
posed as whether or not a system performed a certain function, how it performed it, or how its
performance affects its user. This is an idea of a user-centered evaluation versus a system-
centered evaluation. Saracevic (2000) provided two questions to demonstrate the difference. A
user-centered evaluation might ask, “How well does a given interface provide and support
access, searching, navigation, browsing, and interaction with a digital library?”, whereas a
system-centered evaluation might ask, “How well is the collection or information resources
selected, represented, organized, structured, and managed?” He encourages the use of both, and
for this, both quantitative and qualitative functions will be examined so both areas are
incorporated into the overall comparison.
Another layer in the user experience is the user’s point-of-view. Xie (2008) reported on
the user perspective for evaluating user experience and stated that, “Digital libraries are designed
for users to use. However, most of the research on evaluation of digital libraries has applied
criteria from researchers themselves.” While this study is performed by a student of library and
information science, it could also be considered a user’s perspective as someone who is a
museum visitor (albeit, infrequent) and not highly practiced in the area of website evaluation,
which matches a novice user profile. While the evaluation criteria is quite specific, the author
will also note general sentiments towards each collection. For the purposes of formal research,
each carefully selected factor will be explored, but it does not require a professional to
understand them. “The best way to evaluate digital libraries is to actually use them” (Xie, 2008).
VIRTUALLY THERE 9
This study could be considered an inspiration for others to simply use websites and take notes. If
specific criteria are selected, it becomes easier to compare strengths and weaknesses. Ultimately,
each element builds up to an overall opinion, but many times usability is subconsciously decided
when users either like or do not like their experience on a certain website.
Usability specific to visual resources differs from a traditional library because of the
nature of the content. Zhang, Liu, Li, and Zhang (2013) referred to a study by Yee et al. in 2003
about faceted metadata in image searching, “Overall the participants preferred more faceted
metadata display, and evaluated it as more informative, flexible, and easier to use.” Faceted
searching would give specific control to the users to pick and choose select metadata from which
to expand or eliminate search results. Another area of control for users applies to the look of the
interface itself. In another study mentioned by Zhang et al. (2013), in 2005, Chen, Magoulas and
Dimakopoulos “suggested a flexible interface (e.g. switching visual cues, offering successive
options) to accommodate the preferences of users with different cognitive styles.” This would
take a lot of planning and sophistication to create such a dynamic website, but it would
ultimately provide a user experience that’s practically guaranteed to satisfy all users.
Menard and Smithglass (2014) also addressed issues in regards to access to digital
images with the intention of creating bilingual interfaces. Unfortunately, there are many studies
about general behavior, but no specific studies on current systems, so this evaluation should
begin to fill in the gaps. They did note that “Museums offering images online structure the
information almost exclusively according to collections and using traditional descriptive
methods. We suggest this is one reason for a high level of consistency between museum websites
in terms of search functionalities” (Menard & Smithglass, 2014). This study will reveal that
VIRTUALLY THERE 10
while there are basic structures in common with the following four museums, there are many
other subtleties that do affect the ultimate user experience of each collection.
Metadata is also facing changes with multiple sources of information in the advent of
social tagging. This allows users to provide their own metadata in addition to the descriptive
metadata provided by museum professionals. In a study focused on folksonomies in art
museums, the authors concluded that a blend of professionally provided metadata with user-
supplied tags created the best atmosphere for navigating through images (Boston-Clay, Mahoui,
& Jaebker, 2013). To understand what the user-supplied information is, Boston-Clay, Mahoui,
and Jaebker (2013) explained that “tagging is commonly referred to as ‘social tagging’ whereby
typical users describe and classify resources in a shared environment.” While metadata is critical
in terms of providing accurate information about works of art, when it is complemented with
keywords and terminology more broadly understood by the average user, the search process is
enhanced. What will be interesting to note is whether or not these highly prestigious institutions
allow users to interact with their works.
Methodology
Four museums were selected for this study—The Art Institute of Chicago (AIC),
Museum of Fine Arts, Boston (MFA), National Gallery of Art (NGA), and Philadelphia Museum
of Art (PMA). After gathering a list of major art museums’ websites from across the country,
candidates were narrowed down through quick browsing and searching sessions. Those with the
most sophisticated websites and unique features were included. Two other museums were almost
considered in the study—Los Angeles County Museum of Art and the Museum of Modern Art—
but due to the extensiveness of the qualitative selections and the realization that the last two
VIRTUALLY THERE 11
featured more contemporary and modern approaches (to both their collections and their
websites), the four finalists seemed like a suitable group to evaluate together.
While these collections are not being evaluated by their ability to replicate a literal virtual
tour through the museums’ halls, it was important to examine how well they could provide a
digital tour of their collection, relating to information access and easy navigation through the
virtual halls. Online visitors are likely researching information about artists, specific works,
styles, origin, media, or time periods. How well each museum could aid them in these quests is
critical, as they house so much of this information in both their staff members found at the
museums and in their extensive databases.
For this evaluation, the author used a combination of quantitative and qualitative
measures. Specific checkpoints gathered from the literature review were established before
touring each museum’s digital collection. To allow for a user’s point of view, which was also
discussed in the literature review, some factors for evaluation developed somewhat organically.
The author began with a basic outline of elements to explore, such as interface, searching and
browsing functionality, metadata, social media, and image quality and availability. As the tours
ensued, new concepts or specific measures became apparent. Depending on the experience of the
collection or consistency of results across the board, new questions were created or rating criteria
was altered to create the optimal comparison tool.
Criteria fell into two categories: subjective and objective. Objective measures included
counting the number of advanced search fields and social media links, while subjective measures
included the author making a judgment call on ease of use of the interface and quality of help
resources. Quantitative results are presented in charts in the Findings section, along with detailed
VIRTUALLY THERE 12
qualitative descriptions of each usability element, while the exact steps from the walk-throughs
are available in the Appendix.
The initial general outline included basic starting points for each usability category.
Overall design and layout of the interface was considered. For search and browsing capabilities,
basic and advanced search options were to be explored, as well as faceted search navigation
through results. The ability to control how results are sorted and the transparency so that users
can understand how results are sorted was also noted. The number and efficiency of search
refinements, and similarly, the availability and presentation of metadata was assessed. The
number, size and accessibility of images for downloading, as well as the number of social media
links was also important. Finally, the presence of help resources was also examined. These basic
points were the author’s initial guide, and as the virtual tours began, they were refined and
updated so that they would create the most value and insight into the evaluation.
The specific criteria are included below and are categorized into their respective
subjective and objective headings.
Objective
Presence of elements were counted, and their significance is explored in Findings.
Browse Are there clearly labeled collections to browse from main page?collections Yes or No
Advanced How many advanced search fields are available?search fields Number
Sort results How many ways can users sort results?Number
Search How many categories are available to refine search results?refinements Number
Metadata How many metadata categories actively link to similar works (i.e. author)?links Number
VIRTUALLY THERE 13
Create a Can users create a profile and curate their own collections?Collection Yes or No
Social media How many social media links are available to share images?Number
Subjective
Items were ranked on a 1 to 5 scale, as follows:1 = Missing (if applicable), 2 = Poor, 3 = Fair, 4 = Good, 5 = Excellent
Interface The overall ease of use, layout, and color scheme
Navigate The ease of moving through a list of results, which is particularly importantresults when browsing through hundreds of works or multiple pages of results
Metadata The number of metadata fields included, their presentation, and any supplemental information to enhance the user’s knowledge of the specific piece of art
Image quality The number of images, including angles if the artwork is three-dimensional,as well as size, clarity, and ease of downloading
Help The presence of help resources, either on the search page or with clear links toa help page, or contact information specific to the use of the digital collection
Findings
This section includes both quantified evaluations presented in table formats, accompanied
by quantitative evaluations below. For a detailed walk-through of each museum, please refer to
Appendix A: Walk With Me (page 30). Each category section provides an in-depth comparison
of each collection’s features. Each museum had strengths and weaknesses, but keep in mind,
however, that this is all purely from the point of view of the author. Xie (2008) reminds us,
“Users are not the same. Their evaluation of digital libraries is affected by their preferences,
experiences and knowledge structure. Different users have different preferences.” What might be
intuitive to one might not be clear to others, and vice versa. It would likely benefit the study to
VIRTUALLY THERE 14
include others who might not be as comfortable with this concept, but is interested in searching
the collections, to get a feel for their preferences with visual resource collections.
Objective Results
Table 4.1Objective Scores*Usability factors
AIC MFA NGA PMA
Browse collections
Yes Yes Yes No
Advanced search fields
9 0 3 8
Sort results 1 4 3 1Search refinements
5 3 7 0
Metadata links 2 3 3 4Create a collection
Yes No Yes Yes
Social media 2 3 4 3Total 21 14 22 17
*To quantify yes/no results, yes = 1, no = 0.
Browse collections. In general, all of the museums had clear paths to browse their featured
collections, so this was not a major difference maker in overall user experience. It did detract
from the Philadelphia Museum of Art, though, as this option was missing. Their advanced search
fields would allow the user to search for a classification or curatorial department from a
dropdown menu, which in essence is similar to browsing a collection, but it might be less
carefully curated than the collections of other museums. It also did not provide a different
experience than the act of searching. MFA, on the other hand, seemed to have the most focus on
browsing collections with carefully created unique presentations of these tours.
Some museums are more focused on specialized collections and the experience of a
virtual tour, where some are more concerned with powerful searches, enabling users to create
their own tour through easy access to their entire collection. How is this decided? It would be
VIRTUALLY THERE 15
interesting to explore whether the museums have chosen to do this based on their own decision
or through the wants and needs of their users.
Advanced search fields. The presence of an advanced search may or may not affect a user’s
ability to browse or search a museum’s collection. There was a wide range of approaches in
terms of providing an advanced search. AIC offered nine options, while MFA only offered a
basic search. Both offered a number of search refinements to enable users to quickly focus their
results as necessary. Browsing could be sufficient for some. Menard and Smithglass (2014)
explained, “Image retrieval for most cultural organisations is initiated with user keywords or an
advanced search screen displaying drop-down menus of predefined categories directly related to
collections, and browsing is presented with equal importance.” Pallas and Economides (2008)
added, “Of course, an advanced search engine that returns accurate and relevant results is
important.” Perhaps the effectiveness of the information retrieval is more important than
specified search fields.
AIC had nine advanced search fields, the most of all the collections. This is an adequate
amount for basic users and likely adequate for more advanced users of the collection. There
wasn’t anything particular outstanding about the options, but there wasn’t anything particularly
bothersome either. MFA did not offer an advanced search. This seemed a bit unusual, but their
focus on browsing specific collections made the author think this was a conscious decision to
encourage discovery rather than highly focused searches. NGA was considered to have three
advanced search fields, as this was the max number of fields that could be searched together.
They offered six other specific fields that could be searched separately, so this was not counted
towards an advanced search. PMA offered eight advanced search fields; however, the way they
formatted the keyword search in relation to the advanced search options made the search process
VIRTUALLY THERE 16
a little unclear. Even to an experienced user, the presence of two separate Search buttons made it
unclear that both keywords and advanced search options could be searched together.
The option of an elaborate advanced search versus a basic search paired with excellent
search refinements might not make a difference to overall usability as long as there are many
clear options for users to browse or search these expansive collections. The author’s experience
was that more options available make for a better user experience, but it’s up to the museum to
decide when and where they want these choices to be available for their users—before and/or
during the search process.
Sort results. Zhang et al. (2013) “suggested a flexible interface (e.g. switching visual cues,
offering successive options) to accommodate the preferences of users with different cognitive
styles.” An easy way to do this is in simple adjustments like sorting results.
AIC offered no customization in terms of sorting results, and the default setting was
unclear, which might be frustrating for users moving through a high number of results. There
was a small window in which the user could jump between pages, and there was no way to jump
to the end or know how many pages there are without dividing the total number of results by 10
(after counting the number of results per page). They also had no customization in terms of how
many results may be viewed per page. MFA had an excellent design for sorting results. In all,
they provided eight options—relevance, title, date, and artist—and each may be sorted ascending
or descending. It was also clear what the default setting was and that the user may control how
her results are sorted. Again, MFA placed emphasis on discovery through browsing rather than
through exact searches. NGA had three ways in which to sort results—default, chronological,
and title. It was unclear what the default setting was, but it was still advantageous to have two
other choices, although users could not control the direction of the sorting. PMA gave no
VIRTUALLY THERE 17
flexibility in terms of sorting results. This is a big disadvantage when sifting through many
pages, although they are the only museum to allow users to change the number of results per
page, which does lighten the load in this regard.
Search refinements. Search refinements are another way to allow users to sift through results
efficiently. Zhang et al. (2013) referred to another study that “demonstrated some advantages of
faceted metadata display over a standard image retrieval/browsing interface.” The author was
interested in how many choices the user had in narrowing search results and the experience when
using them.
AIC displayed their search refinements on the right side of the page, which is a little
unusual, but this allows for consistency of page layout, and they draw your attention by
displaying choices in maroon hyperlinks, so the user can easily understand their purpose. There
were a fair number of choices, but perhaps a refinement around date or period of time would be
helpful, especially for an artist like Picasso who covers a wide range. MFA only offered three
basic search refinements. Again, they seem to want to create a browsing atmosphere rather than
offer a fine-tuned search engine. The options were still clear, though their Classification pop-up
list is a little awkward to use (see Appendix A for a detailed description of this feature). On the
other hand, it was easy to manipulate the search refinements with their search summary
information presented on the top of the page. NGA had the nicest search refinements with the
most categories—seven—and the ability of the user to control which refinements they see by
expanding or collapsing each category in the left navigation. Styles appeared to be an ineffective
search refinement, though, as some searches resulted in all choices that had “No Style defined.”
Perhaps the museum is updating the metadata for this feature. PMA had no search refinements,
VIRTUALLY THERE 18
so while the advanced search options are plentiful, there is a big lack of control once those
results are presented.
Metadata links. Metadata is critical to users to properly identify a piece of art, but it can also
connect users to similar works of art by including links to searches for works by the same artist,
in the same style, medium, time frame, etc. Dynamically linked metadata can add another
beneficial layer of discovery for users.
AIC had three linked metadata fields: artist, format, and gallery (if item is on display).
This seems to be the average number of linked metadata fields among museums, so this must be
one element of consistency among museums as mentioned in the literature review. MFA followd
these same guidelines. NGA provided three linked metadata categories, but they are slightly
different—artist, collection or gallery, and on-view items. PMA was the only museum to offer
four linked metadata pieces—artist, classification, curatorial department, and country of origin.
Unfortunately, the placement of these links on the bottom of the page in a separate section with
the label “Explore the Collections” did not seem as user-friendly as the other collections which
included the hyperlinks in the primary metadata next to the image.
Create a collection. In order to appeal to online visitors, including both educators, students, and
enthusiasts, an interactive feature such as creating a personalized collection is critical. The author
did not fully explore the functionality of each, but rather the option to do so.
AIC, NGA and PMA were all onboard with this notion. Unsurprisingly, MFA, with its
extensive collections already available, did not appear to offer a way to create a unique user
collection. It seems that they want to control the curation, which can be seen as a positive or
negative depending on who is looking to browse or use the collection for educational purposes.
VIRTUALLY THERE 19
Some might appreciate the work the MFA has put into presenting ready-made examples, while
others might want the opportunity to pick and choose their own pieces.
Social media. Social media is another indicator that a museum is committed to contributing to
the online world in which we live and that they want to encourage this virtual interaction with
the user. Through social media networks, the user can connect with the museum or connect the
museum with other users.
AIC offered just two links to share their works on social media—Facebook and Twitter.
They only included these links on individual results pages, whereas the MFA included their three
social media icons—Facebook, Twitter, and Google+—immediately on their Collections home
page, and they are present on every page moving forward except for Collections Search Results.
NGA had four social media links—Twitter, Facebook, Instagram, and Pinterest—but they were
not prominently displayed for easy use. They are part of the page footer, so they do appear on
every page, but many times people do not scroll all the way down, and the author tends to look to
the top of the page for social media links. The light gray icons did not contrast much with the
light page background, so they were not eye-catching. PMA included three social media links—
Twitter, Facebook, and Tumblr—but they were somewhat hidden, so they could also easily be
missed by a user. The icons were located on the bottom of the right sidebar, so users would have
to move around the page to find them. For a museum that places so much emphasis on social
interaction, this seems like an unsocial place to put them.
VIRTUALLY THERE 20
Subjective Results
Table 4.2Subjective Scores*Usability factors
AIC MFA NGA PMA
Interface 4 4 5 3Navigate results 4 4 3 3Metadata 5 5 5 4Image quality 4 4 4 4.5Help 3 1 5 4Total 20 18 22 18.5
*These results reflect a ranking on a scale of 1 to 5.1 = Missing (if applicable), 2 = Poor, 3 = Fair, 4 = Good, 5 = Excellent
Interface. The design of a usable interface is key to creating a good user experience. Simply put,
“The design of its user interface (e.g. menus, toolbars, buttons, icons, frames) should facilitate
the visitor in his exploration” (Pallas & Economides, 2008). Design, layout, labeling, and system
performance were all considered when rating each collection’s interface.
Overall, the AIC interface, receiving a score of 4 out of 5, had soothing neutral tones
consistent with the museum’s branding, and accents of maroon, which help highlight actionable
items, such as search refinements and hyperlinked metadata. Generally, each step moved
quickly, and the author was not frustrated with load time, system errors or confusing labels. Still,
the lack of exciting or unexpected features kept it from receiving a full score. The author did
prefer AIC's use of different color backgrounds in its left navigation to MFA’s which help
designate which options are collections and which are other pages, such as provenance, as
opposed to MFA’s left navigation where each link looks exactly the same. The MFA interface,
also with a score of 4 out of 5, was straightforward and quite easy to use. Their advanced search
options and refinements were less powerful than in other museums, but they did offer dynamic
sorting options. Overall, this was a comfortable experience, though nothing was particularly
VIRTUALLY THERE 21
notable. NGA had the most enjoyable interface to use, so it received a perfect score. Layout and
labels, which are critical to navigating a website, were intuitive and clearly explained. Even the
style of the fonts aided in understanding every piece of information—category headers were in
small caps, hyperlinks were presented in a contrasting blue color, artwork titles were slightly
larger and italicized, and bonus information was often bolded to bring attention. Additionally,
having search results update quickly as refinements were selected without waiting for the entire
page to refresh made the process seamless and enjoyable. PMA offered some powerful features,
but the author did not care for the initial size of the interface, which was quite overwhelming on
the screen. There were also many features that were functional, but were not initially clear how
to use. Labeling was either lacking or unclear, and placement of certain elements made for a less
usable interface. For instance, the carousel of images at the top of the individual artwork page
first seemed like a unique way to move through search results, when it was actually displaying
multiple angles. In the particular example, the angles were of the bottom of the piece, so it was
not obvious that it was the same work rather than a separate piece (see Appendix A for a more
detailed description of this feature). Additionally, the Social Tagging page, while novel, could
still be reworked for a more user-friendly presentation. Their apparent bigger-is-better ideal
seemed to come through in their uncomfortably long list of social tags. PMA received a 3 out of
5 for these issues.
Navigate results. Navigating results overlaps with the subjective measures of sorting results and
search refinements, but takes into account the user’s experience in doing so.
AIC, scoring a 4 out of 5, offered nine pages of results at a time, so it was fairly easy to
move between results, but the author would have liked to have seen choices to sort the results
and to understand how the results were sorted by default. The author also would have liked to
VIRTUALLY THERE 22
jump to the last page of results and worked backwards but was unable to do so. The author did
appreciate the ability to move between results from the individual artwork page, but the labeling
was not initially clear. This feature could have been missed had the author not been examining
every element on the page for discussion. MFA also scored a 4 out of 5 for navigation. As
mentioned previously, users had a lot of control over sorting results, but not as much with
refining them. To navigate between pages was not the easiest, but they did show an improvement
over AIC by being able to jump to the first or last page of results. The user cannot move between
results from the individual artwork page, though. NGA scored only a 3 for navigating results due
to the inability to move between results efficiently. While the control offered to the user through
effective search refinements and several sorting options was highly appreciated, if the user was
left with many pages of results, it was hard to move more than a couple of pages at a time. The
lengthy list of search results also made scanning each page of results a slightly longer task. PMA
was also lacking in its navigation, scoring only a 3. While it was the only collection to offer
multiple options for displaying results per page, there were no search refinements or sorting
options, as well as no explanation of the default sort setting. It was better at jumping between
pages than some of the other collections, though.
Metadata. The existence and presentation of metadata is critical to these collections, as many
visitors are using these items for educational purposes. Its presentation is critical for users to
identify works without having to search all over the page, and extra information, particularly in a
narrative form, would be welcomed by users with a little or a lot of pre-existing knowledge. As it
might not be necessary for all users, though, its presentation is also key—the ideal situation
would be a clear indication that supplemental information is available without it overwhelming
the rest of the page.
VIRTUALLY THERE 23
AIC, with a perfect score, presented critical metadata in a concise format, and in some
cases, provided supplemental descriptive information, which will be greatly appreciated by
educators or researchers. The user does not have to exert a lot of effort to find this information,
which is another bonus. One feature of metadata that the author would have changed would have
been the hyperlink for the author. If there were multiple creators attached to the work, along with
country of origin and dates, the entire list was highlighted, but it was only one link for the main
creator, so the exact linking value of metadata was unclear. MFA also received a perfect score.
Metadata was fully available and presented clearly. Where available, supplemental information
was provided. The way they linked creator names, which highlighted only the specific name
rather than all information surrounding it, was preferable to AIC’s linking. NGA is the third
museum to receive a perfect score for metadata. They packed a lot of information into critical
spots, like the search results page, without sacrificing functionality of the website. The metadata
was also presented clearly, the hyperlinked metadata was easy to understand, and the
supplemental information was plentiful and displayed nicely. They even include a map of the
museum for easy location of the physical work if it is on display. PMA received a 4 out of 5 for
metadata. While they included just as much information as the others, even including
supplemental information, the placement of linked metadata and narrative text could easily be
missed by the user.
Image quality. The main reason that users would visit these digital collections would be to see
the art. Therefore, the number of images and their size, quality, and availability for download are
all critical to the overall usability of each collection.
AIC received a 4 out of 5 on this measure since not all of their images enlarge, and some
sculptural pieces only had one angle to view. They performed well for downloading, as the
VIRTUALLY THERE 24
author had no problem saving the biggest available image, which was fairly large, to her
computer. MFA also received a 4 for image quality. While images were very large when
available and could be downloaded, it would have been nice to have multiple angles for three-
dimensional pieces. It would be preferable not to see an option to zoom when the zoom function
is unavailable. On the results page, it also felt unnecessary to provide an additional image pieces
when hovering over an individual result, as many of the images hardly changed. It created a
busier atmosphere than necessary as the mouse moved around the page. Also confusing was the
ability to download the image by right-clicking, while there was also a link to license the image
with specific instructions and a 2-3 week processing timeline, so the user was a little unclear as
to which option to follow depending on usage. NGA received an almost perfect score for image
quality. Not all images were available in very large sizes or with the zoom function, which
prevented it from receiving a 5. The zoom feature for artwork that did have it was amazing, and
the level of detail was incredible to see up close. For users who want to save an image, NGA’s
lightbox feature did make it easy for the user to responsibly download it, but the extra steps
required to visit another page to do so might be a nuisance for some. PMA received the highest
score for image quality. In the same vein as their oversized interface, they offered oversized
images, which is a good thing. While a user cannot download by right-clicking, the download
icon made it just as seamless a process. This was the only museum to offer multiple angles and
detailed views of three-dimensional pieces that the author could find. Unfortunately, pieces that
did include multiple angles were not available to download, which is why it received a 4.5, as the
author wanted to give the museum credit for going above and beyond in this area.
Help. What happens when a user has a question? Xie (2008) wrote, “Very few users used Help
and other features when they encountered problems in the search process.” She explained that
VIRTUALLY THERE 25
help resources had the biggest range of opinions in terms of usability, which helps explain the
disparity of results for this element. It is the author’s opinion that help features are integral to the
usability of each digital collection. One would hope that users might not need help if the
collections are perfectly designed, but given the range of experience of potential visitors, it is
never a bad thing to include help resources. Perhaps the lack of contact information in many
cases is due to the lack of staff time to assist online users.
AIC received a 3 out of 5. They did offer a few key help pointers below their advanced
search fields, but it was unclear if there were any other resources available on the website, and
they did not include any contact information within their digital collections for additional
assistance. MFA received a 1 for help. There did not appear to be any help resources displayed
or linked while moving through the collections pages. NGA received a perfect score for its help.
It offered quick tips in key areas, like the search page, while also including links to more help
guides. There were a number of topics covered in their FAQ section, but the NGA even offered
contact information for people who needed more. Their website was so easy to use, so users
might not need a lot of help, but the diversity of users could present a range of questions not only
relating to use of the database. Either way, help was provided and further assistance was
available. PMA received a strong score for its help resources, a 4 out of 5. It provided an in-
depth list of search tips in an expandable format, which the author felt was a very user-friendly
presentation. They even included a special help section on accented characters for both PC and
Mac users, which was extremely unique.
VIRTUALLY THERE 26
Overall Usability Results
Table 4.3Overall Usability Scores
AIC MFA NGA PMATotal 41 32 44 35.5
Generally, collections that scored the highest on objective measures also scored the
highest on subjective measures, thus resulting in the highest overall usability scores. That being
said, just because a collection might feature many advanced search fields does not translate to the
collection’s performance as a whole or reflect its visual appeal. This is why it was important to
perform qualitative measures in addition to quantitative ones. According to this study, NGA was
the most usable digital collection, followed closely by AIC, while MFA received the lowest
score for usability. NGA’s interface was extremely user-friendly and intuitive but also included
the most help resources. Though they did not offer an extremely complicated advanced search
interface or efficient navigation through results, they excelled in most other categories. AIC was
another strong contender with a slick interface and high scores all around. MFA took a different
approach to their digital collections by providing curated collections for a user to browse and a
less exacting search experience. While it resulted in a lower score for the purposes of this study,
this type of experience might be perfect for some users. Finally, PMA had some outstanding
features, such as excellent images with multiple angles, well-placed help resources and an
extensive advanced search interface, but the overall design and layout proved more difficult to
use and the oversized scale was overwhelming.
Conclusions
The lack of research surrounding the current state of digital collections for art museums is
slim, but relatively straightforward assessments such as these can easily create a discussion of
best practices and areas for improvement. Roth-Katz (2012) mentioned one big hurdle in
VIRTUALLY THERE 27
determining requirements for an excellent digital collection, “Andrew J. Pekarik [from the Office
of Policy and Analysis at the Smithsonian Institution] argues that museums as a field have not
reached consensus on the purpose of their websites.” Without a purpose, it can be difficult to
assess good functionality. At this point, usability can only be assessed in terms of what we know
about current visitation behavior. These users are likely previous or future visitors to the museum
itself, so it’s important to create a strong overall experience with the institution. It is also
important to meet the needs of those who are unable to visit but still want the reliable and
accurate information provided by the museum. Each museum’s digital collection has many
admirable features and seems to be meeting the needs of many users, but there were many less
than perfect scores, so there is room for improvement and innovation in terms of presenting
information and providing access to it.
Though museums have not always had to share so much information publicly, virtual
access should be encouraged. If people are searching for information and images responsibly, we
should honor this practice, rather than forcing them to resort to all-purpose search engines for
unreliable information and illegally reproduced images. It also benefits the museum by
respecting their assets, responsibly promoting their art and services, and proving their
commitment to the community and to the future that blends the physical world with the virtual
one. For these reasons, it seems as though museums are virtually there in providing dynamic and
informative online visits, and for this, visitors can feel like they’re virtually there, enjoying the
art of the digital collections.
VIRTUALLY THERE 28
References
Boston-Clay, C., Mahoui, M., & Jaebker, K. (2013). Exploring the usability of folksonomies in
the art museum community. Proceedings of the 13th ACM/IEEE-CS Joint Conference on
Digital Libraries (JCDL '13), pp. 387-388. http://doi.acm.org/10.1145/2467696.2467760
Hariri, N., & Norouzi, Y. (2011). Determining evaluation criteria for digital libraries' user
interface: A review. Electronic Library, 29(5), 698-722.
Marty, P. F. (2006). Meeting user needs in the modern museum: Profiles of the new museum
information professional. Library & Information Science Research, 28(1), 128-144.
http://dx.doi.org/10.1016/j.lisr.2005.11.006
Menard E., & Smithglass, M. (2014). Digital image access: An exploration of the best practices
of online resources. Library Hi Tech, 32(1), 98-119. http://dx.doi.org/10.1108/LHT-05-
2013-0064
Pallas, J., & Economides, A. A. (2008). Evaluation of art museums' web sites worldwide.
Information Services & Use, 28(1), 45-57. http://dx.doi.org/10.3233/ISU-2008-0554
Patel, M., White, M., Mourkossis, N., Walczak, K., Wojciechowski, R., & Chmielewski, J.
(2005). Metadata requirements for digital museum environments. International Journal
on Digital Libraries, 5(3), 179-192. http://dx.doi.org/10.1007/s00799-004-0104-x
VIRTUALLY THERE 29
Roth-Katz, E. (2012). Access and availability: A study of use policies on art museum library
websites. Art Documentation: Journal of the Art Libraries Society of North America,
31(1), 123-140.
Saracevic, T. (2000). Digital library evaluation: Toward evolution of concepts. Library Trends,
49 (2), 350-369. Special issue on Evaluation of Digital Libraries.
Skov, M., & Ingwersen, P. (2014). Museum web search behavior of special interest visitors.
Library & Information Science Research, 36, 91-98.
Xie, H. I. (2006). Users' evaluation of digital libraries (DLs): Their uses, their criteria, and their
assessment. Library & Information Science Research, 28(1), 433-452.
http://dx.doi.org/10.1016/j.lisr.2006.06.002
Xie, H. I. (2008). Users’ evaluation of digital libraries (DLs): Their uses, their criteria, and their
assessment. Information Processing & Management, 44 (3), 1346-1373.
Wetterlund, K. (2008). Flipping the field trip: Bringing the art museum to the classroom. Theory
Into Practice, 47(2), 110-117. http://dx.doi.org/10.1080/00405840801992298
Zhang, X., Liu, J., Li, Y., & Zhang, Y. (2009). How usable are operational digital libraries: A
usability evaluation of system interactions. EICS '09 - Proceedings of the ACM SIGCHI
Symposium on Engineering Interactive Computing Systems, 177-186.
VIRTUALLY THERE 30
Appendix
Walk with Me
The Art Institute of Chicago
The Art Institute of Chicago (AIC) Collections page has a strong emphasis on browsing
art by collection rather than searching for specific works, but this is a logical choice given the
title of the section. The color scheme is muted gray, tan and white, and the layout is grid-like,
mirroring the top navigation, which is represented in blocks, in this case, mirroring the
museum’s logo.
You may browse collections in two ways: by selecting a representative image in a 15-
image grid at the top of the page, whose exact topic is revealed upon hovering over the image—
and whose functionality I did not realize until I moved my mouse over them. Or you may select
one of the same options from the left navigation.
There are four blocks underneath the collection images: Collection Updates, Quick
search, Interact with the Collections, and About the Collections. Collection Update brings you to
the latest updates. The search section includes a quick search bar with a link to an Advanced
search option. Interact with the Collections allows you to create your own gallery. About the
Collections features a concise description of the page’s purpose.
The left navigation features quite a few options in which to enter the collections: Explore
Themes, Browse A-Z, Search Online Collection, or Search Interpretive Resources, which are
highlighted in a contrasting taupe to the gray page background. Underneath these options are the
15 collection titles, in the gray background color, seemingly to bring more attention to surround
left navigation options. Below the 15 collections are Conservation, Provenance Research Project,
and Books, which I will not explore in this study.
Explore Themes brings you to a list of eight selected themes, again with a representative
image, but this time shown to the right a short paragraph description about each so you can
informatively choose your path. Browse A-Z refers to artist name, which, to the author, was not
intuitive. There is an alphabet grid to begin exploring, but they also provide their search options
in case you decide to change your approach. Search Online Collections brings you to their
advanced search interface. The following free-text fields are present: Keyword, Artist or Culture,
Title, and Accession Number. There are two dropdowns where you may select one choice:
Collection Category and Type. Finally, you may check all or one of the following three choices:
VIRTUALLY THERE 31
On Display, Has Image, and Has Interpretive Resource. Below the search fields, there are five
bullet points with five search tips.
I performed a simple search and used one of their suggestions from the help section: the
artist Picasso. This brings me to the Search Collection Results page, which includes the quick
search bar at the top, with the option to expand and display the advanced search fields if I wished
to change my search immediately. Underneath, I have the message: Displaying records 1 – 10 of
359.
Search refinements are available on the right side of the page, which is an unusual option,
but they are prominently displayed in maroon-colored hyperlinks with “Refine Search” at the
top, so it is obvious what their purpose is. There are five categories in which I can refine my
results, and each refinement includes a number in parentheses indicating how many results I will
get for each choice: On Display or Not On Display, Show only artworks with: Images or
Interpretive Resource, Artist or Culture (Picasso, Pablo being the only choice), Collection
Category, and Object Type. Collection Category includes three categories, but each has a “[+]”
beside it, a clear indication that there are subcategories available to select. Sub-collections
immediately appear when [+] is clicked. When a refinement is selected, the page refreshes with
updated results. You may remove selections by clicking on the maroon “[x]” next to the
refinement element which remains in the navigation, but which is not hyperlinked anymore for
selection.
Again, results are presented in a grid-like display—five rows of two images that expand
based on the length of the descriptive metadata. A small image is displayed in each grid with a
bit of metadata about each record—artist (country and birth/date dates), title, and date, and at the
bottom, I am presented with the choice to Add to my collection (which you have to create an
account to do). Below the results are Pages 1, 2, 3, up to 9, of my results, which I can select
directly or I can use the right arrow to go to the next page. There is no way to jump to the end of
the results, and at most you may move up or down within nine surrounding pages of results.
There is no option to sort my results, and it is unclear what the default sort setting is. You may
go to an individual result page by clicking on the maroon hyperlinked title or by clicking on the
image.
The individual result page, About This Artwork, includes a slightly larger image than
before, again on the top right of the page’s primary content. The metadata includes artist name
VIRTUALLY THERE 32
with country of origin and birth/death dates (in a maroon hyperlink), title, date, format, extent,
and rights information. Underneath there are maroon hyperlinks to its collection and specific
gallery. Below the metadata is a two-paragraph description and history of the work. Below that,
there is a link to Exhibition, Publication and Ownership Histories. Again, I am presented with the
option to add the work to my collection. And finally, there are two more blocks of related
content: Browse Related (which includes collections and styles of art) and related Interpretive
Resources. The image is clickable and expands so that it fills up my entire computer screen as an
overlay with a block of quick metadata at the top, including artist, artist information, title, date,
and copyright information. I am able to right click and save the JPG file to my computer. Back
on the result page, there are four icons underneath the image, which include a description when
hovered over, though they are clear already: Email to a friend, Print this page, Share this artwork
on Facebook, and Share this artwork on Twitter. Below the icons, there are three more choices in
text: View this enlargement, Add to my collection, and Shop for related gifts. To go back to my
search results, I can either click Browse selections, which is in a separate box above the page
content, or I can browse to the previous or following result by clicking on a left or right arrow
next to this, which is a really nice feature.
Museum of Fine Arts, Boston
The Museum of Fine Arts, Boston (MFA) digital collection appears to be a browsing-
focused database. The color scheme is similar to AIC—in this case, it is a combination of soft
grays with red accents, which help direct the user to actionable navigation throughout the
collections. The Collections page features a list of 11 collections down the middle of the page,
with a small representative image on the left, a collection title in red and an abbreviated
description (indicated by a cut off sentence and ellipsis at the end of each paragraph). The left
navigation features a quick search bar at the top, and links to Featured Galleries and the 11
collections, followed by Provenance, Conservation and Collections Management, Libraries and
Archives, Publication and MFA Images. There is a sidebar on the right highlighting a featured
gallery in the same format as the main collections—one image with the collection title and an
abbreviated description. Users may also opt to view this gallery or More Featured Galleries. On
the top right of the page, round icons for easy sharing of artwork are already displayed for email,
Facebook, Twitter and Google+, so users are immediately aware that images can be shared
socially.
VIRTUALLY THERE 33
When you go straight to a collection, you are presented with a larger version of the image
from the Collections page with a lengthy description of the collection. This time, the right
sidebar highlights a related exhibition. Below the description is a red hyperlink leading you to
provenance research. The next section of content below this includes a manually-controlled
carousel of collection highlights with two featured pieces of art and their titles underneath, which
you may click on to bring you to a new page featuring a much larger version of the image with a
description and metadata (when available), along with the same images from the carousel. Back
on the collection page and below the carousel is a list of Collection Tours links. These will bring
you to the same type of image carousel, this time with the specialized tour created by the MFA.
The design of the collection tour differs from the main pages, which gives the feeling of a
special experience. This time, the content is presented on a black background, which really
emphasizes the art and gives the feeling of stepping into a new gallery with painted walls. The
image is large and can be downloaded by right-clicking on it. When you hover over the image, a
few options pop-up on the bottom, including License this Image and Zoom. Both are in red text,
which indicates that they are active links. License This Image brings you to a separate page with
specific instructions on image reuse and how to request permission. I was unable to zoom on the
particular image used for this study, so it would be preferable that it were not presented as an
option, but I assume it is the default display. Metadata is presented on the right side, including
the title, date, artist (another red hyperlink that brings you to a search for that artist with all
his/her works) and artist information, and description. Below this initial information are several
elements that can be expanded or condensed, as indicated by the up arrows next to the section
headers, Description, Details and Multimedia. Description includes a few paragraphs about the
artist and specific work. Details includes dimensions, medium, classification, accession number,
collection and display status. Similar to the artist feature, this item’s classification (paintings)
and collection (Europe) can be clicked on to bring you to all works with this features. At the
bottom of the Details section is another link for MORE INFO, which brings you to the individual
Artwork page, with a large image and all its metadata listed underneath with the same hyperlinks
as before. This page resembles the standard color palette of grays and red accents.
It does not appear that MFA offers an advanced search option, which is unusual. I
performed a simple keyword search. On the Collections Search results page, the left navigation is
replaced with search refinements. The search bar is again at the top, and it displays my search
VIRTUALLY THERE 34
terms. There are three search refinements under the heading “Refine Your Search By”:
Collection, Classification, and On View. Again, MFA is very collection-focused. Collection
offers a clickable list with the number of results for each choice. You may select one at a time,
and the page refreshes as you do. Classification displays a bar with “Select options”, and when
you click on it, a scrollable list pops up with checkboxes next to each choice allowing you to
select more than one option,. This returns items that feature any of these items, rather than every
option. I understand why they have this as a pop-up as the list is less structured and longer than
the categories, so it would make the left navigation longer than the search results, which is messy
and not user-friendly. The number of hits for each option is not included. You must click submit
underneath to apply Classification refinements. Finally, you may select No or Yes for On View,
and these choices also display number of matches.
The search results are displayed with four sorting options at the top. The default setting,
Relevance, is indicated by its gray text, while Title, Date, and Artist are presented in red text.
There is an arrow next to the selected sort option indicating which direction the results are
sorted, and you may change the direction by clicking on it. This is a nice feature. Below the sort
options, a wide gray box tells you your number of results and repeats your exact search options,
including your provided keywords and any refinements you may have added. You may remove
refinements by clicking on the red x to the left of each choice, so you have a lot of control here to
manipulate your results. Results are presented in a 3 x 3 grid, so you have 9 results per page, and
below them, you may navigate through results in a few ways. Pages 1 through 9 are clickable,
and if you want to move beyond those nine pages, you may go to the last choice and move
forward from there, but it does not bring you to the next set of 9 pages. You may also click
NEXT to go to the next page, or LAST, which brings you to the last page of results, so you can
move somewhat swiftly through your results if you have a lot.
Each result is represented by a medium-sized image, where available (if not, a gray box
with the MFA logo is the default placeholder). Underneath the image is the title in a red
hyperlink, the artist (if applicable), date and accession number. Each element except for the
creator is clickable, although date and accession number are not displayed in red—when hovered
over, they display a red underline. Each brings you to the individual result page. When you hover
over the image, it displays another size image (more zoomed out than the display image, which
VIRTUALLY THERE 35
sometimes is a big change, but some images look basically the same) or angle, if available. The
artwork page is the same as described above.
National Gallery of Art
The National Gallery of Art (NGA) has a very neutral palette of white and gray with navy
accents and a clean but traditional website, which matches the tone of the institution. NGA
seems to strike a balance between offering users the choice to browse or search the collection
upon entering to the Collection home page. The main content of the page sits on the left with a 3
x 3 grid of tall rectangles, each with a title, small image, description, and an actionable link, like
View All or More, in cases where there is a link to view only what is On View (a blue hyperlink
located directly underneath the description). The featured collections on this page are mainly
divided by genre—Paintings, Sculptures, Prints, Drawings, Photographs, Decorative Arts, and
Media Arts—in addition to Highlights, Search the Collection, Artists and Recent Acquisitions.
The right sidebar on the page is what I consider to be the traditional left navigation menu,
including Search the Collection, Recent Acquisitions, Provenance Research, Tours and Guides,
Building Maps, Support Us, National Lending Service, and FAQs. Each section has a small “+”
icon to the right of each header, indicating that you can expand or contract for further
information on the subject matter or for links. This is also indicated by the Search section, which
is already expanded by default and includes a brief preview of the searching parameters available
to users.
Upon visiting a collection, in this case Highlights, you arrive on a new page with a 9 x 5
grid of small square- and circle-shaped images, which I like since it allows you to get a feel for
the collection quickly without having to scroll down the page or through the collection only after
you enter it. There is no text, which really highlights the art. When you hover over an image, a
box pops up with a small, but full-size view of the image, author, title, date, viewing status, and
the collection to which it belongs. At the bottom of the page it says “43 objects arranged
chronologically,” which a nice piece of information to include. On the bottom right, there are
three icons. The first, which has a dark gray background indicating this is the current setting, is a
mini grid, so you can tell you are able to change your view. The other two options appear to be a
list view with small images and descriptions or a slideshow view. This is a powerful tool
presented in a clear and concise manner. Finally, on the page footer, there are four social media
links—Twitter, Facebook, Instagram, and Pinterest.
VIRTUALLY THERE 36
Clicking on an image brings you to a new page about the individual artwork. This
includes a larger image on the left, which is featured within a prominent content area with a
slightly darker background at the top of the page. Once again, key metadata is included for
immediate reference, including the artist, artist information, title, date, format, extensions in both
metric and standard measurements (overall and framed), rights information, accession number
(though it is not labeled), and on-view status. The artist, collection and on-view status (only if
the item is on view) are all blue hyperlinks. Clicking on the artist name, collection or on-view
status brings you to a new search results page, executed by the piece of metadata selected.
Clicking on On View brings you to a new page with a simple map of the gallery, and the room in
which this work of art is displayed is highlighted in blue. The specific room information is
clearly labeled at the top, and the same metadata and small image are displayed again underneath
the map. The map is also clickable by room. Embedded at the bottom of the page is a list of
search results for other works on display in the same room. You may click on any other room,
and the header changes to that room title, and the search results update. All of this is extremely
fast, and it is a wonderful feature for both people who might wish to visit the museum in the
future and people who are reflecting on their previous visit.
Directly below this list are four icons paired with text descriptions—zoom, download,
share, and add to favorites. Zoom brings up an overlay of the image with a black background
where you can zoom in to extreme detail. In the upper left corner of the page, a round icon with
IIP displays information about the image viewer you click on it, and it also displays a question
mark and “click for help” when you hover over it. This information is also displayed if you right-
click on the image. After testing a few different works of art, the zoom doesn’t work with each,
but you can zoom in to extreme detail on some. Since you can’t right-click on this image, the
download option allows you to save the image.
Clicking on download brings you to their lightbox page, as indicated by the page footer,
which reads: “Lightbox: My First Lightbox (0)” with a dropdown that would display each
lightbox if you had created any previously. A search has been executed by accession number to
give you the exact piece of art you were just viewing. It is a small image with artist, title, date
and classification underneath, as well as four icons. When you hover, the image is highlighted in
a red border and basic artwork information appears. Hovering over each icon underneath the
image shows that the options at this point are: Add to front of lightbox, Click to view detailed
VIRTUALLY THERE 37
object information and preview image (if available), Click to download a high-resolution image
suitable for publication use, and Quick Download – Click to download an image suitable for
presentation and screen use (1200 pixels). If you choose to download a high-resolution image,
you are prompted to log in, so this is their way of ensuring people are properly downloading
these images. Clicking on more information opens a small window with a slightly larger image, a
quick reference of appropriate metadata on the left hand side (artist, artist info, title, date,
medium, classification, dimensions, credit, accession number, digitization, and image use)
underneath the following options: Preview (Image Only), Add to front of lightbox, Add to back
of lightbox, Print preview with details, and Download lecture image. Again, they are providing a
lot of options and there are a lot of steps, but I think it enhances the experience as the steps are
intuitive, the labels are clear, and the page functions well. At the bottom of the lightbox pop-up,
you can click on the link to View this object record on nga.gov, which brings you to a search
results page in a new tab, retrieved by accession number again.
Back on the individual record page, share displays a small pop-up to e-mail, Tweet, or
Share (on Facebook). In the second section of the page content, there is a small left navigation
menu in the same darker gray background, including Overview, Explore This Work, Inscription,
Provenance, Exhibition History, Bibliography, Technical Summary, and Related Content. The
default display is Explore This Work, which included a lengthy description. What’s nice is that
all of the major identification for the piece is readily available at the top, but there is extensive
additional information on the page, presented in a very usable manner.
Other specific collections have a slightly different design. For instance, Paintings, which
includes multiple sub-collections based on nationality and timespan, offer a landing page with a
featured work of art at the top. There is a left sidebar (in the same format as the right sidebar on
the main Collections page) with the Paintings section expanded, displaying all sub-categories,
while other media are available to visit underneath (like sculpture and prints), and may be
expanded to visit there instead. There is another section of content down the middle of the page
with a smaller featured image, followed by a lengthy history of the collections. The right sidebar
includes several Related Features (expanded by default), Provenance Research, and FAQs (both
collapsed). The FAQs link brings you to a new page with a diverse array of 20 questions about
artwork and an option to e-mail NGA by clicking on “I still have a question.” Clicking on one of
the Paintings collections brings you to another landing page in the same format with a history of
VIRTUALLY THERE 38
the specific collection. A prominent link at the top of the description allows you to “View all
items in [the collection]”. This simply brings you to a search results page for the search
parameters related to the selected collection.
NGA’s Search the Collection page offers what I will call a semi-advanced search. They
have eight free-text fields in which you may enter your own keywords, but you cannot search
them collectively, as indicated by separate Search buttons next to each field. This is comparable
to an enhanced basic search. NGA includes a couple of suggestions for searches in the right
sidebar—by searching only items that are on view now, or works by category—drawings,
paintings, photographs, prints, and sculptures. Specifically, you may search a combination of
artist name, key words in title, and check yes or no for searching for only objects that have
images available. Below, in a section called “More ways to search,” you may search individually
for key words in object information, credit line, provenance name, accession number, exhibition
history, or catalogue raisonné. Each choice has a short explanation to guide the user in her
selection. In general, these seem to offer much more targeted searches.
As you begin to type in the main search fields, artist name or key words in title, matching
options are presented in a pop-up form, so that you may click on it for immediate fill-in. If there
are more than 10 matches, there is a line at the bottom, More Options…, which brings up the
next 10 matches in the list. As you go further down, “Previous Options…” appears at the top to
return to the previous 10 results. This is an unexpected and powerful feature. You may also opt
to use your own search term by clicking outside of the pop-up to ignore the choices.
Now on the Collection Results page, the content is split into two almost equal sized
columns. The search refinements are clearly labeled on the left sidebar with the header, Filter
results by:. There are seven categories with a variety of refining methods: Works on View,
Online Images, Medium, Nationality, Online Editions, Time Span, and Styles. Most choices are
a checkbox, and each refinement choice includes the number of matching results. Beside each
refinement header is a minus sign that indicates you may collapse (and a plus to expand) each
choice, so you have a lot of control over what you see on the page. Works on view and online
images each have one option, to select only items that are on view or that have images online.
Medium and nationality offer multiple results, and underneath the headers are a blue text feature
to Clear All, so you don’t have to manually delete multiple choices, although you can uncheck
them manually. As you select options, the results update quickly without a whole page refresh, as
VIRTUALLY THERE 39
do the remaining matching search refinement options. Time span is a preset dropdown, which
defaults to Before 1300 until After 2000. There are 18 dates available to create your own time
span on either end. Finally, Styles is interesting. In my current search, I can only refine by one
option: “No Style defined,” but I don’t see any styles. Perhaps this metadata is still in progress.
As each refinement is selected or removed, on the right side of the page above the search results
in a prominent gray box, the search terms are updated, which shows you your search parameters
as Filters:. There is a round X icon next to each, so you can also adjust your search here.
Directly below this, you have the choice to sort works by: default, chronological or title.
Below that, the number of results is displayed, and it is clear that there are 20 per page (e.g. 1-20
of 300 results). Just to the right of this are the pages of results. Only the first two are displayed
with an ellipsis, and you must click on next> to move forward in the results, so it isn’t easy to
jump around between pages. As you advance in the pages, you may go back to the first page or
the pages directly preceding or following your current page (e.g. <prev, 1 … 5 6 7 … next>).
The results page is displayed as a long list, which requires you to scroll quite a bit to see
everything, but you also get to see 20 results at a time, which is an improvement over other
collections. Each result is separated by a light gray dividing line. There is a small image on the
right side, if available, and on the left there is a short list of essential metadata, including the
heading “Work of Art,” artist and artist information (in blue, indicating a hyperlink), title (in a
blue, larger italicized font), date, format, extent, collection (in blue) and accession number, and
its on-view status (only On View is in blue).
Clicking on the artist name brings you to a new page with a search for all works by this
artist. At the top of the page is the artist’s name, and the line below includes nationality and birth
and death dates. Between this and the search results (which have been pushed to a lower place on
the page) is a small horizontal navigation menu, including biography, works of art, and related
content. I realize Works of Art is the default display. When you click on Biography, the page
content updates to a detailed history of the artist. Related Content lists any other features on the
website that mention this particular artist. It is noted that not every artist has all of these features.
Clicking on the title of a piece of art brings you to the individual artwork page, which was
previously described.
Philadelphia Museum of Art
VIRTUALLY THERE 40
My initial impression of the Philadelphia Museum of Art (PMA) interface is that it is
large. Images and text are quite overwhelming, and the immediately displayed advanced search
options are quite extensive. There is no landing page for their digital collections—users must go
directly to Search Collections. There is a large horizontal image across the top of the page, and
users may visit its individual page by clicking on the “i” icon in the bottom right corner of the
image. PMA emphasizes the notion of being big in the first sentence of the search page, by
stating, “94,855 objects from the Museum’s collection of over 227,000 are available in the online
collections database.” They want to make it clear just how many items are in their digital
collection, which is a lot. Interestingly, this is the first outright statement of the size of the digital
collection and the entire collection of the four museums reviewed.
The PMA website features a white background with gray and black text and light blue
hyperlinks. This allows the artwork to pop, much like the other collections we’ve seen. There are
two layers to the search page, as indicated by the tab-shaped outline around the content header
“Search”, with a second page devoted to “Social Tagging”, another unique feature to the four
collections. Again, PMA is forthcoming with information to its users, as it includes links to
search help and examples in the introductory text to aid users in their search requests. Clicking
on “search help” updates the content in the main section of the page by expanding a hidden
section below the search bar with search tips and examples. After scrolling down the page, the
advanced search options have just been moved down rather than replaced by the search tips. I
discovered that clicking on Search Help again, in either the introductory paragraph or the
hyperlink below the search bar, collapses the information, if you choose not to have this much on
the page at once.
There is a right sidebar with more navigational options. At the top is a large museum
logo, which brings you back to the homepage, and a mini navigation menu with five pages. The
first, Search Collections, is highlighted in gray, indicating “you are here,” followed by My
Gallery, New Acquisitions, Audio Tours, and ART 24/7. New Acquisitions provides narrative
features about recent news from the museum. Audio Tours includes lists of all the available self-
guided audio tours, which require users to subscribe to the museum’s free podcasts on iTunes.
ART 24/7 explains the process of creating this digital collection, which provides this 24/7 access
to their works. They give a background and encourage people to contribute to the extensive
project. The next section includes links to Support ART 24/7 and Guides to the Museum & its
VIRTUALLY THERE 41
Collections. Then there is a prominent button to Purchase Admission Online. Finally, there is a
clickable image of baked goods with “Dining” overlaid on top. This is the first time I’ve seen a
collection with an emphasis on converting virtual visitors to actual visitors. PMA also
encourages social interaction through their user tagging feature.
Before we move on to the search experience, let’s explore the social tags. First, the
museum supplies a brief overview of social tagging. Below this explanation, they offer the
option to browse by selecting any one of the long list of user tags displayed below. The larger the
word, the more often the tag was used. The list, which is a large run-on sentence of words
separated by more than a traditional space so that users may identify breaks between choices.
The tags also change color as you hover—from light blue to a brighter blue—so you can also
check for words versus strings of words this way. It took me 13 scrolls of my mouse to reach the
bottom of the page, so it is an extensive list. It even includes a number of Chinese symbols.
While the idea is great, and it’s nice to see how much user tagging has happened, perhaps a
better organized presentation of words would be easier to scan. It might be up to the user to
search the page for a social tag in mind.
Back on the Search page, there is a basic search bar at the top of the Search section,
labeled with “Keywords”. There are Search and Reset buttons after the search bar and links to
Search Help and Typing Accented Characters below, which is presented in the same way that
search help is—the section expands directly below the search bar. There are seven advanced
search options which appear to a separate searched from just keywords, as indicated by the
separate Search and Reset buttons below the search fields, but after testing, the user may enter
both a keyword and advanced search terms and search for them all together by using either
search button. The choices are split into two types of refinements: Artist/Maker, Classification,
Country of Work, and Curatorial, in which you select one choice from a dropdown menu, and
Provenance, Audio, and On View, in which you may select only items that have each option.
Artist/Maker is unique in that it appears to offer two search options, but really the first choice
(Artist/Maker beginning with: A, B, C, etc.) controls the dropdown display for Artist/Maker. The
user must select the appropriate letter to view resulting artists, but selecting a letter alone without
selecting from the dropdown does not provide results from all artists matching that letter. Artists
are displayed in last name, first name when applicable; however, commas are missing, so it can
be confusing to read the list.
VIRTUALLY THERE 42
At the top of the search results page, the number of results are displayed. On the next
line, exact search terms are repeated with “Phrase:” preceding them. The text that follows
encourages users to “Create, organize, and share your own galleries with My Museum.” My
Museum is a blue hyperlink. The next line of text tells me that, in this case, I am “Viewing [1-18
of 83], and I have the option to update my results per page via a dropdown from the default 18 to
36 or 54. In between these options and above the search results, I see that there are 5 pages of
results, and I can go directly to any one of them. If there are more than 10 pages, the first 10 are
listed, followed by an ellipses and the last page. There is an option to jump to the next 10 pages.
This is the only way to jump forward. Each set of 10 pages is then displayed, no matter whether
you are at the beginning of the 10 or the end. Again, this is different than other collections. As
you move to the middle of the pages, both the first and last pages are displayed, with options to
move back to the previous 10 pages or the next 10 pages. This is not perfect, but much preferable
to page navigation in other collections. The results are displayed as medium-sized images in a
3x6 grid (since I have left it at 18 results per page). Each image has a light gray border that
changes to bright blue as the image is hovered over, so I know it is clickable. Underneath each
image, the title and artist (or closest information relating to artist if unknown, such as country of
origin) are listed. The title is also in blue, so I can click either the image or the title to go to the
individual work page. If the title includes the keyword that was searched for, it is in bold. This is
a nice feature. Unfortunately, though, there are no search refinements at this point to narrow the
results, nor any way to sort results or understand what the default sort option is.
On the individual work of art page, there is a carousel of four thumbnail images
displaying multiple images of the work. The author thought this was a search result browsing
feature, but upon further inspection, realized that these were simply detailed images of other
angles of the work. Like the results page, each image’s border changes to blue when hovered
over, so the user may go directly to one of the images. Arrows on the left and right sides of the
images allow the user to go to the previous or following set of images if there are more than four.
It is a permanently revolving carousel, so the user will not get stuck on the first or last image.
Underneath, there is a very large image on the left side of the page. There are two icons in the
upper corners. On the left corner are four arrows pointing to each corner, indicating that I can
expand the image even more. This expands the image so that it takes up the entire width of the
main content on the page (the right sidebar remains the same as the initial collection search
VIRTUALLY THERE 43
page), and the only metadata remaining is the title (and alternates, if applicable) underneath the
image. The icon then inverts the arrows to indicate that it will collapse if I click on it again. The
icon in the top right corner is an arrow pointing down to an open box, which indicates the
download image (and says “Download this image” when hovered over). Though I could right-
click on both size images, the download resulted in an empty space .gif file. The downloaded
image, through the download icon, appears to be the same size as the initial image on the
individual result page (a large, rather than extra-large image).
The metadata along the right side includes title, artist and artist information, which is
separated by a light gray horizontal line from the following labeled categories: Geography, Date,
Medium, Dimensions, Curatorial Department, Object Location, Accession Number, and Credit
Line. Underneath the image are two separate boxes of content. On the left side is a Social Tags
box with a light blue background and a “[?]” linked icon, which brings you to the Social Tagging
page. The author would have preferred a brief explanation in a pop-up with a link to the page
rather than immediately going to a new page when it was unclear that this would happen. This
area displays any social tags, or includes a message that there are not any tags associated with the
object. At the bottom of the section, when “[Add Your Own Tags]” is clicked, a message
displays underneath about adding tags appears, along with a free-text field and a button to Save
Tags. There does not appear to be any requirements for setting up an account to do this. I wonder
what kind of monitoring the museum does for this freely available feature. The second area of
content is titled “Explore the Collections.” There is a small box that says 1 love it, and an icon of
a heart underneath, which implies that I could favorite this piece of art by clicking on the heart,
and assumingly, this would update the top to say 2 love it. Underneath is a list of three links:
This Artist/Maker, Classification, Curatorial Department, and Country of Origin with a map of
the world and a small circle indicating this work of art’s origin (and which displays the name of
the country when hovered over). Each links to a search related to the specific link, much like
linked metadata exists in other collections, but provides it in separate links, which could be
easily missed by users. Another piece that could be missed by users is a paragraph of additional
information, which is the narrative supplemental information also included in other collections.
The author almost missed it as it was placed all the way at the bottom of the page.
Users may go back to search results by using the back button, or they have a few options
for navigating results at the top of the page. In the top right corner of the page content, “Viewing
VIRTUALLY THERE 44
[1 of 48]” appears in small gray text. To the right, there are two small icons. The first is a small
3x2 grid of bright blue boxes. There is no information when hovering, but it brings you back to
the grid of search results, which makes sense only after the fact. To the right, there is a small
blue arrow pointing to the right, which brings the user to the next search result. As one moves
forward in the search, a left arrow appears to allow the user to move backwards or forwards.
Recommended