web.hku.hkweb.hku.hk/.../Du-in-press-Academic-social-bookmarking.docx · Web viewRedden (2010), Sun, Suryanto and Liu (2007) and Seki, Qin and Uehara (2010) focused on potential applications

Cited As: Du, H., Chu, S.K.W., Gorman, G.E., & Siu, F.L.C. (in press). Academic social bookmarking: An empirical analysis of the Connotea users. Library & Information Science Research.

Academic social bookmarking: An empirical analysis of Connotea Users

1. Introduction

When people explore the Internet, one of the challenges they face is remembering and retrieving items that they have previously found to be useful. A common approach to managing information found on the Web is the use of personal bookmarks stored on a user’s own computer (Lund, Hammond, Flack & Hannay, 2005; Millen, Feinberg & Kerr, 2005]. Recent research suggests that social bookmarking (SB) offers a more effective means of resolving the difficulty associated with web information discovery and management (Hotho, Jaschke, Schmitz & Stumme, 2006). SB refers to the common indexing of objects and resources, which enables online users to classify useful websites and create shared bookmarks of online resources (Storey, Cheng, Bull & Rigby, 2006), as well as discover resources from other people’s collections (Yanbe, Jatowt, Nakamura & Tanaka, 2007). By allowing users to categorize websites with the use of freely assigned (or controlled) keywords (“tags”), users are able to find and browse their own bookmarks easily (Godwin-Jones, 2006). User-generated tags may further be shared among all users in an SB website, thereby increasing socialization within the community (Goh, Chua, Lee & Razikin, 2009). These tags are also known as “social tags” or “folksonomies”. The term folksonomy comes from the words folk and taxonomy, and is used to describe the vocabulary used for creating and managing social tags (Dye, 2006). Social bookmarks may serve as individual reminders, shared lists, or collective resource libraries (Panke & Gaiser, 2009). With the increasing adoption of SB tools and services, it becomes important to understand users’ opinions and their perspectives regarding the use of SB.

2. Problem statement

Some SB sites, such as Connotea (www.Connotea.org) and CiteULike (www.citeulike.org), have been targeted specifically to academic audiences (Gordon-Murnane, 2006). Different user groups may have different bookmarking preferences and interaction patterns (Lin & Tsai, 2011; Niu & Hemminger, 2011), and so it is possible that academic SB users may have some bookmarking behaviors which distinguish them from general users. For instance, exploratory searches, characterized by less well defined inquiries that emphasize analytical strategies, are often employed by academic researchers (Marchionini, 2006). The popularity of SB services is associated with their potential for improving online information search, exploratory as well as fact-finding (Yanbe et al., 2007), and so one would expect academic users to be attracted by the exploratory search capabilities of SB systems. However, a recent study indicates that the numbers of visitors to connotea.org shows a decreasing trend. (Compete 2010). Such a decline in popularity highlights the urgency in understanding academic users’ bookmarking behaviors and their perceptions of bookmarking, so that SB tools can be designed to be more useful for academic audiences.

Prior studies of SB have mainly looked at tag growth and entropy (Golder & Huberman, 2006; Kittur et al., 2007), tag choice (Sen et al., 2006), tag classification (Al-Khalifa & Davis, 2007), and discussions of the potential academic use of SB tools (Gordon-Murnane, 2006). While substantial research has explored the patterns and organization of tagging and bookmarking behavior of SB users, empirical evidence about how SB usage is affected by different types of users is yet to be explored. Few studies have tried to understand the motivations or incentives behind the behavior of user groups (e.g., Arakji, Benbunan-Fich & Koufaris, 2009), and even less is known about those whose purposes are of an academic or scientific nature. To address this apparent gap in knowledge, the current study collected academic users’ opinions and perspectives on the use of Connotea. The findings are expected to contribute to the understanding and further development of SB as a tool for information management in academic communities.

3. Literature review

SB has gained increasing popularity possibly due to two distinguishing characteristics which enhance its social utility. The first characteristic is the use of tags created by users to organize bookmarks using labels that are meaningful to them (Millen, Yang, Whittaker & Feinberg, 2007). In contrast with the hierarchical and static nature of traditional taxonomies and classification systems, the organizational structure of the tagging used in SB services is emergent and dynamic. The other important characteristic is that bookmark collections typically become public information visible to other people, who may use them for their own purposes (Rader & Wash, 2008; Riddle, 2005). People can share their discoveries and retrieve related information from others who have similar interests (Mejias, 2005). It has been suggested that SB may be a potentially low-cost application for group information management (Grudin, 2006). SB systems provide support for search activities that range from simple “fact-finding” to more exploratory forms of search (Millen et al., 2007). One of the attractions of social tagging systems is that users require no specific skills to obtain the benefits of information organization (Jaschke, Hothos, Schmitz, Canter & Stumme, 2008).

Research issues relating to SB can be categorized into five main topics as shown in Table 1. As SB systems allow users to use their own keywords for bookmark tagging, many researchers have been interested in examining tagging patterns. Tag growth and dynamics, tag choice and tag semantics are frequent foci for SB research. Golder and Huberman (2006) identified regularities in users’ frequency of tagging, their selection of tags and also their activities in SB websites. Factors affecting users’ choice of particular words for tagging were also explored. Similarly, Sen et al. (2006) observed that pre-existing tags affected the kinds of tags that were subsequently created by users. However, this observation was countered by Kipp (2006), who observed that tag selection was not a direct imitation of pre-existing tags, but rather an individual idiosyncratic process.

[Insert Table 1 about here]

Scholars such as Al-Khalifa and Davis (2007) have attempted to classify tags into meaningful semantics. Cattuto, Benz, Hotho and Stumme (2008) suggested relationships between user-created tags and introduced a method to measure such relationships. Sen et al., (2006) investigated the classification of tags, providing insights on tag choices. Redden (2010), Sun, Suryanto and Liu (2007) and Seki, Qin and Uehara (2010) focused on potential applications and future developments of SB tools, with the aim of improving current SB systems and developing more advanced ones with new features and broader applications. Based on participants’ bookmarking and peer interaction patterns in an SB system called “WeShare”, Lin and Tsai (2011) defined four types of user groups: lurkers (seldom performed collective information searching), actives (made more effort to manage their own bookmarks than to review the bookmarks created by others), quoters (made more effort to review others’ bookmarks than to create their own ones), and critics (focused on commenting the bookmarks shared from others).

Research which looks at user perspectives and opinions directly is distinctly sparse when compared with indirect studies of user behavior as reflected in SB activity. This study is a contribution to that underexplored arena.

4. Research methods

4.1. Connotea

SB sites that have gained significant popularity include StumbleUpon, Delicious (formerly del.icio.us), Blinklist, CiteULike, and Connotea. A comparison of the number of visits among five popular academic SB systems (Compete, 2011) indicated that Connotea was the most widely-used in the year 2010 (see Figure 1). Connotea is designed to enable scholars to manage citations and scholarly articles (Rethlefsen, 2008). As with most SB systems, Connotea allows users to tag items, and a user's collection can be made public and shared among colleagues and workgroups across the world (Hammond et al., 2005; Rethlefsen, 2008). Connotea incorporates special functionalities targeted to managing references, such as the automatic collection of metadata.

[Insert Figure 1 about here]

4.2. Participants

To examine perceptions of academic users with a range of familiarity with SB systems, three groups of participants were recruited: experienced users, users with a moderate level of experience, and novice users. Experienced users (Group I) were those who: (a) had created at least 100 bookmarks on the Connotea website, and (b) had made their emails public (e.g., through their bookmarks); thus allowing a means of contact for recruitment. Group I users were recruited from 101 potential participants that were identified on the Connotea website. Thirty-nine responses were collected, yielding a response rate of 38.6%. Moderately experienced users (Group II) included information management students in the University of Hong Kong (HKU) who had experiences using a SB system for group projects prior to their participation in this study. Out of 74 potential Group II participants, 48 responses were collected (65% response rate). Novice users (Group III) included part-time or full-time research assistants in HKU who had no previous experience of using any SB applications prior to participating in the research. Forty potential participants belonged to the last group were invited to register for an account on the Connotea website and use the SB tool for two months. After this period, participants were invited to respond to the online survey, and 32 responses were collected (80% response rate). All potential participants were invited by email to respond to an online survey, and informed consent was obtained from each who agreed to join the study. The research procedures were reviewed and approved by the Ethics Review Committee of the university. A total of 119 users, representing the three user groups, participated. Table 2 illustrates the academic background of all the participants in the study.


4.3. Online survey

The online survey consisted of both close-ended and open-ended questions that investigated users’ bookmarking and tagging behaviors (Appendix A). These questions focused on (1) background of the participants and their habits in using SB systems, particularly Connotea; (2) their ratings of the utilities and features of Connotea; and (3) their opinions on the effectiveness of Connotea. Bookmarking frequency was measured using a 6-point ordinal rating scale with the following values: 1 (never), 2 (once a month or less), 3 (once every two weeks), 4 (once to twice a week), 5 (three to six times a week), and 6 (once daily or more). Participants’ perceptions of the utilities and policies of the SB system were examined using a 4-point Likert-type scale, where 1 referred to “strongly disagree” and 4 denoted “strongly agree”. A balanced 4-point Likert scale which eliminates a neutral mid-point has been shown to minimize social desirability bias and uncertainty in participants’ responses (Garland, 1991). Ratings of 1-2 were interpreted as negative responses and ratings of 3-4 as positive responses. Particular attention was directed towards the users’ perceptions of using the SB website for information discovery and management.

Open-ended questions were also provided so that participants could further elaborate on their scale-type answers when needed. As a result, participants #25 (in Group I), #41 (in Group II), and #28 (in Group III), also provided further written comments in the survey. For those participants who had indicated their willingness to be interviewed, follow-up interviews were arranged to collect in-depth opinions concerning their experiences with using the SB services. Two participants from Group I, one from Group II, and three from Group III were interviewed in the follow-up investigation. Appendix B lists a summary of the open-ended answers and follow-up interview responses by academic participants in the three different groups.

4.4. Handling of “don’t’ know” option

The survey also provided a don’t know option for each question. A number of participants answered with this value for some questions, largely due to their unfamiliarity with the SB features (Figure 2).


These data points were excluded in the analysis. This treatment les to differences in sample size, and may have varying effects on the results. The percentage of don’t know answers in each question was calculated, and has been marked in Tables 4-10 in four different shades of gray for ease of reading. The darker the color in the results tables the higher the percentage of data points being removed from analysis because of the don’t know response and the greater the possible impact on the predictability of the results. Fortunately, the majority of the results in Tables 8, 9, and 10 have no shading. The greatest impacts appear in Table 4 (results for group I), Table 5 (results for the question “It is easy to use” for Group II and III), Table 6 (the result for the question “Using the search box with the choice Find Exact URL” for Group I), and Table 7 (the result of the question “It is useful for managing information in a group” for Group I).

4.5. Analysis

Responses to the online survey were collected and analyzed using non-parametric tests. Descriptive statistics were used to measure central tendency and inferential statistics used the Kruskal-Wallis test to examine the differences in the responses of the three participant groups. Statistical significance was set at p < .05. Statistical analyses were done using SPSS 18.0. Responses to open-ended questions were summarized qualitatively.

5. Results

5.1. Users’ perceptions of bookmarking and tagging behaviors

Table 3 shows five kinds of bookmarking behaviors of the three groups of academic users. Creating bookmarks received the highest rating on average by all groups (Group I = 3.26, Group II = 2.63, and Group III = 2.53), while “copying bookmarks from other users” got the lowest mean rating (Group I = 1.68, Group II = 1.96, and Group III = 1.63). This result suggests that academic users in general are more likely to create their own bookmarks but less likely to use bookmarks created by others. In addition, experienced users (Group I) were found to have significantly higher frequency (once every two weeks) of creating bookmarks than the other two groups (once a month or less).


Experienced users (Group I) created more tags per bookmark comparing to the other two groups (Figure 3). Over 20% of group I users created 4 or more tags per bookmark, but only a small portion (2.1%) of moderately experienced users (Group II) and no novice users (Group III) did so. A majority of Group I (65.5%) and Group II (59.6%) users created 2-3 tags per bookmark. However, Group III users tended to create much fewer tags: 63.3% of them created just 1 tag per bookmark and 36.7% of them created 2-3 tags per bookmark. Connotea policies require users to create at least one tag per bookmark, and it appears that most novice users comply with this minimum requirement without going further.


Academic users’ ratings (from 1 = strongly disagree to 4 = strongly agree') on the usefulness of tags and bookmarks are summarized in Table 4. Participants had generally positive perception about the seven utilities of tags/bookmarks, in which “use tags to find your own bookmarks” received the highest ratings of usefulness by all groups: Group I (M = 3.78), Group II (M = 3.17) and Group III (M = 3.07). In addition, experienced users (Group I) were found to give a significantly higher rating on this utility than the other two groups. “Form a group with friends or colleagues for sharing bookmarks” was rated the least useful by both Group I (M = 2.93) and Group II (M = 2.98) users. Novice users (Group III) rated this utility the second least useful (M = 2.80), and rated “using tags in sharing bookmarks with friends or colleagues” the least useful utility (M = 2.73). This finding suggests that academic users commonly do not share bookmarks with their friends or colleagues.


5.2. Information discovery

All groups of users perceived the automatic collection of bibliographic information to be useful and easy to use. However, some users commented on a limitation of the automatic collection of bibliographic information, noting that the SB website was sometimes inconsistent with the corresponding reference of an article, thus requiring additional time to correct the information manually (Tables 5 and 6).

[Insert Tables 5 and 6 about here]

Generally speaking, positive ratings of usefulness were given by the three groups of users on all eight types of the search functions. This indicates that participants perceived the various search functions useful in supporting information discovery. Significant difference among the three user groups was found only in their views regarding the usefulness of the “related users” function. While moderately experienced users regarded this function the least useful and novice users thought of it the second least useful, experienced users considered it the third most useful of the eight search functions. All three groups rated the function “using the search box with the choice My Library” the most useful search functions. This was consistent with the earlier finding that participants preferred using self-created bookmarks over copying bookmarks created by other users. Compared to the other types of search functions, finding information “through other users who have similar interests” also received relatively high ratings by both Group II and Group III users. Some users pointed out that this function could save time screening relevant information and bookmarks. Both Group I and Group III users rated the function “using the search box with the choice Find exact URL”, the least useful among the eight search functions, while Group II users rated “using Related users” the least useful.

5.3. Information management

The use of social bookmaking for personal and group information management was also examined (Table 7). It appears that all user groups found the SB website easy to use and useful in both personal and group information management. While no significant differences were found among the three groups, novice academic users (Group III) seemed to give higher ratings to the usefulness of SB for personal information management than for group information management.


5.4. Other SB Features

Other SB features were found to be useful (Table 8). “Accessing references and bookmarks from any computer” was reported to be useful by the largest percentage of respondents, while “exporting references and bookmarks to Endnote or other desktop reference managers” was reported to be useful by the smallest percentage of participants. In addition, a majority of experienced users (Group I) found the automatic recognition and filling of references to be useful (68%). In contrast, fewer Group II (27.7%) and Group III (30%) users found this feature to be useful.


5.5. Overall perception

Table 9 shows the users’ overall perceptions of using SB websites compared with the traditional bookmarking methods on a dedicated computer. Significant differences were found among the three groups’ of users’ perceptions on using SB websites to save bookmarks instead of using traditional bookmarking methods. In particular, Group I users were found to have the highest ratings on this point (M = 3.81), while Group II (M = 3.02), and Group III (M = 2.76) users on average gave lower ratings. It was also reported that a majority (61.5%) of experienced users (Group I) eventually built a habit of managing useful websites through Connotea and other SB sites, while most Group II and Group III users still preferred using traditional bookmarking methods (see Table 10).

[Insert Tables 9 and 10 about here]

6. Discussion

The overall findings of the study indicate that academic users generally have positive perceptions of the usefulness and ease of using SB tools for information search and management. Similarly, many functions and bookmarking utilities offered by Connotea were rated positively. However, reported bookmarking use frequency appears to be low among the participants. The highest reported use frequency was “once every 2 weeks”, which was reported by experienced users mainly for creating bookmarks. On the other extreme, novice users reported copying bookmarks from others “once a month or less”. This indicates that while academic users believe that SB tools are quite useful and easy to use, their actual use of such services has not been so pervasive. The participants’ open-ended feedback suggests that while they generally have positive perceptions of SB, the perceived benefits do not outweigh the perceived costs. Group II participant #8 pointed out that users need to spend extra time and effort to become familiar with SB features. This could be the reason why many users were unwilling to switch to SB, which is a relatively new technology. Furthermore, the responses from the focus group study suggest that difficulties in accessing bookmarks may be one of the reasons for low bookmarking and tagging frequencies. Group I participant #30, for instance, mentioned that it was rather inconvenient, as the SB website requires you to log in to access saved bookmarks. A novice user (Group III participant #8) mistakenly perceived that when using SB tools, bookmarks can be saved only from the user’s own computer if the appropriate button is installed on the toolbar.

Comparison across the three different user groups suggests that level of experience in using SB tools is related the use frequency. Responses about methods for managing useful websites suggested that more experienced academic users were more likely to prefer to use SB tools (including Connotea) than traditional methods (such as saving bookmarks on their local computers). Also, experienced users’ perceptions of the advantage of using tools like Connotea over traditional methods were found to be significantly higher than the other two groups. Compared to the experienced users, more intermediate and novice users bookmarked websites on their dedicated computers. These findings highlight the impact of exposure over time. Inexperienced users may not yet feel comfortable using SB tools and require more exposure to Connotea (or other similar SB tools) in order to gain familiarity. The focus group responses also show that less experienced users tended to return to traditional ways of managing websites and searching for information for personal information management or information search purposes, as they were more familiar with the functions and layouts (e.g., Group II participants #12 and #34).

One of the advantages of SB as compared with the traditional ways of saving bookmarks is accessibility from any computer. This may be one of the main reasons why people use SB services, and was reported useful by nearly 80% of participants.

Information discovery does not seem to be the main reason for participants to use Connotea. Higher rating goes to the use of Connotea for personal information management than for group purposes, particularly among novice academic users. The use of tags for respondents to find their own bookmarks and the use of search box with the choice “My Library” both received the highest ratings among a list of SB features and methods. The reported behavior of the participants was consistent with their preferences in personal information management in the survey. All three groups of academic users have higher frequencies in creating bookmarks and in using their self-created bookmarks than any other bookmarking behavior. According to Lin and Tsai’s classification (2011), these results suggest that most academic users are “active” users, who spend more effort on creating and managing their own bookmarks than using the bookmarks created by others. Similar preference for using personal collections of articles was also found in academic scientists (Niu & Hemminger, 2011). Such behavior may be explained by the nature of their work, which usually emphasizes originality and novelty. However, even in the present study, participants reported much lower frequencies of sharing bookmarks with their collaborators. This is contrary to the collaborative nature of SB, in which “sharing” or “social” capability has been described as a key feature (Golder & Huberman, 2006). The results also show that the usefulness of forming a group for sharing bookmarks on Connotea received the lowest ratings among all other common SB behaviors.

According to Markus (1987) and Arakji et al. (2009), users with varied backgrounds and interests can bring more resources to the community to help ensure that there are enough resources available. However, Connotea was conceived particularly for researchers and scientists to share bookmarks and bibliographic information (Redden, 2010). This homogeneous user group may lead to the accumulation of resources which are limited in variety and scope, causing low frequencies in users’ reported bookmarking habits and sharing behavior. Some participants (e.g., Group II participant #22 and Group III participant #31) thought that SB could not gain popularity as quickly as, for instance, Facebook and Twitter because SB websites like Connotea cannot provide a real time connection to those who are not members of the community. Hence, even if users are eager to share information and findings with colleagues, their colleagues also need to users of the same system in order to do more than simply view. Several other participants (Group II participants #12, #22, #34, and Group III participant #31) had the same opinion – not many of their friends or colleagues used SB.

6.1. Limitations

This is an exploratory study. The survey questionnaire was developed to investigate academic users’ general perceptions of SB systems, and their actual bookmarking behaviors. Participants were recruited from Connotea, an academically-oriented SB website, and so results may not all be generalizable to the SB systems, and especially not to non-academic SB systems. Future studies could consider recruiting participants from other academic SB sites or even to confirm the findings. Comparisons to similar research in non-academically oriented SB sites would be interesting. This study only analyzed user perceptions by groups and was mainly concerned about what questions. It might be interesting in future studies to also consider how users first became aware of the SB sites and why they have chosen to use certain sites.

7. Conclusion

Since an SB website requires users’ activities and contributions to maintain its sustainability (Butler, 2001), it is important to understand users’ bookmarking behaviors and their perceptions about the service. The results of this study contribute to further understanding of academic users’ SB behaviors and their opinions, which has not yet been fully understood. The recent decline in the number of visitors on some academic SB websites may be an indicator that SB for academic purposes may not have been a very successful application of the technology. For a technology to become popular, it requires not only a positive user perception of its usefulness, but also of its ease and convenience in use. This empirical investigation may help to improve the design, development, and use of academic SB applications. The benefits of SB systems (flexible personal information management, group collaboration, and enhanced information discovery) make it well worth the effort.

Acknowledgment

· This research is supported by the Science and Technology Planning Project of Guangdong Province (No. 2010A032000002).

· The authors would like to thank K.Y. Yu for her assistance in data analysis.

References

Al-Khalifa, H.S., & Davis, H.C. (2007). Towards better understanding of folksonomic patterns. In Proceedings of the 18th Conference on Hypertext and Hypermedia (pp. 163-166). New York, NY: ACM.

Arakji, R., Benbunan-Fich, R., & Koufaris, M. (2009). Exploring contributions of public resources in social bookmarking systems. Decision Support Systems, 47(3), 245-253.

Butler, B.S. (2001). Membership size, communication activity, and sustainability: A resource-based model of online social structures. Information Systems Research, 12(4), 346-362.

Cattuto, C., Benz, D. Hotho, A., & Stumme, G. (2008). Semantic grounding of tag relatedness in social bookmarking systems. Lecture Notes in Computer Science, 5318, 615-631.

Compete (2011). About Compete.com. Retrieved March 25, 2011 from http://www.compete.com/about/

Compete (2010). Connotea.org. Retrieved August 10, 2010 from http://siteanalytics.compete.com/connotea.org/

Dye, J. (2006). Folksonomy: A game of high-tech (and high-stakes) tag. E-Content, 29(3), 38-43.

Farooq, U., Kannampallil, T.G., Song, Y., Ganoe, C.H., Carroll, J.M., & Giles, L. (2007). Evaluating tagging behavior in social bookmarking systems: Metrics and design heuristics. In Proceedings of the 2007 International ACM Conference on Supporting Group Work (pp. 351-360). New York, NY: ACM.

Garland, R. (1991). The mid-point on a Likert rating scale: Is it desirable? Marketing Bulletin Australia: Australian Department of Education, Science and Training, 2, 66-70.

Godwin-Jones, R. (2006). Tag clouds in the blogosphere: Electronic literacy and social networking. Language Learning & Technology, 10(2), 8-15.

Goh, D.H.-L., Chua, A., Lee, C.S., & Razikin, K. (2009). Resource discovery through social tagging: A classification and content analytic approach. Online Information Review, 33(3), 568-583.

Golder, S.A., & Huberman, B.A. (2006). Usage patterns of collaborative tagging systems. Journal of Information Science, 32(2), 198-208.

Gordon-Murnane, L. (2006). Social bookmarking, folksonomies, and web 2.0 tools. Searcher Mad Database Prof, 14(6), 26-38.

Grudin, J. (2006). Enterprise knowledge management and emerging technologies. In Proceedings of the 39th Hawaii International conference on System Sciences, 3 (pp. 57a). IEEE Xplore Digital Library.

Hammond, T., Hannay, T., Lund, B., & Scott, J. (2005). Social bookmarking tools (I) - A general review. D-Lib Magazine 11(4). Retrieved from www.dlib.org/dlib/april05/hammond/04hammond.html

Hotho, A., Jäschke, R., Schmitz, C., & Stumme, G. (2006). Trend detection in folksonomies. Lecture Notes in Computer Science, 4303, 56-70.

Huang, A.W.C., & Chuang, T.R. (2009). Social tagging, online communication, and Peircean semiotics: A conceptual framework. Journal of Information Science, 35(3), 340-357.

Kipp, M.E.I. (2006). Exploring the context of user, creator and intermediary tagging. In Proceedings of the 7th Information Architecture Summit (Vancouver, Canada). Retrieved from http://iasummit.org/2006/files/109_Presentation_Desc.pdf.

Kittur, A., Chi, E.H., Pendleton, B.A., Suh, B., & Mytkowicz, T. (2007). Power of the few vs. wisdom of the crowd: Wikipedia and the rise of the bourgeoisie. In the Proceedings of SIGCHI Conference on Human Factors in Computing Systems. San Jose, CA.

Lin, C.C., & Tsai, C.C. (2011). Applying social bookmarking to collective information searching (CIS): An analysis of behavioral pattern and peer interaction for co-exploring quality online resources. Computers in Human Behavior, 27(3), 1249-1257.

Lund, B., Hammond, T., Flack, M., & Hannay, T. (2005). Social bookmarking tools (II): A case study, Connotea. D-Lib Magazine 11(4).

Marchionini, G. (2006). Exploratory search: From finding to understanding. Communications of the ACM, 49(4), 41-46.

Markus, M.L. (1987). Toward a ''critical mass'' theory of interactive media. Communication Research, 14(5), 491-511.

Mejias, U. (2005, April). Tag literacy. Retrieved from http://blog.ulisesmejias.com/2005/04/26/tag-literacy/

Millen, D., Feinberg, J., & Kerr, B. (2005). Social bookmarking in the enterprise. ACM Queue, 3(9), 28-35.

Millen, D., Yang, M., Whittaker, S., & Feinberg, J. (2007). Social bookmarking and exploratory search. In L. Bannon et al. (Eds.), Proceedings of 10th European Conference on Computer Supported Cooperative Work (pp. 21-40). Limerick, Ireland: Springer.

Niu, X., & Hemminger, B.M. (2011). A study of factors that affect the information-seeking behavior of academic scientists. Journal of American Society for Information Science and Technology, 63(2), 336-353.

Panke, S., & Gaiser, B. (2009). “With my head up in the clouds”: Using social tagging to organize knowledge. Journal of Business and Technical Communication, 23(3), 318-349.

Rader, E., & Wash, R. (2008). Influences on tag choices in del.icio.us. In Proceedings of the ACM conference on Computer Supported Cooperative Work (pp. 239-248). New York, NY: ACM.

Razikin, K., Goh, D.H.L., Chua, A.Y.K., & Lee, C.S. (2008). Can social tags help you find what you want? Research and Advanced Technology for Digital Libraries, 5173, 50-61.

Redden, C.S. (2010). Social bookmarking in academic libraries: Trends and applications. The Journal of Academic Librarianship, 36, 219-227.

Rethlefsen, M.L. (2008). Electronic resources review. Connotea. Journal of the Medical Library Association, 96(2), 175-178.

Riddle, P. (2005). Tags: What are they good for? Retrieved from http://www.ischool.utexas.edu/~i385q/archive/riddle_p/riddle-2005-tags.pdf

Seki, K., Qin, H., & Uehara, K. (2010). Impact and prospect of social bookmarks for bibliographic information retrieval. In Proceedings of the 10th Annual Joint Conference on Digital Libraries (pp. 357-360). New York, NY: ACM.

Sen, S., Lam, S.K., Rashid, A.M., Cosley, D., Frankowski, D., Osterhouse, J., Harper, F.M., & Riedl, J. (2006). Tagging, communities, vocabulary, evolution. In Proceedings of the 20th Anniversary Conference on Computer Supported Cooperative Work (pp. 181-190). New York, NY: ACM.

Storey, M.A., Cheng, L.T. Bull, I., & Rigby, P. (2006). Shared waypoints and social tagging to support collaboration in software development. In Proceedings of the 20th Anniversary Conference on Computer Supported Cooperative Work (pp. 195-198). New York, NY: ACM.

Sun, A., Suryanto, M.A., & Liu, Y. (2007). Blog classification using tags: An empirical study. Asian digital libraries. Lecture Notes in Computer Science, 4822, 307-316.

Yanbe, Y., Jatowt, A., Nakamura, S., & Tanaka, K. (2007). Can social bookmarking enhance search in the web? In Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries (pp. 107-116). New York, NY: ACM.

Table 1. Research on SB, 2006-2010

Research topics

Author(s)

Year Published

Findings

tag growth and dynamics

Golder & Huberman

2006

- Users exhibit a wide variety in their sets of tags while stable patterns emerge in tag proportions.- Most of the tagging is done not for public benefit but personal purpose.

Kittur et al.

2007

- There was also the decline of elite user influence in del.icio.us.

Farooq et al.

2008

- This study investigates tag growth and patterns and a matrix is hence developed to explain tagging behavior.

semantics

Al-Khalifa & Davis

2007

- The study categorized folksonomy tags into meaningful semantics.

Cattuto, Benz, Hotho & Stumme

2009

- It introduces a systematic methodology in a folksonomy for characterizing different measures of tag relatedness.

tag choices

Radar

2006

- The views towards the concept space of the articles from the user, author and intermediary are different.- It is important for users to have time management information as a tag for articles.- A large number of user terms were related to author and intermediary terms. - In most cases, this was due to the use of newer terminology or differences in approaching problems.

Kipp

2008

- Users’ past tag choices greatly influence future tag choices.

- In del.icio.us, tag selection is governed by individual and idiosyncratic processes instead of direct limitation.

classification of tags

Sen et al.

2006

- Pre-existing tags do affect future tagging behavior, i.e., users tend to follow the distribution of the pre-existing tag.

Sun et al.

2007

- More tags do not necessarily give rise to better classification.

- Moreover, it evaluates a tag expansion algorithm.

Razikin el al.

2008

- It is not true that all tags are useful descriptors for resource sharing.

- The result is not consistent with the Wisdom of Crowds theory.

- It is found that objective tags appear frequently.

Huang & Chuang

2009

- This study identifies ten classes of social tagging which can offer a semiotic solution to the ambiguous and vague tagging in the online communication process.

potential academic use and application

Gordon-Munane

2006

- To make tagging as easy as going to Google, new features should be developed to capture the context and meaning.

- Improving the easiness to tag pages or other media and merging the collaborative tagging with more formal systems could strengthen the power of searching.

Redden

2010

- Academic libraries may use SB as collaborative tools in practical and relatively simple ways.

Seki, Qin & Uehara

2010

- This study revealed that there has been an improvement in information retrieval with the use of social bookmarks.

Table 2. Academic background of the participants (N=119)

Academic background

Group I

(experienced users)

Group II

(moderately experienced users)

Group III

(novice users)

(n = 39)

(n = 48)

(n = 32)

Science and engineering

8

5

14

Social science and humanities

1

43

13

General university research/administration

22

0

3

Business research/administration

1

0

1

Others

7

0

1

Table 3. Bookmarking behaviors

Bookmarking behaviors

Group I

Group II

Group III

Kruskal Wallis

M (SD)

M (SD)

M (SD)

p-value

Creating bookmarks

3.26 (1.349)a

2.63 (0.890)d

2.53 (0.983)f

0.014*

Copying bookmarks from other users

1.68 (0.747)b

1.96 (0.771)d

1.63 (0.609)f

0.085

Using self-created bookmarks

2.62 (1.139)b

2.53 (1.120)e

2.31 (0.931)f

0.485

Sharing bookmarks with collaborators

1.92 (1.360)c

2.02 (1.139)d

1.75 (0.762)f

0.514

Creating tag notes

2.13 (1.277)a

2.15 (0.899)d

1.94 (0.759)f

0.571

Notes: * statistically significant at p < .05.

an = 38, bn = 37, cn = 36, dn = 48, en = 47, fn = 32.

Participants gave ratings based on a scale where 1 = Never, 2 = Once a month or less, 3 = Once every 2 weeks, 4 = 1-2 times a week, 5 = 3-6 times a week, 6 = Once every day or more.

Table 4. Users' perceptions of the usefulness of bookmarks and tags

Criterion

Group I

Group II

Group III

Kruskal Wallis

M (SD)

M (SD)

M (SD)

p-value

It is useful to create a title for a bookmark

2.94 (0.827)a

3.21 (0.690)g

3.03 (0.556)k

0.192

It is useful to form a group with friends or colleagues for sharing bookmarks

2.93 (0.704)b

2.98 (0.766)g

2.80 (0.610)k

0.424

It is useful to use tags to find your own bookmarks

3.78 (0.428)c

3.17 (0.732)g

3.07 (0.583)k

0.000*

It is useful to use tags in finding relevant bookmarks created by other Connotea users

3.00 (0.816)d

3.04 (0.658)g

2.90 (0.759)k

0.629

It is useful to use tags in sharing bookmarks with friends or colleagues

3.19 (0.750)e

3.02 (0.715)h

2.73 (0.583)k

0.025*

It is useful to create tag notes in Connotea

3.24 (0.768)e

3.05 (0.524)i

2.88 (0.612)l

0.082

It is a good policy that Connotea requires its users to create at least one tag per bookmark

3.19 (0.834)f

3.05 (0.714)j

3.04 0.611)m

0.419

Notes: * p < .05.

an = 17, bn = 13, cn = 18, dn = 15, en = 21, fn = 27, gn = 47, hn = 46, in = 37, jn = 40, kn = 30, ln = 24, m n= 25. Participants gave ratings based on a 4 - point Likert scale where 1 = “Strongly disagree”, and 4 = “Strongly agree”.

Percentage of “don’t know” answers is represented in gray; the darker the shade the larger the percentage (see Fig. 2).

Table 5. Users' perception on automatic collection of bibliographic information

Statement

Group I

Group II

Group III

Kruskal Wallis

M (SD)

M (SD)

M (SD)

p-value

It is useful

3.33 (0.816) a

3.03 (0.556) c

3.17 (0.491) d

0.082

It is easy to use

2.92 (0.862) b

2.84 (0.473) b

2.71 (0.772) e

0.399

Notes: an = 24, bn = 25, cn = 30, dn = 23, en = 17.

Participants gave ratings based on a 4 - point Likert scale where 1 = “Strongly disagree”, and 4 = “Strongly agree”.


Table 6. Users' perception on the usefulness of different search functions

Methods

Group I

Group II

Group III

Kruskal Wallis

M (SD)

M (SD)

M (SD)

p-value

Using the search box with the choice "Find Exact URL"

2.10 (0.944)d

2.54 (0.852)g

2.58 (1.121)n

0.147

Using the search box with the choice "Find Exact User"

2.64 (0.790)a

2.41 (0.837)k

2.67 (0.856)d

0.503

Using the search box with the choice "This Collection"

2.95 (0.844)a

2.77 (0.731)g

3.05 (0.669)d

0.344

Through other users who have similar interests

2.87 (0.869)e

3.10 (0.656)h

3.05 (0.575)a

0.504

Using the search box with the choice "All"

2.80 (0.816)c

2.63 (0.751)i

2.88 (0.680)f

0.534

Using "Related Users"

2.88 (1.076)f

2.25 (0.770)m

2.65 (0.832)e

0.031*

Using "Related Tags"

2.83 (1.029)e

2.88 (0.714)l

2.96 (0.767)e

0.891

Using the search box with the choice "My Library"

3.27 (0.874)b

3.17 (0.730)h

3.08 (0.717)f

0.440

Note: * statistically significant at p < .05.

an = 22, bn = 26, cn = 25, dn = 21, en = 23, fn = 24, gn = 35, hn = 42, in = 38, jn = 39, kn = 32, ln = 41, mn = 36, nn = 19.

Participants gave ratings based on a scale where 1 – Not useful, 2 – A bit useful, 3 – Useful, 4 – Very useful.


Table 7. Users' perceptions on the use of SB for information management

Criterion

Group I

Group II

Group III

Kruskal Wallis

M (SD)

M (SD)

M (SD)

p-value

Individual Information Management

It is quick to create bookmarks

3.15 (0.784)a

2.98 (0.794)e

3.21 (0.738)h

0.308

It is easy to save bookmark

3.19 (0.786)b

3.02 (0.707)e

3.19 (0.483)b

0.365

It is useful for managing personal information

2.92 (0.845)a

2.87 (0.647)e

3.04 (0.706)i

0.473

Group Information Management

It is easy to share bookmarks with others

3.04 (0.690)c

3.10 (0.700)f

3.05 (0.394)d

0.854

It is useful for managing information in a group

2.95 (0.686)d

2.89 (0.689)g

2.81 (0.602)j

0.748

Notes: an = 26, bn = 27, cn = 24, dn = 20, en = 47, fn = 41, gn = 38, hn = 28, in = 23, jn = 21.

Participants gave ratings based on a scale where 1 – Not useful, 2 – A bit useful, 3 – Useful, 4 – Very useful.


Table 8. Percentage of users who found various SB features to be useful.

Feature

Group I

(n= 25)†

Group II

(n = 47)†

Group III

(n=30)†

Recognizing references and automatically filling in bibliographic information

68.0%

27.7%

30.0%

Sharing references/bookmarks among all users of the SB website - good for information discovery

48.0%

57.4%

36.7%

Accessing your references/bookmarks from any computer

80.0%

78.7%

80.0%

Exporting your references/bookmarks to Endnote or other desktop reference managers

32.0%

19.1%

16.7%

Notes: †Based on participants who responded to this question. Users could select all those features that they found useful; the percentage shown here is the number of users who have picked this feature over the total number of respondents.


Table 9. Users' perception on SB and traditional way of saving bookmarks

Statement

Group I

Group II

Group III

Kruskal Wallis

M (SD)

M (SD)

M (SD)

p-value

I enjoy using the SB website

3.04 (0.825)a

2.66 (0.645)c

2.69 (0.736)b

0.043*

Using SB websites like Connotea to save bookmarks is better than using the traditional way of saving bookmarks into a dedicated computer

3.81 (0.402)b

3.02 (0.690)d

2.76 (0.926)e

0.000*

Notes: * p < .05. an = 23, bn = 26, cn = 44, dn = 45, en = 25.

Participants gave ratings based on a 4 - point Likert scale where 1 = “Strongly disagree”, and 4 = “Strongly agree”.


Table 10. Percentage of users who use the methods of managing useful websites

Ways of managing useful websites

Group I

(n=26)†

Group II

(n=47)†

Group III

(n=30)†

Bookmarking websites on a computer

42.3%

80.9%

83.3%

Using Connotea

61.5%

21.3%

23.3%

Using other SB sites

61.5%

17.0%

13.3%

Adding them to your own website

7.7%

14.9%

3.3%

Notes: † Based on participants who responded to this question.


Figures

Figure 1. Monthly number of visitors to academic SB websites.

white

Less than 15% [more desirable]

light grey

15% - 30% :

grey

30% - 45% :

dark grey

45% or more [less desirable]

Figure 2. Percentage of “Don’t know” answers.

Note: Number of participants who responded to this particular question in each group are as follows – Group I n = 29, Group II n = 47, Group III n = 30.

Figure 3. Average number of tags per bookmark and the percentage of users who created them.

Appendix A

Survey Questionnaire

1. What is your background?

Student (undergraduate/postgraduate in science & engineering)

Student (undergraduate/postgraduate in the humanities and the social science)

Researcher (academic/university)

Researcher (business)

Researcher (government)

Researcher (non-profit)

Manager/administrator (academic/university)

Manager/administrator (business)

Manager/administrator (government)

Manager/administrator (non-profit)

Others, please specify

2. Main reason for using Connotea:

For my job

For my studies

For personal use

Others, please specify

3. Which country do you mainly live in?

4. Please respond to the following set of statements on your bookmarking behaviors:

[Never/ Once a month or less/ Once every 2 weeks/ 1-2 times a week/ 3-6 times a week/ Once every day or more]

Frequency of creating bookmarks in Connotea

Frequency of copying bookmarks from other Connotea users

Frequency of using bookmarks you have created in Connotea

Frequency of sharing bookmarks you created with your collaborators

Frequency of creating tag notes in Connotea

5. Please indicate how much you agree or disagree with the following statements:

[Strongly Disagree/ Disagree/ Agree/ Strongly Agree/ Don’t Know]

It is useful to create a title for a bookmark.

It is useful to form a group with friends or colleagues for sharing bookmarks.

It is useful to use tags to find your own bookmarks.

It is useful to use tags in finding relevant bookmarks created by other Connotea users.

It is useful to use tags in sharing bookmarks with friends or colleagues.

6. Please indicate how much you agree or disagree with the following statement:


It is a good policy that Connotea requires its users to create at least one tag per bookmark.

7. Can you tell me why you think it is a good (or bad) policy for Connotea to require users to create at least one tag per bookmark?



It is useful to create tag notes in Connotea.

9. Can you tell me why creating tag notes is useful/ is not useful?

10. Number of tags you usually create in Connotea per bookmark:

1

2

3

4

More than 4



The automatic collection of bibliographic information in Connotea is useful.

12. If you selected “Disagree” or “Strongly Disagree” on the question “the automatic collection of bibliographic information in Connotea is useful,” it is because (Please check all that apply to you):

Sometimes Connotea cannot recognize the reference of an article and hence a citation record cannot be created.

The source websites (e.g. PubMed) supporting this function are too few.

Other (Please specify)



It is easy to use the automatic collection of bibliographic information function in Connotea.

14. Can you tell me why you find the automatic collection of bibliographic information easy/ not easy to use?

15. Please rate the usefulness of the following methods of finding information in Connotea:

[Not useful/ A bit useful/ Useful/ Very useful/ Don’t know]

Using the search box with the choice “This Collection” (the collection you are currently viewing)

Using the search box with the choice “My Library”

Using the search box with the choice “All” (all public bookmarks in Connotea)

Using the search box with the choice “Find Exact Tag”

Using the search box with the choice “Find Exact User”

Using the search box with the choice “Fund Exact URI” (e.g., http://www.ncbi.nlm.nih.gov/sites/entrez/)

Using “Related Tags”

Using “Related Users”



Connotea is effective in helping me find useful information through other Connotea members whose interests are similar to me.

17. Can you tell me why you find Connotea effective/ not effective in helping you find information through other Connotea members whose interests are similar to yours?



It is quick to create bookmarks in Connotea.

It is easy to save bookmarks in your Connotea account.

Connotea is useful for managing my personal information (e.g., organizing bookmarks for web pages you found for your research or for personal reasons).

19. Can you tell me why Connotea is useful/ is not useful for personal information management?



Connotea is useful for managing the information in a group (e.g, organizing the web pages/ information you found for a group project).

It is easy to share bookmarks with others.

21. Can you tell me why you find Connotea useful/ not useful for managing information in a group project?



I enjoy using Connotea.

23. Can you tell me why you enjoy/ not enjoy using Connotea?

24. What features in Connotea do you find most useful? (Please check all those features that you find useful.)

Accessing your reference/ bookmarks from any computer.

Exporting your reference/ bookmarks to Endnote or other desktop reference managers.

Recognizing reference and automatically filling in bibliographic information.

Sharing reference/ bookmarks among all Connotea users.

Other (Please specify)

25. What is your usual way of managing useful websites? (Please check all those that apply to you.)

Bookmarking websites on a computer

Using Connotea

Using other social bookmarking sites (please specify)

Adding them to your own webpage

Using other tools or methods (please specify)



Using social bookmarking websites like Connotea to save bookmarks is better than using the traditional way of saving bookmarks into a dedicated computer.

27. What do you think are the disadvantages of Connotea, and how can it be improved?

Appendix B

Summary of open-ended answers and comments by participants

Topic

Participant

Open-ended answers and comments

Inconsistent bibliographic information

Group I #9

Some bookmarks from the journal websites do not store the information in proper format.

Group I #14

I tend to bookmark the PubMed page, which works flawlessly.

Group I #16

Often I have to edit the reference.

Group II #8

It seems to be hard to use, especially not supporting famous paper database like JSTOR and Proquest.

Group II #12

I use Endnote to collect my bibliographic. It seems that Connotea does not support every version of Endnote.

Save time by finding information through other users with similar interests

Group II #3

It can help me to save time for screening what links are useful.

Group II #8

The intention is good to link up scientific knowledge by referring to others' bookmarks.

Group II #18

Other members whose interests are similar to mine have screened and bookmarked useful information in Connotea. It saves my time for screening information by searching in search engines, e.g. Google.

Spend extra time to get familiar with SB features

Group I #30

Connotea is very useful especially for those travel a lot, even though people take time to get used to the features in Connotea.

Group II #5

It takes extra time and efforts to get on with Connotea. User interface is not exceptionally easy to use, especially compare to more popular tools like Google.

Group II #8

The implementation is not good enough and busy people won't invest the time to study in detail how to use the functions.

Difficulties in accessing the bookmarks

Group I #30

Connotes is not so convenient that it requires you to log in first in Connotea, while compare with IE, it just need to click a button and then you can save the websites you want.

Group II #8

I'll prefer using Delicious as its integrated to my browser (Firefox) nicely.

Group II #10

Since I had only installed Connotea is my notebook, so when I was working with the computers in the office, I did not use Connotea.

Group III #8

Before I can use the Connotea in other PCs, I got to create the link on the toolbar. However, I am not successful to do that all the time.

Prefer traditional ways for personal information searching and management

Group II #12

I used to gather information to "My Favourite" file. It is very convenient for me.

Group II #13

I am currently using the Google bookmark function in which I can easily bookmark webpages. I have got used to the Google one.

Group II #34

I seldom use Connotea to share bookmarks with others, because I usually send the links to my friends through msn, email or Facebook.

SB cannot gain popularity quickly

Group II #22

Again, if I would like to share a bookmark with my colleague, I would rather like to e-mail them, or use some other more popular platform like Facebook group or Twitter.

Group III #31

Connotea is unable to provide a real time connection like Facebook and Twitter, for sharing information with friends. It also has to increase its popularity.

Not many friends or colleagues using SB

Group II #12

I seldom use Connotea because not many of my friends use it, unlike Facebook.

Group II #17

If I want to share Connotea bookmarks with friends, they also need to use Connotea as well.

Group II #22

Connotea is not very popular. None of my friend (except my current classmate) has the account of Connotea. This limits what I can get from Connotea.

Group II #34

Indeed, I don't have many friends who use Connotea. Thus, I use it personally.

I:\Sam-publications\published materials\journal articles\final draft\Du (in press) Academic social bookmarking.docx 2

Documents

web.hku.hkweb.hku.hk/.../Du-in-press-Academic-social-bookmarking.docx · Web viewRedden (2010), Sun, Suryanto and Liu (2007) and Seki, Qin and Uehara (2010) focused on potential applications