Challenges of scholarly publications on the Web to the evaluation of science—A comparison of author visibility on the Web and in print journals
Introduction
As the accelerated development of information technology, especially the rapid growth of the Web, is changing the circumstances and consequently the structures and processes of scholarly communication, there is renewed interest in the study of scholarly communication to see how it is being transformed, what the similarities or differences between the new formats of communication and the traditional ones might be, and how the new formats facilitate or inhibit the scholarly communication process (Borgman and Furner, 2002, Cronin, 2001, Zhao, 2003).
Citation analysis and other informetric techniques have been applied successfully to the study of this new phenomenon in scholarly communication. As Zhao and Logan (2002) point out, some studies apply, often with modifications, informetric principles and techniques to study the characteristics and link structures of the Web. Examples include studies on search engines that make use of hyperlink structure (Clever Project, 1999), and so-called “Webometrics” studies (Almind and Ingwersen, 1997, Cronin et al., 1998, Egghe, 2000, Larson, 1996, Rousseau, 1997, Thelwall and Harries, 2004, Turnbull, 2000, Wilkinson et al., 2003). Other studies consider “electronic ingredients” in journal articles—either in reference lists or in abstracts—to gauge the impact of electronic publications on traditional print journal-based scholarly communication (Harter, 1992, Harter and Kim, 1996, ISI, 2004a, Lu, 1999, McCain, 2000, Youngen, 1997). Still others examine scholarly communication patterns demonstrated in research papers published on the Web, and study the differences from, and similarities to, what we have come to expect from print journal-based communication (Goodrum et al., 2001, Lawrence, 2001, The Open Citation Project, 2001, Zhao and Logan, 2002, Zhao, 2003, Zhao, 2004).
As part of a larger research project that aims to systematically compare scholarly communication patterns between the Web and the print world, the present study focuses on the comparison of author visibility between the Web and print journals as revealed from citation analysis, and discusses the challenges of scholarly communication being increasingly conducted over the Internet to traditional scholarly communication system in general and to the common practice of science evaluation based on the databases of the Institute for Scientific Information (ISI) in particular. The present study along with other parts of the project (Zhao, 2004, Zhao and Logan, 2002, Zhao and Strotmann, 2004) may contribute to the understanding of the transition of scholarly communication from print to electronic media, to advancing citation analysis theory and methodology, and to information organization and retrieval on the Web.
Section snippets
Research questions
Within a citation analysis framework, author visibility can either be measured in terms of how frequently authors have been publishing or in terms of how often their published works have been used (cited) by other scholars. Based on this consideration, the research questions to be explored in the present study are as follows.
- •
Are there any significant correlations between author rankings by number of publications identified from the Web and those identified from print journals in the field of
Data collection
The ISI’s Science Citation Index (SCI) and The NEC Research Institute’s CiteSeer1 were used in the present study to collect information on research papers published in print journals and on the Web, respectively. To date, the ISI databases including SCI have been used as the single data source for most of
Findings
A search on “XML” or “eXtensible Markup Language” resulted in 312 papers using CiteSeer (after removing duplicates) and in 374 papers with reference lists using SCI, 268 of which from computer science journals. Among these papers, only 26 were common to both CiteSeer and SCI. The papers from CiteSeer made 4578 citations, and those from SCI made 6782 citations. Among the cited papers from CiteSeer, 22% (987) were proceedings, 22% (991) were Web publications, 2.3% (105) were technical reports,
Discussion
The findings presented above can shed some light on issues of both citation analysis and scholarly communication in transition.
Conclusion
Scholarly communication is increasingly being conducted over the Internet, certainly in the core research areas of computer science but increasingly also in other sciences. This has brought some challenges as well as opportunities to the traditional scholarly communication systems and to the associated evaluation approaches. The present study has explored these through a citation analysis of author visibility in one particular research field—the XML research field. It has revealed different
Acknowledgement
The author wishes to thank Dr. Andreas Strotmann of the Center for Applied Computer Science at the University of Cologne, for his many helpful insights.
This work was supported in part by a Fellowship of the School of Computational Science and Information Technology, Florida State University.
Dangzhi Zhao is an Assistant Professor at the School of Library and Information Studies of the University of Alberta, Canada. Her research interests are in the areas of information systems, scholarly communication, and information technology. Recent research concentrates on the comparison of scholarly communication between the Web and the print world.
References (38)
- et al.
Scholarly publishing in the Internet age: a citation analysis of computer science literature
Information Processing and Management
(2001) - et al.
Informetric analyses on the World Wide Web: Methodological approaches to “Webometrics”
Journal of Documentation
(1997) Data collection methods on the Web for informetric purposes—A review and analysis
Scientometrics
(2001)- et al.
Weaving the web
(1999) - et al.
The semantic web
Scientific American
(2001) - et al.
Scholarly communication and bibliometrics
- Bosak, J. (1997). XML, Java and the future of the Web. Retrieved April 8, 2005, Available from...
- et al.
XML and the second-generation Web
Scientific American
(1999) Supporting and enhancing scholarship in the digital age: the role of open-access institutional repositories
Canadian Journal of Communication
(2004)- Clever Project. (1999). Hypersearching the Web. Retrieved 2000, Available from...
Bibliometrics and beyond: some thoughts on Web-based citation analysis
Journal of Information Science
Invoked on the Web
Journal of the American Society for Information Science
New informetric aspects of the Internet: some reflections, many problems
Journal of Information Science
Psychological relevance and information science
Journal of the American Society for Information Science
Cited by (34)
The distribution of Web citations
2012, Information Processing and ManagementCitation Excerpt :The rapid development of computer and network technology has resulted in the transformation of the traditional scholarly communication system, which has been in place for more than three centuries, by the Internet (Zhao, 2005).
The integration of open access journals in the scholarly communication system: Three science fields
2009, Information Processing and ManagementInformetrics at the beginning of the 21st century-A review
2008, Journal of InformetricsInfgraph: Influential Researcher and Cited Research Analysis Using Citation Network
2022, International Journal of Decision Support System TechnologyIMUW-APP: An instrument for measuring the usability of web applications
2021, Indonesian Journal of Electrical Engineering and Computer ScienceBecoming Metric-Wise: A Bibliometric Guide for Researchers
2018, Becoming Metric-Wise: A Bibliometric Guide for Researchers
Dangzhi Zhao is an Assistant Professor at the School of Library and Information Studies of the University of Alberta, Canada. Her research interests are in the areas of information systems, scholarly communication, and information technology. Recent research concentrates on the comparison of scholarly communication between the Web and the print world.