Skip to main content
Log in

Advantages and limitations in the use of impact factor measures for the assessment of research performance

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Impact factor is a quasi-qualitative indicator, which provides a measurement of the prestige and international visibility of journals. Although the use of impact factor-based indicators for science policy purposes has increased over the last two decades, several limitations have been pointed out and should be borne in mind. The use of impact factor should be treated carefully when applied to the analysis of peripheral countries, whose national journals are hardly covered by ISI databases. Our experience in the use of impact factor based indicators for the analysis of the Spanish scientific production is shown. The usefulness of the impact factor measures in macro, meso and micro analyses is displayed. In addition, the main advantages, such as the great accessibility of impact factor and its ready-to-use nature are pointed out. Several limitations such as the need to avoid inter-field comparisons or the convenience of using a fixed journal set for international comparisons are also stressed. It is worth noting that the use of impact factor in the research evaluation process has influenced strongly the publication strategy of scientists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Amin, M., Mabe, M. (2000), Impact factors: use and abuse, Perspectives in Publishing, 1: 1-6.

    MATH  Google Scholar 

  • Arunachalam, K., Manorama, K. (1988), Are citation-based quantitative techniques adequate for measuring science on the periphery? Scientometrics, 15(5–6): 393-408.

    Google Scholar 

  • Bordons, M., BarrigÓn, S. (1992), Bibliometric analysis of publications of Spanish pharmacologist in the SCI (1984–89). Part II. Contribution to subfields other than “Pharmacology and Pharmacy”, Scientometrics, 25 (3): 425-446.

    Article  Google Scholar 

  • Bordons, M., Zulueta, M. A. (1997), Comparison of research team activity in two biomedical fields, Scientometrics, 40 (3): 423-436.

    Article  Google Scholar 

  • CamÍ, J., FernÁndez, M. T., GÓmez, I. (1993), Producción científica española en Biomedicina y Salud. Un estudio a través del Science Citation Index (1986–1989), Medicina Clínica (Barc) 101 (19): 721-731.

    Google Scholar 

  • CamÍ, J., Zulueta, M. A., FernÁndez, M. T., Bordons, M., GÓmez, I. (1997), Producción científica española en biomedicina y ciencias de la salud durante el período 1990–1993 (Science Citation Index y Social Science Citation Index) y comparación con el período 1986–1989, Medicina Clínica (Barc) 109: 481-496.

    Google Scholar 

  • Garfield, E. (1998), Long-term vs. short-term journal impact: Does it matter? The Scientist, 12 (3): 11-12.

    Google Scholar 

  • GÓmez, I. Bordons, M. (1996), Limitaciones en el uso de los indicadores bibliométricos para la evaluación científica, Política Científica, 46: 21-26.

    Google Scholar 

  • GÓmez, I., FernÁndez, M. T., Bordons, M., Cabrero, A., Morillo, F., Rojo, R. (2001), La actividad científica del CSIC a través del Science Citation Index, Social Science Citation Index y Arts & Humanities Citation Index. Estudio bibliométrico del período 1997–99. Report. CINDOC, Madrid. Institute for Scientific Information (2001) http://www.isinet.com/isi/hot/research/200012/3310412/index.hml

    Google Scholar 

  • LÓpez piÑero, J. M., Terrada, M. L. (1994), El consumo de información científica nacional y extranjera en las revistas médicas españolas: un nuevo repertorio destinado a su estudio, Medicina Clínica (Barc) 102: 104-112.

    Google Scholar 

  • Moed, H. F., Van Leeuwen, Th. N., Reedijk, J. (1998), A new classification system to describe the ageing of scientific journals and their impact factors, Journal of Documentation, 54 (4): 387-419.

    Article  Google Scholar 

  • Moed, H. F., Van Leeuwen, Th. N. (1995), Improving the accuracy of Institute for Scientific Information's journal impact factors, JASIS, 46 (6): 461-467.

    Article  Google Scholar 

  • Noma, E. (1986), Subject classification and influence weights for 3,000 journals. Report. Computer Horizons, Inc./CHI Research.

  • Osareh, F., Wilson, C. S. (1995), Scientific productivity and impact of the third world countries: a citation study, In: Koenig, M. D. E., Bookstein, A. (Eds), Proceedings of the Fifth Biennial Conference of the International Society for Scientometrics and Informetrics. Learned Information, Medford.

    Google Scholar 

  • Seglen, P. O. (1992), How representative is the journal impact factor? Research Evaluation, 2 (3): 143-149.

    Google Scholar 

  • Sen, B. K., Karanjai, A., Munshi, U. M. (1989), A method for determining the impact factor of a non-SCI journal, Journal of Documentation, 45 (2): 139-141.

    Google Scholar 

  • Stegmann, J. (1999), Building a list of journals with constructed impact factors, Journal of Documentation, 55 (3): 310-324.

    Article  Google Scholar 

  • Solari, A., Magri, M. H. (2000), A new approach to the SCI Journal Citation Reports, a system for evaluating scientific journals, Scientometrics, 47 (3): 605-625.

    Article  Google Scholar 

  • Van Leeuwen, Th. N., Moed, H. F., Tijssen, R. J. W., Visser, M. S., van Raan, A. F. J. (2001), Language biases in the coverage of the Science Citation Index and its consequences for interntaional comparison of national research performance, Scientometrics, 51: 335-346.

    Article  Google Scholar 

  • Van Leeuwen, Th. N., Moed, H. F., Reedijk, J. (1999), Critical comments on Institute for Scientific Information impact factors: a sample of inorganic molecular chemistry journals, Journal of Information Science, 25 (6): 489-498.

    Article  Google Scholar 

  • Velho, L. (1986), The meaning of citation in the context of a scientifically peripheral country, Scientometrics, 9 (1–2): 71-89.

    Article  Google Scholar 

  • Zitt, M., Perrot, F., BarrÉ, R. (1998), The transition from “national” to “transnational” model and related measures of countries performances, JASIS, 49 (1): 30-42.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bordons, M., Fernández, M.T. & Gómez, I. Advantages and limitations in the use of impact factor measures for the assessment of research performance. Scientometrics 53, 195–206 (2002). https://doi.org/10.1023/A:1014800407876

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1014800407876

Keywords

Navigation