Bibliometrics: evaluating research via publications

Decorative image. Typewriter, book, notebook, glasses, paper on a wooden desk.

In most academic fields, research findings are principally communicated, verified and assessed through publications, which can be roughly classified into four main categories:

  • articles in scientific journals
  • scientific monographs
  • articles or chapters in edited scientific books
  • articles in scientific conference proceedings

Publishing in other publishing channels, such as research reports, textbooks, articles in professional magazines or newspapers, and popular books, is one means of dissemination of scientific knowledge also to the non-scholarly public.

In general, bibliometric indicators of publishing activity measure the number of publications published by a unit or scholar in a given time period. In many areas of the natural and medical sciences, articles in international refereed journals, often indexed by Web of Science and Scopus (see sections “Web of Science” and “Scopus” of this module), are the primary channel for publishing research results, and thus they cover most of the output. On the other hand, the majority of research outcomes in the social sciences and humanities lie beyond journal articles.

Indicators based on citations received by publications are often used as quantitative measures of research performance. It must, however, be noted that citation indicators can act as a supplementary tool for peer-based evaluation. The impact measured by citations particularly refers to the publication’s international scientific influence, and citation analyses do not capture societal impact well. There are certain advantages of citation measures over peer-based assessments: citation counts are to some extent objective and harder to manipulate, since the research is assessed by the whole research community rather than being limited to the personal knowledge of a few peers. However, the distribution of citations is extremely skewed. Most papers are cited rarely, whereas a few papers can gain even thousands of citations. With regard to individual publications, citations tend to cumulate, and recognition adds further recognition. Also, older papers have had more time to gain citations than recent ones.

In bibliometrics, co-publishing patterns can be explored at various levels: of individual authors, at the domestic inter-organisational level, between sectors (universities, other public research organisations, commercial companies) and internationally. Co-publishing has increased rapidly at all levels. Also, increased internationalisation and international collaboration can be seen in all disciplinary groups. International co-publications are noticed to be cited more often than publications authored by scientists from one country only; domestic inter-organisational collaboration has been shown to increase citation rates compared to articles published by authors from one organisation.

Bibliometric indicators (more information in section “Bibliometric indicators” of this module) can serve well as a supporting tool alongside peer-review for assessment and decision making, and they should be developed further in relation to the various publishing practices in different disciplines. The use of bibliometric indicators requires understanding of both methodology and the studied context. One of the major problems in bibliometric analyses is that scientific disciplines differ, and research conducted within them has different goals, tasks, audience structures and funding structures. Also, the nature of research topics is different in different disciplines. As regards publishing, disciplines differ in terms of the publication types: scientific journals, books and conference proceedings aimed at national or international, scholarly or non-scholarly audiences.

Nowadays increasing interest is also for assessing the impact of research online and in social media (i.e. altmetrics) measured e.g. by the numbers of downloads, views, blog posts, likes and tweets (see for more at 4. Module).

Further information:

Bibliometrics for the University Community. eLearn course. UEF Library. The material is freely accessible.

Bibliometriikkaa yliopistolaisille. eLearn-kurssi. Itä-Suomen yliopiston kirjasto. Materiaaliin voi tutustua vapaasti.

Finnish national guide to publication metrics: Citations.

NordForsk. 2017. Comparing research at Nordic higher education institutions using bibliometric indicators. Covering the years 1999-2014. Policy Paper 4/2017. NordForsk, Oslo.

Responsible Research – Promoting responsible research at The Federation of Finnish Learned Societies. Coordination for Open Science, Publication Forum, The Committee for Public Information (TJNK), Finnish National Board on Research Integrity TENK and the Responsible Research Articles.

Roemer, R.C. & Borchardt, R. 2015. Meaningful metrics: a 21st century librarian´s guide to bibliometrics, altmetrics, and research impact. Association of College and Research Libraries, a division of the American Library Association, Chicago.

San Francisco Declaration on Research Assessment (DORA).

Working group for the responsible evaluation of a researcher. 2020. Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland. Responsible Research Series 5:2020.

(8/2023 JN)

< Previous page: 5. Module
Next page: Viability of bibliometrics >