PILOT logo, post doc PILOT logo, information PILOT logo, literacy PILOT logo, online PILOT logo, tutorial welcome information contents page search and retrieve contents page acquiring and managing information contents page new technologies contents page
digital footprint contents page
copyright contents page IL pilot license contents page databases contents page publication process contents page
Welcome

The main metrics

We will now review some of the main research impact metrics used. In modules 2 and 3 you will be shown how to find and calculate these metrics. You will also get to see some of the issues and limitations of the metrics and the tools used to produce them.

Quantitative measures provide just one part of the picture. They should only be used where it is appropriate to do so and with an understanding that it is extremely difficult to compare across disciplines without taking in to account a discipline's publication and citation patterns.

It is also widely acknowledged that the main citation tools do not offer the ability to produce appropriate and accurate metrics in Humanities, Computer Science and Engineering. These areas are less dependent on journals when compared to other disciplines. The coverage of areas within the social sciences is also limited in a number of tools.

Publication counts: the number of publications produced by an individual, unit or institution is the most basic measure of productivity. These should be treated with caution as a basic count does not take into account:

  • the number of researchers in an institution
  • the disciplines covered
  • the actual publications and the depth of coverage. This point applies to all tools!

Citation counts: Citations are used to measure a paper's or a group of papers' impact or influence. It is important to note that the "count" will vary depending on the tool used. If you use Web of Knowledge you can also view:

  • average citations per paper
  • average citations per institution

Some points to consider:

  • When comparing institutions, you must also consider the number of researchers in each institution.
  • Citation behaviour varies across disciplines, thus for example, an institution with a large medical school and strong emphasis on science will inevitably score higher in citation counts.
  • Citations take time to accrue
  • Average citation counts can be useful when making comparisons, but always compare like with like

H-index: The H-index is covered in detail in module 2. It is a popular metric in the sciences and is used to assess an individual, group of individuals or institution. The H-index (a single number) takes into account productivity (paper counts) and impact (citations). If an individual ahs an h-index of 7, this means that 7 of their papers have been cited at least 7 times each.

As with other key metrics, the H-index can only be used effectively when comparing like with like. As you will see in module 2, the calculation will also depend on the tool used.

Journal impact factor: the JIF is a metric applied to individual journals. JIF represents the average citation counts of papers published in a journal over a two year period. JIF is produced using data from the Web of Science. The JIF is updated every year and published in the Journal Citation Reports (JCR) database. The JIF will be examined in detail in module 3. A word of caution: Look at these three JIF's from 2008:

  • Journal of communication 2.266
  • Quarterly journal of economics 5.048
  • New England Journal of Medicine 50.014

You can see that one journal's JIF is much higher than the rest - yet they are all the journals with the highest JIF in their subject. So again, you should always compare like with like.

GCU logofont +  |   font -  |   next small plane | small plane previous page |  unit home small plane

Creative Commons License

My RI by University College Dublin, Dublin City University, Dublin Institute of Technology, The National University of Ireland, Maynooth and the NDLR adapted by Marion Kelt, Glasgow Caledonian University is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Based on a work at http://www.ndlr.ie/myri/.