Skip to Main Content

Research impact and citation analysis: Measuring impact

Citation analysis

Citation databases track citations included in the reference lists of publications. 

Results of citation analysis will vary dependent upon the database used. This variation may occur because databases index different publication sources, across different publication date ranges. Some variation may also occur due to the inclusion of poor quality data - such as duplicate records, misspelt citations, missing authors and other citation data.

Many databases include citation count data. However, no single database will index all publications by an individual researcher. 

H-Index

Jorge Hirsch proposed the h-index or Hirsch index in 2005 as a means of quantifying the impact and productivity of a scientist. The h-index is calculated on the number and impact of a researcher’s publications.  An h-index of 40 means that a researcher has published 40 papers that each have at least 40 citations. 

More Information: Hirsch, J. (2005). An index to quantify an individual's scientific research output. PNAS 102 (46): 16569-16572. 

NOTE: H-index is included in NURO for University of Newcastle RHD students, researchers and conjoint staff.

Key databases

An author's h-index may be different in each database, since the calculation is based on the indexed content within each resource.

Alternative metrics

Alternative metrics can provide details about the usage of publications, such as the number of:

  • clicks or views online
  • downloads
  • saves or bookmarks into online reference management services such as Mendeley and CiteULike
  • tweets, blog posts, shares and likes on Facebook, bookmarks, etc. in social media.
Strengths of alternative metrics Weaknesses of alternative metrics:
  • Speed - can accumulate more quickly than traditional metrics such as citations.
  • Range - can be gathered for many types of research output, not just scholarly articles.
  • Granularity - can provide metrics at the article, rather than journal level, such as journal impact factors.
  • Detail - can give a fuller picture of research impact using many indicators, not just citations.
  • Non-academic - can measure impact outside the academic word, where people may use but not cite research.
  • Sharing - if researchers get credit for a wider range of research outputs, such as data, it could motivate further sharing.
  • Standards - there are a lack of standards for alternative metrics.
  • Unregulated - could be manipulated or gamed.
  • Reliability - may indicate popularity with the general public rather quality research.
  • Time - there is no single widely used rating or score and alternative metrics can be time consuming to gather.
  • Difficulty - can be difficult to collect, for example bloggers or tweeters may not use unique identifiers for articles.
  • Overload - there are many different metrics and providers to choose from and it can be hard to determine which are relevant.
  • Acceptance - many funders and institutions use traditional metrics to measure research impact.
  • Context - use of online tools may differ by discipline, geographic region, and over time, making alterative metrics difficult to interpret.
    Source: http://library.soton.ac.uk/bibliometrics/altmetrics 
 
Alternative metrics can be found:

Measuring the impact of books

Check for library holdings of the book

  • Trove - National Library of Australia service providing access to printed, electronic, pictorial, newspaper, research, map, music and archival material
     
  • WorldCat (OCLC)  - search portal for over 10,000 libraries worldwide
     
  • The European Library - search portal for European national libraries
     
  • COPAC - search portal for university and national libraries in Great Britain

 

Has the book been cited?

 

Has the book been reviewed?

  • Amazon.com - Includes the full text of some book reviews

  • Google Books - enter author's name, title or title keywords and review. The record includes a link to any editorials or user reviews.

  • JSTOR - Select Advanced Search,  enter words from the title, then select Reviews in the Narrow By: area

  • Project Muse - Enter search terms and under Type of Content choose Reviews, then click Search.

  • ProQuest - Select Advanced Search, enter words from the title, then select Reviews from the Content Type area (left of screen)

  • Web of Science - Select Cited Reference Search, enter the author name in the Cited Author box, the first words of the book title in the Cited Work box and click Search. When results display, Select All, scroll to bottom of screen and select Book Review from the Document types drop down list.

Other questions to ask

  • Has the book been used as a textbook at other universities?
  • Has the book been translated?
  • Has the book been awarded a prize, or noted by an organisation as having made a significant contribution to a field?
  • Has the book is noted on a publisher's best-seller list?
  • Are sales figures available for how many copies sold?
  • Has the book been recommended on Google+?
  • Has the author been invited to present on the topic related to the book or book chapter?
  • Have subsequent editions of the book been published?

 

Source: The Becker List: Impact Indicators

Citation alerts

Keeping track of who's citing your research

Receive email alerts on who is citing your research:​

H-Index alternatives

Whilst a number of alternatives have been proposed to the H-Index, the G-Index is the only one included as a metric in the key databases.