Usage Factor

Last updated: February 2013

Usage-based measures of journal impact and quality

The overall aim of the Usage Factor project is to explore how online journal usage statistics might form the basis of a new measure of journal impact and quality, the Usage Factor (UF). The specific objectives of the project are: to examine the ways in which journal quality is currently assessed; to assess whether the UF would be a statistically meaningful measure; whether it would be accepted by researchers, publishers, librarians and research institutions; whether it would be statistically credible and robust; whether there is an organizational and economic model for its implementation that would be acceptable to the major stakeholder groups.

The project is being executed in three stages, from 2007 onwards. Stage 1 focussed on market research into the overall feasibility and acceptability of the Usage Factor in principle. Stage 2 focussed on modelling and analysis, in which real usage data from COUNTER-compliant publishers was used to test the formula for calculation of UF, as well as the processes for doing so on a sustainable, ongoing basis. The full report on Stages 1 and 2 of the project is provided below. Stage 3 of the project, which commenced in October 2011, is now under way.

One of the first outcomes of Stage 3 of the project is now available, the Draft Release 1 of the COUNTER Code of Practice for Usage Factors.

Draft Release 1 of the COUNTER Code of Practice for Usage Factors

Introduction

The Draft Release 1 of the COUNTER Code of Practice for Usage Factors, is one of the most significant outcomes to-date of the Usage Factor project, and is an important part of this, the final Stage of the project, which will take Usage Factor forward to full implementation. This Draft Release 1 is based on well-established COUNTER standards, procedures and protocols; it is designed to enable the recording and reporting by publishers of credible, consistent and compatible global Usage Factors for online publications hosted by them (and incorporating usage of these publications on other platforms that are capable of delivering COUNTER-compliant usage statistics). While Release 1 of this Code of Practice focuses on Usage Factors for journals, it is envisaged that its scope will be extended in subsequent Releases to cover other online publications, such as books and reference works.

It is our intention that the development of the Code of Practice for Usage Factors should be an open, transparent process, in which we actively seek input from individual researchers and practitioners, as well as from librarians, publishers and other information professionals. Our purpose in publishing this Draft Release 1 now, is threefold:

First, it sets out a formal, detailed standard for the recording, reporting and maintenance of Usage Factors, solidly based on the outcomes of Stages 1 and 2 of the Usage Factor project

Second, it provides a document for interested parties to review and comment upon, which we encourage, as this will greatly help us develop a definitive, implementable Code of Practice.

Third, it provides a framework for selected publishers and other organizations to do more extensive testing of the proposed processes for recording and reporting Usage Factors

When you read through this Draft Release 1 you will find that there are a number of aspects that are incomplete, because the research required to complete them is ongoing and is part of Stage 3 of the Usage Factor project. Noteworthy in this respect are:

The Organizational structure and business model to support Usage Factor

Guidelines for implementation and interpretation of Usage Factor

The Usage Factor subject classification: it has already been decided that Usage Factor will adopt an existing subject classification scheme and candidates are currently being reviewed.

Auditing requirements and tests: these can only be specified once the final version of Release 1 of the Code of Practice is in place.

Strategies for dealing with gaming of the Usage Factor: once a metric becomes widely adopted, attempts will be made to game it and this will happen with Usage Factors. In Stage 2 of the project a series of gaming scenarios were used to test the resilience of Usage Factors to gaming and more extensive tests will be run as part of Stage 3. The results of these tests will provide a basis for the guidance that COUNTER will provide on identifying and dealing with attempts to game the Usage Factor.

While all of these issues are actively being addressed in Stage 3 of the project, we feel that publication of the Draft Release 1, while not fully complete at this time provides a useful indication of the direction in which Usage Factor is going, and we want to share this with you.

The Draft Release 1 of the COUNTER Code of Practice for Usage Factors will be available for comment on the COUNTER website until 30 September 2012. Comments should be sent to Peter Shepherd, COUNTER Director, at: pshepherd@projectCounter.org

Feedback received will be reviewed by the Usage Factor International Advisory Board, together with the results of the Stage 3 tests, and will help us ensure that the COUNTER Code of Practice for Usage Factor meets the needs of the communities it is designed to serve by providing statistically robust metrics in a cost-effective way. We look forward to receiving your comments.

Jayne Marks, Wolters Kluwer Health, Co-Chair, Usage Factor

Hazel Woodward, Cranfield University, Co-Chair, Usage Factor

Draft Release 1 of the COUNTER Code of Practice for Usage Factors
Appendix A: Glossary of Terms relevant to Usage Factor
Appendix B: Specification for Collection of Usage Factor data
Appendix C: Vendor Declaration of Usage Factor Compliance

Stages 1 and 2

Full Report

Journal Usage Factor: results, recommendations and next steps (Download/View report - PDF)

Appendix A: The Journal Usage Factor: exploratory data analysis (Download/View appendix - PDF)

Stage 3

Objectives

The following objectives have been set for Stage 3:

Publication of a draft Code of Practice for the Usage Factor
Further testing of the recommended methodology for calculating Usage Factor for journals
Investigation of an appropriate, resilient subject taxonomy for the classification of journals
Exploration of the options for an infrastructure to support the sustainable implementation of Usage Factor
Investigate the feasibility of applying the Usage Factor concept to other categories of publication in addition to journals

Organization

Co-Chairs:

Jayne Marks, Wolters Kluwer, USA
Hazel Woodward, Cranfield University, UK

Members

Mayur Amin, Elsevier, UK
Kim Armstrong, CIC Center for Library Initiatives, USA
Peter Ashman, BMJ Group, UK
Terry Bucknell, University of Liverpool, UK
Ian Craig, Wiley, UK
Joanna Cross, Taylor & Francis, UK
David Hoole, Nature Publishing Group, UK
Tim Jewell, University of Washington, USA
Jack Ochs, ACS Publications, USA
Matthew Keen, IOP Publishing, UK
Clive Parry, Sage Publications, UK
Jason Price, Claremont College, USA
Ian Rowlands, CIBER, UK
Bill Russell, Emerald, UK
Ian Russell, Oxford University Press, UK
John Sack, HighWire Press, USA
David Sommer, COUNTER, UK
Harald Wirsching, Springer, Germany

Project Director

Peter Shepherd, COUNTER, UK

Go back to the top