Tracking OERs: Technical Approaches to Usage Monitoring for UKOER

Introduction
This set of pages is about technical approaches to monitoring
 * the release of OERs by a project or programme,
 * the use and re-use of these resources, and
 * comments and ratings relating to these resources.

These issues are clearly related to evaluating processes of releasing OERs and their impact, i.e. the extent to which they meet some of the objectives intended for releasing OERs. The technical approaches discussed here can provide lots of numbers but on their own they don't necessarily mean very much. These quantitative measures can be triangulated with data from other methods, such as interviews, questionnaires, focus groups, observations, which provide more information on what the numbers actually mean in terms of the questions that an OER provider may want answered. It is however crucial to be clear what these questions are before trying to decide which of the tracking and other evaluation methods will be used. The actual choice of technical and non-technical measures will depend on the objectives of the individual OER provider or funder. For the UKOER programme there are some requirements for reporting the total number of OERs released, evidence of their use and reuse, evidence of benefit and impact (if available) and indicators of quality such as value to the community and pedagogic value. There is absolutely no mandatory method for how this is achieved by UKOER projects, though there is a strong steer that compelling case studies will be more useful than raw numbers. More information on evaluating OERs is available from the UKOER synthesis evaluation project. See also OLNet Tracking OERs -- requirements and suggested solutions, by Nick Freear.

Methods of Tracking OERs
Bearing in mind that on their own many of these methods represent little more than bean counting, and they only have value if you can give a rationale for counting those particular beans; also bearing in mind that, as with any form of web statistics, the counting is often imprecise and useful for indicating trends but not absolute numbers, the following technical methods may be used for (i) monitoring the number of OERs released, (ii) monitoring their use and reuse, and (iii) monitoring comments about those resources.

Monitoring the number of OERs

 * 1)   Counting resources through host / dissemination services

Monitoring use and reuse

 * 1)   Web usage stats
 * 2)   Google and other online Analytics
 * 3)   URL redirects
 * 4)   Web bugs
 * 5)   Web search

Monitoring comments

 * 1)    Comment aggregation

These pages are based largely on information presented and discussed during the November 2009 "2nd Tuesday" online webinar for the UKOER programme, a recording of which is available (requires Java), as are some of the powerpoint slides used. Many thanks to the other speakers and participants who contributed to that session.

Case studies and examples
As part of the second UKOER programme JISC and the Academy have funded a number of the completed projects from UKOER phase 1 to report back on the results of their efforts to track the use of the OERs they have released. As part of CETIS' support for the programme we will synthesis these reports to provide guidance about tracking OERs.

The guidance will continue to be informed by and contribute to best practice from other initiatives globally.

These include, Scott Leslie's recent research sabbatical at the Open University and his subsequent trial implementation of web bugs with the BCCampus repository OER content.
 * OLNet Fellows podcast

Contact Us
John Robertson

Phil Barker