QTI FAQ Basics

Frequently asked questions about the basics of IMS QTI.

What is IMS QTI?
The IMS Question and Test Interoperability (QTI) specification defines a way of producing assessment information that allows questions, assessments and results to be shared across different computer systems. It is part of a suite of specifications covering a broad range of elearning requirements produced by IMS.

IMS QTI can be implemented on its own, or in conjunction with other IMS specifications. The specifications perhaps most relevant to QTI are:


 * Common Cartridge
 * Basic Learning Tools Interoperability
 * Content Packaging, which enables the description and packaging of learning materials into distributable, interoperable units;
 * Learning Design, which allows learning objects to be aggregated into courses and other kinds of learning experience;
 * Simple Sequencing, which enables the creation of more rigidly structured learning pathways;
 * Access for All, providing support for accessibility requirements;
 * Reusable Definition of Competency or Educational Objective Specification
 * ePortfolio

What does 'QTI' stand for?
‘QTI’ stands for ‘Question and Test Interoperability’.

What is 'IMS'?
‘IMS’ is the IMS Global Learning Consortium, an international non-profit organisation dedicated to the development and promotion of open technical specifications for interoperable learning technology. The initiative was originally known as the ‘Instructional Management Systems project’, but this has since been dropped and now ‘IMS’ in this context doesn’t stand for anything.

IMS was also the acronym for Information Management System(s) - now generally referred to as Information Technology (IT).

What is 'interoperability'?
Interoperability is the ability of content or systems to work together through the use of agreed standards and specifications.

For assessment technology, this means the ability to create questions or assessments in one tool, and to transfer them to and deliver them in other tools without the loss of data.

In an ideal world, systems would be able to transfer content without any human intervention. In reality, the QTI specification is still very much a work in progress. Simple items, such as multiple choice questions, can currently be exchanged very efficiently, but there is still work to be done before more complex items and assessments can be exchanged adequately.

Who defined QTI?
As with all the IMS specifications, the QTI specification has been developed through discussions, led by IMS, with developers, implementers and elearning experts around the world. Dedicated project groups, comprising IMS staff and representatives from various member organisations, work on developing and maintaining the specifications. Public drafts of new specifications or new versions of existing s pecifications are made available to the community for comment before specifications are finally approved.

CETIS participates in the work of the IMS project groups on behalf of all UK Further and Higher Education institutions. CETIS Special Interest Group members are kept informed of developments and activities in their particular sphere of interest and their views and opinions sought, and are therefore able to participate directly in the development of the IMS specifications.

What does the IMS QTI specification consist of?
The most recent version of the specification, version 2.1 public draft revision 2, consists of nine base documents plus an Addendum detailing the changes introduced between revisions of the public draft of v2.1. The nine v2.1 documents include:


 * IMS Question and Test Interoperability Overview, providing a brief introduction to the specification;
 * IMS Question & Test Interoperability Assessment Test, Section, and Item Information Model, which outlines the conceptual model underlying the specification;
 * IMS Question & Test Interoperability XML Binding, which describes how the ASI information model should be implemented in XML;
 * IMS Question & Test Interoperability Results Reporting, covering the handling of results data;
 * IMS Question & Test Interoperability Implementation Guide, which contains example implementations of the specification - it should be noted that these are illustrative and not prescriptive;
 * IMS Question & Test Interoperability Integration Guide, which describes how QTI can be used in conjunction with other specifications, particularly IMS Content Packaging, IMS Learning Design, IMS Simple Sequencing and IEEE CMI;
 * IMS Question & Test Interoperability Conformance Guide, which explains how a system's conformance with the specification may be measured;
 * IMS Question & Test Interoperability Meta-data and Usage Data, containing an application profile of IEEE LOM suitable for assessment resources in general and QTI content in particular;
 * IMS Question & Test Interoperability Migration Guide, explaining how content can be converted from QTI v1.x to the current version, and serves as an introduction to the differences between the specifications for experienced users.

How can I access the specification?
All the IMS specifications can be accessed through the IMS website's specifications page and may be freely viewed online. Downloading and implementing the specifications requires free registration, and provides a royalty-free licence to use the materials. Full details can be found in the licencing section of the IMS site.

What's the difference between a specification and a standard?
A standard is a detailed, formally ratified and fixed technology, format or method which enables the performance of a particular task or activity. A specification can be considered to be a ‘draft standard’, in that it is a step on the path towards formal standardisation, but can be incomplete and is inherently subject to change and development. Conformance to a specification allows a degree of standardisation in advance of the development and formal ratification of a standard; input in the specification development process by commercial organisations can help ensure the longevity of resources produced in conformance with these specifications.

What's the difference between an item and a question?
A question is simply the question that the student is asked to answer (the stem). A QTI item includes the question stem, the possible responses (for a constrained response question), the response processing and scoring, and feedback and hints if included.

What's the difference between compliance and conformance?
‘Compliance’ is the ability to operate in the way defined by a standard, while ‘conformance’ is the ability to operate in the way defined by a specification; in practice, however, these terms are often conflated.

It is not clear what is necessary to claim conformance with a specification: while two systems which comply with a standard are likely to have a high degree of interoperability, conformance of two systems with a specification does not necessarily mean that they will interoperate very well. This is because most specifications are far less fixed and clearly defined than most standards.

What other eassessment standards and specifications are there apart from IMS QTI?
ISO/IEC 23988:2007 - A code of practice for the use of information technology in the delivery of assessments. This is an internationalisation by ISO of the earlier British Standard 7988 of the same title. The standard details guidelines and minimum requirements for organisations using computers as part of the assessment process, and helps them to establish a good code of practice for assessment delivery.

How well defined is QTI?
There is a well-defined core of QTI suitable for creating a wide variety of questions and assessments. However, some less commonly required parts, such as detailed results reporting, are not currently so well defined.

What is the future of QTI?
A large number of commercial vendors and academic institutions have implemented at least part of the QTI v1.2 and v1.2.1 specification in their assessment systems, and a significant degree of interoperability exists between these systems as a result of this. A combination of ambiguities in the more advanced parts of the specification, and a very open approach to layout control have led to some limits on interoperability, and these issues are being addressed in version 2 of the specification.

A public draft of version 2.0 was released in June 2004, with the final version 2.0 specification released in January 2005. Version 2 includes a migration guide to support the reuse of content produced under earlier versions. The first public draft of version 2.1 was released in January 2006, and a second public draft was released in July 2006. Work on version 2.1 is continuing, with several teams using the draft specification to implement this version.

How widely accepted is QTI?
An increasing number of commercial vendors and academic institutions have begun to develop assessment systems which conform to the IMS QTI specification, and claims of QTI compliance are increasingly common in promotional and marketing material. The updating of the specification to version 2.0 addressed many of the item-level issues which may have discouraged uptake in the past; version 2.1 provides powerful test structuring capabilities.

What can IMS QTI do?
QTI is a file format for questions and assessments that allows them to be shared between different software packages. Because QTI is a standardised format, not owned by a single vendor, questions saved as QTI files can be used on many different systems. This means that questions saved in QTI format will still be usable if your college or university switches to a different QTI conformant VLE, and can be shared with colleagues at other institutions that use different systems. Most often QTI questions and assessments are designed for on-line use, and include information for computer marking, but QTI can also be used as a standard file format for printed assessments.

QTI supports a range of common question types such as multiple choice, multiple response, short answer, fill-in-the-blanks, and image hotspot. It can also be used to define longer text answer questions with marking guidelines for human markers although it has almost no support for defining computer marking rules for longer passages of text.

One particularly useful feature of QTI items is the flexibility of feedback that can be given to students, so they are especially suitable for formative assessment applications.

What are the main differences between each version of the QTI specification?

 * Version 1.0, released in June 2000, provided the basis for the specification. Consisting of just three documents – the Information Model, XML Binding and Best Practice and Implementation Guide – it outlined the basic Assessment – Section – Item data structures used to provide interoperability between assessment systems.
 * Version 1.01 (August 2000) added an additional element.
 * Version 1.1 (February 2001) was a more substantial update of v1.0 which introduced a number of new elements and attributes to the specification as well as developing the meta-data model. It also introduced the QTI Lite specification, a subset of QTI v1.1 supporting only multiple-choice questions to allow the construction of the most basic type of QTI-conformant system.
 * Version 1.2, released February 2002, saw the restructuring of the specification into nine separate documents, and a substantial number of amendments. QTI Lite v1.2 was also released at this time, bringing the Lite specification into line with v1.2; no major changes were involved.
 * Version 1.2.1 (March 2003) was released only in the form of an addendum document, detailing the editorial changes and errata corrections with which this update was concerned.
 * Version 2.0, released in January 2005, offered a significant restructuring of the item component of the specification as well as bringing QTI more closely into line with other IMS specifications.
 * Version 2.1, due for release in early 2012, extends this restructuring to the assessment and section levels.

Detailed information on the changes between each version of the specification can be found in the ‘Revision History’ for each of the documents on the IMS QTI specifications site.

What is meant if a specific system is 'QTI conformant'?
There is no hard and fast rule about what 'QTI conformant' means, however it is reasonable to expect any system that claims QTI conformance to be able to either import and render or export QTI questions depending on its designed purpose. Systems that import can be expected to support a wide range of questions with scoring and feedback provided in a way that is consistent with other systems, and software that allows the authoring of QTI conformant items should export files that conform to the rules defined in the QTI specification. There are currently few systems that support QTI assessments, but a conformant importing system should be able to import the individual questions from a QTI assessment for use in a native assessment.

Can I import questions from a QTI conformant system to any other QTI conformant system?
Simple questions, which in practice covers the majority of computer assessment items, can be transferred between any QTI conformant systems, although not all systems support every possible question type. In addition, sometimes formatting can be a problem - most systems use HTML to do text formatting, but some use RTF which the HTML based systems often can't read. Software that is used to import QTI into older systems will also only be able to use the features available in these systems, so, for example, Respondus, which can be used to import QTI into WebCT and Blackboard will not support the full flexibility of QTI marking and feedback because of the limitations of the WebCT and Blackboard quiz systems.

More complicated questions often can't be read by all systems due to different levels of support or different interpretations of the specification, but the situation is improving all the time.

Back to FAQ overview