QTI Profiling WG 28 January 2010

JISC CETIS QTI Profiling Working Group meeting, 28 January 2010, Aston Business School Conference Centre, Birmingham

Purpose of the meeting

IMS QTI v2. is ready, in public draft and with a good number of implementations, with the recent IMS survey receiving a good number of responses that should help to construct a clearer picture of the way in which the specification is being used.

This CETIS working group will support the work necessary for the specification to be finally released, including:


 * development of profile(s) developed by identifying the intersection or areas of overlap between implementations;
 * demonstrating interoperability between profiles;
 * proposing solutions to issues with the current draft.

Profile strategy

This group is particularly focused on using the specification for mathematics, but there is interest from other subject areas such as chemistry and engineering. Ideally, the gorup would like to see the specification being used in non-scientific subject areas such as modern languages, including those using alphabets other than the Latin alphabet. This group's work on mathematics would also provide a useful model of the process for developing domain-specific profiles.

L'Université Pierre et Marie Curie (UPMC) have done signficant work on developing a profile for basic algebra, which handles arithmetic and randomisation very nicely but doesn't do enough with maths expressions, simplifications, etc. Their system was developed for the French schools sector but does provide a very sound basis for part of this work, particularly as there is a lot of basic maths being taught in universities, either remedially or to support 'soft' subject such as teaching and nursing.

It was agreed that a modular approach to profiling would be most appropriate, with a three part model:


 * 'Base' QTI profile developed and published by IMS;
 * General arithmetic profile as developed by UPMC;
 * UK maths profile developed by this group.

Localisation and internationalisation are potentially signficiant, with even basic issues (such as the use of commas or full stops to indicate decimals) having the potential to cause problems. Some required functionality, such as the use of diagrams and graphs, is out of scope for the current work.

Exisiting profiles

A number of existing profiles can be drawn upon for developing this UK maths profile, including QTIEngine, JAssess, APIS, Fetlar, SToMP. The first step is to write up what the intersection between implementions is. Shared non-maths-specific findings should be fed into IMS.

Profiling tool demonstration

Niall Barr of the University of Glasgow demonstrated this tool which can be used to identify the most basic QTI profile based on what parts of QTI are currently being used. The tool can scan either a single item or a folder of items to provide a table indicating what elements are used by those items and which aren't. Niall will scan items submitted to him and create a first indication of what elements have been implemented. There was some concern about the danger of removing potentially useful elements from the specification just because the sample questions don't use them; this will be addressed in future meetings.

Demonstration of Spectatus, the Fetlar virtual appliance

Paul Neve of Kingston University demonstrated this system, which integrates QTIEngine, Minibix and Moodle and provides links to Constructr, MathQuRate and STACK. There's an instance of everything within the virtual appliance, but it can be used with an institutional implementation of Moodle or CARET's Minibix instead.

Issues raised

Request for a maths content variable. MathsVariable is an attribute which just lets you express it inside MathML; we need to be able to express 'a chunk of maths' as a base type. RecordVariable can be used with fields for various elements, but this is implementation-dependent. Custom operators need to be translated, for example between Maxima and Maple, using attributes. However, this is not mature enough or defined enough yet for inclusion in the core specification.

Some glitches with the example packages were noted.

Identifiers in tests differ between the manifest (GUID) and the actual item XML (generally something non-unique like item04). This is probably because Reload was used to generate the packages and followed correct practice for content packaging. There was a lot of debate about this, with the group quite divided on whether the identifiers should be the same in both. it was tentatively agreed that this is a best practice issue rather than part of the specification.

The IMS specification requires the use of IMS Content Packing: should the UK maths profile do so also?

Should there be a reusable CSS file as an adjunct to the specification, as most people work from examples?

Need to establish best practice for issues such as the use of class names for interactions when rendered and in the use of MathML.

Next steps

The group agreed to a follow up online meeting at the beginning of March to look at the first version of the profile produced by analysis of the items submitted to Niall. Wilbert will take the issues raisded by the group to the IMS Quarterly in Orlando and state the group's preference for a modular approach to profiling.