This session looked at various factors surrounding the personalisation of online assessment. The session will featured short presentations from domain experts on various topics to help spark and focus discussion, but the emphasis of the strand was on discussion and participation by all.
The interests of participants were wide-ranging:
Clive Church chaired the session, and opened by discussing some of the issues raised in the previous day's identity, eportfolio and personalisation session:
Dave Millard and Yvonne Howard from the University of Southampton presented their work on modelling the assessment domain in the FREMA project.
FREMA didn't look at portfolios or personalisation, rather concentrating on other factors and expressing services in the eframework in the way JISC required. FREMA illustrates the framework of services available for assessment. Personalisation isn't a scenario in itself, but rather fits into many other scenarios. Enabling personalisation involves rewriting workflows not services, or introducing new services to existing workflows.
The project developed a semantic wiki, available at http://denebola.ecs.soton.ac.uk/wiki, which is intended as a community website for developing and disseminating the model. A semantic wiki is a wiki in which all the pages and linkes are typed, which enables smart searching and analysis. The wiki enables representation of a gap analysis, service expressions for service usage models, service responsbility and collaborations and vocabulary for discussion. It offers opportunities for collaborative refactoring of the model, and a community-orientated way of working. There are many use cases around assessment which haven't yet been described: the wiki offers an excellent way of capturing and analysing this data.
Steve Lay, of the University of Cambridge and co-chair of the IMS Assessment working group which develops and maintains the IMS Question and Test Interoperability specification, discussed some of the issues related to standardised assessment.
Lack of interoperabilty is a barrier to personalisation, yet in some forms of personalised assessment, particularly adaptive testing, there is no desire to interoperate because of commercial sensitivity.
However, the QTI specification does offer opportunities for personalising assessment. Steve demonstrated some of the capabilities of QTI using Graham Smith's Java implementation which can be viewed at http://qtitools.caret.cam.ac.uk/qtiv2/examples/V2examples.html. Interestingly, while Ernest Adams had been careful to warn educators about the 'smoke and mirrors' aspects of game functionality, the Monty Hall question Steve demonstrated uses similar 'cheats' to ensure that the learning outcomes of the question are met.
There was a lot of interest in the range of tools available, and in particular in reusing existing toolkits in modular systems to support personalisation.
Steve's presentation is available from the QTITools website; you can also view his presentation to the identity, portfolio and personalisation session there.
Lou McGill, JISC programme manager with particular responsibility for assessment, outlined JISC's thinking on eassessment and discussed some of the priorities for the domain that have been identified. In a very lively session, she sought feedback from participants on the priorities and workplan that have been identified, and stressed the desire for consultation with the community to help move forwards.
Areas of interest that arose from the discussion included:
Clive Church, who works with both Edexcel and CETIS, outlined some of the issues currently important for awarding bodies.
The main priorities are currently:
CAA is seen as offering cost and time savings and easing the quality assurance process.
The 14-19 specialised diploma and 'componentised' awarding (in which various organisations are responsible for accreditation of components of a diploma which is itself awarded by a diploma awarding body) raise particular challenges. Interoperability is clearly a major priority in this area, and must be addressed urgently given the September 2008 deadline for first delivery.
The Qualifications and Credit Framework is a government-inspired log of all a learner's assessment successes throughout their lifelong learning career. These national activities will inevitably impact on the HE sector.
Kevin Donovan, of http://www.kevindonovan.co.uk, offered an entertaining and stimulating sideways look at personalisation.
Nobody really seems to know what personalisation actually is. It's perceived as adapting content and contexts to suit individuals, fitting the needs of individuals. It is seen in the context of monolithic environments and is a recurring theme across political agendas: it is regarded as being synonymous with 'choice' in contexts such as patient choice and consumer choice.
In reality there is a huge contradiction between personalisation and existing structures, particularly in the public service context. Personalisation isn't necessarily a good thing: a common currency, for example, is an asset rather than a hinderance to the 'average' person. Academic education is seen as more flexible and prestigious than training, yet the most prestigous university courses of all are vocational: law and medicine. Personalisation is just another paradox amongst many!
