Update on SCs and QLTS

You may remember that SCs are being used by the SRA in their newly designed Qualifying Lawyers’ Transfer Scheme (QLTS).  They’re being used in the Objective Structured Clinical Examination (OSCE), currently (and I mean currently…) being run by Kaplan, which won the contract to hold the assessment.  Jenny Rawstorne, SRA, and Dr Eileen Fry, Kaplan, presented on the subject at the Society of Legal Scholars (SLS) Conference, Open B Subject section, in a presentation entitled ‘QLTS – what have the doctors ever done for us?’  Brief summary & thoughts below the fold.

They talked about the regulatory context, and why the medical professions are useful role models.  Jenny pointed out how the Legal Services Act 2007 has affected the work of the SRA in three ways:

• Split the regulatory and representation function of the Law Society
• Prompted move towards outcomes focussed regulation at SRA
• It set out SRA’s regulatory objectives

The QLTS replaced the QLTT, and is an attempt, inter alia, to introduce a fairer, more valid and more reliable assessment of practitioner competence.  Jenny pointed out that the medics have more reliable and tested assessments that we can learn from.  The QLTS is composed of a multiple choice testing knowledge in the reserved areas (National Conference of Bar Examiners in the USA were used to help train Kaplan’s writers), followed by an OSCE that lasts three days.  Methods of standard setting have also been adopted from the medical profession’s approaches to assessment in the OSCE.  This is followed by the TLST – legal writing, research, drafting, done under time constraints.

The statistical methods by which assessment reliability and validity were being tested were explored.  eg use of the co-efficient alpha on reliability; Angoff and Borderline methods on validity. Their conclusions?

1. QLTS  now consisted of much more robust assessment methodologies
2. There was greater focus in assessments on client-facing skills
3. Research will be published by Kaplan
4. Potential impact on LPC, training contract and workbased learning
5. There were also important considerations in this assessment for the Legal Education & Training Review (LETR). 

On point 3, it's essential that Kaplan publishes the research.  I talked to Eileen and Jenny Rawstorne afterwards, and we all agreed on this -- more about that in a later posting.  Regarding point 4, if there are implications for LPC (and why not also BPTC, using SCs in different contexts?), surely there are also good reasons for using SCs in undergrad courses?  That's what Northumbria U. will be doing, starting next month -- Karen Barton and I spent 3.5 days training 16 SCs at Northumbria U Law School.  They'll be used for formative assessment in October and later in the year for summative assessment -- more information over at paulmaharg.com.  We'll be updating on the progress of the initiative here and at paulmaharg.com.  

QLTS is purely an assessment; but assessment never exists in a vacuum.  If badly designed it can take over the learning process, parasitically.  Well-designed, it meshes learning, experience, assessment and motivation.  The standardized client approach does just that because the learning zone is the assessment zone; and within and around that zone we can build an ecology for learning quite different and more extensive than conventional models of legal assessment allow for.