TEF: SLS Subject Level Consultation Response May 2018
31st May 2018
Launch date 12 March 2018 Respond by 21 May 2018
SLS Response
Founded in 1909, and with around 3,000 members, The Society of Legal Scholars is the oldest as well as the largest learned society in the field. The great majority of members of the Society are legal academics in Universities, although members of the senior judiciary and members of the legal professions also participate regularly in its work. The Society’s membership is drawn from all jurisdictions in the British Isles and also includes some affiliated members typically working in other common law systems. The Society is the principal representative body for legal academics in the UK as well as one of the larger learned societies in arts, humanities and social science.
Summary of consultation questions
A contextual point is that the Society, while fully supportive of arrangements that will help ensure high quality in HE in England and Wales while respecting the diversity of approaches to legal education of different law schools, is at present generally sceptical of the value of this particular exercise. The likelihood is that this kind of metrics-driven exercise will lead to behaviour by HEIs that responds to the metrics rather than proper concerns for quality, while at the same time providing little really useful information for students. The pilots may produce evidence that provides reassurance or corroborates the scepticism.
The questions
1 To define ‘subjects’ in subject-level TEF, do you:
a) agree with using level 2 of the Common Aggregation Hierarchy as the classification system (CAH2, with 35 subjects), and if not, what other systems could be used and why?
b) think that specific changes or tweaks need to be made to the definition of the 35 subjects in CAH2, or to the 7 subject groups used in Model B, and if so,
please explain why?
The proposed definitions of “Subjects” are acceptable in that Law is properly recognised as a separate subject. The inclusion of Law with Business in a “Subject Group” is only acceptable on the understanding that there will be (or be the option of) entirely separate submissions for Business and Law.
2 Do you agree that we should have a longer duration and re-application period in subject-level TEF?
Yes. In terms of the possible models set out on p.12 of the Consultation Document, it seems arguable that assessment 2-yearly with a maximum duration of 6 years is appropriate. However, we would suggest the possibility of reapplying after 3 years rather than 4. 4 years is a very long time to be fixed with a label where the next higher rating was very narrowly missed.
3 Should subject-level TEF retain the existing key elements of the provider-level framework (including the 10 TEF criteria, the same suite of metrics, benchmarking, submissions, an independent panel assessment process and the rating system)?
Yes,
4 For the design of subject-level TEF, should the Government adopt:
- A ‘by exception’ approach (i.e. a form of Model A), or
- A ‘bottom up’ approach (i.e. a form of Model B), or
- An alternative approach (please specify)?
Model A should be adopted. Model B involves significantly extra work for an exercise of (we suspect) limited value.
5 Under Model A, do you agree with the proposed approach for identifying subjects
that will be assessed, which would constitute:
- a) the initial hypothesis rule for generating exceptions from the metrics?
- b) allowing providers to select a small number of additional subjects?
Yes.
6 In Model A, should the subject ratings influence the provider rating?
No. The stated disadvantages (CD p.18) are compelling. It is not clear how far students will or be wise to pay attention to provider ratings if subject ratings are available.
7 In Model B, do you agree with the method for how the subject ratings inform the provider-level rating?
Yes.
8 Do you agree that grade inflation should only apply in the provider-level metrics?
Yes, if at all. We are not yet persuaded that a methodology can be devised under which it can be shown that grade inflation in a particular subject or institution is inappropriate.
9 What are your views on how we are approaching potential differences in the distribution of subject ratings?
We support the proposal for the distribution of subject ratings to vary naturally for each subject rather than forcing a uniform distribution, for the reasons given in the CD.
10 To address the issue of non-reportable metrics:
a) do you agree with the proposed approach?
Yes.
b) when assessment occurs, do you prefer that assessors:
- rely on group metrics alongside any reportable subject-level metrics?
- rely on provider metrics alongside any reportable subject-level metrics?
Both need to be used with caution. For example, the metrics for a Business School may not be particularly relevant to a Law School, except, perhaps, in those cases where the Law School is part of the Business School.
- follow an alternative approach (please specify)?
11 Do you:
a) agree that QAA Subject Benchmark Statements and PSRB accreditation or recognition should remain as a voluntary declaration, and if not, why?
Yes.
b) think that there are any subjects where mandatory declaration should apply?
Mandatory declaration should apply, as suggested in the CD, where “accreditation forms an essential element of a student’s progression to employment”.
12 Do you agree with our approach to capturing interdisciplinary provision (in particular, joint and multi-subject combined courses)?
Yes.
13 On balance, are you in favour of introducing a measure of teaching intensity in the TEF, and what might be the positive impacts or unintended consequences of implementing a measure of teaching intensity?
14 What forms of contact and learning (e.g. lectures, seminars, work based learning) should and should not be included in a measure of teaching intensity?
All forms should be included. If this cannot be done there should not be a teaching intensity metric. Students should simply be informed of typical/standard class sizes for a particular course.
15 What method(s)/option(s) do you think are best to measure teaching intensity?
Please state if there are any options that you strongly oppose and suggest any alternative options.
See above.
16 Do you have any other comments on the design of subject-level TEF that are not captured in your response to the preceding questions in this consultation?
14 5 18