30th April 2012
The TEC consultation paper “Changes to the reporting framework for the PBRF 2012 Quality Evaluation” and the report prepared for the TEC by KPMG assessing the preparedness of TEOs for the 2012 Quality Evaluation highlight the extent of problems that have developed with the implementation of the PBRF model in our tertiary institutions. The TEU therefore welcomes the efforts by the TEC to change reporting practices in order to remedy the effects of the inconsistencies in HR practices identified in the KPMG report.
The PBRF model was originally developed as a bulk funding mechanism to recognise and reward research excellence amongst tertiary institutions. However during implementation many of its core principles and ideals have in our view been compromised, often with very negative consequences on achieving the original goals and on our members. Therefore whilst any changes may have flow-on effects, we believe that the extent of discrepancies in preparing and reporting by TEIs make it imperative to implement new reporting modalities. Otherwise, as the TEC notes in its paper, the results of this Quality Evaluation will lack credibility within the sector.
The two options
The TEU’s interest in assessing the options proposed by the TEC is first and foremost to consider the likely impact on our members and their employment. TEU members have reported many practices by TEIs which are, in line with the TEC’s observations in paragraph 46 of the consultation document, inconsistent with the intent and principles of the PBRF. These practices predominantly relate to staff appointments, career progression and performance management. We therefore support a reporting framework that provides the best protection against using PBRF performance as a rationale for making changes to employment conditions. Our members consistently report that it is staff whose TEIs have identified them as likely to occupy the “R” or “R(NE)” who become the target for changes to employment status, excessive management scrutiny, limiting of career progression opportunities and so forth.
For this reason we support Option 2: Excluding “R” and “R(NE)” from the calculation of the AQS. As the TEC notes, this option “serves to remove one of the most significant incentives for TEOs to manage the employment relationships of staff for the purposes of maximising their ‘ranking’ through the Quality Evaluation measure.”
The two sub-options
The TEU supports variant B– not collecting a determination of eligibility in the PBRF census file. While this option may limit the amount of information the Ministry of Education can collect in relation to the Quality Evaluation, that concern is overridden by the need to minimise the ability of TEIs to ‘game’ the system, by managing the eligibility of staff (through various dubious human resources practises).
Changing the reporting threshold
Finally, in regards to the proposal to change the reporting threshold (on the assumption that Option 2 is adopted), we would support such an approach, as it provides the best protection for members to ensure their Quality Categories cannot be inferred from this information. The TEC paper notes that this may mean loss of some of the more detailed information currently available, which is likely to impact on smaller providers. However, ensuring privacy of information for our members is a priority; we therefore support the TEC’s proposal to extend this threshold to the TEO level. It is worth remembering in this instance that the information gathered during the PBRF Quality Evaluation process is a small part of the work that institutions undertake; equally there are many other ways that useful information can be obtained across the sector.
The information gathered from TEOs by KPMG, and the TEC’s subsequent efforts to address the issues identified in the report again highlight the importance of a broader debate on the effectiveness of the PBRF as a funding model, and more specifically its impact on staff and students and the work they are engaged in – teaching, learning and research. The TEU looks forward to being part of this debate.