Supplementary submission on reporting PBRF results

Posted By TEU on Nov 13, 2009 |

Submission of the Tertiary Education Union (TEU) Te HautūKahurangi o Aotearoa

to the Tertiary Education Commission on the supplementary consultation document “Reporting of results”

13 November 2009

For further information please contact:

Jo Scott

Policy Analyst

Tertiary Education Union

Te HautūKahurangi o Aotearoa

Reporting of results (supplementary consultation)


The New Zealand Tertiary Education Union (TEU) Te HautūKahurangi o Aotearoa is the largest tertiary sector union in this country.”  Our membership currently sits at approximately 11,500 members, covering all types of TEOs in the sector.

Prior to the formation of the TEU on 1st January 2009, both AUS and ASTE made submissions on a range of topics related to the PBRF.”  When the PBRF was introduced, we raised a number of concerns about the model, many of which have unfortunately emerged as issues in the sector.”  Our stance on the PBRF model some years on from its introduction is that minimal changes should be implemented as a result of the current review.”  We have taken this position with the current funding model as we believe that leading up to and post-2012 the sector should vigorously debate alternative models that could replace the PBRF, which we regard as fundamentally flawed.

For this reason the TEU holds the view that further changes to the model, other than minor adjustments (such as clarifying guidelines) would put unnecessary pressure on academic and general staff involved in research and on TEOs, would risk further undermining the data obtained so far from assessment rounds, and should therefore be rejected.

TEU response to alternative proposals for reporting of results

As we have noted in previous submissions, TEU does not support the present PBRF model of allocating funding and evaluating research quality, although we do support adequate funding that recognises TEOs research activities.”  In regards to this funding model, we are especially concerned about the use of the individual as unit of assessment, as we have stated in previous submissions.

This supplementary consultation asks respondents to consider an alternative proposal for addressing the issue of reporting of results.”  However because of our ongoing concern about the use of the individual as the unit of assessment, it is unlikely that the union could endorse a continued practice of producing individual results, let alone one that allows disclosure of these results by TEC to employers.

Data verification and internal funding allocations

The TEU understands that the view of institution managers is that individual results are required for two purposes: data verification and internal funding allocations. ” We reject this argument – as we have previously stated, both these requirements can be achieved without the release of individual Quality Categories and Component Scores to employers.”  Verification for the purposes of determining whether all eligible staff have been assessed within the correct discipline groups and by the appropriate panels, and checks for any obvious errors in results, can be undertaken by means that do not require delivery of individuals’ results to employers.

Additionally, Average Quality Score data on academic units allow internal funding allocations to be performed with a reasonable degree of accuracy to a level that then allows heads of department to make the necessary decisions to support improvements in research outputs. ” The use of individual results (other than in the aggregated form of measures such as AQS) for verification and internal budget allocation bear no direct relation to achievement of the aims of the PBRF.”  Neither do they contribute to achievement of the stated purposes of the PBRF, nor its successful implementation. ” At best, data verification and internal allocation bear an indirect ‘downstream’ relationship to the purposes and aims of the PBRF assessment and bulk funding.

Allowing this practice (of disclosure of individual results) to continue simply for the convenience of institutions is unnecessary and uncalled-for, especially when the practice has led to direct, negative impacts on our members’ employment and career progression.”  In our view, both the TEC and TEOs will be able satisfactorily to achieve the purposes for which the PBRF was designed without the disclosure of individuals’ results. ” If TEO managers claim that they cannot manage and improve research quality without individual PBRF data, then that claim only brings into question their competence as academic leaders.

Confidentiality of information

Recently the TEU has become aware of a number of situations where misuse of results has potentially allowed individuals to be identified.”  The risks to our members’ that are inherent in TEOs holding this information, even if this is limited to specific individuals (as suggested in the supplementary consultation document) are in our view too great for the TEU to support disclosure of results to institutions.

The TEU is also concerned that if a TEO has possession of individual results for the purposes of internal fund allocations, and then provides graded levels of PBRF-derived funding to individuals, that this could be taken as information that corresponds to each staff member’s Quality Category. ” Any such dollar-equivalent proxy for the Quality Category could then be interpreted (rightly or wrongly) by heads of department and deans. ” That would effectively undermine the purposes of the proposed compromise to keep information about individual results to “the minimum number of staff necessary”. ” Such an arrangement would render ineffective the privacy and confidentiality protections that were intended by the alternative proposal outlined in the consultation document.

In the ‘Draft Conditions under which individual PBRF data is provided to TEOs’ there is no real incentive for TEOs to respect confidentiality. ” Given the behaviour by some TEOs that has recently been reported to the TEU, we are concerned that the Draft Conditions, if implemented, would be ‘honoured more in the breach than the observance’. ” Furthermore, if any breach of confidentiality is detected by a member of TEU, then neither that member nor the TEU has any effective remedy for that breach, other than to complain about it. ” The ‘victim’ has to police and prosecute his or her own case, and the TEO suffers no penalty if found to be at fault.

The TEU does not relish the prospect of having to support members in complaints against their own employers arising from breaches of the Draft Conditions; and we predict that such breaches will be common, even if difficult to detect.

Privacy Act

We understand that the TEC purports to have an opinion that there might be “reasonable grounds” to disclose individual results to TEOs based on Principle 11 of the Privacy Act. That is, it is claimed that “the disclosure of the information … is directly related to the purposes in connection with which the information was obtained.” ” If this is the case, this opinion asks us to accept that data verification and internal budgeting are directly related to the purposes with which PBRF results were obtained.

We disagree with this position because we believe that it is based upon a misrepresentation of the processes and aims of the PBRF. ” Individual PBRF results are produced by panels for the purposes of calculating the aggregate quality of TEOs’ research outputs.”  These aggregated results are then used to allocate funding to TEOs via a government bulk fund.”  Using these results for internal institutional purposes in their disaggregated form and without the explicit consent of individuals therefore exceeds the stated aims and purposes of the PBRF.”  Clearly therefore using individual results for data verification and internal distributions of funds goes beyond being “directly related to the purposes in connection with which the information was obtained.”

Therefore TEU considers the proposed disclosure of personal information – without individuals’ informed consent – to be unethical and in breach of the Privacy Act.

Academic freedom

The freedom of individual academic staff to engage in research has a clear statutory basis in S. 161 of the Education Act 1989. ” The intent of this section is binding upon the TEC and institutions (subsection 4). ” It requires all parties concerned to observe ‘the highest ethical standards.’ ” As well it clearly describes research in the context of TEOs as a freedom enjoyed by staff (and students). ” Coercive actions taken by employers in implementing the PBRF (especially in the form of performance management processes and actual or threatened disciplinary actions) are contrary to the intent of s. 161. ” Such inappropriate actions by employers are a direct outcome of the competitive and intrusive nature of the PBRF.”  The TEC needs to carefully reconsider what appropriate changes can be made to this system for the next round, if it is to retain the confidence of the sector and comply with the Education Act.

Impact on employment relationships

While individuals’ participation in the PBRF, by way of completion of the evidence portfolio, may be a reasonable requirement of employment, the disclosure of personal information arising from EPs by the TEC to the employer is not an obligation of employment.”  To treat the disclosure of personal PBRF results as a non-consensual obligation may take the issue into difficult employment relations terrain; ceasing this practice will avoid such difficulties.

The TEU’s preferred option

From the present consultation paper, the TEU sees the following option as the only legally and ethically sound one:

II) Only individuals receive Quality Categories

No individual researcher’s results would be provided to TEOs by the TEC. ” TEOs would only receive the summary information providing in the “Evaluating Research Excellence” publication. ” Individual researchers would be able to request their results directly from the TEC and could then advise their TEO if they wish.

In short, each individual researcher should retain the right to control what happens to personal information that TEC may have collected about him or her. ” The TEU cannot see any ethically or legally sustainable alternative to this position.

Print Friendly