TEU submission on "evaluating applied and practice-based research"

Posted By TEU on Jan 30, 2009 |

 Submission of the Tertiary Education Union (TEU) Te HautÅ« Kahurangi o Aotearoa

 on the PBRF Sector Reference Group consultation paper

 â€œEvaluating applied and practice-based research”

 30 January 2009

For further information please contact:

Jo Scott
Policy Analyst
Tertiary Education Union
Te Hautū Kahurangi o Aotearoa

Evaluating Applied and Practice-Based Research

The New Zealand Tertiary Education Union (TEU) Te HautÅ« Kahurangi o Aotearoa is the largest tertiary sector union in this country.  Our membership currently sits at approximately 11,000 members, covering all types of TEOs in the sector.

Prior to the formation of the TEU on 1st January 2009, both AUS and ASTE have made submissions on a range of topics related to the PBRF.  When the PBRF was introduced, we raised a number of concerns about the model, many of which have unfortunately emerged as issues in the sector.  Our stance on the PBRF model some years on from its introduction is that minimal changes should be implemented as a result of the current review.  We have taken this position with the current funding model as we believe that post-2012 the sector should vigorously debate alternative models that could replace the PBRF, which we regard as fundamentally flawed.

For this reason the TEU holds the view that further changes to the model, other than minor adjustments (such as clarifying guidelines) would put unnecessary pressure on academic and general staff involved in research and on TEOs, and would risk further undermining the data obtained so far from assessment rounds, and should therefore be rejected.

Issues – evaluating applied and practice-based research

The concerns raised in this consultation paper about how research in applied and practice-related fields should be evaluated have been voiced by many TEU members, and are frequently raised in discussions on the PBRF.  Further debate and analysis across the sector would in our view be useful, to either allay or substantiate these concerns.  Such a discussion could take place outside of the PBRF process, as a further contribution to debate about research assessment post-2012.

For the purposes of this current review however, the TEU believes that as little disruption to the system (flawed as it is) is the most pragmatic approach at this stage.  If minor adjustments can be made to the processes for evaluating research, which might assist in addressing concerns expressed in the sector about the assessment of applied and practice-based research, then our view is that it would be appropriate to do so.  Anything more than this we believe would be unnecessarily disruptive at this stage of the PBRF model.

Development of research capacity

Maintaining a balance between research and scholarly ambitions and obligations of academic staff, on one hand, and their commitments to students, community and industry, on the other, has always been a difficult task, especially with increasing teaching workloads and staff/student ratios.  The PBRF model has tended to highlight this tension, as the demands brought about by the funding/evaluation model place extra pressure on eligible staff to produce research to a particular level, often without being balanced by a reduction in teaching workloads.

The tension is particularly apparent in those parts of the sector which have more recently been included as ‘research active’ institutions or disciplines (because of the nature/level of programmes they offer), such as teacher education and many of the higher-level programmes now offered in the ITP sector.  These disciplines (which have an applied and practice-based focus to their teaching) have faced significant structural challenges in meeting the requirements of the PBRF quality assessment and funding system, especially that of balancing higher levels of teaching commitments (commonly associated with practice-based fields) with research activity.

These discipline areas generally have not yet developed a significant research culture such as that commonly found in more empirically-based disciplines – rather their focus has been on teaching and other aspects of professional practice.  Within the PBRF system therefore, they will continue to be at a disadvantage for some time yet because of the emergent nature of their research cultures.  Ensuring that research is supported within these discipline areas is crucial if we are to understand and address the many complex social issues we face, as much of their research seeks to address these very concerns.

As the overall lifting of research quality is a central tenet of the PBRF system, the TEU would therefore be supportive of moves by the TEC to further analyse research capacity in the sector with a view to developing strategies (including addressing workload allocation of eligible staff in the newly merged colleges of education and ITP sector) to assist research development in areas identified as needing additional support.

Clarification of panel-specific guidelines

In terms of the specific proposals in the consultation paper, the TEU supports clarifying panel-specific guidelines to ensure that the distinction between research and professional practice activities is clear to TEOs and individuals.  Academic staff undertake a wide range of activities in the course of their jobs, which are not always easily distinguishable as either ‘research or ‘professional practice’ functions.  It is vital that the sector, TEC, management and individual staff have a clear sense of this distinction, and are able to focus energy and resources appropriately.

External representation on assessment panels

Within the current model, the TEU is ambivalent about including members from outside of the sector on assessment panels.  The purpose of the PBRF is to conduct a peer-assessment of research quality, and senior members of the academic community are presumably competent to do this.  There may however be value in further integrating community/industry feedback as part of the research assessment process (such as in evidence portfolios), to highlight the impact the research has had outside of academia.  Such an approach is relevant to all research methodologies, and if adjustments to guidelines for EPs could be made with minimal disruption, then giving panel members greater breadth of information about research impact would in our view be a useful enhancement.

Evaluating peer esteem and research impact

If there is evidence of inconsistency in panel assessment of peer esteem and wider research impact (communities, industry etc), as discussed on page 12 of the consultation paper, then the TEU would support some review of this area, using already-identified exemplars/models developed by other panels to minimise duplication of effort.  As well the TEU would support panels being requested to ask for sector-specific feedback on guidelines for recognising research impact and peer esteem, to ensure that these are relevant to the particular discipline (i.e. considering sources other than citations or other commonly used measures, if this is regarded as appropriate by the discipline).

Diversity of representation on panels

If there is evidence that indicates a lack of representation on panels by researchers with expertise in applied/practice-based research models, then the TEU would be supportive of moves to ensure better representation.  A fundamental tenet of the PBRF system is that of comprehensiveness, which must apply also to panel representation.

We would view the above proposals as being less disruptive options that could assist in allaying some of the concerns expressed about the assessment of applied and practice-based research, without significantly impacting on the current system.


The TEU supports minimal changes to the current system in the area of applied and practice-based research, in line with our position on completely reviewing the assessment and funding system post-2012.

Print Friendly