Jack Heinemann

When I was applying for my first academic position I was advised to ask my referees to not say that I’d be a good teacher, because that was “code” for saying I’d never be a top researcher. Whether or not many of us think the way this mentor did, there is some resemblance in this advice to how the research university works.

Your job application to a research university begins and ends with what you have done, and what you plan to do, in research. To get an interview you have to demonstrate your research worth. Even if the interview includes a separate event to showcase your teaching talents, more of your future colleagues will attend your research seminar.

Your future colleagues often think that they can extrapolate your teaching talents from how slick your lecture is. Indeed, most teaching components of an interview process are just mock lectures to hypothetical undergraduates. You have to be pretty diabolically bad for the job to be lost because of this perfomance.

Endemic in our culture is the quest for knowledge through research. That is not a bad thing! One of the characteristics of a research university that makes it different from primarily tertiary teaching institutions and from primarily research institutions is that research is a vehicle of teaching. Students ride this vehicle with mentors whose choice of research questions is (sometimes and hopefully mostly) influenced by how well they align with the capabilities of students to use them to learn to become independent researchers.

This begs the question then about what is evidence of effective teaching. The most common tool of measurement is the SET (student evaluation of teaching) survey. While this tool has value when applied well and for the right purpose—which sadly is almost never[1]—we have few alternatives. Most universities appear to be too insecure to lead the way away from this situation. However, the search is on for more effective measures of teaching effectiveness.

The advantage of survey tools is scalability. SET not only thrives on big numbers of respondents, but the cost per respondent declines as the number of students increases. Thus the cost to using the tool is predictable and minimal, regardless of whether the information is fit for purpose. Alternatives that involve careful measure of student achievement against carefully defined and possibly customised learning goals are not scalable and therefore potentially much more expensive.

Scalability is likely to guide a criterion for auditioning the next wave of SET companions. A tool I hear more often spoken about is peer review/observation of teaching. There is research that supports the efficacy of this tool, again when applied well and for the right purpose.

Notwithstanding that evidence, poor implementation of peer review of teaching could put us right back to where we are now. How will we know when or if peer review of teaching enriches the culture of learning? The tool is particularly attractive if you believe that we know good teaching when we see it. The challenge is to design a system of peer review that does something SET surveys do not do, to make visible previously invisible good teaching.

Challenges include:

• decades of academics’ defining themselves as good or bad teachers based on SET surveys, a practice that has embedded within the culture the same perceptions of good and bad teaching that students have and which a mountain of evidence has shown can be contraindicative of benefit to students. How will our peers manage, even perceive, the effects of career long SET grooming?
• assessing whether academics are better at guaging the learning happening in students by watching an academic guiding students through a learning activity. If the watcher has the same bias and ability to observe students as the instructor, and neither have access to objective and independent evaluation of changes in the minds of students, then peer review of teaching could be just correlative with SET.
• distinguishing between high acheivement in a poor teaching activity from mediocre achievement in a good teaching activity. The difference is important because improvement in the latter has more potential to improve learning outcomes than does improvement in the former. We academics still primarily use the lecture format to both share our research and demonstrate our teaching ability. Why would we suddenly see that lectures are less effective than other approaches by watching our peers give lectures, especially when some of our colleagues could be much better at giving lectures than we are?

I prefer to identify good teaching through evidence of its effectiveness rather than reference to an internal standard of goodness. One source of evidence comes from comparing peer review evaluations with the outcomes of careful measure of student achievement against carefully defined and possibly customised learning goals. Oops, back to the unscalable and therefore unaffordable standard of measurement!

Measuring poorly can possibly cause more harm than not measuring at all especially when the measurement is strongly linked to promotion. The accountability-through-metrics generation will not like to read this. However, the evidence for this statement is now too large to ignore.[2] But I can throw a bone here and will. Measuring well but infrequently might cause more good than not measuring at all.

There are no easy answers to how to measure effective teaching. In part this is because teachers and learners can have different goals, and teachers can have different goals from one another and in different courses with different learners. There is a lot to measure. This might all sound depressing but to me it isn’t. Academics have a selfish interest in being effective teachers. We are dependent on a society that is made competent through our teaching. So good teaching really does matter.

[1] A particularly blunt statement from the research literature: “…our findings indicate that depending on their institutional focus, universities and colleges may need to give appropriate weight to SET ratings when evaluating their professors. Universities and colleges focused on student learning may need to give minimal or no weight to SET ratings. In contrast, universities and colleges focused on students’ perceptions or satisfaction rather than learning may want to evaluate their faculty’s teaching using primarily or exclusively SET ratings, emphasize to their faculty members the need to obtain as high SET ratings as possible (i.e., preferably the perfect ratings)…” Source: Uttl, B.; White, C.A.; Wong Gonzalez, D. Meta-analysis of faculty’s teaching effectiveness: student evaluation of teaching ratings and student learning are not related. Studies Ed Eval 2016;54:22-42.

[2] Braga, M.; Paccagnella, M.; Pellizzari, M. Evaluating students’ evaluations of professors. Econ Ed Rev 2014;41:71-88. http://www.insidehighered.com/views/2019/06/24/relying-often-biased-student-evaluations-assess-faculty-could-lead-lawsuits-opinion#.XRCVPqdRI5g.twitter

One thing that is puzzling me about Kia Tōpū is – why? What is the strategic objective here? Why have a unifying theme project like this in the first place? And if we do have such a project then why choose food? Is the objective to be more attractive to students? Or to raise our position in the various rankings? Or to secure more external funding? Or to compete head-to-head with Lincoln and take their space?

The original business case document that was circulated has a short section titled “Strategic Case” where it says:

This section of the business case confirms the strategic context for the investment proposal and makes a compelling case for change. It takes previous options considered and develops the preferred approaches to the opportunities while considering defences to competitive threats.
New Zealand is currently exporting $30 billion in agri-food exports, however by the time they reach offshore markets, they have a retail value of $250 billion. Not only is New Zealand losing this gap in value, opportunities are also being lost to respond to and benefit from significant global food market disruption created by new ways of growing food without the need for land or animals, and distribution approaches which remove the need for intermediaries.
By 2050, the world’s population is forecast to hit 10 billion, and global agricultural production will have to grow by 70% by this time if current production and consumption patterns remain unchanged. In reality, to meet the global demand for food, increasing agricultural yields will not be enough. This population growth means that the percentage of arable land per person is decreasing; currently 3% of the Earth’s surface is arable land (of this 11% used for biofuels, 18% for food and 71% for animal feed) and 7% is for pasture. While these pressures exist, about a third of food produced is lost or wasted.
At the same time, climate change will have a drastic impact on food production. For example, it is predicted that by 2050, 40% of the world’s population will suffer from water shortages.
New Zealand has a role to play in feeding the world, and also needs to address its own issues in environmental and production challenges.
The application of new and existing technologies or business models in innovative ways to the three parts of the food supply chain – the production, processing and distribution of food including drink – can provide both opportunities and threats to New Zealand’s largest export sector.

None of this explains why UC is adopting this project. Are we simply being good global citizens?

The VC’s report to Council in September 2018 states:

Kia Tōpū is a new UC research and teaching initiative that aims to contribute to the global challenge of future food and food security. Over the next five years, UC will invest in research and teaching to help develop Kia Tōpū’s vision for the sustainable production, efficient processing and secure distribution of healthy foods across the themes of Food Equity, Food Intelligence and Food Innovation. These themes underpin the four projects that largely underpin Kia Tōpū’s programme of work: programme development, EFTS growth, the development of a research institute, and an online repository. Research and programme development continued on the realisation of Kia Tōpū delivery, with two representatives from each college on each of the main oversight groups – one related to establishing the research institute and one to develop taught programmes. This multi-year, multi-million-dollar investment in interdisciplinary research and teaching will require UC to develop and apply collaboration skills to leverage the contribution UC can make to the work of other institutions including other universities, CRIs and private sector partners.

However, this is also lacking in strategic objective reasoning.

No-one is questioning that food is an important issue (although the world has more kilojoules for more people than ever before). However, water is an important issue and so is climate change and so is energy security. So why food? What comparative advantage did we have in that area?

And no-one is questioning that universities should use their scarce resources for the greatest benefit. Is this where UC can actually make the biggest difference? There is an opportunity cost to this project – has anyone established what we will NOT now be doing? (i.e. what is the opportunity cost). The business case document is strong on benefits and very weak on costs. All projects have benefits – the question is are those benefits worth the costs.

The project may very well be worthwhile but I don’t know as I can’t see what the strategic objective is  and I can’t see any evidence of weighing costs and benefits. I think knowing those two things would be helpful.

Stephen Hickson.

Have a response?  Want to comment?  Comments are open below.

Brash stood to benefit by associating his views with Massey University’s name. The University decision to not host him could be seen as a threat to Brash’s brand. He might regain some loss to his brand by questioning the integrity of the University. Massey was of most value to Brash because its name gave him legitimacy by association; now it is of most use to him if its name is worth less than his own. This controversy creates a danger to academic freedom but only if staff and students of the University unwittingly contribute to an agenda that undermines institutional autonomy.

As I write this, Massey University’s Vice Chancellor Dr. Jan Thomas is in the news because of evidence that she lied and sought to manipulate students hosting a political event featuring Dr. Don Brash. If it is true that she tried to mislead and censor, then the matter is rightly subject to the review of the University’s Council. Her ability to perform her duties without the confidence of staff and students¹ must be central to the Council’s consideration of the case.

However, I fear that her alleged actions are meanwhile being misunderstood as tacit support for Brash’s outrage at not being hosted on a Massey University campus. In my view, the actions of the University were in no way a reasonable impediment to his right of free speech or the silencing of the academic freedom to which students’ are entitled.
Massey University has no obligation to provide infrastructure for Brash to espouse his views. That infrastructure is expensive. Not only are buildings and real estate costly, but the obligations of public institutions to ensure the safety and well-being of those on campus is also expensive. It would be reasonable for Thomas to take that into consideration.

Brash and his supporters have more than the financial and social means to ensure the expression of their views; Massey is not obligated to be the venue (for him). Indeed, if there is an ethical obligation upon Massey University to be a venue for free speech, it is to use its limited financial resources to provide a venue for the airing of views from those less able than Brash to access New Zealand’s ears.

Separate from the issue of free speech, was the cancelling of his speech an attack on the academic freedom of the students who had invited him? In denying Brash use of Massey facilities, did Thomas use autonomy (the institutional form of academic freedom) “as a pretext to limit the rights of”² students?

There would be no question that she did if students wishing to express their views on the same topics as Brash were censored, provided that they are acting within the law. But students don’t have the absolute right to use Brash as a surrogate form of expression. It is expected that students are capable of accurately presenting his views and other students are capable of responding to them.

If you view academic freedom and institutional autonomy as rights, then they are elitist and privileged. Perhaps that is what Parliament intended when it uniquely conferred them upon some tertiary institutions. I think instead that Parliament expected our universities through their staff and students to provide a service using these academic freedoms as one of the essential tools provided to the sector. That service requires we staff and students to use our scholarship to identify and responsibly express unpopular and controversial opinions for the benefit of society, pursue teaching and research of high quality unfettered by the interests of those who may be affected by it, and in doing so to associate ourselves with our University’s name. It does not mean dispatching the hard and risky work to someone else on your campus, avoiding the personal and professional costs of the scholar.

Thomas’ decision is consistent with the centrally important role of autonomy to ensure that academic freedom for some does not come at the expense of academic freedom for others. The role of institutional autonomy is to protect the academic freedom of all within the academic community, and from threats outside of it. Using Brash as a surrogate for the presentation of ideas pits staff and students with contrary views against him, a person who does not have to play by the same rules of scholarship and debate and obligation to respect cultures. This is very different to a contest of ideas among equals in the academic community. It would be reasonable for Thomas to avoid putting staff and students in this position.

We must be careful, staff and students of universities, in how we participate in the debate over Thomas’ actions. She has some explaining to do, especially the alleged threat to student body funding. Indeed, hold her to account for any breach of trust or issue of integrity.

But this is a discussion separate from the academic decisions made by Thomas. It is important to keep her alleged failings as chief executive away from any perception that Brash was wronged. Academic freedom and institutional autonomy includes the right to not promote already privileged views in society. Our public universities do not need to provide free advertising. Our collective responsibility to autonomy is, in the words of the United Nations Educational, Scientific and Cultural Organisation, to “contribute to the public accountability of higher education institutions without…forfeiting the degree of institutional autonomy necessary for their work, for their professional freedom and for the advancement of knowledge.” ² In other words, our exercise of academic freedom should not threaten academic freedom itself.

Jack Heinemann

1 https://www.radionz.co.nz/news/national/366834/don-brash-cancellation-censure-motions-against-vice-chancellor

2 “Recommendation Concerning the Status of Higher Education Teaching Personnel” UNESCO 1997. http://portal.unesco.org/en/ev.php-URL_ID=13144&URL_DO=DO_TOPIC&URL_SECTION=201.html