Kids & family

Tertiary marking

Updated: 17 Feb 2009
Tertiary-marking-hero

Introduction

Tertiary education costs continue to rise. Is quality keeping pace?

Students heading off to tertiary study this year can be assured of one thing: they'll pay a good deal more than their parents did. But can they be assured the quality of the education they're getting is increasing too? The jury's still out.

Tertiary education reform

When Nicky completed her BA in 1980, she graduated footloose and debt-free. Daughter Lily won't be so lucky. To get the same qualification now, Lily will need a loan of at least $12,000 just to cover tuition fees.

Since the mid 1980s, tertiary education has been in an almost constant state of reform. Faced with rising student numbers, governments have devoted considerable attention to funding questions such as who pays and how. Fee hikes and student loans have been the upshot.

The quality of academic research has also been under the spotlight with the creation of the Performance-Based Research Fund and its system for rating researchers and the research quality of tertiary institutions.

Questions of teaching quality (how well subjects are taught and how effectively learning is assessed) have been given far less attention by successive governments. Yet these questions are at the very heart of matters for students.

When it comes to assessment, Dr John Hattie, a professor of education at Auckland University, believes the tertiary education community has a lot to learn from NCEA. He argues that despite "many hiccups in implementation", NCEA is much fairer and has brought about much greater transparency in what's being assessed and what students need to do to achieve.

This kind of transparency is the exception rather than the norm at tertiary level. John Hattie says: "We implicitly trust our academics to know what they value in their subjects, to set examinations and assignments, to mark reliably and validly." But evidence suggests this trust may not always be warranted.

Teaching the teachers

Lecturer

Between 1999 and 2007, tertiary student numbers rose from 306,000 to 484,000. Over the same period the number of qualifications offered increased from 3982 to 5098.

While students now have more choice about what they study, research shows that tertiary teachers may not always have the skills to teach them.

Dr Alison Viskovic, Associate Dean of Faculty of Arts and Communication at the Whitireia Community Polytechnic, did her PhD on tertiary teacher training. She found academic staff often had no formal teacher training and learnt their trade by "trial and error". Many new tertiary teachers can feel quite isolated, she says, with few opportunities to observe others teaching or of being observed by a peer and given feedback. The problem can be worse for part-time or temporary tutors, some of whom may be graduate students with little experience.

Research by Victoria University's Dr Luanna Meyer echoes these findings. She's leading a three-year project looking at assessment policy and practices in four tertiary institutions. She says academic staff are usually appointed on their knowledge and qualifications rather than their teaching expertise. Tuition in how to do the teaching side of the job doesn't necessarily follow once they start work.

Shortcomings in teacher training have also been routinely identified in audits carried out by the New Zealand Universities Academic Audit Unit. Universities do have training and development centres which offer staff courses on teaching and assessment. However, audit reports note there is often relatively low uptake of these courses.

Marking differences

Teacher training gaps often become evident in marking. Research unfailingly finds that academics vary widely in the grades they give. UK researchers and co-authors of 'Developing Effective Assessment in Higher Education', Sue Bloxham and Pete Boyd, say variations of 8 or 9 points out of 25 in essay marking are common.

Not only can grades differ significantly. Markers can also disagree on the positive and negative aspects of students' work. In one UK study, two essays written by students in their final year of an undergraduate degree were graded by 50 different markers. What one marker thought a "well-written" essay was said by another to contain "the language I would expect of someone coming out of secondary school and not a terribly good school at that".

It also needs to be remembered that markers are human. Massey University lecturers Linda Leach, Guyon Neutze and Nick Zepke argue that teachers will view students' work through "changing filters" of values, beliefs, health - and even mood. They also like their job made easier for them: one study found that essays printed in a larger 12-point typeface got significantly higher marks than those in 10 point.

Bar multi-choice tests, some degree of subjectivity may be unavoidable in tertiary assessment. But there's much that institutions can do to ensure the process is as transparent as possible. At a minimum students should receive clear descriptions of assessment criteria and marking schemes (see below).

Grading information

If you're enrolled in tertiary study this year, your institution should provide the following information on how your work will be marked.

Assessment criteria

Assessment criteria are the specific factors the assessor will take into account when marking work. For an essay, they may include the quality of argument, use of evidence, and written expression. These criteria should be readily available to you before assessment.

Grade descriptors

Grade descriptors set out the standards for different grades. They should state clearly what's required to achieve an A, B or C and what constitutes a fail grade.

Marking schemes

Marking schemes combine the assessment criteria for an assignment with the grade descriptors. Good marking schemes help make outcomes transparent to students. They also go some way to ensuring marking is consistent - across new and experienced markers - by setting out what factors must be taken into account when awarding a grade.

Even with explicit marking criteria, markers will still need to use a degree of professional judgement. But the marking scheme helps make it clear how this judgement is arrived at.

Moderation

Institutions' assessment regulations may require a proportion of assignments to be moderated or "check-marked" by another academic staff member. Check-marking won't normally involve all assignments: it may focus on a few from the top, middle, and bottom; or it may just focus on grades that are borderline (on the cusp between one grade and the next).

Provided it's underpinned by robust marking schemes, check-marking can help promote common standards among teaching staff and so can improve consistency. Your institution's regulations on check-marking should be readily available.

Quality control

A student

Some changes to quality control are in the wings. The New Zealand Qualifications Authority (NZQA) has been given the task of developing a new quality-assurance system for the tertiary sector, apart from universities. The New Zealand Vice Chancellors Committee will retain primary responsibility for quality assurance in universities.

Changes for other tertiary organisations are set to be phased in between 2009 and 2013. Just what they will deliver in quality improvements is too early to tell.

The new system relies on organisations assessing their own performance. The results of this self-assessment are then subject to external evaluation - which will look at a range of areas including staff training and assessment processes. But a major shortcoming is that there's no specific detail on what constitutes "good practice" in these areas.

Some guidance may come from Ako Aotearoa, the National Centre for Tertiary Teaching Excellence, set up in 2007 with funding from the Tertiary Education Commission. So far the centre's been establishing contestable funding pools to support projects that will improve tertiary teaching. However, the results of any large-scale research will be some way off: the first major projects to be funded were announced only in February.

What's to be done

We think the UK's Code of Practice for the Assurance of Academic Quality and Standards in Higher Education could be worth looking at. Developed by the UK Quality Assurance Agency, among other matters the code recommends that institutions publish and use clear assessment criteria and marking schemes to ensure marking is carried out fairly and consistently. It also stresses the need for tertiary institutions to train their staff in assessment.

We think New Zealand students should expect the same of tertiary providers here. Some providers already do a lot towards meeting these criteria but practice is uneven - not only between institutions but also within institutions. Something similar to the UK code could help improve standards across the sector.

Much could also be learned from Germany's system. German universities are assessed routinely against a range of indicators both quantitative (student/teacher ratios) and qualitative (students' opinion of the quality of teaching). The results are readily available online. For students like Lily - who are facing fees of $12,000 (see Tertiary education reform) - this kind of information would be a huge help when it comes to making decisions about study.

How to complain

If you've been given a mark that you think is unfair, what can you do?

  • The first step is to have a go at sorting it out informally. Discuss the mark with the lecturer or tutor concerned.

  • If you can't resolve the issue informally, check your institution's policies for having your mark reconsidered and what formal procedures you need to follow for this. The head of department (or equivalent) should also be able to provide advice on the steps to take.

  • You can also make a complaint to the Office of the Ombudsmen if you feel you've been treated unfairly by a publicly owned tertiary institution (a university, institute of technology, polytechnic, wananga, or college of education). But you should try to resolve the issue with the institution first.

  • Your complaint involves a private organisation? You can use the Quality Commission Scheme set up by the New Zealand Association of Private Education Providers. But you must first try to resolve the issue through the organisation's internal complaints process.


Our view

  • Teaching quality has been the poor relation in tertiary reforms. The proposed quality-assurance system is a step in the right direction but more work is needed on the details of what constitutes good practice.

  • Tertiary institutions can do a lot to improve teaching and assessment practices. Training for tertiary teachers needs to be a priority - being an expert on a subject is one thing; being able to teach the subject is something different altogether.

  • To make the assessment process as transparent as possible, institutions need to give students good information on how their work is graded.


More information


Report by Jessica Wilson