Here's 10 things you need to know ahead of the NZ election

By Nicole Satherley, Andrew Sporle and Lara Greaves*

The closer we come to election day on October 14, the more media focus we’ll see on political polls. Poll results are often used to project the makeup of parliament, despite them being snapshots of the polling period rather than reliable predictions.

Indeed, the accuracy of past political polls in New Zealand has been open to question. And the way results are framed can sometimes cause more confusion than provide useful context.

No poll is perfect. But understanding the quality of a poll and the results it produces requires knowing something about how the poll was designed and carried out.

We recently completed a guide to understanding public opinion polling in New Zealand that describes the important features of polls to look out for. These factors determine their quality and should be considered when making conclusions about the results.

Technical details (sample size, margin of error and so on) about a given poll are usually available. So what information is important to consider? Here are ten things to think about when evaluating a political poll.

Another TV political poll showing support for left/right blocs within the margin of error - but this one prompted some pretty creative interpretations from political reporters and pundits https://t.co/FZsSTXTSEf

— Mediawatch (@MediawatchNZ) July 22, 2023

1. Sample size

Contrary to common misconceptions, good quality results about the New Zealand population can be obtained from polls with as few as 500 to 1,000 participants, if the poll is designed and conducted well.

Bigger samples lead to less random variation in the results – that is, the differences between the results from a sample of the population compared with the whole population. But bigger samples are more expensive to collect, and they don’t make up for poor sampling design or polling process.

2. Target population

It should be clear which group of people the results are about. Results about sub-groups (for example, women in a certain age group) should be treated more cautiously, as these are associated with smaller samples and therefore greater error.

3. Sampling method

Sampling design is crucial. It determines how well the poll sample matches the target population (such as people intending to vote). Polls should be conducted with an element of choosing people at random (random sampling), as this achieves the best representation of the population.

Polls that allow for self-selection and that do not control who can participate – such as straw polls on media and social media sites – will end up over-representing some groups and under-representing others. This leads to biased, inaccurate results.

4. Sample weighting

When characteristics about the population and the sample are known – such as the percentage of women, age or region – “weighting” increases the contribution of responses from groups under-represented in the sample to better match the population of interest.

This is achieved by making responses from under-represented respondents count more towards the results of the total poll.

Weighting cannot be used, however, to correct unknown differences between the poll sample and the total population. The distribution of population characteristics, like gender and age, are known through the census, and can be adjusted in the sample with weighting.

But we don’t have known population characteristics for other things that may affect the results (such as level of interest in politics). Good sampling design, including elements of random sampling, are the best way to ensure these important but unknown characteristics in the poll sample are similar to the whole population.

New Zealand poll shows Labour and National in tightening contest three months out from election https://t.co/GuqQUktUEA

— Guardian World (@guardianworld) July 19, 2023

5. Poll commissioner and agency

Knowing who paid for the poll is useful, as there may be vested interests at play. Results could be released selectively (for example, just those favourable to the commissioning organisation). Or there may be a hidden agenda, such as timing a poll around particular events.

Equally, we can be more confident in poll results when the polling agency has a strong track record of good practice, particularly if they follow national and international codes of best practice.

6. Poll timing

Knowing when the poll was conducted, and what was happening at the time, is important. Poll results describe public opinion at the time the poll was conducted. They aren’t a prediction of the election outcome.