The independent panel of experts reviewing the quality of data assembled for Census 2018 have disagreed with Statistics NZ's judgement that the information on ethnicity is of "high" quality and they say there's "significant variability in the quality of ethnicity data by ethnic group".
The Census External Data Quality Assurance panel was convened by the Government Statistician in August 2018 to look at the data assembled in the troubled and much delayed census.
Its first report on the data quality was issued on Monday in conjunction with release of the first actual census information. A second and final report will be issued in December.
The panel says data on ethnicity is a critical census output, and high-quality census data are particularly important for groups that have special status, rights or interests.
"The panel has taken a broader view of the needs of users of ethnicity data than simply the ethnicity variable itself. We rate the quality of ethnicity data as ‘moderate’, rather than Stats NZ’s rating of ethnicity as ‘high’ quality.
"There is significant variability in the quality of ethnicity data by ethnic group," the panel says.
It says this reflects different patterns of non-response, and the reliance on different alternative data sources, which have different quality characteristics.
"We find that 2018 Census ethnicity data are of high quality for European; moderate to high quality for Māori; and moderate quality for some Pacific groups."
The panel has a particular issue with the comparability of the data and says that the 2018 census "should be treated as a break in the time series, and that comparisons with ethnicity data prior to 2018 should be undertaken with extreme caution, particularly for Māori and Pacific ethnic groups."
The panel has endorsed the statistical approaches used by Stats NZ to mitigate non-response to the census survey.
However, it says the unprecedented use of administrative data to augment census data raises questions around ethics, social licence (i.e., tacit approval from the New Zealand public), cultural licence (collective mandate for the trusted use of Māori data), and Māori data sovereignty.
"While the panel has been advised of the statutory legitimacy of the record linking that has enabled the new methodology to be adopted, we remain unclear about the social and cultural licence to do so.
"There has not yet been a comprehensive and open public consultation with New Zealanders, including with the groups most affected by the use of alternative data, to gauge the acceptability of the revised census approach."
Apart from the issues over ethnicity, the panel says there are also "key limitations" on the quality of analyses on some other issues that are seen as of less critical importance.
Household and Families data "will generally be of low quality and will not enable comparisons with 2013".
The analyses of many population groups of importance to government, ethnic communities, local authorities, Māori and service providers will be affected by the lower responses to much of the census questionnaire.
"Comparative analyses with earlier censuses, or comparisons between groups defined by the categories will be of lower quality. This ranges across the information that can only be obtained from the census questions including about different ethnic groups, occupations, forms of employments status, travel origin and destination analyses, iwi membership, age group studies, activity limitations, religious group membership, languages spoken and smoker characteristics," the panel says.
Information that relates people to dwellings will be "incomplete" as just under 8% of the population cannot be placed in a specific dwelling, even though they can be located in New Zealand at small area (meshblock) level.
And the panel notes that Stats NZ will not publish iwi data as official statistics due to insufficient data quality.
"In this regard Stats NZ have not met their Treaty obligations to Māori. Our final report will consider statistical strategies that might be pursued to produce useful estimates for iwi, should iwi wish for that work to be done."
8 Comments
From reading the report, it looks like Stats NZ has done a good job of recovering the data from a very poor starting position (albeit caused by their own collection issues). The admin data approach is sound and they have been able to get good data in areas where admin data exists. The worst areas are the ones which are only answered on a Census form such as Iwi and household composition.
We welcome your comments below. If you are not already registered, please register to comment.
Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.