As is normal, interest.co.nz kindly reproduced the last Raving in which I aired my concern about the quality of some official statistics.
Statistics NZ (SNZ) provided a thoughtful response to interest.co.nz (appended) and a senior manager from SNZ contacted me for a friendly discussion that I appreciated.
However, the discussion with the SNZ official left me concerned about judgements made at SNZ.
The review of the relationship between reported GDP and employment growth in this Raving reinforces my suspicions about the accuracy of especially the labour market data but possibly also the GDP data.
It is possible the source or sources of inaccuracy are outside the control of SNZ while a budget constraint may be part of the problem. Preparing official statistics in a small country with budget constraints isn't an easy task. But whatever the cause(s) I recommend that great caution is used in interpreting reported employment growth.
This casts a shadow over the accuracy of the reported unemployment rate while I am somewhat wary of the accuracy of reported GDP growth and I am inclined to conclude that something needs to be done to improve the quality of decision making at SNZ.
What is wrong with the picture painted by this chart?
Allowing for sampling errors that create random variation in the reported numbers there used to be a reasonably sensible relationship between the reported GDP and employment growth prior to 2012 as shown in the chart. The highest correlation between the two at 0.76 - akin to a 76% mark in an exam - was with GDP growth leading employment growth by one quarter. Reflecting this, the red GDP growth line in the chart has been advanced or shifted to the right by one quarter. This fits with what I believe should be expected (i.e. stronger economic growth being followed roughly one quarter later by higher employment growth and weaker GDP growth being followed roughly one quarter later by lower employment growth).
However, since 2012 the peak correlation, still with GDP growth leading employment growth by one quarter, is only 0.46 that can be roughly interpreted as a 46% mark in an exam (i.e. a fail mark). The green boxed area in the chart highlights the period since 2012 when reported employment growth has at times been radically out of line with what should have been expected given GDP growth. In my opinion this raises the most doubt over the accuracy of reported employment growth but my gut feeling, aided by my analysis of the key drivers of economic growth that is contained in our monthly economic reports, is that reported GDP growth may also be off the mark.
Between 1992 and 2011 reported GDP growth averaged 3% per annum while reported employment growth averaged 1.9%. It makes sense that GDP growth runs around one percentage point above employment growth reflecting the contribution of productivity growth to GDP growth. Over this period there were rare exceptions in which reported employment growth was stronger than reported GDP growth that may reflect sampling errors as much as anything. By contrast, in the last three years reported GDP growth has averaged 3% per annum while reported employment growth has averaged 3.5%. To me as an economist with 30 years' experience this just doesn't make sense. Last year this was the result of SNZ including two lost groups of employees for the first time and not making an adjustment to include them on the historical numbers meaning reported employment growth significantly overstated actual growth as discussed in the last Raving - more on this below. Maybe part of the problem is underreporting of actual GDP growth.
In my assessment reported employment growth for 2013 and 2014 should be completely ignored. I severely doubt that employment fell 1.7% in 2013 just to rebound 4.8% in 2014 while over this period GDP growth remained relatively stable. It seems to be pretty clear that the problem is mainly with reported employment growth that may taint the reported unemployment rate but it is still possible there are problems with the accuracy of reported GDP growth.
I see no defence for knowingly reporting inaccurate employment growth
In my opinion SNZ made a major error in judgement in reporting 5.8% employment growth for 2016 when it knew actual growth was weaker. The reported super-charged growth was the result of including two groups of previously overlooked employees in the official numbers starting in the 2016 June quarter and not revising the earlier data to incorporate estimates for these two groups.
Based on what I was told by the SNZ official who kindly rang the quarterly labour force survey didn’t provide the information needed to estimate the impact of the inclusion of the two groups of previously overlooked employees. Consequently, SNZ didn't have a sound basis for grossing up the earlier employment numbers to incorporate estimates for the lost employees. Maybe this was a good judgement but to me there is no defence for knowingly reporting hugely inaccurate employment growth.
On reflection after the discussion with the SNZ official it seems to me the sensible solution was to run the old survey alongside the new survey for at least one quarter. If this had been done it would have been possible to estimate the impact of the inclusion of the previously overlooked employees. This obviously would involve more money and comes back to my point that one of the issues over the quality of some of New Zealand's official economic data may be budget constraints.
As I suggested in the last Raving, maybe SNZ should be a beneficiary of the increase in government spending to be announced in the May Budget; subject to it being targeted at and contingent on increased quality in at least the major economic data (e.g. GDP, employment growth, unemployment rate). But budget constraint or not, as a long time user of official economic data I believe it was a major error of judgement on the part of SNZ for knowingly reporting inaccurate employment growth for 2016. Equally, I struggle to understand why red flags weren't raised at SNZ over the disparities between reported employment and GDP growth in 2013 and 2014 (i.e. something should have been done to rectify the inaccuracies). Consequently, I am inclined to conclude that something needs to be done to improve the quality of decision making at SNZ.
---
Statistics New Zealand's response to the last Raving:
Statistics New Zealand agrees with Interest.co columnist Rodney Dickens that it is important official figures are robust - they must also be impartial and represent our best understanding at the time.
Dickens’ column (Opinion February 28, 2017) discussed the accuracy of official numbers, and how revisions and errors can affect the confidence of our user community.
Stats NZ welcomes feedback of this nature, and the opportunity it provides for us to improve.
We agree that official statistics are vital for decision-makers from the Reserve Bank and government policymakers, to investors of all kinds, whether they are putting money in the bank or setting up a corner dairy.
As Dickens points out there have been errors with some figures recently. Stats NZ is transparent about errors, often found by ourselves - we own up to these errors and make changes as quickly as possible.
In the case of revisions, these are usually driven by new or improved data that affect historical results, or changes to the underlying measure – in both cases we aim to better represent the real life situation of a changing society, economy and environment.
Where possible, we try to give our users as much notice as possible about the planned revisions. We explain the changes and even in some cases publish a revised back-series before new revised data comes out.
In both the case of error corrections and revisions, our goal is to get the most accurate data out to everyone.
Stats NZ welcomes close scrutiny of official figures - it is in everyone’s interest to make sure they are accurate. We welcome questions from economists, media or anyone else to help them understand the data.
Rachael Milicich
General Manager Products, Services & Insights | Ngā Hanga, Ratonga, Kitenga
Customer Strategy & Delivery | Rautaki Kiritaki me te Whakaratonga
Statistics New Zealand | Tatauranga Aotearoa (www.stats.govt.nz)
*Rodney Dickens is the managing director and chief research officer of Strategic Risk Analysis Limited.
10 Comments
I find I'm turning away from conventional measures as they are meaningless and if not intentionally manipulated they are affected by the methodology.
I've started look at actual tax revenue as they seems to be a more reliable indicator. Public debt to GDP is only 24.6% according to google but debt to tax revenue is about 92%. So our public debt doesn't look that great when examined in the same way as household debt, yet 24.6% debt to GDP is supposedly reasonable.
Of course the US is at 450% with debt to tax revenue so no wonder there's a huge drag on the US economy.
Maybe it's time to look at other statistics. GDP says that things are ok. GDP per capita is negative and suggests things are not good at all. Public debt to tax revenue and household debt suggest that we have too much public and private debt which is likely to become problematic.
Whether the public debt is 24 or 32% its not a huge issue ie I am not aware that any country has had an issue at this figure, 200% yeah maybe. Of course what is a concern is while we carry this debt the National party gives everyone an election bribe this year instead of paying the debt off.
Yet we consider the right economically sensible and the left not.
"Other statistics" I'd like to see why GDP is so far off. Example I/we know oil production peaked in 2004/5 and has been on a production plateau ever since and GDP has been lackluster since globally as well. Close to a decade of correlation should make ppl think. I am wondering if the size of (growing) debt is making GDP look bigger in some way than it really is, certainly when it comes time to pay it back (or write it off) that will have a big impact.
Private debt is a concern on several levels, a) it seems these days we buy and sell houses to one another rather than run a real business makes me wonder just how healthy our economy is and if that is distorting GDP? b) the level of private debt is monstrous IMHO, who is going to pay it? As long as it stays in private hands (ie those holding the debt take the loss) one one level I do not care, but as a tax payer I do care about it being dumped on the public purse when things go pear shaped. All the people with savings of course are voters and they will vote not to take losses but pass them onto their kids and grandkids just as the worl'd economy goes into permanent decline. Ergo its going to be default, the only Q is who the defaulter is. Now at least National has shown some spine in letting Solid energy collpase and not take on the debt. Labour? not much confidence they have any balls at all.
The debt to GDP figure seems ok until we go into a recession then the numbers get much worse. With debt to tax income of 92% that's not a good time to implement tax cuts, but at least that stupid talk has stopped for now. Just be aware that if there is a recession and the tax take drops that ratio will shoot up a lot.
Rapid GDP growth is gone, but also the improvement of people's standard of living is gone above a certain level of GDP. GDP growth isn't the one stop shop for improving the welfare of all citizens.
One article I've read suggests once you get above 100-110% household debt that this reduces GDP growth. We might not be in the situation yet despite it being 168% due to our historically low interest rates as debt servicing costs are still low. Once interest rates go up it will be interesting to observe.
I have worked with Stats before and its not easy to get a clear picture with 100% certainty about anything .
When there is a sudden spike in numbers ( in either direction) , you always have to ask why .
The fact that the measurement criteria changed in 2016 should have resulted in the prior years figures being adjusted to make the new inclusions meaningful when compared to prior years
We welcome your comments below. If you are not already registered, please register to comment.
Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.