ANZ's chief economist Sharon Zollner and her predecessor Cameron Bagrie hit back at criticism of the bank's Business Outlook Survey from Rodney Dickens

ANZ NZ's current and former chief economist have hit back at criticism of ANZ's Business Outlook Survey from independent economist Rodney Dickens.

A long-time critic of the ANZ survey, Dickens has again attacked it. In an article published by interest.co.nz on Wednesday Dickens argues the survey has been "corrupted by some respondents using it as a political football," and "some questionable analysis is now being used to make the survey appear more useful than it is."

Out last week, the latest iteration of the ANZ survey showed business confidence and businesses’ expectations of their own activity fell again in March, largely due to concerns around residential construction.

Both Sharon Zollner, ANZ's chief economist, and Cameron Bagrie, her predecessor, acknowledge responses to the business confidence question in the survey do display political bias.

"There is indeed a political bias to the survey, most notably in the headline Business Confidence, which asks respondents about general business conditions in 12 months’ time. We have been very open about that; see for example our November 2017 report," says Zollner.

"There is a much smaller degree of bias in the other questions, particularly those asking about performance and concrete intentions on various aspects of running their business. It has been my observation that in general, observers over-estimate the impact the Government has on business cycle turning points and this observed bias is likely a manifestation of that, not ‘gaming’ of the survey."

Additionally Zollner says ANZ's analysis "does indeed find" political bias in the export intentions series, "which it hadn’t occurred to us to check."

"Once this is taken into account, the recent fall, which as we wrote in the March ANZBO release 'doesn’t square with anecdotes', looks much less dire, though it still suggests growth in non-food manufactured exports is going to zero in Q1. The bias could be explained by many things, eg concerns about potential higher labour costs under a left-leaning Government, rather than ‘gaming’ of the survey," Zollner says.

"We weren’t aware of the bias in the export intentions, and we will take that into account going forward." 

Nonetheless she says ANZ's economists "object strongly" to assertions their Business Outlook Survey is misleading or "corrupted by some respondents using it as a political football."

"Dickens tells a good story but we’ve tested it as far as is possible, and found no supporting evidence," Zollner adds.

'Rodney can’t even get the survey name right'

Bagrie, now managing director and chief economist at Bagrie Economics, is even more direct saying; "Rodney can’t even get the survey name right so little wonder he is confused."

"It’s a business outlook survey with a business confidence question not a business confidence survey. The business confidence question does display a political bias so is to be ignored," says Bagrie.

"It would be useful if Rodney focused on the composite growth indicator from the survey, like everyone else tends to, as opposed to attacking individual components of the survey. His article looks to be 'marketing'."

"I can remember Rodney putting the boot in during the global financial crisis saying the RBNZ shouldn’t be cutting rates, and us (me) bank economists didn’t know what was going on. Enough said," Bagrie adds.

Here's Zollner's response to Dickens' criticism in full.

Claim 1: The survey is gamed

There is indeed a political bias to the survey, most notably in the headline Business Confidence, which asks respondents about general business conditions in 12 months’ time. We have been very open about that; see for example our November 2017 report.

There is a much smaller degree of bias in the other questions, particularly those asking about performance and concrete intentions on various aspects of running their business. It has been my observation that in general, observers over-estimate the impact the Government has on business cycle turning points and this observed bias is likely a manifestation of that, not ‘gaming’ of the survey.

We looked closely at the impact that new respondents and/or businesses who didn’t usually respond had on the survey in the months after the election. If the survey was indeed being gamed to send the new Government a message, one would expect an influx of negative responses from enthusiastic new and returning survey participants. We found no evidence of this – indeed, for many series the new respondents were more positive.

The mid-2015 fall in the survey suggested as an example of ‘gaming’ was in fact the result of the dairy downturn. If you look at the regional data, it is easy to see the pain was real. Retail sales in Waikato ground to a complete halt in 2015, for example.

Claim 2: Business confidence has a poor correlation with GDP growth

Yes of course it does. The question asks about their view on general business conditions 12 months ahead, not economic growth per se. And many things change over the course of 12 months. It is hard enough for those of us who are paid to predict the economy one year ahead to do so; it is hardly surprising that businesses aren’t very good at it either. The [NZIER] QSBO measure is very similar.

We downplay the headline business confidence measure as a predictor of GDP growth in our releases for exactly that reason. Rather, we use this question for insights on the mood of businesses and it prompts us to discover more about what’s influencing confidence in our discussions with businesses.

Claim 3: Own activity has a political bias

Yes it does, though to a much smaller extent than the headline business confidence. Correlation analysis is useful for understanding the broad relationship between variables, but regression analysis is superior for assessing bias. A statistical regression model with a binary variable for which political party is in power provides an estimate of how much it matters, as well as an indication of how certain it is that it matters.

For ANZBO own activity, the bias is statistically significant, but with or without it, own activity picks the economics swings and roundabouts pretty well (figure 1) and is picking the economy continued to gently lose steam in Q1.

 

But own activity isn’t actually the best predictor of GDP in the ANZBO. As we’ve pointed out in our reports, capacity utilisation actually explains it best. For this series, there is no evidence of political bias. There is always significant variation in GDP caused by things like drought that business surveys cannot hope to capture, but it captures the broad cycles pretty well, including the GFC, the influence of commodity prices, and the recent slowdown.

Claim 4: QSBO own activity outperforms ANZBO own activity as a GDP predictor

The best GDP indicator out of the QSBO is own activity experienced. Firms know better what they did in the past three months than how the next three months will go, naturally enough. It is not reasonable to compare the forecasting performance of an expected versus experienced question.

However, the QSBO survey also has an expectations question that is broadly the same as our survey. The two match up closely. The QSBO expected activity series also has a statistically significant political bias when used to model GDP. It does fit GDP a bit better than its ANZBO equivalent, but the experienced activity question is far superior.

Claim 5: The chart of residential construction intentions is misleading

The breakdown in the correlation is due to the 2015 period, when intentions fell but actual consents rose. Again, it comes down to your interpretation of what was happening at that time. One possible interpretation is that respondents were ‘gaming’ the survey to bring about interest rate cuts.  A more credible explanation is that it was widely believed by both firms and economists that the sharp fall in dairy prices was going to derail the broader economy, but in the end, the strength in Auckland housing more than offset it.

The relationship did change in 2008, as indeed many economic ‘truths’ did, but a decade of data is sufficiently long to assess the new relationship. Assuming economic relationships don’t change is a sure recipe for some pretty poor forecasting, as everyone who tries to predict inflation can attest.

The unusually large amount of public sector building activity slated for the years ahead that may well mean the correlation between private sector intentions and actual consents remains weak. But that doesn’t mean that a gauge of private sector intentions is not useful or informative. Residential construction intentions fell in every single region of the country in March. That is worth knowing.

There is no statistical evidence for political bias in this data.

Claim 6: The chart of export intentions is misleading

Our export intentions series has had decent explanatory power since 2010 (a full 8 years of data). However, our analysis does indeed find political bias in this series, which it hadn’t occurred to us to check. Once this is taken into account, the recent fall (which, as we wrote in the March ANZBO release, 'doesn’t square with anecdotes') looks much less dire, though it still suggests growth in non-food manufactured exports is going to zero in Q1.

The bias could be explained by many things, eg concerns about potential higher labour costs under a left-leaning Government, rather than ‘gaming’ of the survey.

Conclusion

All up, we object strongly to assertions that the Business Outlook Survey is misleading or “corrupted by some respondents using it as a political football”. Dickens tells a good story but we’ve tested it as far as is possible, and found no supporting evidence.

There is indeed political bias in some of the ANZBO data, likely reflecting firms’ over-estimation of the influence of policy on the economic outlook, and we have been very open about that. We weren’t aware of the bias in the export intentions, and we will take that into account going forward. But for most series in the Business Outlook survey, apart from the headline business confidence series (which we have long downplayed as a predictive variable), the bias is insignificant, statistically or economically.

The ANZ Business Outlook survey provides a timely, useful read on the state of the business cycle, and has been extremely helpful in recognising the recent growth slowdown in real time.

The details of the survey tell us about much more than where GDP is going. The survey provides us with important colour about the specific conditions that businesses face. While conditions vary from business to businesses, the survey tells us that many are experiencing somewhat slower turnover, rising costs, profitability strains, credit constraints, and difficulty finding labour. Our many conversations with businesses across the country reveal that these concerns are very real.

And below is Bagrie's response in full.

Well Rodney can’t even get the survey name right so little wonder he is confused.

It’s a business outlook survey with a business confidence question not a business confidence survey. The business confidence question does display a political bias so is to be ignored.

The economics team has been clear for years what the key indicators are in the survey.

The composite gauge from the survey – which uses real activity, employment, investment and profitability – is one of the best indicators in the market. Composite indicators perform better than individual ones. 

Rodney has written these articles putting the boot into the business outlook survey before, and mixing up the 'survey' with the business confidence question.  I don’t think him saying business confidence is not a good indicator is anything new.  Everyone is aware of that so we focus on the rest of the survey which provides good insights.

It would be useful if Rodney focused on the composite growth indicator from the survey (like everyone else tends to) as opposed to attacking individual components of the survey.  His article looks to be “marketing”.

I can remember Rodney putting the boot in during the global financial crisis saying the RBNZ shouldn’t be cutting rates, and us (me) bank economists didn’t know what was going on. Enough said. 

*This article was first published in our email for paying subscribers early on Thursday morning. See here for more details and how to subscribe.