2013 SHS: A few questions for the upcoming CPI update ‘Ask an Expert’ Q&A

Figure 1 SHS response rates 1997-2008
Source: The Importance of the Long Form Census to Canada, David A. Green and Kevin Milligan, Canadian Public Policy – Analyse de politiques, vol. xxxvi, no. 3 2010

This Thursday, Statistics Canada will be updating the Consumer Price Index (CPI) basket weights with the latest results from its 2013 Survey of Household Spending (SHS). Hopefully its corresponding CPI update ‘Ask an Expert’ Q&A, scheduled the following day, proves more informative than its 2011 National Household Survey (NHS) Q&A session.

While this CPI basket update will be the second undertaken since a major SHS redesign in 2010, little information about the survey has been made publicly available since then. Statscan stopped producing public use micro data as well as data quality reports for the SHS immediately following the redesign.  The official reason: “There will be no public use microdata file (PUMF) for SHS 2010 due to resource constraints.”

That Statscan regularly cites ‘resource constraints’ even as it contends they haven’t affected the agency is one thing. That it regularly does so to justify withholding data from the public is a concern. Other national statistical agencies likewise facing resource constraints, including those in the US and the UK, seem loathe to target public data dissemination. Not only does doing so inconvenience data users, it undermines public confidence.

Since the release deals with the specific use of the SHS to weight the CPI basket, the main question is whether / how well the survey results are representative of expenditure patterns of different households across Canada. The two common sources for potential problems are sampling error (whether the selected sample households accurately represent the target population) and non-sampling error (like under-coverage and non-response).

The 2013 SHS was based on the Labour Force Survey (LFS) sampling frame in place at the time, which was based on the 2001 census. In addition to accounting for geography (its ‘primary sampling unit’), the LFS sampling frame includes ‘special strata’ to better account for aboriginal, immigrant and high-income households (and yes, that data is, or rather, was, derived from the long-form census  – more on that in an upcoming post).

Assuming the LFS sampling method provides a sufficiently representative cross-section of non-institutionalised Canadian households (it doesn’t, but more on that another time), that leaves potential sampling errors as a possible source of bias in the SHS.

For a selected household to be part of the survey, it has to be living in the residence selected in the sample at the time the survey is conducted, with at least one household member able to respond on its behalf available to do so (coverage). That member would also have to agree to take the survey (response) as the SHS is voluntary. Statscan also has to find the survey response ‘acceptable’.

The SHS is conducted in two parts. The interview, administered to all surveyed households, collects information on broader expenditure as well as income components. To make it more like the US Consumer Expenditure Survey (CEX), in 2010 Statscan introduced a diary, distributed to half of surveyed households, which it uses to collect more detailed expenditures (and to check against the interview data for verification).

The response rates for the 2013 SHS survey components were 67% and 46%, respectively. For comparison, the response rates for the 2013 CEX survey components were 67% and 61%, respectively.

Overall response rates are less informative than response rates by different household characteristics, since different types of households living in different parts of the country have different consumption patterns, which can have a significant impact on CPI basket item weights.

While Statscan provides some information on geographic disparities – for example, only five hundred usable diaries were received from Ontario, its nearly 5 million households comprising the most populous and diverse province in the country.

Wealth/income inequality in Canada sees a disproportionately greater share of income, along with a disproportionately even greater share of discretionary income, concentrated among fewer households. As such, not having a representative share of high-income households in the SHS could significantly impact the survey results, and ultimately the CPI basket weights.

Statscan makes a point of emphasising the priority given to sampling high-income households: “The high-income household strata are allocated a larger share of the sample than the other strata, where an allocation proportional to stratum size is used.” However, that’s the first and last mention of high-income households in the 2013 SHS Guide. It’s worth noting the LFS sample frame defined ‘high-income’ as total household income over $125,000 based on the 2001 Census.

Statscan used to publish somewhat more detailed data quality indicators. From its 2009 SHS data quality report: “This report… covers the usual quality indicators that generally help users interpret data, such as coefficients of variation, response and non-response rates, slippage rates and imputation rates.” The 2009 report included non-response and under-coverage rates for high-income households (for the interview; the diary was rolled out in 2010).

Under-representation of high-income households (as well as their respective incomes, which exclude capital gains) in the national household spending survey is not a trivial / trifling matter. In fact, economists from across US federal statistical agencies recently published a report seeking to address this very issue with the US household spending survey. Is the Consumer Expenditure Survey Representative by Income? (NBER, 2013) notes: “Given the plutocratic nature of the CPI, the relationship of income and spending on different types of categories suggests that under-representation of high income families in the CE could be biasing the CPI.”

The solution seems simple enough: Just make the SHS mandatory. Unfortunately, there’s a long history that precludes such a possibility. The predecessor of the SHS, the Survey of Family Expenditure (FAMEX) was once a mandatory survey, before over-zealousness and questionable competence on Statscan’s part killed it. (A valuable lesson for the ongoing National Household Survey / long-form census debate, that’s obviously not been learned.)

There are far more questions than there are answers regarding the SHS at this point. As it stands, it’s not too far a stretch to say that using the SHS to weight the CPI basket could be biasing the Canadian index. Unfortunately, questionable Statscan decisions of the past have all but assured there’s no easy fix to the problem going forward.

Fluctuating self-employment data a sign of Canada’s weak labour market (Includes Correction*)

Chart 1 Part-time self-employment by sex, seasonally adjusted and unadjusted
               January 1987 to December 2000Chart 1 Part-time self-employment by sex, seasonally adjusted and unadjusted Jan1987-Dec2000

 Source: CANSIM Table 282-0019 Labour force survey estimates (LFS), Statistics Canada
*downloaded February 11, 2015

 

Chart 2 Part-time self-employment by sex, seasonally adjusted and unadjusted
               January 2001 to December 2014Chart 2 Part-time self-employment by sex, seasonally adjusted and unadjusted Jan2001-Dec2014

 Source: CANSIM Table 282-0019 Labour force survey estimates (LFS), Statistics Canada
*downloaded February 11, 2015

Developing…

Note: Chart data was seasonally adjusted using TRAMO-SEATS from Banco de España. For more information, see Empirical Evaluation of X-11 and Model-based Seasonal Adjustment MethodsU.S. Bureau of Labor Statistics, December 2007.

November 2014 LFS: An alternative view. 23,000 jobs lost, or 12,300 new ‘entrepreneurs’ found?*

Statistics Canada’s November 2014 Labour Force Survey (LFS) release  generated the typical headlines, like Canada lost 10,000 jobs in November (courtesy CBC News). The associated write-ups highlighted bank economists’ ‘cautious optimism’, as they have for a number of years now. While Statscan’s brief report twice mentioned significant self-employment gains, both month-to-month and year-over-year, most national news reports (like CBC News’) omitted any such mention. As the saying goes, you can only lead a horse to water.

The almost statistically insignificant changes reported in the latest and last LFS release of the year nevertheless merit mention, if only for reflecting the year that was(n’t) in the Canadian labour market.

Continue reading

2016 NHS: Long-form census fiasco as FAMEX redux

Shady
2011 NHS: How much less we now know, illustrated * 2011 NHS: How much less we now know, illustrated *

Canada’s 2011 long-form census fiasco was brought to the fore again last week with the introduction of Bill C-626. The private member’s bill seeks to amend the Statistics Act to mandate the long-form survey and provide the Chief Statistician with greater administrative autonomy. While there’s no debating its voluntary replacement rendered useless results at exorbitant cost, the proposed legislation, even if successful, won’t necessarily assure the long-form census’ future success.

It’s been written those who cannot remember the past are condemned to repeat it. Although never mentioned in the ongoing public discourse, what’s happening with the long-form census is basically the Survey of Family Expenditures (FAMEX) all over again. And that didn’t turn out well.

Continue reading

Is Canadian housing price inflation an intended policy outcome?

Bank_of_Canada

During the press conference following the Bank of Canada (BoC) July 2014 Monetary Policy Report, bank governor Stephen Poloz was asked whether the Bank was making up excuses – likely in response to his “serial disappointment” remark – to avoid raising its target rate. A lower rate benefits the federal government in a number of ways, primarily by lowering its debt servicing and public spending costs. However, the dangers of an extended period of low interest rates include excessive household debt accumulation, particularly mortgage debt.

But what if the BoC views excessive housing price inflation as a key economic driver – and that view is affecting its policy rate decision?

Continue reading