Employment Media Transparency

November 2013 LFS: Why the monthly Labour Force Survey release shouldn’t be called “the jobs report” (updated December 13, 2013)

As practically all of these ‘alternative’ Labour Force Survey reports start out, the LFS monthly movements are notoriously unreliable. Most economists know it’s become even more so in recent years following The Great Recession, for a number of reasons. One of the easier to explain is the issue of survey sample size. Another issue highlighted in this month’s release is the increase in questionable self-employment, jobs which it’s somewhat cynical to attribute to positive economic growth. Hopefully readers who’ve taken to referencing the LFS as “the jobs report” pause to reconsider whether that’s appropriate. StatsCan publishes another survey of business payrolls that would better fit the description, although the Survey of Employment Payroll and Hours (SEPH) is not without its issues since the recession either.

Standard errors suggest increasingly unreliable data

StatsCan takes great pride in noting its LFS in-scope sampling rate is relatively ten times that of the US Bureau of Labor Statistics’ Current Population Survey (CPS). Which is true. That in and of itself, however, does not mean it always gives a reliable picture of changes in the Canadian labour market – not even for the most basic stats like the size of the labour force or total employment. As a concession to greater transparency, earlier this year StatsCan began to publish the standard errors for the LFS sample.

Table 1 – Standard errors for Labour Force and Employment Estimates

Month Labour Force (thousands) Employment (thousands)
Est. ∆ Std.Err. 1 Est. ∆ Std.Err. 1
Apr -0.5 29.1 12.5 28.7
May 80.9 29.1 95 28.7
Jun 7.1 29.1 -0.4 28.7
Jul -14.2 29.2 -39.4 28.9
Aug 40.9 29.2 59.2 28.9
Sep -25.1 29.2 11.9 28.9
Oct 13.2 29.2 13.2 28.9
Nov 12.4 29.2 21.6 28.9

1.Average standard error of change in two consecutive months (see “Sampling Errors” in Data Quality for further explanations).

Source: Monthly Labour Force Survey (LFS) release, The Daily, Statistics Canada

As shown in Table 1, the standard errors for the broadest LFS stats, total labour force and employment,  exceeded the estimated changes in six of the eight months since StatsCan began to disclose the errors. Readers are encouraged to visit the link provided for StatsCan’s official explanation. In a nutshell, a standard error greater than the estimated change in a given month means the estimate is less reliable (the greater the error relative to the change, the more unreliable the estimated change).

That’s simply a sample size issue. Given the size of StatsCan’s sample, there’s a minimum level of change required monthly for the survey to register a statistically significant change for a given stat.

It also looks like StatsCan’s been readjusting its standard errors only once every six months, despite the fact the sample changes slightly every month.

 Self-employment and actual employment

Research over the years, including StatsCan’s own, has shown economic downturns are prone to spells of rising self-employment. We’ll be blunt in summarising the issue: Many of those aren’t real jobs, but unemployed Canadians trying to make the best of a bad situation.

During an economic downturn, during which employers are reluctant to hire due to soft demand, StatsCan’s LFS releases occasionally report monthly spikes in self-employment. Most of those spikes tend to be in the “unincorporated” and/or “no paid help” categories, and they tend to disappear in subsequent months just as suddenly as they appeared.

It’s usually impossible to tell whether they’re actual jumps or sampling, seasonal-adjustment snafus. While almost all the reported October to November 2013 job gains (19.1 of 21.6 thousand) were self-employment, the standard error on that increase in self-employment was 25.2 thousand – meaning there was a good chance there was no gain at all.

To sum up: In addition to the strong likelihood many of those November 2013 self-employment jobs weren’t real (at least the “unincorporated”, “no paid help” type),  the reported gains were statistically unreliable.

All things considered, it’s really a stretch to call the monthly LFS release “the jobs report”. The November 2013 release stands as an excellent example of why it shouldn’t be.

Leave a Reply

Your email address will not be published. Required fields are marked *