Employment Media Transparency

November 2013 LFS: Why the monthly Labour Force Survey release shouldn’t be called “the jobs report” (updated December 13, 2013)

As practically all of these ‘alternative’ Labour Force Survey reports start out, the LFS monthly movements are notoriously unreliable. Most economists know it’s become even more so in recent years following The Great Recession, for a number of reasons. One of the easier to explain is the issue of survey sample size. Another issue highlighted in this month’s release is the increase in questionable self-employment, jobs which it’s somewhat cynical to attribute to positive economic growth. Hopefully readers who’ve taken to referencing the LFS as “the jobs report” pause to reconsider whether that’s appropriate. StatsCan publishes another survey of business payrolls that would better fit the description, although the Survey of Employment Payroll and Hours (SEPH) is not without its issues since the recession either.

Employment Media Transparency

July 2013 SEPH: Yet another reason to ignore monthly employment survey figures, plus update on the ‘unclassified’ jobs boom

Over the weekend, a critical media report, looking at the apparent contradiction between the July 2013 Labour Force Survey (LFS) and Survey of Employment Payroll and Hours (SEPH) releases, was brought to our attention. As noted (repeatedly) on these pages, analysing the month-to-month changes reported in Statistics Canada employment survey releases is a fool’s errand. The report deserves an ‘A’ for effort, for at least casting a critical eye on the figures. It gets an ‘incomplete’ otherwise.

Financial security Governance Media Poverty Transparency

2011 NHS: A few notes on the Income release

Inevitably, media sources will cite the 2011 NHS income data, set for release tomorrow, as a realistic portrait of Canadian household and individual incomes. It will not, and could not, be owing to the nature of the source survey. The following provides a brief overview as to why the 2011 NHS income data will be unreliable.  The role of income data in the federal government’s decision to cancel the long-form Census is also discussed.

Governance Media Transparency

2011 NHS: Closing notes on data quality

Statistics Canada has made a point of (mis)informing Canadians about the reliability and usability of the 2011 National Household Survey (NHS) data. While Canadians were told that the decline in response rates from prior long-form Census did result in greater data suppression at pretty much every sub-provincial level, most of the data was still published and represented as being of sufficiently high quality. Canadians were told not to be reluctant to use said data. The media for the most part followed this advice, reporting on the first couple of releases with few if any concerns.  Chief Statistician Wayne Smith even made a point of criticising those who cautioned Canadians about the reliability of the published data, stating that critics were doing a “disservice to Canadians”. The following discusses the significantly lowered data quality standards Statscan adopted for the 2011 NHS in order to (mis)represent its data as fit for publication.

Civil liberties Foreign policy Human rights Justice Media

The sorry state of Canadian civil liberties: Where facts can be disputed as hate crimes, and vice versa – by the same group


The map montage pictured above is a hate crime, according to Friends of Simon Wiesenthal Center. Why/how you may ask? Apparently it is provocative and incites hatred and contempt (PDF). Four time-lapsed maps, a statistic and a two-word descriptor. Seems fairly innocuous.

If anything, one would think a story accusing another nationality of starting the next Holocaust would be more worthy of being labeled a hate crime, especially if it turned out to be false. But the good people at Friends of Simon Wiesenthal Center didn’t denounce that story as a hate crime. That’s because the Simon Wiesenthal Center was busy promoting and perpetuating that possible hate crime.

Media Trade and investment

Canadian telecom Big 3: ‘Sweetheart deals for U.S. giants are a bad call for you’ – but not for us

‘Fair for Canadians’. ‘Cynical’ doesn’t do this ad justice.

For readers who can’t be bothered to watch the video (understandably), Canada’s Big 3 telcos, Bell, Telus and Rogers, are vehemently opposed to the federal government barring them from buying up their smaller competitors, a couple of currently struggling wireless telcos that entered the Canadian market only a few short years ago. They’re equally opposed to the federal government setting aside for smaller start-ups half of the wireless spectrum blocks that will be auctioned shortly. They’re afraid that a large international firm, such as Verizon, may swoop in and at once purchase one (or both) of the small telcos as well as the reserved wireless spectrum – which the government would be more than happy to see. Speculation about potential competition has already shaved billions off the Big 3’s market caps, which speaks to the lack of competition they claim is alive and well in Canada.


Inaccurate measures of poverty: The problem with reporting ‘poverty rate drop’ without referencing the metric

A recently published column, courtesy of a certain PostMedia News  pundit, criticized the Canadian media for not reporting a June 27, 2013 Statscan release.

“’Poverty rate’ drops to lowest level ever and media turns a blind eye” read the headline on Andrew Coyne’s piece on July 22nd.

Briefly, there are a number of assumptions and methodologies underlying Statscan’s low-income measures, including  the LICO-AT (after-tax low-income cut-offs), that suggest they shouldn’t be used as measures of poverty. In case it wasn’t obvious to some, Statscan specifically cautions readers that the LICOs are not measures of poverty. The author seems to be aware of this, conveniently placing quotes around the term ‘poverty rate’ in the title.

Had the author consulted a few knowledgeable business/ economy journos, he likely would have had the answer to the seeming media silence. In summary:

1. The media is waiting for the 2011 NHS Income and Housing Release, scheduled for August 14, 2013.

2. The media is increasingly skeptical/critical of the Statscan releases, specifically the validity of both the data and metrics used.

Long-form Census and low-income

The Survey of Consumer Finances (SCF) and, more recently, Survey of Labour and Income Dynamics (SLID) were (past-tense; check the links for details) used to estimate income for the purpose of reporting annual low-income incidence. The Survey of Family Expenditures (Famex) and, more recently, Survey of Household Spending (SHS)1, have been used as the basis for the annually updated low-income cut-offs.

The one thing all those surveys had in common was they were all voluntary. And that was OK, so long as there were other surveys that could be relied on to (re)weigh the voluntary response samples to make them representative of the general population. The most important among those were the short and long-form Census, the only mandatory household surveys in Canada according to the Statistics Act. Or so most people assumed (see s.8 and s.19 of the Act), until the long-form Census was cancelled and replaced by the voluntary 2011 National Household Survey in 2010.

The final release of the Census cycle, Income and earnings (along with Housing and shelter costs), was always the most anticipated by the media. The long-form Census provided the most reliable and comprehensive measure of income, broken down by individual and household demographics, publicly available in Canada. The big headline grabbers were the direction and magnitude of income changes, along with changes in low-income incidence, relative to prior Census year(s).

Interestingly, the same ‘news’ outlet chastising the media for not reporting a relatively inconsequential July 27 Statscan release was the only one to take exception to Statscan’s report on stagnating middle-class incomes in its 2006 Census Income release.  ‘StatsCan sets off its own class war,’2 read the headlines of a Terence Corcoran column on May 2nd, 2008.

But back to the subject of the long-form Census and low income. As noted, it has been the biggest headline of the biggest release of the Census cycle (see 199620012006 Census Income releases).  Among the notable headlines the 2006 release generated were: “Almost 900,000 children live in poverty” from Loryanne Anthony for The Canadian Press; “The Canadian dream? 25 YEARS, 53 BUCKS” from Michael Valpy in The Globe and Mail.

Media anticipation of 2011 NHS Income release

Most of the detailed low income research in Canada, including Statscan’s own, references the Census low-income data for the reason already mentioned:it was the most comprehensive available.

Most experienced business / economy writers know this, which likely explains the scant coverage given  the June 27, 2013 low-income release from the Survey of Labour and Income Dynamics: the media is waiting for the major 2011 NHS low-income release scheduled for August 14, 2013.

If the last two 2011 NHS releases are indicative, the coverage will likely focus less on the Statscan’t analysis and more on how suspect the data is.  Readers may recall that the 2011 NHS immigration data was not in line with administrative data from the Department of Citizenship and Immigration Canada. As goes immigration, so goes visible minority data. And more than a few people questioned the 2011 NHS Aboriginal population estimates; As John Kolkman succinctly put it, This makes no sense.

And, one suspects, neither will the low income data from the 2011 NHS. Recent immigrants, visible minorities and Aboriginals tend to be among the disadvantaged groups over-represented (relative to their population share) in the low-income data. That these groups are no longer accurately represented in the population data will make the low-income data less reliable.

Media scepticism

Following The Great Depression, Canadians have seen spikes in household debt, precarious employment, lay-offs  – which newsrooms weren’t spared,  and social assistance at the same time government income security programs were clawed back in the name of austerity. The anecdotal evidence would make any reasonable person suspect of data indicating Canadian low-income incidence fell significantly over the same period.

Scepticism in the media is a good thing; It encourages more in-depth research and analysis. Unlike our seemingly oblivious Postmedia pundit, most business / economy journos likely looked up the 2011 LICO After-tax table and realised that the cut-offs were completely meaningless as a measure of poverty. As mentioned, no reasonable person would conclude that a couple living in a major Canadian urban centre could get by on $23K annually.

Some probably dug a little deeper and discovered that Statscan has increasingly adopted measures of low income that seem to serve no purpose other than to reduce the reported low income incidence (see here). All the more disappointing, given that the U.S. Census Bureau has taken the opposite approach with its Supplemental Poverty Measure (see Figure 7 in Income, Poverty, and Health Insurance Coverage in the United States: 2011 – PDF).


In closing, a note on data ‘cherry-picking’: Our perturbed PostMedia pundit has chastised the media for not reporting a data point which he asserts is indicative of a decline in Canadian poverty. However, he does not hold the same view of data and data sources that do not reconcile with his perspective. In fact, when several long-standing federal government advisories that compiled and published statistics on actual poverty were axed, he supported the decision.

“’Government isn’t obliged to bankroll its critics” read the headline on Andrew Coyne’s piece on May 4th, 2012.

Intellectually dishonest perhaps best sums up this approach to reporting. Then again, the PostMedia News pieces referenced in the preceding were all conveniently written as Op-Eds, not news items.


1 The low-income cut-offs are relative to average family expenditure based on the Survey of Household Spending. The survey’s methodology appears to excludes wealthier households (those that spend 70% or less of their earnings in a given year), likely biasing the cut-offs downward (see here).

2 In addition to more women working in 2005 relative to 1980, family composition, the proportion of the population living in a family, not to mention Statscan’s definition of family, all markedly changed over the referenced 25 year period. That’s why individual income is the preferred unit of historical comparison, and why the criticised Statscan report used it instead of family income.

Employment Financial security Media Poverty

Q: How does a McDonald’s employee get by in the new Mconomy? A: By getting a second McJob, according to McDonalds

Kudos to ThinkProgress for catching this. In case it gets removed, below is a copy of the McDonald’s employee Budget Journal (see p.4)

Of interest to Canadian readers: Even though the same McDonalds employee’s U.S. dollar (USD) income in Canadian dollars(CAD) would leave a Canadian McDonalds employee even further behind, that same worker wouldn’t be considered poor in Canada. According to the official Canadian low-income cut-off table, that McDonalds employee’s income should be more than sufficient for two.

Governance Media Transparency

2011 NHS: Community organizations officially left in the dark

Today’s first release of the 2011 National Household Survey (NHS) data confirmed what we had previously written on December 6, 2012. It appears the data quality was so poor that Statistics Canada decided to release neither data at the dissemination area (DA) nor the census tract (CT) levels. These are more commonly referred to as ‘community-level’ data.

NHS Focus on Geography Series

2011 National Household Survey: Data tables

NHS User Guide > Chapter 6 – Data dissemination for NHS standard products

It’s unclear at this point whether the community-level data will be released at a later date, or only provided on a paid-access basis. Given the obviously problematic data quality, the latter would be ill-advised.

As an example, the small community that received the Statscan letter referenced in the December 2012 post had a population of about 65,000 in 2006. That community was swallowed up in amalgamation and is now part of a census subdivision (CSD) with a population of 1,600,000. The lowest level geography made available in today’s 2011 NHS release was CSD. According to Statscan, the non-response rate for that amalgamated CSD was 21%. To put that in context, non-response in that small community’s now amalgamated CSD was equivalent to 5 communities its size. From a statistical perspective, that community has effectively disappeared.

That was the biggest news from yesterday’s 2011 NHS release: No community-level data (both DA and CT) was released.

Editor’s Note:
For context, census tract data has been available since 1941, dissemination area data since 2001 (enumeration area data, the DA equivalent prior to 2001, dates back to 1961).

Update (26/06/2013):
An update to this post can be found here.

Governance Media Transparency

2011 NHS: News since long-form Census cancellation of summer 2010

The media has once again taken up writing about the 2011 long-form Census cancellation in preparation for the first releases from its replacement, the 2011 National Household Survey (NHS). Readers would be hard-pressed to distinguish the media write-ups in recent days from those dating back to summer 2010.

What happened back in 2010

Munir Sheikh, who in his brief time as Chief Statistician gutted the agency’s social and environmental programs along with analytical research, somehow ended up a hero after resigning over the long-form Census cancellation.  In the original article that broke the Census cancellation story brief mention was made of cuts to programs and analytical research, along with an unsettling culture change coinciding with Mr. Sheikh’s arrival. The media failed to follow up, presumably because it didn’t fit with the developing narrative.

The official opposition at the time played politics with the issue – deservedly, it went on to suffer the greatest electoral defeat in party history. The federal government ultimately conceded to the addition of a couple of language questions to the short-form Census questionnaire.

There was no shortage of hypotheticals re the drop in data quality that would result from the changeover from a mandatory to a voluntary long-form. Reference was made to how municipalities could suffer from lower quality data that was necessary for urban planning and how social/community groups working with vulnerable communities could potentially suffer in the same way.

The cancellation of the Participation and Activity Limitation Survey (PALS) shortly thereafter received significantly less attention, though it was not completely unexpected: The PALS sample relied on a couple of questions on the long-form Census. The government promised to replace the PALS with another survey shortly thereafter.

The media eventually discovered that the voluntary 2008 Census test, conducted in preparation for the 2011 Census, contradicted then Industry Minister Tony Clement’s claim that StatsCan had assured him a voluntary survey could provide data of comparable quality to the long-form Census.  In 2008, less than half of the long-form test questionnaire recipients responded to it.

All the while the federal government Gong Show about protecting Canadians from long-form Census questions asking for the number of bathrooms in their homes went on largely unabated (despite the obvious fact the questionnaire included no such question).

What’s happened since

Munir Sheikh pulled his golden parachute and landed at Queen’s University. The career government hatchet man (PDF), in no small part thanks to the mistaken hero narrative, ended up heading The Commission for the Review of Social Assistance in Ontario (PDF). Not surprisingly, among the report’s recommendations was a cut to benefits for the province’s disabled. We’ve touched on the impact The Great Recession that began in 2008 had on the poor and disabled in Canada’s largest province. (Notably, during his brief tenure Mr. Sheikh directed StatsCan employees not to use the word “recession”.)

The federal opposition made some noise about the impact the loss of the long-form Census would have on federal official languages policy, going so far as to pursue a court challenge on the basis of the Official Languages Act. What it seemed to wilfully ignore was the greater role the long-form played in evaluating the performance of Canada’s legislative/statutory framework. Among other things, the Census Guide 2B 2006 noted:

Questions 7 and 8 provide information on the number of people in Canada who have difficulties with daily activities, and whose activities are reduced because of a physical condition, a mental condition, or a health problem. The results are used to help Statistics Canada find out more about the barriers these persons face in their everyday lives…

Question 17 provides information about the ethnic and cultural diversity of Canada’s population. This information is required under the Multiculturalism Act (s. 3.(2)(d)) (PDF) and the Canadian Charter of Rights and Freedoms (PDF). It is also used extensively by ethnic and cultural associations, as well as by agencies and researchers, for activities such as health promotion, communications and marketing.

Questions 18, 20 and 21 provide information about Aboriginal or First Nation, Inuit and Métis peoples that is used to administer legislation and employment programs under the Indian Act and the Employment Equity Act. The information is also used by researchers and Aboriginal governments and associations to explore a wide variety of demographic and socio-economic issues. 

Question 19 tells us about the groups that make up the visible-minority population in Canada. This information is required for programs under the Employment Equity Act, which promotes equal opportunity for everyone.

The PALS relied on long-form Census questions 7 and 8 for its sample. The PALS sample became less of a concern when the government scrapped it shortly after scrapping the long-form Census. In 2012, PALS was quietly replaced by the Canadian Survey on Disability (CSoD), which uses the less reliable National Household Survey (NHS) for its sample. Unlike PALS, and every major StatsCan social survey before it, the CSoD is now a voluntary survey based on a sample taken from another voluntary survey. This will be the fate of all StatsCan social surveys that previously relied on the long-form census.

Like numerous required periodic reports on the operation of federal statutes, the Minister’s Annual Report on the Operation of the Canadian Multiculturalism Act cites the Census as its data source on immigrant and ethno-cultural communities. In addition to the loss of the long-form Census, the Longitudinal Survey on Immigrants to Canada is classified inactive, likely to be discontinued.

Canadians with disabilities and Canadian ethno-cultural communities are among the vulnerable minorities on whom data would become less reliable with the change from the Census to the voluntary NHS. The preceding are concrete examples of how the loss of the long-form Census has already had an adverse impact.

An example of how the loss of the mandatory long-form has already affected community organisations was previously touched on. Even if it ultimately does release data for lower-level geographies (census tracts, dissemination areas),  StatsCan’s uncertainty only six months prior to the first scheduled 2011 NHS releases speaks to the questionable quality of the data collected. (It will likely be compelled to release it eventually, irrespective of data quality.)

Going forward a few years, Canada’s Labour Force Survey (LFS), as notoriously unreliable as it already is, will become even more so. As succinctly stated here:

the long-form data is the basis for just about all of Statistics Canada’s important social measurements. The unemployment rate, for instance, is compiled from the monthly Labour Force Survey, but the sample used in that survey is based on the census data. Once the census data becomes voluntary, the unemployment rate will be considered less reliable, taking the heat off governments in times of rising unemployment.

Given that the LFS is used to determine qualifying hours and weeks entitlement for Employment Insurance (EI) benefits for the unemployed, the more unreliable LFS data will mean more questionable EI benefits denials and shorter benefits entitlements.

What’s the end game

A lot of the chatter following the government’s decision to scrap the long-form Census in 2010 accused it of trying to substitute ideology in the place of fact-based public policy. It’s plausible. However, the government seemed to go out of its way to make the point it could likely collect better, more accurate information at lower cost from administrative data.  A quick browse of online comments in support of the government’s position at the time appeared to repeat this ‘administrative database’ idea, elaborating on how certain Scandinavian countries don’t take a Census but rather rely on administrative records in lieu of a census..

The ‘ideologically-driven’ narrative seemed to ignore the government’s penchant for passing legislation curtailing Canadian civil liberties in the name of protecting Canadians from fill-in-the-blank (pedophiles, youth gangs, terrorists, etc). Little consideration was given to the possibility the long-form Census was cancelled in the name of protecting Canadians’ privacy, with the intent of passing legislation intended to egregiously breach same said privacy. It was not that long ago that a secret longitudinal administrative database (under a previous government) was cancelled upon discovery for violating Canadians’ privacy.

Canada Scraps Citizen Database
Wired News Report   May 30, 2000

Privacy Commissioner applauds dismantling of database
Office of the Privacy Commissioner of Canada, May 29, 2000

That ‘super-database’ was innocuously named the Longitudinal Labour Force File (LLFF), HRDC PPU 335.

Given the options, well-informed Canadians would likely choose the far less intrusive long-form census over a secret government super-database that was previously found in breach of their privacy rights.