In Canada, a national census is conducted every five years in May by Statistics Canada. The census provides important demographic and statistical data used in a wide variety of applications. The results help determine the number of Members of Parliament for each Province and Territory and the amount of any federal transfer payments. The data from the census is also used by both public agencies and various private organizations to help plan essential services like health care, police, fire, education, transportation and housing.
Traditionally, the census has been administered by distributing a mandatory short form 8-question survey to 100% of households and a more detailed mandatory 53-question “long-form” (implemented in 1971) to one-in-five households. Ahead of the 2011 census, the conservative government decided to scrap the mandatory long-form survey in favor of a voluntary National Household Survey (NHS). The conservatives said the change from mandatory to voluntary was required due to privacy related complaints, yet this argument is difficult to understand given census data is already depersonalized for statistical purposes. In other words, it cannot be traced back to individuals.
So what happened to the census in Canada and why does it matter?
The decision to replace the long form was extremely controversial especially since the government failed to consult with organizations that work closely with Statistics Canada. Enraged at the change in policy, Munir Sheikh, Canada’s Chief Statistician, resigned in July 2010. In an open letter Mr. Sheikh wrote: “I want to take this opportunity to comment on a technical statistical issue which has become the subject of media discussion. This relates to the question of whether a voluntary survey can become a substitute for a mandatory census. It cannot”.
Unmoved by the protests and complaints from a myriad of organizations that rely on quality census data, the government went ahead with the NHS for the 2011 Census. The 2011 census still featured the basic 8-question short form (distributed to every household), but the traditional mandatory long-form was replaced by the new voluntary household survey. In an attempt to salvage a decent response rate, distribution of the 2011 household survey was increased to one-in-three households instead of the traditional one-in-five.
So what did the 2011 NHS results look like?
More than 98% of Canadian households responded to the short-form census (a better response rate than in 2006), but the response rate to the new voluntary NHS was just 68.6% compared with the typical response rate for the mandatory long-form which has consistently been around 94%. Even more troubling, a number of communities had a response rate lower than 25% and some communities simply did not respond at all. As a result, StatsCan had to sharply lower the point at which results are suppressed at the neighborhood level. In 2006, the point of suppression was 25%, but for the 2011 NHS it was 50%. In spite of these changes to the suppression rate, StatsCan was unable to report data for approximately 25% of the census subdivisions or municipalities.
However, if the NHS were held to the standards of the 2006 census, the responses from 67% of Canadian neighborhoods would not be considered reliable, and therefore excluded. Unfortunately, this means the NHS data is not reliable at the community level for planning purposes and the new methodology renders the NHS incompatible with previous mandatory long-form censuses for comparison purposes.
Because of its breadth and high-response rate, the mandatory long-form census has consistently been one of the most reliable data sources in Canada. Because this data is considered representative, data from the mandatory long-form census has been used as an anchor, reducing the risk of bias in other StatsCan surveys. All of this changed in 2011 with the introduction of the NHS.
The Impact of the NHS on Retailers
The NHS has had a significant impact on many industries that count on reliable census data to make key decisions. John Rose, the City of Edmonton’s chief economist said; “In an era of tighter government budgets and uncertainty from falling oil prices, “once you’ve made the decision to invest … and you find out the numbers are wrong, how do you unmake them? We want to make sure we get it right in these days where every capital dollar is constrained.”
This is the exact situation most Canadian retailers are faced with today. There’s an increasing pressure to make every dollar count when making location decisions. Retail is becoming more and more polarized as the gap between rich and poor widens. Losing visibility into these extremes with the 2011 NHS means that critical real estate decisions have been made without understanding the true picture on the ground. The problem is even more troubling for retailers who don’t have a loyalty program where customer behavior can be tracked. The sad reality is we know less about Canada today than we did about Canada back in 2006.
It will be interesting to see if the response rate of the upcoming 2016 census is affected by the missteps of 2011. As someone who works with census data every day, I will be watching closely as the results start to trickle in. No matter what industry, I understand a robust and unbiased data sample is mandatory for reliable decision making. So everyone be sure to fill-out and send in your census forms this year, no excuses!
Learn more about how Tango helps retailers and restaurant companies make better location decisions.
