Catalogue no. 12-586-X
ISBN 978-0-660-08114-4
Statistics Canada’s
Quality Assurance Framework
Third Edition - 2017
Release date: April 21, 2017
Revision date: July 19, 2017
Standard table symbols
The following symbols are used in Statistics Canada
publications:
. not available for any reference period
.. not available for a specic reference period
... not applicable
0 true zero or a value rounded to zero
0
s
value rounded to 0 (zero) where there is a meaningful
distinction between true zero and the value that was rounded
p
preliminary
r
revised
x suppressed to meet the condentiality requirements
of the Statistics Act
E
use with caution
F too unreliable to be published
* signicantly different from reference category (p < 0.05)
How to obtain more information
For information about this product or the wide range of services and data available from Statistics Canada, visit our website,
www.statcan.gc.ca.
You can also contact us by
email at STATCAN.infostats-infostats.STA[email protected]
telephone, from Monday to Friday, 8:30 a.m. to 4:30 p.m., at the following numbers:
• Statistical Information Service 1-800-263-1136
• National telecommunications device for the hearing impaired 1-800-363-7629
• Fax line 1-514-283-9350
Depository Services Program
• Inquiries line 1-800-635-7943
• Fax line 1-800-565-7757
Note of appreciation
Canada owes the success of its statistical system to a
long-standing partnership between Statistics Canada, the
citizens of Canada, its businesses, governments and other
institutions. Accurate and timely statistical information could not
be produced without their continued co-operation and goodwill.
Standards of service to the public
Statistics Canada is committed to serving its clients in a prompt,
reliable and courteous manner. To this end, Statistics Canada has
developed standards of service that its employees observe. To
obtain a copy of these service standards, please contact Statistics
Canada toll-free at 1-800-263-1136. The service standards are
also published on www.statcan.gc.ca under “Contact us” >
“Standards of service to the public.”
Published by authority of the Minister responsible for Statistics Canada
© Minister of Industry, 2017
All rights reserved. Use of this publication is governed by the Statistics Canada Open Licence Agreement.
An HTML version is also available.
Cette publication est aussi disponible en français.
Document revision control
Revision no.
Date of issue
[dd-mm-yyyy] Author(s) Brief description of change
1.0 21-04-2017 Quality Secretariat (QS) Approved version
2.0 11-07-2017 QS • “Directive on Obtaining Access to Records Maintained by Other Organizations” was updated to “Directive
on Obtaining Data under the Statistics Act”
• “Guidelines for the Use of Publicly Available Data” was updated to “Guidelines on data available to the
public”. Corresponding updates to text were made to reflect this change of terminology.
• “Directive on Record Linkage” was updated to “Directive on Microdata Linkage”
• “Generic Record Linkage Process Model” was updated to “Record Linkage Project Process Model”
Statistics Canada – Catalogue no. 12‑586‑X
3
Statistics Canada Quality Assurance Framework
Table of contents
Preface ......................................................................................................................................... 4
Introduction ................................................................................................................................. 5
Guiding Principles ...............................................................................................................5
Background to the QAF ......................................................................................................7
Organization of the QAF ......................................................................................................7
The Corporate Environment ....................................................................................................... 8
A. Quality commitment ........................................................................................................8
B. Sound implementation of statistical methods ..............................................................13
C. Assurance of condentiality, privacy and security .......................................................17
The Statistical Program ............................................................................................................ 22
D. Management of input data and relations with data providers ......................................22
E. Allocation and management of resources ....................................................................27
F. Management of relations with data users and stakeholders .........................................31
Statistical Outputs..................................................................................................................... 35
G. Relevance .....................................................................................................................35
H. Accuracy and reliability .................................................................................................38
I. Timeliness and punctuality .............................................................................................41
J. Accessibility and clarity .................................................................................................44
K. Coherence and comparability .......................................................................................48
L. Interpretability and management of metadata ..............................................................52
References ................................................................................................................................. 55
Statistics Canada – Catalogue no. 12‑586‑X
4
Statistics Canada Quality Assurance Framework
Statistics Canada Quality Assurance Framework
Preface
The role of Statistics Canada is to provide credible and relevant statistical information to the public, to inform
decision-making and to help Canadians better understand their country. The trust of the Canadian people is
earned and maintained through transparent and responsible management of statistical information and of the
statistical system that produces it.
The Quality Assurance Framework (QAF) serves as the highest-level governance tool for quality management at
Statistics Canada. The QAF describes, in general terms, the measures the Agency
1
has put in place to manage
quality and provides guidance to statistical program areas as they develop and implement quality management
strategies to meet their users’ needs. The QAF is used in conjunction with Statistics Canada management
practices, such as those described in the Agency’s Quality Guidelines.
Simply stated, the QAF sets out the principles that guide the work of Statistics Canada. It may also serve as a
useful reference for those interested in the production of ofcial statistics. I encourage its use and support its
regular revitalization.
I offer my thanks and congratulations to the numerous Statistics Canada experts who have contributed to the
production of the 3
rd
edition of the Quality Assurance Framework, as well as to the Quality Secretariat who led and
coordinated this effort under the guidance of the Methods and Standards Committee.
Anil Arora
Chief Statistician
1.
The term “The Agency” refers to Statistics Canada.
Statistics Canada – Catalogue no. 12‑586‑X
5
Statistics Canada Quality Assurance Framework
Introduction
Statistics Canada’s mission statement “Serving Canada with high-quality statistical information that matters”
expresses the Agency’s
2
mandate to provide all sectors of Canadian society with access to a trusted source that
serves their information needs. Statistics Canada denes the quality of its ofcial statistics in terms of their tness
for use. Maintaining condence in the Agency through the management and assurance of quality is essential to
the success of Statistics Canada.
The Quality Assurance Framework (QAF) describes the strategies Statistics Canada has put in place to facilitate
and ensure effective management of quality in all its statistical programs and organizational initiatives. Underlying
these strategies are eight guiding principles.
Guiding Principles
Quality is multi-dimensional
Statistics Canada has identied six dimensions of statistical information to dene its quality and evaluate its tness
for use.
Relevance reects the degree to which statistical information meets user needs.
Accuracy reects the degree to which statistical information correctly describes the phenomena it was designed
to measure.
Timeliness refers to the delay between the end of the reference period to which statistical information pertains
and the date on which the information becomes available.
Accessibility refers to the ease with which statistical information can be obtained.
Coherence reects the degree to which statistical information is logically consistent and can be brought together
with information from other sources or different time periods.
Interpretability reects the availability of supplementary information (metadata) necessary to understand, analyze
and utilize statistical information appropriately.
The six dimensions are overlapping and interrelated and achieving an appropriate level of quality in all dimensions
is required, as failure in any one of them will compromise the tness for use of an information product. Statistics
Canada strives continuously to nd innovative methods and data sources that can lead to achieving higher levels
of quality in one or more dimensions without adversely impacting others.
Quality is relative, not absolute
Management of quality must be in conjunction with other important factors including the data needs of users
and stakeholders, costs and response burden. As when managing the dimensions of quality, Statistics Canada
counts on innovations, in areas such as data integration, to fulll user needs with high quality data at lower
cost and lessened response burden. It also recognizes that effective management of quality does not demand
maximization of quality over all other factors. Rather, it is the result of striking an appropriate balance between the
resources available to the Agency and the information needs of its data users and stakeholders. Efforts to improve
the quality of ofcial statistics take into account factors such as existing budgets, availability of specialized
resources and response burden.
Every employee has a role to play in assuring quality
Statistics Canada’s management of quality reects the principle, as stated by renowned American statistician W.E.
Deming, that “Quality comes not from inspection, but from improvement of the production process.”
3
That is, it is
not possible to achieve quality by merely “inspecting” a nal product. Rather, quality must be built into processes
from the outset. Success in assuring quality at Statistics Canada requires the sound application of knowledge
2.
The term “The Agency” refers to Statistics Canada.
3.
W. Edwards Deming (1982) Out Of The Crisis. The MIT Press. (2
e
ed. 1986), p 29.
Statistics Canada – Catalogue no. 12‑586‑X
6
Statistics Canada Quality Assurance Framework
and expertise by employees at all levels within the Agency – in short, quality is “everyone’s business” at Statistics
Canada. An essential component of this strategy is recruitment and professional development programs that lead
to a motivated and competent workforce.
Quality must be built in at each phase of the process
As operations at all stages can impact the quality of outputs, effective quality assurance requires measures at
multiple phases of the statistical process and consideration of the impact of each phase on the process as a
whole. Modelling the statistical process by dividing it into phases has proven to be an effective management tool.
One such reference framework is the Generic Statistical Business Process Model, in which the principal phases
are “specify needs”, “design”, “build”, “collect”, “process”, “analyze”, “disseminate” and “evaluate”. A quality
management structure can be conceptualized by considering each cell of the matrix dened by these phases and
the six dimensions of quality. It is important to note that effective management of quality does not necessitate
similar measures at all phases.
Balancing the dimensions of quality is best achieved through a team approach
The use of multidisciplinary teams ensures that the dimensions of quality and other important factors including
cost and user needs are effectively managed. Subject-matter experts bring knowledge of content, stakeholder
needs and relevance while mathematical statisticians (methodologists) contribute a sound foundation in statistical
methods and expertise regarding accuracy. Operations personnel have experience in collection and processing
methods, are well-placed to consider practicality and efciency, and represent eld staff and respondents.
Systems personnel ensure the informatics point of view is considered in design and implementation, and bring
specialist knowledge of technology standards and tools. Teams are supported by a committee of senior managers
who provide a forum for resolving issues and, when necessary, give guidance related to data quality trade-offs,
especially with respect to timeliness and cost.
Quality assurance measures must be adapted to the specific program
At Statistics Canada, responsibility and accountability for quality assurance lie with the Agency’s statistical
programs. The Agency develops and maintains quality management strategies and tools that program areas
can adapt to their individual needs. Within individual programs the challenge is to achieve an appropriate level of
quality by effectively balancing program objectives, evolving user and stakeholder needs, costs, response burden,
and the various dimensions of quality.
Users must be informed of data quality so that they can judge whether the statistical information is
appropriate for their particular use
Some dimensions of quality, such as timeliness, can be observed directly by users. However for most other
dimensions, users require objective information about data quality to evaluate tness for use. Often, the Agency
is the sole source of such information. Both quantitative measures, such as coefcients of variation and response
rates, and qualitative information, such as a description of sources of error, are necessary.
Quality assurance is a continuous practice
There is constant evolution of the social and economic conditions in Canada, as well as in the user and
stakeholder environments. Consequently, quality is not self-sustaining and will deteriorate in the absence of
regular review and refreshment. In particular a “relevance gap” may open if ofcial statistics do not keep pace with
the changing needs of the Agency’s users and stakeholders. It is further incumbent on Statistics Canada to ensure
its methods remain at the forefront of those used by national statistical ofces. This is achieved through a culture
that promotes a continuous search for new and innovative sources and methods.
Statistics Canada – Catalogue no. 12‑586‑X
7
Statistics Canada Quality Assurance Framework
Background to the QAF
The Statistics Canada QAF was rst produced in 1997 and was updated in 2002. This 3
rd
edition was inspired by
the generic National Quality Assurance Framework template developed by a United Nations Statistics Division
Expert Group. In particular, this version expands the scope of the Statistics Canada QAF by discussing quality
management in the Agency’s corporate environment and statistical programs.
In the QAF, several references are made to the organizational structure and operating procedures of Statistics
Canada. An overview of the functional (reporting) structure of the Agency is given below.
• Statistics Canada is headed by the Chief Statistician, an appointment at the deputy minister level within
the Government of Canada.
• The Chief Statistician is supported by an Executive Management Board, which consists of assistant chief
statisticians who each represent one field of the Agency. A eld covers a broad grouping of commitments,
such as those related to economic statistics or to corporate services.
• Fields are sub-divided into branches, and each branch is made up of divisions. Each division is led by
a director.
Organization of the QAF
The QAF consists of twelve stand-alone chapters, each dened by a quality management theme. All chapters
share a common three-part approach. The rst, Description, introduces concepts to be discussed in the chapter,
denes relevant terms and provides context and background information. Issues and factors that impact the
Agency’s capacity to achieve goals related to the concepts in the chapter are discussed, including relationships to
concepts described in other chapters. Assessment lists objectives essential to successful operationalization of the
concepts in the chapter. Objectives are expressed in generic and qualitative terms, and represent ideals to which
the Agency strives. Implementation demonstrates achievement of these objectives through references to specic
Statistics Canada activities that contribute to quality, grouped by the objectives in Assessment.
Statistics Canada – Catalogue no. 12‑586‑X
8
Statistics Canada Quality Assurance Framework
The Corporate Environment
A. Quality commitment
Description
Trust in the integrity of a national statistical ofce (NSO) is essential. A crucial element of this trust is belief that the
NSO’s choices are based on sound criteria and are motivated by statistical considerations, and that the NSO is
insulated from undue inuence of data users, funding partners and other stakeholders.
The term quality commitment is used to describe values, policies, decisions and practices consistent with the
ideals and culture described above. Realizing quality goals requires a combination of NSO-wide and program-
specic initiatives. Effective management of quality strikes an appropriate balance between the resources
available to the NSO and the information needs of its data users and stakeholders. Many of the forces that
threaten the ability of an NSO to maintain its quality commitment are beyond its control. The economy and society
to which ofcial statistics pertain are changing at an unprecedented pace, and the demand for information about
them, both in breadth and depth, is growing as well. Meanwhile, in a continuing long-term trend, households
and businesses are becoming less inclined to participate in surveys. Advancements in technology and changes
in behaviour, combined with the availability of alternate sources of statistical information, contribute to the
expectation of greater efciency and expediency. Additionally, the emergence of new priorities demanding public
funds have increasingly required NSOs to fund improvements and new initiatives through savings harvested
internally. It is therefore incumbent upon the Agency to be continuously looking for innovative methods and new
data sources to allow it to fulll its mandate. In spite of all these pressures, an NSO must be managed such that
the institutional environment entrenches drivers of quality. In this way, the NSO can continue to uphold and sustain
its quality commitment.
Assessment
Statistics Canada’s adherence to its quality commitment is assessed by evaluating the extent to which the
Agency:
1. ensures its employees function to the highest professional standards in terms of both competencies
and ethics
2. promotes transparency, impartiality, objectivity and professional independence through its business
practices and organizational structure
3. bases its programs on sound methods and scientic principles, and ensures they are consistent with
recognized standards and best practices and are supported by a statistical methods group
4. follows through on its quality commitment operationally, throughout the statistical process
5. ensures that exogenous factors and other unanticipated changes do not hinder its capacity to meet its
quality commitment
6. utilizes objective and transparent criteria to evaluate and monitor the degree to which it fullls its
quality-related commitments in its programs and outputs.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to stand behind and follow through on its quality
commitment in its statistical programs. Below is a list of such initiatives, in groups that correspond to the items
under Assessment.
Statistics Canada – Catalogue no. 12‑586‑X
9
Statistics Canada Quality Assurance Framework
A.1 Statistics Canada ensures its employees function to the highest professional standards
in terms of both competencies and ethics
• Provide guidance to all staff on their responsibilities and obligations and on expectations of them as
Statistics Canada employees, especially pertaining to condentiality, privacy and security. Related
management initiatives include: overarching policies and codes; a binding commitment through oath;
training and certication on condentiality, privacy and security; and guidelines for proper use of electronic
networks.
f Code of Conduct
f Values and Ethics Code (http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=25049)
f Policy on Conict of Interest and Post-Employment
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=25178)
• Coordinate recruitment campaigns to identify and attract professionally competent and motivated
candidates from the relevant academic disciplines. The processes include national recruitment teams
and campaigns for professional groups including mathematical statisticians (methodologists), computer
scientists, economists and social scientists.
f Integrated Business and Human Resources Plan
• Staff vacancies through objective and fair selection processes, to benet both employees and the Agency.
Statistics Canada: utilizes generic competitions at regular intervals; sets out objective criteria to assess
knowledge, abilities and personal suitability and to assign successful candidates to openings; and ensures
the participation of senior managers on selection committees that evaluate candidates.
• Foster the professional and personal development of staff through learning opportunities, performance
feedback and developmental opportunities. Statistics Canada’s policies outline its commitment to
continuous learning. Learning is promoted through development programs for new recruits, a vast
catalogue of courses delivered in-house (including a set of required courses) and by the Canada School of
Public Service, access to courses at local universities and colleges and support for completion of degree
programs. Further, guidance on professional development is provided through regular performance review,
mentoring and support activities. Opportunities for employees to expand their professional experience are
available through deployment, special assignments and exchanges.
f Policy on Learning, Training and Development
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12405)
• Maintain collaboration and exchanges with the scientic, academic and business communities through
participation of employees in professional associations and other peer gatherings.
• Stimulate creativity, innovation and the promotion of professionalism, particularly at the “grassroots” level.
f Innovation Channel
• Build a positive, exciting and healthy workplace through employee wellness programs.
A.2 Statistics Canada promotes transparency, impartiality, objectivity and professional
independence through its business practices and organizational structure
• Convey information regarding the Agency’s priorities, initiatives and progress to the public, stakeholders
and employees. Various reports are listed on its website, including its report to Parliament, the Corporate
Business Plan and internal audit reports.
f Departmental Reports (http://www.statcan.gc.ca/eng/about/reports2)
Statistics Canada – Catalogue no. 12‑586‑X
10
Statistics Canada Quality Assurance Framework
• Commit to providing all Canadians with equitable and timely access to Statistics Canada’s data products
and information releases. The Agency publishes release dates well in advance. The pre-planned release
schedule is publicly available and any pre-release access is communicated transparently. Procedures
exist for cases where data are released prematurely. Further, it is required that all releases pass through
a uniform dissemination service, where The Daily is Statistics Canada’s ofcial release bulletin. Policies
ensure that communications are effectively managed and are responsive to information needs of the public.
f Policy on Ofcial Release
f Directive on the Distribution of Registered Statistics Canada Products
f Release schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
f The Daily (http://www.statcan.gc.ca/dai-quo/index-eng.htm?HPA)
f Policy on Communications and Federal Identity
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=30683)
f Directive on the Security of Sensitive Statistical Information
f Information and Privacy Breach Protocol (http://www.statcan.gc.ca/eng/about/pia/generic/section8)
• Produce and disseminate standard methodological documentation. Statistics Canada policies describe the
procedures for communicating data quality information. These guidelines translate into the development
of an integrated metadata repository where denitions, data sources and methods for each survey and
statistical program are published. As well, user guides accompany a data release to provide background
details on the data and methods. Additionally, technical reports explain procedures used in the creation of
the statistics.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Integrated Metadatabase
f Denitions, Data Sources and Methods (http://www.statcan.gc.ca/eng/concepts/index)
A.3 Statistics Canada bases its programs on sound methods and scientific principles,
and ensures they are consistent with recognized standards and best practices and are
supported by a statistical methods group
• Base choices of information sources and statistical methods and decisions regarding dissemination on
sound criteria motivated by statistical considerations.
• Foster innovation through a statistical methods research and development program that is centrally funded.
f Methodology Research Block Fund
• Share expertise within the Agency and externally to evaluate and improve new methods, using accepted
scientic principles. This sharing is facilitated through committees that confer on matters relating to the
utilization of efcient statistical methods, peer review of papers and presentations and publication of an
Agency journal on statistical methods.
f Advisory Committee on Statistical Methods
f Policy on Peer and Institutional Review
f Survey Methodology Journal
• Put in place policies and procedures to ensure that standard concepts, denitions and classications
are consistently applied throughout the Agency. For example, an Agency-level committee advises on the
development and application of statistical standards and metadata.
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
f Methods and Standards Committee
• Provide expertise and leadership on design and implementation of sound statistical and survey methods.
At Statistics Canada, such services are provided to statistical program areas by a core statistical
methods group that is centrally located within the Agency. The Methodology Branch consists of a group
Statistics Canada – Catalogue no. 12‑586‑X
11
Statistics Canada Quality Assurance Framework
of mathematical statisticians (methodologists) with education and experience in relevant disciplines. In
addition to guidance on statistical and survey methods, the Methodology Branch also plays a leading role in
promoting and supporting quality assurance within the Agency.
A.4 Statistics Canada follows through on its quality commitment operationally, throughout
the statistical process
• Provide tools and support to foster building quality into all aspects of statistical programs and to ensure all
employees are aware of and fulll their responsibilities regarding the quality of processes and products.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Quality Assurance Framework
f Directive and Guidelines for the Validation of Statistical Outputs
f Quality Secretariat
• Assist statistical program areas in the implementation of sound methods through the statistical methods
group. This group plays a leadership role in promoting common methodologies and generalized solutions.
The group guides program areas in the proper integration of new methods through adherence to
recognized procedures such as pilot tests and parallel runs. Members of the statistical methods group form
part of multidisciplinary project teams. As well, centres of expertise, or resource centres, exist for assisting
with specic issues (e.g., Time Series Research and Analysis Centre).
f Generalized Systems Resource Centre
f Resource and support centres
A.5 Statistics Canada ensures that exogenous factors and other unanticipated changes do
not hinder its capacity to meet its quality commitment
• Establish a consistent process for submission, evaluation and funding allocation of proposals. The planning
process considers a long-term horizon. It works on an annual planning cycle and covers all stages of
planning where nancial, human resources and informatics needs are all considered together. It includes a
review of corporate priorities and integrates risk management, investment planning, and evaluation into the
planning process.
f Integrated Strategic Planning Process
f Integrated Business and Human Resources Plan
f Information Technology Plan
• Forecast regular and strategic investments necessary to preserve the quality and continuity of statistical
programs over time. This forecast covers an extended horizon (10 years) and considers the cyclical nature
of specic statistical programs (e.g., ve-year Census cycle, 10-year redesign cycle for certain surveys)
when planning redesigns and other program modications or enhancements. A separate fund exists for
initiatives that cannot be covered by program area base budgets.
f Continuity and Quality Maintenance Investment Plan
f Continuity and Quality Management Fund
f Investment Proposals
• Create corporate initiatives to assure quality in light of changing environments. Corporate committees
evaluate impending changes and decide on a course of action. As well, program areas document Business
Continuity Plans for use in case of a business interruption.
f Human Resources Committee
f Information Technology Architecture Committee
f Security Coordination Committee
Statistics Canada – Catalogue no. 12‑586‑X
12
Statistics Canada Quality Assurance Framework
• Implement effective project management by providing tools and guidelines. A centralized ofce provides
support in the development of common processes and tools to improve the timely delivery of projects in
a cost-effective fashion while adhering to quality standards and meeting client needs. As a result, a set of
standard project management processes, templates and tools is in place.
f Departmental Project Management Ofce
f Departmental Project Management Framework
A.6 Statistics Canada utilizes objective and transparent criteria to evaluate and monitor the
degree to which it fulfills its quality-related commitments in its programs and outputs
• Evaluate statistical program areas regularly through internal audits, quality reviews and other techniques.
Through the reviews, strengths and weaknesses are identied and current best methods are shared
corporately. As well, feedback from data users and stakeholders is solicited and used to guide program
areas. As a result, risk mitigation strategies are strengthened. External audits of statistical programs and
benchmarking to external indicators are conducted as required.
f Audit and Evaluation Plan
• Ensure measures of quality are made available to data users.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Directive on Documenting Statistical Metadata
• Use objective measures to track differences between quality targets and realized values, such as producing
corporate performance indicators and monitoring of corrections to data releases.
Statistics Canada – Catalogue no. 12‑586‑X
13
Statistics Canada Quality Assurance Framework
B. Sound implementation of statistical methods
Description
Sound implementation of statistical methods means that the design is carried out as specied, errors are
avoided and proper verication and documentation are completed. Sound implementation of statistical methods
also requires alignment with current best methods and standards. The resulting coherence and consistency
enhance relevance and utility.
Statistics Canada develops mandatory rules and guidelines that govern the acquisition, production and
dissemination of statistical information. These rules and guidelines are referred to as “statistical standards”, and
their purpose is to ensure uniformity across statistical programs. The Agency also encourages the use of common
tools, approved methods and other current best methods in order to improve coherence and comparability,
reduce the cost of design and production, and increase robustness and simplicity.
Assessment
Statistics Canada’s sound implementation of statistical methods, as well as the use of statistical standards and
current best practices, are assessed by evaluating the extent to which the Agency:
1. ensures correct implementation of survey designs and of mitigation of risks to quality
2. promotes and supports the use of statistical standards by program areas
3. contributes to the development and use of conceptual frameworks, standard names and denitions, and
best practices
4. promotes and supports the use of consistent data acquisition, processing and dissemination methods
across its statistical programs
5. effectively manages and communicates information regarding redesigns and other changes to statistical
methods.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to ensure sound implementation of statistical
methods and to promote the use of statistical standards and current best practices in its statistical programs.
Below is a list of such initiatives, in groups that correspond to the items under Assessment.
B.1 Statistics Canada ensures correct implementation of survey designs and of mitigation
of risks to quality
• Implement a comprehensive “matrix” approach to quality management by considering quality assurance
at all phases of the statistical process and by considering the multi-dimensional nature of quality. For
example, program areas identify and verify that changes in variable denitions, international standards,
parameter values and other inputs are correctly incorporated into operations and systems at the beginning
of each processing cycle. They monitor collection and processing operations, by using process control and
other methods, and make adjustments as required. Moreover, they reconcile estimates and outputs with
other comparable statistical and administrative sources on similar subjects, including previous estimates
from the same statistical program, and conduct comprehensive post-release program review and analysis.
f Generic Statistical Business Process Model
(http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0)
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Quality Assurance Framework
f Directive and Guidelines for the Validation of Statistical Outputs
Statistics Canada – Catalogue no. 12‑586‑X
14
Statistics Canada Quality Assurance Framework
• Implement effective project management by providing tools and guidelines. A centralized ofce provides
support in the development of common processes and tools to improve the timely delivery of projects in
a cost-effective fashion while adhering to quality standards and meeting client needs. As a result, a set of
standard project management processes, templates and tools is in place.
f Departmental Project Management Ofce
f Departmental Project Management Framework
• Use a multidisciplinary team approach to ensure expertise in all areas, effective exchange of ideas and
balanced decision-making. Typically, subject-matter experts are responsible for content, stakeholder
needs and relevance. Mathematical statisticians (methodologists) contribute a solid foundation in statistical
methods and expertise in accuracy and its relation to other dimensions of quality. Operations personnel
have experience in collection and processing methods, and bring knowledge of eld work, respondent
relations and cost. Systems experts ensure informatics considerations are represented and bring specialist
knowledge of technology standards and tools.
• Identify and thoroughly understand potential sources of error and of breach in quality. Risk mitigation
strategies are integrated into survey design and implementation. Mitigation of risks to quality is assessed
and reported periodically.
B.2 Statistics Canada promotes and supports the use of statistical standards by
program areas
• Develop and maintain a protocol for statistical standards. Promote and monitor the adoption of statistical
standards uniformly across statistical programs. Statistical standards apply to concepts, denitions,
frameworks, units, variables, classication systems, protocols and target populations. Exceptions to
statistical standards are justied and documented.
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
• Develop, maintain and disseminate statistical metadata in a corporate repository. The elements in the
repository are updated, changes are tracked and users are informed.
f Denitions, Data Sources and Methods (http://www.statcan.gc.ca/eng/concepts/index)
f Integrated Metadatabase
f Directive on Documenting Statistical Metadata
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
• Support statistical standards with administrative and governance structures and tools. A corporate
committee assists and advises on the development, approval and application of statistical standards
and metadata within the Agency’s programs. A specic division is responsible for all the classications
and standards - including industry, product, occupation and education classications, geography and
economic accounts - used by Statistics Canada.
f Corporate Management Framework (http://www.statcan.gc.ca/eng/about/cmf)
f Methods and Standards Committee
f Standards Division
f Standards Bulletin
• Provide guidance on the interpretation of statistical standards. If appropriate, dene classes of standards
that differ by the degree of obligation to adhere to them (i.e., compulsory Statistics Canada standards,
recommended standards and program-specic standards).
f Denitions, Data Sources and Methods (http://www.statcan.gc.ca/eng/concepts/index)
Statistics Canada – Catalogue no. 12‑586‑X
15
Statistics Canada Quality Assurance Framework
B.3 Statistics Canada contributes to the development and use of conceptual frameworks,
standard names and definitions, and best practices
• Participate in the development of national and international standards and other frameworks, and
encourage consistency with them. Statistics Canada participates in numerous development groups, such
as those of the United Nations Statistics Division and the United Nations Economic Commission for Europe.
The Agency is also involved in groups developing standard classication systems such as the North
American Industrial Classication System and the National Occupational Classication-Statistics.
f Denitions, Data Sources and Methods (http://www.statcan.gc.ca/eng/concepts/index)
• Participate in various external fora, such as working groups or subject-matter expert groups, to work with
other national statistical ofces in recognizing and promoting national and international best practices and
cutting-edge methods.
f International Cooperation (http://www.statcan.gc.ca/eng/about/cooperation)
• Set up internal cross-program committees to ensure that quantities being estimated bear relation to each
other through use of consistent terminology, commonly formulated questions and comparable variable
denitions. In particular, maximize consistency both within (e.g., between quarterly and annual estimates,
and between preliminary and nal estimates) and across statistical programs.
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
f Policy on the Review and Testing of Questionnaires
f Questionnaire Design Resource Centre
• Ensure that program areas periodically assess compliance with standards and frameworks as well as
consistency and comparability with related administrative data, other estimates from Statistics Canada and
estimates from elsewhere.
B.4 Statistics Canada promotes and supports the use of consistent data acquisition,
processing and dissemination methods across its statistical programs
• Use standardized frameworks and systems to support process management.
f Generic Statistical Business Process Model
(http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0)
f Generic Statistical Information Model
• Develop and use common frames and processing environments. Examples of these are the Business
Register, Household Survey Frame Service, Social Survey Processing Environment and the Integrated
Business Statistics Program.
• Optimize the use of generalized systems, resource centres and centralized services.
f Generalized Systems Resource Centre
f Resource and support centres
• Ensure internal consistency of outputs during operations. In particular, ensure that arithmetic or accounting
operations do not lead to discrepancies (e.g., rounding) and that outputs of complementary processes are
integrated properly.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Directive and Guidelines for the Validation of Statistical Outputs
• Implement an appropriate structure for approval and testing when developing and implementing new
software applications and other processing tools. Develop appropriate guidelines and other oversight tools
and structures, as needed.
Statistics Canada – Catalogue no. 12‑586‑X
16
Statistics Canada Quality Assurance Framework
• Ensure consistency of message and convenient user access through a corporate dissemination framework
and access tools. Examples of such tools are The Daily and tables from the Agency’s socioeconomic
database CANSIM.
f Statistics Canada Dissemination Model
f Directive on Media Relations
f Directive on Corrections to Daily Releases and Statistical Products
f The Daily (http://www.statcan.gc.ca/dai-quo/index-eng.htm?HPA)
f CANSIM (http://www5.statcan.gc.ca/cansim/a01?lang=eng)
• Require that program areas collect and store data required for post-release program review and analysis.
B.5 Statistics Canada effectively manages and communicates information regarding
redesigns and other changes to statistical methods
• Produce quality reports and other supporting methodological information needed for users to make
meaningful comparisons over time and with other sources of information. Program areas provide users with
information on changes to concepts, denitions, classications and methods, as well as deviations from
international standards and other practices.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Directive on Documenting Statistical Metadata
f Directive on the Management of Aggregate Statistics
• For redesigns and other signicant changes in survey methodology or international standards, explain
breaks in series and develop methods for reconciliation. Produce historical revisions when deemed
appropriate. For example, a data series is updated when a classication framework is revised.
• Solicit feedback from users, in particular their experience in producing historical series, comparing
estimates between statistical programs, and comparing the Agency’s outputs with those of other sources.
Identify inadequacies and issues.
Statistics Canada – Catalogue no. 12‑586‑X
17
Statistics Canada Quality Assurance Framework
C. Assurance of confidentiality, privacy and security
Description
A national statistical ofce (NSO) is entrusted with the authority to compel individuals, businesses and institutions
to provide information. Implicit in this provision are certain obligations regarding the NSO’s “stewardship”
4
of
information provided to it. The NSO must justify the benets of obtaining information and minimize the burden that
its collection imposes, in terms of both invasion of privacy and demands on respondent time and effort. The NSO
must also safeguard its data holdings, in particular information attributable to individual respondents, from
unauthorized disclosure, access or use.
Three important concepts in data stewardship are condentiality, privacy and security. Confidentiality is the
protection of information attributable to an identiable unit
5
from unauthorized disclosure. Privacy, in the statistical
context, is the protection of respondents from intrusive enquiries and the provision that individuals have some
access to and control over information about themselves. Security refers to the policies and practices NSOs
use to prevent condential information from being improperly disclosed, accessed or used. Disclosure control
refers to the development and implementation of measures that identify and address data items that could expose
condential information in outputs such as data tabulations and public-use microdata les. Examples of disclosure
control measures include cell suppression and data perturbation.
The Statistics Act enshrines the authority of Statistics Canada to collect, process and publish statistical
information. It also prescribes the responsibilities of the Agency to protect the condentiality of identiable
individual responses. The Statistics Act gives Statistics Canada authority to obtain information from citizens
and businesses through mandatory surveys, and also guarantees the Agency access to records maintained by
organizations. The Statistics Act also provides the Agency with discretion to authorize the release, by executive
order, of data that would otherwise be protected.
Other federal laws, primarily the Privacy Act and the Access to Information Act, establish the responsibilities of
all federal departments regarding personal information held in government records. The Privacy Act requires that
individuals be informed of the authority under which data are collected and the intended uses of the information.
Information collected by Statistics Canada is for statistical purposes, dened as description or analysis of
characteristics of a population to which the individual belongs. As the Privacy Act also recognizes statistical
purposes as a legitimate secondary use of information, Statistics Canada is able to additionally obtain information
collected elsewhere for other purposes. In particular, the Agency is able to obtain information collected for
administrative
6
purposes. However the Privacy Act dictates that information collected for statistical purposes must
not be used for administrative purposes. The Access to Information Act ensures that respondents have the right to
obtain information they provided to Statistics Canada and other federal departments.
Assessment
Statistics Canada’s efforts pertaining to condentiality, privacy and security are assessed by evaluating the extent
to which the Agency:
1. develops and implements policies and practices to fulll the obligations of the Statistics Act, the Privacy
Act, the Access to Information Act and any other statutes
2. minimizes the intrusiveness of data collection through use of alternate sources of information and effective
design of data collection instruments
3. facilitates response by informing respondents of their rights and obligations, the importance of responding
and the protection their data will be afforded
4. implements organizational, physical and technological measures to protect the condentiality and ensure
the security of statistical data
4.
See “Data Stewardship at Statistics Canada” by Gordon Brackstone and Pamela White.
5.
Examples of units include persons, households, enterprises and institutions.
6.
Administrative purposes are those related to making decisions about the individual to whom the information pertains. An example is determining eligibility for a government program based
on financial information pertaining to a specific person, household or business.
Statistics Canada – Catalogue no. 12‑586‑X
18
Statistics Canada Quality Assurance Framework
5. provides training and other measures to instill a culture that promotes awareness of issues and obligations
related to condentiality, privacy and security
6. implements effective disclosure control measures, in particular through extensive verication of outputs.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to protect condentiality, minimize the impact
on privacy and provide data security in its statistical programs. Below is a list of such initiatives, in groups that
correspond to the items under Assessment.
C.1 Statistics Canada develops and implements policies and practices to fulfill the
obligations of the Statistics Act, the Privacy Act, the Access to Information Act and any
other statutes
• Develop policies to ensure the Agency carries out its responsibilities and obligations, as a federal institution,
regarding privacy, condentiality and security.
f Directive on Access to Information and Privacy
f Directive on Sensitive Statistical Information
f Policy on Access to Information
(http://www.tbs-sct.gc.ca/hgw-cgf/oversight-surveillance/atip-aiprp/ai/index-eng.asp)
f Policy on Privacy Protection (http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=12510)
f Companion guide to the Statistics Act
• Dene directives regarding the release of identiable information by executive order (discretionary release),
as provided in the Statistics Act. One justication for discretionary release is permission provided by
respondents to release data about themselves (waivers).
f Directive on Discretionary Disclosure
f Obtaining a Discretionary Disclosure: Guidelines
f Directive on Informing Survey Respondents
C.2 Statistics Canada minimizes the intrusiveness of data collection through use of
alternate sources of information and effective design of data collection instruments
• Before undertaking any new collection, determine if equivalent data are available from other sources.
f Directive on Obtaining Administrative Data under the Statistics Act
• Reduce response burden through data collection and processing initiatives such as record linkage, data
sharing or joint data collection activities (where surveys that collect similar or related data are integrated or
combined). Study and implement innovative methods, for example electronic collection, satellite imaging
and scanner data.
f Business Response Management Committee
f Statistics Act (Data Sharing under Sections 11 and 12)
(http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Directive on Microdata Linkage
• Assess impact on privacy prior to commencing any new data collection initiative.
f Directive on Conducting Privacy Impact Assessments
• Use focus groups and questionnaire testing to uncover factors that will impact data collection. Identify
sensitive questions and subjects and evaluate the impact of the mode of collection on the provision of
such information. Ensure response is not inhibited by characteristics such as excessively long or onerous
questionnaires.
f Policy on the Review and Testing of Questionnaires
f Questionnaire Design Resource Centre
Statistics Canada – Catalogue no. 12‑586‑X
19
Statistics Canada Quality Assurance Framework
C.3 Statistics Canada facilitates response by informing respondents of their rights and
obligations, the importance of responding and the protection their data will be afforded
• Provide respondents with information on Statistics Canada and the survey in which they have been
asked to participate. Explain to respondents the benets of ofcial statistics and the importance of their
participation.
f Statistics Act (http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Directive on Informing Survey Respondents
f Information for survey participants, including video on importance of participation
(http://www.statcan.gc.ca/eng/survey/index?HPA=x)
f Information for Business Survey Respondents
(http://www.statcan.gc.ca/eng/survey/participant06)
• Inform respondents of any planned or possible record linkage of their survey responses to data from other
surveys or administrative les. If required, obtain permission for linkage.
f Directive on Microdata Linkage
• For longitudinal surveys, inform respondents of the repeated nature of data collection and that their
information will be combined with that collected in past or future cycles, as applicable.
• Inform respondents of any data sharing or joint-collection agreements to which their data are subject. When
applicable, allow survey respondents to opt out of such use of their data.
f Statistics Act (Data Sharing under Sections 11 and 12)
(http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
• Convey to respondents the importance of using recommended mechanisms to return questionnaire
data, the risks associated with other means of transmission and the importance of providing data by the
specied deadline.
f Directive on the Transmission of Protected Information
C.4 Statistics Canada implements organizational, physical and technological measures to
protect the confidentiality and ensure the security of statistical data
• Specify the characteristics of information to which “sensitive statistical information” provisions apply and
develop appropriate policies and practices to safeguard such information. Some examples of sensitive
statistical information are: information attributable to specic respondents (e.g., completed questionnaires,
printouts of individual survey records); information which could be exploited for nancial or other gain
(i.e., information not yet released to the public); and human resources management and other personal
information of employees.
f Directive on the Security of Sensitive Statistical Information
• Address issues related to condentiality, privacy and security in risk management and contingency
planning.
f Integrated Risk Management Guidelines
f Information and Privacy Breach Protocol (http://www.statcan.gc.ca/eng/about/pia/generic/section8)
• Elaborate a reporting and management structure for assurance of condentiality, privacy and security and
promotion of the importance of these issues. At Statistics Canada, a corporate committee on microdata
access provides direction about matters pertaining to access to information, privacy and condentiality
obligations, with other specic responsibilities accorded to certain senior manager positions.
f Microdata Access Management Committee
f Policy on Privacy and Condentiality
• Dene corporate approval, monitoring and documentation processes for all record linkage initiatives.
f Directive on Microdata Linkage
Statistics Canada – Catalogue no. 12‑586‑X
20
Statistics Canada Quality Assurance Framework
• Dene guidelines for retention and documentation of statistical information.
f Policy on Document Management
f Policy on Information Management
f Directive on the Management of Aggregate Statistics
• Implement secure electronic storage and transmission of information, including controlled network access.
f Information Technology Security Policy
f Directive on Management of Information Technology
f Directive on the Transmission of Protected Information
• Limit internal access to sensitive information to only those who require it (“need to know”). Review and
update such permissions regularly through a centralized system.
f Corporate Access Request System
• Control access to premises. Statistics Canada restricts access and entry to buildings, requires that access
(identication) cards be worn by staff members and visitors, and requires specic approval through a
centralized system for after-hours access.
f Security Practices Manual
f Corporate Access Request System
• Apply strict protocols regarding external user access to statistical microdata for research purposes. These
protocols apply to external researchers working at Statistics Canada’s ofces and at its Research Data
Centres. They also cover the use of microdata accessed externally via universities, or through remote
access.
f Policy on Microdata Access
f Directive on the Use of Deemed Employees
f Directive on the Security of Sensitive Statistical Information
f Research Data Centres (http://www.statcan.gc.ca/rdc-cdr/index-eng.htm)
f Federal Research Data Centres (http://www.statcan.gc.ca/eng/rdc/frdc)
f Canadian Centre for Data Development and Economic Research
(http://www.statcan.gc.ca/cder-cdre/index-eng.htm)
f Data Liberation Initiative (http://www.statcan.gc.ca/eng/dli/dli)
f Real Time Remote Access (http://www.statcan.gc.ca/eng/rtra/rtra)
C.5 Statistics Canada provides training and other measures to instill a culture that promotes
awareness of issues and obligations related to confidentiality, privacy and security
• Provide a range of training on condentiality, privacy and security. Address these themes in the orientation
course for all new employees and in agship courses. Develop and implement new training initiatives.
Provide employees with tips and best practices.
f Survey Skills Development Course (http://www.statcan.gc.ca/eng/survey/household/8008)
• Require all staff (both regular and deemed employees) to sign a legal condentiality commitment prior to
accessing any work materials. Currently a new employee’s director administers an oath on the rst day
of work.
• Require all staff to refresh their condentiality commitment at regular intervals. Currently employees
complete an online course as part of renewing their building access card.
Statistics Canada – Catalogue no. 12‑586‑X
21
Statistics Canada Quality Assurance Framework
• Require all staff to review and accept a policy outlining responsibilities and obligations of Statistics Canada
employees. The policy includes post-employment requirements. Currently review and acceptance of this
policy is required annually, as part of employee performance review.
f Code of Conduct
f Values and Ethics Code (http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=25049)
f Policy on Conict of Interest and Post-Employment
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=25178)
• Require all staff to review and accept a policy outlining proper use of Agency electronic networks. Currently
acceptance of this policy is required every 90 days to maintain an active user account.
f Policy on the Use of Electronic Networks
f Network Use Policy
f Network Use Policy Agreement
• Ensure employees are aware of the potential penalties for wilful breaches of statistical condentiality and of
the procedures to be followed if they become aware of a breach of statistical condentiality.
f Statistics Act (Section 17(1)(b), Section 30(c))
(http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Information and Privacy Breach Protocol (http://www.statcan.gc.ca/eng/about/pia/generic/section8)
C.6 Statistics Canada implements effective disclosure control measures, in particular
through extensive verification of outputs
• Develop and maintain expertise in disclosure control and make it available to all statistical program areas.
Continue research efforts in identifying new types of risks, measuring disclosure risk and protecting
statistical data condentiality.
f Disclosure Control Resource Centre
• Remove names, personal or business identication numbers and other unique identiers from internal
databases and data les, when not required for statistical purposes.
f Directive on the Management of Statistical Microdata Files
• Promote and use standardized methods, software and other tools for condentiality protection such as that
for condentiality (G-Cond), and for tabulations (G-Tab). These methods and tools are reviewed through
corporate committees responsible for methods and standards, and for microdata access.
f Methods and Standards Committee
f Microdata Access Management Committee
• Implement approval procedures, drawing upon the expertise of Agency specialists in the corporate
committee on microdata release, prior to release of statistical microdata les and of aggregated data.
f Microdata Release Committee
f Policy on Microdata Release
f Guidelines for the Release of Microdata Files
• Ensure appropriate measures (e.g., anonymization, physical security) are in place for researcher access to
microdata les. Properly vet outputs, including those that are generated using complex survey analysis and
other statistical techniques.
Statistics Canada – Catalogue no. 12‑586‑X
22
Statistics Canada Quality Assurance Framework
The Statistical Program
D. Management of input data and relations with data providers
Description
The three main sources of input data to Statistics Canada are respondents, administrative data providers
7
and repositories of data available to the public. Respondents, such as individuals, households, enterprises
or administrations, provide data about an entity (typically themselves, but possibly an entity about which they
can report) using a data collection instrument, the most common example of which is a survey questionnaire.
Administrative data providers collect information about a population of interest to them for their own purposes
(i.e., regulatory, nancial administration, service delivery) and provide this information to Statistics Canada.
The general term data available to the public refers to data that are available to any user, obtained with or
without licence, and with or without payment of a fee. This includes, but is not exclusive to, open data, commercial
data and web data. Statistics Canada obtains respondent and administrative data within a legal or regulatory
framework that covers the Agency’s and data provider’s rights and responsibilities regarding collection and use.
When obtaining data available to the public, the Agency is bound by terms and conditions that apply to all who
access and use the data.
Provision of high quality input data is vital to the success of a national statistical ofce (NSO). Access to
high-quality input data is achieved primarily through the NSO effectively managing relations with input data
providers, supported by legislative or regulatory requirements and complemented by goodwill and belief in the
value of quality ofcial statistics. Another factor that contributes to the high quality of input data is the consistency
in the Agency’s collection and pre-processing methods. This consistency leads to shared and standardized
treatments during preliminary data processing operations.
Statistics Canada’s objective in using administrative data or data available to the public for statistical purposes is
to improve the balance between relevance, data quality, response burden and cost. An important responsibility of
the Agency is to justify the necessity of any new data collection activity, given its cost and the burden it imposes.
In particular, when administrative data or data available to the public are of sufcient quality and adequately match
the concepts being measured, they should be considered over direct collection. A further responsibility of the
Agency, to both input data providers and the general public, is protecting the privacy, condentiality and security
of all data provided to it, with the provisions applying equally to survey data and administrative data.
Assessment
Statistics Canada’s efforts pertaining to obtaining and managing high-quality input data, in particular through its
relations with respondents and administrative data providers, are assessed by evaluating the extent to which the
Agency:
1. justies the necessity of any new data collection
2. puts policies and practices in place to optimize its use of data that are already available
3. encourages and supports respondent participation through effective communication
4. monitors and manages response burden
5. develops and implements effective, innovative and user-friendly collection tools and provides response
options
6. collaborates effectively with administrative data providers and other partners in the information industry
7. provides for secure and efcient acquisition and management of administrative data through a corporate
approach supported by common tools.
7.
Statistics Canada’s Policy on the Use of Administrative Data Obtained under the Statistics Act provides definitions of administrative data, administrative data providers and other terms.
The Policy also outlines objectives and context for the Agency’s acquisition and use of administrative data, and defines roles and responsibilities.
Statistics Canada – Catalogue no. 12‑586‑X
23
Statistics Canada Quality Assurance Framework
Implementation
Statistics Canada undertakes a broad range of specic initiatives to effectively obtain and manage the Agency’s
input data and to build and maintain effective relations with input data providers. Below is a list of such initiatives,
in groups that correspond to the items under Assessment.
D.1 Statistics Canada justifies the necessity of any new data collection
• Before undertaking any new collection, determine if equivalent data are available from other sources. For
example, a request to access administrative (tax) data or publicly available nancial reports of corporations
could be made as an alternative to collection of nancial information from respondents.
f Directive on Obtaining Administrative Data under the Statistics Act
f Guidelines on Data Available to the Public
• Introduce a formal review process for new data demands (i.e., increasing sample size of existing surveys or
undertaking new surveys) for business and household surveys.
f Business Response Management Committee
f Household Survey Strategy
• Reduce response burden through data collection and processing initiatives such as record linkage, data
sharing or joint data collection activities (where surveys that collect similar or related data are integrated or
combined). Study and implement innovative methods, for example electronic collection, satellite imaging
and scanner data.
f Business Response Management Committee
f Statistics Act (Data Sharing under Sections 11 and 12)
(http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Directive on Microdata Linkage
• Assess the impact on privacy prior to commencing any new data collection initiative.
f Directive on Conducting Privacy Impact Assessments
• Limit the collection of respondent-provided information that is used exclusively for data-management or
other survey-support reasons such as editing or imputation.
D.2 Statistics Canada puts policies and practices in place to optimize its use of data that
are already available
• Work actively with partners to collectively optimize the collection and sharing of data. Maintain key contact
with other federal government departments and other partners including provincial focal points and
industry associations. Ensure consistency through standardized protocols for objectively evaluating the
quality, relevance and utility of new sources of administrative or data available to the public.
f Policy on the Use of Administrative Data Obtained Under the Statistics Act
f Directive on Obtaining Administrative Data Under the Statistics Act
f Quality and Evaluation Framework for Administrative Data
• Keep an inventory of administrative data les available to Statistics Canada programs.
f Administrative Data Inventory
f Acquisition of Administrative Data
• Promote and support the use of record linkage, small-area estimation and other techniques that bring
together information from various sources.
f Directive on Microdata Linkage
f Record Linkage Resource Centre
f Record Linkage Project Process Model (http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-605-
X&objType=2&lang=en&limit=0)
Statistics Canada – Catalogue no. 12‑586‑X
24
Statistics Canada Quality Assurance Framework
• Seek alternatives to traditional survey methodology and administrative microdata, in particular when
direct collection is not feasible or is not cost-effective, such as for rare and hard-to-reach populations and
sub-populations of large numbers of small units. Examples of such techniques include statistical modelling,
proxy response and synthetic estimation.
• Learn about innovative data gathering and treatment methods (e.g., “Big Data”) and gauge their relevance
to Statistics Canada. Establish ties with information-industry partners in the private sector, academia and
research institutes.
D.3 Statistics Canada encourages and supports respondent participation through
effective communication
• Instill condence in respondents by emphasizing guarantees of privacy and protection of personal
information provided by federal laws and Statistics Canada policies.
f Statistics Act (http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Policy on Privacy Protection
• Provide respondents with information on Statistics Canada and the survey in which they have been
asked to participate. Explain to respondents the benets of ofcial statistics and the importance of their
participation.
f Statistics Act (http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Directive on Informing Survey Respondents
f Information for survey participants, including video on importance of participation
(http://www.statcan.gc.ca/eng/survey/index?HPA=x)
f Information for Business Survey Respondents (http://www.statcan.gc.ca/eng/survey/participant06)
• Convey the appreciation of Statistics Canada by thanking respondents in publications and, when possible,
giving back to respondents information accruing from their survey participation.
• Maintain lines of communication by soliciting respondent feedback, responding to complaints and
concerns and using the information obtained to improve programs and processes. This communication
strategy includes a single point of contact for respondent inquiry and an approach where social media
(including Facebook, Twitter and YouTube) are used to provide access to statistical information and to foster
engagement, co-operation and information-sharing with respondents. Further, an ombudsman is available
to help business survey participants by responding to their concerns.
f Stay connected (http://www.statcan.gc.ca/eng/sc/index)
f Ombudsman for Business Survey Respondents (http://www.statcan.gc.ca/eng/survey/ombudsman)
• Initiate outreach to promote an awareness of Statistics Canada within the Canadian public. Examples of
such approaches are regular presentations by the Chief Statistician, relations with the media such that
Statistics Canada results are announced to the public, and through the maintenance of a web and social
media presence.
D.4 Statistics Canada monitors and manages response burden
• Put in place controls to prevent a disproportionately high response burden on particular respondents or
particular categories of respondents. This strategy entails such measures as offering periodic reporting
exemptions for respondents selected for multiple surveys during a time interval, establishing upper limits for
the length of time respondents spend in the sample for a specic survey, and taking respondent availability
and preference into account when scheduling interviews.
• Consider response burden in the sample allocation phase. For example, identify rare populations from
the Census of Population or administrative data to avoid asking identication questions to a broad base
of respondents. Response burden can also be minimized through techniques including sample rotation
(e.g., Labour Force Survey rotating panel design, where one-sixth of the sample is replaced each month),
coordinated allocation and use of differential sampling rates (e.g., selection probabilities proportional to size
for skewed populations of enterprises).
Statistics Canada – Catalogue no. 12‑586‑X
25
Statistics Canada Quality Assurance Framework
• Retain information to quantify response burden and set targets for its reduction over time.
f Response Burden Reduction Efforts (http://www.statcan.gc.ca/eng/about/rbre)
f Household Survey Strategy measures of response burden
• Coordinate all data collection, irrespective of mode, through a corporate collection service. Statistics
Canada uses a single portal to plan for all collection needs and an integrated collection and operations
system, and achieves consistency in input les through standard methods.
f Collection Front Door
f Collection Planning and Research Division
f Integrated Collection and Operations System
D.5 Statistics Canada develops and implements effective, innovative and user-friendly
collection tools and provides response options
• Use focus groups and questionnaire testing to identify impediments to response. These strategies
include detecting sensibilities and social acceptability, minimizing the impact on privacy and degree of
invasiveness, and detecting the impact of questionnaire length and “respondent-friendliness”.
f Policy on the Review and Testing of Questionnaires
f Questionnaire Design Resource Centre
• Build and deploy an electronic data reporting infrastructure. This infrastructure includes the development of
electronic questionnaire applications as part of the Corporate Business Architecture modernization, and the
adoption of internet response as the default mode for the Census of Population.
f Electronic Questionnaire Service
f Corporate Business Architecture
• Adapt to changes in technology and behaviour by developing applications for cellphone and tablet
response, integrating methods to reach cellphone-only households and initiating the use of tablets by
interviewers.
• Where possible and appropriate (i.e., for large or complex enterprises), adapt data collection to conform to
the respondent’s accounting system.
D.6 Statistics Canada collaborates effectively with administrative data providers and other
partners in the information industry
• Build and sustain good working relations with providers and custodians of administrative data. In particular,
ensure contact between counterparts at a variety of levels and functions, including chief executives,
program managers and operational staff. Consult with other federal government departments, provincial
government departments or focal points, and industry associations. Strengthen the statistical value of
administrative data by communicating Statistics Canada’s requirements in terms of coverage, conceptual
denitions and timeliness. Keep abreast of organizational and operational changes at provider agencies
and remind providers of Statistics Canada’s commitments to data security, condentiality and use of data
for statistical purposes only.
• Solidify access to data by entering into data sharing agreements and by working to support terms of
legislation that give the Agency access, for statistical purposes, to information of other government
departments and organizations. For example, Memoranda of Understanding have been signed with Canada
Revenue Agency for tax and payroll data.
f Statistics Act (http://laws-lois.justice.gc.ca/eng/acts/S-19/FullText.html)
f Directive on Obtaining Administrative Data under the Statistics Act
Statistics Canada – Catalogue no. 12‑586‑X
26
Statistics Canada Quality Assurance Framework
D.7 Statistics Canada provides for secure and efficient acquisition and management of
administrative data through a corporate approach supported by common tools
• Ensure an Agency-level coordination of external collaboration (interdepartmental, intergovernmental and
international) led by the division responsible for administrative data. Coordinate and support efforts with the
appropriate administrative structures, governance, resource allocation and communications plans.
f Administrative Data Division
• Require all projects that deal with methods or processes associated with administrative data to pass
through Agency-level approval. This practice of using a corporate committee for administrative data
management fosters development and implementation of consistent practices, and achieves long-term
benets including increased relevance, gains in efciency and reduction of response burden.
f Administrative Data Management Committee
• Maintain a repository of information on administrative data holdings and initiatives within the Agency. This
repository includes tools for use of administrative data (i.e., Metadata Explorer, Tax Data Navigator), an
inventory of administrative data les that have been obtained or are in the process of being obtained by the
Agency and a collection of relevant, high-level documentation on administrative data.
f Administrative Data Inventory
• Standardize pre-processing of administrative data. Create and maintain centralized databases of
processed administrative data, including tax data. Require that “downstream” processes draw from these
databases, rather than processing administrative data independently.
• Ensure the condentiality, security and integrity of administrative data provided to Statistics Canada.
Protect administrative data with equivalent physical, technological and organizational provisions as survey
data obtained directly by the Agency. Apply enhanced security measures when warranted by the sensitivity
of administrative data, such as for tax data or health information. Ensure that only authorized staff has
access to specic databases, controlled through a centralized system.
f Policy on Privacy and Condentiality
f Information Technology Security Policy
f Directive on the Security of Sensitive Statistical Information
f Directive on the Transmission of Protected Information
f Corporate Access Request System
Statistics Canada – Catalogue no. 12‑586‑X
27
Statistics Canada Quality Assurance Framework
E. Allocation and management of resources
Description
The production of most Statistics Canada outputs is supported nancially through government revenues allocated
to the Agency. Other resources also requiring constant management include human resources, information
technology and infrastructure.
It is important to identify and harvest efciencies in order to fund improvements and new initiatives and to provide
a cushion against funding stresses. Consequently the Agency strives to minimize the resources consumed
(efciency) while ensuring that outputs are t for use (effectiveness).
Assessment
Statistics Canada’s allocation and management of resources are assessed by evaluating the extent to which the
Agency:
1. implements effective strategies for organization and management
2. ensures continued production of relevant, high-quality and timely information through effective long-term
planning and priority-setting
3. pursues efciencies through centralization of resources, consolidation of services and use of generalized
systems
4. pursues efciencies in administrative services, infrastructure and technology
5. implements effective project management
6. implements human resources management practices that optimize the contribution of its workforce and
demonstrate its commitment to quality.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to effectively allocate and manage resources
in its statistical programs. Below is a list of such initiatives, in groups that correspond to the items under
Assessment.
E.1 Statistics Canada implements effective strategies for organization and management
• Corporately optimize decision-making. Decisions regarding resource management and allocation are
ultimately decided by the Executive Management Board (EMB). However, various internal committees, with
specialties in particular subjects, recommend decisions to the EMB. These recommendations are based on
reviews that consider the Agency’s overall priorities, gaps, risks and budget.
f Corporate Business Plan (http://www.statcan.gc.ca/eng/about/bp)
f Corporate Risk Prole
f Field Planning Boards
f Integrated Risk Management Guidelines
• Objectively evaluate the Agency’s use of resources by fully participating in external audits, scheduling
regular internal evaluations and reviews to evaluate objectives in relation to cost, and producing corporate
performance indicators that quantify program efciency.
• Maintain the capacity (staff, infrastructure) to produce custom products in response to client requests for
cost-recovery work.
f Information on Custom Surveys (http://www.statcan.gc.ca/eng/cs/overview)
• Mitigate vulnerability to funding stresses by setting aside sufcient contingency funds, in part through
instilling a culture that promotes efciency.
• Centralize control and monitoring of nancial administration, human resources management and
information technology infrastructure at the Agency level.
Statistics Canada – Catalogue no. 12‑586‑X
28
Statistics Canada Quality Assurance Framework
E.2 Statistics Canada ensures continued production of relevant, high-quality and timely
information through effective long-term planning and priority-setting
• Implement a sound planning and priority-setting process at the Agency level. This process considers a
long-term horizon. It works on an annual planning cycle and covers all stages of planning where nancial,
human resources and informatics needs are all considered together. It includes a review of corporate
priorities and integrates risk management, investment planning, and evaluation into the planning process.
f Integrated Strategic Planning Process
f Integrated Business and Human Resources Plan
f Information Technology Plan
• Forecast regular and strategic investments necessary to preserve quality and continuity of statistical
programs over time. This forecast covers an extended horizon (10 years) and considers the cyclical nature
of specic statistical programs (e.g., ve-year Census cycle, 10-year redesign cycle for certain surveys)
when planning redesigns and other program modications or enhancements. A separate fund exists for
initiatives that cannot be covered by program area base budgets.
f Continuity and Quality Maintenance Investment Plan
f Continuity and Quality Management Fund
• Target efciencies by regularly conducting Agency-wide reviews of systems and practices. Statistics
Canada has in place the Corporate Business Architecture which is a long-term Agency-wide review
and modernization that covers the business processes, business rules, computer systems and internal
organizational and physical infrastructure that Statistics Canada uses to carry out its main business of
collecting, analyzing and publishing statistical information. By consolidating processes and standardizing
systems where necessary, cost savings are achieved while still maintaining high standards of quality and
timeliness.
f Corporate Business Architecture
• Communicate plans and priorities to employees and stakeholders. Besides publicly available documents,
internal communication includes the Chief Statistician’s annual address and subsequent article published in
the Statistics Canada electronic employee newsletter.
f Departmental Reports (http://www.statcan.gc.ca/eng/about/reports2)
E.3 Statistics Canada pursues efficiencies through centralization of resources,
consolidation of services and use of generalized systems
• Limit the number of business processes and computer systems in place; that is, encourage re-use of
existing systems. Foster ease of transition of employees between projects by consolidating computer
applications and minimizing the number of hardware and software products (tool kits) in use.
• Ensure efcient allocation by centralizing specialized resources and services. Examples include informatics
and methodology resources, frame infrastructures for economic and social surveys and administrative data
processing operations.
• Establish single corporate service areas for statistical services to generate and capture economies of scale.
Mandate their use by requiring justication of local exceptions. This plan involves common processing
environments for business and social surveys, a corporate collection service and a single collection system.
f Integrated Collection and Operation System
• Develop, promote and support the use of a suite of Statistics Canada generalized systems and common
tools.
f Generalized Systems Resource Centre
• Streamline data processing activities. In particular, eliminate rework (e.g., editing at multiple stages of the
survey process) and pay particular attention to resource-intensive steps such as collection and manual
processes such as editing and coding. For example, selective editing can be used to prioritize application
Statistics Canada – Catalogue no. 12‑586‑X
29
Statistics Canada Quality Assurance Framework
of effort to where it will have the greatest impact on estimates, and responsive collection uses paradata to
optimally schedule interviews.
E.4 Statistics Canada pursues efficiencies in administrative services, infrastructure and
technology
• Introduce transformation and modernization initiatives for infrastructure. This includes physical space
reduction and modernization, replacement of landline telephones with cellphones and migration of
department-specic email systems to a common framework.
f Cost Effective Telephone Services Infrastructure
f Email Transformation Initiative
• Undertake periodic modernization of the computer network and informatics infrastructure. This includes
improvements to informatics security, standardization of employee software toolkit (desktop) and migration
of local processing platforms to centralized ones (e.g., SAS Grid).
f Network Transformation Initiative
• Centralize and modernize delivery of administrative and other support services. In particular: streamline
and standardize through use of Agency-standard business processing systems for human resources
management and nancial administration; replace paper-based methods by electronic request and
approval; and consolidate editorial and translation services.
E.5 Statistics Canada implements effective project management
• Require the use of a standard project management process, templates and tools covering the entire
project lifecycle. This process includes project “gating” (approval and monitoring) at the eld level within the
Agency.
f Departmental Project Management Framework
f Departmental Project Management Ofce
f Field Planning Boards
• Require supplemental procedures for evaluation of the informatics component of new projects.
f Informatics Technology Architecture Committee (includes the Technical Review Committee and the
Security Review Committee)
f Integrated Security and Risk Framework
• Facilitate project management by mapping projects to statistical process models.
f Generic Statistical Business Process Model
(http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0)
• Minimize misunderstanding and ensure project goals are respected through effective change management.
f Changes, Issues and Risk Management Tool
f Change Advisory Board
• Provide tools for monitoring and evaluating project resources. Performance reports, monthly nancial
reporting and project dashboards are required of each program.
f Departmental Project Management Framework
E.6 Statistics Canada implements human resources management practices that optimize
the contribution of its workforce and demonstrate its commitment to quality
• Effectively manage appointment and promotion. Centralized recruitment programs for core professional
groups, generic competitions for all management levels and comprehensive “career streams” for all major
groups are all standard practices which contribute to the effective use of resources.
Statistics Canada – Catalogue no. 12‑586‑X
30
Statistics Canada Quality Assurance Framework
• Ensure Statistics Canada’s human resources meet the Agency’s needs in terms of both strength and
qualication. Regular Agency-wide performance monitoring and feedback (including training needs), talent
management and mentoring are ways of enhancing staff performance.
f Training Policy and Framework
• Anticipate future needs through effective human resources management and planning, including
succession planning.
f Integrated Business and Human Resources Plan
• Cultivate employee motivation, professionalism and engagement. Statistics Canada motivates employees
by fostering innovation, as well as by a positive and healthy workplace including physical exercise facilities,
exible work opportunities and a Wellness Centre.
f Innovation Channel
Statistics Canada – Catalogue no. 12‑586‑X
31
Statistics Canada Quality Assurance Framework
F. Management of relations with data users and stakeholders
Description
Building and sustaining good relations with its data users and stakeholders is vital for the success of a national
statistical ofce (NSO). Data users and stakeholders transform statistical information into knowledge, as needed
for political debate, decision-making or research. When statistical information is t for use, data users and
stakeholders gain condence in the NSO, return to it for future information and, in some instances, provide the
NSO with the support it needs to carry out its work.
The focus of this chapter is activities Statistics Canada undertakes to build and sustain relations with data users
and stakeholders. It is important to note that some data users and stakeholders may interact with Statistics
Canada in other capacities, such as survey respondents or providers of administrative data.
Assessment
Statistics Canada’s efforts pertaining to managing its relations with data users and stakeholders are assessed by
evaluating the extent to which the Agency:
1. makes information about itself readily available
2. improves its programs through the participation of data users and stakeholders
3. ensures all Canadians have equitable and timely access to its data products and information releases
4. meets the communications and support services needs of specialized groups of data users and
stakeholders
5. partners and collaborates effectively with other federal departments and agencies and with provincial,
territorial and local governments.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to build and maintain effective relations with
its data users and stakeholders. Below is a list of such initiatives, in groups that correspond to the items under
Assessment.
F.1 Statistics Canada makes information about itself readily available
• State the Agency’s mandate and objectives clearly on its website.
f About Statistics Canada (http://www.statcan.gc.ca/eng/about/about?MM=)
f Statistics Canada (the Agency) (http://www.statcan.gc.ca/eng/about/agency?MM=)
f Mandate and objectives (http://www.statcan.gc.ca/eng/about/mandate)
f We are Statistics Canada (video) (http://www.statcan.gc.ca/eng/about/video/statcan)
f Fundamental Principles of Statistics (video) (http://www.statcan.gc.ca/eng/about/video/fpos)
• Publicly identify the Agency’s stakeholders and the uses to which it puts information it collects.
f Stakeholders (http://www.statcan.gc.ca/eng/about/stakeholders)
f Partners (external links) (http://www.statcan.gc.ca/eng/reference/partners-eng)
f How data are used (http://www.statcan.gc.ca/eng/about/used)
• Convey information regarding the Agency’s priorities, initiatives and progress to the public, stakeholders
and employees. Various reports are listed on its website, including its report to Parliament, the Corporate
Business Plan and internal audit reports.
f Departmental Reports (http://www.statcan.gc.ca/eng/about/reports2)
f Chief Statistician’s annual address and subsequent article in the Statistics Canada electronic
employee newsletter
Statistics Canada – Catalogue no. 12‑586‑X
32
Statistics Canada Quality Assurance Framework
F.2 Statistics Canada improves its programs through the participation of data users
and stakeholders
• Regularly engage in consultation with Canadians to gauge satisfaction, seek input and test functionality,
using a variety of methods. For example, the Agency consults with: users, as part of program review
and evaluation; business and industry associations and labour unions, in particular to learn about
information needs and reporting preferences; not-for-prot organizations including community groups,
social organizations and volunteer groups; and interested groups on particular programs (e.g., Census of
Population content). When possible, the results of consultation are made available to the public.
f Consulting Canadians (http://www.statcan.gc.ca/eng/consultation/index-eng?MM)
f Consultation FAQs (http://www.statcan.gc.ca/eng/consultation/faq-eng)
f Client satisfaction (http://www.statcan.gc.ca/eng/consultation/calend-t1-eng)
• Use advisory groups of experts to ensure the Agency’s programs remain relevant in light of emerging needs
and trends, to verify its methods remain sound and to foster exchanges that shed light on stakeholder
needs and priorities. These groups include the National Statistics Council and executive and professional
advisory committees on statistical methods and in each of the major subject-matter areas.
f Advisory Groups (http://www.statcan.gc.ca/eng/about/relevant)
• Maintain extensive contacts with international, scientic and intergovernmental organizations and
membership in international organizations to ensure the Agency’s outputs meet international standards and
that Canadian data continue to be comparable with those of other countries.
f Federal Government Departments (http://www.statcan.gc.ca/eng/reference/goc-gdc-eng)
f Canadian Statistical Organizations (http://www.statcan.gc.ca/eng/reference/organ-eng)
f International Organizations and Expert Groups
(http://www.statcan.gc.ca/eng/reference/other-autres-eng)
f International Cooperation (http://www.statcan.gc.ca/eng/about/cooperation)
F.3 Statistics Canada ensures all Canadians have equitable and timely access to its data
products and information releases
• Publish release dates well in advance. The pre-planned release schedule is publicly available and any
pre-release access is communicated transparently. Further, procedures exist for cases where data are
released prematurely.
f Release Schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
f Directive on the Security of Sensitive Statistical Information
f Information and Privacy Breach Protocol (http://www.statcan.gc.ca/eng/about/pia/generic/section8)
• Require that all releases pass through a uniform dissemination service. Currently The Daily is Statistics
Canada’s ofcial release bulletin.
f Policy on Ofcial Release
f Directive on the Distribution of Registered Statistics Canada Products
f The Daily (http://www.statcan.gc.ca/dai-quo/index-eng.htm?HPA)
f Policy on Communications and Federal Identity
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=30683)
• Correct errors in a structured, timely and transparent fashion.
f Directive on Corrections to Daily Releases and Statistical Products
• Minimize the barrier of cost by providing a set of standard products free of charge on the Statistics Canada
website.
Statistics Canada – Catalogue no. 12‑586‑X
33
Statistics Canada Quality Assurance Framework
• Produce and promote a broad range of statistical products, and dene a fair and balanced policy for
funding these on a cost-recovery basis. For example, research papers, analytical products, specialized
tabulations, as well as the design and implementation of customized surveys, are available.
f Research Papers and Analytical Products (http://www5.statcan.gc.ca/researchers-chercheurs/
index.action?lang=eng&start=1&end=25&search=&sort=0&themeId=0&date=&series=&author=&the
meState=-1&dateState=-1&seriesState=-1&authorState=-1&showAll=false&univ=7&MMI)
f Information on Custom Surveys (http://www.statcan.gc.ca/eng/cs/overview)
f Pricing of Cost-Recovery Projects
f Policy on Service Standards for External Fees
F.4 Statistics Canada meets the communications and support services needs of specialized
groups of data users and stakeholders
• Work with the media and others to maximize awareness, in particular by encouraging re-dissemination of
Statistics Canada information products through use of modern information technology and an open-data
licence agreement that requires no payment of fees for specic data products. This approach includes
applications for smartphone and tablet access, a presence on social media (including Facebook, Twitter
and YouTube), a regular blog and a facility to interact with Agency specialists.
f Stay connected (http://www.statcan.gc.ca/eng/sc/index?MM)
f Statistics Canada blog (http://www.statcan.gc.ca/eng/sc/blog?MM)
f Chat with an expert (http://www.statcan.gc.ca/eng/chatwithanexpert?MM)
f Statistics Canada Open Licence Agreement (http://www.statcan.gc.ca/eng/reference/licence-eng)
f Directive on Licensing
• Provide special and dedicated services for members of the media.
f Media Enquiries (http://www.statcan.gc.ca/dai-quo/smr01/smr01_000-eng.htm)
f Directive on Media Relations
• Facilitate researcher access to data without compromising data security and condentiality. Apply
protocols regarding external user access to statistical microdata for research purposes. These protocols
apply to external researchers working at Statistics Canada’s ofces and at its Research Data Centres. They
also cover the use of microdata accessed externally via universities, or through remote access.
f Policy on Microdata Access
f Directive on the Use of Deemed Employees
f Directive on the Security of Sensitive Statistical Information
f Research Data Centres (http://www.statcan.gc.ca/rdc-cdr/index-eng.htm)
f Federal Research Data Centres (http://www.statcan.gc.ca/eng/rdc/frdc)
f Canadian Centre for Data Development and Economic Research
(http://www.statcan.gc.ca/cder-cdre/index-eng.htm)
f Data Liberation Initiative (http://www.statcan.gc.ca/eng/dli/dli)
f Real Time Remote Access (http://www.statcan.gc.ca/eng/rtra/rtra)
• Put in place a commitment to serve clients in a prompt, reliable, courteous and fair manner and maintain
lines of communication with them. This commitment includes a single point of contact for respondent
inquiry and a follow-up response to complaints and concerns.
f Standards of Service to the Public (http://www.statcan.gc.ca/eng/about/service/standards)
• Explain the value and benets of quality ofcial statistics to data users and stakeholders.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Directive on Documenting Statistical Metadata
Statistics Canada – Catalogue no. 12‑586‑X
34
Statistics Canada Quality Assurance Framework
F.5 Statistics Canada partners and collaborates effectively with other federal departments
and agencies and with provincial, territorial and local governments
• Maintain close bilateral relationships with key federal departments and agencies through connection at
the senior management and other levels. These relationships are supported through participation of the
Chief Statistician in policy and program discussions with other deputy ministers, through participation in
federal government research initiatives and through inter-departmental committees. These contacts foster
an awareness of respective needs and information priorities, and allow the Agency to keep abreast of
emerging needs.
f Advisory Groups (http://www.statcan.gc.ca/eng/about/relevant)
f Inter-departmental committees
• Collaborate and liaise with provincial and territorial focal points and other representatives. Determine
provincial and territorial data requirements, consult on current statistical activities, harmonize provincial and
territorial statistics to produce national estimates, and coordinate the dissemination of Statistics Canada’s
products to provincial and territorial governments.
f Federal-Provincial-Territorial (FPT) Consultative Council on Statistical Policy
f Subcommittees of FPT Consultative Council on Statistical Policy
• Provide special liaison, consultation and coordination in the areas of health, education and justice, which
are primarily under provincial jurisdiction. Examples include assistance to the Board of Directors of the
Canadian Institute for Health Information and participation in the Canadian Education Statistics Council and
the Justice Information Council.
• Maintain lines of communication with local and regional governments.
Statistics Canada – Catalogue no. 12‑586‑X
35
Statistics Canada Quality Assurance Framework
Statistical Outputs
G. Relevance
Description
Statistics Canada aims to provide its data users and stakeholders with information on subjects that are important
to them, and in a format and within a time frame that facilitates their research, analysis, decision-making and
communications. The relevance of statistical information reects the degree to which it meets the needs of data
users and stakeholders.
Managing relevance requires ensuring that the Agency’s programs remain aligned with information needs as
they evolve. Being aware of changing priorities and having the exibility to respond to them are vital to ensuring
continued relevance.
Assessment
Statistics Canada’s efforts to maintain relevance of its statistical programs are assessed by evaluating the extent
to which the Agency:
1. captures emerging trends and changes to the Canadian sociological and economic framework, and thus
identies information gaps in its current statistical programs
2. communicates information about its mandate, values, priorities and management practices to data users
and stakeholders
3. ensures continued production of relevant, high-quality and timely information through long-term planning
and priority-setting
4. supports refreshment of current statistical programs and promotion of new initiatives by identifying and
redirecting savings in existing programs and by procuring additional funding.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to maintain relevance in its statistical programs.
Below is a list of such initiatives, in groups that correspond to the items under Assessment.
G.1 Statistics Canada captures emerging trends and changes to the Canadian
sociological and economic framework, and thus identifies information gaps in its
current statistical programs
• Seek input from data users and stakeholders so that Statistics Canada keeps abreast of changes in society,
culture and commerce and the impact of these changes. This engagement is accomplished through such
practices as meetings of the Chief Statistician and senior managers with counterparts internationally and in
other federal departments, federal–provincial–territorial meetings, use of advisory groups of subject-matter
and technical experts, and advocacy groups. Further, liaison with businesses, not-for-prot organizations,
consortia of data users, the media and other secondary distributors of data are also used.
f Advisory Groups (http://www.statcan.gc.ca/eng/about/relevant)
f National Statistics Council
f Inter-departmental committees
f Federal-Provincial-Territorial (FPT) Consultative Council on Statistical Policy
f Subcommittees of FPT Consultative Council on Statistical Policy
f Stakeholders (http://www.statcan.gc.ca/eng/about/stakeholders)
f International Organizations and Expert Groups
(http://www.statcan.gc.ca/eng/reference/other-autres-eng)
• Regularly consult with Canadians to monitor data user and stakeholder satisfaction with the current
statistical programs, using a variety of methods. For example, the Agency consults with: business and
Statistics Canada – Catalogue no. 12‑586‑X
36
Statistics Canada Quality Assurance Framework
industry associations, small-business advocacy groups and labour unions, in particular to learn about
information needs and reporting preferences; not-for-prot organizations including community groups,
social organizations and volunteer groups; and interested groups on particular programs (e.g., Census of
Population content). When possible, the results of consultation are made available to the public.
f Consulting Canadians (http://www.statcan.gc.ca/eng/consultation/index-eng?MM)
f Consultation FAQs (http://www.statcan.gc.ca/eng/consultation/faq-eng)
f Client satisfaction (http://www.statcan.gc.ca/eng/consultation/calend-t1-eng)
• Supplement the current statistical program with additions funded on a cost-recovery basis. Develop and
maintain the capacity to respond quickly to demands for information on matters of current interest. Allow
additions funded on a cost-recovery basis to become part of the core statistical program, if appropriate.
f Information on Custom Surveys (http://www.statcan.gc.ca/eng/cs/overview)
• Gather information on the use of Statistics Canada’s products and services. Statistics Canada’s division
responsible for dissemination tracks data users and the uses to which they put statistical products. Usage
and other statistics are compiled along with bibliometric and webometric analyses. Feedback from data
analysts identies gaps and weaknesses in data holdings.
G.2 Statistics Canada communicates information about its mandate, values, priorities and
management practices to data users and stakeholders
• State the Agency’s mandate and objectives clearly on its website.
f About Statistics Canada (http://www.statcan.gc.ca/eng/about/about?MM=)
f Statistics Canada (the Agency) (http://www.statcan.gc.ca/eng/about/agency?MM=)
f Mandate and objectives (http://www.statcan.gc.ca/eng/about/mandate)
f We are Statistics Canada (video) (http://www.statcan.gc.ca/eng/about/video/statcan)
f Fundamental Principles of Statistics (video) (http://www.statcan.gc.ca/eng/about/video/fpos)
• Publicly identify the Agency’s stakeholders and the uses to which it puts information it collects.
f Stakeholders (http://www.statcan.gc.ca/eng/about/stakeholders)
f Partners (external links) (http://www.statcan.gc.ca/eng/reference/partners-eng)
f How data are used (http://www.statcan.gc.ca/eng/about/used)
• Convey information regarding the Agency’s priorities, initiatives and progress. Various reports are listed on
its website, including the report to Parliament, the Corporate Business Plan and internal audit reports.
f Departmental Reports (http://www.statcan.gc.ca/eng/about/reports2)
• Provide supporting information to data users and stakeholders regarding the content, quality and timing of
Statistics Canada’s products. The pre-planned release schedule is publicly available. At release, standard
documentation is disseminated such as the denitions, data sources and methods for each survey and
statistical program, user guides, quality reports and technical reports. The above information includes
changes to concepts, denitions, classications and methods, as well as any deviations from international
standards and other practices.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Release Schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
f Integrated Metadatabase
f Directive on Documenting Statistical Metadata
Statistics Canada – Catalogue no. 12‑586‑X
37
Statistics Canada Quality Assurance Framework
G.3 Statistics Canada ensures continued production of relevant, high-quality and timely
information through long-term planning and priority-setting
• Manage priority-setting and planning to ensure continued relevance. Weigh competing needs objectively,
based on usage statistics and other information, when elaborating Agency-level priorities, and set
program-level priorities accordingly. Assess quality of existing statistical programs and adapt to changing
needs efciently and quickly through responsive allocation and exible deployment of resources.
An integrated strategic planning process includes a review of corporate priorities and integrates risk
management, investment planning, and evaluation into the planning process.
f Integrated Strategic Planning Process
f Integrated Business and Human Resources Plan
f Information Technology Plan
• Forecast the regular and strategic investments necessary to preserve quality and continuity of statistical
programs over time. This forecast covers an extended horizon (10 years) and considers the cyclical nature
of specic statistical programs (e.g., ve-year Census cycle, 10-year redesign cycle for certain surveys)
when planning redesigns and other program modications or enhancements. A separate fund exists for
initiatives that cannot be covered by program area base budgets.
f Continuity and Quality Maintenance Investment Plan
f Continuity and Quality Management Fund
• Continuously review programs from the perspective of evolving data user and stakeholder needs, and
make necessary adjustments to program content. The annual strategic planning sessions provide a forum
for addressing relevance issues, as does the annual risk assessment activity. The division responsible for
dissemination identies products for which demand has declined.
• Conduct external audits of statistical programs and benchmarking to external indicators, as required.
G.4 Statistics Canada supports refreshment of current statistical programs and promotion
of new initiatives by identifying and redirecting savings in existing programs and by
procuring additional funding
• Target efciencies by regularly conducting Agency-wide reviews of systems and practices. Statistics
Canada has in place the Corporate Business Architecture which is a long-term Agency-wide review
and modernization that covers the business processes, business rules, computer systems and internal
organizational and physical infrastructure that Statistics Canada uses to carry out its main business of
collecting, analyzing and publishing statistical information. By consolidating processes and standardizing
systems where necessary, cost savings are achieved while still maintaining high standards of quality and
timeliness.
f Corporate Business Architecture
• Regularly review the cost-effectiveness of statistical programs, in particular those required to meet
legislative or regulatory requirements. Manage survey collection, processing operations, dissemination
practices and quality assurance procedures so that the resulting statistics will meet needs in a
cost-effective fashion. Look for operational efciencies, in particular by managing and exploiting
interdependency among statistical programs.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
• Partner with stakeholders, especially other federal government departments, to nance valuable additions
to the national statistical database, in particular to meet legislative or regulatory requirements or to provide
information for policy development and evaluation.
Statistics Canada – Catalogue no. 12‑586‑X
38
Statistics Canada Quality Assurance Framework
H. Accuracy and reliability
Description
Statistics Canada aims to develop, produce and disseminate information that presents its data users and
stakeholders with a true reection of reality. The accuracy of statistical information refers to the degree to which
it correctly describes the phenomena it was designed to measure. The accuracy of statistical estimates is usually
quantied by the evaluation of different sources of error, where the magnitude of an error represents the degree
of difference between the estimate and the true value. Common sources of error include coverage, non-response,
measurement and processing. For estimates derived from survey data, an additional error source is sampling
error, which reects the fact that the estimates are computed from samples, rather than the entire population.
Related to accuracy is the concept of reliability, which reects the degree to which statistical information,
consistently over time, correctly describes the phenomena it was designed to measure. That is, reliability
characterizes repeated observations of accuracy, over time. Reliability applies both to multiple measurements
of the same phenomenon (e.g., preliminary, nal and revised estimates) and to a series of measurements (e.g.,
monthly estimates of the employment rate).
A denitive assessment of accuracy and reliability is impeded by the fact that an estimate needs to be compared
with a true value that possibly cannot be expressed quantitatively or that can be obtained only at great expense.
Consequently, much evaluation of accuracy and reliability is qualitative, described by the steps taken to
mitigate sources of error. One exception is sampling error, for which a sound mathematical framework exists
and quantitative measures, estimates of variance, are typically calculated and made available. The variance is a
measure of precision. In contrast to accuracy, precision expresses how close the estimates are to their average
value, rather than to the true value. Other measures of precision, such as coefcients of variation and condence
intervals, can be derived from the variance. The term bias is used to describe average differences between
estimates and the true value. Bias is generally due to systematic or non-random sources of error.
In managing accuracy, investment in a strong statistical methods research program increases the capacity to
improve accuracy without increasing cost or impacting timeliness, through the incorporation of state-of-the-art
statistical techniques.
Assessment
Statistics Canada’s efforts pertaining to accuracy and reliability are assessed by evaluating the extent to which the
Agency:
1. incorporates quality assurance measures into program and process design, implementation and execution
2. manages and monitors accuracy during implementation and execution of its statistical programs and
processes
3. assesses accuracy and reliability, both pre-release and post-release, and communicates the results.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to promote accuracy and reliability in its
statistical programs. Below is a list of such initiatives, in groups that correspond to the items under Assessment.
Statistics Canada – Catalogue no. 12‑586‑X
39
Statistics Canada Quality Assurance Framework
H.1 Statistics Canada incorporates quality assurance measures into program and process
design, implementation and execution
• Design and implement a process quality management plan, which includes quality assurance actions
and practices to ensure that the process reliably produces the desired results and products, and
indicators reecting the effectiveness of the quality assurance actions and practices. The process quality
management plan: includes results from intermediate processing steps; uses information obtained to
correct problems detected and reduce error incidence; and also includes process control of manual
operations, as needed.
f Directive and Guidelines for the Validation of Statistical Outputs
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
• Establish accuracy targets for key estimates (variable and domain). Program areas dene requirements for
corporate performance indicators of accuracy, including the contribution of non-response adjustment.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Guidelines on coefcients of variation
f Corporate performance indicators (coefcient of variation)
• Maintain or improve accuracy while minimizing response burden and cost by identifying and considering
alternate sources of the same or similar data, including existing survey data and administrative data. The
reduction in response burden is managed in part by the corporate committee responsible for administrative
data management.
f Administrative Data Management Committee
• Study and evaluate new and alternative statistical methods. Consider their impact on accuracy, reliability
and other dimensions of quality as part of the evaluation of design alternatives. The research is guided by
an advisory committee of experts and supported by a research fund.
f Advisory Committee on Statistical Methods
f Methodology Research Block Fund
• Develop expertise in statistical techniques and improve accuracy through incorporating such renements.
Examples of such techniques include calibration, benchmarking and seasonal adjustment.
f Time Series Research and Analysis Centre
• Establish, maintain and periodically evaluate survey frames. The use of centrally-maintained frames ensures
up-to-date and consistent coverage and classication of target populations for survey programs. Program
areas assess coverage of their target population by the proposed survey frame, for the entire population
and signicant sub-populations.
• Objectively weigh trade-offs between accuracy and other factors, including relevance, timeliness, cost and
response burden, both before deciding to undertake a survey and when planning each aspect of it.
H.2 Statistics Canada manages and monitors accuracy during implementation and
execution of its statistical programs and processes
• Execute a process quality management plan. Monitor and interpret indicators reecting the effectiveness
of quality assurance actions and practices. Process quality monitoring is integrated into regular on-going
management functions.
• Assess and validate administrative and other source data using standardized guidelines for detecting and
correcting errors and assessing accuracy.
f Administrative Data Management Committee
Statistics Canada – Catalogue no. 12‑586‑X
40
Statistics Canada Quality Assurance Framework
• Ensure effective questionnaire design through adequate justication for each question asked, standardized
terminology, banks of common questions, and appropriate pre-testing and review. A corporate resource
centre is mandated to review and evaluate each questionnaire against these standards.
f Questionnaire Design Resource Centre
f Policy on the Review and Testing of Questionnaires
• Implement adequate measures for encouraging response, following-up non-response and compensating
for total and item non-response.
H.3 Statistics Canada assesses accuracy and reliability, both pre-release and post-release,
and communicates the results
• Evaluate the accuracy and reliability of estimates. Program areas conduct an evaluation after each survey
cycle and document results. They track differences between target and realized values of measures of
accuracy and analyze sampling errors and non-sampling errors in the post-release evaluation of each
survey cycle. Reliability can then be studied by analyzing errors over time. Program areas identify best
practices, lessons learned and insights gained, and then recommend improvements. This information is
shared with other programs within Statistics Canada.
• Address quickly and uniformly errors detected in published estimates and inform users transparently.
f Directive on Corrections to Daily Releases and Statistical Products
• Ensure that preliminary or revised data are clearly identied as such to users, and that users are provided
with the accuracy indicators necessary to make effective use of them. Produce and make public
information that quanties differences between preliminary, nal and revised estimates, and use this
information to improve the survey process.
• Report data quality and methodology information necessary for data users. Require discussion of sources
of error and estimates of accuracy for key variables and domains as part of data quality and methodology
information provided to users. Periodically compile and disseminate quality reports. Include both a
qualitative and quantitative analysis of all types of errors.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Directive on Documenting Statistical Metadata
f Corporate performance indicators (coefcient of variation and response rate)
Statistics Canada – Catalogue no. 12‑586‑X
41
Statistics Canada Quality Assurance Framework
I. Timeliness and punctuality
Description
Statistics Canada strives to make its data products available as quickly as possible and to respect its
commitments to data availability. The timeliness of a data product is dened as the length of time between the
end of the reference period (or the reference date) to which the data relate and the date the product is made
available. Punctuality refers to the difference between planned and actual availability.
Alternate denitions may be used for other products and services. For example, timeliness of a custom data
request may be dened as the length of time between the receipt and delivery of a request. For new initiatives,
the denition of timeliness may be extended to include development time. Using a new survey as an example,
timeliness may be dened as the length of time between the commitment to conduct the survey and the release of
its data.
These denitions view timeliness and punctuality from the user perspective (i.e., relative to release or delivery), and
will be the basis of discussion in this chapter. Despite efforts to introduce innovation and to effectively manage
resources, gains in timeliness may require or lead to affecting other dimensions of quality. Consequently, the
impact on other dimensions of quality must be considered when evaluating efforts to improve timeliness.
Assessment
Statistics Canada’s efforts pertaining to timeliness and punctuality are assessed by evaluating the extent to which
the Agency:
1. denes policies that reect its commitment that Canadians have equitable and timely access to its data
products and information releases
2. effectively manages user expectations and other issues related to timeliness and punctuality
3. communicates release dates transparently and ensures equitable and timely access to data
4. implements methods to evaluate timeliness and punctuality and to improve outcomes.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to promote timeliness and punctuality in its
statistical programs. Below is a list of such initiatives, in groups that correspond to the items under Assessment.
I.1 Statistics Canada defines policies that reflect its commitment that Canadians have
equitable and timely access to its data products and information releases
• Address both scheduling of release dates and how users and stakeholders are informed about them. The
policies outline release procedures and suggest timeliness targets, and they allow for exibility (i.e., release
procedures appropriate for the type of data product, and timeliness targets that reect the length of the
reference period). Timeliness targets are guided by international dissemination standards.
• Outline the procedures for modifying release dates. Users and stakeholders are informed of the reasons for
any changes and of new release dates. Further, the policies outline scheduled revisions to data products
and inform users about unplanned corrections as quickly and transparently as possible.
• Eliminate incentives for interested parties to inuence or delay a particular release for their own benet
through transparency regarding release dates.
• Implement practices to support the Agency’s commitment to monitor release dates and track divergence
from them.
f Policy on Ofcial Release
f Directive on the Distribution of Registered Statistics Canada Products
f Policy on Communications and Federal Identity
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=30683)
Statistics Canada – Catalogue no. 12‑586‑X
42
Statistics Canada Quality Assurance Framework
I.2 Statistics Canada effectively manages user expectations and other issues related to
timeliness and punctuality
• Consult data users and stakeholders on issues related to timeliness and punctuality. This is done
by engaging with data users and stakeholders through such practices as meetings of the Chief
Statistician and senior managers with counterparts internationally and in other federal departments,
federal–provincial–territorial meetings, internal and external advisory committees and working groups.
f Advisory Groups (http://www.statcan.gc.ca/eng/about/relevant)
f National Statistics Council
f Professional Advisory Committees
f Inter-departmental committees
f Federal-Provincial-Territorial (FPT) Consultative Council on Statistical Policy
f Subcommittees FPT Consultative Council on Statistical Policy
f Stakeholders (http://www.statcan.gc.ca/eng/about/stakeholders)
f International Organizations and Expert Groups
(http://www.statcan.gc.ca/eng/reference/other-autres-eng)
• Give explicit consideration to timeliness in design, planning and project management. A production
schedule is created in consultation with the collection, processing and dissemination teams. The schedule
is effectively managed through collaboration between the project team and Statistics Canada’s corporate
collection service. Conicts, lapses and other threats to punctuality are identied and evaluated based
on their impact and likelihood. Contingency plans and other recourses are elaborated and employed as
needed.
• Include quality assurance, peer and professional review and other preventative measures in the timelines.
Trade-offs between timeliness, other dimensions of quality and cost are managed effectively so that tness
for use, scal responsibility and other Statistics Canada priorities are not compromised.
f Policy on Peer and Institutional Review
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
• When feasible, make preliminary data or other leading indicators available, with guidance on their quality
and appropriate use. Differences in magnitude and direction between preliminary, revised and nal
estimates are tracked and used to assess the timeliness–accuracy balance. Persistent or predictable
biases in preliminary data are identied and, when possible, addressed in estimation.
• Effectively manage user expectations regarding timeliness and data availability through communication
of Agency priorities and constraints. Feedback on the impact of timeliness on relevance to users and
stakeholders is of particular interest to the Agency.
I.3 Statistics Canada communicates release dates transparently and ensures equitable and
timely access to data
• Publish release dates well in advance. A single release calendar includes all key economic indicators and
major releases planned in the next 12 months. The release calendar is easily accessible with a link on the
main page of The Daily and gives users and stakeholders sufcient advance notice of upcoming releases.
f Release schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
• Require that all releases pass through a uniform dissemination service, where The Daily is Statistics
Canada’s ofcial release bulletin. It is made public at a standard time and port of entry (i.e., 8:30 a.m.
Eastern time).
f Policy on Ofcial Release
f Policy on Communications and Federal Identity
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=30683)
Statistics Canada – Catalogue no. 12‑586‑X
43
Statistics Canada Quality Assurance Framework
f Directive on the Distribution of Registered Statistics Canada Products
f The Daily (http://www.statcan.gc.ca/dai-quo/index-eng.htm?HPA)
• Correct errors in a structured, timely and transparent fashion.
f Directive on Corrections to Daily Releases and Statistical Products
• Work with the media and others to maximize exposure and awareness of Statistics Canada’s releases.
Releases are scheduled so that they do not overshadow each other and do not coincide with other
major (planned) news events. User and media anticipation of data releases is built through effective
communication and outreach activities. Various social media (including, Facebook, Twitter, YouTube,
Statistics Canada blog and “Chat with an expert”) are used to communicate with users. A media
spokesperson is always available on release day. Further, many free products are available on the website
and the re-dissemination of Statistics Canada information products is encouraged through an open-data
licence agreement that requires no payment of fees for specic data products.
f Directive on Media Relations
f Stay connected (http://www.statcan.gc.ca/eng/sc/index)
f Statistics Canada blog (http://www.statcan.gc.ca/eng/sc/blog?MM)
f Chat with an expert (http://www.statcan.gc.ca/eng/chatwithanexpert?MM)
f Statistics Canada Open Licence Agreement (http://www.statcan.gc.ca/eng/reference/licence-eng)
I.4 Statistics Canada implements methods to evaluate timeliness and punctuality and to
improve outcomes
• Develop and implement effective project management tools. A departmental project management ofce
provides support in the development of common processes and tools to improve the timely delivery
of projects in a cost-effective fashion while adhering to quality standards and meeting client needs. A
supporting framework exists as a set of standard project management processes, templates and tools.
f Departmental Project Management Ofce
f Departmental Project Management Framework
• Develop, calculate and publish quality indicators related to timeliness and punctuality and use them to
improve outcomes.
f Corporate performance indicators (timeliness and punctuality) J. Accessibility and clarity
Statistics Canada – Catalogue no. 12‑586‑X
44
Statistics Canada Quality Assurance Framework
J. Accessibility and clarity
Description
Statistics Canada’s goal is to make its ofcial statistics and data products available to meet the information needs
of all Canadians. Accessibility refers to the ease with which users are able to identify, obtain and use statistical
products and services. Clarity refers to the degree to which metadata and other information are provided so that
users are able to locate and select products or services that correspond to their needs.
Accessibility and clarity apply to all Statistics Canada products and services: statistical products (i.e., aggregate
tables, analytical articles), data access programs and other requests (i.e. custom surveys, specialized tabulations,
services). They also apply to metadata and other support provided with a product or service. Many factors that
inuence data availability and access, such as dissemination policies and data access systems, are determined
by Agency-wide initiatives. Individual program areas also play a vital role by designing statistical products and
providing them in formats that meet user needs. User experience is an important aspect to consider when
evaluating accessibility and clarity; i.e., from beginning a search, to identifying and obtaining a product or service
and completing the related work.
The Agency must balance the increasing demand for data and the desire to minimize barriers to access with
its obligation to protect the privacy and condentiality of respondents and to ensure equitable access for all
Canadians.
Assessment
Statistics Canada’s efforts pertaining to accessibility and clarity are assessed by evaluating the extent to which the
Agency:
1. publicizes products and services, including the availability of metadata and other supplementary resources
2. delivers products and services in an impartial and open manner, and ensures equitable access by
minimizing cost barriers
3. facilitates the nding of information about products and services so that users can make informed
decisions about which ones meet their needs
4. enables the use of its products, by providing clear procedures on accessing them, providing products in
formats compatible with user requirements, and ensuring barrier-free redistribution of its data
5. fosters correct use of information products by using clear, audience-appropriate presentation, combined
with informative and useful resource materials
6. supports and promotes its data-access programs and services.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to promote accessibility and clarity in its
statistical programs. Below is a list of such initiatives, in groups that correspond to the items under Assessment.
J.1 Statistics Canada publicizes products and services, including the availability of
metadata and other supplementary resources
• Work with the media and others to maximize awareness, in particular by encouraging re-dissemination of
Statistics Canada information products through use of modern information technology and an open-data
licence agreement that requires no payment of fees for specic data products. This approach includes
applications for smartphone and tablet access, a presence on social media (including Facebook, Twitter
and YouTube), a regular blog and a facility to interact with Agency specialists.
f Stay connected (http://www.statcan.gc.ca/eng/sc/index?MM)
f Statistics Canada blog (http://www.statcan.gc.ca/eng/sc/blog?MM)
f Chat with an expert (http://www.statcan.gc.ca/eng/chatwithanexpert?MM)
f Statistics Canada Open Licence Agreement (http://www.statcan.gc.ca/eng/reference/licence-eng)
f Directive on Licensing
Statistics Canada – Catalogue no. 12‑586‑X
45
Statistics Canada Quality Assurance Framework
• Provide special and dedicated services for members of the media.
f Media enquiries (http://www.statcan.gc.ca/dai-quo/smr01/smr01_000-eng.htm)
f Directive on Media Relations
• Publish release dates well in advance. A single release calendar includes all key economic indicators and
major releases planned in the next 12 months. The release calendar is easily accessible with a link on the
main page of The Daily and gives users sufcient advance notice of upcoming releases.
f Release schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
J.2 Statistics Canada delivers products and services in an impartial and open manner, and
ensures equitable access by minimizing cost barriers
• Require that all releases pass through a uniform dissemination service, where The Daily is Statistics
Canada’s ofcial release bulletin. It is released at a standard time and port of entry (i.e., 8:30 a.m. Eastern
time) each business day.
f Policy on Ofcial Release
f Directive on the Distribution of Registered Statistics Canada Products
f Release schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
f The Daily (http://www.statcan.gc.ca/dai-quo/index-eng.htm?HPA)
f Policy on Communications and Federal Identity
(http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=30683)
• Promote availability by specifying requirements for non-release or restricted release of data. Ensure
decisions regarding release are independent of funding source or stakeholder.
f Directive on Access to Information and Privacy
• Minimize the barrier of cost by providing standard products, for each statistical program, free of charge on
the Statistics Canada website.
• Produce and promote a broad range of statistical products, and dene a fair and balanced policy for
funding these on a cost-recovery basis. For example, research papers, analytical products, specialized
tabulations, as well as the design and implementation of customized surveys, are available.
f Research Papers and Analytical Products (http://www5.statcan.gc.ca/researchers-chercheurs/
index.action?lang=eng&start=1&end=25&search=&sort=0&themeId=0&date=&series=&author=&the
meState=-1&dateState=-1&seriesState=-1&authorState=-1&showAll=false&univ=7&MMI)
f Information on Custom Surveys (http://www.statcan.gc.ca/eng/cs/overview)
f Pricing of Cost-Recovery Projects
f Policy on Service Standards for External Fees
J.3 Statistics Canada facilitates the finding of information about products and services so
that users can make informed decisions about which ones meet their needs
• Provide modern search capabilities on a single, well-known point of entry to Statistics Canada’s online
presence, as well as a catalogue and customer service to help users nd the desired products. Manage
inquiries clearly and consistently.
f Statistics Canada website (http://www.statcan.gc.ca/)
f Standards of Service to the Public (http://www.statcan.gc.ca/eng/about/service/standards)
• Provide clear descriptions of products and services, as well as links to complementary information, such as
methodological notes and quality measures through the Integrated Metadatabase.
f Integrated Metadatabase
Statistics Canada – Catalogue no. 12‑586‑X
46
Statistics Canada Quality Assurance Framework
• Ensure that Statistics Canada products and metadata are stored in centralized repositories in a logical
and organized fashion. The dissemination framework and the socioeconomic database CANSIM are two
such examples.
f Statistics Canada Dissemination Model
f CANSIM (http://www5.statcan.gc.ca/cansim/a01?lang=eng)
• Ensure that past products, including those produced before electronic release and storage,
are properly archived.
f Web Archiving Directive
J.4 Statistics Canada enables the use of its products, by providing clear procedures on
accessing them, providing products in formats compatible with user requirements,
and ensuring barrier-free redistribution of its data
• Tailor outputs to conform to medium-specic content guidelines and standards.
f Our Commitment to Accessibility (http://www.statcan.gc.ca/eng/reference/accessnotice)
• Ensure that outputs are available in alternate formats to accommodate users with special needs (e.g.,
outputs in large print or Braille available free of charge, upon request).
• Ensure that outputs are compatible with common user software.
• Develop tools that automate data retrieval and display, and offer them free of charge.
• Encourage citation, quotation, and transmission (re-dissemination) of Statistics Canada information
products by the media and other users through the use of modern information technology and social
media, and an open-data licence agreement that requires no payment of fees for specic data products.
f Policy on Document Management
f Directive on Licensing
f Statistics Canada Open Licence Agreement (http://www.statcan.gc.ca/eng/reference/licence-eng)
• Provide channels for users to interact with Statistics Canada, to communicate their needs, to get support,
and to provide feedback.
J.5 Statistics Canada fosters correct use of information products by using clear,
audience-appropriate presentation, combined with informative and useful
resource materials
• Ensure the consistency of messages by using a controlled media-spokesperson strategy.
f Directive on Media Relations
• Ensure clear and comprehensible presentation of information products by adhering to publication
guidelines and other content standards.
f Statistics Canada Publishing Guidelines
f Policy on Highlights of Publications
• Ensure consistency of information products over time by specifying standard concepts, denitions and
classications, and adopting best practices.
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
• Minimize errors and inconsistencies in information products by implementing quality assurance procedures,
including staff training and institutional and peer reviews.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Policy on Peer and Institutional Review
Statistics Canada – Catalogue no. 12‑586‑X
47
Statistics Canada Quality Assurance Framework
• Ensure users are able to obtain supporting materials, such as metadata, user guides, help lines, quality
reports and supplemental software.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
• Share analytical tools developed by Statistics Canada to foster the consistent use of the data and to make
typical data transformations easier for users.
J.6 Statistics Canada supports and promotes its data-access programs and services
• Facilitate researcher access to microdata without compromising data security and condentiality, by using
secure remote-access tools and exible stafng arrangements.
f Policy on Microdata Access
f Directive on the Use of Deemed Employees
f Directive on the Security of Sensitive Statistical Information
f Research Data Centres (http://www.statcan.gc.ca/rdc-cdr/index-eng.htm)
f Federal Research Data Centres (http://www.statcan.gc.ca/eng/rdc/frdc)
f Canadian Centre for Data Development and Economic Research
(http://www.statcan.gc.ca/cder-cdre/index-eng.htm)
f Data Liberation Initiative (http://www.statcan.gc.ca/eng/dli/dli)
f Real Time Remote Access (http://www.statcan.gc.ca/eng/rtra/rtra)
• Facilitate access to Statistics Canada’s staff and expertise through consultation, participation in peer
groups, and international and private-sector collaboration.
• Dene objective release or suppression criteria for custom tabulations and other special requests.
Implement a vetting process that includes condentiality measures.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Policy on Microdata Release
f Guidelines for the Release of Microdata Files
• Ensure that software and other non-data products, such as generalized systems, are marketed effectively
to benet Statistics Canada and users.
f Policy on Software Dissemination
Statistics Canada – Catalogue no. 12‑586‑X
48
Statistics Canada Quality Assurance Framework
K. Coherence and comparability
Description
Statistics Canada aims to develop, produce and disseminate statistics that are consistent and comparable over
time and across jurisdictions, and can be used in conjunction with information from other sources. Coherence
of statistics refers to the extent to which they are logically consistent in terms of denition and measurement and
thus can be reliably combined in different ways and for various uses. Comparability refers to the extent to which
differences over time or among sources can be attributed to changes in the true values of the statistics, and not to
changes in denition or measurement.
The use of standard concepts, denitions, classications and target populations promotes coherence and
comparability, as does the use of common methodology, statistical techniques and processes across surveys.
Lack of coherence or comparability is generally attributed to two primary sources: differences in concepts and
denitions; and differences in collection, processing and dissemination methods. In the rst case, the target of
measurement is not consistent; in the second, the process of measurement introduces inconsistency.
Assessment
Statistics Canada’s efforts pertaining to coherence and comparability are assessed by evaluating the extent to
which the Agency:
1. applies concepts, denitions, frameworks and protocols uniformly to ensure consistency across its
statistical programs and to facilitate comparison over time and with alternate sources of related information
2. designs and implements common methods for data collection, processing and dissemination to minimize
inconsistency during operations and to manage change between cycles
3. veries outputs, including comparisons with estimates from other sources and related information
4. provides documentation and other supplementary information and applies statistical techniques to help
users make meaningful comparisons over time and with other sources.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to promote coherence and comparability in its
statistical programs. Below is a list of such initiatives, in groups that correspond to the items under Assessment.
K.1 Statistics Canada applies concepts, definitions, frameworks and protocols uniformly to
ensure consistency across its statistical programs and to facilitate comparison over
time and with alternate sources of related information
• Develop and maintain a protocol for statistical standards. Statistical standards apply to concepts,
denitions, frameworks, units, variables, classication systems and target populations. The Agency
promotes and monitors the adoption of statistical standards uniformly across statistical programs.
Exceptions to statistical standards are justied and documented.
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
• Maintain and disseminate information on statistical standards in a corporate metadata repository. The
elements in the repository are updated, changes are tracked and users are informed.
f Denitions, Data Sources and Methods (http://www.statcan.gc.ca/eng/concepts/index)
f Integrated Metadatabase
f Directive on Documenting Statistical Metadata
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
Statistics Canada – Catalogue no. 12‑586‑X
49
Statistics Canada Quality Assurance Framework
• Provide structures and tools that support administration and governance of statistical standards. A
corporate committee assists and advises on the development, approval and application of statistical
standards and metadata within the Agency’s programs. A specic division is responsible for all the
classications and standards - including industry, product, occupation and education classications,
geography and economic accounts - used by Statistics Canada.
f Corporate Management Framework (http://www.statcan.gc.ca/eng/about/cmf)
f Methods and Standards Committee
f Standards Division
f Standards Bulletin
• Participate in the development of national and international standards and other frameworks, and
encourage consistency with them. Statistics Canada participates in numerous development groups, such
as those of the United Nations Statistics Division and the United Nations Economic Commission for Europe.
The Agency is also involved in groups developing standard classication systems such as the North
American Industrial Classication System and the National Occupational Classication-Statistics.
f Denitions, Data Sources and Methods (http://www.statcan.gc.ca/eng/concepts/index)
• Set up cross-program committees to ensure that quantities being estimated bear relation to each
other through use of consistent terminology, commonly formulated questions and comparable variable
denitions. In particular, maximize consistency both within (e.g., between quarterly and annual estimates,
and between preliminary and nal estimates) and across statistical programs.
f Policy on Standards (http://www.statcan.gc.ca/eng/about/policy/standards)
f Policy on the Review and Testing of Questionnaires
f Questionnaire Design Resource Centre
• Participate in various external fora, such as working groups and subject-matter expert groups, to
engage with other national statistical ofces in recognizing and promoting national and international best
practices, arithmetic and accounting identities and other standards. Ensure communication of information
so that program areas are acquainted with and guided by national and international best practices and
cutting-edge methods.
f International Cooperation (http://www.statcan.gc.ca/eng/about/cooperation)
• Ensure program areas periodically assess compliance with standards and frameworks as well as
consistency and comparability with related administrative data, other estimates from Statistics Canada and
estimates from elsewhere.
K.2 Statistics Canada designs and implements common methods for data collection,
processing and dissemination to minimize inconsistency during operations and to
manage change between cycles
• Use standardized frameworks and systems to support process management.
f Generic Statistical Business Process Model
(http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0)
f Generic Statistical Information Model
• Develop and use common frames and processing environments. Examples of these are the Business
Register, Household Survey Frame Service, Social Survey Processing Environment and the Integrated
Business Statistics Program.
• Optimize use of generalized systems and corporate services.
f Generalized Systems Resource Centre
f Resource and support centres
Statistics Canada – Catalogue no. 12‑586‑X
50
Statistics Canada Quality Assurance Framework
• Ensure internal consistency of outputs during operations. In particular, ensure that changes to denitions or
other inputs are incorporated, arithmetic or accounting operations do not lead to discrepancies
(e.g., rounding)
8
and that outputs of complementary processes are integrated properly.
f Quality Guidelines
(http://www5.statcan.gc.ca/olc-cel/olc.action?objId=12-539-X&objType=2&lang=en&limit=0)
f Quality Assurance Framework
f Directive and Guidelines for the Validation of Statistical Outputs
f Quality Secretariat
• Implement an appropriate structure for approval and testing when developing and implementing new
software applications and other processing tools. Develop appropriate guidelines and other oversight tools
and structures, as needed.
• Ensure consistency of message and convenient user access through a corporate dissemination framework
and access tools such as The Daily and tables from the Agency’s socioeconomic database CANSIM.
f Statistics Canada Dissemination Model
f Directive on Media Relations
f Directive on Corrections to Daily Releases and Statistical Products
f The Daily (http://www.statcan.gc.ca/dai-quo/index-eng.htm?HPA)
f CANSIM (http://www5.statcan.gc.ca/cansim/a01?lang=eng)
K.3 Statistics Canada verifies outputs, including comparisons with estimates from other
sources and related information
• Reconcile estimates and outputs with other comparable statistical and administrative sources on similar
subjects, including previous estimates from the same statistical program. Any differences are identied and
substantiated.
f Directive on the Management of Aggregate Statistics
• Elaborate a strategy for detecting and correcting errors in previously released data. Retroactive corrections
are implemented, if necessary.
f Directive on Corrections to Daily Releases and Statistical Products
• For redesigns and other signicant changes in survey methodology or international standards, explain
breaks in series and develop methods for reconciliation. Produce historical revisions when deemed
appropriate. For example, a data series is updated when a classication framework is revised.
• Solicit feedback from users, in particular their experience in producing historical series, comparing
estimates between statistical programs, and comparing the Agency’s outputs with those of other sources.
Identify inadequacies and issues.
K.4 Statistics Canada provides documentation and other supplementary information and
applies statistical techniques to help users make meaningful comparisons over time
and with other sources
• Document changes to concepts, denitions, classications and methods, including changes between
preliminary and nal estimates. Changes are expressed in quantitative terms when possible.
f Directive on Documenting Statistical Metadata
8.
Ensuring consistency and comparability does not necessarily imply full numerical consistency.
Statistics Canada – Catalogue no. 12‑586‑X
51
Statistics Canada Quality Assurance Framework
• Produce and distribute quality reports and other supporting methodological documentation that address
internal consistency, comparability over time and comparability with other statistics. Changes to important
parameters and methods are emphasized, as well as deviations from international standards and other
practices.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
• Provide quantitative and qualitative information that allows users to compensate for differences in
periodicity, survey population, variable denitions and processing methods when comparing data from
different sources and comparing data over time.
f Integrated Metadatabase
f Directive on Media Relations
• Develop expertise in statistical techniques and produce data products that incorporate such renements.
Examples of such techniques include calibration, benchmarking and seasonal adjustment.
f Time Series Research and Analysis Centre
f Advisory Committee on Statistical Methods
f Methodology Research Block Fund
Statistics Canada – Catalogue no. 12‑586‑X
52
Statistics Canada Quality Assurance Framework
L. Interpretability and management of metadata
Description
Statistics Canada has a responsibility to provide both statistical information and the necessary tools and guidance
to ensure its proper interpretation and appropriate use. Without such tools and guidance, statistical information
is of limited use and its interpretation becomes subject to the bias of the user. There is also a risk of information
products becoming a liability to the Agency if they are misunderstood or misinterpreted as supporting a particular
agenda. Interpretability refers to the ease with which users are able to understand, properly analyze and draw
correct inferences from statistical information. Statistical metadata are dened as “information about statistical
data and the statistical business process”.
9
Thus managing interpretability primarily involves providing metadata
10
so that the statistical information can be understood and used appropriately.
The information needed to understand statistical data falls under three broad headings. Concepts, variables
and classifications that underlie the data dene what is being measured or estimated in the statistical
information product, and are required by users to assess the product’s relevance to their needs. Information on
the methodology of data collection, processing and dissemination addresses how the concept is being
measured, in particular whether methods were scientic, objective and carefully implemented. If users are aware
of the methodology, they can choose the appropriate analytical tools. Data-quality measures quantify how well
the concept was measured and help users associate the appropriate level of condence with the results.
Assessment
Statistics Canada’s efforts pertaining to interpretability are assessed by evaluating the extent to which the Agency:
1. enshrines metadata requirements in policies and practices
2. provides users with the basic information they need to interpret data
3. strives to ensure the correct interpretation of its data releases by the media and the public
4. establishes metadata requirements for public-use microdata les and custom data products.
Implementation
Statistics Canada undertakes a broad range of specic initiatives to provide, manage and disseminate metadata
and thus promote interpretability in its statistical programs. Below is a list of such initiatives, in groups that
correspond to the items under Assessment.
L.1 Statistics Canada enshrines metadata requirements in policies and practices
• Dene policies that specify metadata requirements for every statistical product and data release, including
the provision of information on data quality and methodology. Responsibility is directly assigned to ensure
that these requirements are met.
f Directive on Documenting Statistical Metadata
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
• Dene a governance for metadata. To ensure that statistical metadata are managed as a corporate
resource, governance for statistical metadata needs to be established at the Agency level. This will allow
the management of statistical metadata to be coordinated across the Agency, in particular for statistical
metadata that cross domains, organizational groups or systems.
f Metadata Architecture Modernization
9.
The source of this definition of statistical metadata is Statistics Canada’s Strategy for Statistical Metadata Management. In summary, it defines statistical metadata as information relating
to designing, producing and disseminating statistical products. It distinguishes statistical metadata from other information that contributes to the production of statistical products, including
regulatory, financial management, administrative and security information.
10.
Statistical metadata are a specific type of metadata, which is commonly defined as “information about information”. As the focus is statistical information, the term “metadata” will be used
to refer to statistical metadata.
Statistics Canada – Catalogue no. 12‑586‑X
53
Statistics Canada Quality Assurance Framework
• Dene a metadata management structure. The Agency’s metadata architecture includes the establishment
of operational governance as well as formalization of centres of responsibility for statistical metadata.
f Strategy for Statistical Metadata Management
• Ensure that metadata are aligned with statistical business processing standards.
f Generic Statistical Business Process Model
(http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0)
• Ensure that metadata are reviewed regularly and updated when needed, particularly prior to data releases.
• Dene processes to ensure that the exchange of metadata between projects is transparent and efcient.
L.2 Statistics Canada provides users with the basic information they need to interpret data
• Establish an integrated metadata repository that contains the information needed to describe each of
Statistics Canada’s data holdings.
f Directive on Documenting Statistical Metadata
f Integrated Metadatabase
• Ensure that metadata are released concurrently with the data product. At release, standard documentation
is produced and disseminated such as the Integrated Metadatabase (which publishes denitions, data
sources and methods for each survey and statistical program), user guides (which accompanies a dataset
release to provide background details on the data and methods), and technical reports (which explain
procedures used in the creation of the statistics). Program areas also produce quality reports and other
supporting methodological information needed for users to make meaningful comparisons over time
and with other sources of information. The above reports provide information on changes to concepts,
denitions, classications and methods, as well as any deviations from international standards and other
practices.
f Policy on Informing Users of Data Quality and Methodology
(http://www.statcan.gc.ca/eng/about/policy/info-user)
f Release Schedule (http://www.statcan.gc.ca/dai-quo/cal3-eng.htm)
f Integrated Metadatabase
f Directive on Documenting Statistical Metadata
• Provide metadata in formats that are convenient to users and in language that is understood by users and
not obscured by statistical jargon.
• Include the usefulness and adequacy of metadata in discussions with data users and stakeholders.
f Consulting Canadians (http://www.statcan.gc.ca/eng/consultation/index-eng?MM)
f Consultation FAQs (http://www.statcan.gc.ca/eng/consultation/faq-eng)
f Client satisfaction (http://www.statcan.gc.ca/eng/consultation/calend-t1-eng)
L.3 Statistics Canada strives to ensure the correct interpretation of its data releases by the
media and the public
• Implement a clear, timely and consistent communications strategy. Such a strategy ensures that the media
and the public understand and draw the correct inferences from information in a data release, in particular
when they rst read or hear about it.
• Publicly address misinterpretations of the Agency’s data.
f Policy on Highlights of Publications
f Directive on Media Relations
Statistics Canada – Catalogue no. 12‑586‑X
54
Statistics Canada Quality Assurance Framework
L.4 Statistics Canada establishes metadata requirements for public-use microdata files and
custom data products
• Provide a record layout and documentation of concepts and denitions.
• Provide a “codebook”, data dictionary or other source of coding and classication details.
• Provide metadata in a format compatible with the data le (i.e., in the same software).
f Policy on Information Management
Statistics Canada – Catalogue no. 12‑586‑X
55
Statistics Canada Quality Assurance Framework
References
Brackstone, Gordon. 1999. “Managing data quality in a statistical agency”. Survey Methodology. Vol. 25, n
o
2.
Statistics Canada Catalogue n
o
12-001-X, pp. 139-149.
http://www.statcan.gc.ca/pub/12-001-x/1999002/article/4877-eng.pdf (accessed August 29, 2016)
Brackstone, Gordon and Pamela White. 2002. “Data Stewardship at Statistics Canada.” Joint Statistical Meetings
of the American Statistical Association. New York, New York. August 11-15.
http://ww2.amstat.org/sections/SRMS/Proceedings/y2002/Files/JSM2002-000460.pdf
(accessed November 15, 2016)
Corporate Planning Division, Statistics Canada. March 2013. Continuity and Quality Maintenance Investment Plan
(CQMIP): Frequently Asked Questions and Guidelines.
Deming, W. Edwards. 1982. Out of the Crisis. The MIT Press (2
e
ed. 1986).
European Statistical System. n.d. Quality Assurance Framework of the European Statistical System (version 1.2).
http://ec.europa.eu/eurostat/documents/64157/4392716/ESS-QAF-V1-2nal.pdf/bbf5970c-1adf-46c8-afc3-
58ce177a0646 (accessed August 29, 2016)
European Statistical System. September 28, 2011. European Statistics Code of Practice.
http://ec.europa.eu/eurostat/documents/3859598/5921861/KS-32-11-955-EN.PDF/5fa1ebc6-90bb-43fa-888f-
dde032471e15 (accessed August 29, 2016)
Ofce of National Statistics (UK). September 2013. Guidelines for measuring statistical quality (Version 4.1).
http://www.ons.gov.uk/ons/guide-method/method-quality/quality/guidelines-for-measuring-statistical-quality/
guidelines-for-measuring-statistical-output-quality.pdf (accessed August 30, 2016)
Statistics Canada. n.d.a. International Statistical Fellowship Program. Last updated: June 14, 2016.
http://www.statcan.gc.ca/eng/about/cooperation/isfp (accessed August 29, 2016)
Statistics Canada. n.d.b. Review of the July 2014 Labour Force Survey release. Last updated April 8, 2015.
http://www.statcan.gc.ca/eng/about/ir/lfs-2014 (accessed August 30, 2016)
Statistics Canada. 2015a. Corporate Business Plan Statistics Canada 2015-2016 to 2017-2018. Last updated
July 9, 2015.
http://www.statcan.gc.ca/eng/about/bp (accessed November 15, 2016)
Statistics Canada. 2015b. Policy on the Use of Administrative Data Obtained Under the Statistics Act.
Last updated March 14, 2016.
http://www.statcan.gc.ca/eng/about/policy/admin_data (accessed November 15, 2016)
Statistics Canada. June 3, 2011. Integrated Strategic Planning Process at Statistics Canada – An Overview.
Statistics Canada. April 2009. Report of the Task Force on Corporate Business Architecture. Last updated
March 19, 2015.
UK Statistical Authority. January 2009. Code of Practice for Official Statistics: Edition 1.0.
https://www.statisticsauthority.gov.uk/wp-content/uploads/2015/12/images-codeofpracticeforofcialstatisticsjanu
ary2009_tcm97-25306.pdf (accessed August 29, 2016)
Statistics Canada – Catalogue no. 12‑586‑X
56
Statistics Canada Quality Assurance Framework
United Nations Economic and Social Council. January 29, 2014. Fundamental Principles of Official Statistics.
http://unstats.un.org/unsd/dnss/gp/FP-New-E.pdf (accessed August 29, 2016)
United Nations Economic Commission for Europe. December 2013. Generic Statistical Business Process Model
(Version 5.0, December 2013).
http://www1.unece.org/stat/platform/display/GSBPM/GSBPM+v5.0 (accessed August 29, 2016)
United Nations Statistics Division. n.d. Annex 1: Template for a generic national quality assurance framework
(NQAF), as developed by the Expert Group on National Quality Assurance Frameworks.
http://unstats.un.org/unsd/dnss/docs-nqaf/2012-13-NQAF-E.pdf (accessed August 29, 2016)
United Nations Statistics Division. 28 August 2013. “NQAF 1-19 Checklist.” NQAF Checklist 2013 (for the review of
NQAF lines 1-19).
http://unstats.un.org/unsd/dnss/docs-nqaf/NQAF%20CHECKLIST%202013_28augFINAL.xls
(accessed August 30, 2016)
United Nations Statistics Division. February 8, 2012. Guidelines for the Template for a Generic National Quality
Assurance Framework (NQAF).
http://unstats.un.org/unsd/dnss/docs-nqaf/GUIDELINES%208%20Feb%202012.pdf (accessed August 29, 2016)
United Nations Statistics Division. February 3, 2012. Glossary (compiled by the Expert Group on National Quality
Assurance Frameworks)
http://unstats.un.org/unsd/dnss/docs-nqaf/NQAF%20GLOSSARY.pdf (accessed August 29, 2016)
Statistics Canada – Catalogue no. 12‑586‑X
57
Statistics Canada Quality Assurance Framework
