Guidance on Designing Healthcare External Evaluation

Authors:

Triona Fortune, Elaine O’ Connor and Barbara Donaldson

Guidance on Designing

Healthcare External

Evaluation Programmes

including Accreditation

2015

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Table of contents

Acknowledgements iii

Foreword ISQua iv

Foreword World Bank & WHO vi

List of Tables vii

Glossary of Terms viii

Introduction 2

Chapter 1: Why develop an external evaluation programme? 4

1.1 The growing demand for external evaluation in health and social care 4

1.2 Models of external evaluation 5

1.3 Beneﬁts of external evaluation 8

1.4 Challenges for external evaluation programmes 10

Chapter 2: Establishing the fundamentals 12

2.1 Deﬁning the purpose of the external evaluation programme 12

2.2 Deﬁning the scope of the external evaluation programme 15

2.3 Establishing the role of government 18

2.4 Determining incentives 21

2.5 Developing relationships with stakeholders 24

Chapter 3: Setting up the external evaluation organisation 27

3.1 Establishing a preliminary board or advisory committee 27

3.2 Proposing a governance board and framework 28

3.3 Funding of the programme 30

3.4 Setting up strategic, operational and ﬁnancial management systems 33

3.5 Timeframes 35

Chapter 4: Developing the standards 37

4.1 The role of standards 37

4.2 Principles for standards 38

4.3 Referencing to quality dimensions 39

4.4 Developing the measurement system 40

Chapter 5: Developing assessment methodologies 43

5.1 Selection, training and evaluation of surveyors 43

5.2 Developing the survey management process 46

5.3 Establishing the accreditation / certiﬁcation process 48

5.4 Quality Assurance 51

Chapter 6: Evaluating systems and achievements 52

6.1 Measuring performance internally 52

6.2 Evaluating independently 53

6.3 Monitoring by regulatory agencies 53

6.4 Accrediting the external evaluation bodies 53

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Table of contents

Conclusions 54

References 55

Bibliography 58

Useful web resources 62

Appendix 1: Case Studies 64

Appendix 1a. Danish Case Study 64

Appendix 1b: Jordanian Case Study 67

Appendix 1c. New Zealand Case Study 69

Appendix 1d: Practice Incentive Program (PIP) 72

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

8-11 Lombard Street East, Dublin 2, D02 Y729, Ireland.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

III

Acknowledgements

This document is based on the Toolkit for Accreditation Programs, 2004 developed by Charles

Shaw. The International Society for Quality in Health Care would like to thank the following for

their contributions to the development of this document:

Reviewers

Mark Brandon, AACQA - Australia

Stephen Clark, AGPAL - Australia

Heleno Costa Junior, CBA - Brazil

Carsten Engel, IKAS - Denmark

Eric de Roodenbeke, International Hospital Federation

Carlos Goes de Souza, CHKS - UK

Helen Healey, DAP BC - Canada

Salma Jaouni, HCAC - Jordan

Thomas Leludec, HAS - France

Hung-Jung Lin, JCT - Taiwan

Lena Low, ACHS - Australia

Kadar Marikar, MSQH - Malaysia

Wendy Nicklin, Accreditation Canada

BK Rana, NABH - India

Charles Shaw, Independent Consultant

Paul van Ostenberg, JCI - USA

Kees van Dun, NIAZ - The Netherlands

Stuart Whittaker, COHSASA - South Africa

Hongwen Zhao, WHO

Nittita Prasopa-Plaizier, WHO

Akiko Maeda, World Bank

Dinesh Nair, World Bank

Rafael Cortez, World Bank

A special acknowledgement to Akiko Maeda and the World Bank for supporting this project.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Foreword

Accreditation is an important tool for improving the care delivered by healthcare systems, and

one of the key roles of the International Society for Quality in Health Care (ISQua) has been to

accredit the accreditors. However, accreditation has to evolve to be beneﬁcial. An increase in

requests - especially from developing economies - for advice on establishing an accreditation

programme prompted ISQua to review two of its major tools: the Toolkit for Accreditation

Programs, 2004

, and Checklist for Development of New Healthcare Accreditation Programs,

2006

. The last decade has seen considerable changes, worldwide, to healthcare systems

and external evaluation programmes. To reﬂect these changes, a revision to the existing

guidance was deemed inadequate and this new Guidance manual was therefore developed. We

believe this document will be suitable for a much wider audience; it is designed for countries,

governments and policy makers within public or private, primary, secondary or tertiary

healthcare systems. It is also intended as an aid for funding and development agencies such as

the World Bank, international aid agencies, the World Health Organization (WHO), Ministries of

Health, other government agencies, groups and organisations who want to improve the quality

and safety of healthcare in their country, region or specialty area.

It has now been almost 100 years since the ﬁrst external evaluation programme, known as

accreditation, was established. Nearly every country currently has some form of external

evaluation, whether voluntary or mandatory. There are both “aﬁcionados” and critics of

healthcare accreditation. Anyone who has dealt with accreditors coming into their site has

likely felt that they were arbitrary, or focused on things that were less than important. However,

accreditation gets organisations to pay attention to things they might otherwise prefer to ignore

or put off. While it is sometimes voluntary, following a series of adverse events policymakers

then change it to mandatory in response. While traditionally accreditation was a programme

for developed economies, developing countries are now equally as interested. This document

has extended its scope beyond healthcare accreditation programmes to include other external

evaluation programmes such as certiﬁcation and licensing as they apply to organisations, not

individual practitioners. These programmes have different scopes and organisational coverage

but are based on the same principle of evaluating and improving performance against a deﬁned

set of standards, using external evaluators, to improve the safety and quality of health services

for the public.

Accreditation is not a panacea to address all quality improvement issues but it can provide

a systematic approach that identiﬁes areas where improvements are necessary, and when

mandatory, can “lift all the boats”, including some of the less strong entities within our

healthcare systems. When used with tools such as checklists and supported by technology, it

can become a powerful instrument for healthcare reform.

Developing an external evaluation system is a process that should be designed according to

each countries’ proﬁle. Firstly, the purpose should be clear and secondly, depending on the

desired outcome, a decision should be made as to whether a voluntary or mandatory system

is appropriate. This document is not designed as a rigid guideline, rather as a diverse range

of practices which should be discussed. It includes advice on best practices for governance,

developing standards and assessment methodologies. It also includes real case studies from

both developed and developing countries.

Healthcare continues to evolve; some of the key changes occurring today are that populations

are ageing, while technology is becoming smarter and the relationships between providers

and patients are tilting so that patients are much more empowered, and they are becoming

our partners. We all need to strive to reach country speciﬁc and global goals such as the World

Health Organization’s mandate on Universal Health Coverage (UHC) by 2020.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Governments will ultimately be responsible for providing UHC and they will be required to

demonstrate efﬁcient use of limited public funds while providing safe quality healthcare.

External evaluation systems can provide this assurance.

ISQua believes that accreditation can continue to be a powerful force for improvement in the

quality of care that is delivered. However, like all quality improvement initiatives, it must evolve

with the times to reﬂect the needs of our healthcare systems.

Professor David W. Bates

President International Society for Quality in Health Care

August 2015

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Foreword

World Bank and the World Health Organization

The public has a growing awareness of and expectation for their healthcare to be accountable,

safe, of high quality and responsive to their needs. Globally, healthcare costs are rising, putting

increasing burdens on both governments and healthcare organisations, as they try to meet

the growing challenges with limited resources. Governments are working towards Universal

Health Coverage (UHC) as a way to ensure that their populations have equitable access to safe,

high quality services, without suffering ﬁnancial hardship. The critical question remains: how

can countries maximise access whilst maintaining safe and quality services within affordable

margins?

External evaluation programmes, which include accreditation, certiﬁcation and licensing of

healthcare institutions, are among measures that can help improve organisational efﬁciency and

effectiveness as well as the safety and quality of services. However, implementation of these

programmes is not uniform. This may be due to a lack of resources or expertise or, importantly,

due to a lack of operational ‘know-how’ on the implementation of such programmes.

This report aims to provide a practical guide for setting up an external evaluation programme

at both a national and an organisational level. It will help governments and policy makers to

identify and determine health systems’ priorities and gaps, so they can re-orient healthcare

systems and policies to meet such growing challenges. The report offers a range of approaches

and practical steps on the setting up of external evaluation programmes, including creating an

enabling environment and developing human and system capacities.

Better implementation of external evaluation programmes can contribute to improved safety

by requiring services to meet standards, and by encouraging quality improvement through

organisational and individual professional development. Such programmes, if adopted

and implemented appropriately and consistently, will contribute to a more resilient, more

accountable, and more effective healthcare system in the long run.

It is hoped that this report will encourage governments and healthcare organisations to adopt

and implement external evaluation programmes in order to achieve safe, high quality, resilient

and sustainable health systems and services.

Timothy Grant Evans

Senior Director

Health Nutrition and Population Global Practice

The World Bank Group

Marie-Paule Kieny

Assistant - Director General

Health Systems and Innovation

World Health Organization

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

VII

List of Tables

Table Page No.

Table 1: Deﬁnitions of accreditation, certiﬁcation and licensing 8

Table 2: Comparison of capacity building and regulatory external evaluation 13

Table 3: Potential composition of a preliminary / interim board or advisory

committee

VIII

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Glossary of Terms

Accountability

Responsibility and requirement to answer for tasks or activities. This

responsibility may not be delegated and should be transparent to all

stakeholders.

Accreditation

A self-assessment and external peer review process used by health and

social care organisations to accurately assess their level of performance

in relation to established standards and to implement ways to

continuously improve the health or social care system.

Assessment

Process by which the characteristics and needs of patients, groups,

populations, communities, organisations or situations are evaluated or

determined so that they can be addressed. The assessment forms the

basis of a plan for services or action.

Assessor

Person who evaluates characteristics and needs. For external evaluation,

an assessor identiﬁes and evaluates evidence that set criteria are being

met and makes recommendations for action to address any gaps. Also

auditor, surveyor, external evaluator.

Benchmarking

Comparing the results of services’ or organisations’ evaluations to

the results of other interventions, programmes or organisations, and

examining processes against those of others recognised as excellent, as a

means of making improvements. Also benchmark.

Certiﬁcation

Process by which an authorised body, either a governmental or non-

governmental organisation (NGO), evaluates and recognises either an

individual, organisation, object or process as meeting pre-determined

requirements or criteria. The pre-determined requirements are set out in

standards which are developed speciﬁcally for the purpose of assessment.

The standards assess the performance of the organisation, object, process

or person, may focus on speciﬁc aspects of performance and may address

more than legal requirements.

Clients

Individuals or organisations being served or treated by the organisation.

Also patients, consumers, service users.

External

evaluation

Process in which an objective independent assessor gathers reliable

and valid information in a systematic way by making comparisons to

standards, guidelines or pathways for the purpose of enabling more

informed decisions and for assessing if pre-determined and published

requirements such as goals, objectives or standards have been met. An

organisation, object, process or individual may be assessed and evaluation

may be undertaken by peers, including organisations and professionals,

private professional auditors or consultants, purchasers / funders /

insurers, consumers / patients or governments.

Health Outcome

Health state or condition attributable to treatment, care or service

provided.

Leader

An individual who sets expectations, develops plans and implements

procedures to assess and improve the quality of the organisation’s

governance, management, clinical and support functions and processes.

Leadership

Ability to provide direction and cope with change. It usually involves

establishing a vision, developing strategies for producing the changes

needed to implement the vision, aligning people, motivating and inspiring

people to overcome obstacles.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Licensing

Process by which a governmental authority grants permission to an

individual practitioner or health and social care organisation to operate or

engage in an occupation or profession. Licensing regulations are generally

established to ensure that an organisation or individual meets minimum

standards to protect health and safety. The output of licensing is the

awarding of a document or licence allowing an organisation or person to

provide a service within a speciﬁed scope.

Medical tourism

Travel of people to another country for the purpose of obtaining medical

treatment in that country.

Organisational

peer assessment

A process whereby the performance of an organisation is evaluated by

members of similar organisations. Also peer review.

Outcome

standards

Standards which address the results, consequences or outcomes of the

performance and measurement of activities, systems and functions.

Patient

centredness

Focus on the experience of the patient / client from their perspective,

minimising vulnerability and maximising control and respect. Also patient

/ client focus.

Patient / Client

journey

The patient / client path through the care or treatment process – entry,

assessment, planning, delivery of care or treatment, evaluation, follow-up

and across services and providers. Also client continuum of care.

Process

standards

Standards which address the interrelated processes of different

organisational and clinical functions and activities.

Quality

improvement

Ongoing response to quality assessment data about a service, in ways

that improve the processes by which services are provided to clients. Also

continuous quality improvement (CQI).

Regulation

Is a form of external evaluation by which a body, who is authorised

by law, assesses an organisation or a person against pre-determined

requirements. The pre-determined requirements are derived from

legislation and therefore, the regulator may take a number of actions in

the event of non-compliance.

Risk mitigation

A systematic reduction in the extent of exposure to a risk and / or the

likelihood and consequences of its occurrence.

Self-assessment

A process by which an organisation evaluates its own performance against

set criteria or standards, identiﬁes strengths and gaps, and plans actions

for improvement.

Standardisation

Process of developing and implementing technical, service or other

standards; that can help to maximize compatibility, interoperability, safety,

repeatability or quality.

Structure

standards

Standards which address the relatively stable characteristics of healthcare

providers, their staff, tools and resources, and physical and organisational

settings.

System

A set of interacting or interdependent processes forming an integrated,

whole function or activity.

Transparency

Operating in such a way that it is easy for others to see what actions

are performed; a principle that allows those affected by administrative

decisions, business transactions or charitable work to know not only the

basic facts and ﬁgures but also the mechanisms and processes. Usually

requires documented policies and procedures.

Universal health

coverage

The goal of all people having access to and obtaining health promotion,

preventive, curative, rehabilitative and palliative health services they need,

of sufﬁcient quality to be effective, without suffering ﬁnancial hardship to

avail of them.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Introduction

The purpose of this document is to guide countries, agencies and other groups in the process of

setting up new health or social care external evaluation organisations or programmes. It is also

intended as an aid for funding and development agencies such as the World Bank, international

aid and technical cooperation agencies, World Health Organization, Ministries of Health, other

government agencies, groups and organisations who want to improve the quality and safety

of healthcare in their country, region or specialty area. It revises the International Society for

Quality in Health Care (ISQua) Toolkit for Accreditation Programs, 2004

, and ISQua Checklist for

Development of New Healthcare Accreditation Programs, 2006

. This document has extended

its scope beyond healthcare accreditation programmes to include other external evaluation

programmes such as certiﬁcation and licensing as they apply to organisations, not individual

practitioners. These programmes have different scopes and organisational coverage but are

based on the same principle of evaluating and improving performance against a deﬁned set

of standards or criteria, using external evaluators, to improve the safety and quality of health

services for the public.

Accreditation can be deﬁned as a self-assessment and external peer review process used by

health and social care organisations to accurately assess their level of performance in relation

to established standards and to implement ways to continuously improve the health or social

care system. Certiﬁcation is a process by which an authorised body, either a governmental or

non-governmental organisation, evaluates and recognises an organisation as meeting pre-

determined requirements or criteria. Licensing is a process by which a governmental authority

grants permission for a healthcare organisation to operate. Licensing regulations are generally

established to ensure that an organisation or individual meets minimum standards to protect

health and safety. For the purpose of this document we will refer to an accreditation body but

this includes any external evaluation programme as the principles remain the same.

The document refers mainly to healthcare organisations but is also applicable to social care

organisations. In it, the term external evaluation is used to cover accreditation, certiﬁcation,

licensing and other standards based assessment programmes. The term survey is used to refer

to survey, assessment and audit. The term surveyor is used to include surveyors, assessors and

auditors.

Research and experience have identiﬁed the beneﬁts of external evaluation programmes such

as improved organisational efﬁciency and effectiveness, improved safety and quality, better risk

mitigation, improved leadership, reduced liability costs, better communication and teamwork,

increased satisfaction of users and staff, and better patient care. However, there are challenges

in setting up these programmes. The principal threats to new external evaluation programmes

appear to be inconsistency of government policy, unstable politics, unrealistic expectations

and lack of professional / stakeholder support, continuing ﬁnance and / or incentives. The

effectiveness and sustainability of an external evaluation organisation or programme depends

ultimately on many variable factors in the particular healthcare environment of the country

or organisation involved. It also depends on the kind of programme concerned, and how it is

implemented.

To be sustainable, external evaluation programmes need ongoing government and / or private

support, a sufﬁciently large healthcare market size, stable programme funding, diverse

incentives to encourage participation, and continual reﬁnement and improvement in the external

evaluation organisation’s operations and service delivery.

This guide addresses the variables of policy, organisation, methods and resources. It outlines

the reasons why an external evaluation programme might be developed, describes the different

models, and highlights the beneﬁts and challenges associated with external evaluation.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

It then provides guidance on the steps that need to be taken in establishing a new external

evaluation organisation including:

Establishing the fundamentals of scope and purpose, and deﬁning the important roles of

government and incentives in the external evaluation organisation / programme.

Setting up of the external evaluation organisational structure including: establishing an

advisory committee; developing relationships with stakeholders; designing a governance

framework; embedding the values of fairness and transparency; and getting outside

assistance and funding.

Establishing governance and management systems including: stafﬁng; ﬁnancial and

information systems; and risk management and performance improvement systems. It also

highlights the importance of allowing enough time for these stages.

Developing the standards to be used by the organisation and the system for measuring their

achievement.

Developing the surveyor and survey management systems including: the selection and

training of surveyors; the designing of processes and technology for managing surveys

and other events; developing and establishing education services; and determining and

establishing the process for awarding accreditation or certiﬁcation status.

Integrating into all these systems and processes ways of measuring and evaluating

performance.

This document reﬂects the best practice guidelines and standards developed by the

International Society for Quality in Health Care (ISQua) as part of its International Accreditation

Programme (IAP): ISQua Guidelines and Standards for External Evaluation Organisations, 4th

Edition Version 1.1, 2014

; ISQua Guidelines and Principles for the Development of Health and

Social Care Standards, 4th Edition Version 1.1, 2014

; and ISQua Surveyor Training Standards

Programme, 2nd Edition 2009

The appendices include case studies outlining how three different healthcare external

evaluation organisations were established. Two of the organisations featured are accreditation

organisations. The third featured organisation is an assessment organisation established

primarily to assess against government-mandated standards for compulsory certiﬁcation.

Appendix 1d describes an Australian Practice Incentive Programme that demonstrates how

accreditation can be used as a lever to encourage quality improvement.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Chapter 1: Why develop

an external evaluation

programme?

This chapter introduces what a healthcare external evaluation programme is; describes some

of the different models of external evaluation; outlines the beneﬁts of such programmes; and

highlights the challenges which may be encountered in establishing such programmes.

1.1 The growing demand for external evaluation in health and

social care

There is growing worldwide demand, concern and focus on quality and safety in

healthcare. UUniversal Health Coverage (UHC) is now a key agenda item for the World

Bank and the World Health Organization and many countries have adopted or are about

to adopt this system of equal healthcare for all. The goal of universal health coverage is

to ensure that all people obtain the health services they need without suffering ﬁnancial

hardship when paying for them. This requires:

A strong, efﬁcient, well-run health system with good governance

A system for ﬁnancing health services in an efﬁcient and equitable way

Access to essential medicines and technologies and good health information

systems

A sufﬁcient capacity of well-trained, motivated health workers

There is increasing support from governments, and from funding agencies, for

mechanisms, such as accreditation, to support UHC. Governments will ultimately be

responsible for providing UHC and they will be required to demonstrate efﬁcient use

of limited public funds while providing safe quality healthcare. External evaluation

provides assurances that healthcare facilities have quality systems in place and

the data to demonstrate the required level of service provision. Depending on the

comprehensiveness of the standards against which health service performance is being

measured, external evaluation programmes such as accreditation and certiﬁcation can

contribute to quality improvement, risk mitigation, patient safety, improved efﬁciency and

accountability, and can contribute to the sustainability of the healthcare system. They

can provide information on how well health services are being delivered, identify issues,

and assist the decision-making of funders, regulators, healthcare professionals and the

public. External evaluation supports transparency, benchmarking and accountability, so

that government funding is allocated in a fair and equitable way and supports a culture of

change and quality and an increased focus on risk.

Patients expect to receive safe care and are demanding quality services that meet their

needs. They expect to be treated with respect, to receive services of an appropriate and

consistent standard that are delivered with care and skill, that minimise risk and harm,

comply with legal, professional and ethical standards, and that facilitate continuity of

care. Patients need to receive information about their condition and treatment in a way

they can understand, to be able to make informed choices about their treatment and to

be communicated with openly and honestly. They want the right to complain if services do

not meet their needs and expect action to be taken to address the problem.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

They have a right to trust that their health provider or hospital has systems and

processes in place to provide such patient-centred, reliable, efﬁcient, effective and

responsive care. An external evaluation programme based on best practice standards

will make a signiﬁcant contribution to achieving this.

With preventable error rates estimated to be 83% in developing and transitional countries

and a 30% rate of adverse events associated with deaths, these countries require not only

more resources to improve the safety and quality of care, but a political environment,

policies and mechanisms that support quality initiatives. The contribution of external

evaluation organisations centred on promoting improvements, applying standards and

providing feedback is being increasingly recognised in these countries. Preventable

error rates of over 10% in developed countries are also unacceptable. A ﬂourishing

accreditation programme is one element of the institutional basis for high quality

healthcare

1.2 Models of external evaluation

External evaluation

Is a process by which an objective independent assessor gathers reliable and valid

information in a systematic manner by making comparisons to standards, guidelines

or pathways for the purpose of enabling more informed decisions and for assessing

if pre-determined and published requirements such as goals, objectives or standards

have been met. An organisation, object, process or individual may be assessed and

evaluation may be undertaken by peers, including organisations and professionals,

private professional auditors or consultants, purchasers / funders / insurers, consumers

/ patients or governments.

The distinguishing features of external evaluation are as follows:

It is a formal process

The object being assessed is an organisation, object, process or individual person

Assessment is undertaken by an objective, independent assessor

Assessment is against pre-determined and published requirements / criteria

It is designed so that decisions are not inﬂuenced by those being assessed

The assessment results in a deﬁned output

There are a number of models of external evaluation and it should be acknowledged

that there can be confusion regarding terminology due to the diverse applications of

the external evaluation models. Examples of external evaluation models include the

following:

Accreditation

Accreditation may be deﬁned as a self-assessment and external peer review process

used by health and social care organisations to accurately assess their level of

performance in relation to established standards and to implement ways to continuously

improve the health or social care system. Although primarily applied in relation to

organisations, processes may also be accredited. Accreditation standards assess

the organisation’s or process’s ability to fulﬁl its core mission and may address more

than legal requirements. They are usually recognised as optimal, evidence-based and

achievable and are designed to encourage continuous improvement

. The output of

accreditation is a report summarising the ﬁndings of the assessment and a recognition

decision regarding the accreditation status.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Accreditation is one of the longest established models of external evaluation. It is a

self-assessment and external peer review process that assesses the entire organisation

including both clinical and management processes and activities. Traditionally, health

and social care organisations engaged in accreditation on a voluntary basis and

accreditation schemes were provided by non-governmental agencies. However, there

has been a shift over time towards greater governmental involvement in accreditation

with the development of national government funded accreditation programmes and a

shift from voluntary to mandatory participation in such schemes. For example, in 2011

the Australian Health Ministers endorsed the National Safety and Quality Health Service

(NSQHS) Standards and a national accreditation scheme. As a result, all hospitals and

day procedure services and the majority of public dental services across Australia now

need to be accredited to the NSQHS Standards. Private health service organisations are

required to conﬁrm their requirements for accreditation to any standards in addition to

the NSQHS Standards with the relevant

health department. Prior to 2011, participation in accreditation was voluntary for

Australian hospitals

Certiﬁcation

Certiﬁcation is a process by which an authorised body, either a governmental or non-

governmental organisation, evaluates and recognises either an individual, organisation,

object or process as meeting pre-determined requirements or criteria. The pre-

determined requirements are set out in standards which are developed speciﬁcally

for the purpose of assessment. The standards assess the performance of the

organisation, object, process or person, may focus on speciﬁc aspects of performance

and may address more than legal requirements. The output of certiﬁcation is a report

summarising the ﬁndings of the assessment and a recognition decision regarding the

certiﬁcation status.

Certiﬁcation may be used by governments or other authorised agencies to assess the

compliance of healthcare facilities or speciﬁc departments / services within those

facilities with a set of standards. The focus is usually on essential elements being in

place rather than on continuous quality improvement. The standards and certiﬁcation

may not be organisation-wide, but may apply to a particular service, e.g. physiotherapy.

Governments may authorise independent assessment organisations to assess health and

social care providers’ compliance with government-mandated standards.

An example of a certiﬁcation scheme is ISO: the International Organization for

Standardization. ISO provides standards, e.g. ISO 9000 Quality Management, against

which organisations or functions may be certiﬁed by ISO accredited certiﬁcation bodies or

organisations

. Although originally designed for the manufacturing industry, e.g. medical

devices, these have been primarily applied to radiology and laboratory systems in

healthcare, and more generally to quality systems in hospitals and clinical departments.

Conformance with ISO standards is assessed by professional quality auditors and any

non-conformance is followed up with a subsequent audit.

When applied to individuals, certiﬁcation usually implies that the individual has received

additional education and training, and demonstrated competence in a specialty area

beyond the minimum requirements set for registration or licensing. For example, a

doctor may be certiﬁed by a professional specialty board in the practice of obstetrics

There can be confusion between the terms accreditation and certiﬁcation and they are

often used interchangeably. However, accreditation usually applies only to organisations,

while certiﬁcation may apply to individuals, as well as organisations.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Regulation

Regulation is a form of external evaluation by which a body, authorised by law, assesses

an organisation or a person against pre-determined requirements. The pre-determined

requirements are derived from legislation and therefore, the regulator may take a

number of actions in the event of non-compliance.

Licensing

Licensing is a process by which a governmental authority grants permission to an

individual practitioner or health or social care organisation to operate or engage in an

occupation or profession. Licensing regulations are generally established to ensure that

an organisation or individual meets minimum standards to protect public health and

safety.

The output of licensing is the awarding of a document or licence allowing an organisation

or person to provide a service within a speciﬁed scope.

Organisational licensing or registration is granted following an on-site inspection

to determine if minimum health and safety standards have been met. Maintenance

of registration or licensure is an ongoing requirement for the health or social care

organisation to continue to operate and care for patients or clients.

Individual or professional licensing or registration is usually granted after some form of

examination or proof of education and may be renewed periodically through payment of a

fee and / or proof of continuing education or professional competence

Countries may have more than one model of external evaluation in operation in speciﬁc

sectors. For example, hospitals may be required to be licensed and meet speciﬁc

government-mandated standards in order to be able to provide health services in a

particular country, but may still engage voluntarily in organisational accreditation

or certiﬁcation programmes for speciﬁc departments in the facility e.g. laboratory

certiﬁcation programmes. Individual healthcare practitioners may need to be registered

with their professional body in order to be employed in a hospital but they may also

voluntarily undergo additional education in order to be certiﬁed in a respective ﬁeld by a

professional specialty board.

The key characteristics of accreditation, licensing and certiﬁcation are set out in Table 1.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Table 1: Deﬁnitions of accreditation, certiﬁcation and licensing

Process Participation Issuing

organisation

Object of

evaluation

Components /

Requirements

Standards

Accreditation Voluntary or

mandatory

Non-

governmental

organisation

(NGO) or

government

authority

Organisation Compliance

with published

standards, on-

site evaluation;

compliance may

not be required

by law and / or

requlations

Set at a

maximum level

to stimulate

improvement

over time

Certiﬁcation Voluntary or

mandatory

Authorised

body, either

government or

NGO

Individual Evaluation of

pre-determined

requirements,

additional

education

/ training,

demonstrated

competence in

speciality area

Set by national

professional or

speciality boards

Organisation

or component

Demonstration

that the

organisation

has additional

services,

technology or

capacity

Industry

standards

(e.g. ISO 9000

standards)

evaluate

conformance

to design

speciﬁcations

Licensing Mandatory Governmental

authority

Individual Regulations to

ensure minimum

standards,

exam, or proof

of education /

competence

Set at a

minimum level

to ensure an

environment

with minimum

risk to health

and safety

Organisation Regulations to

ensure minimum

standards, on-site

inspection

1.3 Beneﬁts of external evaluation

External evaluation has contributed to improving the quality and safety of healthcare

for nearly 100 years and the majority of the published literature relates to accreditation.

Research on the beneﬁts of certiﬁcation, regulation and licensing is sparse. It must

be acknowledged that historically there has been limited evidence of the impact of

accreditation but in recent years more empirical research has been undertaken to

identify and quantify the beneﬁts.

Some of the speciﬁc beneﬁts of accreditation identiﬁed in the literature include impacts

on structural elements of quality improvement in healthcare organisations such as

leadership, governance and management, and process elements such as organisational

performance

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

From a leadership, governance and management perspective, accreditation is

perceived as: providing a framework for helping to create and implement systems and

processes that improve operational effectiveness and advance positive health outcomes;

providing organisations with a well-deﬁned vision for sustainable quality improvement

initiatives; and as a means of demonstrating credibility and a commitment to quality and

accountability.

From an organisational performance perspective, some of the identiﬁed beneﬁts include:

Increases healthcare organisations’ compliance with quality and safety standards

Stimulates sustainable quality improvement efforts and continuously raises the bar

with regard to quality improvement initiatives, policies and processes

Decreases variances in practice among healthcare providers and decision-makers

Highlights practices that are working well. Promotes the sharing of policies,

procedures and best practices among healthcare organisations

Accreditation has also been perceived as having an impact on team working by

strengthening interdisciplinary team effectiveness and promoting capacity building,

professional development and organisational learning

Similarly, a recent synthesis of 122 empirical studies that examined either the processes

or impacts of accreditation programmes concluded that research evidence generally

presents health service accreditation as a useful tool to stimulate improvement in

health service organisations and to promote high quality organisation processes.

Some of the cited studies found that accreditation promotes standardisation of care

processes; increased compliance with external programmes or guidelines; development

of organisational cultures conducive to quality and safety; implementation of continuous

quality improvement (CQI) activities; and superior leadership. There was limited evidence

showing positive associations between accreditation and patient outcome measures.

However, this was attributed to poor research design

A comparison of accreditation in low- and middle-income countries versus higher-

income countries showed all programmes promote improvements, apply standards

and provide feedback. Accreditation programmes are contributing to incremental

improvements in quality systems and clinical processes in health systems around the

world and are one element of the institutional basis for high-quality healthcare

A recent review examining the use of economic evaluation techniques in health services

accreditation research identiﬁed that no formal economic evaluation of health services

accreditation has been carried out to date. It also highlighted that the impact or

effectiveness of accreditation has been researched with a variety of foci and to differing

degrees. The research design of some studies, particularly those that are observational

or qualitative in nature, makes it difﬁcult to provide statistically robust evidence for

the efﬁcacy of accreditation or causality. The lack of a clear relationship between

accreditation and the outcomes measured in beneﬁt studies makes it difﬁcult to design

and conduct economic appraisal studies where a more robust understanding of the costs

and beneﬁts involved is required. In turn, the absence of formal economic appraisal

means it is challenging to appraise accreditation in comparison to other methods to

improve patient safety and quality of care

While the evidence for the direct impact of accreditation on patient / client outcomes

is inconclusive, the available research suggests that accreditation may contribute to

improving health outcomes by strengthening interdisciplinary team effectiveness and

communication and by enhancing the use of indicators for evidence-based decision

making

. The challenge for mature external evaluation systems is to become more

outcome driven. This reduces the burden of audit but also helps to highlight its beneﬁts.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

1.4 Challenges for external evaluation programmes

The principal threats to new external evaluation programmes include: inconsistency of

government policy; unstable politics; unrealistic expectations; and lack of professional /

stakeholder support, continuing ﬁnance and / or incentives. To be sustainable, external

evaluation programmes need a number of elements to be in place, including some of

the following: ongoing government and / or private support; a sufﬁciently large health

or social care market size; stable programme funding; diverse incentives to encourage

participation; and continual reﬁnement and improvement in the external evaluation

organisation’s operations and service delivery

(refer Chapter 2).

To be sustainable and credible, new programmes need sufﬁcient numbers of trained

and skilled personnel and a realistic timeframe for the development of the programme.

They need to demonstrate objectivity and independence with transparent procedures for

the assessment of healthcare services and for decisions on accreditation or certiﬁcation

awards. The expectations of governments and stakeholders about what the external

evaluation programme can achieve need to be realistic, in line with the purpose and

scope for which it has been designed and resourced, and in line with the government’s

broader strategy or policy for healthcare quality and safety. Within that strategy or policy

there needs to be a balance between the objectives of external control or regulation

and internal organisational development or improvement. Attempts to prescribe and

control every process of a complex system like a healthcare organisation or service,

which cannot be understood as simply a sum of a number of discrete and predictable

processes, will evoke resistance from staff, and can be counterproductive in terms of

quality and safety. Health and social care staff need to be motivated and committed to

improving quality rather than directed and sanctioned.

Expectations of accredited or certiﬁed health or social care services can be

unrealistically high. The external assessment of organisations for the purposes of

accreditation or certiﬁcation is based on an on-site survey or assessment of compliance

with, or achievement of, standards. This is a snapshot in time and does not guarantee,

nor is it meant to guarantee, ongoing performance at the same level. However, external

evaluation organisations who themselves engage in an external evaluation process, such

as ISQua’s International Accreditation Programme (IAP) are expected, as part of this

process to monitor the continued maintenance of standards and quality improvements by

the organisations they have accredited or certiﬁed, e.g. submission of action plans and

reports of their implementation, periodic self-assessment or external reviews, random

reviews, follow-up of signiﬁcant complaints or sentinel events.

Given the amount of effort and money invested worldwide in external evaluation and

regulation of healthcare delivery, and the common pursuit of valid standards and

reliable measurement, there are economic and technical reasons to share research and

experience more actively in the international community.

A study comparing European hospitals in terms of quality and safety was found to be

challenging because of the different hospital accreditation and licensing systems in each

country; the different indicators collected; different deﬁnitions of the same indicators;

different mandatory versus voluntary data collection requirements; different types of

organisations overseeing data collection; different levels of aggregation of data (country,

region, hospital); and different levels of public access to such data.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

This means that patients are unable to make informed choices about where they receive

their healthcare in different countries and some governments will remain in the dark

about the quality and safety of care available to their citizens as compared to that

available in neighbouring countries

Ongoing research is needed into the beneﬁts and limitations of external evaluation

in healthcare. To measure the impact of any new programme, before and after

measurements are needed of the indicators that the programme is intended to address.

This chapter has introduced different external evaluation models and has outlined the

beneﬁts of external evaluation and the challenges associated with establishing a new

programme. The following chapters will present the factors that need to be considered

when deciding which external evaluation model to adopt in a country and the steps to be

undertaken when setting up an external evaluation organisation and programme.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Chapter 2: Establishing

the Fundamentals

This chapter outlines the initial decisions that need to be made when a new external

evaluation programme is being established: the purpose of the programme; its scope; the

role of government; and the incentives that may be needed to ensure health and social

care organisations participate. It also highlights the importance of identifying who the main

stakeholders may be and what external inﬂuences for the programme will look like.

2.1 Deﬁning the purpose of the external evaluation programme

One of the ﬁrst steps in the development of a new external evaluation programme is to

determine its purpose.

Factors to consider in determining the purpose of an external evaluation programme or

organisation include the following:

Developmental or regulatory

According to the World Bank

, governments regulate health services in order to guide

private activity and achieve national health objectives. Regulation can be used for control,

with instruments that use the force of law to ensure that services provided adhere to

legal requirements. Instruments that aim to control include: licensing, restrictions on

dangerous clinical practice and registration. Examples include: basic legislation on

health personnel such as registration and licensing requirements, which can also be

used to set minimum requirements for health services or facilities to operate. Regulation

can also use ﬁnancial or non-ﬁnancial incentives that change the behaviour of private

healthcare providers. The advantages of using incentive-based regulation is that it

avoids the informational, administrative and political constraints that control-based

interventions entail. Accreditation, certiﬁcation and contracts are examples of incentive-

based regulation. However, in developing countries, regulation is often ineffective

because of the low level of enforcement and insufﬁcient resources.

Standards-based external evaluation

Standards-based accreditation is a programme that contributes to developing an

organisation, and is designed to improve the quality as well as the safety of health

services.

Accreditation programmes monitor and promote, via self and external assessment,

healthcare organisation performance against pre-determined optimal standards

. They

also aim to contribute to the provision of high quality and safe healthcare services and to

improve patient health outcomes.

Certiﬁcation may be similarly standards-based and use a rating system that encourages

improvement over time but its focus is usually more on continuing compliance with

criteria and the standards may be more limited. Licensing may be used when the priority

is ensuring basic health and safety requirements are met in order for a healthcare

organisation to operate and will usually be facility focused.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Values and objectives underpinning a new programme

A survey of healthcare accreditation organisations revealed that quality improvement

was the reason healthcare organisations participated in accreditation. On the other hand,

the government agenda commonly focused more on the protection of public money and

public health as a priority, meaning reducing variation in practice to increase efﬁciency

and improve patient safety, consistent with WHO global initiatives

Values or principles may relate to features such as leadership, a system and process

approach, multidisciplinary teamwork, capacity building and training, patient

centredness, devolved decision-making and accountability, evidence-based decisions for

continuous improvement and performance-based incentives.

Objectives of external evaluation programmes identiﬁed in some developing countries

have included: improving leadership of a quality health system; improving resources and

capacity of the system and staff; improving performance by clearly deﬁning the roles

and responsibilities of staff at all levels; developing the structures, systems and capacity

to support quality improvement; strengthening the focus and role of health service

consumers and other stakeholders; and improving health services through systematic

implementation of standards.

The following table compares capacity building and regulatory external evaluation

approaches

Table 2: Comparison of capacity building and regulatory external evaluation

Capacity building Regulatory

Purpose Dynamic, organisational

improvement

Static, control

Terminology Accreditation, certiﬁcation Licensing, regulation

Governance Non-governmental

organisation, stakeholders

National / regional

government agency

Primary Customers Healthcare providers Government

Secondary customers Patients, professions,

healthcare insurers

Population, politicians,

public ﬁnance

Incentives for healthcare

organisations to

participate

Ethical, commercial Legal, mandatory

Uptake Voluntary self-selection to

available programs

All institutions in all

sectors

Standards Deﬁned by non-governmental

organisation, optimal,

achievable, encourage quality

improvement

Deﬁned by regulation,

minimal acceptable

Funding Self-ﬁnancing State

Cross-border mobility Limited by language, culture Limited by political

borders

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Possible purposes or objectives of an external evaluation programme might be to:

Improve the performance of health services by setting and measuring the

achievement of standards

Increase public safety and reduce risks associated with injury and infections for

patients / clients and staff

Increase public conﬁdence in the quality of healthcare services

Promote accountability of health services to funders and the public.

How do these values and objectives relate to plans for health reform in

general, and to the national quality strategy in particular?

The next important step is to identify if there are plans for health and / or social care

reform in the country or region and if there are any national or regional quality strategies

or plans in place. Reform plans outline the changes that a government intends to

make to a particular sector and outlines the speciﬁc actions that it will take to achieve

those reforms. For example, a government may outline in a reform plan that it intends

to establish an external evaluation organisation and what the role or purpose of this

organisation will be. A quality strategy provides an agreed direction and identiﬁes the

most important activities for improving quality in the health and social care sector in the

country or region. It helps to identify the strengths of the system and also the constraints

that prevent the provision of a quality service. A quality strategy may outline the role

or will help to identify or clarify the role that external evaluation is expected to play in

achieving the country or region’s quality vision.

These factors will guide all further decisions - the role of the government, relationships

with stakeholders, the governance and management framework, the standards or

criteria to be used for assessment, the assessment process, and the outcome of

licensing, certiﬁcation or accreditation.

The case study examples below provide further insight into the factors that inﬂuenced

the establishment of external evaluation agencies in different jurisdictions.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Case Studies – Foundation of the programme

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

The Danish accreditation programme (DDKM) was established as part of the

“National Strategy for Quality Development in the Healthcare System – Joint Goals

and Action Plan 2002-2006”. The strategy was developed by the national, regional and

local political authorities in cooperation with stakeholder organisations, representing

professionals and consumers.

At that time, a number of hospitals already had positive experiences with

accreditation provided by international accreditors – one of the intentions of the

strategy was to spread this to the entire healthcare system, based on a Danish

model.

Health Care Accreditation Council (HCAC)

Country: Jordan

The HCAC is the national healthcare accreditation agency of Jordan. Several reasons

were stated for why the programme was developed including to improve the quality

of hospitals and to enhance medical tourism. In addition, it was a response to public

complaints of poor quality of care and a need to improve the entire healthcare system

in the country.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

The commencement of the Health and Disability Services (Safety) Act on 1 July 2002

represented a signiﬁcant change in the regulatory environment in the New Zealand

health and disability sector. This Act replaced several previous pieces of legislation

and changed the way in which residential and hospital services were licensed

or registered. In addition, the Act introduced health and disability standards for

hospitals, rest homes and residential disability services aimed at improving safety

levels and quality of care that became mandatory from 1 October 2004. The Act

required that designated audit agencies (DAAs) are approved by the Director General

of Health for the purpose of auditing these services to those standards.

2.2 Deﬁning the scope of the external evaluation programme

Once the purpose is established it is important to deﬁne the initial scope of the

programme. The purpose of a new external evaluation programme may depend on

the government’s priorities, the national health reform or quality strategies, available

funding, the commitment of stakeholders and the problems or issues that need to be

addressed.

Factors to consider in deﬁning the scope of the external evaluation programme include

the following:

Primary or hospital care?

Traditionally, accreditation has been developed for hospitals or aged care facilities and

then moved outwards towards home support, hospice and other community services and

then to regional networks or networks of preventive and curative services.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

However, in developing countries the most urgent need may be for improved primary

and community care and the programme will initially be developed to cover primary

care clinics and outreach services, although there may be some resource advantages in

developing primary care and hospital programmes at the same time.

Often it is easier to develop facilities based programmes ﬁrst, starting with core

standards and external evaluation for single institutions, e.g. acute hospitals, polyclinics

or health centres. Standards can then be developed for more specialised services, e.g.

rest homes or hospice care or mental health, followed by the linkages between them,

preventive health or health networks, and they can then be covered by the programme.

Assessment of single units, services or departments could offer large organisations a

gradual entry to a full programme but it does not carry the beneﬁts of integration and

organisation consistency. It may hide the opportunities for improvement which frequently

lie in communication between services rather than within them. However, there are

many service speciﬁc external evaluation programmes which are operated either by a

larger generic programme, or by a provider or association which works only in that area,

e.g. palliative care, laboratory medicine, speech therapy, autism, general practice, aged

care, and community services.

Some programmes have started with tertiary hospitals and services, with the intention

of expanding to secondary care services later. Some programmes in North America (e.g.

Accreditation Canada) accredit entire health networks and regions and are applying

accreditation across the continuum of care. Some governmental programmes in Europe

address public health priorities (such as cardiac health, cancer services) by assessing

local performance of preventive to tertiary services against national service frameworks.

In such programmes, measures may include the application of evidence-based

medicine (process) and the measurement of population health gain (outcome) but many

health determinants, e.g. housing, education and poverty, remain outside the scope of

healthcare external evaluation programmes.

However, current best practice is to provide a programme that focuses on the patient or

client and their journey through the service, hospital, network or care programme and

the continuity of service or care for that individual or family across the entire continuum

of care.

Historically, external evaluation programmes have set their scope in a way which

compartmentalises care and service rather than optimising quality outcomes for the

patient or client.

Public or Private coverage?

Most external evaluation programmes offer services to both public and private sector

services, although some are restricted to either the public or private sector. Evaluating

across sectors has advantages to healthcare organisations in facilitating the focus on the

patient or client journey, providing a level playing ﬁeld for comparing and benchmarking

potential competitors, to surveyors in learning from another sector and to self-ﬁnancing

programmes in having a larger potential market. Sometimes either the private or

public sector has the size, resources and incentives such as funding incentives, medical

insurance and competitive advantage to adopt an external evaluation programme earlier.

Medical tourism is another large incentive. To attract patients who are crossing national

borders in search of affordable and timely healthcare, private and public health services

need accreditation or certiﬁcation to demonstrate their competence and safety.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Many medical tourism companies are now involved in organising cross-border health

services and it has been recommended that the care they arrange should only be at

accredited international health facilities. Other recommendations include the medical

tourism companies themselves having to undergo an accreditation review; standards to

ensure patients make informed choices; and continuity of care as an integral feature of

cross-border care

The case studies provide some further insights into how the scope of external evaluation

agencies in different jurisdictions was determined.

Case Studies – Scope of the programme

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

Public and private hospitals, pharmacies, municipalities (primary care services,

including long-term care), ambulance providers and General Practitioners (GPs) all

participate in DDKM.

Health Care Accreditation Council (HCAC)

Country: Jordan

The HCAC is the national healthcare accreditation agency of Jordan. The organisation

sets standards for hospitals, primary healthcare centres, family planning and

reproductive health, transport services (ambulances), cardiac care, and diabetes

mellitus. HCAC surveys against the standards and awards accreditation. HCAC also

provides consultation and education to prepare healthcare facilities for accreditation

and offers certiﬁcation courses.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

The commencement of the Health and Disability Services (Safety) Act on 1 July 2002

represented a signiﬁcant change in the regulatory environment in the New Zealand

health and disability sector. This Act replaced several previous pieces of legislation

and changed the way in which residential and hospital services were licensed or

registered. HDANZ’s scope was determined by the Safety Act – the assessment of

standards is a legal requirement for public and private hospitals, rest homes and

residential disability services. Standards New Zealand (SNZ) is responsible for the

New Zealand standards and this includes others such as for home support, allied

health, and day surgery procedures.

Critical mass: economy, consistency, equity, objectivity

Larger countries can achieve economies of scale; smaller countries (perhaps with a

population of less than 5 million), or large ones which choose to devolve the process

to regional government, e.g. Italy, or ethnic groups, e.g. Aboriginal, have to share the

considerable costs of infrastructure and development among a smaller number of

healthcare organisations (giving higher unit costs). If the surveyor workforce is voluntary,

this may also mean having a smaller choice of surveyors (giving more potential for

conﬂict of interest). However, there are options such as contracting or employing a

smaller paid surveyor workforce or contracting surveyors from other countries for

surveys.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Other options for enhancing the opportunities for smaller programmes include:

Sharing a programme with a neighbouring region or state which has similar culture

and language

Designing one national programme, rather than several regional ones

Providing national standards, guidelines or tools for regional agencies or

designated assessment organisations

Using a single organisation to provide multiple accreditation programmes

Using the same organisation or agency as a centre for research and development

of other quality methods, e.g. performance indicators, clinical guidelines, patient

surveys, technological assessment

Obtaining accreditation services from another region or state.

2.3 Establishing the role of government

The development of an external evaluation programme may be part of broader health

reforms, or part of an overall governmental strategy for quality improvement and a

transition from a centralised system to one which is more open and independent. It may

be necessary for the health ministry to re-deﬁne its own duties and responsibilities in the

context of a reformed organisational structure of the health system.

The relationships between departments of government which have a major impact on

quality may be unclear. The roles of agencies responsible for such areas as public health,

blood products, pharmaceuticals or medical devices and inspectorates responsible for

such aspects as control of the environment, safety, radiation at national or local level

need to be clariﬁed as part of the overall quality plan. Dissemination of this structure and

plan would also provide an opportunity to develop a strategy for active communication of

the aims and operation of an integrated quality system.

Government controlled or not?

Speciﬁc to external evaluation is the question of whether the programme should

be organised and administered directly and solely within the ministry of health, like

licensing, or by an independent body totally unconnected to government, or by something

between these two extremes – which has become more common. The legitimate and

necessary role of government is the licensing of healthcare facilities, using basic safety

standards or criteria. Licensing of individual medical practitioners may be a government

function but is usually carried out by a medical council. However, there are challenges

for governmental external evaluation programmes which include:

Inconsistent policy and management with changes in government

Reviewing and updating standards consistently and in a timely way

Public perception of government that is too low to make them credible assessors of

healthcare

Conﬂict of interest between government roles as purchaser, regulator and insurer,

and lack of independence and continuity

Delegation of powers to local areas, which may result in multiple government

programmes duplicating development and ongoing costs of running the

programmes.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Some countries, such as France and Saudi Arabia, have made participation in

accreditation by healthcare organisations legally compulsory, but most countries

merely authorise the functions of the external evaluation organisation. Two-thirds of

accreditation organisations surveyed in 2010 were supported by enabling legislation.

However, many independent programmes thrive without it. Five accreditation

organisations were struggling or inactive, despite being supported by a published

government strategy. If enabling legislation is not essential and national strategies often

change with ministers and governments, external evaluation organisations must choose

reliable partners for survival

Need for government support

To be successful, external evaluation programmes often need government support and

collaboration and to be recognised as an important part of the national health quality

strategy. The support may be through funding, providing incentives for participants

such as limiting other forms of inspection or audit, or recognising the programmes as a

legitimate and essential part of the overall health quality strategy.

Some functions, such as the deﬁnition of standards, the assessment of compliance

and the grading of awards may be totally independent or may be shared between

government and independent external evaluation organisations. Some governments,

for example, New Zealand, have developed or approved standards that they require

healthcare organisations to meet. However, the government have devolved the process

of assessment of compliance with the standards and follow-up to ensure the standards

are being maintained to independent designated auditing, accreditation or certiﬁcation

organisations. These organisations in turn need to be internationally recognised by a

3rd party accreditor such as ISQua. In Australia, a similar system operates through

the Australian Commission on Safety and Quality in Health Care which has developed

national quality and safety standards. The accreditation of healthcare organisations who

meet the national quality and safety standards has been devolved.

The mandatory requirement for external evaluation, as in the above examples, is an

increasing trend as governments seek to improve the quality and safety of health

services.

Key roles which governments might play in supporting external evaluation include:

Enabling the external evaluation process, e.g. through policy decisions such as by

reciprocal recognition of assessments; joint development of standards; avoiding

conﬂict such as perverse incentives and competing mechanisms for assessment

Providing leverage, e.g. by according preference to accredited or certiﬁed facilities,

services or networks such as reimbursement tariffs and payment procedures

Using accreditation or certiﬁcation as a criterion in its own purchasing decisions,

e.g. in deﬁning preferred providers and contract monitoring

Regulating individuals and institutions, e.g. by ensuring consistency and distinction

between licensing and accreditation

Acknowledging or endorsing accreditation or certiﬁcation programmes against

deﬁned criteria to maintain standards, avoid duplication and potential exploitation

Providing ﬁnancial support in establishing programmes and / or contributing to the

funding of programmes’ continuing development.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

The extent of government support and involvement in the external evaluation programme

may also depend on the country’s overall stage of development. In developing countries,

where there may be a more limited health industry or where professional organisations

may not have the resources or ﬁnancial capacity to initiate an external evaluation

programme, government organisations may be needed to establish such programmes.

For example, in Kenya, the National Health Insurance Fund (the insurer) manages

accreditation; their standards, known as the Kenya Quality Model, were developed by

a broad coalition of professionals outside of the Insurance Fund and are supported by

the Ministry of Health. In Ghana in West Africa, the National Health Insurance Scheme

originally placed responsibility for accreditation within government; that task is now

being transferred to an independent body

The case studies highlight the nature of the relationships between external evaluation

agencies and governments in different jurisdictions.

Case Studies – Role of the government

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

IKAS and the Danish accreditation programme (DDKM) were established by an

agreement between the regional and local political authorities, who are responsible

for delivering healthcare, and the national government that sets the overarching

political priorities, including the economic frame, and is the healthcare legislator

and regulator. The ﬁrst step in the development of DDKM was the development of a

cooperation agreement between the government and the regions of a joint model for

quality assessment which included provisions for the funding for DDKM. IKAS is a

formal independent organisation but the government provides part of the funding for

IKAS.

Health Care Accreditation Council (HCAC)

Country: Jordan

The HCAC is a private, not-for-proﬁt shareholding company registered under the

Ministry of Trade and Industry.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

The Safety Act required that designated audit agencies (DAAs) who monitor

compliance with health and disability standards for hospitals, rest homes and

residential disability services are approved by the Director General of Health for

the purpose of auditing these services to those standards. HDANZ is a private,

independently owned company. It is linked to government as a Ministry of Health

(MOH) approved designated auditing agency and for these services HDANZ submits

the audit report to the MoH who issues the certiﬁcate. HDANZ was designated as an

approved designated auditing agency in October 2002.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

2.4 Determining incentives

If the external evaluation programme is not mandatory, evidence suggests that incentives

are useful to promote and sustain it. Possible incentives for healthcare organisations to

participate in an external evaluation programme include:

Organisational development: self-assessment, team-building, benchmarking,

guided pathways

Increased public funding such as health insurance fund payments moderated

by accreditation or certiﬁcation status, additional government subsidy, e.g.

per accredited or certiﬁed bed, or some other linkage to core funding or

reimbursement

Effective exchange of data between external evaluation programmes and insurance

programmes to inform their purchasing decisions and payments

Preference from private insurers: insurers prefer to deal with facilities or services

whose clinical and management processes have been independently veriﬁed; they

also make reimbursement simpler and faster for such organisations

Market advantage: public recognition brings status and advantage in a competitive

market which can attract patients / clients, staff and income

Reduction of liability insurance costs: premiums reﬂect reduced risk rating

Exemptions from regulatory inspection: e.g. the state issues a licence to an

accredited or certiﬁed facility on the basis that accreditation or certiﬁcation

standards include and exceed licensing standards (“deemed status”); this may be a

condition of receiving public funding

Linkage to training posts: status conditional on accreditation or certiﬁcation

National quality competitions: for example, making accreditation or certiﬁcation

status one of the judging criteria.

Healthcare organisations may be discouraged from participating in an external

evaluation programme by:

The cost in terms of time, management, and money

Fears about the outcome - sanctions for shortcomings, loss of staff morale if

denied the award of accreditation or certiﬁcation, misuse of performance data, and

of gaining the award and then losing it when standards get more demanding

Lack of recognition for the resources invested

Lack of information about the beneﬁts

Resistance from healthcare professionals and other staff and the failure to recruit

clinical and other staff champions

The difﬁculties of effecting culture change without external support and

Failure to recognise and celebrate the achievements of participating organisations.

Consideration also needs to be given at this time to the issue of consequences when

organisations do not achieve or meet the accreditation or certiﬁcation standards to

the acceptable level. What are the consequences, if any, for these organisations? For

example, do the consequences include ﬁnancial sanctions?

The case studies provide examples of some of the incentives put in place for external

evaluation programmes.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Case Studies – Incentives for external evaluation programmes

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

DDKM (Danish accreditation programme) is not required by any legislation, but is

based on agreements as follows:

Public hospitals: all hospitals participate by agreement between National and

Regional governments

Private hospitals: voluntary, but participation is a prerequisite to obtain a contract

to treat patients for the regions (also required by some insurance companies)

Pharmacies: voluntary, ﬁnancial incentive in place

Municipalities (primary care services, including long-term care): voluntary, no

incentives in place

Ambulance operators: prerequisite to obtain contract with Regions

General practitioners: mandatory (with some minor exceptions) by agreement

between the Regions and the Organisation of General Practitioners in Denmark;

ﬁnancial compensation as part of the agreement.

Health Care Accreditation Council (HCAC)

Country: Jordan

Accreditation is voluntary. There are no incentives (laws, regulation, insurance

requirements) in the country for accreditation.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

The Safety Act 2002 introduced health and disability standards for hospitals, rest

homes and residential disability services aimed at improving safety levels and quality

of care that became mandatory from 01 October 2004. Under the Safety Act 2002,

service providers such as hospitals, rest homes and residential disability service

providers must be certiﬁed. From September 2005, physiotherapy services were

required to be certiﬁed if they wished to provide services under the New Zealand

Accident Compensation Scheme (ACC) physiotherapy services contract. From

September 2012, health funders made certiﬁcation mandatory for home support

providers and from March 2013, a health insurance provider Southern Cross Health

Society made certiﬁcation mandatory for their afﬁliated providers.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Practice Incentive Program (PIP)

Country: Australia

The Australian Government introduced the Practice Incentive Program (PIP) in 1998.

The PIP is aimed at supporting general practice activities that encourage continuing

improvements and quality care, enhance capacity and improve access and health

outcomes for patients

In the 2015-16 Australian Government Budget, in excess of $1.5bn over four years

was allocated to the PIP to support the continuation of incentive payments to general

practices.

The PIP is used as a lever by government to inﬂuence behavioural change within

the general practice environment. To access payments under the PIP, practices

must meet the eligibility requirements, including that a practice must be accredited

or registered for accreditation against the Royal Australian College of General

Practitioners (RACGP) Standards for general practices and must maintain full

accreditation.

Approximately 80% of all practices that meet the RACGP deﬁnition of a general

practice participate in accreditation and, therefore, may access PIP payments.

There are three types of payments available under the PIP

1. Practice Payments

The majority of payments through the PIP are made to practices and focus on those

aspects of general practice that contribute to quality care. These payments are

intended to support the practice to purchase new equipment, upgrade facilities or

increase remuneration for GPs working at the practice.

2. Service Incentive Payments

Service Incentive Payments (SIPs) are generally made to GPs to recognise and

encourage the provision of speciﬁed services to individual patients. The Cervical

Screening, Asthma and Diabetes incentives have service incentive payment

components, and the Aged Care Access Incentive is a service incentive payment

only.

3. Rural Loading Payments

Practices participating in the PIP, with a main practice location situated outside

capital cities and other major metropolitan centres, are automatically paid a rural

There are ten individual incentives available to general practices and GPs under the

PIP

: (See Appendix 1d for further information)

Since the inception of the PIP in 1998, successive Australian Governments have

committed to ongoing funding for the program; and during this time, have retained

the requirement that a practice must be accredited, or registered for accreditation,

and must maintain full accreditation in order to access such payments.

Given the level of participation in accreditation by Australian general practices, it can

be assumed that the highly incentivised PIP has been instrumental in encouraging

practices to engage in the process, and in turn has had a positive impact by

supporting practices to focus on improvements and quality outcomes.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

2.5 Developing relationships with stakeholders

Another key exercise at this stage is to identify or map out the other main stakeholders

in the quality and safety arena in the country or region; their role; and their link to the

external evaluation programme. This may be different for each country or region and

this exercise will help to establish what external inﬂuences for the programme will look

like and what the nature of the relationship with the other stakeholders should be. For

instance, if the external evaluation organisation does not itself manage related functions

at a national or regional level, then it needs to deﬁne communications and relationships

with other departments and agencies to harmonise the setting and assessment of

healthcare standards, to avoid waste and conﬂict between systems, and to minimise the

“burden of audit” on healthcare organisations. A new organisation should seek where

possible to integrate and build upon existing systems of standards and inspections.

For example, by establishing a process to recognise existing ISO or mandated audits.

In addition, there are a number of organisations internationally who deﬁne and assess

standards, and with whom they could usefully collaborate, ISQua being one.

Key stakeholders with whom the external evaluation organisation may consider

developing relationships with include the following:

Consumer groups

Representatives of a recognised consumers’ council or association should be involved in

the creation and support of the proposed external evaluation organisation as a means of

making health services more transparent and accessible to the public. They should help

deﬁne what standards and services the public should expect from healthcare providers,

and develop and promote reliable and consistent methods for measuring them. They may

assist with developing a consumer code of rights. Consumer and patient representatives

may also be part of the advisory committee of the external evaluation organisation and

later sit on the governance board.

Regulatory inspectorates and other external agencies

These might include statutory bodies with responsibility for areas such as ﬁre safety,

radiation, medical device safety, hygiene and health data collection agencies. The

relationship between the country’s or region’s ISO accreditation organisation and the

health service accreditation or certiﬁcation organisation needs to be explored and

deﬁned. Relationships also need to be built with the assessment organisation that

certiﬁes laboratories, x-ray departments or other technical services and organisations to

relevant ISO standards, to understand each other’s needs and requirements and possibly

coordinate activities and assessments.

Key relevant legislative requirements such as for buildings, health and safety in

employment, equal opportunities, consumer rights or waste management can be more

speciﬁcally referenced in the external evaluation organisation’s standards in consultation

with the relevant agencies responsible. Speciﬁc technical standards or regulatory

requirements relating to safety such as infection control, ﬁre safety, equipment safety

and emergency preparedness can be integrated into the standards as criteria and

assessed as part of the survey or assessment visit.

Most accreditation or certiﬁcation organisations assume that statutory inspections are

carried out as intended, and expect to examine safety certiﬁcates, such as for radiation

protection as part of their own surveys, but in some countries the statutory radiation

protection agency does not have the resources to carry out its own inspections and may

turn to the accreditation or certiﬁcation organisation to provide its own expertise.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

A process needs to be developed to determine which alternative evaluations are robust

enough to be accepted as proof of compliance.

Public and community health bodies

Links between these bodies and the external evaluation organisation would give an

opportunity to share data to describe the impact on population and community health and

on the performance of providers and the healthcare delivery system. Where countries

currently employ inspectors to regulate healthcare facilities, the inspectors’ role could be

modiﬁed to include assisting local facilities to prepare for external evaluation surveys by

the organisation when it is established, and to monitor the implementation of the ensuing

recommendations for improvement. This would require initial and continuing education

programmes.

Technical agencies

Relationships with agencies for aspects such as health technology assessment, clinical

guidelines, clinical pathways and patient / consumer safety are useful, especially to

enable consultation and advice on the development of appropriate evidence-based

standards and for keeping information and communications current.

Professional bodies

Independent bodies such as medical academies or councils will offer wisdom and

advice to the organisation and be recognised for that purpose. Other bodies responsible

for such duties as supervising training or licensing or registering clinicians (doctors,

nurses, dentists, pharmacists, allied health professionals) will contribute to the setting of

standards and to their local assessment.

In particular, the role of professional chambers, associations and colleges needs to be

deﬁned with respect to:

Professional regulation

Setting and monitoring of clinical performance standards

Monitoring of clinical practice according to these standards

Development and dissemination of quality improvement methods.

The functions of statutory bodies should be deﬁned in relation to voluntary associations

and to the external evaluation organisation. The organisation should work with local

government ministries, insurance funds and professional associations and chambers to

develop consistent incentives for measurable achievement of agreed national standards

of process and outcome in primary, ambulatory and hospital care.

Health insurance funds

Using contracted service providers offers an alternative to the traditional centralised

model in healthcare management. In several countries, laws on healthcare insurance

specify that only accredited organisations, from either the public or private sector, have

the right to sign contracts to provide services under compulsory insurance. The external

evaluation organisation can work with health insurance funds to help them obtain and

protect best value from available funding by recognising accreditation or certiﬁcation for

its impact on quality improvement.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

External assistance

A further group of stakeholders with whom an external evaluation organisation may

interact would be individuals, organisations or groups providing external assistance.

External assistance is available from a number of sources including:

International external evaluation businesses or initiatives

International aid organisations and technical corporations

International experts

Neighbouring external evaluation organisations

ISQua.

Assistance may be for any part or all of the components of an external evaluation

programme. Before engaging formal external assistance, it is important that:

The project speciﬁcations have been scoped out and are appropriate

Competency criteria for selection of external assistance include relevant

experience with health or social care standards based external evaluation

References and advice are sought from experienced accreditation or similar

organisations and ISQua.

Most accreditation organisations have based their standards on existing research,

clinical practice guidelines, input from experts and other accreditation and technical

standards. New organisations can, in consultation with the owners of these standards,

choose a model that best reﬂects their purpose, scope and cultural context, and then

adapt those standards or build on them to make them appropriate to the local context.

It is important that the standards adhere to the ISQua Guidelines and Principles for the

Development of Health and Social Care Standards

as these are accepted as best practice

by organisations and so that they can become internationally accredited (See Chapter 4

for more information).

ISQua’s Guidelines and Standards for External Evaluation Organisations

and for

Surveyor Training Standards Programme

provide guidance on what structures, systems,

processes and evaluation methods need to be in place to be a best practice organisation.

When organisations seek ISQua accreditation, they get assistance with their self-

assessment and they can have a mock survey prior to an international accreditation

survey.

Information speciﬁc to healthcare external evaluation is widely available - see web links

in the bibliography section.

The next chapter will focus on the initial steps involved in setting up an external

evaluation organisation including how to involve and engage with other stakeholders as

part of this process.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Chapter 3: Setting up

the External Evaluation

Organisation

This chapter focuses on the process of establishing an external evaluation organisation and

the different stages in this process. This process may be different for each country or region

depending on government policy, the stakeholders involved and the size of the health or social

care sector. The case study examples outline the approaches adopted in different countries.

3.1 Establishing a preliminary board or advisory committee

The impetus for setting up an accreditation or certiﬁcation organisation may come from

a number of possible stakeholders: Ministry of Health, health professional associations,

consumer organisations, private insurers, university departments, voluntary membership

societies, health service charities or aid organisations. The initiative may come from

a company or group of individuals who see a market opportunity, e.g. as assessors

of government standards. If the purpose of the programme is clear, it is not difﬁcult

to identify whom it will serve and whom it will affect. Traditional, profession-driven

programmes have tended to build links with regulators and consumers, thus becoming

more accountable and transparent. More recent programmes have been more inﬂuenced

by commercial providers and insurers or actively supported by government.

One way of involving relevant stakeholders who have or will have an interest in the

success of the new organisation is through setting up a preliminary board or an advisory

committee to establish the organisation. This enables them to feel they have a stake in

the organisation and its work and to provide advice and expertise.

The preliminary board or advisory committee will provide guidance and direction on the

practical aspects of establishing the external evaluation programme including:

Clarifying the role of the external evaluation programme in the context of other

departments and agencies working in the quality and safety arena in the country or

jurisdiction e.g. other external evaluation programmes

Funding of the external evaluation programme

Governance framework for the external evaluation organisation

The use of external assistance for development and delivery of the external

evaluation programme.

The composition of the interim board or advisory committee will be unique for each

country depending on government policy and the range of stakeholders working in

the quality and safety arena. Some members from this board or committee may form

the basis for the governance board in the established organisation. Table 3 outlines

suggested members of a preliminary board or advisory committee.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Table 3: Potential composition of a preliminary board or advisory committee

Stakeholder

Group

Examples of representatives

Government Ministry of Health and / or other related departments e.g. Finance.

Local government e.g. municipality, canton, oblast level

Consumer

groups

Recognised national consumer council / association or advocacy

organisation

External

evaluation

organisations

Regulatory and other external evaluation agencies working in the

quality and safety arena in the country or jurisdiction e.g. statutory

bodies with responsibility for areas such as health and safety,

radiation, medical devices, medicines, regulatory inspectorates,

certiﬁcation agencies

Service

providers

Public and private providers in country or region e.g. national

representative bodies such as national hospital association or

national disability service providers association / forum

Professional

bodies

Independent bodies with responsibility for the licensing or

registration of health and social care professionals or the

supervision of training such as medical academies or councils

Academia Universities or colleges who deliver education and training

programmes for health and social care professionals

Technical

agencies

National agencies with a speciﬁc role e.g. health technology

assessments, clinical guidelines and pathways, patient / consumer

safety

Independent Independent experts, neighbouring external evaluation

organisations, international external evaluation initiatives

3.2 Proposing a governance board and framework

One of the ﬁrst tasks for the interim board will be to develop a draft governance

framework for the external evaluation organisation or programme, with a formal

constitution, governance board and draft policies and procedures. For credibility and

in line with best practice, a commitment should be made that the organisation will be

established in line with the ISQua Guidelines and Standards for External Evaluation

Organisations

(currently 4th edition, 2014, but note that these are updated on a regular

basis and the latest ones should always be obtained).

3.2.1 Governance body

If it is to be a non-governmental organisation, it is preferable for the organisation to have

a board comprising and accountable to the various stakeholder organisations rather

than the government. The board should represent professional, public and governmental

interests and bring personal qualities to the governance of the organisation, such as

ﬁnance, legal and public relations, but be dominated by none of them. For example, in

Malaysia accreditation programmes are delivered by the Malaysian Society for Quality in

Health (MSQH), which was established by the Malaysian Ministry of Health in association

with the Private Hospital Association and the Malaysian Medical Association. All three

organisations are represented on the board of MSQH

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Typically, independent boards include consumers; representatives of professional

associations such as nurses, managers and doctors; industry associations such as

hospitals or rest homes; funding agencies; and statutory bodies. Some boards are

now appointed according to skillsets, expertise and experience rather than chosen by

representative stakeholder organisations because of the perceived conﬂicts of interest

the representative members may have, being the provider, consumer and sometimes

also purchaser of the external evaluation. Government representatives in particular may

have a perceived conﬂict of interest.

Public involvement goes beyond the sharing of information; it also demands the sharing

of authority. Many external evaluation organisations have representatives of patients and

the public in their governance structure to ensure their involvement in the development

of policy and standards and in ensuring that agreed procedures are followed throughout

the external evaluation process.

As per good governance practice, members of the governing body must be oriented to

their roles and have ongoing information and education to assist them in their role. They

should be guided by a set of governance policies.

Case Studies – Composition of governing board

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

IKAS and DDKM were established by an agreement between the regional and local

political authorities, who are responsible for delivering healthcare, and the national

government that sets the overarching political priorities, including the economic

frame, and is the healthcare legislator and regulator. The government is represented

on the board of IKAS; the Chair of the Board is a government representative, a

Director of the Danish Health and Medicines agency.

Health Care Accreditation Council (HCAC)

Country: Jordan

The board of directors is made up of representatives for all healthcare sectors in

Jordan, medical and nursing professions, and education.

3.2.2 Governance framework

The external evaluation organisation needs to be set up as a legal entity, or a part of one,

with clear legal responsibilities for all its external evaluation activities. If it is part of a

Ministry or government agency, this independence is particularly important.

The organisation’s governance arrangements need to be clearly described in a deed,

constitution or similar document that deﬁnes powers, accountability and responsibility

including:

The composition of the governing body

The process for appointing its members

Lines of accountability including lines of accountability out of the legal entity

The terms of reference of the governing body and any of its committees

Responsibility and rules for making decisions such as on accreditation or

certiﬁcation awards.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

The organisation requires a clear vision and mission or purpose and strategic direction

to provide the basis for the organisation’s planning and direction and must be guided by a

deﬁned set of values which are reﬂected in all services and activities. It is also important

that the organisation has an explicit set of ethical principles to inform all decision-

making and a code of conduct outlining the expected behaviours of those working in and

/ or on behalf of the organisation. Other responsibilities for overseeing, monitoring and

approval also need to be deﬁned

3.2.3 Committing to fairness and transparency

External evaluation organisations which have succeeded in making improvements

in client healthcare organisations have generally done so by stimulating internal

motivation and commitment to self-assessment and change. This requires a culture

of transparency and acceptance of personal and organisational responsibility among

management, clinicians and other staff. However such a culture is not universal,

especially in hierarchical systems. External evaluation organisations cannot rely on

health professionals’ ethics and self-regulation to ensure an open and fair culture that

promotes quality improvement. The commitment to fairness and transparency must be

built into the governance framework and the ways of leading the organisation.

In setting up the new external evaluation organisation, a commitment must be made that

it will:

Use transparent and objective systems, decision-making and reporting

Be free from undue inﬂuence by any party

Avoid conﬂicts of interest

Establish a fair complaints and appeals system

Design and publish procedures for contracting, facilitation, assessment, reporting

and accreditation or certiﬁcation decisions to promote conﬁdence and

Put arrangements in place that ensure that external evaluation activities are

strictly separated from consultancy.

This commitment should be deﬁned in policies, including one requiring accreditation or

certiﬁcation decisions to be made solely based on the relevant standards, the ﬁndings

of the surveyors / assessors and other objective evidence related to the standards. A

growing trend is for decisions on accreditation status to be made based on a formulaic,

mathematically oriented approach, which avoids any perception of bias

3.3 Funding of the programme

Most new external evaluation organisations require at least two years to establish their

organisation and / or programme, longer before they are sustainable, and longer still

before they are self-ﬁnancing. In short, political and ﬁnancial support generally needs to

be consistent beyond the term in ofﬁce of most health ministers and many governments.

External funding from government, health insurers, aid organisations or other partners

will be required for:

Establishment of the external evaluation organisation

Initial development and testing of the standards

Marketing

Possibly subsidising the running of the organisation for the ﬁrst few years or a year

after break-even.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

However, the initial set-up costs may be much less for external evaluation organisations

whose role is to accredit or certify health or social care organisations against

government-mandated standards or similar. In this situation, there is an identiﬁed

potential client pool, there will be guaranteed payment of costs of the assessment by

either the clients or the government and there may be a shorter time period in which

clients are required to be assessed (See the example from HDANZ in the Case Studies

section).

For most other organisations the number of potential client health or social care

organisations will be a key determinant of programme costs, as will other factors such

as whether the programme:

Is a single national programme, regional or sector speciﬁc

Is limited initially to a priority focus, e.g. nursing homes, or to the entire health

system

Is supplementing or replacing existing external assessments

Is development focused, requiring training and education of clients

Develops its own standards

Employs specialist expertise.

One of the major potential costs for an external evaluation organisation will be the

surveyor workforce and in particular whether they are paid or voluntary. Traditionally,

accreditation organisations have relied upon participating accredited institutions to

provide or loan staff to work as surveyors and to promote the concept of peer review.

Certiﬁcation agencies usually employ or contract their assessment personnel on a

paid basis, sometimes supplemented by technical experts. However, accreditation

organisations are now also increasingly paying surveyors as employed or contracted

personnel, or using a mix of both paid and voluntary. The organisation would need to

consider factors such as the availability of suitable personnel in the country to act as

surveyors; the feasibility of suitable personnel being released by their organisations to

work as surveyors; and the number of and costs of employing full or part-time surveyors

when deciding on which approach to take.

Thorough system design and testing will be another cost, as will the investment in

communications, information management and marketing.

Although a sustainable external evaluation organisation and its programme are

constantly under development, the start-up costs may last 3-5 years before a tested and

valued product is sufﬁciently marketable to begin to recover operational costs from client

organisations. Whether they choose to participate, or whether they can afford to, depends

on the incentives and sanctions provided and existing operating budgets.

During the ﬁrst year, the organisation may manage with a small core staff, several

working groups and low overheads; however costs increase rapidly with the addition

of, surveyor training, document production and the direct costs of ﬁeld testing. In some

countries external expertise is required and must be factored in to the start-up costs. At

the next stage, when the initial development is completed and the organisation is ready

to offer accreditation or certiﬁcation, it may face another challenge; the faster the rate

of uptake, the faster it must invest to build capacity. Funding should be proﬁled to reﬂect

this growth.

At the same time as obtaining funding, incentives need to be negotiated if possible.

The case studies outline the experiences of external evaluation agencies in different

countries in terms of funding arrangements.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Case Studies – Funding

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

Set-up costs

When IKAS was being established, a decision was made to seek external assistance

to help with the establishment of the organisation and the development of the

accreditation programme. A request for tender was issued to international

accrediting organisations to provide consultancy services for the establishment

of IKAS and the development of DDKM. The United Kingdom based international

accreditation organisation CHKS was awarded the contract to assist with the

establishment of IKAS as an accreditation organisation; the development of

standards; and the training of surveyors.

Funding of the accreditation scheme

IKAS is an independent organisation but receives an index-linked annual grant from

the central government, regions and local government. Public clients such as public

hospitals or pharmacies do not have to pay any fees to participate in DDKM. Other

private clients pay a fee that covers direct expenses plus an overhead.

Health Care Accreditation Council (HCAC)

Country: Jordan

Initial funding

The original funding to develop the HCAC came through the Jordan Healthcare

Accreditation project funded by the United States Agency for International

Development (USAID) and grants. The HCAC is a private, not-for-proﬁt shareholding

company registered under the Ministry of Trade and Industry. Since March 2013,

HCAC has been ﬁnancially sustainable through charging fees for services offered

including surveys, education and consultation.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

HDANZ is a private, independently owned company. It is linked to government as a

Ministry of Health (MOH) approved designated auditing agency. HDANZ audits these

services on behalf of the MOH and submits audit reports to the MOH who then issues

the certiﬁcates to the services.

Service providers pay fees to HDANZ for survey and monitoring visits. Certiﬁcation

has been mandatory for the MOH Safety Act since October 2002. From September

2005, it became mandatory for physiotherapy services if they wanted to provide

services under the New Zealand Accident Compensation Scheme (ACC)

physiotherapy services contract. From September 2012, health funders made

certiﬁcation mandatory for home support providers and from March 2013, a health

insurance provider Southern Cross Health Society made certiﬁcation mandatory for

their afﬁliated providers.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

3.4 Setting up strategic, operational and ﬁnancial management

systems

Once the governance board has been established and the governance framework has

been developed, the next step is to staff the external evaluation organisation and to

develop the management systems.

3.4.1 Stafﬁng the organisation

The most important task of any board is to appoint the chief executive, with the

appropriate skills and experience for the role. The governing board may delegate

accountability, authority and responsibility for managing the external evaluation

organisation to a chief executive. The responsibilities for managing the organisation, the

level of authority and the chief executive’s relationship and accountability to the board

need to be deﬁned in a job description or similar document. It is also the board’s role to

conﬁrm strategic and operational plans, to receive regular reports on achievement of

goals and targets and to review the chief executive’s performance annually against set

performance targets

After the chief executive has been employed, personnel need to be selected, trained

and paid, including employed staff, seconded staff, e.g. surveyors, and sub-contractors

e.g. legal, statistical, marketing, communications. Sometimes ﬁnancial and information

technology staff are contracted.

In larger organisations, staff may be structured into functional units such as:

Survey planning and management

Surveyor recruitment and development

Standards research, development and revision

User education and development

Technical support staff – ﬁnancial, human resources, information management

Administration.

Smaller organisations can be sustained on very few core staff if they have signiﬁcant

support from unpaid experts and staff seconded from employment in health and social

care services. Stafﬁng numbers and skill levels need to be planned and transparent

policies developed for recruitment, selection and appointment; orientation; health and

safety; ongoing training; and regular performance assessment. Personnel records with

deﬁned content need to be established for all staff.

It is important that the lines of responsibility within the external evaluation organisation

are clearly deﬁned; made known to all staff; and that there are processes in place to

ensure that staff and surveyors are free from inﬂuence by those who have a direct

interest in the services and accreditation / certiﬁcation decisions. The lines of authority,

responsibility and allocation of functions in the external evaluation organisation may be

outlined in an organisational chart or organogram. The lines of responsibility may be

outlined to staff as part of their orientation and updates provided whenever there is a

change of responsibilities.

A ﬁnancial system needs to be set up to develop budgets and record and track income

and expenditure and past, current and projected ﬁnancial positions. It needs to be able to

produce timely reports to assist staff to manage their budgets. Control and audit systems

will be needed to protect assets and ensure the transparency of ﬁnancial transactions.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

3.4.2 Developing the system for ﬁnancial sustainability

Initial budgeting is challenging and depends on how much funding is received for

development or how much of the set-up costs need to be included in the budget.

Provision usually needs to be made for external assistance and expertise. Some

organisations consider guided facilitation and / or training on the survey standards

and process as an integral part of the development process. Others provide separate

consultancy (including general education and development) for which they charge a fee

which can be budgeted for.

If client healthcare organisations are required to pay on an event basis, ongoing costs

will depend on the length and depth of surveys (which are inﬂuenced by the standards),

length of the survey cycle, mid-term monitoring system, the efﬁciency of scheduling,

survey logistics, report handling and award adjudication. Budgets have to predict

when events such as training, on-site surveys, and mid-term surveillance visits will

occur and how much they will cost. Any postponement or cancellation can negatively

affect anticipated cash ﬂow. Some organisations include all documentation and direct

survey costs, e.g. surveyor travel and accommodation, into a single-price package per

survey but costs and revenues are still dependent on the event occurring. A number of

accreditation organisations have moved to a membership or subscription based ﬁnancial

system, whereby clients become members of the accreditation programme and are

charged a regular annual fee based on anticipated costs over the whole accreditation

cycle, including overheads, education, guidance, standards, tools, survey and mid-

term progress visits. While it still requires budget forecasting of the number and type

of clients, it limits the uncertainty of whether and when events will happen and has

contributed to the ongoing sustainability of a number of accreditation organisations.

A marketing programme and budget will be needed by most new external evaluation

organisations to publicise itself, the services it offers and the beneﬁts of its programme

to attract healthcare providers. Getting a sustainable market share of client

organisations will be fundamental to its success. Wider marketing and publicity will be

needed for potential insurers, funders and the general public.

3.4.3 Establishing information systems

Information management covers both technological and paper based information,

including educational and marketing resources. Internal information systems are

essential for planning, operations and ﬁnance, but they also need to have the capacity

to collect, aggregate and compare data over time within and between participating

organisations, standards and surveyors, such as:

Data of compliance with achievement of individual criteria or standards

Proﬁles of participating organisations

Calculation of standard scores, function scores, and overall score for each

organisation

Aggregated results for comparison over time, function and place

Proﬁles of individual surveyors and their participation

Survey scheduling and management

Overall impact of programme.

Data which show that participating organisations have made improvements associated with

the programme since the ﬁrst (baseline) contact are essential to demonstrate the value of the

programme to the healthcare system

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

3.4.4 Addressing risk management and performance improvement

The external evaluation organisation must model the safety and quality approach

it expects from its client organisations. A robust risk management framework that

identiﬁes and manages risks and promotes safety must be implemented. While most

of these organisations demonstrate a safety culture, it needs to be demonstrated by

establishing a quality improvement policy and framework. Essential to this will be the

documentation of policies and procedures for all functions, the development and use

of key quality indicators which can be monitored and benchmarked over time or with

similar organisations, the use of audits and reviews to ensure compliance with policies

and procedures, documented quality improvement projects and a transparent complaints

system that is available to staff, surveyors, clients and other stakeholders

3.4.5 Providing education services

Most external evaluation programmes provide a variety of education and training as

an essential component of their services. Education services need to be systematically

designed and implemented to meet quality standards and client needs. These include:

Induction and development of staff

Orientation and ongoing education of members of the governing board

Initial and continuing training of surveyors

General preparation of participating organisations and their staff as a basic

component of their participation

Speciﬁc methods of internal quality improvement required to meet external

evaluation standards, such as infection control, risk management, performance

measurement, patient / client surveys – these are usually additional to services

covered by fees and are charged separately

Quality improvement programmes for the health or social care sectors in general.

These training and education programmes and courses and their resources need to be

planned, scheduled and costed. Information provided needs to be kept up-to-date and

based on current research and evidence. Trainers and educators, whether internal or

external, need to have the competence and expertise to deliver the programmes.

3.5 Timeframes

The most commonly underestimated resource is the time needed to plan, design, build

and deliver a sustainable new external evaluation organisation. The pace at which this

can be done is limited largely by factors outside the control of the organisation, notably

by the prevailing culture and attitudes towards leadership, innovation, improvement,

team-working and transparency.

In practice the development stages, which may overlap, are:

Policy decision to develop an external evaluation organisation / programme and

deﬁning its scope

Option appraisal on existing models and their adaptation

Setting up the organisation structure and obtaining of funding

Development and testing of standards

Development and testing of assessment methodologies

Surveyor selection and training

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Pilot testing, education and marketing campaigns

Revision of standards and methods based on feedback from piloting

First “live” surveys

First accreditation / certiﬁcation recognition status decisions.

This process is likely to take at least two years but can take much longer (The case

studies outline the order of development and the timescales involved for the three

different agencies. Please refer to Appendices 1 a, b and c for further information.).

Taking time to establish communication with all stakeholders and the public and

continual updating of information as the organisation develops, is essential for success.

The following chapters focus on and provide more detail in relation to the development

and testing of standards; the development of assessment methodologies and

mechanisms for evaluating systems and performance.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Chapter 4: Developing the

standards

This chapter focuses on the different elements required when developing standards. It includes

the use of quality dimensions and the importance of a reliable and valid measurement scale.

The standards used or developed by external evaluation organisations are the most fundamental

element of their programme. While not always realistic, it is advisable to consider what

evaluation methodology will be used while the standards are still in the development phase. The

standards will help to inform the public what to expect from health and social care providers and

will act as a benchmark against which providers and the government can measure quality. The

standards will form the framework for self-assessment and internal audits.

Standards development can often commence prior to the setting up of governance and

management systems in the external evaluation organisation and can take two or more years

to complete. Funders may want to know the shape and content of the standards before they

commit to funding the organisation. Separate funding is often available for the standards

development process.

4.1 The role of standards

An external evaluation organisation’s standards have to reﬂect its purpose and cover

the key functions and processes of the healthcare or social care sectors that are being

evaluated. Similarly, if standards are owned or mandated by government, they need to

reﬂect the purpose for which government intends them. They have to reﬂect legislative

requirements, safety and good practice. They should be relevant, understandable,

measurable, beneﬁcial and achievable (RUMBA)

Standards also need to be realistic and reﬂect the availability of resources, especially in

developing countries where resource limitations can signiﬁcantly impact a healthcare

organisation’s ability to achieve optimal performance. For example, Malaysia and

Thailand began with relatively achievable accreditation standards but committed to

continue updating and improving these over time. In this context, Malaysia has published

the 4th edition of their hospital standards since the accreditation programme began in

1999. Thailand has also made progressive changes, introducing a stepwise recognition

programme in 2004 and patient safety goals in 2006

. Standards can also be prioritised

and incremental improvements made in achieving them can be recognised and rewarded.

In India, the National Accreditation Board for Hospitals & Healthcare Providers (NABH)

has developed Pre-Accreditation Entry Level certiﬁcation standards, in consultation with

various stakeholders in the country, whose aim is to introduce quality and accreditation

to healthcare organisations as their ﬁrst step towards awareness and capacity building.

Once organisations have met the Pre-Accreditation Entry Level certiﬁcation standards,

they can then prepare and move on to the next stage –Progressive Level and can then

work towards Full Accreditation status. This methodology provides a step by step phased

approach for healthcare organisations

The long-established accreditation organisations generally began with standards and

surveys which reﬂected management units, e.g. departments. They also tended to focus

on structures, e.g. stafﬁng arrangements, funding, equipment or committees. Most

programmes now focus their standards and assessments on a client focused continuum

of care or patient’s journey rather than management units and on processes and

outcomes rather than structures.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

However, for developing countries, basic structural standards may still be an important

starting point. External evaluation may be primarily a vehicle for taking stock and

developing greater equality of structure and access where the healthcare system

has wide regional and social divisions. In this case, the health system must be able

to mobilise resources in order to respond appropriately to the priorities which are

objectively demonstrated through the external evaluation process. For example,

participants from external evaluation organisations in low and middle-income

countries attending a 2013 workshop in Bangkok, Thailand highlighted that standards

are important in their countries to improve the overall quality of care and not just to

differentiate between hospitals that pass an accreditation visit and those that do not.

In many low and middle-income countries, institutions that fail to meet standards may

still be the only available source of care for parts of the population and therefore, it is

important that there is a focus on improving the care they do provide

4.2 Principles for standards

Standards are developed and written in many different ways and are designed to meet

the purpose and scope of the particular external evaluation programme, as discussed in

Chapter 2. However, they must be user-friendly, able to meet the purposes for which they

have been designed, and be able to measure achievement in a consistent way. Evidence-

based mechanisms by which standards are developed, promulgated, reinforced, audited

and evaluated are needed. Linking the writing of standards, including the wording,

structure, design, focus and content, to demonstrating improved outcomes requires

further investigation

ISQua has focused on addressing this gap by developing principles to guide the

development of health and social care standards and enable their assessment and

accreditation. These were originally developed in 2000, and revised on numerous

occasions. The most recent 4th edition was published in 2014

. The principles are based

on the Institute for Medicine (IOM) quality dimensions

, of effective quality performance,

efﬁcient organisational performance, safety and patient focus. The ISQua Principles

(2014)

also give guidance on how to develop and measure standards. ISQua recommends

that the development and content of all standards should meet its internationally

accepted best practice principles.

The purpose of some external evaluation organisations is to assess, and sometimes

certify, health and social care organisations against government standards or the

standards of another external evaluation organisation, perhaps adapted to local

circumstances. For the credibility of its own assessments, these organisations should

encourage the owners of the standards to get them ISQua accredited.

The ISQua Principles cover all the functions of a healthcare or social care organisation,

from governance, to management, to client care, to quality. They are:

1. Standards Development: Standards are planned, formulated and evaluated through

a deﬁned and rigorous process.

2. Standards Measurement: Standards enable consistent and transparent rating and

measurement of achievement.

3. Organisational Role, Planning and Performance: Standards assess the capacity and

efﬁciency of health and social care organisations.

4. Safety and Risk: Standards include measures to manage risk and to protect the

safety of patients / service users, staff and visitors.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

5. Patient / Service User Focus: The standards focus on patients / service users and

reﬂect the continuum of care.

6. Quality Performance: Standards require service providers to regularly monitor,

evaluate and improve the quality of services

Steps for developing standards in line with the ISQua Principles for Standards

include:

Reviewing other external evaluation organisation standards, current research

and evidence, recognised guidelines, recommendations from WHO and other

professional organisations and experts

Incorporating legislative, technical and safety requirements

Incorporating best practice where evidence is available

Ensuring the standards are client focused, cover the functions or systems of a

whole organisation or service, address the dimensions of quality, and support

quality improvement

Consulting stakeholder groups, including consumer groups

Involving stakeholders in standards development committees and working groups

Developing the rating system for measuring compliance with / against the

standards

Testing the standards and the way they are rated through self-assessment and pilot

surveys

Using feedback from testing to improve the standards and rating system

Developing guidelines to assist users to interpret and apply the standards

Ensuring the standards are approved by the external evaluation organisation

governing body

Applying for ISQua standards accreditation.

This development process may take two years or more if the standards are being fully

developed. With the rapidly changing healthcare environment, 12 months would be an

appropriate timeframe for organisations adapting other organisations’ standards.

4.3 Referencing to quality dimensions

Standards can be grouped around quality dimensions to demonstrate their relationship

to quality. The six quality dimensions as deﬁned within the Institute of Medicine (IOM)

report Crossing the Quality Chasm, are the most commonly referenced

Safe S

Timely T

Efﬁcient E

Equitable E

Effective E

Patient-centered P

By deﬁning the dimensions of quality, organisations are able to ensure that their

inclusion can be justiﬁed but can also measure achievement in relation to those

dimensions, demonstrating that quality is not an optional extra but the essence of a good

and acceptable service. When standards are developed the criteria should address all of

the quality dimensions.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Codes of patient / consumer rights have now been developed or adopted in many

countries. These are designed to protect an individual’s rights when they access

health or social care services and describe what their rights are when accessing such

services. In some jurisdictions, the codes of patient / consumer rights are speciﬁed in or

underpinned by legislation and service providers are required to have processes in place

to meet them. In such cases the codes of patient / consumer rights may be referenced

in the standards as this will provide a means of assessing how service providers are

meeting patient / consumer rights. In other countries, codes of patient / consumer

rights have been developed by organisations such as national consumer or advocacy

organisations and service providers may adopt them on a voluntary basis. Referencing

the codes of patient / consumer rights in standards is one way of helping to ensure that

standards are focused on the patient / consumer. This in turn will help service providers

to focus on delivering patient / consumer focused care that meets their needs and

protects their rights.

Mature accreditation organisations have now moved to designing their standards to

reﬂect the patient / consumer journey or pathway and then surveyors may, as part of the

survey process, trace or follow selected patients’ / consumers’ journeys to check at each

stage if the standards were met for that individual and their family.

Many sets of standards label some criteria as core or compulsory, usually based on

safety and risk. The core criteria are usually then required to be met or a deﬁned ratio

of them met, e.g. 80%. These core criteria may be used for licensing or regulation

purposes.

4.4 Developing the measurement system

The rating scale should reﬂect the purpose of the standards, be transparent and enable

users to rate and measure standards, criteria or elements consistently. A yes / no scale

is good for determining compliance or non-compliance with a criterion or standard,

especially for measuring structural elements, so its use should reﬂect the nature of the

standards. It leaves less scope for recommendations for improvement where a criterion

is mainly met, but some elements are missing.

Likert-type rating scales are particularly suited for standards with a strong quality

improvement approach, e.g. 3, 5 or 7 point scales, often with descriptions for each point

or some of the points. These descriptions may relate to principles such as compliance,

consistency, evidence and implementation.

There is a tendency for assessors to favour a middle or neutral point, so an even point

scale such as a four point scale can give a clear cut-off point as to whether the criterion

is met or not but still provide a graduated measure of how well it is met or how badly it is

not met. The clearer the descriptors, the more consistent the assessments are likely to

be.

As well as a measurement system for rating each measureable criterion, element or

standard, a system is needed to determine if the standards are met overall which will

be the basis for awarding accreditation or certiﬁcation where that is applicable. In a

study comparing the organisational attributes of accreditation programmes in low- and

middle-income countries with those in higher-income countries, it was found that the

low- and middle-income countries’ programmes were more likely to use a formulaic

mathematically oriented approach to make accreditation decisions

. Traditionally,

accreditation organisations relied on accreditation panels to make decisions but this

was not always a transparent process, the basis of the decision was not always clear, it

could be more prone to bias or external inﬂuence and was also likely to result in appeals

against the decision.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Therefore, best practice is to determine overall achievement of standards based on a

formula which includes the level of achievement of or compliance with the measureable

elements of the standards, risk and other elements of the standards such as core criteria

or high priority criteria.

Some organisations measure only at the criterion level, so their overall decision will

be based on achievement of criteria while others use the overall ratings of the criteria

within each standard to rate achievement of the standard, so their overall decision will

be based on achievement of the standards. For example, the methodology could be that

all core or compulsory criteria must be met, or all criteria or standards must be met at a

deﬁned level such as 3 or 4 on a 4 point scale, or no standards must be rated at below a

certain level.

Like the rest of the standards, the rating scale needs to be developed in consultation with

stakeholders and the satisfaction of users regularly assessed. As with the standards

themselves, the rating scale needs to be tested and piloted before use to ensure it is

reliable and can produce consistent and fair results.

The case study examples highlight the approaches to standards development adopted in

different countries.

Case Studies – Development of standards

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

Range of standards

IKAS has developed all standards used in its programmes. They were ﬁrst developed

for hospitals and community pharmacies. Standards have since been developed

for primary care services, delivered by municipalities, and for ambulance services.

Currently standards are being developed for general practitioners and specialist

physicians. Over the coming years, all healthcare professions who operate outside of

hospitals in their own ofﬁce or premises will be covered.

Development process

Standards were developed by theme groups (for related groups of standards) of

standard developers, consisting of senior professionals, appointed by the Regions

and the Association of Danish Pharmacies. IKAS and HQS / CHKS served as advisors

and secretariat for the groups.

Rating scale

Compliance with standards is assessed by scoring a number of elements (for the

hospital standards roughly 450) according to a four point scale (Fully / Largely /

Partially / Not Met), where the two upper levels indicate a satisfactory performance

(except for certain safety critical standards, where only Fully Met is considered

satisfactory). Any element not met to satisfaction will require follow up, and if

not corrected, results in accreditation with comments. An Accreditation Award

Panel decides, guided by certain rules, whether the nature and / or amount of the

comments preclude accreditation – if so, status as “not accredited” is awarded and

published.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Health Care Accreditation Council (HCAC)

Country: Jordan

Range of standards

As the national accreditation agency of Jordan, HCAC sets standards for hospitals,

primary healthcare centres, family planning and reproductive health, transport

services (ambulances), cardiac care, and diabetes mellitus. HCAC surveys against the

standards and awards accreditation.

Development process

All the standards are developed in Jordan. No standards developed by other

organisations are used. Hospital standards were developed ﬁrst, then standards for

primary care centres, family planning and reproductive health, transport services

(ambulances), cardiac care, and diabetes mellitus.

Rating scale

Standards are classiﬁed as critical, core and stretch.100% of critical standards must

be met; and a speciﬁed percentage of both core and stretch standards must be met in

order for a service to be accredited.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

HDANZ is a private, independently owned company. It is linked to government as a

Ministry of Health (MOH) approved designated auditing agency. HDANZ audits these

services on behalf of the MOH and submits audit reports to the MOH who then issues

the certiﬁcates to the services.

Service providers pay fees to HDANZ for survey and monitoring visits. Certiﬁcation

has been mandatory for the MOH Safety Act since October 2002. From September

2005, it became mandatory for physiotherapy services if they wanted to provide

services under the New Zealand Accident Compensation Scheme (ACC)

physiotherapy services contract. From September 2012, health funders made

certiﬁcation mandatory for home support providers and from March 2013, a health

insurance provider Southern Cross Health Society made certiﬁcation mandatory for

their afﬁliated providers.

The rating scale for compliance against the health and disability sector standards is:

CI = Continuous improvement

FA = Fully attained

PA = Partially attained

UA = Unattained

The Ministry of Health uses the assessment ratings to determine certiﬁcation. The

length of certiﬁcation can vary from one to four years depending on the level of

achievement of the standards.

The next chapter outlines the factors to be considered in developing assessment

methodologies.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Chapter 5: Developing

assessment methodologies

This chapter explores factors to be considered in the development of the assessment

methodology such as the selection, training and evaluation of surveyors; the development

of the survey management process; and the establishment of processes for determining the

accreditation or certiﬁcation status.

A survey against standards can be achieved by either a desk top review or an on-site survey. Desk

top reviews may be suitable for some specialities such as diagnostic imaging or clinical pathways

such as stroke care. For organisations an on-site survey is recommended, which can be planned

or unannounced.

Surveyors are the main interface of the external evaluation organisation with its clients, and

the survey is the key event on which the clients will judge the organisation. It is essential that

surveyors and the survey and award processes are managed consistently, transparently and well.

5.1 Selection, training and evaluation of surveyors

Accreditation organisations generally use the term “surveyors” while certiﬁcation

organisations usually use the terms “assessors” or “auditors” to describe the personnel

who visit, assess and draft reports. Regulatory bodies may use the term “inspectors”. They

are central to the credibility, objectivity and sustainability of the organisation. Accreditation

surveyors are generally regarded as peer reviewers – doctors, nurses, managers and

allied health professionals – who understand the work their peers do but their role is

to assess processes and systems rather than their peers’ performance. Auditors are

professional quality auditors, usually certiﬁed as such, who can audit or assess across

industries and do not need to be a healthcare professional peer. In this guide the term

“surveyor” is used to cover all assessment personnel and the term “survey” to cover all

external assessments.

Paid or voluntary?

As previously highlighted in Chapter 3 (See Section 3.3 Funding of the programme),

accreditation organisations have traditionally relied upon participating accredited

institutions to provide or loan staff to work as surveyors and to promote the concept

of peer review. This has the advantage of reducing survey costs, maintaining the

acceptability and independence of peer review, and sharing the experience and knowledge

of accreditation widely throughout the health system. However, it assumes that there

are personnel with enough experience who are able and willing to be seconded by their

employers to be trained as surveyors without creating a conﬂict of interest. To maintain

skill levels and currency with standards and systems, surveyors should be expected to

undertake a minimum number of working days (usually ten) a year. It can be a challenge

for them to get released from their full-time job for this amount of time.

Certiﬁcation organisations usually employ or contract their assessment personnel on

a paid basis, supplemented by technical experts. However, as highlighted in Chapter 3

(See Section 3.3 Funding of the programme), accreditation organisations are now also

increasingly paying surveyors as employed or contracted personnel, or using a mix of both

paid and voluntary. Surveyors mostly come from a health background and have previously

been involved in accreditation programmes.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

The advantages of having a more stable workforce of paid surveyors is their greater

availability, the reduced number of surveyors needed, reduced demand for recruitment

campaigns and new training programmes and more reliable and consistent performance

of the role because of the increased frequency of undertaking surveys and writing

reports.

Selecting and appointing

The ﬁrst steps in developing a surveyor workforce are to:

Determine the number, skill mix and mix of paid / employed or voluntary surveyors

needed for the planned programme of work (the numbers will need to be increased

as more organisations join the programme)

Deﬁne the required competencies, including personal attributes, professional

qualiﬁcations and experience, knowledge and skill sets relevant to the programme.

The number of surveyors to be recruited should be estimated from the volume of

surveys planned, their duration (in terms of surveyor days), the number of days each

surveyor would provide per year, the number of surveyors withdrawing each year and

the paid / voluntary mix of surveyors. Their professional background, culture and skills

should reﬂect the function and scope of the programme. Recruitment may be done by

advertising in relevant publications, sending notices to all potential client organisations

and professional associations, and directly approaching likely candidates.

Surveyors should be appointed through a clearly stated and fairly applied process in

accordance with the deﬁned competencies and the numbers determined. Competencies

could include:

Personal attributes, including the ability to communicate effectively and to work as

a team member

Professional qualiﬁcations and experience, usually at a senior level

Current healthcare or social care sector knowledge

Skills in the areas covered by the programme.

Whether surveyors are seconded (on their usual salary), or employed directly by the

external evaluation organisation, they must be committed to comply with the rules of that

organisation, particularly with respect to conﬁdentiality and independence. If the external

evaluation organisation employs them directly, it may have to accept additional legal

responsibility and have to provide additional liability insurance.

Training to be a surveyor and undertaking the role is a form of professional development

and is recognised as such by many professional colleges and associations. Surveyors

become familiar with the standards and survey processes and are able to learn for their

own practice from what they observe in the organisations in which they survey. They in

turn become educators of the staff they survey, able to identify areas where they can

improve and best practice methods or tools they could use.

Training

After selection, surveyors will then need to be either employed or contracted, and

trained and oriented to the role. Training cannot begin until at least draft standards and

procedures are available. In established organisations, training is provided by existing

surveyors and staff; new organisations generally use expertise from other programmes,

at least for initial training.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

The initial training programme can be of one to ﬁve days duration and should cover topics

such as:

Standards’ interpretation

Survey process

Interviewing and observation skills

Documentation review

Speciﬁc areas, e.g. safety, infection control

Report writing techniques

Trainees then need to be evaluated to determine their suitability for the role. Mock

assessments are often included so that trainees can demonstrate their aptitude. They

then usually go on one or more survey visits as observers or trainees with a mentor to

accustom them to the role and further test their suitability. They need manuals and other

resources to assist them. Programmes are increasingly using technology on-site for the

recording of the assessment and use of this also needs to be part of the training.

The surveyor training programme of accreditation organisations in low- and middle-

income countries tend to be surveyor certiﬁcation programmes and organisations

in developed countries are also moving in this direction. Such certiﬁcation provides

a recognised status for the surveyor but may also provide the opportunity for more

rigorous evaluation of performance and ongoing training and development. Certiﬁcation

programmes generally expect their auditors or assessors to be certiﬁed.

Ongoing development and evaluation

Surveyors must be provided with ongoing training and development opportunities,

and be evaluated regularly to ensure their ongoing competence. External evaluation

organisations need to deﬁne criteria for selecting, training, retraining and deselecting

surveyors. Some organisations have an independent committee to monitor inter-

rater reliability of the survey and rating performance of surveyors and / or satisfaction

surveys by an independent third party, as well as in-house survey team assessments.

It is common to ask client organisations to evaluate the standards, the survey, and the

performance of the surveyors after the external survey. These evaluations are most

useful if they relate to the individuals rather than just the team. All these reports, and

participation in continuing training, contribute to the systematic appraisal of each

surveyor.

Where there is a surveyor certiﬁcation programme, surveyors must meet the annual

requirements to maintain their certiﬁcation.

The ISQua Surveyor Training Programme Standards (2009)

provide guidance on setting

up these training programmes which can then be ISQua accredited. The ISQua Guidelines

and Standards for External Evaluation Organisations also contain a standard (Standard 6)

on surveyor management

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

5.2 Developing the survey management process

Contracting with the client organisation

There should be a deﬁned process to ensure that participating organisations are aware

of their rights and responsibilities in relation to the external evaluation programme, and

that they understand the procedures and responsibilities of the programme. This usually

involves a standard contract or service agreement between the applicant healthcare or

social care organisation and the external evaluation programme.

Training and educational support are often provided by the programme for the staff of the

client organisation as an integral part of the preparatory process. This may include for

example: project manager training, standards interpretation, and internal assessment

and self-assessment training. Where self-assessment is a component of its programme,

the external evaluation organisation’s staff can guide the client as to how to undertake

and complete this. Self-assessment against the published standards develops insight

and commitment, and reduces the burden of external assessment because it helps

organisations to identify, understand and resolve their own problems. Many programmes

consider this internalisation to be a key factor in the rapidly increasing compliance with

standards which can be demonstrated in participating organisations in the months prior

to external survey. It is important to determine what is included within the programme

fees and what training / educational support is provided at an additional fee.

Many programmes provide facilitators, such as programme staff or trained surveyors,

to support client organisations to prepare on ﬁrst entering the programme, and to feed

back to the programme any problems with systems or processes. This acknowledges that

the early external surveys are as much a test of the standards, surveyors and procedures

as they are of the organisation being visited. The facilitators should not be permitted to

take part in or inﬂuence the external survey. They can arrange training, participate as a

trainer, advise clients on interpretation of the standards or what needs to be in place to

meet the standards but they can only provide generic advice that is freely available in the

public domain. They must not give any advice on how things should be done or provide

any technical assistance such as preparing or producing documentation or procedures,

or giving client-speciﬁc advice, instructions or solutions. This would be regarded as

consultancy which must be strictly separated from external evaluation activities.

A pre-survey review or mock survey can also be a valuable part of preparation. It

identiﬁes whether the client organisation is interpreting the standards correctly and

has appropriate documentation as evidence of how it meets different criteria as well as

indicating the client’s progress towards survey readiness. It also provides a good practice

run for staff so they know what to expect from the actual survey.

Planning and conducting the survey

Planning the scope of the survey, duration and the size of the survey team should be

transparent, based on the needs of the organisation and the policies of the external

evaluation body. The surveyor team for the external survey should include an appropriate

mix of skills and experience and avoid conﬂict(s) of interest. A more experienced team

leader is generally chosen to guide the process. Dates for the external survey are usually

set 6-12 months in advance to allow for self-assessment and preparation and possibly a

mock survey.

The standards must be incorporated into a tool in which surveyors can make ﬁndings,

ratings and recommendations for improvement. The self-assessment can be included in

the tool if this is part of the process. The tool may be on paper or loaded into a tablet or

similar technological device.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Site visits may extend from half a day for one surveyor for a small rural primary care

clinic to two weeks for large teams for a healthcare network. Small hospitals or rest

homes often use two people for two days (four surveyor days); larger ones commonly use

three people for ﬁve days. Time for surveyor preparation, travel, team brieﬁng and report

completion must be added to these “on-site” estimates. At the end of the visit, most

organisations provide time for the team to prepare a report back of ﬁndings which they

present at a meeting to the leadership of the client organisation and preferably also to

staff. This enables the client to correct any errors at the time and means there should be

no surprises when they receive the ﬁnal report.

The efﬁciency of the survey visit and the transparency and consistency of the process

can be improved through the provision of tools and guidelines to assist the surveyors;

thorough preparation by the organisation being surveyed and the surveyors; the timely

submission of complete and accurate self-assessments and other pre-visit documents;

a realistic survey timetable; explicit sampling procedures; speciﬁed documents being

made readily available for review on site; and time management. Increasing the number

of surveyor days may not help, but will certainly increase the complexity and cost of the

visit.

Writing the report

The surveyors write a report of their ﬁndings and rating of achievement against the

standards either while still on-site at the end of the visit or afterwards. Doing this

electronically contributes to the speed with which the report can be submitted. New

external evaluation organisations should include the e-generation of the report as part of

their programme if possible. It is important that strict timelines are put on this process,

otherwise the surveyors can get back to their usual workplace and try to catch up on

that work before ﬁnishing the report. A delay at this stage leads to a delay in making the

award decision which is frustrating for the client. The report is submitted to the external

evaluation organisation which must have processes for editing and reviewing the reports

to ensure they are complete, accurate, balanced, constructive and consistent with the

intent of the standards.

Performance indicators

The external evaluation organisation should determine what indicators it requires its

client organisations to monitor. These should cover the different management, safety

and clinical functions of the healthcare organisation and may include things such as

complaints, patient / client satisfaction, staff satisfaction, staff turnover, ﬁnancial ratios,

adverse events, accidents, clinical indicators such as falls and infections, and medication

errors. These demonstrate that the client organisation has the capacity to generate and

analyse performance data as part of an internal quality improvement programme and is

using the results to make improvements.

Sometimes the collection, analysis and publication of the results of indicator data is

part of the scope of the external evaluation organisation. In these cases, there must

be processes to ensure the indicators have standardised deﬁnitions and numerators

and denominators, that the data collected is clean, complete, accurate and timely. The

data can then provide comparable measures of achievement over time for a healthcare

organisation or between similar organisations in terms of processes and outcomes in

clinical, safety, ﬁnancial or other areas

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

5.3 Establishing the accreditation / certiﬁcation process

Responsibilities for accreditation / certiﬁcation

The external evaluation organisation is responsible for setting the criteria for

determining accreditation or certiﬁcation status, and the decision on whether or not to

grant accreditation status is made in accordance with the criteria on the basis of the

ﬁndings in the survey report. These criteria should ensure:

Transparency for organisations being accredited or certiﬁed, for surveyors and for

the public

Consideration for the clients of the service and their safety

Decisions based on the achievement of the standards

Consideration of how accreditation or certiﬁcation status will facilitate further

quality improvement

Consistency between award decisions

A non-adversarial process for appeals.

Basis for recognition decisions

Earlier programmes based the recognition decision or accreditation status primarily

on the capacity for good clinical care, demonstrated by compliance with accreditation

standards, but the emphasis has now shifted towards overall performance. Newer

accreditation programmes, especially in developing and under-resourced countries, may

need, at least initially, to focus on and to reward the existence of basic infrastructure

and demonstrated progress towards, rather than absolute compliance with, the

published standards. Different programmes may have different priority concerns, e.g.

critical functional areas such as patient care, infection control, quality improvement or

management of the environment; patient safety goals such as patient identiﬁcation, high

alert medications, wrong-site surgery or communication among caregivers; or areas of

difﬁculty such as information ﬂow, patient records or medical equipment surveillance.

In case there is any dispute about the recognition decision, a transparent, independent

and clearly described appeals process is necessary.

Timeframe for recognition decisions

Having worked hard to prepare for the external survey, staff and management of

client organisations are eager to receive a timely decision from the external evaluation

organisation. Many programmes still aim to provide the majority of decisions within

two months of the survey, although those using electronic technology for reports and

formulaic criteria for decision making are able to make the decisions much quicker. As

the delay increases, the report and decision become increasingly irrelevant, staff become

demotivated and improvement is not sustained. The adjudication process must therefore

be transparent and thorough, but also timely.

Duration and maintenance of accreditation

Accreditation status is normally awarded for a period of between one and four years.

Sometimes there are different grades of achievement, e.g. conditional, or with

commendations, or exemplary. ISQua criteria now require monitoring by the external

evaluation organisation of the continued maintenance of standards and quality

improvements by accredited or certiﬁed organisations.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Monitoring could include submission of an action plan following the award with

timeframes for making improvements recommended in the report and regular

updates on progress with implementation. In most programmes, the majority of

report recommendations after the external survey are about improving systems in the

organisation rather than about increasing resources and, as with the preparation, the

organisation should be incurring much of that cost anyway so it should not be a barrier

to improvement. Other monitoring may require a review of speciﬁed documents that

were deemed incomplete, inadequate or missing; annual or mid-term visits and random

reviews. Longer intervals between external surveys tend to instil a false sense of security

and remove the momentum for internal improvement.

Publication of results

The extent and methods of public disclosure of survey ﬁndings and accreditation or

certiﬁcation awards must be agreed in advance by the external evaluation organisation

and the various stakeholders. The public should have access to information about which

organisations are accredited or certiﬁed. Some organisations are now publishing the

survey reports or a summary of them. Regulatory bodies are usually mandated to publish

full reports.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Example: Public disclosure of accreditation reports: Japan

The Japan Council for Quality Health Care (JCQHC) was founded in 1995 and has developed

standards and criteria for accreditation and began carrying out on-site assessments in

1997. Japan has a universal health insurance system and so Japanese people have a right

to receive medical care at any healthcare organisation and hospitals cannot refuse any

patients. Hospital accreditation is voluntary and requires an application fee. Hospitals

receive scores for each item in all areas with comments from JCQHC in the standard

accreditation process. There are two forms of disclosure of hospital accreditation reports in

Japan:

1. Self-disclosure to the public directly by hospitals;

2. Disclosure by the JCQHC with agreement from the hospital concerned.

Hospitals are not permitted to disclose only selected parts of their accreditation report as

the purpose of disclosure of accreditation reports is to give consumers access not only to

favourable aspects of the report but also to information about those aspects of the service

that require improvement. The data disclosed by the JCQHC to the public include summary

comments and accreditation scores for all the items assessed.

A study was performed in Japan to examine the association between accreditation scores

and the disclosure of accreditation reports. This included a questionnaire to hospitals

who disclosed their accreditation reports to gather data about hospital characteristics

along with perceptions about the public disclosure of accreditation reports. A total of 547

of the 817 hospitals accredited by JCQHC participated in the study. Comments about the

disclosure of accreditation reports were categorised into ﬁve general subject areas: (1)

impact of disclosure on the public, (2) advantages to the hospital, (3) risks to the hospital,

(4) JCQHC disclosure, and (5) hospital self-disclosure of information—that is, voluntary

disclosure by the hospital by, for example, a pamphlet or a notice on all billboards in

the hospital. Feedback from participating hospitals, highlighted that most hospitals

(60%) perceive disclosure as good for consumers and hospitals; with most hospitals who

disclosed their reports to the JCQHC (80.5%) agreeing that “disclosure provides incentives

for improving the quality of care because consumers in the community read accreditation

reports”.

A total of 508 (93%) of the participating hospitals disclosed their accreditation reports on

the JCQHC website. Public hospitals were signiﬁcantly more committed to public disclosure

than private hospitals, and larger hospitals were signiﬁcantly more likely to participate in

public disclosure than smaller hospitals. Accreditation scores were positively related to

the public disclosure of hospital accreditation reports. Scores for patient focused care and

efforts to meet community needs were signiﬁcantly higher in actively disclosing hospitals

than in non-disclosing hospitals. Among the large hospitals, scores for safety management

were signiﬁcantly higher in hospitals advocating disclosure than in non-disclosing hospitals.

Most hospitals who agreed to disclosure by the JCQHC (410/508 – 80.7%) reported that

their public disclosure was helpful. A total of 489 of the 547 respondents (89.4%) indicated

that they also disclosed their accreditation reports themselves: 366 disclosed only their

accreditation status and 123 disclosed more than this. The study found that signiﬁcantly

more of the hospitals who agreed to disclosure of their report by the JCQHC also released

information than those who were not in favour of disclosure by the JCQHC.

The study ﬁndings suggest that public disclosure of accreditation reports should be

encouraged to improve public accountability and the quality of care. The authors highlighted

that there is a need for further research to explore the interaction between public

disclosure, processes and outcomes

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

5.4 Quality Assurance

External evaluation organisations need to be able to demonstrate their integrity,

objectivity and reliability. Mechanisms include:

The programme’s standards, survey processes and criteria for accreditation or

certiﬁcation awards are made publicly available

Surveyors are selected, trained and evaluated against explicit published criteria

Survey teams are tailored to each individual client organisation, according to

published criteria, to avoid any conﬂict of interest

The survey team reports initial ﬁndings back to the client organisation before

leaving the site, especially in relation to those likely to generate recommendations,

in order to check the observation and to ensure there are no surprises later

Team reports are prepared and agreed jointly and in compliance with procedures

which are often deﬁned in a surveyors’ handbook

Team reports are independently checked within the external evaluation

organisation for content, consistency and compliance with procedures

Final draft reports are referred to the client organisation for veriﬁcation before the

accreditation or certiﬁcation decision

Accreditation or certiﬁcation awards are made by a panel or staff independent of

the process, based on the team’s report and in line with deﬁned decision-making

criteria or formulae, not by the team itself

The ﬁnal chapter will look at evaluation systems that need to be established.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Chapter 6: Evaluating

systems and achievements

External evaluation organisations need to set an example of quality improvement within their

own organisation. This includes deﬁning, monitoring and improving their own performance. This

chapter outlines some of the mechanisms which external evaluation organisations can employ

to do this.

6.1 Measuring performance internally

Internal audits, indicators and quality improvement projects will form part of the overall

quality framework of the organisation.

Indicator data routinely collected by external evaluation organisations and reported to

governing boards include:

Recruitment, drop-out of participating organisations

Denial rate (proportion of organisations refused accreditation or certiﬁcation)

Report turnaround times (from survey date to ﬁnal report or to award decision)

Financial performance, such as actual against budget and various ﬁnancial ratios

Website hits

Surveyor recruitment, training and evaluation

Client satisfaction with surveyors, education services, the survey process, the

survey visit and other products provided by the programme

Staff satisfaction

Surveyor satisfaction

Satisfaction of other stakeholders.

The ISQua organisation standards require the external evaluation organisation to

evaluate the performance of various functions (such as governance, human resources

management, surveyor and survey management and accreditation or certiﬁcation

processes and outcomes), by collecting data on deﬁned indicators and other measures of

performance, analysing it, making improvements and evaluating achievements

External evaluation organisations typically undertake many development and

improvement initiatives. These need to be treated as quality projects and the objectives,

actions, timeframes, responsibilities, progress and results documented. These project

documents will form an important part of the evidence needed when the organisation

undergoes its own external evaluation survey through ISQua.

Audits need to be scheduled, results documented and actions taken as a result recorded

and evaluated. Audits can address a number of areas; for example, audits can be

conducted of staff, surveyor and client records; award decisions; health and safety; and

the complaints register.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

6.2 Evaluating independently

Independent evaluations of new accreditation organisations have been commissioned,

often by governments or as conditions of receiving initial development funding. Examples

from Australia, South Africa, Zambia and the UK document beneﬁts perceived by

organisations and their users, but include little data on individual or population health

improvements

A WHO study of external quality assessment programmes for maternal and child health

concluded in 2002 that these bring beneﬁts to clients, the community, staff and the

service, summarised as

The linkages, networks and structures which have been developed and / or

improved to inﬂuence the political, legislative, economic, socio-cultural and public

health environment within which services operate (enabling mechanisms)

The reorganisation and / or development of the healthcare delivery systems at the

service level

The change in attitude and / or development of skills and knowledge of health

service staff

Improvements to health facilities and equipment

A client-centred and clients’ rights approach to healthcare whereby services

consult with and support clients, are needs based and able to deliver better care to

clients and the community.

6.3 Monitoring by regulatory agencies

Some regulatory bodies, e.g. in USA and Canada, monitor independent accreditation

programmes, primarily by representation on the governing board or by checks on

selected surveys. The federal government follow The Joint Commission into 5% of

surveys in “deemed status” hospitals within a few weeks of the visit to validate reports;

the National Committee for Quality Assurance (NCQA) in USA has a proportion of co-

visits; and the Accreditation Association for Ambulatory Health Care (AAAHC) has a

similar proportion of post-accreditation validation surveys of ambulatory care centres.

In South Africa, the provincial government, which is also the contractor, provides

monitoring by co-visiting. In New Zealand, the Ministry of Health arranges monitoring

audits of 5% of all certiﬁcation audits undertaken by independent designated audit

agencies

6.4 Accrediting the external evaluation bodies

The International Society for Quality in Health Care’s (ISQua’s) International Accreditation

Programme has been in existence since 1999 and “accredits the accreditors”. The

scope of the programme has been extended from the evaluation of national healthcare

accreditation organisations, their standards and surveyor training, to include other

standards based certiﬁcation and audit organisations.

The International Accreditation Programme (IAP) provides three products for health and

social care external evaluation bodies:

Survey and accreditation to international standards for external evaluation

organisations

Standards assessment and accreditation to international principles for healthcare

and social care standards

Assessment and accreditation of surveyor training programmes.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

The international standards for external evaluation organisations are the outcome of

several years of development, testing, peer review and consultation with the international

accreditation community. They were designed to address the quality of all aspects and

functions of an accreditation body, broadly incorporating the International Standards

Organisation (ISO) requirements for certiﬁcation bodies, the Baldrige criteria for

performance excellence, and criteria for organisational excellence from the accreditation

standards of a number of national accreditation bodies. These standards assess the

key business functions as well as best practice in assessment methodologies, surveyor

management and award recognition.

The standards and principles and their criteria are intended to guide external evaluation

organisations in their development by identifying best practice processes and systems

and providing an assessment process and recognition system for achievement of these.

Many smaller and developing programmes cannot justify the resources required for full

international recognition but they could embark on a deﬁned progression of development

and standardisation starting from self-assessment, to peer review, and aiming eventually

for international accreditation.

ISQua provides technical and advisory services such as self-assessment review and

mock surveys to assist external evaluation organisations develop their programmes and

prepare for international accreditation.

ISQua requires at least one set of the organisation’s standards to be ISQua accredited

before the organisation can enter the organisation accreditation programme.

Conclusions

This document has aimed to highlight some of the questions, issues and challenges which need

to be addressed before deciding on and implementing an external evaluation programme. The

decisions made must be speciﬁc to the values, health policies or strategies and organisations of

individual countries, regions and care sectors. Steps have been identiﬁed that need to be taken

to ensure that the foundation is set for a sustainable organisation. The order may be different,

but the fundamentals must be established ﬁrst. Some steps may be done in parallel, for

example obtaining funding, negotiating incentives and developing standards, or establishing the

governance framework and management systems.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

References

1 Shaw C. Toolkit for Accreditation Programs. Melbourne: The International Society for

Quality in Healthcare, 2004.

2 International Society for Quality in Health Care. ISQua Checklist for Development of New

Healthcare Accreditation Programs: Guidance for governments, agencies and other

groups. Melbourne: The International Society for Quality in Healthcare, 2006.

3 International Society for Quality in Health Care. Guidelines and Standards for External

Evaluation Organisations 4th Edition Version 1.1 July 2014. Dublin: International Society for

Quality in Health Care, 2014.

4 International Society for Quality in Health Care. Guidelines and Principles for the

Development of Health and Social Care Standards 4th Edition Version 1.1 July 2014. Dublin:

International Society for Quality in Health Care, 2014.

5 International Society for Quality in Health Care. Surveyor Training Standards Programme.

Dublin: International Society for Quality in Health Care, 2009.

6 World Health Organization. Universal Health Coverage [Internet]. Geneva: World Health

Organization. Available from: http://www.who.int/universal_health_coverage/en/

7 Braithwaite J, Shaw C.D, Molodovan M, Greenﬁeld D, Hinchcliff R, Mumford V, Kristensen

MB, Westbrook J, Nicklin W, Fortune T and Whittaker S. Comparison of health service

accreditation programs in low- and middle- income countries with those in higher income

countries: a cross-sectional study. International Journal for Quality in Health Care. 2012; 24

(6): 568-577.

8 Rooney AL and van Ostenberg PR. Licensure, Accreditation and Certiﬁcation: Approaches

to Health Services Quality. Quality Assurance Methodology Reﬁnement Series. Bethesda:

Center for Human Services, Quality Assurance Project; 1999.

9 Australian Commission on Safety and Quality in Health Care. Accreditation and the NSQHS

Standards [Internet]. Sydney: Australian Commission on Safety and Quality in Health Care.

Available from: http://www.safetyandquality.gov.au/our-work/accreditation-and-the-nsqhs-

standards/

10 International Organization for Standardization (ISO). About ISO [Internet]. Geneva:

International Organization for Standardization. Available from: http://www.iso.org/iso/

home/about.htm

11 Nicklin W. The Value and Impact of Health Care Accreditation: A Literature Review. Ottawa:

Accreditation Canada; 2014. Available from: http://www.accreditation.ca/sites/default/ﬁles/

value-and-impact-en.pdf

12 Hinchcliff R, Greenﬁeld D, Moldovan M, Westbrook JI, Pawsey M, Mumford V and

Braithwaite J. Narrative synthesis of health service accreditation literature. BMJ Qual Saf.

2012; 21:979-991.

13 Mumford V, Forde K, Greenﬁeld D, Hinchcliff R and Braithwaite J. Health services

accreditation: what is the evidence that the beneﬁts justify the costs? International Journal

for Quality in Health Care. 2013; 25 (5): 606-620.

14 Beaumont M. Recherche sur l’efﬁcacite du programme d’agrement du Conseil canadien

d’agement des services de sante: Methodologie et resultants. Maitrise en administration

des services de sante: Faculte de medicine, Universite de Montreal; 2002.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

15 Shaw C.D, Braithwaite J, Moldovan M, Nicklin W, Grgic I, Fortune T and Whittaker S.

Proﬁling health-care accreditation organizations: an international survey. International

Journal for Quality in Health Care. 2013; 25 (3): 222-231.

16 Burnett S, Renz A, Wiig S, Fernandes A, Weggelaar AM, Calltorp J, Anderson J.E., Robert G,

Vincent C and Fulop N. Prospects for comparing European hospitals in terms of quality and

safety: lessons from a comparative study in ﬁve countries. International Journal for Quality in

Health Care. 2013; 25 (1): 1-7.

17 The World Bank. Health Systems Regulation [Internet]. Washington D.C.: The World Bank.

Available from: http://www.worldbank.org

18 Al-Awa B, De Wever A, Melot C and Devreux I. An overview of patient safety and

accreditation: a literature review study. RJMS. 2011; 5: 200-223.

19 Turner L. Quality in healthcare and globalisation of health services: accreditation and

regulatory oversight of medical tourism companies. International Journal for Quality in

Health Care. 2011; 23(1): 1-7

20 Smits H, Supachutikul A and Mate KS. Hospital accreditation: lessons from low- and

middle-income countries. Globalization and Health. 2014; 10 (65).

21 Australian Government Department of Human Services. Practice Incentive Program

Guidelines. Adelaide: Australian Government Department of Human Services; 2013.

Available from: http://www.humanservices.gov.au/healthprofessionals/services/practice-

incentives-programme/

22 Australian Government Department of Health. Budget 2015 – 2016 Portfolio Budget

Statements 2015 -16 Budget Related Paper No. 1.10 Health Portfolio. Canberra: Australian

Government Department of Health; 2015. p96. Available from: http://www.health.gov.au/

internet/budget/publishing.nsf/Content/2015-2016_Health_PBS_sup1/$File/2015-16_

Health_PBS_0.0_Complete.pdf

23 Australian Government Department of Human Services. Practice Incentives Program

[Internet]. Adelaide: Australian Government Department of Human Services; (undated).

Available from: http://www.humanservices.gov.au/health-professionals/services/practice-

incentives-programme/

24 Malaysian Society for Quality in Health. MSQH – Committee Members [Internet]. Kuala

Lumpar: Malaysian Society for Quality in Health. Available from: http://www.msqh.com.my/

msqh/ct-menu-item-3/2013-07-29-06-13-01/ct-menu-item-9

25 Shaw CD. Developing hospital accreditation in Europe. Copenhagen: WHO Regional Ofﬁce

for Europe; 2004. Available from: http://www.unifesp.br/hsp/acred.pdf

26 National Accreditation Board for Hospitals & Healthcare Providers. Introduction – Hospital

Entry Level [Internet]. New Delhi: National Accreditation Board for Hospitals & Healthcare

Providers. Available from: http://nabh.co/Hospital-EntryLevel.aspx

27 Greenﬁeld D, Pawsey M, Hinchcliff R, Moldovan M and Braithwaite J. The standard of

healthcare accreditation standards: a review of empirical research underpinning their

development and impact. BMC Public Health Services Research. 2012; 12: 329.

28 Institute of Medicine (IOM). Crossing the Quality Chasm: A New Health System for the 21st

Century. Washington D.C.: National Academy Press, 2001. Available from: http://www.iom.

edu/Reports/2001/Crossing-the-Quality-Chasm-A-New-Health-System-for-the-21st-

Century.aspx

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

29 Ito H and Sugawara H. Relationship between accreditation scores and the public disclosure

of accreditation reports: a cross sectional study. Qual Saf Health Care. 2005; 14 (2): 87-92.

30 Mahaffey A. In Pursuit of Quality Improvement: A review of experiences of external quality

assessments at reproductive health services. Geneva: World Health Organization, 2003.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Bibliography

Alkhenizan A and Shaw C. Impact of accreditation on the Quality of Healthcare Services: a

systematic review of the literature. Ann Saudi Med. 2011; 31 (4): 407-416.

Al-Awa B, De Wever A, Melot C and Devreux I. An Overview of Patient Safety and Accreditation: A

Literature Review Study. RJMS. 2011; 5: 200-223.

Al Tehewy M, Salem B, Habil I, and El Okda S. Evaluation of accreditation program in non-

governmental organizations’ health units in Egypt: Short-term outcomes. International Journal

for Quality in Health Care. 2009; 21 (3): 183-189.

Auras A and Geraedts M. Patient experience data in practice accreditation - an international

comparison. International Journal for Quality in Health Care. 2011; 22 (2); 132-139.

Braithwaite J, Greenﬁeld D, Westbrook J, Pawsey M, Westbrook M, Gibberd R, Naylor J, Nathan

S, Robinson M, Runciman B, Jackson M, Travaglia J, Johnston B, Yen D, McDonald H, Low

L, Redman S, Johnson B, Corbett A, Hennessy D, Clark J and Lancaster J. Health service

accreditation as a predictor of clinical and organisational performance: a blinded, random,

stratiﬁed study. Qual Saf Health Care. 2010; 19: 14-21.

Braithwaite J, Westbrook J, Johnston B, Clark S, Brandon M, Banks M, Hughes C, Greenﬁeld

D, Pawsey M, Corbett A, Georgiou A, Callen J, Øvretveit J, Pope C, Suñol R, Shaw C, Debono D,

Westbrook M, Hinchcliff R, and Moldovan M. Strengthening organizational performance through

accreditation research: the ACCREDIT project. BMC Res Notes. 2011; 4: 390.

British Standards Institution. A standard for standards – Principles of standardization. BS 0

2011. London: British Standards Institution; 2011. Available from: http://www.iso.org/sites/PEG/

docs/PEG%20Documents/04_bs02011.pdf

Bukonda N, Tavrow P, Abdallah H, Hoffner K, and Tembo J. Implementing a national hospital

accreditation program: the Zambian experience. International Journal for Quality in Health Care.

2002; 14 (Supplement 1): 1-7.

Chandra A, Glickman SW, Ou F-S, Peacock WF, McCord JK, Cairns CB, Peterson ED, Ohman EM,

Gibler WB, and Roe MT. An Analysis of Society of Chest Pain Centers Accreditation to American

College of Cardiology / American Heart Association Non-ST Segment Elevation Myocardial

Infarction Guidelines Adherence. Annals of Emergency Medicine. 2009; 54 (1): 17-25.

Chung K-P and Yu T-H. Are quality improvement methods a fashion for hospitals in Taiwan?

International Journal for Quality in Health Care. 2012; 24 (4): 371-379.

Cleveland EC, Dahn BT, Lincoln TM, Safer M, Podesta M and Bradley E. Introducing health

facility accreditation in Liberia. Global Public Health. 2014; 10 (65).

Devkaran S and O’Farrell PN. The impact of hospital accreditation on clinical documentation

compliance: a life cycle explanation using interrupted time series analysis. BMJ Open. 2014; 4:

e005240. doi: 10.1136/bmjopen-2014-005240

de Walcque C, Seuntjens B, Vermeyen K, Peeters G, and Vinck I. Comparative study of hospital

accreditation programs in Europe. Health Services Research (HSR). Brussels: Belgian Health

Care Knowledge Centre (KCE); 2008. KCE reports 70C, D/2008/10.273/03. Available from: https://

kce.fgov.be/sites/default/ﬁles/page_documents/d20081027303.pdf

El-Jardali F, Jamal D, Dimassi H, Ammar W, and Tchaghchaghian V. The impact of hospital

accreditation on quality of care: Perception of Lebanese nurses. International Journal for Quality

in Health Care. 2008; 20 (5): 363-371.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Fortes MT, de Mattos RA, and de Faria Baptista TW. Accreditation or accreditations? A

comparative study about accreditation in France, United Kingdom and Cataluña. Rev Assoc Med

Bras. 2011; 57 (2): 234-241.

Gratwohl A, Brand R, Niederwieser D, Baldomero H, Chabannon C, Cornelissen J, de Witte

T, Ljungman P, McDonald F, McGrath E, Passweg J, Peters C, Rocha V, Slaper-Cortenbach I,

Sureda A, Tichelli A, and Apperley J. Introduction of a Quality Management System and Outcome

after Hematopoietic Stem-Cell Transplantation. J Clin Oncol. 2011; 29 (15): 1980-1986.

Greenﬁeld D, Travaglia J, Braithwaite J, and Pawsey M. Unannounced Surveys and Tracer

Methodology: Literature Review. Sydney: Centre for Clinical Governance Research, University of

New South Wales, 2007.

Greenﬁeld D, Travaglia J, Braithwaite J, and Pawsey M. Intra-rater and Inter-rater Reliability in

Health Care Accreditation: Literature Review. Sydney: Centre for Clinical Governance Research,

University of New South Wales, 2007.

Greenﬁeld D and Braithwaite J. Health sector accreditation research: a systematic review.

International Journal for Quality in Health Care. 2008; 20 (3): 172-183.

Greenﬁeld D, Pawsey M, and Braithwaite J. What motivates professionals to engage in the

accreditation of healthcare organizations? International Journal for Quality in Health Care. 2011;

23 (1): 8-14.

Greenﬁeld D, Hinchcliff R, Westbrook M, Jones D, Low L, Johnston B, Banks M, Pawsey M,

Moldovan M, Westbrook J, and Braithwaite J. An empirical test of accreditation patient journey

surveys: randomized trial. International Journal for Quality in Health Care. 2012; 24 (5): 495-500.

Greenﬁeld D, Hinchcliff R, Moldovan M, Mumford V, Pawsey M, Westbrook JI, and Braithwaite

J. A multimethod research investigation of consumer involvement in Australian health service

accreditation programmes: the ACCREDIT-SCI study protocol. BMJ Open. 2012; 2: e002024. doi:

10.1136/bmjopen-2012-002024

Greenﬁeld D, Moldovan M, Westbrook M, Jones D, Low L, Johnston B, Clark S, Banks M, Pawsey

M, Hinchcliff R, Westbrook J, and Braithwaite J. An empirical test of short notice surveys in two

accreditation programmes. International Journal for Quality in Health Care. 2012; 24 (1): 65-71.

Greenﬁeld D, Hinchcliff R, Pawsey M, Westbrook J, and Braithwaite J. The public disclosure of

accreditation information in Australia: stakeholder perceptions of opportunities and challenges.

Health Policy. 2013; 113 (1-2): 151-159.

Greenﬁeld D, Pawsey M, Naylor J, and Braithwaite J. Researching the reliability of accreditation

survey teams: Lessons learnt when things went awry. Health Information Management Journal.

2013; 42 (1): 4-10.

Greenﬁeld D, Civil M, Donnison A, Hogden A, Hinchcliff R, Westbrook J, and Braithwaite J. A

mechanism for revising accreditation standards: A study of the process, resources required and

evaluation outcomes. BMC Health Services Research. 2014; 14: 571.

Greenﬁeld D, Kellner A, Townsend K, Wilkinson A, and Lawrence SA. Health service

accreditation reinforces a mindset of high-performance human resource management: Lessons

from an Australian study. International Journal for Quality in Health Care. 2014; 26 (4): 372-377.

Greenﬁeld D, Hinchcliff R, Hogden A, Mumford V, Debono D, Pawsey M, Westbrook J, and

Braithwaite J. A hybrid health service accreditation program model incorporating mandated

standards and continuous improvement: Interview study of multiple stakeholders in Australian

healthcare. Int J Health Plann Mgmt. 2015. doi: 10.1002/hpm.2301

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Guérin S, Le Pogam M-A, Robillard B, Le Vaillant M, Lucet B, Gardel C, Grenier C, and Loira

P. Can we simplify the hospital accreditation process? Predicting accreditation decisions

from a reduced dataset of focus priority standards and quality indicators: results of predictive

modelling. BMJ Open. 2013; 3: e003289. doi: 10.1136/bmjopen-2013-003289.

Gyani GJ and Krishnamurthy B. The National Accreditation Board for Hospital and Health Care

Providers Accreditation Programme in India. World Hospitals and Health Services. 2014; 50 (1):

9-12.

Hinchcliff R, Greenﬁeld D, Westbrook JI, Pawsey M, Mumford V and Braithwaite J. Stakeholder

perspectives on implementing accreditation programs: a qualitative study of enabling factors.

BMC Health Services Research. 2013; 13 (437).

Jaafaripooyan E, Agrizzi D, and Akbari-Haghighi F. Healthcare accreditation systems: further

perspectives on performance measures. International Journal for Quality in Health Care. 2011; 23

(6): 645-656.

Jaafaripooyan E. Potential pros and cons of external healthcare performance evaluation

systems: real-life perspectives on Iranian hospital evaluation and accreditation program. Int J

Health Policy Manag. 2014; 3 (4): 191-198.

Leatherman S, Ferris T, Berwick D, Omaswa F, and Crisp N. The role of quality improvement in

strengthening health systems in developing countries. International Journal for Quality in Health

Care. 2010; 22 (4): 237-243.

Lind A, Edward A, Bonhoure P, Mustafa L, Hansen P, Burnham G, and Peters DH. Quality of

outpatient hospital care for children under 5 years in Afghanistan. International Journal for

Quality in Health Care. 2011; 23 (2): 108-116.

Menachemi N, Chukmaitov A, Brown LS, Saunders C, and Brooks RG. Quality of care in

accredited and nonaccredited ambulatory surgical centers. The Joint Commission Journal on

Quality and Patient Safety. 2008; 34 (9): 546-551.

Mumford V, Greenﬁeld D, Hinchcliff R, Moldovan M, Forde K, Westbrook JI, and Braithwaite

J. Economic evaluation of Australian acute care accreditation (ACCREDIT-CBA (Acute)): study

protocol for a mixed method research project. BMJ Open. 2013; 3: e002381. doi:10.1136/

bmjopen-2012-002381

Mumford V, Greenﬁeld D, Hogden A, Debono D, Gospodarevskaya E, Forde K, et al. Disentangling

quality and safety indicator data: A longitudinal, comparative study of hand hygiene compliance

and accreditation outcomes in 96 Australian hospitals. BMJ Open. 2014; 4: e005284. doi: 10.1136/

bmjopen-2014-005284

National Advisory Group on the Safety of Patients in England. A promise to learn – a

commitment to act: Improving the safety of patients in England. 2013. Available from: https://

www.gov.uk/government/uploads/system/uploads/attachment_data/ﬁle/226703/Berwick_

Report.pdf

Pomey MP, Lemieux-Charles L, Champagne F, Angus D, Shabah A and Contandriopoulos AP.

Does accreditation stimulate change? A study of the impact of the accreditation process on

Canadian healthcare organizations. Implement Sci. 2010; 5 (31). doi: 10.1186/1748-5908-5-31

Ruelas E, Gómez-Dantés O, Leatherman S, Fortune T, and Gay-Molina JG. Strengthening the

quality agenda in health care in low- and middle-income countries: questions to consider.

International Journal for Quality in Health Care. 2012; 24 (6): 553-557.

Sack C, Scherag A, Lütkes P, Günther W, Jöckel K-H, and Holtmann G. Is there an association

between hospital accreditation and patient satisfaction with hospital care? A survey of 3700

patients treated by 73 hospitals. International Journal for Quality in Health Care. 2011; 23 (3): 1-6.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Sack C, Lütkes P, Günther W, Jöckel K-H, and Holtmann GJ. Challenging the holy grail of

hospital accreditation: A cross sectional study of inpatient satisfaction in the ﬁeld of cardiology.

BMC Health Services Research. 2010; 10: 120.

Sax S and Marx M. Local perceptions on factors inﬂuencing the introduction of international

healthcare accreditation in Pakistan. Health Policy Plan [Internet]. 2013 November [cited 2014

June 11]; doi:10.1093/heapol/czt084

Shaw C, Bruneau C, Kutryba B, De Jongh G, and Suñol R. Towards hospital standardization in

Europe. International Journal for Quality in Health Care. 2010; 22 (4): 244-249.

Shaw C, Groene O, Botje D, Sunol R, Kutryba B, Klazinga N, Bruneau C, Hammer A, Wang A,

Arah OA and Wagner C. The effect of certiﬁcation and accreditation on quality management in 4

clinical services in 73 European hospitals. International Journal for Quality in Health Care. 2014;

26: Number S1: 100-107.

Shaw C, Groene O, Mora N, and Sunol R. Accreditation and ISO certiﬁcation: do they explain

differences in quality management in European hospitals? International Journal for Quality in

Health Care. 2010; 22 (6): 445 – 451.

Shaw C, Kutryba B, Braithwaite J, Bedlicki M, and Warunek A. Sustainable healthcare

accreditation: messages from Europe in 2009. International Journal for Quality in Health Care.

2010; 22 (5): 341-350.

Siddiqi S, Elasady R, Korshid I, Fortune T, Leotsakos A, Letaief M, Qsoos S, Aman R, Mandhari A,

Sahel A, El-Tehewy M, and Abdellatif A. Patient Safety Friendly Hospital Initiative: from evidence

to action in seven developing country hospitals. International Journal for Quality in Health Care.

2012; 24 (2): 144-151.

Tabrizi JS, Gharibi F and Wilson AJ. Advantages and Disadvantages of Health Care Accreditation

Models. Health Promotion Perspectives. 2011; 1 (1): 1-31.

The Mid Staffordshire NHS Foundation Trust Public Inquiry. Report of the Mid Staffordshire NHS

Foundation Trust Public Inquiry. London: The Stationery Ofﬁce; 2013. Available from: http://www.

midstaffspublicinquiry.com/report

Touati N and Pomey MP. Accreditation at a crossroads: Are we on the right track? Health Policy.

2009; 90 (2-3): 156-165.

University Research Co., LLC. Jordan Healthcare Accreditation Project Final Report June 17,

2007 – March 17, 2013. Bethesda: University Research Co., LLC, 2013.

Webster TR, Mantopoulos J, Jackson E, Cole-Lewis H, Kidane L, Kebede S, Abebe Y, Lawson

R, and Bradley EH. A brief questionnaire for assessing patient healthcare experiences in low

income settings. International Journal for Quality in Health Care. 2011; 23 (3): 258-268.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Useful web resources

The International Society for Quality in Health Care (ISQua) is not responsible for external

website content. Please note that many organisations have English language content on their

websites and where possible the direct link to such material is provided. However, in some

instances the website content is only available in the native language.

Accreditation Canada http://accreditation.ca/

Agency for Quality and Accreditation in

Health and Social Welfare, Croatia

http://aaz.hr/

American Accreditation Council http://www.americanaccreditationcouncil.

com/

American Association for Accreditation of

Ambulatory Surgery Facilities International

http://www.aaaasﬁ.org/

American Association of Blood Banks http://www.aabb.org/

Australian Aged Care Quality Agency http://www.aacqa.gov.au/

Australian Commission on Safety and Quality

in Health Care

http://www.safetyandquality.gov.au/

Australian General Practice Accreditation Ltd

(AGPAL)

http://www.agpal.com.au/

Canadian Accreditation Council http://www.cacohs.com/

CHKS, United Kingdom http://www.chks.co.uk/

Consortium for Brazilian Accreditation (CBA) http://www.cbacred.org.br/

DAA Group Ltd http://www.daagroup.co.nz/

DNV GL Business Assurance http://www.dnvba.com/

Global-Mark Pty Ltd http://www.global-mark.com.au/

Haute Authorité de Santé, France http://www.has-sante.fr/portail/

jcms/r_1455134/fr/about-has

Health Accreditation Service, Columbia http://www.icontec.org/

Health and Disability Auditing New Zealand

Ltd (HDANZ)

http://www.healthaudit.co.nz/

Health and Disability Auditing Australia Pty

Ltd

http://www.hdaau.com.au/

Health Care Accreditation Council, Jordan http://www.hcac.jo/

IKAS, The Danish Institute for Quality and

Accreditation in Healthcare

http://www.ikas.dk/IKAS/English.aspx

Japan Council for Quality Health Care http://jcqhc.or.jp/pdf/top/english.pdf

Joint Commission International http://www.jointcommissioninternational.

org/

Joint Commission of Taiwan http://www.tjcha.org.tw/FrontStage/aboutus_

en.html

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Malaysian Society for Quality in Health http://www.msqh.com.my/

Ministry of Health New Zealand – Health and

Disability Services Standards

http://www.health.govt.nz/our-work/

regulation-health-and-disability-system/

certiﬁcation-health-care-services/health-

and-disability-services-standards

National Accreditation Board for Hospitals

and Healthcare Providers, India

http://www.nabh.co/

Quality Innovation Performance, Australia http://www.qip.com.au/

Joint Commission of Taiwan http://www.tjcha.org.tw/FrontStage/aboutus_

en.html

The Australian Council on Healthcare

Standards

http://www.achs.org.au/

The Healthcare Accreditation Institute (Public

Organisation), Thailand

http://www.ha.or.th/

The Council for Health Service Accreditation

of Southern Africa

http://www.cohsasa.co.za/

The Diagnostic Accreditation Program,

British Columbia, Canada

http://www.dap.org/

The Netherlands Institute for Accreditation in

Healthcare (NIAZ)

http://en.niaz.nl/

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Appendix 1 - Case Studies

Appendix 1a.

IKAS – Danish Institute for Quality and Accreditation in Healthcare

Country: Denmark

Contributed by: Carsten Engel

Foundation of the programme

The Danish accreditation programme (DDKM) was established as part of the “National Strategy

for Quality Development in the Healthcare System – Joint Goals and Action Plan 2002-2006”. The

strategy was developed by the national, regional and local political authorities in cooperation

with stakeholder organisations, representing professionals and consumers.

At that time, a number of hospitals already had positive experiences with accreditation provided

by international accreditors – one of the intentions of the strategy was to spread this to the

entire healthcare system, based on a Danish model.

IKAS is formally an independent organisation, but IKAS and DDKM were established by an

agreement between the regional and local political authorities, who are responsible for

delivering healthcare, and the national government that sets the overarching political priorities,

including the economic frame, and is the healthcare legislator and regulator.

The government provides part of the funding for IKAS. The government is represented on the

Board of IKAS; the Chair of the Board is a government representative (a director of the Danish

Health and Medicines Authority).

Development steps

The following steps describe the initial development of DDKM. The programme has since been

extensively developed, based on the experiences obtained.

1. Cooperation agreement between the government and the regions on the establishment of a

joint model for quality assessment, including provisions for the funding for DDKM (2004)

2. Appointment of a Board by the parties to the cooperation agreement and endorsement of

bylaws for IKAS

3. Establishment of IKAS as an organisation (2005)

4. Tender for consultancy by an established international accreditor, resulting in a contract

with HQS / CHKS for support to develop standards, establish IKAS as an accreditation

organisation, and train surveyors

5. Development of ﬁrst two sets of standards (hospitals and pharmacies) by theme groups

(for related groups of standards) of standard developers, consisting of senior professionals,

appointed by the Regions and the Association of Danish Pharmacies. IKAS and HQS / CHKS

served as advisors and secretariat for the groups.

6. Public hearing, which for the hospital standards resulted in an extended revision by an

editorial group with members from IKAS and the Regions, followed by a second hearing.

7. Pilot testing of standards for usability (for clients) and understandability

8. Submission of standards for ISQua accreditation

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

9. Development of an IT system to support implementation and external assessment

10. Development of a rating system

11. Development of information for hospitals and pharmacies and holding a series of courses

for key persons in client organisations

12. Development of a survey methodology, described in a handbook

13. Selection and training of surveyors

14. Appointment and training of an Accreditation Awards Panel

15. Development and implementation of processes to process survey reports

16. Preparation for ISQua accreditation as an external evaluation organisation and of the

surveyor training programme (obtained early 2011).

The standards actually led the development process; steps 9 – 15 overlapped each other and the

later phases of standard development, but continued up to the commencement of surveys, 1½

years after ﬁnalising the standards.

The ﬁrst survey was conducted 4½ years after the establishment of IKAS.

Funding & incentives

In terms of funding, IKAS has an index-linked annual grant from the central government,

regions and local government. There are no fees for public clients or pharmacies. Other private

clients pay a fee that covers direct expenses plus an overhead.

The programme is not required by any legislation, but is based on agreements as follows:

Public hospitals: all hospitals participate by agreement between National and Regional

governments

Private hospitals: voluntary, but participation is a prerequisite to obtain a contract to treat

patients for the regions (also required by some insurance companies)

Pharmacies: voluntary, ﬁnancial incentive in place

Municipalities (primary care services, including long-term care): voluntary, no incentives in

place

Ambulance operators: prerequisite to obtain contract with Regions

General practitioners: mandatory (with some minor exceptions) by agreement between the

Regions and the Organisation of General Practitioners in Denmark; ﬁnancial compensation

as part of the agreement.

Standards and measurement

IKAS has developed all standards used in its programmes. They were ﬁrst developed for

hospitals and for community pharmacies. Standards have since been developed for primary care

services, delivered by municipalities, and for ambulance services. Currently standards are being

developed for general practitioners and specialist physicians. Over the coming years, all health

care professions providing ofﬁce-based services, outside of hospitals, will be covered.

Compliance with standards is assessed by scoring a number of elements (for the hospital

standards roughly 450) according to a four point scale (Fully / Largely / Partially / Not Met),

where the two upper levels indicate a satisfactory performance (except for certain safety

critical standards, where only Fully Met is considered satisfactory). Any element not met to

satisfaction will require follow up, and if not corrected, results in accreditation with comments.

An Accreditation Award Panel decides, guided by certain rules, whether the nature and / or

amount of the comments preclude accreditation – if so, status as “not accredited” is awarded

and published.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Assessment methodology and focus

The assessment methodology used is external survey with extensive use of tracer methodology.

The focus is on exploring the implementation of safe processes and investigating the use of

quality data for improvement activities.

Quality improvement is fundamental. There is an extensive set of national quality registers

in Denmark, and one of the purposes of DDKM is to support and assess that data is not just

collected, but also used for quality improvement. Demonstration of completed and evaluated

improvement activities is required from the second accreditation cycle.

Surveyors are active senior health care professionals who are contracted for 15 survey days per

year. In addition, they are obligated to participate in continuous training activities.

Barriers

Development of an accreditation programme from scratch is much like building a bridge while

you are crossing it. Even with the best support from consultants, there are a lot of lessons to

be learned when the programme is applied in practice. A full pilot test, including complete

implementation and external assessment, would be ideal, but would add a considerable delay.

Lessons learned

One lesson learned is that while it adds to the legitimacy of the programme that standards

are developed involving a large number of healthcare professionals as standard developers, a

strong editorial process is needed if this is to result in a uniform and balanced standard set.

Furthermore, these types of standard developers will almost exclusively focus on the standards

as implementation guides; it may be a challenge to assess performance in a reliable and

uniform way. To support reliable assessment, the standards must include a lot of guidance for

surveyors, both as to methodology and to rating, while avoiding surveys becoming exercises of

“ticking check boxes”.

We have underestimated the need to communicate that the standards are different from

regulatory rules. The latter contain speciﬁc directions that must be strictly adhered to, whereas

many (albeit not all) standards express a goal to strive for or require the client to deﬁne the

speciﬁcs, according to local needs and priorities. You will meet clients asking to be told exactly

what to do, and you will meet examples of “over implementation”, where clients demand their

staff to rigidly apply the same standardised procedures to all patients; an example could be

hospitals believing that the standards require them to screen all patients for malnutrition,

regardless of the likelihood for a certain patient or type of patient to be malnourished. This is, in

our experience, an important source of resistance to accreditation among staff.

Our surveys are announced and are preceded by a lot of preparation by the clients. Many of

their staff perceive this as building a nice picture to show the surveyors, but not necessarily

giving a fair picture of the real performance; the risk is that preparing for accreditation is

seen by staff as a show, designed to obtain a certiﬁcate, more than as a value adding activity.

Doing unannounced or partially unannounced surveys would no doubt add to the face validity

of accreditation. We are currently preparing a controlled study to investigate the merits of

unannounced surveys.

One typical way to articulate resistance is to ask for the evidence for accreditation. While you

must argue that accreditation is a complex intervention that cannot be backed by evidence of the

same type as a drug treatment, design of a formal evaluation as part of the programme should

be considered.

More information, including accreditation standards, can be found at http://www.ikas.dk/IKAS/

English.aspx

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Appendix 1b.

Health Care Accreditation Council (HCAC)

Country: Jordan

Contributed by: Ed Chappy

Foundation of the programme

Several reasons were stated for why the programme was developed including to improve the

quality of hospitals and to enhance medical tourism. In addition, it was a response to public

complaints of poor quality of care and a need to improve the entire healthcare system in the

country.

The HCAC is the national healthcare accreditation agency of Jordan. The organisation sets

standards for hospitals, primary healthcare centres, family planning and reproductive health,

transport services (ambulances), cardiac care, and diabetes mellitus. HCAC surveys against the

standards and awards accreditation. HCAC also provides consultation and education to prepare

healthcare facilities for accreditation and offers certiﬁcation courses.

The HCAC is a private, not-for-proﬁt shareholding company registered under the Ministry of

Trade and Industry. The board of directors is made up of representatives for all healthcare

sectors in Jordan, medical and nursing professions, and education.

Development steps

1. Decision on funding and incentives

2. Standards or criteria development if applicable

3. Survey / Assessment management processes

4. Development of manuals, tools, education programmes for clients or others

5. Selection and training of surveyors / assessors

6. Type of proposed governance board and framework, constitution

7. Setting up of governance board, governance policies and procedures

8. Development of management systems, strategic and operational plans

9. Accreditation / Certiﬁcation processes

10. Monitoring, review and evaluation systems

11. Development and use of website, portal or other electronic aids

A decision was taken to develop standards, prepare 17 pilot hospitals from the public, private,

university, and military sectors for accreditation and then create the agency based on demand

for accreditation.

The ﬁrst set of hospital standards were developed in 2005, surveyors trained in 2006 and

the agency (HCAC) established in December 2007. The ﬁrst hospital accredited using HCAC

standards was in March 2008.

The ﬁrst services developed were consultation and education services to prepare hospitals

for accreditation and mock and accreditation surveys. Then preparation of primary healthcare

centres to meet standards and mock and accreditation surreys for them were added. Later,

local and regional consultation and education surveys and certiﬁcation courses for infection

prevention staff, risk managers, and quality improvement coordinators were added.

Funding and incentives

The original funding to develop the HCAC came through the Jordan Healthcare Accreditation

project funded by the United States Agency for International Development (USAID) and grants.

Since March 2013, HCAC has been ﬁnancially sustainable through charging fees for services

offered including surveys, education and consultation.

Accreditation is voluntary. There are no incentives (laws, regulation, insurance requirements) in

the country for accreditation.

Standards and measurement

All the standards are developed in Jordan. No standards developed by other organisations are

used. Hospital standards were developed ﬁrst, then standards for primary care centres, family

planning and reproductive health, transport services (ambulances), cardiac care, and diabetes

mellitus.

Standards are classiﬁed as critical, core and stretch.100% of critical standards must be met;

and a speciﬁed percentage of both core and stretch standards must be met in order for a service

to be accredited.

Assessment methodology & focus

Mock and accreditation surveys are used. The focus is on quality improvement.

Surveyors are certiﬁed for two years and are paid per survey. Staff are trained as surveyors but

are only used in emergencies when a surveyor is ill or for other reasons cannot do a survey.

Challenges

The main challenge was deciding where the organisation was going to be placed in the country

– Ministry of Health, other government agency, professional association, or as an independent

company. The second challenge was to determine how it would be funded.

Lessons learned

Every country must develop their system based on their needs and goals.

See what other countries are doing but create your own system.

Many activities can be done in parallel and you do not need to wait until one task is done

before proceeding to the next (do not have to wait for the agency to be developed before

standards are developed).

Recognise that accreditation is a business and look at the agency as any other business with

strategic, business, and operational plans and business processes.

Do not neglect the need to market accreditation to the population as well as healthcare

facilities and professionals.

Partner with clients and maintain a relationship after and between accreditations.

Look at accreditation as a means of improving the entire healthcare system, not just

hospitals.

See accreditation as one means to quality, not the only means.

Always seek ways to do things better, which may be different from what everyone else is

doing.

ISQua Accreditation International Accreditation Programme (IAP)

Appendix 1c.

Health and Disability Auditing New Zealand Ltd (HDANZ)

Country: New Zealand

Contributed by: Jim duRose

Foundation of the programme

The commencement of the Health and Disability Services (Safety) Act on 1 July 2002 represented

a signiﬁcant change in the regulatory environment in the New Zealand health and disability

sector. This Act replaced several previous pieces of legislation and changed the way in which

residential and hospital services were licensed or registered. In addition, the Act introduced

health and disability standards for hospitals, rest homes and residential disability services

aimed at improving safety levels and quality of care that became mandatory from 1 October

2004. The Act required that designated audit agencies (DAAs) are approved by the Director

General of Health for the purpose of auditing these services to those standards.

HDANZ became designated in October 2002. In 2004 3rd party accreditation was with

International Accreditation New Zealand (IANZ). Due to a change in IANZ’s legislation they

could no longer accredit HDANZ and in December 2008 HDANZ decided to proceed with ISQua

accreditation. The objective was to have a seamless transition from IANZ and this was achieved

by August 2009. Also, as of December 2008, the Ministry of Health did not require 3rd party

accreditation but a few months later this became a requirement to maintain designation.

HDANZ’s scope was determined by the Safety Act – the assessment of standards is a legal

requirement for public and private hospitals, rest homes and residential disability services.

Standards New Zealand (SNZ) is responsible for the New Zealand standards and this includes

others such as for Home Support, Allied Health, and Day surgery procedures.

HDANZ is also 3rd party accredited with ISQua in order to audit and certify services to these

standards.

HDANZ is a private, independently owned company. It is linked to the government as a MoH

approved designated auditing agency and for these services, HDANZ submits the audit report to

the MoH who issues the certiﬁcate

Development steps

1. Type of proposed governance board and framework, constitution

2. Decision on funding and incentives

3. Development of management systems, strategic and operational plans

4. Setting up of governance board, governance policies and procedures

5. Survey / Assessment management processes

6. Accreditation / Certiﬁcation processes

7. Selection and training of surveyors / assessors

8. Monitoring, review and evaluation systems

9. Development of manuals, tools, education programmes for clients or others

10. Development and use of website, portal or other electronic aids – HDANZ had a website

early on but web based assessment tools were introduced in 2008.

The ﬁrst assessment was undertaken approximately 6-8 months after HDANZ was established.

International Accreditation Programme (IAP) ISQua Accreditation

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

There was no trial period but pre-audit “gap analysis” work was commonplace for most services

before they completed their ﬁrst assessment in 2003 / 04.

At ﬁrst, HDANZ provided assessment services for all services under the Safety Act: rest homes,

geriatric hospitals, maternity, surgical, hospice, mental health, disability services and addiction

services. These continue but also HDANZ certiﬁes home care, allied health / physiotherapy

services, day surgery / ofﬁce-based services and community services. HDANZ also completes

funder contract auditing with NGO providers for a wide range of personal health and mental

health and addiction services. General practice reviews are completed on behalf of Primary Care

Organisations (PHO). HDANZ also assists the Royal College of General Practitioners (RNZCGP)

with their Cornerstone general practice accreditation programme by independently reviewing

reports and issuing a recommendation for accreditation.

Funding & incentives

Service providers pay fees to HDANZ for survey and monitoring visits. Certiﬁcation has been

mandatory for the MoH Safety Act since October 2002. From September 2005, it became

mandatory for physiotherapy services if they wanted a special contract from the Accident

Compensation Corporation (ACC). From September 2012, health funders made it mandatory

for Home Support providers. From March 2013, Southern Cross Health Society insurance made

certiﬁcation mandatory for their afﬁliated providers.

Standards and measurement

Standards New Zealand is responsible for the standards. In 2003, the main standards were

Health and Disability Sector Standards and this includes Infection Control and Restraint

Minimisation. These were updated in 2008. In 2003, Home and Community Support Standards

were issued by SNZ and these were updated in 2012. In 2005, Allied Health Standards and Day

stay surgery standards were issued by SNZ.

The rating scale is:

CI = Continuous improvement

FA = Fully Attained

PA = Partially attained

UA = Unattained

The Ministry of Health uses the assessment ratings to determine certiﬁcation. The length

of certiﬁcation can vary from one to four years depending on the level of achievement of the

standards.

Assessment methodology & focus

Audit teams are formed for on-site visits and reporting to the relevant standards. This includes

documentation, observation, client records sampling, tracer methodology, and interviewing of

staff, management, clients and family.

Quality improvement is the focus and at the same time the provider has to have achieved the

standards being assessed, noting that areas identiﬁed for further work (PA / UA ratings) have to

have progress reported and are reviewed at the surveillance audit.

Assessors are paid per event and in addition to the two operational company Directors who

audit there is one employee auditor. HDANZ maintains two separate auditor networks; one

is for DAA / other services which includes about 20 assessors and is a mix of lead, clinical,

consumers, technical experts, cultural and ﬁnancial auditors and the other is for Physiotherapy

services with an auditor network of 8 auditors.

International Accreditation Programme (IAP) ISQua Accreditation

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

Challenges

In October 2002 there were 10 DAAs and all but HDANZ had a formal status in either ISQua

health accreditation at the time or non-health ISO certiﬁcation.

Development of HDANZ’s services and the infrastructure to deliver a range of audits.

Setting up the quality management system.

Lessons learned

Early investment in a customer relationship database was very important and then later

improved at identifying sub-groups for marketing and other information.

The two key drivers for this business are a) operational efﬁciency with competent

administration staff and b) assessor competency.

Added value for governance and robust organisational management from maintaining a 3rd

party accreditation status.

Sound decision as growth occurred to structure into programmes.

Costs need to be closely monitored and managed as they can easily escalate otherwise.

Outsourcing the ﬁnancials in 2009 was a positive decision.

Maintaining NZQA auditor training course approval for credibility and HDANZ purpose

despite not being a revenue generator.

2008 investment into a marketing course reaped substantial dividends.

To be perceived as the expert.

Board / governance development in hindsight could have been more of a priority earlier on.

Appendix 1d.

Practice Incentive Program (PIP)

Country: Australia

Contributed by: Steve Clark

The Australian Government introduced the Practice Incentive Program (PIP) in 1998. The PIP is

aimed at supporting general practice activities that encourage continuing improvements and

quality care, enhance capacity and improve access and health outcomes for patients

In the 2015-16 Australian Government Budget, in excess of $1.5bn over four years

was

allocated to the PIP to support the continuation of incentive payments to general practices.

The PIP is used as a lever by government to inﬂuence behavioural change within the general

practice environment. To access payments under the PIP, practices must meet the eligibility

requirements, including that a practice must be accredited or registered for accreditation

against the Royal Australian College of General Practitioners (RACGP) Standards for general

practices and must maintain full accreditation.

Approximately 80% of all practices that meet the RACGP deﬁnition of a general practice

participate in accreditation and, therefore, may access PIP payments.

ISQua Accreditation International Accreditation Programme (IAP)

Guidance on Designing Healthcare External Evaluation Programmes including Accreditation

There are three types of payments available under the PIP

1. Practice Payments

The majority of payments through the PIP are made to practices and focus on those aspects

of general practice that contribute to quality care. These payments are intended to support

the practice to purchase new equipment, upgrade facilities or increase remuneration for GPs

working at the practice.

2. Service Incentive Payments

Service Incentive Payments (SIPs) are generally made to GPs to recognise and encourage

the provision of speciﬁed services to individual patients. The Cervical Screening, Asthma and

Diabetes incentives have service incentive payment components, and the Aged Care Access

Incentive is a service incentive payment only.

3. Rural Loading Payments

Practices participating in the PIP, with a main practice location situated outside capital cities

and other major metropolitan centres, are automatically paid a rural loading.

There are ten individual incentives available to general practices and GPs under the PIP

After-hours Incentive, supporting general practices to have appropriate arrangements in

place that ensure their patients have access to quality after-hours care.

Asthma Incentive, which aims to encourage GPs to better manage the clinical care of people

with moderate to severe asthma.

Cervical Screening Incentive, which aims to encourage GPs to screen under-screened

women for cervical cancer, and to increase overall screening rates.

Diabetes Incentive, which aims to encourage GPs to provide earlier diagnosis and effective

management of people with established diabetes mellitus.

eHealth Incentive, which aims to encourage general practices to keep up-to-date with the

latest developments in eHealth and adopt new eHealth technology as it becomes available.

GP Aged Care Access Incentive, which aims to encourage GPs to provide increased and

continuing services in Residential Aged Care Facilities.

Indigenous Health Incentive, which aims to support general practices and Indigenous health

services to provide better healthcare for Aboriginal and / or Torres Strait Islander patients,

including best practice management of chronic disease.

Procedural GP payment, which aims to encourage GPs in rural and remote areas to maintain

local access to surgical, anaesthetic and obstetric services.

Quality Prescribing Incentive, which aims to encourage practices to keep up-to-date with

information on the quality use of medicines.

Teaching payments, which aim to encourage general practices to provide teaching sessions

to undergraduate and graduate medical students who are preparing for entry into the

Australian medical profession.

Since the inception of the PIP in 1998, successive Australian Governments have committed to

ongoing funding for the programme; and during this time, have retained the requirement that a

practice must be accredited, or registered for accreditation, and must maintain full accreditation

in order to access such payments.

Given the level of participation in accreditation by Australian general practices, it can be

assumed that the highly incentivised PIP has been instrumental in encouraging practices to

engage in the process, and in turn has had a positive impact by supporting practices to focus on

improvements and quality outcomes.

International Society for Quality in Health Care

Joyce House, 8-11 Lombard Street East

Dublin 2, Ireland

Ph: +353 1 670 6750 Fax: +353 1 671 0395

Web: www.isqua.org