Technical Articles

SAINT: A Public Sector Integrity Assessment Tool

SAINT stands for Self-Assessment INTegrity, a tool developed by the Netherlands Court of Audit in cooperation with the Ministry of the Interior and the Bureau of Integrity of the city of Amsterdam. By using the SAINT tool, public sector organizations can assess their vulnerability to integrity violations and resilience in response to those violations. SAINT also yields recommendations on how to improve integrity management. This article, based on the SAINT manual, outlines the concept of integrity, the basic principles of SAINT and its components, and its design and operation as implemented in a 1-day workshop.

SAIs are well placed to promote the integrity of the public sector by contributing to accountability and transparency. SAINT may help SAIs to assess integrity risks and the resilience of integrity management systems.

The Concept of Integrity

Integrity is not a simple concept to define. Many overlapping and distinct definitions are used. The term integrity is derived from the Latin in-tangere, meaning untouched. It refers to virtue, incorruptibility, and the state of being unimpaired. Integrity is closely related to the absence of fraud and corruption, but it also entails common decency. In this context, it is a positive and broad concept related to ethics and culture. The SAINT tool also uses a broad and positive definition of the term integrity.

Integrity means more than simply observing rules and laws. The law provides a lower limit and a minimum moral starting point. An integrity policy calls for a combination of repression and prevention. On the one hand, an organization must adopt measures to take if its staff act inappropriately (repression). On the other, it must do all it can to remove temptations that might induce civil servants to act inappropriately (prevention). Priority should be given to prevention. Not only is it is more effective, but on balance the investment is many times smaller than the cost of repairing damage caused by inappropriate behavior.

The Concept of Integrity

  • Self-assessment: SAINT is a self-assessment tool. The organization itself must take the initiative to test its integrity. Thus, the assessment draws on the knowledge and opinions of the staff. The organization reveals its own weaknesses and the staff make recommendations on how to strengthen resilience.
  • Targeted at prevention: The self-assessment tool is targeted at prevention. It is not designed to detect integrity violations or to punish (repress) unacceptable conduct but to identify the main integrity weaknesses and risks and to strengthen the organization's resilience in the face of those weaknesses and risks.
  • Raising general integrity awareness: The SAINT workshop significantly increases awareness of integrity. The participants' collective discussions about the importance of integrity are of great value.
  • Learning to think in terms of vulnerability and risk: The SAINT workshop teaches the organization how to think in terms of vulnerability and risk. During the workshop, the participants identify the main vulnerabilities and risks and then make recommendations on how to minimize them.
  • Concrete management report/action plan: The end product of the SAINT workshop is a concrete management report/action plan. Under the expert leadership of a trained moderator, the participants formulate recommendations for their own organization. The report explains to management where urgent measures must be taken to strengthen the organization's resilience in response to integrity violations.

Outline of the SAINT Workshop

SAINT is a self-diagnosis tool that is presented in a 1-day workshop. Table 1 outlines the steps in the process and the order in which they are taken. SAINT consists of four modules that are presented in morning and afternoon sessions.

Table 1: Steps in the SAINT Workshop Process Session	Module	Process Step
Morning	Module 1	a. Analysis of processes
		b. Selection of most vulnerable processes 
		c. Analysis of the integrity risks of the most vulnerable processes 
		d. Selection of the main risks 
Afternoon	Module 2 	Assessment of the maturity of the integrity management system
	Module 3 	a. Formulation of a general management report and action plan
		b. Preparation of a concrete management report and action plan
	Module 4 	Evaluation of the workshop

Module 1a: Analysis of Processes. The first step is to analyze the primary and secondary processes relevant to the organization. By way of preparation, the organization must draw up a full list of its primary and secondary processes and send it to the moderator before the workshop. The workshop can then get off to a "hot start."

Module 1b: Selection of the Most Vulnerable Processes. In this step, an estimate is made of the vulnerability-i.e., the potential exposure to integrity violations-of all the processes named in step 1a. The participants ultimately choose the two or three most vulnerable processes so that the related risks can be identified in the next step (1c).

Module 1c: Analysis of the Integrity Risks of the Most Vulnerable Processes. In this step, participants analyze the integrity risks-i.e., the concrete risks of integrity violations-of the processes selected in step 1b as being the most vulnerable.

The characteristics of a vulnerable process are known from both research literature and practice. They are summarized in table 2.

Table 2: Characteristics of Vulnerable Processes
Elements for assessing vulnerability	Vulnerable areas /activities /actions
Relationship between the government and the public/businesses	Collection 	assessments, taxes, import duties, excise duties, fees, charges 
	Contracting 	tenders, orders, assignments, awards
	Payment 	subsidies, benefits, allowances, grants, sponsoring
	Issuance 	permits, passports, driving licenses, identity cards, authorizations, inspections
	Enforcement 	supervision, control, inspection, prosecution, detection, justice, punishment
Management of public property	Information 	national security, confidential information, documents, dossiers
	Money 	cash/giro via budgets, premiums, expenses, bonuses, allowances, etc.
	Goods 	purchase, management and consumption (stocks, computers)

Processes that have one or more of these characteristics are vulnerable to integrity violations. The left-hand column contains two characteristic elements for assessing vulnerability. Processes in which there is intensive contact with "clients" are more vulnerable to violations because there are more opportunities and temptations. The same is true of processes that involve valuable public assets.

In addition to the vulnerability caused by characteristics of a function or process, factors inherent in certain circumstances can increase vulnerability. Table 3 lists examples of factors that increase vulnerability.

Table 3: Factors That Increase Vulnerability
Management and staff
Management dominated by a single person or small group
Staff has powers to be obstructive
Staff loyalty extremely limited 
Organizational culture
Not customary to hold each other responsible
Lack of opportunity or safety to discuss difficult questions 
Nature of the work 
Discretionary powers/solo action
Political pressure, time pressure, pressure from market parties or members of the public
Complexity 
Complex financial/legal relationships
 Young organization/short or quickly set-up project
Combination of public and private (commercial) functions

While the factors above are not integrity risks in themselves, they can increase vulnerability because they increase the probability of a violation occurring and the consequences (impact) of a violation.

Using the knowledge about vulnerabilities and risks, the participants analyze the main integrity risks for each vulnerable process. A list of the risks identified for each process is then drawn up.

Module 1d: Selection of the Main Risks. In this module, the main integrity risks are selected from the list drawn up in step 1c. Based on the aggregated individual scores of the participants, the top five greatest perceived risks for each process are listed and consensus is reached on the scores.

Module 2: Assessment of the Maturity of the Integrity Management System. In this module, the participants assess the maturity of the integrity measures that together form the organization's integrity management system. SAINT uses an integrity control framework based on research literature and international standards set by organizations such as the United Nations and the World Bank. The system is divided into 14 clusters, which are subdivided into three blocks (general, hard, and soft controls), as shown in figure 1.

Integrity Control Management System

The hard controls are, as the term suggests, concerned chiefly with regulations, procedures, and technical systems. The soft controls are designed to influence behavior, working atmosphere, and organizational culture. The clusters in the general controls category are more wide ranging or have a mix of hard and soft elements.

During the workshop the participants assess the maturity of all the measures by awarding them points. SAINT has a relatively simple classification model consisting of the four maturity levels and associated selection criteria shown in table 4.

Table 4: Maturity Levels and Associated Selection Criteria for Integrity Measures
Level 	Criteria 
1	I do not know of the measures' existence
2	I know of the measures' existence
I think the measures are not implemented/observed
3	I know of the measures' existence
I think the measures are implemented/observed 
I do not know if the measures work/are effective
 4	I know of the measures' existence
I think the measures are implemented/observed
I think the measures work/are effective

In principle, the highest level is the required maturity level. In certain organizations, however, some measures will be less relevant or not applicable. This will become clear when the maturity level is scored and will be taken into account when the measures are prioritized.

Modules 3a and 3b: Management Report and Action Plan. This module reveals the link between the most vulnerable processes (1c) and the measures (2). The central question is which measures are the most appropriate to make the most vulnerable processes more robust. Subsequently, the participants are asked to suggest how the organization can improve and implement the most important measures. These suggestions form the building blocks for the management report and action plan.

Module 4: Evaluation of the workshop. At the end of the SAINT workshop, the participants are asked to answer a series of questions to evaluate the workshop itself.

SAINT in an International Context

The results from the SAINT workshops held in The Netherlands, including one by the Netherlands Court of Audit itself, are very promising. We believe sharing and explaining the concepts of SAINT may be beneficial to other INTOSAI members. However, we feel that pilot workshops may be desirable to further test the tool's applicability in different cultural and technical environments. We also believe that widespread use of SAINT among SAIs would make it possible to learn from others' experiences and would also facilitate further improvement of the instrument itself.

For additional information about SAINT, or if your SAI would be interested in pilot testing the tool, please contact the authors at For additional information about SAINT, or if your SAI would be interested in pilot testing the tool, please contact the authors at h.benner@rekenkamer.nl or i.dehaan@rekenkamer.nl.