SAI PMF a tool for all SAIs, including jurisdictional SAIs! Updated indicators to assess jurisdictional activities
By Eduardo Ruiz and Irina Sprenglewski, senior managers at INTOSAI Development Initiative (IDI)
In a continuous drive to keep the SAI Performance Measurement Framework (SAI PMF) relevant and useful for SAIs, INTOSAI approved a new version of the framework in November 2022 which contains a revised set of indicators assessing jurisdictional activities.
The SAI PMF and Global Uptake
Since the SAI Performance Measurement Framework (SAI PMF) was introduced in 2010, an impressive 96 SAIs have completed a SAI PMF assessment. While SAI PMF has become an established tool among SAIs globally, uptake so far has been more limited among SAIs with jurisdictional competencies, with only 12 being jurisdictional-model SAIs out of the 96.
SAI PMF is a specifically designed tool for SAIs to assess their current situation and performance across its key functions, processes, and outputs. As an assessment against the INTOSAI Framework of Professional Pronouncements (IFPP) and other established international good practices for external public auditing, SAI PMF is recognised as a basis for establishing needs to inform SAI strategic plans and capacity development efforts. Its overall objective is to help improve SAI performance to strengthen public financial management and foster accountability and transparency by leading by example.
A key feature of SAI PMF is that it should be a useful tool for all SAIs. This entails revising and improving the framework when necessary, since it is not static and needs to consider changes in the environment. As a minimum, SAI PMF needs to reflect changes in the underlying INTOSAI standards to ensure an objective assessment. To that end, the adoption in 2019 of INTOSAI-P 50 “Principles of jurisdictional activities of SAIs” which integrated the jurisdictional activities into the IFPP. Moreover, feedback from assessors applying the framework and a consultation conducted in 2020 also indicated that there were some areas for improvement.
An inclusive process
The inclusive revision process involved a multitude of organizations and was conducted under the strategic lead of the INTOSAI Capacity Building Committee (CBC). The technical development was conducted by a group of senior practitioners from SAIs France, Brazil, Greece and Madagascar and from the INTOSAI Development Initiative (IDI) as the SAI PMF operational lead. Together, the group produced a draft set of indicators that was later tested and fine-tuned in two workshops carried out with colleagues from SAIs Portugal, Tunisia, and Morocco. The SAIs involved reflect the existing variety of jurisdictional models, which was important to ensure the revised indicators work across SAIs, and this inclusive approach has greatly contributed to the final product. Subsequently, and in accordance with INTOSAI procedures, all SAIs with a jurisdictional function were invited to provide comments on the revised indicators, with the highlight of the endorsement of the SAI PMF 2022 version at INCOSAI in Rio de Janeiro.
Revising the Indicators Assessing Jurisdictional Activities
Revision scope
SAI PMF outlines key areas to be assessed, represented by domains A-F. The scope of this revision were the indicators and dimensions specifically targeting SAIs with a jurisdictional function (indicators SAI-8, 18, 19 and 20).
The indicators span the range of jurisdictional activities from control of regularity of the accounts and management operations (abbreviation: control of the accounts) and the subsequent legal proceedings. This implies that SAI PMF identifies two competences related to these two main areas. By the same token, it acknowledges that legal proceedings can be initiated in different ways: by irregularities identified in an audit or when conducting the control of the accounts, and from reports and tips from third parties.
A new set of adjusted indicators
Firstly, the review included an alignment with INTOSAI-P 50 which strengthens the framework. However, INTOSAI-P 50 mainly captures the legal proceedings and is at the “principle” level. This entails that the control of the accounts is not extensively covered and there is no standard (ISSAI) describing how the principles should be implemented. It was therefore necessary to include specific criteria based on good practices. These build strongly on the criteria developed by the earlier task team that was included in the SAI PMF 2016 version.
Secondly, the review team needed to correct some shortcomings revealed when applying the indicators these past years – for example, reflecting newer approaches to control of the accounts, based on sampling and risk assessment. Other specific examples covered four specific indicators identified in the review process.
SAI-8 (iv) “Coverage of the control of regularity of the accounts and management operations”
Jurisdictional activities include checking the accounts and the supporting documentation for irregularities. Which accounts shall the SAI check? There are two main scenarios.
- Several SAIs, particularly in CREFIAF, are required to check all public accounts. In several cases, this has led to a struggle with backlogs of accounts pending to be controlled if the SAI does not have the necessary resources to conduct the controls in a timely manner. In this scenario it may still be possible for the SAI to plan and programme its controls in a manner that allows most of the accounts to be subject to control within a defined period. The remaining accounts can be sampled, based on the level of risk they represent.
- In other countries, the legal framework allows SAIs to select the accounts that should be controlled based on considerations such as risks and materiality. In this scenario, SAIs are better positioned to allocate resources to examining the key accounts.
To assess coverage, the SAI can therefore select between two options depending on their mandate, adapting the assessment of coverage to the mandate and legal requirements of the SAIs.
SAI-18: Jurisdictional Legal Framework and system to ensure quality of the control of the accounts
This indicator assesses the foundation for jurisdictional activities. Firstly, it assesses whether there is a legal framework in place that governs the jurisdictional activities, establishing the liability regime for its public managers (including accountants). The indicator focuses on respect for basic principles such as legality, fairness, impartiality, and contradiction.
Secondly, the indicator assesses the processes the SAI has put in place to ensure the competencies of the controllers, and the quality of the control of the accounts.
SAI-19: Jurisdictional Activities
The SAI-19 indicator assesses how jurisdictional activities are carried out in practice. It examines the planning and implementation phase of the control of the accounts as well as the subsequent legal proceedings looking into both the decision-making process and the final decision. It includes adhering to key principles such as fairness, impartiality, and collegiality as well as reflecting key roles such as the public prosecutor.
SAI-20: Results of Legal Proceedings
The aim of this indicator is to assess the performance of the SAI in ensuring that the exercise of the jurisdictional activities leads to notified and implemented judgements. Ultimately making sure that the sanction of the personal liability is effective. The results of controls and legal proceedings are decisions, such as judgments, orders, and legal ordinances against public managers (including accountants). The indicator includes the notification and publication of results, as well as the follow-up by the SAI on the implementation of results.
Use SAI PMF for Better Jurisdictional Activities
To a large extent, the aspiration for SAI PMF from 2010 has been fulfilled. The aim was to create a framework that can be applied in all types of SAIs, regardless of governance structure, mandate, national context, and development level. To date approximately 50% of INTOSAI SAIs have used the SAI PMF as a basis to better understand, manage and improve performance. SAI PMF has also been used to facilitate internal and external communication with key stakeholders about SAIs’ capacity development needs.
A SAI PMF assessment would allow the SAI to demonstrate how its jurisdictional activities contribute to compensate for losses suffered by a public entity and/or to sanction the personal liability, either financial or disciplinary of individuals found guilty. It also serves for making the SAIs more accountable by reporting performance and monitoring progress over time. Conducting regular repeat SAI PMF assessments is a key approach to monitor performance change.
The work does not stop there, and it is important to maintain the relevance of the framework which is highlighted in the SAI PMF strategy 2023-28. SAIs with a jurisdictional function can now access the improved SAI PMF 2022 version that is even more tailored to assessing jurisdictional activities, following principles rooted in the international standards. The framework can be found on the IDI Website: Resources (idi.no).
Hopefully, SAI PMF will continue to help SAIs make a difference by strengthening their capacities and performance for the next many years.
