International Journal of Government Auditing – October 2011
Representatives from four SAIs (Estonia, Poland, Slovenia, and the United Kingdom) constituted the peer review team for the SAI of the Slovak Republic. In this article, the two Polish members of the team share their reflections on this experience and what they learned through it.
During the course of its work, the peer review team gained an appreciation for the many strengths of the Supreme Audit Office (SAO) of the Slovak Republic. The team recognized that the SAO is an institution open to change and modern working methods. It commended the SAO’s professionalism, the quality of its audit activities, the development of its audit methodology, the growing importance and availability of the SAO’s audit findings to its stakeholders, and its cooperation with the media.
The peer review team had many long discussions with its Slovak colleagues concerning the role, mandate, and functioning of their respective SAIs. Based on their audit experience, the team formulated recommendations to the SAO regarding
This article presents some reflections of the Polish members of the peer review team resulting from those discussions. Its purpose is to show how the interactions among the members of the peer review team and with the entity being reviewed can provide knowledge and insights that are useful to all the parties involved and can be applied to the different organizations, mandates, and working methods of the individual SAIs. For us, the peer review was a great opportunity to compare and rethink our own solutions in the following areas: strategic goals, sources of audit topics, territorial organization of our SAI, development of performance auditing, the readability of audit reports, and approaches to audit activity assessment. The following sections discuss some of the insights we gained in these areas during the course of the peer review and give examples of the differing practices of the peer review team members’ SAIs in each area. We also discuss the overall benefits of a peer review for those involved in performing the review.
Many SAIs have adopted strategic documents that set out the basic directions and principles of their activity. While comparing the experience of various SAIs during the peer review, we observed that the general purpose of an SAI’s activity is not always clearly defined. Undoubtedly, the basic task of SAIs is to audit and report on expenditures of public funds. While in some countries (e.g., Austria, the Czech Republic, France, the Netherlands, and Spain) the constitution sets out the purpose of SAI activities, in most countries this is not the case. For example, the Constitution of Poland sets out the scope of auditing but does not specify the purpose of its SAI’s activity.
Therefore, it is important to answer strategic questions about the ultimate purpose of an SAI’s audit activities: is it to assess the government as a whole or the performance of individual bodies?
When selecting issues to audit, strategic objectives, risk analysis, materiality and the potential impact or added value of the planned audit are often taken into account. However, the concepts of risk and materiality can be further developed to better use an SAI’s resources to audit priority areas. Should the SAI conduct more audits at the strategic level (e.g., sectoral policies and governmental programs)? The Web sites of the SAIs of the peer review team contain examples of audits focused on assessing essential problems or whole areas. These include audits of the system of subsidies for state-owned enterprises, financial management in the central administration, the stability of the banking system, energy security, privatization of the shipbuilding industry, and implementation of a program to fight cancer.
As the peer review team was analyzing the way in which the SAO develops its audit plans, a question arose about whether SAIs should consider informal suggestions about audit topics made by the Parliament, the government, other central state bodies and the public. A valid case can be made both for and against doing this.
Each year, the Auditor General of Estonia asks the parliamentary Committee on Public Accounts to propose audit topics. To the extent that it is possible, the proposed topics are included in the audit plan, except for those issues that have been already audited and topics that are political in nature.
In 2010, the Court of Audit of Slovenia’s 63 audits included 24 audits required by law, 20 audits related to issues selected based on risk analysis, and as many as 19 audits (almost 30 percent) selected from the 409 proposals received from various state bodies, companies, associations, and citizens.
Every spring the President of the Polish SAI (NIK) asks for suggestions to help the NIK select audit issues. In 2010, parliamentary committees submitted 81 suggestions; the Speaker of the Senate submitted 15; the President of Poland submitted 6; the Prime Minister submitted 40; and the Ombudsman submitted 7. The 2011 audit plan included most of the suggestions in its 294 audit topics.
During the course of a typical year, citizens suggest numerous issues for SAIs to audit— for example, cases of suspected misuse of substantial resources, unclear tender proceedings, or other issues of interest. While ad hoc audits of such issues can be conducted, an SAI has the final word on how its often scarce resources are to be spent.
Regional offices of SAIs often employ a large portion of their staff (in the case of Polish NIK, about 50 percent). The NIK has found that it can enhance its long-term potential by strengthening its regional offices, especially by dedicating part of their resources to managing national audits.
Because most SAIs conduct, or want to conduct, performance audits, the peer review team considered ways to expand performance audit work. The Peer Review Checklist: Appendix to ISSAI 5600 identified questions regarding the extent to which an SAI uses different types of performance audit tools, such as process-based studies, organizational studies, impact and outcome studies, cost-benefit analysis, and service and quality management studies. A detailed description of performance auditing methods is presented in ISSAI 3000: Standards and guidelines for performance auditing based on INTOSAI’s Auditing Standards and practical experience and Appendix to ISSAI 3100.
Because the Slovak and Polish languages are fairly similar to each other, we were able to review a number of original audit documents in Slovak, which allowed us to consider the issue of the readability of audit reports. Discussions with the other members of the peer review team showed that many SAIs have problems in this area. For example, even if audit reports are brief, their style, layout, and contents may diminish their clarity and, consequently, their impact. While the language and style of audit reports must be formal enough to ensure that they are accurate and verifiable, the reports must also be accessible to the general public.
Audit reports can begin by describing the key conclusions of the findings to encourage the reader to read the whole document. It is also useful to present executive summaries in special typeface at the beginning of audit reports. The report contents can be divided into parts by types of findings and presented using understandable titles. Additionally, SAIs’ annual reports and reports on individual audits can be enriched with trend analyses, lists comparing findings with data from previous years, and systemic conclusions presenting SAI opinions on the audited issues.
In many SAIs, the audit quality process comprises continuous assessment that takes place during the course of an audit and postaudit assessment.
Continuous external assessment of audit activities is a controversial issue because of the risks it poses to an SAI’s independence. Only a few SAIs practice it. For instance, in the United Kingdom, the National Audit Office’s financial audits have been subject to external review since 2001. The Comptroller and Auditor General chose to have this type of assessment to certify the quality of the SAI’s activities based on commonly accepted criteria. The SAI of Estonia also has an external assessment of its financial audit quality. In these cases, the company chosen to do the external assessment (usually an international audit company) reviews several audits selected using statistical sampling.
A peer review provides a unique opportunity to consider the basic principles and purposes of an SAI’s functioning and to compare them with the experience of other SAIs. Therefore, peer reviews are valuable experiences for both those who conduct them and for those who are reviewed as they seek to find solutions to problems that they share. We found many similarities between the challenges that the Polish SAI and the SAI of Slovakia have to respond to and, as a result, learned a great deal from each other. Both the Polish NIK and the Slovak SAO—like most SAIs in the world—face a key challenge in obtaining significant findings and recommendations and communicating them effectively to our readers, among whom citizens are the most important. Our discussions gave us a great opportunity to compare and rethink our solutions to this challenge and to challenges in the other areas mentioned in this article.
The discussions during the peer review process also helped us identify the strengths of our SAI: numerous performance audits and effective communication with the public through our professional Web site. At the same time, we learned from the Supreme Audit Office of the Slovak Republic about being open to the experiences of others, striving to use new technical solutions, and presenting brief audit reports to the public.
Another group of benefits for INTOSAI as a whole relates to our experience applying ISSAI 5600: Peer Review Guideline and its ready-to-use checklist to this peer review. We found the checklist helpful in quickly screening all important matters. After the peer review, we were able to provide Subcommittee 3 of the Capacity Building Committee with feedback on problems that we had faced with these materials and our proposals for improvements. This was the first opportunity to provide feedback since ISSAI 5600 was approved at the XX INCOSAI in November 2010.
This article does not, however, discuss the results of the peer review. These are contained in the peer review report that will soon be available in English and Slovak on the SAO Web site (www.nku.gov.sk).