Cover Story


International Journal of Government Auditing – October 2011
Special Issue on Peer Review

Background is map of the world with no markings.  The title superimposed on the background is 'Peer Review:  A Quality Assurance Tool for SAIs.'
An interview with Dieter Engels, President of the German SAI and Chairman of the INTOSAI Subcommittee to Promote Best Practices and Quality Assurance through Voluntary Peer Reviews

In the field of science journalism, peer review has a track record dating back to the 17th century. In external government auditing, peer review is a relatively new tool that has gained increasing importance in the INTOSAI community in recent years.

Subcommittee 3 of INTOSAI’s Capacity Building Committee (CBC) promotes best practices and quality assurance through voluntary peer reviews. According to information provided by the subcommittee, 24 peer review projects have been successfully completed since 1999. More than 30 supreme audit institutions (SAI) have participated in at least one peer review, either as the reviewed or reviewing SAI. Currently, five INTOSAI members are being reviewed by peers.

This edition of the International Journal of Government Auditing focuses on peer reviews in recognition of their growing scope and importance within INTOSAI.

The support team of CBC Subcommittee 3 interviewed Prof. Dr. Dieter Engels about the various approaches to peer review that SAIs have adopted and the German SAI’s own experiences with peer review.

Question:

Why do SAIs carry out peer reviews?

Dr. Engels:

Peer reviews are carried out for a variety of reasons. In most cases, a peer review is used as a quality assurance tool to answer the question, Who audits the auditor? It helps SAIs carry out their audit work in compliance with professional standards; it also helps them enhance their practices and procedures. For example, when the German SAI participated in a peer review of the Austrian SAI in 2010, we investigated how the Austrian SAI could improve the way it carries out its core functions of auditing and providing advice.

Peer reviews can also provide the basis for elaborating an SAI’s strategic development plan. When Estonia was preparing for its imminent accession to the European Union, its SAI underwent a peer review whose results provided key guidance for defining its future role and mandate.

Question:

You have just outlined the benefits for the reviewed SAIs. What incentives are there for SAIs to assume the role of reviewing SAI?

Dr. Engels:

Peer reviews provide benefits to all participants. Their merits are not limited to the reviewed SAI. The reviewing SAI gains a deeper insight into the procedures and methods of a peer organization. Thus, it can identify good practices by comparing its own structure and procedures to those of the reviewed SAI. To date, the German SAI has participated in four peer reviews, and each time we identified helpful suggestions and ideas for our own work.

Question:

All this sounds very positive. What about potential risks? And, if there are any, how can they be addressed?

Dr. Engels:

Nobody likes to be criticized and, therefore, the decision to undergo a peer review requires courage, openness, and confidence in one’s own strengths. As a reward for this courage, the SAIs involved should deal with each other on an equal footing without any bias and in a spirit of friendship and cooperation. This does not mean that the process will be dominated by words of appreciation and that criticism is undesirable. A peer review yields optimum benefits only if the partner SAIs engage in a sincere, frank, and constructive dialogue and the recommendations and proposals for improvement are based on sound arguments.

To avoid ambiguities or misunderstandings during a peer review, the participating SAIs should agree in advance on a framework for the process. For instance, it is essential to agree on the peer review’s objectives, timetable, and procedural steps and how its costs will be handled. These basic conditions should be spelled out in writing in a document known as a memorandum of understanding (MOU).

In most cases, the organizational structure and audit environment of the reviewed and the reviewing SAIs will differ from each other. Therefore, the peer review team should receive all information it needs to familiarize itself with the legal, political, economic, and societal environment of the reviewed SAI. This will help ensure that the reviewed SAI can accept and implement the recommendations the peer review generates.

Question:

What areas do peer reviews normally focus on?

Dr. Engels:

There is no one generally applicable rule. Since peer reviews are carried out voluntarily, the participating SAIs are free to decide on their scope and content.

On one hand, the peer review may follow a quite comprehensive approach, looking into all audit activities and the entire structural organization of an SAI. This was the case for the peer review of the European Court of Auditors that was conducted in 2008.

On the other hand, peer reviews may focus on one or several specific fields of activities. As a case in point, I would mention the peer review of the Peruvian SAI, conducted in 2008, which addressed five selected fields of activity, including relations with Parliament and processing of petitions.

Question:

Should the results of peer reviews be published?

Dr. Engels:

The decision about whether or not to publish a peer review report lies exclusively with the reviewed SAI. Nevertheless, experience has shown that those SAIs that have undergone a peer review have usually decided to make the results available to the general public.

Personally, I welcome such publication. An institution that deals with its strengths and weaknesses with equal openness creates confidence in itself and enhances the effectiveness of its work. Moreover, other SAIs can also benefit from the peer review report by, for instance, using it as a yardstick to design their own peer reviews.

Question:

What other tools are available to help SAIs considering or definitely planning to have a peer review?

Dr. Engels:

As you may know, the Subcommittee to Promote Best Practices and Quality Assurance through Voluntary Peer Reviews has drafted the Peer Review Guideline, which was adopted as ISSAI 5600 at the XX INCOSAI held in South Africa. The guideline is a useful tool that helps both the reviewing and reviewed SAI to successfully complete all stages of the peer review process. The document places particular emphasis on the contents of the MOU, whose importance I have already pointed out.

The guideline has an appendix, the Peer Review Checklist, that contains detailed questions that can be asked in the course of a peer review. The checklist provides guidance to the team of reviewers depending on the scope of the review and the fields to be reviewed.

ISSAI 5600: Peer Review Guideline and Checklist

The Peer Review Guideline developed by Subcommittee 3 of INTOSAI’s Capacity Building Committee (CBC) was adopted by the XX INCOSAI as ISSAI 5600. The guideline is available on the ISSAI Web site at www.issai.org and on the CBC Web site at cbc.courdescomptes.ma under “Guides & Materials.”

The Peer Review Guideline makes recommendations for all stages of the peer review process. It discusses preliminary questions (such as the objectives of a peer review and the choice of partner SAIs) as well as specific advice on how to prepare, implement, follow up on, and evaluate a peer review exercise.

A central chapter deals with the framework conditions to be agreed upon. Good practice cases based on earlier peer reviews supplement recommendations for the provisions that the memorandum of understanding should include.

The Peer Review Checklist: Appendix to ISSAI 5600 provides guidance for conducting a peer review. The checklist includes questions designed to help understand the national environment of the SAI to be reviewed as well as the SAI’s structural organization, internal rules, audit standards, and audit procedures. It is also available on the CBC and ISSAI Web sites.

Cover of 'Peer Review Guideline.'


The peer review documentation posted on the Capacity Building Committee’s Web site (cbc.courdescomptes.ma) is another useful tool. At present, this documentation includes MOUs and reports of 20 peer reviews (see the list of the documentation on p. 5). It can serve as reference material that will help SAIs make informed decisions about carrying out a peer review.

Question:

You already mentioned that the XX INCOSAI adopted the Peer Review Guideline. Developing it was a major task of the peer review subcommittee. What issues will the subcommittee address in the years to come?

Dr. Engels:

While we have finished the guideline, we have not fully accomplished our mission. INTOSAI’s Strategic Plan 2011–2016 assigned several tasks to our subcommittee to enhance the environment for conducting voluntary peer reviews. The subcommittee wants to continue its efforts to assist SAIs as much as possible with preparing for and implementing peer reviews.

Specifically, one of our aims is to supplement and further refine the peer review documentation. Therefore, we are always anxious to receive professional input from SAIs that are conducting or undergoing a peer review. We also appreciate it very much if MOUs, reports of results, and other peer review documents are shared with us to enhance our documentation.

We are looking for new ways of presenting peer review materials to facilitate research and reliance on the documents.

Furthermore, we collect feedback on the applicability and the utility of the Peer Review Guideline and its checklist. For this purpose, the guideline includes a concise survey at its end inviting all INTOSAI members to share ideas and suggestions with us. Our subcommittee will use this feedback to review and refine the guideline.

If you have questions, wish to provide feedback on the Peer Review Guideline, or wish to share relevant documents with other SAIs, please contact the German SAI at international@brh.bund.de.

Special Issue on Peer Review

This issue highlights some of the peer review initiatives being carried out across the INTOSAI community. In addition to the interview with the chair of the INTOSAI peer review subcommittee in the editorial, two articles deal with the just-completed peer review of the Slovak Republic.

  • The first article was written by Dr. Ján Jasovský, President, Supreme Audit Office of the Slovak Republic, and shares insights from perspective of the entity being reviewed.
  • The second article was written by Pawel Banaś and Jacek Mazur, Supreme Audit Office of Poland, two members of the peer review team. They share their reflections from the perspective of the team performing the peer review.

INTOSAI’s peer review efforts are coordinated by subcommittee 3 of the Capacity Building Committee. The Journal wishes to express its appreciation to the subcommittee, chaired by the SAI of Germany, for its invaluable assistance in preparing this issue.

Photo: Members of the INTOSAI subcommittee on peer review.
Members of the INTOSAI subcommittee on peer review.