Peer Review: A Road to Building Trust, Accountability, and Credibility

Mr. Nelson directs the U.S. Government Accountability Office’s (GAO) preparations for external peer review and its annual internal inspection program, as well as quality assurance training and audit policy development and implementation. He consults domestically and internationally on audit quality assurance systems, peer review, and compliance issues, and has led international peer reviews. Prior to moving to his current position, Mr. Nelson was Director of GAO’s International Relations and Trade group. He has received numerous awards and citations for his work and for mentoring other staff.

Why is peer review important to Supreme Audit Institutions (SAIs)?
Let me start with how peer review can help SAIs achieve their goals. Overall, SAIs want and need to have credibility. This includes being viewed as having achieved the capability and capacity to reliably assess government performance, as well as the ability to identify significant risks. Peer review can be very useful in this regard, as it provides an independent assessment of the reviewed SAI’s state of readiness.

What are some specific benefits of peer review?
Peer review can provide strategic benefits in the form of enhanced credibility, which can bolster the SAI’s influence and ability to produce change in the government’s effectiveness and efficiency. It can offer operational benefits as well, by helping assure the SAI’s systems and processes align with its strategic goals and objectives, and that quality assurance systems are operating as intended.

Peer review also provides an opportunity for continuous learning and improvement as a result of insights gained from external parties, who may bring different perspectives to issues the reviewed SAI is facing. Similarly, reviewers may identify good practices the reviewed SAI has adopted that could be beneficial at their own SAIs.

What are some key peer review challenges?
Peer review requires considerable time and resources from the reviewed SAI, as well as from the reviewers. Thus, it is a significant investment and commitment. This can be a challenge for SAIs facing financial and staffing constraints. Another challenge that can arise during a peer review is a lack of clarity on the review’s objectives.

In addition, instability in audit or quality assurance policies, procedures, or processes leading up to a peer review can create challenges for conducting the review. When there have been changes in these areas, the reviewed SAI should take steps to identify them, and the audits to which they applied, to facilitate the review process. Further, identifying the reasons for the changes and their intended effects on quality can enhance the review team’s understanding of the implications for the review. Discussing with the peer review team new or planned initiatives that will occur during the peer review period helps ensure there are no surprises.

What are some of the keys to success in peer review?
Let me outline a few specifics:

• Planning, preparation, and communication are the keys to success in a peer review. For example, the reviewed entity should gather reliable information and documentation relevant to the policies, procedures, systems, and audits that will be in the scope of the review.
• Electronic access to key policies and procedures, as well as statistics and other information about the audit office, facilitates peer reviews. Including an overall index or table of contents for this information, and keeping it updated throughout the review, is very helpful.
• Candid discussions about expectations for the peer review help all parties achieve a common understanding of the process, goals, and objectives of the review. This open communication is essential for ensuring the benefits of peer review are fully realized and surprises avoided.
• Clearly defining for the peer reviewers key terms, abbreviations, and practices—especially those that are unique to the audit entity and not commonly used—facilitates understanding. It is helpful for the reviewed entity to consider terms or expressions that could be confusing because of language differences, or due to different meanings or uses.
• A collaborative approach to considering changes to the conduct of the peer review in light of shifting circumstances can help ensure all parties’ perspectives and interests are considered in the final decision.

What do you see as a potential peer review risk?
Great question, and one that highlights the importance of two-way communication between the reviewed entity and the review team throughout the peer review and during the development of the draft report.

One of the key challenges is to assure that the review team understands the reviewed SAI’s operating environment from early on in the process. This can help the peer review team tailor its work to be most useful and constructive in view of the reviewed SAI’s particular context. This question also highlights the need for a good feedback mechanism and regular exchanges between the reviewed SAI and the review team.

Who are the key stakeholders in a peer review?
I would say the SAI and its staff, Parliament, and Government all have an interest in the peer review outcome. Obviously, positive results are a source of pride for the SAI’s leadership and staff. Additionally, positive results can serve to enhance the reviewed SAI’s credibility in Parliament and Government. As I said earlier, the goal is to be viewed as a trusted source of information and analyses on the government’s performance.

Any parting comments?
Peer review, like quality, doesn’t cost, it pays—and in numerous ways!