International Journal of Auditing – April 2014
The following article is a based on a keynote presentation by John Reed and Jean Cinq-Mars given at the 15th Meeting of the INTOSAI Working Group on Environmental Auditing held in Tallinn, Estonia, June 2013. The presentation was subsequently developed into a full Discussion Paper by the CCAF, Vérificateur général du Québec, and the Office of the Auditor General of Canada. This is a condensed version of the Discussion Paper, which can be found on the website of the CCAF at www.ccaf-fcvi.com. The presentation in Tallinn also inspired the creation of a new INTOSAI WGEA Research Project on “How to Increase the Impact of Environmental Audits,” which is being co-led by the SAIs of Lesotho and Cameroun.
Performance auditors desire to have a positive impact on the programs and entities they audit. Similarly, performance auditors focusing on environmental, health, and safety issues want to see their recommendations implemented and programs and entities to improve as a result. Ultimately, they want environmental quality and the health and safety of all citizens to benefit from their audits.
The authors hold the view that it is possible to increase the impact of environmental performance audits and to improve environmental quality through careful audit topic selection, planning, execution, reporting, communication, and innovation.
Performance audits focus on a wide variety of topics. One audit will focus on a single question in a single departmental program, while another will look at several complex issues in several programs managed by many departments. Some audits focus on economy or effectiveness; others, on efficiency. Many audits examine compliance with policies, laws, and regulations while others focus on the management systems and controls that support such compliance. Most audits look at results. While environmental performance audits deal with a specialized subject, they also fall in the categories listed above.
The success of all performance audits rests on the same necessary foundations: a solid methodology, qualified people, and a sound knowledge of the subject matter.
Audit offices recognize the importance of solid methodology. Methodology that complies with professional standards, adopts best practices, and reflects key principles of quality assurance and quality control will add value and help bring about change.
To apply audit methodology as intended and complete audits in a timely manner requires strong audit teams, consisting of the right people with the right skills for each project. The most effective performance auditors usually possess a combination of key skills, particularly professional judgment, critical thinking, creativity and innovation, and the ability to lead and supervise, and to manage relationships and communications, both internally and externally.
Finally, to have an impact, auditors must select the right issues to audit, prepare a report that addresses the main questions convincingly, and communicate their conclusions effectively—all of which require a sound knowledge of the subject matter. For environmental auditors, this generally means having a good understanding of current environmental issues and of relevant environmental laws, regulations, policies, standards, and international agreements (such as those involving climate change, protection of endangered species, and waste management). It may also involve consulting advisors and specialists who have related experience with the audit topic.
A well-selected and planned audit is more likely to bring about significant change and add value. The following are recommended steps to take in the planning phase of an environmental performance audit.
The INTOSAI WGEA has produced guidance materials for collaborative audits and has many examples of such audits on its website, such as the Coordinated International Audit on Climate Change conducted by 14 national audit offices and based on 33 individual audits. The WGEA also has a large database of audits conducted by national audit offices around the world. Between 1993 and 2011, national audit offices in over 100 countries conducted more than 3,200 financial, compliance, and performance audits related to the environment.
A key step in audit execution and examination is to anticipate and continually assess data needs. In the planning phase, the types, sources, and limitations of evidence and data are identified. Also at this stage, auditors should try to determine the types of qualitative and quantitative data analysis they will carry out on the evidence and how it may be presented. In the examination phase, auditors need to continually assess whether the anticipated type of data and evidence is available and still relevant and if not, make any needed adjustments to the audit plan.
The data and evidence found during the examination phase leads to observations. Performance audits, like all audits, compare a situation that exists with the way it should be, based on suitable criteria. The gaps between the two result in audit “findings” or “observations.” Examples of common audit findings include:
To answer the burning question, “Why do these deficiencies occur?” auditors should perform root cause analysis, which can support effective recommendations that lead to solutions that prevent the problem from recurring. One caution, however: root causes that are traced to the merits of policy, availability of resources, or partisanship can be difficult for legislative auditors to address.
By the end of the examination phase, auditors have usually gathered evidence from various sources, all of which has to be considered, analyzed, and retained or discarded. In this evidence lies the story the audit will eventually tell. The challenge is to determine what the main messages will be and how to best present them in a clear, convincing manner.
The most effective reports answer the following questions:
In addition, auditors should strike the right tone and communicate a balance of positive and negative findings that reflects the evidence gathered.
Auditors should use the roll-up technique for developing good audit reports. This technique involves filtering information and prioritizing messages for inclusion in the report. Auditors examine the large amounts of information normally collected during an audit in order to identify usable evidence and include in the report only those facts, observations, and conclusions that are material, significant, and/or of high risk. In the environmental domain, significance and risk can and should relate to human health, ecosystem functioning, and the financial consequences of environmental degradation.
Auditors should use their reports to educate. Environmental questions are often complex and it may be necessary to explain important concepts in a report’s introduction to help readers fully understand audit findings and their significance. The introduction is an ideal place to provide basic context and background information and make links to any related economic and social aspects.
Report authors should use plain language and avoid being too technical. Similarly, reports that help readers identify with the topic and care about the audit findings will have more influence. This can be achieved by using concrete case studies, such as ones about potential health impacts in urban areas.
Ultimately, the impact of an environmental performance audit will depend on the quality of the recommendations and on their implementation. Too often, recommendations are prepared at the end of the audit, seemingly as an afterthought. Instead, the thought process for recommendations should start during the examination phase.
Auditors can make strategic recommendations by focusing on the “pivot points” (discussed earlier) in relevant decision-making processes.
They can also aim to create a “domino effect.” The domino effect refers to situations in which changes to one element of a system trigger changes in its other elements; the effect is greater when system elements are closely interlinked and when recommendations are targeted at a key point in the decision-making process. For example, introducing a carbon tax will create a domino effect in society and in the economy since the tax is likely to result in reduced consumption of carbon-intensive products, lower greenhouse gas emissions, increased use of public transport, better air quality, improved health for citizens, and lower health care costs.
Recommendations that are superficial (for example, “the entity does not have a strategy, so we recommend it develop a strategy”) or superfluous (for example, “the entity should continue to do…”) are unlikely to lead to significant changes. In some cases they may be required as a first step, but to be effective, recommendations should:
Another way to increase the impact of an environmental performance audit is to reach out to audiences beyond the usual suspects that can help to reinforce audit messages and impact. For example, environmental auditors can contact specialized media, journals, academics, and members of civil society, including grassroots organizations and youth.
If one audit can have an impact, two audits can have even more. Conducting one or a series of environmental audits and later conducting a follow-up audit to determine progress in resolving deficiencies and implementing recommendations can ensure that audit work will have a sustainable impact. Indeed, when entities know that a follow-up audit might (or will) take place, they are more likely to take concrete actions.
Follow-up audits usually occur a few years after the original audit—enough time for entities to implement recommendations. In most cases, only one follow-up will be done. But sometimes it is good to have a long-term plan for more follow-up audits, especially in the environment domain, where issues tend to have long time frames.
Governments are tasked with managing and addressing key environmental issues, from the impacts of climate change to urban smog. Audit offices have the mandate to assess the implementation and effectiveness of the management of these issues and many more.
Environmental audits can affect change in these areas. But this requires careful attention to audit topic selection, planning, execution, reporting, and communication.
By focusing on these elements, auditors can increase the impact of environmental performance audits and, as a result, improve environmental quality through better and more effectively managed programs. Even auditors play a role in the concept behind the saying, “We don’t inherit the Earth from our parents— we borrow it from our children.”
The authors would like to thank the following individuals for their contributions to this paper: Pierre Frechette, Kimberley Leach, Neil Maxwell, George Stuetz and Scott Vaughan.