Highlights

International Journal of Government Auditing – October 2013


Coordinated Audits as a Capacity Building Strategy

An international coordinated audit combines several audits on the same topic conducted by different SAIs in their respective countries according to an integrated planning approach. This strategy has traditionally been used to study transnational issues, such as environmental resource management. It has also been used in federal states to evaluate programs in which regional governments play an important (and usually complementary) role. The final report can thus provide a useful snapshot of how a pecific issue is being dealt with at a national or regional level.

Recently, coordinated audits have also been used as a robust capacity-building mechanism. They allow methodological and technical knowledge to be applied to a real-life problem, resulting in a concrete product—the audit report—that is sector- and region-specific. This concept builds upon the INTOSAI Development Initiative (IDI) practice of using pilot audits as a component of an integrated impact learning strategy. Pilot audits and coordinated audits each provide distinct advantages; the latter facilitates close long-term collaboration among SAIs and the production of a consolidated audit report on a major regional issue.

The coordinated audit process offers multiple opportunities for capacity building. Auditors keep abreast of current methodology and deepen their knowledge of the specific subject matter involved in the audit through a combination of training approaches, such as online courses and in-person workshops. During the research and planning phases, outside experts familiar with the field and regional intricacies can play a key role in ensuring that auditors have the information and skills they need to conduct their work. In later phases, when the audit is being carried out and the results discussed, auditors can continue to glean insights through contact with these experts.

OLACEFS is consolidating a multifaceted capacity-building strategy centered on coordinated audits as a key component of its impact-learning approach. Thus far, it has proven successful. The strategy builds on guidelines from the Capacity Building Committee's Guide for Cooperative Audit Programs Between Supreme Audit Institutions and includes the acquisition of knowledge and skills during each phase of the audit process.

The phases can be summarized as follows, though adaptations can be made for each initiative:

  1. Selection of a high priority audit topic
  2. Feasibility study, including definition of audit type and refinement of scope and objectives
  3. Audit planning and preparation
    3.1   Selection of audit teams and their coordinators
    3.2   Capacity building on audit techniques and tools (online and in-person)
    3.3   Capacity building on the specific audit topic
    3.4   Workshop or reference panel with specialists on the audit topic
    3.5   Preliminary studies (national level)
    3.6   Compilation/consolidation of the preliminary studies
    3.7   Development of the audit plan (ISSAI compliant)
  4. Audit execution in each country
    4.1   Monitoring of the audit schedule and plan
    4.2   Online meetings to exchange information on progress in each country
  5. Technical meeting to discuss audit results and determine the structure of the consolidated report
  6. Publication and dissemination of the consolidated report
  7. Monitoring of recommendations
  8. Critical assessment of the initiative as a whole and discussion about continuity of the cooperation efforts

The use of coordinated audits as a capacity-building strategy reinforces the contemporary government audit paradigm, in which audit work demands not only methodological skills but also expertise in the fields in which SAIs focus their audit work. Coordinated audit initiatives facilitate knowledge sharing and pooling of resources among SAIs with a view to deepening auditors’ understanding of the audit topic at hand. The audit teams and coordinators can recruit specialized consultants, conduct in-depth preliminary studies, and hold reference panels and seminars.

International standards and best practices can be effectively disseminated to individual SAIs and auditors through the coordinated audit strategy. There is a growing consensus in the INTOSAI community regarding the importance of implementing the International Standards of Supreme Audit Institutions (ISSAI), and coordinated audits provide an ideal platform for this process. Coordinators can ensure that coursework, planning, and general methodology are ISSAI compliant; in this manner, auditors from various countries become familiar with and begin applying practices that are fully aligned with ISSAIs, thereby facilitating efforts at their home SAIs to update local standards and methods. IDI has recognized this particular benefit and included coordinated audits as part of its ISSAI Implementation Initiative (3i Program).

Like many institutions focused on capacity building, OLACEFS has discovered the tremendous benefits of distance learning and is increasingly investing in online courses to train larger numbers of auditors at a lower cost. Since international coordinated audits involve staff in various countries, online training is an integral part of the long-term capacity-building strategy involved. When this training is followed up with in-person workshops and seminars, the strategy is particularly effective in preparing the audit teams for field work and developing institutional capacity in a sustainable manner.

Most importantly, what sets coordinated audits apart as an impact learning strategy is that auditors put the skill set they have acquired into concrete practice, maintaining international collaboration among peers and specialists for an extended period. In sum: auditors receive theoretical training in the topic they are auditing from specialists in the field, as well as in ISSAI-compliant audit methodology, through distance learning coursework combined with in-person training; they immediately apply the knowledge attained, in collaboration with peers from various SAIs, through each step of the audit process. The consolidated audit report, which provides a unique perspective on a vital regional issue—though potentially an invaluable contribution in and of itself—is the icing on the cake.

The OLACEFS Experience

In 2011, OLACEFS’ Regional Capacity Building Committee (CCR) included coordinated audits in its five-year strategic plan. An OLACEFS subgroup comprising Mercosur[1] SAIs (EFSUR) had concluded a preliminary coordinated audit in 2010 on the Mercosur Structural Convergence Fund (FOCEM) and was planning a more in-depth evaluation of a specific program financed by FOCEM. In 2011, Brazil’s SAI, which held the Executive Secretariat of EFSUR, partnered with CCR and the German cooperation agency GIZ, through the OLACEFS/GIZ Program, to conduct the first coordinated audit that closely followed the capacity-building strategy outlined above.

This coordinated audit of the Mercosur Free from Foot-and-Mouth Disease Action Program (PAMA) directly involved five SAIs and was successfully concluded in September 2012. Three more coordinated audit projects have been conducted or are currently underway, all in the framework of CCR’s strategic plan and with the support of OLACEFS/GIZ.

In 2012, Colombia’s SAI coordinated an audit on the AGRO 2003–2015 Plan for Agriculture and Rural Life in the Americas to assess adoption and monitoring of the plan in nine Latin American countries. In 2012, Brazil’s SAI laid the groundwork for a coordinated audit on oversight of public revenue from oil and gas production. Eight SAIs participated in the preliminary study, four of whom have committed to the entire coordinated audit process, whose planning was finalized in May 2013. Finally, training of auditors has begun for an international audit on water resources coordinated by Argentina’s SAI. There are 13 participating SAIs and, in addition to offering online coursework, organizers provided a training and planning workshop held in June 2013. Coordinated audits are being planned for 2014 on topics relating to biodiversity and information technology.

In conjunction with these audit initiatives, CCR has coordinated training in performance audit concepts and methodology through a distance learning course offered by Brazil’s SAI that is fully ISSAI compliant. (Due to time constraints, in one case—the Agro Plan audit—the contents were consolidated and administered through an in-person course). The course offerings were not limited to the audit teams but rather extended to all of OLACEFS. From 2011 to May 2013, 166 Latin American SAI staff members successfully concluded the full online performance auditing coursework (190 total, including the consolidated mini-course for the Agro Plan). Dozens of these auditors have participated or will be participating in the entire audit cycle, including fieldwork, using the performance audit methods learned.

The following table summarizes these courses, all of which were offered as a component of OLACEFS coordinated audit initiatives. The figures include all OLACEFS students who successfully completed the courses, not just members of the audit teams.

Table 1: Performance Audit Courses in OLACEFS since 2011

Graphic: Table 1: Performance Audit Courses in OLACEFS since 2011

Note: Numbers for the PAMA and Oil and Gas initiatives include only those who successfully completed the course with passing grades; as of the writing of this article, grades had not yet been divulged for the course related to water resources.

The most recent edition of the course, completed in May 2013, introduced a significant development that paves the way for expanded offerings. Since the course was originally designed in Portuguese for Brazilian auditors, and is offered by Brazil’s SAI through CCR, prior editions had involved exclusively Brazilian tutors (with a working knowledge of Spanish). In 2012, staff from Argentina’s SAI reviewed and improved upon the Spanish translation of the course. Deepening this close collaboration, three previous Argentine students of the course were incorporated as tutors in this latest edition, offered in the framework of the water resources audit initiative.

The PAMA Coordinated Audit

The PAMA initiative provides a good illustration of the various steps involved in OLACEFS’ capacity-building strategy using coordinated audits. Brazil’s SAI was tasked with identifying activities financed by FOCEM that merited a more in-depth evaluation. PAMA was chosen not only because of its multinational scope, but also because it met the standards of risk, significance, and feasibility, in accordance with ISSAI 3100. Previous audits conducted by Brazil’s SAI had identified weaknesses in border control posts and highlighted the risk of a possible foot-and-mouth disease outbreak that would result in untold millions of dollars of losses, considering that the Mercosur region is home to 250 million head of cattle.

The SAIs of Argentina, Paraguay, and Bolivia agreed with the proposal to audit PAMA and began forming their audit teams. Additionally, the SAI of Venezuela, though not a direct beneficiary of PAMA, opted to conduct a parallel audit on its national foot-and-mouth disease program and participated in the various planning, training, and reporting stages of the coordinated initiative.

The capacity-building efforts involved in the PAMA program had two major components. The first was the online performance auditing course offered by Brazil’s SAI via CCR from August to November 2011, which aimed at providing all team members with sufficient understanding of concepts, procedures, and techniques needed for the audit. As previously mentioned, the course offering was extended to all of OLACEFS, and 43 auditors from 13 SAIs participated. To facilitate planning and execution of the PAMA audit, three specific tools were emphasized: the design matrix, a SWOT[2] analysis, and the findings matrix.

The second component was a week-long seminar held in Brasilia in September 2011 for audit team members. The meeting had three complementary objectives. First, tutors from the online course led in-person workshops on audit methodology to consolidate understanding of tools explained in the first module of the online course. Second, experts in the field, including a consultant from the Pan American Foot and Mouth Disease Center (PANAFTOSA) and specialized staff from Brazil’s Ministry of Agriculture, led detailed discussions on the audit topic.

Lastly, audit teams undertook joint planning based on the methods and techniques acquired. The central component of this planning was the development of a preliminary design matrix with audit questions, criteria, sources, and expected results, thus permitting all SAIs to follow similar lines of research, investigation, and analysis. In addition, participants agreed upon milestones and deadlines for the subsequent phases of the audit. After the seminar, each SAI had two weeks to provide further suggestions and approve a final version of the design matrix.

The five SAIs then began performing their national audits, following the design matrix with the adaptations they deemed necessary. The coordinating SAI monitored the audit schedule, held discussions via email and phone contacts, received news of progress from each SAI, and circulated key program documents and information. Further, it produced an additional tool, a potential findings matrix, which presented possible findings, both positive and negative, related to each audit question. The SAIs’ findings matrices, which all followed the same template, helped systematize collected data, perceived causes and effects, and potential recommendations for each audit finding, allowing the teams to assess whether the finding was robust enough to be included in the final audit report.

In May 2012, after most SAIs had finalized their fieldwork and had at least begun preparing their national audit reports, a two-day technical meeting was held in Asunción, Paraguay, with members of all five audit teams to present and discuss results of the national audits, agree upon the general outline and format of the consolidated report, and adjust the schedule for the final steps of the audit process.

The consolidated report was finalized in October 2012, and executive summaries were published in Spanish, Portuguese, and English. The results were presented at the OLACEFS General Assembly in November, and each SAI was responsible for disseminating the report to its legislature, Ministry of Agriculture, and other interested parties. In addition, all SAIs agreed to monitor audit recommendations and, every other year, send a brief report to the Executive Secretariat of EFSUR on the current status of recommendations in each country. The Executive Secretariat will prepare consolidated monitoring reports based on this information.

As a final step, the coordinating SAI prepared an assessment report on the PAMA initiative as a whole and presented it to the OLACEFS community as a technical topic for the General Assembly. Comments received from seven SAIs were incorporated into the final report. The results were discussed extensively at the General Assembly with a view to applying the model and lessons learned to future coordinated audits.

Coordinated Audit of the Oil and Gas Sector

The coordinated audit of the oil and gas sector that is currently underway in OLACEFS provides a second illustration of the capacity-building potential of this approach. This topic was selected because of the sector’s strategic importance to energy policy worldwide, the inherent challenges involved in overseeing the sector due to its complexity and sensitivity, and its special relevance for the region, as shown in the data in figure 1 on 2011 government revenue from the sector as well as the industry’s overall share of GDP.

Figure 1: The Importance of the Oil and Gas Sector in Latin American Countries


Country

2011 Government Revenue from Sector (in billions of US dollars)

Total industry share of GDP

Argentina

$3.2

5%

Brazil

$14.6

12%

Colombia

$4.9

8%

Mexico

$68.3

9%

Peru

$2.0

Not available


Source: Data extracted from the preliminary report OLACEFS-GIZ–Project  B.2.9 – March 2013, written by the company Energias Renováveis–EnerRio.

This topic is important to SAIs internationally. In the same period as the OLACEFS coordinated audit initiative, AFROSAI-E is developing, in conjunction with IDI and the Extractive Industries Transparency Initiative (EITI), an audit manual for the extractive industries with work to date focused on the oil production.

As part of the audit preparation phase, the SAIs designated their audit teams and signed them up for the previously mentioned online performance auditing course. Team members unable to participate in the 2012 course were enrolled in the course held from February to May 2013.

To identify shared challenges and refine the scope and objectives of the coordinated audit, it was necessary to have a broad information gathering survey and comparative analysis of institutional models and oversight in each of the region’s major producers. In other initiatives, the coordinating SAI carried out this step, but in this case a specialized consulting firm was hired to conduct the study under close supervision of the project coordinators. This preliminary study contributed knowledge and insight on the sector and allowed for training material based on concrete data to be prepared.

Since audit team members were engaged in each step of the study, this information-gathering phase also provided an initial opportunity to apply concepts and techniques learned in the online course. For instance, team members helped conduct interviews at ministries and government agencies and evaluated their institutional models and auditing mandates.

The results of the consulting firm’s study were consolidated and led to the audit scope being further refined to focus on systems for measuring production and calculating royalties and other payments due to government. As an added benefit, the study recommended other high priority topics for future audits in the sector.

The study also served as the conceptual basis for a seminar organized to train the audit teams and develop an audit plan. At the event, held May 20–24, 2013, in Brasilia, the consultants presented basic concepts and preliminary information from the study, and the SAI teams discussed and agreed upon a design matrix for the coordinated initiative to guide the execution of the national audits.

The next steps include the following:

  • Establishing schedules and executing the audit plan in each country, with close monitoring by the coordinating SAI through online meetings.
  • A face-to-face meeting to discuss a potential findings matrix and standard template for individual SAI reports.
  • Preparing national audit reports.
  • A follow-up face-to-face meeting to discuss results and begin preparation of the consolidated audit report.
  • Finalizing, translating, and publishing the consolidated report and executive summary.

By identifying common weaknesses and best practices, this coordinated audit should strengthen oversight of the oil and gas sector in Latin America and, through the exchange of experiences, in other regions. In addition, the initiative seeks to further refine the coordinated audit methodology as a capacity-building strategy in OLACEFS and other regions.

Conclusion

The updated 2013–2015 OLACEFS-CCR Strategic Plan envisions furthering SAI capacity building by offering a series of impact learning activities. This portfolio includes coordinated audits in critical policy areas that will continue to combine training programs in the specific subject matter and audit methods with hands-on application of the knowledge acquired.

We anticipate that the coordinated audit approach to impact learning will continue to be refined and involve a growing number of SAIs and multilateral partners. We further hope it will lay the groundwork for broader cooperation that might include opportunities for South-South initiatives. Increasingly, as the coordinated audit initiatives in OLACEFS have shown, knowledge can be acquired and transferred among developing country SAIs as regional groups work in coordination with INTOSAI bodies and multilateral institutions.

[1] Mercosur, the Common Market of the south, is a regional trade agreement between Argentina, Brazil, Paraguay and Uruguay, with several other associate members. It was founded in 1991 to promote free trade and the fluid movement of goods, people, and currency.

[2] Strengths, weaknesses, opportunities and threats.

Coordinated audits and multilateral cooperation initiatives

In addition to their capacity-building benefits, coordinated audits are well suited to multilateral cooperation initiatives. They strike an ideal balance between commitments and benefits among participants; allow for cost sharing and economies of scale; and involve timetables, milestones, and deliverables, thus providing concrete indicators to measure output.

As discussed in the preceeding article, the support of GIZ, Germany’s cooperation agency, through the OLACEFS/GIZ Program, has been instrumental in the success of coordinated audit initiatives in OLACEFS. That support has been particularly important in conducting preliminary studies on the audit topic; organizing workshops, reference panels, and technical meetings among auditors; and translating and publishing consolidated reports. Further, the OLACEFS/GIZ program is planning to support the design and implementation of new online courses for CCR that will be critical components of future coordinated audit initiatives.

Other agencies and multilateral organizations have expressed interest in collaborating with coordinated audits in OLACEFS. By working with these international organizations, SAIs can have access not only to the benefits already mentioned, but also to cutting-edge knowledge from the organizations’ experts and to stakeholders with a long history of work in critical policy areas. This process can lead to an exchange of expertise that will also benefit multilateral organizations and development agencies.

Using coordinated audits as part of a broader cooperation strategy can build important networks and synergies and open new multilateral communication channels. SAIs and multilateral organizations can work together to create a common agenda, including new approaches, priorities, and critical policy areas to be approached from different perspectives.

In building this agenda, it is worth noting that certain government functions are especially important not only because they account for a significant amount of public expenditure, but also due to their critical impact on people’s welfare. This impact is generally present in public policies and programs related to health and education, basic infrastructure projects, and income transfer programs for poverty reduction. These policy areas, which are highly relevant for social and economic development, require constant attention from SAIs.

In the case of OLACEFS, they are also areas in which the work of Latin American SAIs could most benefit from the expertise of various international organizations. Infrastructure—due to the steep rise in expenditures in Latin American countries in the past few years—and initiatives related to the UN Millennium Development Goals are areas where the knowledge and experience accumulated by multilateral organizations can be instrumental in improving the quality and depth of SAI audits.