Highlights

International Journal of Government Auditing – July 2013


Measuring the Financial Impact of Audit Reports

editor's note: This article describes an approach to measuring the financial impact of audit reports. SAIs may find it useful as they consider procedures for their own institutions.

SAIs seek to enhance public confidence in government institutions by identifying information about the extent to which government programs are effective and efficient. Regardless of whether the audited entities themselves implement good governance practices and meet high standards of accountability, SAIs should certainly evaluate and assess the performance of their own audit activities. Ultimately, one of the most difficult—yet most compelling—aspects of performance is the outcome; for the SAI, one key outcome is the financial impact (whether cost savings or increased revenue collection) achieved by implementing audit recommendations.

ISSAI 3000: Standards and Guidelines for Performance Auditing Based on INTOSAI’s Standards and Practical Experience states that following up on the impact of audit recommendations is important to making audit reports more effective and evaluating SAI performance.

During the 19th UN/INTOSAI Symposium on Government Audit in March 2007, however, few SAIs were able to show that they were using forms to measure the impact of their own activities. Most of them agreed that the financial impact of audit work is difficult to quantify.

Principles to Measure the Financial Impact of Audit Reports

The most commonly used efficiency measures directly related to audit reports are actual versus budgeted hours, percent of time spent in administrative tasks, percent of audit recommendations adopted, and the number of audit engagements. However, the ultimate goal for an SAI performance measurement system should be to measure the financial impact of SAI work. This is extremely important to the SAI’s ability to market itself to government officials and other stakeholders.

Obviously, there is no unique framework to measure the financial impact of audits and audit recommendations. However, some common practices may be helpful in conducting such measurements.

Establish a Follow-up System

A valuable starting point and the most fundamental process involves establishing and maintaining a follow-up system to monitor the disposition of audited entities’ responses to recommendations. To effectively monitor this impact, the SAI should

  • establish the time frame within which the auditee will have completed implementing the recommendations,
  • evaluate the audited entity’s actions in response to the recommendations, and
  • determine the number of recommendations that were completely implemented.

Determine Cause and Effect

Determine the cause-and-effect relationship between the recommendation and the corrective action. This step requires some prudence because the complexity of some corrective actions may mean that the SAI’s contribution to them can vary between 0 and 100 percent. The SAI can take 100 percent credit for the impact of a recommendation that is very specific and prescriptive and that the auditee implemented exactly as stated.

An indirect causal link may also be identified when, for example, the entire audit report and not a specific recommendation, contributes to an improvement that results in a financial impact. In this case, the ratio attribution may vary between 5 and 50 percent.

In some cases, the ratio attribution may reach only 5 percent—for example, when part of the audit activities had a minor role in the corrective action. In any case, it is very important that the SAI develop its own criteria for attributing ratios for the extent to which the SAI’s work contributed to a financial impact in an audited activity. Further, the SAI should regularly evaluate the strength of the relationship that has been attributed and determine whether there are any repeated impacts. To ensure this proper evaluation, a team must be designated to study the adequacy of the causal link.

Identify the Fiscal Year Involved

Financial impacts must have been realized within, or before, the fiscal year in which they are reported, and the auditee must be able to demonstrate the cost savings, additional revenues, or other financial impacts. In all cases, the SAI should not take credit for any future impact, even if achieved, as a result of the continuation of the new controls.

Agree Upon the Financial Impact

In accordance with ISSAI 400: Reporting Standards in Government Auditing (paragraph 24), facts and recommendations are generally agreed upon with the audited entity to ensure that they are complete, accurate, and fairly presented in the audit report. In the same way, the audited entity must also agree upon financial impact. Senior management of the audited entity should agree on the impact, and either internal or external auditors should also evaluate the impact. In this way, achievement against the impact target will be built only on demonstrable savings, will be rigorously tested, and will be subject to careful scrutiny.

Identify the Net Savings or Additional Revenue

Financial impact as a component of quantitative impact should be determined as net savings or net additional revenue generated. Therefore, the cost of implementing the recommendations will be covered by the amount saved. The SAI should consider the following factors:

  • the degree of effort and cost needed to correct the reported condition,
  • the impact that may result should the corrective action fail, and
  • the time period involved.

SAIs using this approach will need to spend time and effort, create a good recording system, and secure the cooperation of auditees.

Financial Impact Form

While discussions and interviews with the auditee are often important in the measurement process, it is also crucial to prepare a form that includes all the principles for measuring financial impact and documents the extent of the impact. The SAI should take into account important questions, such as the following, when developing this form:

  • Does the form ensure that the SAI provides sufficient evidence to demonstrate the financial impact?
  • Did the SAI verify the financial impact?
  • Did the SAI take into account the costs of implementing the recommendations when calculating the financial impact?
  • Did the SAI take into account the costs of implementing the recommendations when calculating the financial impact?

Stakeholders in the Financial Impact Process

A team should be created in each audit department to assess financial impact and develop an accurate database for any expected impacts. Such teams should also be responsible for monitoring progress toward measuring impact and reporting the results to senior SAI management.

Heads of individual divisions or departments within the SAI may approve any estimated financial impact that does not exceed a fixed amount. Impact estimates that exceed that amount may need to be approved by a more senior SAI manager.

Planning to Measure Financial Impact

Early coordination with audited entities before starting an audit engagement is crucial to

  • explain the reasons for implementing audit recommendations and the benefits of measuring their financial impact,
  • identify the information and evidence available in the audited entity and assess its relevance to measuring the financial impact, and
  • secure prior agreement on the methodology for calculating the financial impact.

It is also possible to educate some officials in the audited entities on these matters by periodically holding workshops that allow those officials to discuss their difficulties in implementing recommendations and to clarify the methodology used to measure their financial impact.

The financial impact process should be included in the audit approach, and auditors must assess the possibility/likelihood of achieving such impacts. During the preliminary phase of audit engagement, the audit team should evaluate which audit area would lead to the greatest potential financial impact.

This impact can only be accurately estimated through integrated and reliable information on the costs and performance of audited entities. Accordingly, to track the financial impact of audit recommendations, the SAI must identify that information early on and determine how to coordinate with the auditee for monitoring.

Carrying Out the Measurement Process

The implementation of audit recommendations is a key factor in measuring financial impact. The following are some of the important steps in the measurement process:

  • During the opening meeting with the auditees, the audit team should clarify the purpose of measuring financial impact and discuss the methodology and approach to the measurement process.
  • The head of the audit department may propose additional audit engagements to measure and clarify the usefulness and timeliness of measuring financial impact.
  • The SAI and the audited entity should agree on the time to be allocated for measuring financial impact.
  • The use of some Computer Assisted Audit Techniques (CAATs) to analyze data and extract results may be very helpful in identifying and measuring more impacts.

Most SAIs are able to report examples of nonfinancial impact, such as improvements in governance, planning, objective setting, or resource allocation. They may even be able to describe anecdotally the wider economic benefits resulting from SAI work, such as improved productivity from patients returning to work more quickly from hospital. However, valid measurement of the actual financial impact of audit results still requires additional efforts by either professional associations or SAIs.

For additional information, please contact the author at adnen.maali@gmail.com.