International Journal of Government Auditing – January 2013
The Office of the Auditor-General of Pakistan is committed to producing high quality audit reports and contributing to improved service delivery in the public sector. Driven by this vision, we developed our Financial Audit Manual (FAM) based on International Auditing Standards (IAS) and introduced it in 2006. The manual advocates the use of a risk-based methodology in public sector auditing and provides quality checks at all stages of an audit cycle. We revamped our training programs and developed and diffused knowledge products to embed the FAM in our practices to enable auditors to use the FAM effectively in their work.
At that time, we had a Central Quality Assurance Committee (CQAC) to ensure quality in our products. The CQAC’s mandate was limited to reviewing, improving, and finalizing audit reports of field audit offices (FAO). The CQAC reported that introducing the FAM had improved the quality of our audit reports to a certain extent. However, other quality-related problems remained.
Using benchmarking techniques, we followed a structured participative approach in developing and institutionalizing our quality management framework (QMF) in 2010. It uses prescribed standards as benchmarks and examines the extent of FAO compliance with these benchmarks in audit processes and reports. Although the QMF has been operative only for about 2 years, it has already improved quality of our audit processes and reports. This article shares our experience in designing and institutionalizing our QMF.
Elements of Our QMF
SAIs should establish a quality management system that responds to the specific challenges they face. The system should help them test the existence, relevance, adequacy, and operational effectiveness of quality controls within the SAI. In addition, it should provide mechanisms to not only identify and fix gaps in compliance with prescribed audit standards but also locate bottlenecks requiring changes in policy, systems, and procedures. The system should allow different tiers of an SAI to make meaningful contributions toward quality improvement.
We integrated the requirements of the FAM and sectoral guidelines in the design of our QMF. In addition to ISSAI 40, we also factored in relevant EUROSAI and ASOSAI standards and guidelines. As intended, our QMF is consistent with the system of quality management envisaged in the preceding paragraph. It captures quality at all stages of an audit cycle and acts as an effective decision support system for quality management, as explained below:
Quality Assurance Mechanism: This mechanism provides reasonable assurance that our personnel comply with the professional standards and that our reports are appropriate in the circumstances. The QMF puts the responsibility for quality assurance processes with the heads of FAOs. The processes include updating the permanent file, preparing the planning file, reviewing audit planning and audit completion checklists, filling in the “Post-Audit Quality Assurance Checklist,” maintaining documentation and ensuring the completion of audits within the planned timelines, and reviewing quality assurance and quality control reports. The QMF specifies roles and responsibilities of heads of FAOs, audit wings, and other officials in these processes. Periodic quality reviews track the extent to which FAO officials have taken the required actions. Standardized templates/checklists are included in the QMF to facilitate reviews.
Quality Control Mechanism: This mechanism checks the effectiveness of the quality assurance mechanism in the FAOs and evaluates and grades FAO audit reports. An Audit Quality Management Wing (AQMW) was set up to perform these tasks. It is headed by a senior quality management specialist (SQMS) who supervises the work of a team of quality management specialists (QMS). One QMS has been placed in each of our six audit wings and reports to the SQMS. The AQMW is at sufficiently higher level within the SAI to work independently. It is staffed by experts from the marketplace, but we plan to gradually replace them with our regular officers.
Quality Improvement Mechanism: Like any effective benchmarking process, the QMF supports continuous improvements in audit quality. FAOs get QCR reports showing what needs to be done to promote compliance with prescribed standards and are required to take actions agreed to with the findings of the QMSs, since the follow-up QCRs will invariably check and report on the implementation status of those actions. Audit wings get reports on quality aspects of audit operations from FAOs and the SQMS and are expected to design and implement preventive and corrective measures. Thus, continuous feedback in the form of periodic reports from internal and external quality mechanisms and timely implementation of corrective measures promote continuous improvement in audit quality. The AQMW closely monitors QMF implementation to identify opportunities to refine it.
Expect and Respond to Organizational Resistance to Change
Introducing and institutionalizing a QMF in the SAI of Pakistan was a gargantuan effort designed to bring the performance of audit wings and FAOs under closer organizational scrutiny. It was a radical change, as auditors were not used to their work being externally reviewed for quality. Thus, we anticipated some resistance to change and adopted a strategy to counter it. Our experience offers the following lessons that may be relevant to other SAIs interested in strengthening their quality management regimes.
Aligning audit codes and manuals with International Standards on Auditing or ISSAIs is the first critical step in improving the quality of SAI work. However, significant progress in improving quality will occur only if auditors apply prescribed standards in their work, which is easier said than done. Our QMF demonstrates how an SAI may handle this challenge. It makes a clear distinction between quality assurance and quality control processes and shows how the two, working together, can increase compliance with prescribed auditing standards and enhance the effectiveness of an SAI’s audit operations. Introducing a QMF will trigger resistance to change with an SAI and the latter needs to take appropriate steps to mitigate this risk. In the SAI of Pakistan, the impact of implementing the QMF is already visible in improved compliance with prescribed standards and the quality of audit reports.
For additional information, contact the author at firstname.lastname@example.org.
 The term knowledge product refers to templates (i.e., audit planning and reporting templates) and guidance notes included therein to help auditors apply FAM good practices in their work. Some of products are available on the SAI Pakistan’s website http://www.agp.gov.pk.
 The QMF document is available on the SAI of Pakistan’s website: http://www.agp.gov.pk.
 ISSAI 40 requires that the quality control regime support this goal.
 Based on the FAM, the SAI of Pakistan has produced 24 such guidelines that facilitate the application of the FAM in a specific sector. For example, audits conducting information system (IS) audits will use the sectoral audit guidelines for IS audit.
 Since the introduction of the QMF, the AQMW has conducted QCRs of the planning and execution phases.
 The SAI Pakistan developed a Handbook of Quality Controls in Performance Auditing in November 2011. The QCCs will appraise performance audit reports against criteria that capture the significance of performance audit topics, the objectives and methodology used by audit teams, and the language and structure of performance audit reports as laid down in the handbook.