Beyond Compliance: Elevating Audit Impact through Behavioral Insights
Authors: Carlos Morales Castro, Follow-up Area Manager; Alexia Umaña Alvarado, Follow-up Area Supervisor; Verónica Cerdas Benavides, Audit Supervisor and former SAI Young Leader.
Through their work, Supreme Audit Institutions (SAIs) are compelled to promote positive changes in people’s lives. To achieve this purpose, auditing stands out as one of the most important tools at SAIs’ disposal to increase the public value generation. By carrying out audits, a SAI can evaluate whether processes are executed in compliance with established norms, identify performance improvements (under the usual principles of effectiveness, efficiency, and economy), or conclude on the financial information of an entity.
However, even if a good audit is conducted, the value chain that it generates can be broken if the recommendations issued by the SAI are not fulfilled by the auditees or are not taken with the necessary commitment to ensure the sustainability of the corrective actions indicated. Therefore, the design and follow-up of audit recommendations are essential elements for promoting audit impact. How can this be addressed from an innovative perspective? One possible answer is by applying behavioral sciences.
Cognitive biases and auditing
As part of the 2022-2023 SAI Young Leaders Program promoted by the INTOSAI Development Initiative (IDI), the General Comptroller Office of the Republic of Costa Rica developed a project entitled Use of behavioral insights and identification of cognitive biases in audit process for a better design and follow up of audit recommendations. This project aimed to build capacities in audit teams for identifying cognitive biases and applying behavioral insights techniques in the audit process, in order to promote better-designed recommendations, more empathetic and effective follow-up, and increased compliance with the recommendations by the auditee. The project hoped to transform their vision from imposition to persuasion and convincing.
To contextualize the project’s theme, it is necessary to point out what a cognitive bias is. Following Chin (2022), a cognitive bias is a systematic error in thinking that occurs when people process and interpret the information around them, affecting the judgment formation and decision-making process. This term was introduced in the 1970s by psychologists Amos Tversky and Daniel Kahneman, who laid the foundations for the development of behavioral economics. For his contributions in this field, Daniel Kahneman won the Nobel Prize in Economics in 2002. (He shared the Prize with economist Vernor Smith, who were recognized by his contributions in experimental economics.)
As human beings, any person can make mistakes in decision-making, due to cognitive biases. By relying on mental shortcuts (heuristics), audit teams may undervalue or overvalue evidence or misinterpret the available information. Furthermore, cognitive biases can influence perception and drive teams towards erroneous conclusions. These systematic errors in thinking also affect interactions with stakeholders, including audited organizations, which can affect their subsequent willingness to comply with audit recommendations. For these reasons, this project was about being conscious of these unconscious biases.
Figure 1: Potential benefits of applying behavioral insights in auditing process
Low-cost, high-impact innovation
For all these reasons, the application of behavioral insights emerged as a low cost but high-impact innovation. This is very relevant given the fiscal crisis in Costa Rica, with a 61,1% debt/GDP ratio (in December 2023). The application of behavioral insight is almost a zero-cost intervention, the main requirement being awareness and knowledge-sharing.
In addition, a project like this allows the SAI of Costa Rica to be innovative in a wider sense: usually people tend to associate innovation with technology, even going as far as turning it into a purpose itself rather than being a tool for problem-solving. By definition, innovation is about implementing significant changes in processes, products or the organization with the purpose of improving the results, in this case: the identification of cognitive biases and the application of behavioral insights strengthen the objectivity and professional judgment, at the same time, promote a better understanding inside and outside the organization.
In this line, the first step was socializing the framework with audit teams. Activities were carried out to raise awareness about the concept of cognitive bias and how they can jeopardize the objectivity and professional judgment of the audit team. The main cognitive biases (e.g., confirmation bias, anchoring, authority bias or social norms) and the relationship with auditing were explained. As part of these awareness activities, a webinar with Organization for Economic Cooperation and Development (OECD) and SAI-Chile for the OLACEFS community were conducted (in Spanish below) and a special podcast episode was developed (available below).
Additionally, behaviors that undermine the audit impact at critical contact-point between auditor and auditee were identified. Based on this information, the General Comptroller Office of the Republic of Costa Rica developed a pilot plan to implement intervention strategies collaboratively. Finally, behaviors affecting the follow-up of recommendations were analyzed, and mitigation strategies were proposed based on the “ABCD” framework developed by OECD, which stands for attention, belief formation, choice and determination.
Once the main milestones in the audit process were identified, according to different stages (planning, examination, communication, and follow-up), activities with the most interaction between participants in the audit process were established, and behavioral adjustment strategies were proposed to promote greater objectivity, assertive communication, and a proactive relationship not only with audited entities, but also among the internal areas involved in audit studies.
From theory to practice
In this context, the General Comptroller’s Office of the Republic of Costa Rica applied the proposed improvements resulting from the collaborative work to 7 audits conducted in 2023. The General Comptroller’s Office of the Republic of Costa Rica has audit executing units, as well as a specialized area for the development of the follow-up stage, whose objective is to promote improvements in public management. For each audit team, a contact-person from the follow-up area was selected to help manage the audit recommendation follow up process. The teams also requested a closer interaction between the audit team and the follow-up team from an earlier stage, instead at the ending of communication stage. This approach allowed both the auditors and the follow-up team to gain a deeper understanding of the context of the audited issues, the nature, and conditions of the organizations to minimize uncertainty and promote internal cohesion. Also, it facilitates how the follow-up personnel can contribute to the audit’s development and promotes internal efficiency by dedicating less time to understanding the audited issues.
Figure 2: Main activities and results
Regarding links with audited organizations, work sessions were held promoting the use of proactive language to reshape the auditee’s perspective and transform their conception of the SAI. This allowed the audit to be seen as an opportunity for change, a window of opportunity for improvement, and the generation of impact for the population demanding quality processes, products, and services. Additionally, aligning the auditor’s expectations with those of the auditee helped to find points of convergence and enhance short-term solutions.
Another change resulting from this project was the co-creation of recommendations between the audit team and the follow-up personnel as the audit findings were configured. This aimed to strengthen the will to change, propose viable solution alternatives, and clarify the desired results, reducing the cognitive load on the auditee and audit teams.
Moreover, using technological tools and predictive models on recommendation compliance, the Office of the General Comptroller of Costa Rica is leveraging accumulated data on recommendation follow-up to continue improving their drafting. For example, analyzing the total number of words, specific terms (verbs used), granted deadlines, and institution characteristics helps identify trends that may unconsciously limit or hinder auditees. Thus, substituting them to foster commitment and eliminate information overload is possible.
A more specific, brief, and consensually drafted recommendation, using a proactive tone and coupled with more empathetic and closer follow-up, can be key for promoting compliance from a conviction standpoint, along with the possibility of sustainable changes over time.
The results obtained from implementing this project indicate that change is both real and desirable by the various stakeholders. The foray into behavioral insights requires small but substantial adjustments within reach of auditors and auditees. This promotes experience sharing and diversity of perspectives, enriching audit products and fostering accountability, with a focus on value creation as part of the continuous improvement sought by an SAI.
