Technical Articles



International Journal of Auditing – October 2007

 

Effectiveness Examinations and Impact Assessments

Editor’s note: Rigsrevisionen, the Danish SAI, has initiated three projects aimed at gaining experience in and deepening skills related to effectiveness examinations and impact assessment methodologies.

The report on a 2006 peer review of Rigsrevisionen, the Danish SAI, recommended that the agency place increased emphasis on effectiveness examinations. The report’s recommendations coincided with Rigsrevisionen’s plans to apply more advanced methods when carrying out examinations of effectiveness. As a result of the peer review report, Rigsrevisionen initiated three major examinations that focus on effectiveness and impact assessment.

Effectiveness and Impact

Effectiveness examination is the overall term applied to analyses of the impact of government efforts and projects. An effectiveness examination may include an evaluation of goal achievement, impact, and the relation between impact and resources. Impact refers to the results of an effort in the short, medium, or long term.

The examinations were initiated to assess the impact of the efforts in the Ministry of
Social Affairs and in the Ministry of Refugee, Immigration and Integration Affairs and in the area of industrial promotion within several Danish ministries. The three examinations are also part of a development project in Rigsrevisionen designed to gain experience and skills in the field of impact assessment and will provide a platform for drawing up guidance on how to work within the area of effectiveness in the future.

Prerequisites of Examinations

Rigsrevisionen’s preliminary examination showed that the ministries had focused on impact assessment in recent years but had rarely considered how to measure the future impact of their efforts when developing legislation or establishing a new support scheme. As a result, the ministries often did not have sufficient data to carry out valid impact assessments. Because the ministries did not collect the necessary data, the three project groups have found it difficult to identify the areas best suited for impact assessments and obtain usable data.

Rigsrevisionen is collaborating closely with the Danish Institute of Governmental Research (AKF),1 which has extensive experience with impact assessments. The AKF will provide feedback on professional aspects of the three examinations and contribute views and suggestions concerning methodology, the framework of analysis, and the interpretation of results.

Assessing impact requires coordination and a close dialogue with the institutions concerned. Furthermore, to successfully assess the results of the effectiveness examinations, the entity and Rigsrevisionen must fully agree on the intended impact of a given effort.

Impact examinations have a dual purpose. First, the individual examination should provide concrete evidence of the immediate impact. This part of the examination is based on concrete impact assessments and thereby adds a new dimension to Rigsrevisionen’s current evaluations of the impact of ministries’ efforts.

Second, the examinations are designed to map out/shed light on the ministries’ work with impact and provide specific recommendations on how to develop the area. This part of the examination is based on methodologies previously applied by Rigsrevisionen, and it includes an assessment of how the ministries are working with impact in selected areas based on goals, strategies, and action plans.

Looking to the Future

When the reports on the examinations are finished in the spring of 2008, the process will be evaluated, as one of the goals of the examinations is to build knowledge, experience, and skills within the field of impact examination. In the long term, the work with examinations is expected to provide Rigsrevisionen with guidance on how to work with impact assessments in the future.

The effectiveness examination work will also help to develop new skills within Rigsrevisionen. Regardless of the format chosen for future work with impact assessment, employees will be better able to review impact assessments and evaluations carried out by the entities.

When the results of the three examinations are available, the Rigsrevisionen will publish the most significant results and experiences gained in a future article.

For additional information, please contact the authors at rt@rigsrevisionen.dk and ane@rigsrevisionen.dk.


1The AKF conducts research into subjects of relevance to the public sector, especially municipalities and regions. It also carries out independent research at an international level. The results of its work are disseminated in scientific journals and in publications for the general public.