Assessment of the Maturity and Impact of Tax Incentives for Regional Development: Application of the Control Framework for Public Policies and Impact Assessments

By: Marcos Araújo Mortoni Silva and Rafael Encinas

Background: Audited Public Policy 

In 2022, the Brazilian Court of Audit (TCU) and the Comptroller General of the Union audited the Automotive Regional Development Policies (PADR) of the Brazilian government. These policies, created in the late 1990s, granted tax credits to automobile manufacturers that established factories in less developed regions of Brazil. 

The PADR costs R$5 billion per year in tax benefits and currently benefits four companies, which have factories located in the cities of Goiana and Belo Jardim, in the state of Pernambuco, and Anápolis and Catalão, in the state of Goiás.

Objectives and Audit Questions 

The audit was structured to allow for a comprehensive opinion on the current situation of the PADR regarding its maturity as public policies and their results. Figure 1 presents the analytical framework and audit questions that were developed for the work.

Figure 1 – Analytical framework of the PADR Audit

Findings

The audit found that: 

• a) The PADR were not formulated based on a prior and consistent diagnosis of a public problem, which made it difficult to define objectives and analyze alternatives to determine the most efficient approach to addressing the causes of a public problem. As a result, the PADR currently lack a logical framework, failing to demonstrate how they will address the causes of a public problem at the lowest possible cost.
• b) The key roles of direction, supervision, and coordination of implementation, monitoring, and ex-post evaluation processes were not established for the policies and are not being performed, leaving the PADR in an inert state, with the federal government unaware of their results. This condition hampers the accountability of these policies, as the executive branch fails to present taxpayers with the performance of the PADR and its responsibility for them. 
• c) The more than R$50 billion spent on the PADR since 2010 did not bring significant socio-economic improvements to the territories where the beneficiary car factories are located, resulting in a high cost per job created.

Methodology and Analytical Framework 

Policy Maturity 

The development of the questions and procedures for the analysis of policy maturity in the PADR was based on the methodological and material foundations of the Control Framework for Public Policies (RCPP) of the TCU, which was also developed based on INTOSAI ISSAI 100, 300, 3000, and 3100. The RCPP presents an analytical rationale structured in control blocks (Figure 2), designed in line with the phases of the policy cycle, providing a conceptual basis and a toolkit for guiding public policy analyses (Figure 3).

Figure 2 – Analytical structure of the RCPP and policy cycle

Figure 3 – RCPP – toolkit for public policy analysis

The RCPP served as an integrating reference for relevant analytical and methodological perspectives in public policies in the audit of the PADR, facilitating dialogue between the team and the internal supervisors and decision-makers of the TCU on common ground. In addition, since incentives and constraints influence agents in their decisions, the audit was innovative and also considered this context in the analysis of the PADR. 

Thus, consolidated perspectives from the literature, such as public choice theory and agency theory, were adopted for the formulation of questions, analysis of responses, and justification of the characterization of the causes and effects of the audit findings. This analytical framework was highly useful in demonstrating the material relevance of accountability for the effectiveness of governments in public policies.

Impact Assessment

To assess the impact of the PADR, the Synthetic Control Methodology, proposed by Abadie and Gardeazabal (2003), was used. According to Abadie (2021, p. 392), the method was developed to estimate the effects of aggregate interventions, i.e., interventions that are implemented at an aggregate level, affecting a small number of large units (such as cities, regions, or countries), on an outcome of interest. According to Athey and Imbens (2017, p. 9), it is “arguably the most important innovation in the policy evaluation literature in the last 15 years.” 

The method is primarily used in comparative case studies, comparing the evolution of the outcome variable in the unit that received the government intervention with a set of untreated units that, when combined into a synthetic unit, form the counterfactual of the treatment unit. 

Although the PADR currently benefits four automotive factories, the impact assessment was only possible for a factory inaugurated in 2015 due to data availability, as the national indicator databases did not have information prior to the inauguration of the other factories. 

In Brazil, municipalities are grouped into two levels of regions (IBGE, 2017): immediate and intermediate. Immediate regions are characterized by integration into the urban network, based on nearby urban centers that satisfy the immediate needs of the populations. Intermediate regions correspond to an intermediate scale between the federal unit and the immediate region, delimited by the inclusion of metropolises, regional capitals, or representative urban centers for a set of immediate regions (IBGE, 2017, p. 20). Both types of regions were defined as the territory for impact assessment. 

Figure 4 shows the map of the state of Pernambuco with the division of its immediate and intermediate regions. The immediate region where the evaluated factory was installed is the region shaded in dark green, composed of eleven municipalities and a population of approximately 350,000 inhabitants. The light green-shaded region is the intermediate region, which encompasses the immediate region of the factory, consisting of 72 municipalities, with a total population of 5.7 million.

Figure 4: Map of the state of Pernambuco, with the immediate and intermediate regions where the factory was installed

The synthetic control evaluation indicated a significant impact on per capita GDP, total employment, technical-scientific employment, and total remuneration in the immediate region that benefited from the PADR, but no impact in the intermediate region, as shown in Figure 5.

Figure 5: Results of the synthetic control analysis for per capita GDP, total employment, total remuneration, and technical-scientific personnel

In the case of total employment, the proportion to the population in the treated immediate region reached 14.64% in 2019, while in the synthetic control, it was 11.4%. The difference of 3.24% over the population of the region corresponds to 11,258 jobs generated. 

In 2019, the factory received R$4.6 billion (USD 1.1 billion) through tax incentives, which corresponds to R$388 million (USD 92.4 million) per month, resulting in a monthly cost of the policy per job created of R$34,000 (USD 8,100), a high cost considering the national minimum wage of R$998 (USD 238) or the amounts transferred by other social programs, such as Bolsa Família, which averaged R$186 (USD 44). 

The results of the impact assessment were supported by employment generation data from national records, which indicated low job creation in other sectors of the economy, with a migration of jobs to the automotive sector. Furthermore, information on the factory’s input purchases showed that only 6% of suppliers were from the target regions of the policy, while 94% were acquired from companies in more developed regions. 

The conclusions are aligned with studies from multilateral organizations such as the OECD (2015) and the World Bank (Kronfol & Steenbergen, 2020), which state that tax incentives alone do not have the ability to attract investments, as other factors are equally or more important in the decision-making process of private agents.

Applicability in Audits 

The maturity analysis of public policies adopted in the audit of the PADR can be replicated by Supreme Audit Institutions (SAIs), with contextualizations as necessary. 

The applicability of the content of the RCPP needs to be verified on a case-by-case basis, as its criteria are largely based on Brazilian legal frameworks, as well as the competencies and jurisprudence of the TCU. On the other hand, the analytical framework of the RCPP and its toolkits, developed based on the phases of the policy cycle, can be fully replicated by SAIs, being useful for the assembly of audits aimed at a comprehensive analysis of public policies. 

The adoption of theoretical perspectives that consider the behavior of agents can also be replicated for the maturity analysis of policies. In this case, not as a normative compliance criterion, but as a scientific basis to guide investigative perspectives and explain the causes of findings in audits of public policies. 

The impact assessment conducted in the audit of the PADR can also be used in audits that evaluate public policies in various sectors. The synthetic control method allows for the construction of a counterfactual in case studies, so that the impact of health, education, economic development, environmental, and other policies on large aggregates such as municipalities, states, or countries can be evaluated.